Agentic AI systems — LLMs that plan, use tools and act autonomously — introduce risks that traditional threat models miss. Threat modeling them before you build is the cheapest security you will ever buy. This guide explains how to think about agentic threats and what to map.
Why agents need their own threat model
Classic threat modeling assumes deterministic components with well-defined trust boundaries. An agent breaks those assumptions: it takes non-deterministic actions based on untrusted natural-language input, and it crosses trust boundaries by design when it calls tools and reads external data. The core question shifts from “what inputs are validated?” to “what is the worst an attacker can make this agent do?”
The trust boundaries to map
- Input channels: every place untrusted text can reach the model — user chat, retrieved documents, tool outputs, connected data sources. Each is a prompt injection vector.
- Tool surface: every capability the agent can invoke, its scope, and what it can reach. This is where MCP and tool-chain risks live.
- Authorisation boundary: where “decide” ends and “act” begins — the point your code must enforce permissions, never the model.
- Data boundary: what the agent can read and exfiltrate, across tenants and sensitivity levels.
The attacks to design against
- Goal hijacking: injected instructions redirect the agent toward the attacker’s objective.
- Tool abuse & chaining: combining legitimate tools to reach an illegitimate outcome (read data → send email → exfiltrate).
- Privilege escalation: using the agent’s standing permissions to do what the attacker cannot do directly.
- Sandbox escape: breaking out of the execution environment into the host or network.
Design principles that fall out of the model
- Least-privilege tools and short-lived, scoped credentials.
- Authorisation enforced in code, outside the model.
- Human approval gates for irreversible actions.
- Isolation of untrusted content and sandboxed execution.
- Full logging of decisions and tool calls for detection and forensics.
These are the same principles behind our practical guide, how to secure an AI agent.
How we help
We run structured agentic AI threat modeling at the design stage to find these risks before code exists, then validate the built system with AI agent penetration testing. Together they cover the architecture and the implementation.
Frequently asked questions
When should we threat model — before or after building? Before. Design-stage modeling is far cheaper than retrofitting controls after an incident.
Is threat modeling a substitute for testing? No — it guides design; testing proves the built system holds. You want both.
Designing an agentic system? Explore agentic AI threat modeling or get a fixed-price quote.