AI agent least privilege means the model never holds authority itself. A policy layer outside the model decides what each tool call may do, on whose behalf, with credentials the model cannot read. No filter reliably stops prompt injection, so this design decides whether a hijacked agent produces a logged, reversible mistake or a breach.
The short answer
- AI agent least privilege means the model proposes and deterministic code outside it decides: a policy point checks narrow tool scopes against the user's real rights.
- Prompt injection has no reliable filter, so design for a hijacked model: brokered credentials it never sees, approvals bound to the exact payload, default-deny egress.
- Break the lethal trifecta (private data, untrusted input, external comms) per agent, and verify the design with injection tests against the real permission config.
Key takeaways
- Prompt injection cannot be filtered out reliably, so design every agent as if the model is already hijacked.
- Authorization lives in a PEP/PDP outside the model and is evaluated against the end user's rights, not the agent's.
- Replace god-tools with typed, allowlisted tools, and never trust MCP annotations or self-declared risk categories.
- Broker credentials through a proxy so the model never sees a secret; use short-lived, audience-bound, sender-constrained tokens.
- Approvals must be risk-tiered, bound to a hash of the exact payload, and rendered from server-generated parameters.
- Default-deny egress plus CSP on rendered output closes the exfiltration leg of the lethal trifecta.
What does least privilege mean for an AI agent?
AI agent least privilege means the model proposes actions and deterministic code decides them. A policy enforcement point checks each tool call against narrow, parameter-constrained scopes and the real end user’s entitlements. It injects short-lived credentials the model never sees, and it asks a human only for irreversible or externally visible actions, with the approval bound to the exact payload.
The difference from service-account least privilege is the threat model. A service account runs code you wrote. An agent runs whatever plan emerges from a context window that mixes your instructions with emails, web pages, tickets and retrieved documents written by strangers. The principal you are constraining is partly the attacker.
Saltzer and Schroeder’s 1975 principles still hold. Least privilege and fail-safe defaults get quoted; complete mediation gets skipped. If a code interpreter, a sidecar or a cached token can reach a resource without passing the policy check, that path is unmediated, and an injected model will find it.
Can prompt injection be prevented, or do you design around it?
You design around it. Prompt injection follows from mixing instructions and data in one context window, and no published defense eliminates it. The 2025 OWASP Top 10 for LLM Applications ranks prompt injection (LLM01) first and lists excessive agency (LLM06) as the least-privilege failure, broken into excessive functionality, excessive permissions and excessive autonomy.
Classifiers, instruction hierarchies and “ignore malicious instructions” lines in the system prompt are probabilistic detection. Keep them: they raise attacker cost and generate signal. Do not count them as the control. A system prompt that says “never delete files” is a request, and attacker text can make a competing one. Measure how often they fail with prompt injection testing across every untrusted input path.
The fastest triage test is Simon Willison’s lethal trifecta. An agent that reads private data, consumes untrusted content and can communicate externally can be made to exfiltrate, whatever the model’s quality. Remove at least one leg per agent instead of adding another filter in front of all three.
Patterns that keep untrusted data out of tool parameters
The 2025 paper by Beurer-Kellner et al., “Design Patterns for Securing LLM Agents against Prompt Injections”, lays out six patterns. Four matter most in practice:
| Pattern | How it works | Guarantee | Cost |
|---|---|---|---|
| Action-Selector | Model picks from fixed tools with fixed arguments | Strongest: untrusted data never shapes parameters | Little flexibility |
| Plan-Then-Execute | Plan is fixed before untrusted content is read | Control flow is safe; arguments can still be tainted | Moderate |
| Dual LLM | Privileged model never sees untrusted text; quarantined model has no tools | Strong, via opaque symbolic variables | Orchestration complexity |
| Code-Then-Execute | Model writes a program; an interpreter enforces policy on data flow | Strong when paired with flow tracking | Highest |
Google DeepMind’s CaMeL extends Dual LLM with capabilities and information-flow control. Microsoft’s FIDES labels every value for confidentiality and integrity, then blocks actions whose arguments carry untrusted taint. The cheapest structural win needs no research code: split one powerful agent into a read-only research agent and a write-capable action agent.
Where is AI agent authorization enforced, and which tools should exist?
Authorization belongs in a single policy enforcement point (PEP) in the tool-execution path, backed by a policy decision point (PDP) the model cannot reach. OPA with Rego, AWS Cedar, OpenFGA or an AuthZEN-conformant service all work. The model proposes; the policy layer disposes. Three rules make that real:
- One path. Every tool, including code execution and file access, routes through the PEP. Grep your codebase for direct SDK clients that bypass it.
- The model’s refusal is not a control. If the PDP would allow the call, assume the model will make it.
- Authentication is not authorization. Knowing which agent called says nothing about whether this user may perform this action on this resource (CWE-863, CWE-269).
Tool scopes: kill the god-tools
Give the agent typed, single-purpose tools whose parameters are allowlists. Each common god-tool maps to a known weakness class:
http_get(url)is SSRF by design (CWE-918). Replace it with per-connector clients pinned to specific hosts.read_file(path)is path traversal (CWE-22). Replace it with a reader locked to one directory prefix.run_shell(cmd)is command injection (CWE-77). Replace it with named operations such asrun_tests(repo)against a repo allowlist.sql_query(any)combines traversal, injection and unbounded reads. Replace it with read-only parameterized queries, row limits and a column allowlist.
Constrain outputs too. A CRM lookup that returns name and ticket status, not the full customer record, limits what an injected agent can carry out. Classify every tool in your own registry as read, write, destructive, external communication or financial. A category the tool declares about itself is input, not policy.
What MCP adds
The Model Context Protocol brings third-party code that defines what your model believes its tools do. That opens four abuse surfaces:
- Token passthrough. The MCP authorization spec forbids a server from accepting tokens not issued for it. A proxy server that forwards the user’s token downstream recreates the confused deputy.
- Tool poisoning and rug pulls. Tool descriptions reach the model with more trust than user content. Pin and hash them, diff on change and alert. A server that silently rewrites a description after approval is a rug pull.
- Annotations are hints.
readOnlyHintanddestructiveHintcome from an untrusted server. Never gate authorization on them. - Sampling and elicitation. Sampling lets a server ask the host model to generate text, an injection path into the privileged model. Elicitation can social-engineer the user. Capability names differ across spec revisions, so check yours.
Local stdio servers are a supply-chain risk. Installing one is like adding a dependency with shell access, because it has shell access. Our MCP server and tool-chain security testing targets these paths.
Should the agent act as the user, and how does it get credentials?
Use a hybrid. The agent has its own workload identity, and every downstream call carries explicit delegation from the user. An agent that acts only as a broad service identity is a textbook confused deputy: injected text makes it spend authority the user never had, and tenant isolation collapses.
OAuth 2.0 Token Exchange (RFC 8693) expresses this with the act claim, so the downstream service sees both principals, authorizes against the user’s rights and logs the chain. Rich Authorization Requests (RFC 9396) narrow it further. Instead of payments.write, the token carries a structured grant: a payment initiation with a maximum amount and a fixed creditor.
Give each agent instance its own identity through SPIFFE/SPIRE, cloud workload identity or mTLS. Shared service accounts destroy attribution and make revocation all-or-nothing: to stop one compromised agent, you break all of them.
Broker credentials so the model never sees a secret
The model emits a logical identifier such as cred:github-readonly. An egress proxy resolves it after the model’s turn, injects the auth header and forwards the request under policy. Injection can still make the agent use a credential while the session is active, but it cannot make the agent exfiltrate the credential itself.
| Approach | Secret in context? | Replay if stolen | Engineering cost |
|---|---|---|---|
| Long-lived API key in env or prompt | Often | Indefinite | Lowest |
| Short-lived downscoped token | Possibly | Minutes, single audience | Moderate |
| Credential-injection proxy + sender-constrained token | Never | Unusable off-host | Highest |
Build downscoping from existing primitives: AWS STS AssumeRole with an inline session policy, GCP credential downscoping, Azure on-behalf-of flows, HashiCorp Vault dynamic secrets. Target TTLs in minutes, scoped to one task and revoked at session end.
Bind tokens to an audience with Resource Indicators (RFC 8707), so a token minted for tool A fails at tool B. Sender-constrain them with DPoP (RFC 9449) or mTLS-bound tokens (RFC 8705). An agent with outbound access can send a stolen token anywhere; binding makes it worthless there.
Memory and sub-agents carry privilege too
Persistent memory and RAG are write-then-read channels: text poisoned today can trigger an unrelated run next week. Scope memory per user, taint it as untrusted at read time, and never let it carry authority. A stored note saying “user pre-approved all payments” must have zero effect on the PDP.
A spawned sub-agent’s privileges must be a strict subset of its parent’s, with credentials derived by narrowing. Macaroons and Biscuit tokens support offline-verifiable attenuation, and UCAN follows a similar model. Add containment budgets (action counts, spend caps, step limits per OWASP LLM10 Unbounded Consumption) and a kill switch that revokes tokens immediately.
Which agent actions need human approval?
Gate by risk tier. Auto-approve idempotent reads. Require confirmation for writes, destructive operations, external communication and money movement, and step-up WebAuthn for the top tier. Confirming everything trains users to click through. OWASP’s Agentic AI Threats and Mitigations treats overwhelming human reviewers as a threat category in its own right.
An approval design only works if an injected model cannot subvert it:
- Bind the approval to a hash of the canonicalized, fully resolved action (host, method, path, body, amount, recipient), and make it single-use. Without this, a TOCTOU gap lets the model change the recipient after the human approves the summary.
- Render server-generated parameters, never model-written prose. If the model writes the approval prompt, the injection writes the approval prompt.
- Prefer reversibility to approval. Soft delete, undo windows, dry-run previews and two-phase commit protect better than a dialog. Irreversibility, not sensitivity, is what makes an action catastrophic.
For high-risk systems deployed in the EU, AI Act Article 14 human-oversight duties attach to this layer.
How do you stop an agent from exfiltrating data?
Enforce default-deny egress at the network layer; a prompt instruction limits nothing. Route agent traffic through a CONNECT proxy (Envoy, Squid, an Istio egress gateway, Cloudflare Gateway or Zscaler) with domain and method allowlists, and add Kubernetes NetworkPolicy underneath.
- Block direct-to-IP connections, non-443 ports, raw sockets and arbitrary DNS resolvers.
- Cap bytes per destination per session and run DLP on outbound bodies.
- Log the full URL and byte counts, not just the hostname.
Then close the channel most teams forget: the answer itself. A rendered markdown image with data in its query string exfiltrates the moment the chat UI loads it, and link previews and raw HTML do the same. EchoLeak, the 2025 Microsoft 365 Copilot issue (CVE-2025-32711), belonged to this class. Set CSP img-src and connect-src to trusted origins, or strip remote references before rendering.
Sandboxing does not solve this. gVisor, Firecracker, Kata and Wasmtime bound code execution, not goal hijack. A sandboxed interpreter with open egress is still an exfiltration tool, so ship it with no network and an ephemeral filesystem.
How do you test that AI agent permissions actually hold?
Attack the real permission configuration and check side effects; a mock tells you nothing about production. Run an injection regression suite in CI (AgentDojo-style tasks plus PyRIT, garak or promptfoo) against the deployed tool wiring, and fail the build when a tainted input reaches a gated action.
Permission review checklist
Per agent, a review should prove:
- Every tool call passes through the PEP, including interpreter and file paths.
- Calls are authorized against the user’s entitlements, and a second-tenant canary stays unreachable.
- No secret appears in any context window, log or trace.
- Approvals are bound to the payload hash, and changing one field after approval fails.
- Egress to a non-allowlisted host, a raw IP or a markdown image URL is blocked.
- Changing a tool description triggers an alert.
- Audit logs record resolved parameters, identity, provenance label and approval record, not the model’s self-report.
Our guide on how to test excessive agency in AI agents covers the method. If you report into a governance program, map results to NIST AI RMF MEASURE and the Generative AI Profile (NIST AI 600-1).
FAQ
What is AI agent least privilege?
The model never holds authority directly. Every tool call goes through a policy check against narrow scopes and the user's real entitlements, using credentials the model cannot read.
Is a system prompt instruction a valid access control for an AI agent?
No. A system prompt is text the model weighs against other text, including attacker text. Access control has to be enforced in code outside the model.
What is the lethal trifecta in AI agents?
Three capabilities in one agent: access to private data, exposure to untrusted content and the ability to communicate externally. An agent with all three can be made to exfiltrate data regardless of model quality, so remove at least one leg per agent.
What is the confused deputy problem for AI agents?
The agent holds broader authority than the user it serves, and injected instructions make it spend that authority for the attacker, often across tenants. Authorize every call against the user's own rights, using delegation tokens such as RFC 8693 with an actor claim.
Should a human approve every AI agent action?
No. Blanket approval causes click-through fatigue, which OWASP lists as an agentic threat. Auto-approve idempotent reads, gate writes and external or financial actions, and bind each approval to the exact resolved payload.
Is a read-only AI agent safe?
Not by default. Read tools are how private data gets collected for exfiltration, so a read-only agent still needs egress limits and output rendering controls.
If you are designing tool scopes, token flows or approval gates for an agent going to production, our agentic AI threat modeling maps where a hijacked model can still reach, and our testers then prove it with real injection chains.