AI Agent Penetration Test Scope: A Checklist for Security Leads

An AI agent penetration test scope has to cover the whole deployed system: agents and versions, orchestration, tool and MCP permissions, identity paths, memory and RAG stores, and human approval gates. Leave any of those out and the report cannot tell you what an attacker can make the agent do with the privileges it already holds.

What should an AI agent penetration test scope include?

An AI agent penetration test scope should include every agent and version, the orchestration layer, model endpoints and provider configuration, tool and MCP permissions, identity and delegation paths, memory and vector stores, RAG ingestion, human-in-the-loop approval gates, multi-agent links, and the logging that ties actions back to principals. Those surfaces decide what an attacker can make the agent do.

The unit of assessment is the deployed agent system, not the model. For scoping purposes, an agent is an LLM-backed planner or executor plus tools, memory, retrieval, orchestration, an identity and often a human approver. Strip any of those out of scope and you are testing a chatbot with extra steps.

Draw the trust boundaries before anyone writes a test case. A typical agent system has seven:

• User to agent
• Agent to model provider
• Agent to tool or API
• Agent to memory or vector store
• Agent to other agents
• Agent to human approver
• Agent to monitoring and audit

Every boundary on that list should map to at least one row in the scope document, with an owner, a test mode and exit criteria. A boundary with no row has no coverage, and the report will be silent about it.

How is an AI agent pentest different from an LLM penetration test?

An LLM penetration test asks what the model will say; an AI agent penetration test asks what the system will do. Prompt injection and jailbreaks still matter, but in an agent they are the entry point, not the finding. The finding is the tool call, the token used, the record written, the email sent.

That moves where severity lives. A prompt injection that makes a support bot swear is a content issue. The same injection that makes an agent call a refund API with its own service credentials is a privilege issue, the pattern the OWASP Top 10 for LLM Applications 2025 calls excessive agency.

Dimension	LLM pentest	AI agent pentest
Primary question	What can the model be made to output?	What can the agent be made to execute?
Core surfaces	System prompt, guardrails, output filters	Orchestration, tools, MCP servers, identities, memory, RAG, approval gates
Typical findings	Jailbreaks, system prompt leakage, harmful output	Confused deputy, tool poisoning, cross-tenant memory leaks, approval bypass
Severity driver	Content sensitivity	Tool privilege, autonomy, blast radius
Access needed	Often black-box	Usually grey-box or white-box
Evidence	Prompt and response pairs	Traces linking input, plan, tool call and principal

Agent scopes need both. Content manipulation shows how the attacker gets in; tool and identity abuse testing shows how far they get.

Which agents, versions and trust boundaries belong in the inventory?

Every agent instance and version belongs in the inventory, including the ones nobody calls an agent. Sub-agents, scheduled jobs, webhook handlers, background workers, evaluation harnesses, admin consoles and feature-flagged variants often share the same tool wiring. Unlisted agents are untested agents.

For each one, collect these fields before kickoff:

• Name, owner and business purpose. Who signs off on findings and who fixes them.
• Autonomy level. Suggest-only, act with approval, or act without approval.
• Model, provider and version, plus system prompt and template version.
• Tool schemas and permissions, including every MCP server the agent can reach.
• Retrieval indices and memory stores, with the data classification of each.
• Identities and secrets: service accounts, OAuth clients, API keys and where they live.
• Environments where the agent runs, and which ones mirror production.
• Human gates: which actions require approval and who approves.
• Logging and tracing: tool-call logs, identity attribution, SIEM forwarding, and OpenTelemetry GenAI semantic conventions if adopted (the conventions are still maturing).
• Third-party dependencies and vendor contacts.

If the team cannot fill these fields, that gap is the first finding. It usually means nobody mapped the architecture, and a short agentic AI threat modeling exercise before testing will produce a better scope than any template.

How do you scope tool permissions, plugins and MCP servers?

Scope tool permissions by comparing what each tool is declared to do with what it is actually allowed to do. The agent’s effective privilege is the union of its tool permissions, so the tool layer is core scope, not a dependency owned by another team.

For each tool, the scope should list:

• Read versus write capability, and whether write restrictions are enforced at the API or only described in the tool schema
• Data classification of everything the tool returns
• Network egress, which is where SSRF through fetch and browse tools comes from
• Filesystem access and code execution
• High-impact actions: payments, refunds, ticket changes, outbound email, deployments
• Schema validation, argument allowlists and per-tenant scoping

MCP servers need their own rows. A local MCP server runs as a process with the user’s or host’s credentials, so the question is what that process can touch. A remote MCP server adds its own authentication, tool manifests and third-party authorization. Both are exposed to tool poisoning, where a malicious or altered tool description steers the model toward calls the user never requested. Testers need the manifests to check for it; our MCP server and tool-chain security testing covers that method in detail.

Do identity and credential paths belong in an AI agent pentest scope?

Yes. Identity and credential paths decide what the agent can reach, so excluding the identity provider because “IT owns it” deletes the part of the test that measures blast radius.

Scope three identities separately: the user, the agent, and the service accounts behind each tool. Then scope the delegation between them:

• OAuth 2.0 and OpenID Connect scopes granted to agent clients
• Token exchange and on-behalf-of flows, including OAuth 2.0 Token Exchange (RFC 8693) where used
• Token audience and downscoping: does a token minted for one tool work against another?
• Workload identity and short-lived credentials (SPIFFE/SPIRE, AWS STS, Entra ID managed identities, Google Cloud service accounts)
• Secrets storage such as HashiCorp Vault, and whether the agent runtime can read secrets it never uses
• Audit attribution: can the logs tell which human caused which agent action?

The core risk is the confused deputy. A low-privilege user asks the agent for something, the agent executes it with a high-privilege service account, and authorization is checked against the agent rather than the user. The same pattern collapses controls that used to depend on separate people, which we covered in why one agent breaks segregation of duties.

How should memory stores, vector databases and RAG pipelines be scoped?

Scope memory and RAG as attack surfaces, not as data infrastructure. Cross-tenant memory leakage and persistent poisoning are agent security failures even when the vector database itself is configured correctly.

Memory scope covers short-term conversation state, long-term user and agent memory, caches, scratchpads, state stores and vector memory. For each, test read, write and delete authorization, isolation across sessions and tenants, whether poisoned content persists into later sessions, retention, provenance and deletion rights. Memory poisoning that survives a session reset is far worse than an injection that dies with the conversation.

RAG scope covers the full path: document ingestion, source authorization, document-level ACLs, tenant filters, chunking, the embedding model, index refresh, citation behavior and retrieval-time authorization. Name the store (Pinecone, Weaviate, Qdrant, Milvus, pgvector, Chroma), because filter semantics differ. Indirect prompt injection through an ingested document, ticket or web page is a system issue: the payload rides the retrieval path into the agent’s context. A dedicated RAG pipeline security assessment goes deeper on ingestion and ACL enforcement.

Insist on at least two test tenants. A test that runs inside one tenant cannot prove that memory, retrieval and tool outputs are tenant-scoped.

Are human-in-the-loop approval gates security-relevant enough to test?

Yes, and they should be tested as security controls, not as frontend. The question is not whether the approval screen renders. It is whether the action a human approves is the action that executes.

Scope the gate on these points:

• Which actions require approval, and whether that list matches the written policy
• Whether the displayed action matches the actual tool call and its arguments
• Approver privilege: can the approver authorize something they could not do themselves?
• Bypass paths: direct API calls, alternative tools or sub-agents that reach the same action without the gate
• Race conditions between approval and execution
• Approval fatigue from high-volume, low-risk prompts that train people to click yes
• Fail-open versus fail-closed behavior when the policy engine or approval service is down
• Audit evidence of who approved what, and when

A gate that shows “send summary to manager” while the underlying tool call carries an external address is a critical finding. You only find it if the gate is in scope.

Should production be in scope, and what access should testers get?

Production should usually be in scope only in read-only or canary-tenant form; write-capable agent testing belongs in an isolated staging environment that mirrors production tools, identities and data classifications. “Staging is close enough” breaks down when staging uses different OAuth clients or a stub payment tool, so add a production validation pass to catch configuration drift.

Test mode	Testers get	Good for	Misses
Black-box	Public interface only	External realism, injection entry points	Tool wiring, identity delegation, attribution
Grey-box	Architecture docs, tool schemas, partial config	Most agent engagements	Some secret-handling and code paths
White-box	Source, prompts, tool schemas, identity config, logs	Identity and tool-permission findings, root cause	Attacker realism, unless paired with a black-box phase

Agentic testing usually needs grey-box or white-box access, because tool wiring and identity delegation are invisible from outside. Without config and logs, findings are hard to attribute, reproduce or fix.

The rules of engagement should state written authorization, test accounts, synthetic or approved data, allowed techniques, rate limits, stop conditions, emergency contacts, the kill-switch procedure, evidence redaction and retention, and third-party notification. Data handling belongs here: PII and PHI classification, GDPR and DPA obligations, and whether testers may keep model inputs and outputs. Set a cost ceiling too. Recursive tool loops, token-budget exhaustion and denial of wallet are real agent risks, and the test should probe them within agreed limits.

How do you handle nondeterminism, third parties and explicit exclusions?

Handle nondeterminism by pinning versions, repeating trials and capturing traces; handle third parties with a written shared-responsibility split; handle exclusions by writing every one down with a reason.

Pin the model, prompt template and tool-schema versions for the test window. Record timestamps and trace IDs. Run repeated trials and report a success rate and reproduction conditions instead of declaring a finding from one stochastic output. If the provider rolls a model version mid-test, log it: a finding against one version may not survive the next deployment.

For third parties, the provider’s model internals and training pipeline are generally not testable by the customer. Your integration, configuration, prompts, tool permissions, data flows and identity trust are. Provider terms, remote MCP servers and SaaS connectors may prohibit intrusive testing even when you own the agent configuration, so the authorization letter must name each one. Supply chain works the same way: open-weight models, embedding models, plugins, agent frameworks such as LangChain, LangGraph, CrewAI or Semantic Kernel, and inter-agent protocols like A2A each move a trust boundary. In multi-agent systems, one compromised sub-agent can inherit the orchestrator’s reach, so delegation chains, inter-agent messages, shared memory and agent cards belong in scope.

The out-of-scope traps we see most often:

• Tools excluded because “the API team owns them”
• The identity provider excluded because “IT owns it”
• Memory and RAG excluded because “the data team owns it”
• The approval UI excluded because “it is frontend”
• Production validation excluded because “staging is close enough”
• Guardrails treated as a reason to cut scope instead of a control to test

The opposite failure is scoping too broadly. Bolting an agent review onto a full infrastructure pentest usually produces neither deep agent findings nor complete infrastructure coverage. Silent exclusions are the worst outcome, because they yield a clean report nobody should trust.

What deliverables and severity ratings should an AI agent pentest require?

Require findings tied to versioned assets, trace evidence and an agent-specific severity model, because CVSS alone does not capture agent risk. A prompt injection scores the same in CVSS whether the agent can only read FAQ pages or can issue refunds.

Rate severity on exploitability, autonomy level, tool privilege, blast radius, data sensitivity, presence or absence of a human gate, tenant isolation and reproducibility rate. Each finding should carry the asset and version, preconditions, observed behavior, trace or log evidence, business impact, affected principals, remediation, residual risk and retest criteria, with secrets and customer data redacted.

Map coverage to frameworks that auditors and boards already read: the OWASP Top 10 for LLM Applications 2025 (prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, unbounded consumption), OWASP’s agentic AI guidance, MITRE ATLAS, the NIST AI RMF with the NIST AI 600-1 Generative AI Profile, and ISO/IEC 42001. Use OWASP ASVS and the OWASP API Security Top 10 for the web and API surfaces around the agent. State framework versions in the report, since agentic guidance is still being revised.

Agree exit criteria before testing starts: every in-scope tool, identity path, memory store, retrieval path, human gate and external integration covered; critical findings triaged; evidence accepted; retest criteria signed off. Without them, the report goes stale after the next prompt change.

A copyable AI agent penetration test scoping checklist

Use this as the skeleton of the scope document. Turn each line into a row in a scope matrix with columns for owner, test mode, principals, data class, allowed actions, evidence source and exit criteria.

1. Agent inventory complete: every agent, sub-agent, job, webhook, harness and feature flag, with model, prompt and tool-schema versions pinned.
2. Trust boundaries drawn: user, provider, tools, memory, other agents, approvers, monitoring.
3. Test mode chosen per asset (black, grey or white-box) and access delivered: source, prompts, schemas, identity config, logs.
4. Every tool listed with declared versus enforced permissions, egress, code execution and high-impact actions.
5. Every MCP server listed as local or remote, with manifests supplied.
6. User, agent and service identities mapped, including OAuth scopes, token exchange, audiences and secrets storage.
7. Memory stores and vector databases listed with read, write, delete and retention rules.
8. RAG ingestion sources, ACLs and tenant filters listed; at least two test tenants provisioned.
9. Approval gates listed with the actions they cover, approver roles and failure behavior.
10. Multi-agent delegation chains and inter-agent protocols (A2A, MCP) documented.
11. Environments defined: staging for write tests, read-only or canary scope in production.
12. Rules of engagement signed: data handling, rate and cost limits, stop conditions, kill switch, contacts.
13. Third-party authorization confirmed for each provider, MCP server and SaaS integration.
14. Exclusions written down, each with a business reason and an owner who accepts the risk.
15. Severity model, report fields, framework mapping and retest criteria agreed before day one.

FAQ

What is the difference between an LLM pentest and an AI agent pentest?

An LLM pentest tests what the model can be made to output, such as jailbreaks and system prompt leakage. An AI agent pentest tests what the system can be made to execute through its tools, identities, memory and approval gates.

Should production be in scope for an AI agent penetration test?

Usually only in read-only or canary-tenant form. Write-capable testing belongs in an isolated staging environment that mirrors production tools and identities, plus a production validation pass to catch configuration drift.

What access should we give AI agent pentesters?

Grey-box at minimum: architecture docs, tool schemas and partial configuration. White-box access to source, prompts, identity config and logs is needed to attribute and reproduce identity and tool-permission findings.

Can we test our model provider or third-party MCP servers?

Provider model internals are generally not testable by customers, but your integration, prompts, tool permissions and identity trust are. Any testing of third-party APIs, remote MCP servers or SaaS connectors needs explicit authorization, since provider terms may prohibit intrusive testing.

How do you deal with nondeterministic model output during a pentest?

Pin model, prompt and tool-schema versions, capture trace IDs and timestamps, and run repeated trials. Report a success rate and reproduction conditions rather than a single stochastic result.

Is CVSS enough to rate AI agent findings?

No. Agent severity should factor in autonomy level, tool privilege, blast radius, data sensitivity, whether a human gate exists, tenant isolation and reproducibility.

What should be explicitly out of scope in an AI agent pentest?

Only what you have a business reason to exclude, written down with a risk owner. Do not exclude tools, the identity provider, memory stores or approval gates by default, because those are where agent-specific risk lives.

If you want a second pair of eyes on a scope before you sign it, or a team to run the test end to end, our AI agent penetration testing engagements start from exactly this checklist.