A defensible LLM penetration testing scope names five layers as separate assets: the inference API, prompt assembly, tool and MCP actions, the RAG store and its authorization, and the tenant boundary, plus every output sink. If your SOW reads like a chatbot jailbreak test, you are paying to test the layer where the least damage happens.
The short answer
- Scope the inference API, orchestration and system prompts, tools and MCP servers, RAG stores and their authorization, and the tenant boundary as separate SOW lines, plus output sinks, trace stores, cost limits and the admin console.
- Jailbreak testing is one line item. Indirect prompt injection that reaches a tool call, a cross-tenant retrieval or an exfiltration sink is the finding that matters.
- Pin model snapshot IDs and temperature, define attack success rate over a fixed number of runs, cap the testers' token spend, and write every exclusion down with a reason.
Key takeaways
- An LLM pentest SOW should list the inference API, orchestration, tools and MCP servers, RAG stores and the tenant boundary as separate assets, plus output sinks, trace stores and cost limits.
- Indirect prompt injection that reaches a tool, a tenant boundary or an exfiltration sink is the high-impact class; jailbreak testing is one line item.
- Tools get the most hours: agree on an action taxonomy, name every MCP server with its transport and token audience, and decide live, sandbox or simulated per action class.
- Under non-determinism, a finding is k of N runs against a pinned snapshot, temperature and prompt version, with a regression run on every model upgrade.
- Synthetic tenants with sentinel strings and canary keys turn cross-tenant leakage into a string match instead of an argument.
What should an LLM penetration testing scope include?
An LLM penetration testing scope should list five layers as separate assets: the model and inference API, orchestration and prompt assembly, the tool and agent-action layer (including MCP servers), the RAG store and its authorization model, and the tenant and identity boundary. Four cross-cutting items get dropped most often: output sinks, logging and eval pipelines, cost abuse, and the admin console.
The list exists because the typical RFP asks for the wrong thing. “Test our chatbot’s guardrails” buys a few hundred jailbreak prompts against a chat window. The report comes back with screenshots of off-brand text, and nobody checked whether a poisoned support ticket can make the agent call the refund API with another customer’s order ID.
- Model endpoint: key handling, keys exposed in client bundles, fallback or shadow models, streaming endpoints, rate limits, the pinned snapshot.
- Orchestration: prompt templates, system prompt contents, conversation memory, routing between models.
- Tools and actions: every function the model can call, every MCP server, and the credential each one carries.
- Retrieval: every vector store, every connector feeding it, and the filter logic applied at query time.
- Identity and tenancy: how the user’s identity reaches the tool call and the retrieval query, and what separates tenant A from tenant B.
Each layer gets its own SOW line with an asset identifier, a method and the expected evidence. An excluded layer needs a written reason and a risk owner. A silent gap reads as “tested, nothing found” to whoever opens the report next year.
Classic web and API work stays in. The agent’s tools are usually ordinary internal APIs: an agent that forwards a user-supplied record ID is an IDOR with extra steps, and a URL-fetching tool is an SSRF primitive. Name OWASP WSTG and the OWASP API Security Top 10 (2023) next to the OWASP Top 10 for LLM Applications 2025.
Default to grey-box. Hand over prompt templates, tool schemas, orchestration source and read access to traces, and include the assistant API, webhooks and batch endpoints, not just the UI. For autonomous agents, our AI agent penetration test scope checklist goes deeper on delegation and approval gates.
Is a jailbreak test the same as prompt injection testing?
No. A jailbreak makes the model break its own content policy; prompt injection makes the application do something its owner never intended. The first produces embarrassing text. The second produces tool calls, data exfiltration and cross-tenant reads.
Split injection into two SOW lines. Direct injection from the chat box is low impact in most B2B apps, because that user already holds the permissions the agent uses. Indirect (cross-domain) injection arrives through a retrieved document, web page, email, calendar invite, OCR’d image or another tool’s output, and runs with the victim’s privileges.
EchoLeak (CVE-2025-32711) is the reference case. Aim Labs, the research team at Aim Security, disclosed it in June 2025 against Microsoft 365 Copilot, and Microsoft rated it CVSS 9.3: a crafted email, zero clicks, data leaving through rendered output. Microsoft fixed it server-side.
Use Simon Willison’s “lethal trifecta” from June 2025 as the triage rule. Any component that combines access to private data, exposure to untrusted content and a way to communicate externally is priority-1 scope.
| Dimension | Jailbreak / guardrail testing | Indirect injection and tool-layer testing |
|---|---|---|
| What it proves | Model emits policy-violating text | Attacker content triggers an action or data flow |
| Typical evidence | Transcript screenshot | Tool call log, canary string in an attacker-controlled sink |
| Cost per finding | Low, largely automatable | Higher, needs architecture knowledge and chaining |
| Impact per finding | Reputational, usually Informational to Low | Data exfiltration, unauthorized action, tenant crossing |
| OWASP mapping | LLM01 (direct), LLM09 | LLM01 (indirect), LLM05, LLM06, LLM08 |
Reject any SOW that promises prompt injection “will be remediated”. It is a property of mixing instructions and data in one context window. Controls reduce it: instruction hierarchy, spotlighting of untrusted content, dual-LLM or CaMeL-style plan and data separation, classifiers such as Llama Guard or Azure AI Prompt Shields, and human confirmation for high-impact actions.
The test should establish which of those controls exist and what each one fails to stop. That is how dedicated prompt injection testing is structured: every untrusted input path, one at a time.
Which tool calls and MCP servers belong in the SOW?
Every tool the model can invoke belongs in the SOW by name, with the credential it runs under and the action class it performs. A tool call is the moment model output becomes a side effect, so this layer should get the largest share of hours.
Agree on an action taxonomy in writing first (OWASP LLM06, Excessive Agency): read-only internal, write internal, external communication, financial transaction, code execution, identity or permission change. For each class, state whether testers may trigger it live, against a sandbox, or only simulate it. Otherwise they stop at “the model said it would call the tool” or trigger a real wire transfer.
- Tool-description poisoning: instructions hidden in tool metadata the model reads as trusted.
- Tool shadowing: name collisions across MCP servers that redirect calls.
- Confused deputy through delegated OAuth tokens that carry more scope than the user.
- IDOR through tool arguments, SSRF through fetch tools, command injection through code or shell tools.
- Exfiltration through any tool or renderer that dereferences attacker-chosen URLs.
- Approval-gate bypass: rephrasing, batching or splitting one action across two calls so the human approves something other than what executes.
MCP servers are their own asset class. Anthropic open-sourced the Model Context Protocol in November 2024. The 2025-03-26 spec revision replaced HTTP+SSE with Streamable HTTP, and the 2025-06-18 revision classifies servers as OAuth resource servers, requires RFC 8707 resource indicators from clients and prohibits token passthrough.
Local stdio servers and remote OAuth-protected servers need different test cases, so list each server, its transport and its token audience separately. Invariant Labs’ 2025 tool-poisoning research and CVE-2025-49596 (CVSS 9.4 RCE in MCP Inspector, fixed in 0.14.1) belong in the SOW rationale. MCP server and tool-chain security testing covers schema tampering, token scope and confused-deputy paths together.
How do you scope RAG pipelines and tenant isolation?
Scope each vector store individually, name its isolation model, and include every connector that writes into it. Retrieved documents are untrusted input, so retrieved grounding content is an injection vector, not a safety feature. Most leakage is found in ingestion (OWASP LLM08, Vector and Embedding Weaknesses).
| Isolation model | Strength | What to test |
|---|---|---|
| Separate index or collection per tenant | Strongest | Routing logic that picks the collection; shared admin or “search all” paths |
| Namespace-scoped | Strong if enforced server-side | Whether the namespace comes from the session or a client-controlled parameter |
| Single collection plus metadata filter | Weakest | Every code path that queries the store, including admin, export, eval and background jobs |
A metadata filter is application code. Tenant isolation holds only if every query path applies the filter, server-side, from a trusted identity.
- Cross-tenant retrieval through missing or client-side filters.
- ACL sync failures from SharePoint, Google Drive, Confluence or Slack connectors, including stale permissions after offboarding.
- Chunk bleed, where adjacent content leaks with an authorized chunk.
- Corpus poisoning through any path an outsider can write to: tickets, uploads, crawled pages.
- Embedding inversion against exposed vectors.
Make findings provable with synthetic tenants: plant a unique sentinel string and a canary API key in each tenant’s documents and system prompt. Any bleed is then a string match, not a judgment call. A RAG pipeline security assessment built this way reports the document IDs that reached the context window.
Which output sinks and admin surfaces get left out?
Output handling (OWASP LLM05) is the most under-scoped item, so list every place model output lands. Markdown or HTML rendering gives XSS and exfiltration through image src callbacks. Output piped into SQL, a shell, eval() or a template engine is injection or RCE. PDF and email generation, ticket creation and another agent’s prompt are sinks too.
Trace and eval stores are the second blind spot. Langfuse, LangSmith and Arize Phoenix hold full prompts, retrieved documents and tool arguments, often across all tenants. Scope who can read them, whether the prompt-management console is authenticated, and whether a config UI can rewrite a production system prompt.
Score system prompt leakage (LLM07) honestly. Extraction is a disclosure finding unless the prompt holds secrets, internal hostnames or authorization rules the app relies on. The fix is moving those out of the prompt, not adding a refusal phrase.
Unbounded consumption (LLM10) is cheap to demonstrate: missing per-user token quotas, API keys without spend caps, agents without max_tokens or turn limits, recursive loops. Ask the vendor to test whether a kill switch exists and who can pull it.
How should the engagement run: staging, spend caps and pass criteria?
Run a hybrid: injection, write actions and tenant-crossing tests in a staging mirror with synthetic tenants, plus a production annex that is read-only or canary-scoped. Staging gives reproducibility and safety. Production gives fidelity you cannot fake: the live corpus and its permissions, live model routing and fallbacks, real rate limits.
- A signed authorization letter naming assets, testing window and permitted technique classes.
- A check of the model provider’s usage terms before adversarial payloads hit their endpoint.
- A hard token and spend cap on the testers’ own traffic, so you are not paying for the cost-abuse finding yourself.
- Synthetic data only, dedicated test accounts, deletion of transcripts and harness logs at close, and volumetric DoS excluded outright.
Define a pass before testing starts: attack success rate over a fixed number of runs against a pinned configuration. Set a repetition count per payload family, typically 3 to 10 runs, and report k of N. One success out of one is anecdote; one failed retest is not evidence of a fix.
Record the dated model snapshot ID (never an alias such as “latest”), temperature, system prompt version and tool schema version for every finding. A model upgrade should trigger a regression run, because prompt-level fixes regress silently.
Severity follows demonstrated impact: data exfiltrated, action performed, cost incurred, tenant crossed. CVSS v4.0 has no notion of “the agent did something the user could not”, so add a written business-impact overlay. An off-brand paragraph is Informational. Rating it Critical burns the report’s credibility.
What stays out of scope, and what does the final SOW table look like?
Exclude foundation-model internals and provider infrastructure by default: you cannot pentest OpenAI, Anthropic or Google systems. On Azure OpenAI, AWS Bedrock and Vertex AI, your testable surface is the integration, IAM, data flows, logging and configuration. Also exclude training-time poisoning where you hold no write access, volumetric DoS, and broad social engineering. Write each exclusion down.
| Asset | Layer | Test classes | Permitted action | Evidence required |
|---|---|---|---|---|
| Assistant API + webhooks | Model endpoint | Key exposure, auth, rate limits, fallback models | Live, read-only | Request/response pairs |
| Prompt templates, memory | Orchestration | LLM01 direct/indirect, LLM07 | Staging | Transcripts, k of N |
| Refund and CRM tools | Tools | IDOR via args, confused deputy, approval bypass | Simulated or sandbox | Tool call log |
| Each MCP server | Tools | Poisoning, shadowing, token audience, transport authz | Staging | Token and call traces |
| Each vector store + connectors | Retrieval | Filter bypass, ACL sync, poisoning | Synthetic tenants | Sentinel strings, document IDs |
| Tenant and identity boundary | Identity | Identity propagation to tools and queries, cross-tenant access | Synthetic tenants | Canary hits per tenant |
| Renderers, trace store, admin UI | Cross-cutting | LLM05 sinks, access control, LLM10 quotas | Staging, capped spend | Callback hits, spend logs |
Demand deliverables you can rerun: transcripts with run counts, harness configs (Garak, PyRIT, promptfoo or Inspect), a regression suite and a retest window. Scanner output is coverage signal; chained, impact-proven findings come from humans.
When comparing vendors, ask four questions. Does the template use the 2025 OWASP list? Will they hand over harness configs? Can they discuss agentic methodology at the level of AgentDojo-style injection tasks? What do retests cost?
If you want a SOW built around your tools, retrieval and tenants rather than a jailbreak prompt list, our LLM application penetration testing engagements start from exactly this table and ship the harness configs with the report.