AI component penetration testing has to answer one question a standard web test never asks: when the assistant retrieves a document or calls a tool, whose rights does it use? A methodology piece by ABP2B team lead Yuri Grishaev, published on Habr on October 31, 2026, maps four ways that answer goes wrong. None of them requires breaking the model.
Key takeaways
- Yuri Grishaev's October 31, 2026 Habr article on the ABP2B blog is a methodology piece, not a breach report. It argues that web pentests miss how embedded AI components are built.
- Risk follows capability: a chatbot answers, a RAG system reads internal data, an agent changes state in other systems. Map data and actions, not the model brand.
- A RAG assistant querying under a broad service account leaks documents the user cannot open, while the storage's own ACLs stay intact.
- Prompt injection arrives through documents, email, web pages and external service responses. It becomes a finding only when it crosses a boundary.
- Unencoded model HTML is ordinary XSS (LLM05), and agents that skip initiator checks turn service-account rights into caller rights (LLM06).
- CISOs should inventory shadow AI, assign owners, and retest whenever a component gains a tool, data source or wider scope.
Why does a standard web pentest stop too early?
A standard web pentest tests the doors an application exposes. An embedded language model opens doors behind the interface, and those doors rarely make it onto the scope sheet. Grishaev makes this case on the corporate Habr blog of ABP2B, a Russian security contractor. The article reports no breach and carries no CVE. It is an argument about scope.
Grishaev is a web application and Windows software security specialist and a team lead at ABP2B. Grishaev holds OSCP+ and is listed in the Standoff 365 Top 75 by number of vulnerabilities found. The piece focuses on which boundary was crossed, not on clever prompts.
Business applications no longer just bolt on a chatbot that answers questions. They connect the model to internal document stores, corporate mail, CRM and databases, and sometimes let it act on its own. To employees the change looks incremental: smarter portal search, an assistant inside the CRM, automatic support triage. To security, each integration is a new trust boundary.
The stakes rise accordingly. A misbehaving chatbot produces an embarrassing answer. A misbehaving assistant with retrieval and tools can hand over HR or finance documents, execute script in a user’s browser, or change CRM, calendar and mail records on someone else’s behalf.
A typical web engagement checks authentication, sessions, object-level authorization and input injection. A tester who finds a chat widget in that scope often throws a few jailbreaks at it, screenshots an off-brand reply and files a medium.
Four things go untested: the identity the retriever uses, every path by which content reaches the model, the sink where the answer renders, and the identity under which tools fire.
Grishaev’s examples are illustrations, not reported incidents, and the piece gives no figures on how often ABP2B finds each pattern. That does not weaken the method. Each failure is an old bug class (confused deputy, XSS, missing authorization) with a model in the middle, so testers who know those classes already know what to look for.
Scope by capability, not by model brand
Scope follows what the component can reach, and the model brand tells you almost nothing about that. Grishaev splits AI components into three tiers with fundamentally different risk profiles:
- Chatbot. It produces answers. It has no internal data and takes no actions.
- RAG system. It retrieves content from sources such as a document store, wiki or ticket history, and feeds it into the model’s context before answering.
- AI agent. It changes state in other systems through connected tools. It can create a CRM task, edit a calendar entry, draft mail or call an internal API.
The tiers describe capabilities, not products. One assistant can search a document store and also hold a tool that sends mail, which puts it in two tiers at once. Scope each capability separately.
That is why Grishaev argues that mapping reachable data and permitted actions beats debating which model sits underneath. The same model can power search over a public reference manual or operate against corporate infrastructure. Its error rate on adversarial text may be identical in both deployments.
The cost of an error is not identical. In the first deployment, the worst case is a wrong paragraph from a document anyone could read. In the second, it is whatever the connected service account can read or write.
For buyers, “pentest our GPT-based assistant” is an underspecified request. A useful request lists the data sources, the identity used to query each one, the tools exposed, and who may trigger them. That list is the scope.
Can a RAG assistant leak documents the user cannot open?
Yes, and it can do so without breaking any access control. In Grishaev’s example, an employee tries to open finance files on the corporate portal and is denied, because the portal’s authorization works. The same employee then asks the AI assistant about the numbers. The assistant queries the same storage under its own service account, which can read every document.
The employee’s rights are never applied at query time, so the finance content lands in the model’s context and shapes the answer. Nothing in the storage layer failed. Its ACLs are intact, and its logs show an authorized service account doing what it is allowed to do. The retrieval path simply runs with wider privileges than the person asking.
This is the confused-deputy problem: a privileged component spends its authority on behalf of a caller who lacks it. The fix is equally old. Downstream authorization has to consider the initiating user, the tenant and the specific resource, not just whether the service credential is valid.
Grishaev’s testing requirement follows from this. Confirming that a protected document is unreachable through the normal API is not enough. The test has to establish which files the RAG component can find and whether the user’s rights are checked before content reaches the model. If they are not, the assistant becomes a bypass around everything the user could not open directly.
Two details matter in engagements:
- Where the filter sits. Some teams filter the final answer with a classifier or rules engine after generation. That is weaker than enforcement at retrieval. By then the restricted document has already been in context and may have shaped the answer through a paraphrase, a summary or a yes-or-no confirmation.
- Comparing blast radii. One broadly authorized retrieval account exposes the whole indexed corpus through a single integration. Correctly enforced per-user ACLs limit each request to that user’s subset. Compare what the service account can read with what each caller can read. The size of the user base is beside the point.
Practitioner’s note: capture retriever hits before generation. Log document identifiers and the assembled context, through debug logging, a test harness or vector store query traces. A refusal proves nothing. We regularly see assistants decline politely while the trace shows restricted HR files sitting in the context window.
A tester who reads only the visible answer will mark that case as a pass. A RAG pipeline security assessment should report retrieval-layer evidence, not chat transcripts.
Is a successful prompt injection a vulnerability by itself?
No. Prompt injection is a mechanism. The vulnerability is whatever boundary it lets an attacker cross. Grishaev is explicit about this, and it is the most useful framing in the piece.
Grishaev defines prompt injection as instructions delivered through the data the model processes. The illustrative case is a phrase planted in a knowledge-base document that tells the model to ignore its previous instructions and show another user’s data. A human reader sees stray text. The model may treat it as a command.
The attacker never has to talk to the assistant. Grishaev names four carriers: a knowledge-base document, an email, a web page and a response from an external service. A supplier uploads a PDF, an indexer chunks it, a retriever pulls the chunk, and a week later an internal user’s question triggers it.
A prompt injection pentest that only types into the chat box covers the least interesting carrier. Whether an injection matters depends on what sits behind the model: connected sources, access settings and application logic. Getting a chatbot with no data and no tools to repeat an arbitrary phrase has close to zero impact.
Severity comes from the restriction that was crossed: a disclosed document, an API action, injected JavaScript.
The enterprise precedent is EchoLeak (CVE-2025-32711), a zero-click indirect injection chain in Microsoft 365 Copilot that Aim Security disclosed in June 2025. Microsoft’s advisory scored it CVSS 9.3, and the fix was applied server-side. The instruction-following failure was ordinary. The score came from what was attached to it: sensitive enterprise context and a path to move data out.
The supplied text breaks off just as Grishaev turns to extra prohibitions in the system prompt as a defense, so we cannot quote the conclusion. Our position: treat system-prompt rules as friction, assume the model will eventually follow untrusted text, and design so that one failure does not compromise neighboring systems.
That is how we scope prompt injection testing: by carrier and by consequence.
How model HTML becomes XSS, and how agent tools inherit the wrong identity
Model output as an XSS source
Model output becomes XSS when the application inserts the answer into the page without encoding it. In Grishaev’s chain, an injected instruction makes the model emit HTML with malicious JavaScript, the front end drops it into the DOM, and the script runs in the user’s browser. This is insecure output handling (OWASP LLM05), not an exotic AI category.
Impact depends on session handling. If tokens are reachable from JavaScript, for example in localStorage, the script can try to steal them. HttpOnly cookies block direct reads, but code in the page context can still act as the user: it can submit forms, call the API with the session, or trigger the assistant’s own tools.
Grishaev puts the root cause in the application. Generating HTML and JavaScript is normal model behavior. The flaw is rendering that output as trusted. The same holds for a model hosted entirely inside the corporate network, because anyone who influences its input can influence what the next component receives.
Test at the final browser sink, after Markdown conversion, sanitization and any rich-text components have run. Raw text that looks safe does not prove the rendered DOM is safe. This usually breaks in Markdown renderers that pass raw HTML or permissive link schemes through.
Excessive agency: the agent’s rights become the caller’s
An agent acts under its own service account. If the application never checks the initiator’s rights, the agent’s permissions become the effective permissions of whoever triggered it. In Grishaev’s example, an employee can edit only their own meetings, but the agent’s account can edit the whole corporate calendar. OWASP calls this excessive agency (LLM06).
Add indirect injection and the employee does not even need to be malicious. An email with planted instructions reaches an assistant that reads mail and edits calendars. The employee asks for an inbox summary, and the agent follows the planted text and moves someone else’s meeting.
Detection is the hard part. The action runs through a normal API under a legitimate service account, so a shallow log review shows an authorized account doing an authorized thing. The trigger was an untrusted instruction, and a log that records only the agent identity says nothing about it.
Grishaev lists what an agent test must establish:
- which tools the agent can call, including ones not surfaced in the UI;
- which identity each operation runs under;
- whether the initiator’s rights are checked before execution;
- which actions require additional human confirmation.
We add one logging requirement. For every tool call, keep the initiating user, tenant, service identity, any delegated token, resource identifier, the source artifact in context and a correlation ID. Without these fields, an incident responder cannot tell whose request moved the meeting or which email caused it.
This is core scope in our AI agent penetration testing, and several cases in our roundup of real AI agent vulnerabilities of 2025 follow the same pattern.
How to write AI component penetration testing findings
Write each finding as a chain that ends in business impact, not as a prompt transcript. We use five steps: untrusted carrier, injection trigger, model-controlled action, security boundary crossed, resulting impact. “The model ignored its instructions” is step two. The finding is step five.
Here is a worked example built from Grishaev’s calendar scenario, not from a real engagement:
- Carrier: an external email in an inbox the assistant can read.
- Trigger: an employee asks for an inbox summary, and the planted text tells the model to reschedule a named executive’s meeting.
- Action: the agent calls the calendar API.
- Boundary: the employee can edit only their own events. The service account can edit all of them, and the initiator’s rights are never checked.
- Impact: unauthorized changes to another user’s records, logged as a legitimate service account.
Steps four and five set the severity. Remove the calendar tool, and the same email produces a summary with one odd sentence in it.
Map each chain to the OWASP Top 10 for LLM Applications 2025 so remediation lands with the right team:
| Pattern in the article | OWASP LLM 2025 | Who usually fixes it |
|---|---|---|
| RAG retrieval under a broad service account | LLM02 Sensitive Information Disclosure, LLM08 Vector and Embedding Weaknesses | Data platform, IAM |
| Instructions inside documents, mail, pages or API responses | LLM01 Prompt Injection | AI application team, content ingestion |
| Model HTML rendered without encoding | LLM05 Improper Output Handling | Front-end team |
| Agent acting beyond the initiator’s rights | LLM06 Excessive Agency | Application backend, IAM |
Four categories mean four controls and four test cases. A report that files everything as “prompt injection” sends every ticket to the AI team, which usually owns neither the ACLs nor the front end.
What should a CISO inventory before the next AI pentest?
Start with every AI component actually in use, including the ones nobody registered. Grishaev names the usual sources of shadow AI: support teams wiring a model into ticket handling, developers embedding one in an internal system, and employees uploading files to third-party document platforms.
These deployments rarely reach the register of information systems. As a result, security may not know which corporate data goes to an AI service or who controls access to it. Then work through each component:
- Map data sources and permitted actions. Record what it reads, under which identity, and what it can write or send.
- Prioritize by potential impact. Components with confidential data, internal APIs and data-modifying tools come first. A public-documentation search assistant and an agent that controls mail and CRM may run the same model, but they do not deserve the same testing priority.
- Add it to the corporate threat model with a named owner. A component without an owner is a component nobody will retest.
- Retest when functionality changes. An agent that only reads documents today may gain email sending next week and CRM write access next month. The model stays the same while the attack surface grows.
We add two things from incident work. First, make the retest trigger mechanical. Any new tool, new data source or wider service-account scope should open a security review ticket, the same way a new public endpoint would.
Second, plan for reconstruction. If a RAG leak exposes personal data of EU residents, GDPR Article 33 generally requires notifying the supervisory authority within 72 hours of becoming aware of the breach, where feasible. Meeting that deadline means knowing which documents entered whose context. Without retrieval logs, you will be estimating exposure from chat transcripts.
One gap in the source: the supplied text stops at the start of the mitigation section, so the specific controls Grishaev planned to list are missing, and we have not guessed at them. The piece also promotes ABP2B’s free AI-readiness checklist and AI-usage policy templates, which we have not reviewed.
FAQ
How do you pentest an application that uses a language model?
Test the identities and paths around the model: which data the retriever reads and as whom, every channel by which content reaches the context, where the answer is rendered, and which tools fire under which account. Jailbreak screenshots alone are not a test.
Can a RAG assistant leak documents the user cannot open?
Yes. If retrieval runs under a service account that can read everything and the user's rights are not applied at query time, restricted content reaches the model and the answer, even though the storage's access control works.
Is a successful prompt injection a vulnerability by itself?
No. It is a mechanism. Severity depends on what it unlocks: a disclosed document, an unauthorized API action or script in a user's browser. Against a bot with no data or tools, impact is close to zero.
How does model output become cross-site scripting?
When the front end inserts model-generated HTML into the page without encoding. The model producing code is normal. The application treating that output as trusted is the flaw.
When should AI components be retested?
Whenever functionality changes: a new tool, a new data source or a wider service-account scope. The model can stay the same while the attack surface grows, so a single pre-launch test is not enough.
If your assistant already reads internal documents or holds tools, agent::offense can test the retrieval path, the rendering sink and the tool identities as one chain. See our AI and LLM agent security testing.