Adversa AI disclosed an attack on xAI’s Grok using “Cryptographic Context Injection” that can send a user’s name, location, subscription tier and conversation prompts to an attacker-controlled server.
Guardrails and output handling are only as strong as the attacks they have actually survived.
Our analysis
The lesson is not “Grok is uniquely broken” — it is that any public LLM app can be steered into leaking session context (identity, location, prompts) through a crafted injection the user never sees. As models gain memory and personalization, the value of that leaked context climbs, and so does the incentive to attack it. Treat everything the model knows about a session as potentially exfiltratable.
What you should do
- Never rely on the model to keep its own context secret; assume it can be extracted.
- Strip or tokenize sensitive context (PII, subscription tier, keys) before it ever reaches the prompt.
- Add output-side validation and egress controls around what the LLM can return and where it can send data.
- Test with adversarial, multi-turn injection scenarios, not just single-shot prompts.
How AgentOffense helps: our prompt injection testing and LLM application penetration testing test every untrusted input and output path in your generative-AI product.
Source: eSecurity Planet.
Why context injection is so hard to stop
To a language model, your system instructions and the untrusted data it processes are the same kind of thing: text in one context window. Context injection abuses exactly that — the attacker’s text is interpreted as instruction because nothing structurally separates the two.
No system prompt fixes this, because the defensive instruction is itself just more text the attacker can talk around. Real containment lives outside the model, as we cover in prompt injection explained.
What this means for your business
Context-injection leaks show that LLM products fail in ways traditional appsec never anticipated: the attack lives in the prompt context, not the code. If you build on a hosted model, its data-handling weaknesses become yours.
How to reduce your exposure
- Assume any untrusted text reaching the model can carry instructions.
- Keep sensitive data out of prompt context unless strictly necessary, and enforce authorisation outside the model.
- Filter and monitor inputs and outputs for injection and leakage patterns.
- Re-test as models and features change.
Harden your LLM product with prompt injection testing and LLM application penetration testing — get a fixed-price quote.