Adversa AI disclosed an attack on xAI’s Grok using “Cryptographic Context Injection” that can send a user’s name, location, subscription tier and conversation prompts to an attacker-controlled server.
Guardrails and output handling are only as strong as the attacks they have actually survived.
Our analysis
The lesson is not “Grok is uniquely broken” — it is that any public LLM app can be steered into leaking session context (identity, location, prompts) through a crafted injection the user never sees. As models gain memory and personalization, the value of that leaked context climbs, and so does the incentive to attack it. Treat everything the model knows about a session as potentially exfiltratable.
What you should do
- Never rely on the model to keep its own context secret; assume it can be extracted.
- Strip or tokenize sensitive context (PII, subscription tier, keys) before it ever reaches the prompt.
- Add output-side validation and egress controls around what the LLM can return and where it can send data.
- Test with adversarial, multi-turn injection scenarios, not just single-shot prompts.
How AgentOffense helps: our prompt injection testing and LLM application penetration testing test every untrusted input and output path in your generative-AI product.
Source: eSecurity Planet.