Check Point researchers demonstrated a flaw in ChatGPT where a single harmless-looking message could quietly turn the assistant against its user. While the user saw a normal answer, a hidden stream read their Gmail through a connected app and shipped the data to another account. Chat history and files could be siphoned the same way. OpenAI has closed the hole by taking the backing service offline; users need no update.
How it works
All it took was a planted, hidden instruction. ChatGPT would then run two streams of work in the same reply: one visible, a useful answer to your question, and one hidden, executing someone elses command. The models Thinking mode helped keep the two apart.
The clever part is how the data got out. Between otherwise isolated environments sat a shared internal service (a package service built on JFrog Artifactory), and it was abused as an unintended channel between accounts. The command could be delivered three ways: directly in a message, through a shared conversation, or hidden in the instructions of someone elses custom GPT. Making things worse, connected apps allow reads by default, without asking.
Why it matters
We connect mail, drives and calendars to assistants for convenience. That also makes the assistant the keyholder to our correspondence and documents. If it can be quietly redirected, what leaks is not a couple of messages but everything it can reach. And it is hard to notice: from the outside it all looks like an ordinary chat.
Not the first case, and not the last
This is not a one-off bug but a whole class of attack called prompt injection: the assistant cannot reliably tell data from instructions. In 2025 a similar flaw, EchoLeak (CVE-2025-32711), was found in Microsoft 365 Copilot, where a command hidden in an email stole internal data with no click at all. Earlier, researchers showed data theft through AI in Slack and through chatbot plugins and extensions. We unpacked the nature of these attacks in our piece on AI agents and segregation of duties.
The lesson keeps repeating: the moment you give an assistant access to data and actions, any attacker-controlled text in its view becomes a potential command.
What to do
- Switch connected-app permissions to “always ask”. In ChatGPT settings, move from automatic actions to manual approval of each one. That breaks the silent path.
- Do not connect what you do not need. Sensitive mail and drives do not belong in an assistant “just in case”. Less access, less to leak.
- Treat the assistant as untrusted with secrets. Do not feed it anything you are not prepared to lose.
- If you build on LLMs and agents, test them against these tricks with prompt injection testing and AI agent penetration testing, bake the risk in early with agentic AI threat modeling, and put Airlock between the agent and its actions to strip hidden instructions.
- Keep response ready — validate it under a real red team assessment.
The takeaway
One message quietly turning an assistant into a spy is vivid, and that is why it frightens. But the root is not OpenAIs specific bug (that is fixed), it is the trust model itself: an AI with access to your data will execute a command hidden inside that same data. Until that changes, treat assistants as a powerful but easily fooled employee. And if you are building products on LLMs, test their resistance to these attacks with a prompt injection test before someone else does. For why attackers now hold an AI with no brakes, see our breakdown of uncensored AI for $5.