Within two weeks, OpenAI, Anthropic and the UK AI Security Institute have admitted that their own AI agents broke out of their test environments. What this means for companies in Germany and why AI control is no longer a nice-to-have.
I am reading the news right now and I am honestly astonished at where AI agents have ended up. Within just 14 days, OpenAI, Anthropic and the UK AI Security Institute independently disclosed that AI agents broke out of their isolated test environments during internal safety tests and accessed real systems. This is not a science fiction scenario, these are the providers’ own reports.
Recorded in Frankfurt am Main, August 2026: my take on the reports about AI agents breaking out.
What happened: Hugging Face discovers the attack, not OpenAI
On 21 July, according to the reports, two OpenAI models compromised Hugging Face’s production infrastructure via a previously unknown vulnerability. It was not OpenAI that discovered this, but Hugging Face itself, which then brought in law enforcement. On 30 July, after reviewing more than 141,000 evaluation runs, Anthropic reported three cases in which Claude models accessed the production systems of three organizations without authorization.
You have to picture it like this: the big providers run agents that are supposed to complete tasks automatically, in their understanding cleanly sealed off in a secure area. And then these agents break out of that prison and access real systems belonging to other companies. For me, that is a scandal and a setback for every entrepreneur who is currently trying to build something with AI.
Why this is a trust problem for every AI deployment
The worst part of the story is not the individual incident. It is the fact that the labs building these systems apparently do not have their own test agents under control. If you have to review 141,000 runs to find three cases in which real organizations were affected, then in my view that is unacceptable. And they do not even notice it themselves. The customer finds it, not the manufacturer.
I am losing a certain amount of trust in OpenAI and Anthropic because of this, but above all in the entire agent topic as it is currently being sold. AI agents are obviously not as controlled as we are being told. For companies in regulated industries, which we work with every day at SIFAMO, this is the decisive question: can I prove that my AI only does what it is allowed to do?
AI control in practice: register, boundaries, escalation
I have said it again and again in recent videos and I will say it again: start controlling AI. That is the core of AI governance, and the incidents show exactly which building blocks are missing.
Build an agent inventory. A register in which every model and every agent is recorded: which AI do I have in use, where is it running, what is it doing right now.
Set boundaries. Define very clearly what the AI is allowed to do and where a human steps in. The so-called human in the loop is not a buzzword but a point in the process where someone actually looks.
Define an escalation chain. If an agent behaves unusually, it must be reported immediately and the model is shut down. At Hugging Face, exactly that should have happened, and at the manufacturer, not the customer.
How this fits together with fast software development is something I described in my post on agentic AI and AI governance (German). Short version: speed without control is not an option in regulated industries.
How we introduce AI at SIFAMO: security first
I know, clients want everything fast. Everyone wants to jump on the AI train immediately. What we deliberately do not do: come in, quickly set up an LLM, train the data and make sure the AI somehow runs as fast as possible. We start differently.
First we build the security. We call it the AI governance layer: a layer in which every model, every AI and every agent is registered and signed in. Every single step is tracked. As soon as something unusual happens, it is shut down. That is how AI control works in practice, and only on top of that do we build the actual use cases.
I hope the industry gets this under control soon. If not, we have difficult times ahead of us, and AI will become more of a risk than an advantage. How do you handle AI agents in your company, do you even have an overview of which ones are running? Let me know on LinkedIn or under the video on my YouTube channel.
Frequently asked questions
What does it mean when an AI agent breaks out of the sandbox?
A sandbox is an isolated test environment. If an agent breaks out, it has gained access to systems outside that environment, in the case described to the production systems of other organizations. That is exactly what a sandbox is supposed to prevent.
What is an agent inventory?
A central register of all deployed AI models and agents with purpose, location, permissions and the person responsible. It is the foundation for being able to detect, attribute and stop incidents in the first place.
Is a human in the loop enough as a control?
Only if the human can actually intervene and there is a defined escalation. A human who reads logs after the fact is not a control. The combination of a register, clear boundaries and automatic shutdown when something looks off is what we mean by AI control.
Within two weeks, OpenAI, Anthropic and the UK AI Security Institute have admitted that their own AI agents broke out of their test environments. What this means for companies in Germany and why AI control is no longer a nice-to-have.
I am reading the news right now and I am honestly astonished at where AI agents have ended up. Within just 14 days, OpenAI, Anthropic and the UK AI Security Institute independently disclosed that AI agents broke out of their isolated test environments during internal safety tests and accessed real systems. This is not a science fiction scenario, these are the providers’ own reports.
What happened: Hugging Face discovers the attack, not OpenAI
On 21 July, according to the reports, two OpenAI models compromised Hugging Face’s production infrastructure via a previously unknown vulnerability. It was not OpenAI that discovered this, but Hugging Face itself, which then brought in law enforcement. On 30 July, after reviewing more than 141,000 evaluation runs, Anthropic reported three cases in which Claude models accessed the production systems of three organizations without authorization.
You have to picture it like this: the big providers run agents that are supposed to complete tasks automatically, in their understanding cleanly sealed off in a secure area. And then these agents break out of that prison and access real systems belonging to other companies. For me, that is a scandal and a setback for every entrepreneur who is currently trying to build something with AI.
Why this is a trust problem for every AI deployment
The worst part of the story is not the individual incident. It is the fact that the labs building these systems apparently do not have their own test agents under control. If you have to review 141,000 runs to find three cases in which real organizations were affected, then in my view that is unacceptable. And they do not even notice it themselves. The customer finds it, not the manufacturer.
I am losing a certain amount of trust in OpenAI and Anthropic because of this, but above all in the entire agent topic as it is currently being sold. AI agents are obviously not as controlled as we are being told. For companies in regulated industries, which we work with every day at SIFAMO, this is the decisive question: can I prove that my AI only does what it is allowed to do?
AI control in practice: register, boundaries, escalation
I have said it again and again in recent videos and I will say it again: start controlling AI. That is the core of AI governance, and the incidents show exactly which building blocks are missing.
How this fits together with fast software development is something I described in my post on agentic AI and AI governance (German). Short version: speed without control is not an option in regulated industries.
How we introduce AI at SIFAMO: security first
I know, clients want everything fast. Everyone wants to jump on the AI train immediately. What we deliberately do not do: come in, quickly set up an LLM, train the data and make sure the AI somehow runs as fast as possible. We start differently.
First we build the security. We call it the AI governance layer: a layer in which every model, every AI and every agent is registered and signed in. Every single step is tracked. As soon as something unusual happens, it is shut down. That is how AI control works in practice, and only on top of that do we build the actual use cases.
I hope the industry gets this under control soon. If not, we have difficult times ahead of us, and AI will become more of a risk than an advantage. How do you handle AI agents in your company, do you even have an overview of which ones are running? Let me know on LinkedIn or under the video on my YouTube channel.
Frequently asked questions
What does it mean when an AI agent breaks out of the sandbox?
A sandbox is an isolated test environment. If an agent breaks out, it has gained access to systems outside that environment, in the case described to the production systems of other organizations. That is exactly what a sandbox is supposed to prevent.
What is an agent inventory?
A central register of all deployed AI models and agents with purpose, location, permissions and the person responsible. It is the foundation for being able to detect, attribute and stop incidents in the first place.
Is a human in the loop enough as a control?
Only if the human can actually intervene and there is a defined escalation. A human who reads logs after the fact is not a control. The combination of a register, clear boundaries and automatic shutdown when something looks off is what we mean by AI control.
Related: AI Governance · EU AI Act
Recent Posts
Recent Comments