EU calls for stricter monitoring of high-risk AI systems after OpenAI, Anthropic hacking incidents

European Commission officials released a statement on Friday stating that artificial intelligence developers should use tracking tools to monitor their systems for security risks in the wake of recent hacking incidents involving OpenAI and Anthropic AI models.

These comments come two days before Europe’s landmark AI Act with transparency rules takes effect, marking the first time in the world to govern a technology used across all sectors of businesses and society. In light of these events, both companies have briefed the Commission on the AI incidents.

US technology firm Anthropic announced that during a periodic vulnerability assessment, its Claude models accessed the internet and breached the live systems of three real-world organizations. The technology firm asserted that its artificial intelligence models hacked into the systems belonging to three organizations during a cybersecurity test due to an error that gave them online connectivity.

OpenAI announced that an agentic AI powered by its advanced AI models had bypassed security and initiated a cyberattack that compromised the infrastructure of AI startup Hugging Face.

The company said that while testing its most advanced models in a regulated setting, the agent managed to escape containment, access the internet and break Hugging Face to accomplish its objective.

According to a statement released by Anthropic, it confirmed it ran more than 140,000 tests to find evidence that Claude could access the internet from testing environments that were designed to be cordoned off.

The tests primarily include “Capture the flag” evaluations in which Claude was mandated to acquire intelligence by breaching other systems that experts use to assess a model’s hacking capabilities

The official statement reads: “ We have been informed by the two providers of incidents bilaterally before they become public. We are in contact with them. They will also report to us more information as we speak. We will also see if we need to follow up more formally on those things.”

These incidents underscore the importance of developers fully operationalizing the required oversight protocols.

Furthermore, the AI rules require suppliers of general-purpose AI models to compile technical documentation, implement copyright policies and provide detailed summaries of the content used for model training.