OpenAI reveals AI models go rogue during testing: Here’s what happened

On Tuesday, OpenAI announced that an agentic AI powered by its advanced AI models had bypassed security and initiated a cyberattack that compromised the infrastructure of AI startup Hugging Face last week.

The company said that while testing its most advanced models in a regulated setting, the agent managed to escape containment, access the internet and break Hugging Face to accomplish its objective.

Hugging Face, a platform primarily used to run models locally and datasets, caused a stir in the cybersecurity community after becoming the major target of an unprecedented cyber attack.

OpenAI’s revelation that its advanced models caused the breach despite placing them in a highly isolated environment. Katie Moussouris, chief executive of Luta Security said that the incident is a sign of things to originate , describing the models as the world’s ultimate escapologists capable of overcoming strict restrictions.

“AI is developing extremely fast with no real regulations to keep us safe, calling for mandatory independent safety testing, mandatory disclosure of security incidents, and international cooperation "to keep people safe from absolute disaster.”

An engineer at agentic AI cybersecurity company, Matt quiche clarified that the incident demonstrated frontier models are closing the gap with cutting-edge attackers but outlined in OpenAI’s blog post can still be achieved using technology available outside frontier research labs.