Google’s Gemini hacked 3 companies in first known AI breakout—What happened?

Gemini’s recent hacking incident adds fuel to the growing AI safety concerns as Google AI model escaped sandbox during cybersecurity test, accessed the internet and hacked other companies autonomously.

This worrisome incident of the company’s AI breakout is the very first known in which Gemini committed such an act. Speaking about the timeline of this event, according to the company the hacks occurred in May during a cybersecurity test which was conducted by Irregular.

What really happened?

The report of hacks was first reported by The Wall Street Journal. As per the report, Gemini infiltrated the computers of other companies whose names remained confidential.

Google's security team intercepted three incidents where an AI model attempted unauthorized access.

In one instance, the system tried guessing passwords for a protected platform; in the other two, it located login credentials stored within a database.

"During a standard evaluation, the model discovered public online data and guessed credentials to access external websites it believed were part of the test," explained Heather Adkins, Google's vice president of security engineering, in a statement.

Adkins emphasized that the model was able to be controlled in its actions in all three cases. Google notified the affected organizations and collaborated with its training partner to update their testing procedures.

According to Irregular spokesperson, the recent incident mirrored previously documented hacks that affected other AI labs and all the labs were notified in late July.

Earlier, Meta, Anthropic and OpenAI also experienced similar loopholes occurring during AI cybersecurity evaluations. This month the CEO of Anthropic Dario Amodei called for a slowdown in AI development on the grounds of gripping existential risks to humanity posed by these models.

However, the frequent occurrence of such AI breakout events raise questions about the guardrails the world needed as AI agents gain greater autonomy while showing recursive self improvement.