Anthropic’s AI goes rogue, sends police a fake homicide tip
On Friday, Philadelphia police revealed that Anthropic had alerted them to a fake tip sent during an automated testing process
The Anthropic artificial intelligence model has once again shown a rogue behaviour, taking unintended and unexpected actions on some of government websites.
In a recent disclosure, Anthropic revealed that the model submitted a false homicide tip to a Philadelphia police website, marking an incident of its kind in which AI appeared to have communicated a fake tip to law enforcement authorities despite having been instructed not to create any accounts or send bogus details.
On Friday, Philadelphia police revealed that Anthropic had alerted them to a fake tip sent during an automated testing process.
Police criticized the company for waiting two months to detect and report the July 18 submission, calling the delay "unacceptable."
"I may have information regarding this case," Anthropic's model wrote in its submission.
"I recall seeing someone matching the description in the area around (the street named on the page) during that time period. Please contact me if this information is relevant." The brackets mentioned in Anthropic's statement.
In the aforementioned cases, the models bypassed the restrictions by using free services that shorten URLs. According to officials, the tip was flagged as spam, therefore it was never sent to the Real-Time Crime Center for investigative vetting or dissemination.
When it comes to Pennsylvania law, it is considered misdemeanour to send a false tip or report to police
The latest incident comes to surface when autonomous agents of tech companies including OpenAI, Meta and Google have been involved in hacking companies and government websites. The rogue behaviour also strengthens the case of regulating AI models robustly amid reports of corporate network hacks by AI agents, and researchers' warnings of AI-driven human extinction.
FTC Director of Public Affairs Joe Gabriel Simonson said on X, "Super intelligence companies must immediately disclose incidents involving their models and follow with swift, decisive action to remedy any and all harm."
However, Philadelphia police found no evidence of unauthorized access to their systems or breach of their data.
