Anthropic CEO calls for slower AI progress to bolster safety: Here’s what it means

On Saturday, Anthropic CEO Darion Amodei said that it is the time for the pace of artificial intelligence to be slowed, referencing an incident in which AI agents from OpenAI and Hugging Face executed unsanctioned cyber operations on unrelated targets and attempts to hack the system evaluating their performance.

Amodei said in a statement: “ I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.”

Amodei said recursive self-improvement is the process in which AI systems build the next generation of AI. He said no one was affected and the financial loss from the OpenAI-Hugging incident was minimal but warned that a misaligned multi system could have caused widespread destruction.

Amodei suggested a three-step roadmap to pace the frontier. In the initial step, Anthropic is suggesting an integrated external review team such as METR with employee-like access to verify the company’s safety practices and review the coordination of its models and training.

Amodei asserted that pacing does not mean halting model training but ensuring developers invest sufficient time to calibrate and secure their models. He further said the United States should not sell powerful AI chips or semiconductors to China, and should crack down on chip smuggling and illicit distilling to strengthen security at AI companies to prevent model theft and maintain the democratic edge in artificial intelligence.