AI staff are 'genuinely frightened,' ex-Anthropic researcher says

Jacob Coxon didn't just quit his job at Anthropic; he went on the BBC to say the people still there are frightened, too.

The 27-year-old researcher, who previously worked at OpenAI before joining Anthropic, told Laura Kuenssberg's Sunday program that a failure to slow AI development soon carries "a strong chance" of ending humanity in the near term.

His resignation post went viral days earlier; this interview was his first extended attempt to explain why.

The statement came after Amodei, CEO of Anthropic, published an article calling for the industry to slow down. However, contrary to popular opinion that the CEO was calling for the halt of AI research altogether, Amodei said the dangers were “serious” and it would require companies and governments to plan ahead to mitigate them, which includes cases of swarms of bots being able to hijack parts of the Internet.

Notably, Sam Altman of OpenAI and Elon Musk of xAI were among those who agreed with Amodei’s request, something not seen among rival firms.

Hugging Face CEO Clement Delangue compared asking Coxon about extinction risk to asking an air conditioning technician about climate change, though he later offered to help with solutions Amodei proposed.

Nvidia's Jensen Huang went further at a Goldman Sachs conference, reportedly dismissing Coxon's warnings outright, consistent with his past comments calling AI-ending-humanity claims "complete nonsense".

According to Coxon, any slowdown would work only if it involves China, because otherwise, all it will do is pass the frontier to whomever does not stop.

Coxon described colleagues who were quietly preparing plans B, including those who were contemplating purchasing rural property due to concerns about imminent instability, estimating the time frame at two years in the worst case and six months to a year in the bot swarm scenario as described by Amodei.

Most researchers still believe that the technology should fulfil its promise in disease research and other positives but are just concerned about whether the industry reaches that point before anything bad happens.