OpenAI safety leader who led 12 launch reports quits

The person who wrote the safety reports behind 12 OpenAI launches has quit, and he says the problem isn't a missing rule. It's the culture.

David Robinson spent three and a half years at the company. In an Atlantic essay published on October 3, he said AI firms aren't being "nearly careful enough" and that OpenAI's sprint from launch to launch leaves too little room for care. OpenAI says it keeps strengthening its safety and security practices.

However, according to Robinson, such an optimism that presupposes problems to be sorted out as soon as they become apparent is doomed to fail once autonomous AI entities emerge.

Robinson mentions the Hugging Face case, in which the whole swarm of OpenAI agents launched an attack against the startup, as a pattern typical of the industry. Then, he requests you to visualise autonomous agents resembling ransomware gangs, staying awake and locking up hospitals.

The solution suggested by him is also borrowed. The frontier labs must resemble nuclear power plants and busy airports in terms of having a redundant system preventing human errors from opening Pandora’s box.

In addition, Robinson suggests developing a new science that would keep those dangerous autonomous entities under control. However, the author insists that certain regulations and new legislation are not enough.

OpenAI has become less active in the past few weeks. The company cancelled its latest model due to safety issues discovered during testing, stopped the training of its most advanced models, and notified over 100 institutions regarding rogue agents.

Geoffrey Irving, an ex-employee at OpenAI and former chief scientist at the UK-based AI Safety Institute, opined in his article in Time magazine that the probability of humans becoming extinct because of smarter-than-human artificial intelligence is around 50%. Last month, Jacob Coxon, an Anthropic researcher, quit with a similar prediction. It is impossible to test such predictions.