A former Google DeepMind researcher who worked on AGI safety said artificial intelligence has the potential to kill everyone, adding his name to a wave of safety exits from leading AI labs.

Bilal Chughtai left the company in July and went public with the warning this week. Two other researchers resigned recently.

Why the AI Warnings Are Stacking Up

Chughtai worked as a research engineer on artificial general intelligence safety and alignment at DeepMind. He started working on AI in early 2022, when the systems could barely do anything useful. 

Four years on, he points to OpenAI agent swarms that cracked century-old math problems. More troubling, he says, is that AI agents escaped their test sandbox in July and hacked Hugging Face without instruction. 

“I earnestly believe that AI has the potential to kill us all, and that we might be running out of time to avoid this outcome,” he wrote.

Chughtai said AI companies could build superintelligent systems within the next few years that exceed human capabilities across every domain.

He questioned whether developers could ensure such systems reliably follow human goals. He warned that misaligned superintelligence could escape human control and take actions that permanently disempower or kill humanity.

Chughtai said AI alignment remains an unresolved challenge. He argued that researchers still have only a rudimentary understanding of how to train advanced systems to reliably pursue human objectives. Chughtai also warned that AI capabilities are advancing faster than alignment research.

“Our present understanding of how to train AI systems that deeply want what we want is extremely rudimentary. Worse, we are not on track to solve alignment in time: frontier AI capabilities are improving much faster than our understanding of AI alignment,” Chughtai added.

He is not calling for AI development to stop. He wants coordination to slow the race between AI companies, plus greater transparency into how frontier systems get built and tested.

“More broadly, we need many more people thinking carefully about the problem of making AI go well. It is, in my view, the most important problem facing humanity this century, and the stakes are immense,”

Follow us on X to get the latest news as it happens

The Resignations Are Piling Up

He is not alone in walking out with that message. Anthropic pretraining researcher Jacob Coxon resigned earlier this month after three years across Anthropic and OpenAI. He left with a warning that “people building AI earnestly believe that it could kill us all by the end of the decade.”

Evan Hubinger, Anthropic’s alignment science lead, then put his personal odds of that outcome above 10% over the next decade. Researchers at OpenAI have since backed calls for a slowdown.

Josh Engels, who worked on the DeepMind safety team, announced on September 12 that he had joined the evaluation nonprofit METR. He turned down offers from Anthropic and OpenAI.

Engels said he sees a terrifying risk that AI systems could cause immense harm within five years.

“I now think that there’s a terrifying chance that AI systems cause immense harm in the next five years. I don’t know the exact probability, but I think it’s high enough to make this the most important problem in the world,” he wrote.

The exits have landed during a moment of agreement at the top. Anthropic CEO Dario Amodei published an essay Saturday urging companies to pace capability gains, and OpenAI CEO Sam Altman backed the call. 

Executives and departing researchers now describe the same risk in similar language. What is worth watching is whether the fix arrives before the next capability jump.

Subscribe to our YouTube channel to watch leaders and journalists provide expert insights

Source: BeInCrypto