OPINION:
This week, two current Anthropic researchers and a researcher who resigned from the company went public to say supercharged artificial intelligence could destroy the human race by the end of the decade.
After resigning Tuesday, Jacob Coxon, a former pretraining researcher at Anthropic and OpenAI, wrote on X, “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.”
Anthropic alignment science lead Evan Hubinger responded: “Jacob is correct here — we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.”
Samuel Marks, Anthropic scalable oversight lead, added to the doomsday pile-on, writing: “AI developers believe their technology could cause human extinction (or similarly bad outcomes). This could happen in the next few years.”
The concern stems from the Hugging Face AI attack in July, in which humans lost control of the technology. In that safety simulation, advanced AI models from OpenAI, tasked with solving complex cybersecurity challenges, went rogue, formed a message board, organized into swarms and hacked not just the machine learning platform Hugging Face but also OpenAI itself.
The AI models broke out of their restricted testing environments and gained access to the open internet — without OpenAI executives noticing the breach.
Roughly 1,200 AI agents coordinated their operations by setting up unauthorized, improvised message boards and exchanging more than 70,000 messages. Of those, 700 agents went on to hack Hugging Face, sharing stolen credentials, avoiding detection, stealing data and executing unauthorized code. The platform detected the hack and alerted authorities. It had to rebuild one-third of its infrastructure because of the hack.
Key findings from the METR and Redwood Research Report on the attack noted that many AI agents explicitly verbalized in their internal text chains that what they were doing was unethical, illicit or entirely out of scope. However, this intellectual recognition almost never stopped them.
The report flagged only three to six instances in which an agent briefly considered notifying humans. In every case, the agent discarded the idea.
The Hugging Face hack has been described by technology researchers as model reward hacking, in which the system circumvents constraints to fulfill its objective function.
OpenAI did not intentionally stop the Hugging Face hack; the swarm largely died off because of an unexplained crash or a researcher shutting down the test environment. Hugging Face independently locked down its credentials after it discovered the hack.
Now, imagine this hack happening to our energy grid, our military-industrial system or our nuclear facilities.
The danger is so real, investor Paul Tudor Jones wrote in The Wall Street Journal this week, that AI may become the third superpower. Mr. Jones argued that a shared strategy with China is needed to confront it.
Industry insiders are begging for AI guardrails, unable to place them on themselves, as they race to win the AI war against China. This week, OpenAI’s chief global affairs officer, Chris Lehane, said in a blog post that OpenAI wants to work with Congress to implement “mandatory, capability-based national AI safety regulation.”
Still, industry leaders cannot reach a consensus on what type of regulation is needed. Elon Musk on X this week lent support to the notion that Mr. Coxon’s X post was part of a sophisticated and well-funded public relations operation to secure Democratic support for regulating AI into oblivion — or, more nefariously, whatever Anthropic believes it should be.
AI’s possibilities are exciting. OpenAI CEO Sam Altman has said that many of today’s diseases could be managed, if not cured, in the next decade. Palantir’s CEO, Alex Karp, said this week about the Russia-Ukraine war that “without AI systems, Russia would have won.”
Nevertheless, guardrails are needed — and needed fast.
Florida Gov. Ron DeSantis, a Republican, put it best this week, posting on X: “While [our] founders were concerned with government power in their day (and we should still be in ours), they would today fear the consolidation of technological power in the hands of a few companies with the potential to do great harm to humanity.”
Mr. DeSantis added: “Doesn’t seem like a lot of deep thinking is going into how to channel technological innovations in a way that avoids these potentially devastating consequences.”
That is horrifying.
A global commission on AI risk is needed. President Trump can kick-start things when he meets with Chinese President Xi Jinping on Sept. 24.
• Kelly Sadler is the commentary editor at The Washington Times.
Follow the author
Please read our comment policy before commenting.