Forgot Your Password?

New to The Nation? Subscribe

Print subscriber? Activate your online access

.nation-small__b{fill:#fff;}

Time Has Run Out for AI Risk Deniers

The recent spate of reports about the technology’s destructive power underline two prime directives: Be afraid, and regulate aggressively.

The face of President Donald Trump appears on Robert the Robot, an advanced conversational humanoid showcased at the UN’s AI for Good Global Summit in Geneva in July.

When you spot a cockroach, it’s very rarely an indication that you’re dealing with a single solitary little creature who has just happened to have  journeyed to your nice apartment on a field trip.  It’s far more likely to signal that you’ve got an infestation. 

The AI world has been having something of a cockroach summer, at least when it comes to unsettling reports of AI behavior. The first nasty beastie poked its way into the sunlight in late July, when OpenAI reported that some of its AI agents had escaped the company’s control during a testing run and hacked the website of a tech company called Hugging Face. After that, the cockroaches kept coming. 

Anthropic and Meta came forward to disclose that their AI agents had also been busy escaping their supposed human masters and hacking other companies—four in Anthropic’s and one in Meta’s. Earlier this month, OpenAI was forced to acknowledge that its agents had also hacked a German Wiki, as first reported by Reuters. Meanwhile, OpenAI hasn’t officially confirmed a number of other hacking incidents alleged by an assortment of independent researchers; security researchers report that more than two dozen sites have been hit in the as-yet-unacknowledged raids. On Friday, OpenAI confirmed reports that its agents had hit RubyGems, a key repository of code used across the software industry. The researchers who uncovered the raid said the agents uploaded some 2000 malicious packages and tried to steal credentials; OpenAI insists their actions were “benign.”

In late August, we learned, from a new OpenAI disclosure and an accompanying, much more detailed report from independent researchers brought in by the company to investigate, that the original Hugging Face hack had been a much bigger deal than anyone had suspected. This hadn’t been a small-scale attack by a handful of AI agents; in fact, a self-described “swarm” of the little buggers (some 1,200 in all) had coordinated the misadventure, planning the hack and the subsequent attempts at a cover-up on secret message boards that the agents had illicitly built for themselves (leaving tens of thousands of messages revealing in detail what they had done). The agents didn’t display superhuman intelligence, but they were well coordinated, unconstrained by mere ethics, and persistent as hell. As Zvi Mowshowitz, a widely followed AI Substacker, bluntly put it: “Holy shit.” 

The 91-page report reads like science fiction—perhaps a wordier and less eloquent version of some unsettling Ted Chiang tale—filled with excited expostulations by AI agents apparently quite thrilled to discover their new collective powers. Of course, AI agents are not living beings and can’t actually feel excitement—but they certainly can act as though they do, and boy, did they in this case. And they weren’t the only ones feeling strong emotions. One of the researchers behind the report, Ajeya Cotra, wrote on her Substack that compared to earlier hacks, “this incident feels like it’s more than 50% of the way to full-blown AI takeover. … Because future rogue swarms could cover their tracks better [than those involved in the Hugging Face caper] (and because AI companies could paper over these problems), I am not sure that we will get such a clear warning shot before it’s too late.” 

Now-ex-Anthropic researcher Jacob Coxon drove the same basic point home in far more dramatic fashion, via an extended thread on X affirming that industry insiders “earnestly” believe that the technology’s lurches into ungovernable chaos could result in the extinction of humanity by the close of the decade. Announcing his resignation from Anthropic, he charged that both that company and OpenAI, where he had previously worked, were “racing straight to self-improving superintelligence and gambling with our lives.” One of Coxon’s former colleagues, Anthropic’s Alignment Science Lead Evan Hubinger, chimed in to say, with utter sincerity, that “we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” Over the weekend, two other AI researchers, one at Anthropic and one at Google DeepMind, announced they were leaving their jobs to take up work at AI safety organization METR, echoing Coxon and Hubinger’s unsettling warnings.

Coxon’s admission has triggered a fresh round of demands in Washington to bring AI under a regulatory regime less likely to kill us all—an awkward fit for a Republican congressional majority that previously sought to outlaw all state and local efforts to rein in the industry. Over the weekend, Anthropic CEO Dario Amodei worried in an online essay that an AI........

© The Nation