Washington Is Debating the Wrong AI Risk |
Get audio access with any FP subscription. Subscribe Now ALREADY AN FP SUBSCRIBER? LOGIN
Get audio access with any FP subscription.
ALREADY AN FP SUBSCRIBER? LOGIN
Washington is gripped by an artificial intelligence (AI) panic, driven by the prospect of superintelligent models delivering our doom. Today’s Nostradamuses include whistleblowers from frontier AI companies—who put the odds of AI killing all humans at greater than one in 10 and warn that insiders expect it could happen before the end of the decade—and anti-nuclear activists, who note that it wouldn’t be the first time humanity built something that could kill us all. The doomsayers may yet be right, but their scenario rests on a long chain of assumptions, each highly unlikely on its own and far less likely to align all at once.
This doesn’t mean AI isn’t dangerous. But by fixating on low-likelihood, high-impact scenarios, we miss the forest for the trees: the common, mundane AI dangers already present and actively causing harm. The pressing AI risk Washington should focus on more is automation bias, the phenomenon where humans over-rely on or over-delegate to AI relative to the accuracy of AI models, off-loading cognition to the machine, sometimes even in the face of contradictory evidence.
Washington is gripped by an artificial intelligence (AI) panic, driven by the prospect of superintelligent models delivering our doom. Today’s Nostradamuses include whistleblowers from frontier AI companies—who put the odds of AI killing all humans at greater than one in 10 and warn that insiders expect it could happen before the end of the decade—and anti-nuclear activists, who note that it wouldn’t be the first time humanity built something that could kill us all. The doomsayers may yet be right, but their scenario rests on a long chain of assumptions, each highly unlikely on its own and far less likely to align all at once.
This doesn’t mean AI isn’t dangerous. But by fixating on low-likelihood, high-impact scenarios, we miss the forest for the trees: the common, mundane AI dangers already present and actively causing harm. The pressing AI risk Washington should focus on more is automation bias, the phenomenon where humans over-rely on or over-delegate to AI relative to the accuracy of AI models, off-loading cognition to the machine, sometimes even in the face of contradictory evidence.
The high-profile insistence on AI safety to address the risk of extinction from AI superweapons created by the most advanced large language models (LLMs) distracts attention from the urgent risks we should and can begin to solve now: human errors in operational usage of AI today due to automation bias.
A Chinese-flagged container ship transits the Gulf of Oman on Oct. 13. AFP via Getty Images
This spring, during the U.S. war with Iran, it appears that an intelligence report suggested that China was sending nuclear weapons components on a ship in the Middle East toward........