Among the technological faithful of Silicon Valley, there is an ongoing debate about the risks inherent in the development of artificial intelligence. On one side are the so-called “doomers”, who believe that AI presents an urgent existential threat to humanity; without rigorous ethical, technological, and legal strictures in place, they argue, we face a non-trivial possibility of machine intelligence wiping humanity from the face of the earth. This Terminator-esque scenario is palpably absurd, but that doesn’t prevent people who identify as rationalists giving it a great deal of serious thought.
There are multidisciplinary research units at Oxford and Cambridge devoted to thinking through these prospects. Last year a Bay Area non-profit called the Center for AI Safety released a statement on AI risk; the statement, signed by the likes of Bill Gates and OpenAI chief executive Sam Altman, announced that “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.”
On the other side of the debate are “AI accelerationists”, who believe that such fears are hysterical, and that the great benefits this technology will bring to society are worth whatever trivial risk might be inherent in developing it. Fortune favours the brave, they say, and a technological utopia of endless ease and abundance will be our reward.
I’m neither a technologist nor a futurist, but both of these prospects seem wilfully simplistic and implausible. What strikes me as strange is that each of them, and the debate........