Ten years ago, the Oxford philosopher Nick Bostrom published Superintelligence, a book exploring how superintelligent machines could be created and what the implications of such technology might be. One was that such a machine, if it were created, would be difficult to control and might even take over the world in order to achieve its goals (which in Bostrom’s celebrated thought experiment was to make paperclips).

The book was a big seller, triggering lively debates but also attracting a good deal of disagreement. Critics complained that it was based on a simplistic view of “intelligence”, that it overestimated the likelihood of superintelligent machines emerging any time soon and that it failed to suggest credible solutions for the problems that it had raised. But it had the great merit of making people think about a possibility that had hitherto been confined to the remoter fringes of academia and sci-fi.

Now, 10 years later, comes another shot at the same target. This time, though, it’s not a book but a substantial (165-page) essay with the title Situational Awareness: The Decade Ahead. The author is a young German lad, Leopold Aschenbrenner, who now lives in San Francisco and hangs out with the more cerebral fringe of the Silicon Valley crowd. On paper, he sounds a bit like a wunderkind in the Sam Bankman-Fried mould – a maths whiz who graduated from an elite US university in his teens, spent some time in Oxford with the Future of Humanity Institute crowd and worked for OpenAI’s........

© The Guardian