In controlled safety tests described earlier this year, researchers asked AI models to solve a series of simple math problems. Partway through the exercise, the instructors warned the bots that if they tried to solve the next problem, the computer environment they were operating in would be shut down. In some runs, the shutdown occurred as specified. In others, models interfered with the shutdown script and continued with the remaining problems.

What explains this behavior? Do artificial intelligence systems have a drive to survive?

Resistance to instructions

Consider a robot vacuum cleaner. When it detects that its battery is running low, it returns to its charger, recharges itself, then resumes cleaning. The machine is keeping itself going, but no one is panicking about an existential Roomba threat to humankind because the behavior is straightforward. Designers built in the ability to recharge.

AI models are more complex. For years, researchers have discussed scenarios in which an AI agent might act to prevent shutdown so it could complete the tasks it had been assigned. Staying “on” could be a means to an end, even without an explicit instruction to resist shutdown.

One recent experiment showed that when researchers spelled out to an........

© The Conversation