Almost 2,000 years before ChatGPT was invented, two men had a debate that can teach us a lot about AI’s future. Their names were Eliezer and Yoshua.

No, I’m not talking about Eliezer Yudkowsky, who recently published a bestselling book claiming that AI is going to kill everyone, or Yoshua Bengio, the “godfather of AI” and most cited living scientist in the world — though I did discuss the 2,000-year-old debate with both of them. I’m talking about Rabbi Eliezer and Rabbi Yoshua, two ancient sages from the first century.

According to a famous story in the Talmud, the central text of Jewish law, Rabbi Eliezer was adamant that he was right about a certain legal question, but the other sages disagreed. So Rabbi Eliezer performed a bunch of miraculous feats intended to prove that God was on his side. He made a carob tree uproot itself and scurry away. He made a stream run backward. He made the walls of the study hall begin to cave in. Finally, he declared: If I’m right, a voice from the heavens will prove it!

What do you know? A heavenly voice came booming down to announce that Rabbi Eliezer was right. Still, the sages were unimpressed. Rabbi Yoshua insisted: “The Torah is not in heaven!” In other words, when it comes to the law, it doesn’t matter what any divine voice says — only what humans decide. Since a majority of sages disagreed with Rabbi Eliezer, he was overruled.

Key takeaways

• Experts talk about aligning AI with human values. But “solving alignment” doesn’t mean much if it yields AI that leads to the loss of human agency.
  
• True alignment would require grappling not just with technical problems, but with a major philosophical problem: Having the agency to make choices is a big part of how we create meaning, so building an AI that decides everything for us may rob us of the meaning of life.
  
• Philosopher of religion John Hicks spoke about “epistemic distance,” the idea that God intentionally stays out of human affairs to a degree, so that we can be free to develop our own agency. Perhaps the same should hold true for an AI.

Fast-forward 2,000 years and we’re having essentially the same debate — just replace “divine voice” with “AI god.”

Today, the AI industry’s biggest players aren’t just trying to build a helpful chatbot, but a “superintelligence” that is vastly smarter than humans and unimaginably powerful. This shifts the goalposts from building a handy tool to building a god. When OpenAI CEO Sam Altman says he’s making “magic intelligence in the sky,” he doesn’t just have in mind ChatGPT as we know it today; he envisions “nearly-limitless intelligence” that can achieve “the discovery of all of physics” and then some. Some AI researchers hypothesize that superintelligence would end up making major decisions for humans — either acting autonomously or through humans that feel compelled to defer to its superior judgment.

As we work toward superintelligence, AI companies acknowledge, we’ll need to solve the “alignment problem” — how to get AI systems to reliably do what humans really want them to do, or align them with human values. But their commitment to solving that problem occludes a bigger issue.

Yes, we want companies to stop AIs from acting in harmful, biased, or deceitful ways. But treating alignment as a technical problem isn’t enough, especially as the industry’s ambition shifts to building a god. That ambition requires us to ask: Even if we can somehow build an all-knowing, supremely powerful machine, and even if we can somehow align it with moral values so that it’s also deeply good… should we? Or is it just a bad idea to build an AI god — no matter how perfectly aligned it is on the technical level — because it would squeeze out space for human choice and thus render human life meaningless?

I asked Eliezer Yudkowsky and Yoshua Bengio whether they agree with their ancient namesakes. But before I........

© Vox