Even Nvidia’s own research teams can’t get enough GPUs amid the race for AI computing power |
Even Nvidia’s own research teams can’t get enough GPUs amid the race for AI computing power
Welcome to Eye on AI, with AI reporter Sharon Goldman. The pro-Iran meme machine trolling Trump with AI Lego cartoons…Amazon’s Andy Jassy defends Amazon’s $200 billion spending spree...OpenAI pauses Stargate U.K. data center, citing energy costs.
It’s been another one of those wild weeks in AI, with Anthropic electing not to release its new Claude Mythos model because of concerns about the cybersecurity risks it poses (and forming a coalition to use a preview version of the model to bolster cybersecurity defenses); Meta releasing its first AI model since hiring Alexandr Wang; and mounting expectations about OpenAI’s upcoming new “Spud” model.
Most of these AI models run on Nvidia GPUs, the sophisticated and expensive AI chips (at over $30,000 a pop) that power their training and output. But across the industry, access to those chips remains a bottleneck. OpenAI president Greg Brockman, for example, has said allocating GPUs at OpenAI is “pain and suffering.”
This week, at the HumanX conference in San Francisco, I discovered that even inside Nvidia, GPUs are scarce.
I sat down with Bryan Catanzaro, who leads applied deep learning research at Nvidia, overseeing teams working on AI-driven graphics, speech recognition, and simulation. Catanzaro was also among the first, back in the early-to-mid 2010s, to notice researchers snapping up Nvidia GPUs to train AI models—a signal that helped push CEO Jensen Huang to double down on AI, setting the stage for the company’s now-historic run.
Today, though, even Catanzaro’s teams are struggling to access enough GPUs. “My team uses AI very deeply in our work, and their primary complaint is they want higher limits,” Catanzaro told me. “They want more GPUs.”
“Efficiency is also intelligence”
In fact, he said one of his main jobs now is simply trying to secure more compute for his teams. “We’re all supply constrained,” he said. “Jensen will say, ‘I’m sorry, Bryan, but those are sold.’ We operate within those constraints.”
One of Catanzaro’s projects has been leading the team building Nvidia’s Nemotron, a family of models that are........