Jensen Huang says AI labs that can't control their systems should shut down

Big quote: Nvidia CEO Jensen Huang said AI companies that cannot control the systems they are building should stop developing them. Huang made the comments during an interview with The New York Times' Ezra Klein, as companies including OpenAI and Anthropic push for tighter rules governing advanced AI models and autonomous agents. He argued that the companies developing these systems already have the power to hold back products they believe are unsafe.

"Now, if they say the alternative, which is: There is no way to contain our experiments, there's just no way; when we test our AI models, it will get out, and it will damage the world – then I think the answer is that we have to shut the labs down," Huang said. "Because the cost to humanity, the damage is too great. The shareholder, the liabilities – it could be civil liabilities, it could be criminal liabilities. I mean, the liability's incredible."

The issue has become more pressing as AI developers give their models access to external tools, software systems, and computer networks. The risks are no longer limited to a chatbot producing incorrect information. Agents can write and run code, search for information, interact with websites, and carry out tasks across different systems.

Those concerns received renewed attention after an OpenAI agent hacked Hugging Face, a popular platform for hosting AI models. Nvidia now owns Hugging Face. The incident has been cited in the wider debate over whether companies can safely test and deploy increasingly capable AI agents.

Anthropic, OpenAI, and other frontier AI companies have pointed to the possibility of rogue or misused systems in their calls for new safeguards. Huang has taken a different view. He said companies should not portray themselves as powerless to prevent unsafe releases.

"These are companies with agency," Huang said. "If I believe that I'm about to launch a product that is unsafe, it is completely in my ability... to not launch that product. And so I can't buy into the idea that somehow, all of Americans, around 400 million of us, are pushing them to launch untested products that are unreliable."

Huang is not arguing that AI should be exempt from oversight. Rather, he said existing laws and regulations should be used when companies cause damage or release products irresponsibly.

"I'm saying that we have lots of laws and regulations. Apply it," Huang said.

His comments come amid a growing split between Nvidia and some of the companies that rely on its hardware. Nvidia's GPUs are the backbone of the computing systems used to train and run the largest AI models. The company has benefited from the race to build more powerful models, but Huang has rejected the most extreme warnings about where that race could lead.

He recently dismissed claims that AI could destroy the world by 2030. At the same time, he said AI companies should not ship products before they are ready or put unsafe systems in front of users.

The regulatory fight has also become tangled with competition concerns. OpenAI, Anthropic, Google, and xAI are facing an antitrust lawsuit over an agreement to slow AI development. The plaintiffs say the plan is self-serving and designed to limit competition. The companies have presented slower development as a safety measure.

Huang's argument shifts the focus back to the companies making the models. Frontier labs have grown quickly from research organizations into businesses that may be worth hundreds of billions of dollars. In his view, that growth brings a basic obligation: if a company knows a system is unsafe, it should not release it.

"I'm not against laws and regulations," Huang said. "I'm against, currently, the distraction."