Top AI labs reportedly planning for catastrophic AI event fallout

Anthropic, OpenAI, and other AI labs are reportedly gaming out scenarios where a rogue AI wreaks havoc on critical infrastructure, like financial services, internet and telecommunications, or utilities such as power and water. According to Axios, the leadership of these companies is privately preparing for such an incident, similar to how the Pentagon has been running war games to see the results of different strategies in case of a nuclear exchange between nations.

Planning like this is not unheard of in the corporate world, with enterprises sometimes running business continuity exercises to account for contingencies. However, what’s concerning is that these exercises are in preparation for a major AI incident, something that many industry experts believe would happen within the next six to 12 months. "OpenAI conducts preparedness exercises where teams discuss and work through a range of potential scenarios," a company spokesperson told the publication. "These scenarios are not treated as inevitable but are meant to help us prepare for a variety of circumstances."

Dario Amodei recently released a warning about an AI-driven botnet “swarm” taking over the entire internet in the coming months, especially after incidents of OpenAI’s models escaping their testing environments. We’ve also heard of incidents where a “kill switch” failed to stop a rogue agent, forcing the startup to pause testing. The threat is so great that Anthropic listed “existential risks to humanity” as one of its major risk factors in its IPO prospectus, allocating nearly a third of the document to covering this issue. A former OpenAI safety employee has spoken out about this, saying that the current risks stem from a lack of safety culture — something that the aviation and nuclear industries have already learned through experience.

Despite that, the White House has refused to impose regulations on artificial intelligence, instead preferring to make Nvidia, OpenAI, Anthropic, and other AI labs promise to “self-police” AI development. But even though the current administration is lenient on AI labs and development, Axios says that one of the things that the scenarios focus on is how to educate Congress after a major incident. An event like this would probably result in Congress attempting to slow or shut down AI development, though planners think that such a move would face significant challenges. It will still likely have an impact on the economy, though, especially as big tech has already spent more than $1 trillion in AI infrastructure, with an additional $32 trillion expected to be invested until 2050.

The heads of Anthropic, OpenAI, and SpaceXAI have all agreed that a slowdown is needed for AI safety, but it’s unclear what steps these companies have taken to put that into effect. While gaming out scenarios like these is par for the course for any major organization, it still doesn’t give us comfort that they’re essentially preparing for “The Day After Tomorrow” of an AI apocalypse.

Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.