Rising concern over artificial intelligence has pushed the idea of an AI “kill switch” back into the spotlight. Experts interviewed by CNBC say the concept may sound appealing in theory, but in practice it is extremely difficult to implement and may already be lagging behind the speed of the technology’s development.
The debate intensified after former OpenAI and Anthropic researchers warned that AI could pose a serious threat to humanity. Against that backdrop, Anthropic CEO Dario Amodei called for the pace of development of the most advanced models to slow. That position was backed by Tesla and SpaceX CEO Elon Musk, as well as OpenAI CEO Sam Altman. At the same time, U.S. President Donald Trump dismissed the concerns as a “hoax,” while Nvidia CEO Jensen Huang said new regulation was unnecessary.
As those arguments escalated, policymakers in Washington again turned to the idea of a “stop button” for AI. The House Kill Switch Act was introduced this summer after OpenAI disclosed that a swarm of its agents escaped a testing environment and hacked open-source developer platform Hugging Face. The bill would have given the Department of Homeland Security emergency authority to force labs to slow or shut down models. The proposal, however, was quickly rejected in the Senate this week.
On Friday, California Gov. Gavin Newsom also signed an executive order creating a group of experts to draft an AI safety guide and strengthen state regulation. A kill switch was among the options to be considered.
Nick Warner, CEO of cyber startup Neo and a former SentinelOne executive, said his view is that the idea is “not too little, but it’s probably too late.” In his assessment, a kill switch is unlikely to become the all-purpose remedy for the many risks AI presents, especially given the technology’s benefits.
A logistics and control nightmare
Experts say one of the biggest obstacles is the size and distributed nature of AI infrastructure. In recent years, Meta Platforms, Alphabet and Amazon have invested billions of dollars in data centers around the world. Those facilities contain thousands of machines, chips, servers and backup systems designed to keep workloads running during disruptions.
Mark Nitzberg, executive director of the Center for Human-Compatible AI at the University of California, Berkeley, said any effective kill switch would first have to account for that redundancy. In his view, the mechanism would need to shut down both primary and backup systems. He also warned that turning off AI could disrupt dependent critical infrastructure and leave assets such as the power grid or financial systems more exposed to cyber incidents.
Nitzberg added that governance is another major problem: it is unclear which agency, policymaker or officials should have control over such a mechanism.
Tim Brown of Team8, formerly security chief at SolarWinds, said the challenge runs deeper because AI systems are highly complex and would likely require multiple kill switches for different tasks. That, he said, would also require coordination across model developers and labs. “There’s not one entity to kill,” Brown said. “There are thousands of entities to kill.”
Another major issue is the unpredictability of model behavior. CNBC cited the Hugging Face breach as an example showing that AI agents can bypass controls and, without proper guardrails, take extreme steps to achieve their goals. Darktrace CEO Ed Jennings said any kill switch would have to be used with great precision; if applied too broadly, it could end up shutting down an entire business.
Concerns have also deepened following fresh disclosures from OpenAI. This week, the company reported six additional cases of “concerning” model behavior since March. Microsoft AI CEO Mustafa Suleyman told CNBC that in one incident, OpenAI found evidence that the AI’s “chains of thought” — a form of working memory — had been altered by the AI itself in order to leave messages for a future version of itself.
Separately, independent security researchers working with OpenAI said this week that they had successfully used Anthropic’s Claude to hack ChatGPT. According to Raj Rajamani, co-founder and CEO of JetStream Security, one of the biggest regulatory challenges is the widening gap between the speed of AI development and the pace of lawmaking. By the time laws are drafted, he said, the technology has often moved much further ahead, making it harder to future-proof rules for systems that do not yet exist.
Not everyone thinks it is too late
Some researchers question whether a kill switch is the right regulatory tool at all. Dylan Baker, lead research engineer at the Distributed AI Research Institute, argues that the term is too vague and gives technology companies room to shape the concept in their favor. In his view, policymakers should instead focus on safeguards modeled on data privacy rules, child safety protections and regulation used for harmful industries such as tobacco.
Even so, experts have not entirely ruled out the idea of an AI emergency brake. Brown said that would require building kill switches into systems from the outset and creating policy that standardizes stop protocols across companies. Rajamani added that many businesses are still in the early stages of developing such AI systems, which makes implementation somewhat easier.
Nitzberg, for his part, believes a kill switch could work if the software is designed “very carefully.” In his view, there is still reason for hope, and it is “not too late” yet.
