Companies developing increasingly powerful artificial intelligence systems may eventually need to be legally required to install independently verifiable “kill switches” that can shut down dangerous AI tools, according to Anthropic co-founder Jack Clark.
Clark, one of the seven founders of AI company Anthropic, told the BBC that AI developers already have different mechanisms for pulling the plug on their systems, but governments may ultimately need to make such safeguards mandatory.
“Should you mandate for companies to definitely have a kill switch? Is that kill switch verifiable by a third party?” Clark asked, arguing that both questions should form part of the broader policy debate around artificial intelligence.
He said society may eventually want to establish rules requiring companies to demonstrate that their AI systems can be safely shut down if they become dangerous.
Clark’s comments come amid growing concern among AI researchers and executives about the speed at which the technology is advancing and the possibility that increasingly autonomous systems could pose serious risks.
Anthropic CEO Dario Amodei recently called for the pace of AI development to slow and for the technology to be subjected to closer monitoring.
Clark said allowing AI to develop as a “totally unregulated industry” would be a mistake.
“We are rolling dice with immense risks,” he said. “And the point is, we have to change the course of this industry.”
However, he questioned the usefulness of assigning a specific probability to the possibility of AI causing human extinction.
His comments came after Anthropic scientist Evan Hubinger said he personally believed there was a greater than 10 per cent chance of human extinction from AI within the next decade.
Computer scientist and Nobel Prize winner Geoffrey Hinton, widely known as the “Godfather of AI”, also recently told the BBC that a 10 per cent probability of AI killing all humans was “not unreasonable”.
