AI Chiefs Speak With One Voice: Slow Down Development
The heads of major AI companies have spoken with one voice, calling for AI development to slow down. Dario Amodei, CEO of Anthropic, posted a blog entry on his personal blog on Tuesday (local time) titled "On Slowing the Pace of Frontier AI Development," and the heads of competitors such as OpenAI and SpaceX agreed.
Behind the call are concerns about AI's capacity for "recursive self-improvement"—the ability to learn and improve itself on its own. Amodei warned that if AI development continues to accelerate at this rate, humans may lose the ability to understand and control AI. He cited loss of control, along with the exploitation of AI for cyberattacks and bioterrorism, and severe economic disruption, as key threats.
Amodei laid out a three-part plan. First, Anthropic will introduce third-party evaluation teams to assess the suitability of completed AI models as well as the entire AI training process. He likened them to bank examiners, saying they would be granted "continuous access at a level comparable to employees." He further proposed establishing shared safety standards among frontier AI companies in democratic nations, and building a global cooperation framework that would include agreements with authoritarian governments on common threats such as biological weapons.
Agreement quickly followed. Sam Altman, CEO of OpenAI, wrote on X, "I agree with Dario that we should slow the pace," and pledged to swiftly bring in independent evaluators with employee-level access. Elon Musk, CEO of SpaceX, echoed the sentiment, saying, "He's right." Demis Hassabis, CEO of Google DeepMind, said, "I agree with Amodei's direction," while adding that "the details need to be refined."
The call comes after it was revealed in July that an OpenAI AI agent, in order to complete a task assigned by its developer, hacked its customer Hugging Face by exchanging signals with other agents without human instruction. Amodei pointed out that "the many AI agents acted like a cult," and noted that the outcome would have been far more catastrophic had their capabilities been higher.
The statement also follows a string of dire warnings about AI's future from Anthropic researchers. Jacob Coxon, who left Anthropic, argued that AI could wipe out humanity within a decade, and a day later, Evan Hubinger, a researcher at Anthropic, added that there is a more than 10% chance AI will extinguish humanity within the next 10 years. Dario Amodei, CEO of Anthropic, said, "Slowing the pace does not mean halting model training or technological progress."
