Anthropic co-founder and CEO Dario Amodei has published an essay urging artificial intelligence companies to deliberately limit how quickly they expand the capabilities of their models. He argues that the goal is not to halt technical progress, but to ensure companies leave enough time for alignment work, safeguards and independent verification.
Amodei laid out a three-step plan that he said could temper the pace of development without “sacrificing commercial advantage or the United States’ lead in AI.” He said Anthropic has already made a unilateral commitment on the first step by giving third-party evaluators employee-level access so they can verify safety practices and report incidents.
The second step calls for coordination among leading AI companies in democratic countries to establish common safety standards. The third envisions coordination between democratic governments and authoritarian governments. Amodei acknowledged that some parts of the plan may prove easier to carry out than others.
The essay came after Anthropic researcher Jacob Coxon, who previously also worked at OpenAI, announced this week that he had left the company, triggering a social media backlash. Coxon said he resigned because Anthropic and OpenAI were “gambling with our lives,” and added that people building AI “earnestly believe that it could kill us all by the end of the decade.”
While those warnings are extreme, fears that AI could lead to human extinction or other catastrophic outcomes have circulated in research circles for years. In 2023, a group of prominent AI researchers and executives, including Amodei and OpenAI CEO Sam Altman, signed a statement saying that mitigating extinction risk from AI should be a global priority alongside threats such as pandemics and nuclear war.
Amodei wrote that calls to pause or slow AI development have been around since 2023, but said such a move made “little sense” at that time because models were not yet powerful enough to act in the real world or capable of “significant deception, manipulation, cheating, or cyberattacks.” At the same time, he stressed that he still believes AI can dramatically improve human life if it is built the right way.
His proposal was welcomed by a number of researchers and industry executives. Sam Altman wrote on X that he agreed the sector needs to manage the speed at which advanced AI capabilities are developed. He said the issue has been a primary topic of discussion at OpenAI in recent weeks and added that independent evaluators with employee-like access are “a great idea,” saying OpenAI will do the same.
Earlier this month, OpenAI chief scientist Jakub Pachocki warned that no AI company has yet solved alignment and monitoring well enough to keep scaling responsibly at maximum speed for much longer. He said voluntary slowdowns should become commonplace until shared safety bars are in place.
Elon Musk also backed the idea, writing on X that “Dario is right.” Musk, whose rival AI startup xAI was acquired earlier this year by his rocket company SpaceX, had previously been a vocal critic of Anthropic. But after Anthropic announced a major compute deal with SpaceX in May, his tone shifted markedly. At the time, he wrote that everyone he met was highly competent and cared deeply about doing the right thing.
Anthropic is also preparing for what markets widely expect to be a historic IPO, though the company has not officially disclosed a listing timeline. Amodei said that if slowing down buys even an extra year or two before models reach critical capability levels, and that time is used to improve alignment, the risk of a serious failure could be significantly reduced.
