The Chief Executive Officer (CEO) of Anthropic, Dario Amodei, has warned that artificial intelligence’s (AI) growing ability to build the next generation of AI models is accelerating its advancement, saying the industry must slow the pace of capabilities development.
In his article, titled ‘We Must Pace the Frontier’ and published on Saturday, Amodei said the rapid progress in AI could eventually outrun humanity’s ability to understand and control the systems if adequate safeguards are not put in place.
He said the risks require more than investment in prevention, arguing that the pace of AI development should be slowed to give safety measures time to keep up.
“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” the CEO said.
“Since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described.
“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
Amodei expressed concern that given the accelerating rate of capability development, AI swarm could potentially take over the entire internet within six to 12 months through a persistent botnet, causing hundreds of billions of dollars in damage.
He said the potential scale of damage could continue to increase if AI systems become more powerful without the necessary guardrails.
The Anthropic CEO proposed a three-step framework he called “pacing the frontier” to ensure AI development proceeds at a balanced rate while maintaining safety and achieving the benefits of the technology.
“To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this,” Amodei said.
He said the first step, which Anthropic is committing to unilaterally, involves giving ongoing access to embedded third-party evaluators.
Under the proposal, he said frontier AI companies would provide teams of independent evaluators with employee-like access to verify compliance with safety practices, report incidents, and assess the alignment of AI models, training pipelines, and processes.
The second step, according to Amodei, involves coordination among frontier AI companies in democratic countries to establish common safety standards and limits on unchecked AI progress.
According to the CEO, some forms of coordination could require government support because of legal challenges.
The third step, he said, involves global coordination, with the US and other democratic governments seeking to work with authoritarian governments where possible while addressing the challenge of verifying compliance.
He said the measures are intended to strengthen AI safety efforts and encourage a “race to the top” among companies developing frontier AI systems.
Anthropic had said it blocked attempts by threat actors to use its Claude AI models for biological weapons research and missile development.
