Artificial intelligence company Anthropic has urged governments and leading technology firms to consider a coordinated global pause on the development of the most advanced AI systems, warning that emerging models are beginning to show early signs of behaviours that could become difficult to control.
In a new report, the San Francisco-based company behind the Claude AI models said that temporarily slowing frontier AI development would “likely be a good thing” for global safety. However, it cautioned that unilateral action by a single company or country would be ineffective, as competitors would continue advancing rapidly in the absence of shared restrictions.
Anthropic argued that any meaningful pause would require agreement between major AI developers across multiple countries, particularly the United States and China, along with enforceable mechanisms to verify compliance. Without such coordination, it said, commercial and geopolitical pressures would make restraint difficult to sustain.
The company warned that the absence of a global framework could force firms and governments into making safety-critical decisions under intense competitive pressure. It compared the challenge to nuclear arms control agreements, while noting that AI systems are far harder to monitor due to the ease of concealing training processes compared with physical weapons infrastructure.
The proposal has already drawn criticism from parts of the tech industry and policymakers in Washington, where some officials argue that focusing on extreme risk scenarios may overstate current dangers and risk slowing innovation. Critics also warn that restrictions on AI development could allow rival powers, particularly China, to gain strategic advantage in a key emerging technology.
Despite this pushback, the White House has acknowledged the capabilities of some of Anthropic’s advanced systems, including its restricted “Mythos” model, which is currently available only to selected organisations due to cybersecurity concerns. It is not publicly accessible.
US President Donald Trump has also recently raised the possibility of cooperation with China on AI safety during diplomatic discussions, while simultaneously signing an executive order requiring a 30-day government review of the most powerful AI models before public release.
Anthropic said internal data suggests AI systems are already accelerating their own development, creating a feedback loop that could eventually lead to what researchers describe as “recursive self-improvement,” where AI systems begin improving themselves with limited human input.
While the company stressed that such a scenario has not yet materialised and is not guaranteed, it warned that the timeline could be shorter than many governments expect. It also noted that the role of humans in AI development is steadily diminishing as systems become more capable of performing increasingly complex tasks.
The company said it plans to convene policymakers, researchers and industry leaders in the coming months to explore possible governance structures, arguing that international coordination may be essential to managing the next phase of artificial intelligence development safely.
You must be logged in to post a comment Login