AI firm warns, advanced models could escape human control

Artificial intelligence company Anthropic is advocating a global pause on the development of the most powerful AI systems, warning that the latest models are beginning to show signs that they could eventually evade human control.

The San Francisco-based company, which develops the Claude family of AI models, said in a report that a worldwide slowdown in frontier AI development would “likely be a good thing”.

However, it cautioned that if only one company halted development, competitors would simply continue advancing the technology.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep pace with the advancement of the technology,” the company said.

Anthropic noted that any meaningful pause would require the participation of multiple leading AI companies across several countries, particularly the United States and China, with all parties agreeing to halt development simultaneously under arrangements that could be independently verified.

The company has faced criticism from others within the industry, as well as from officials in the White House, who argue that its emphasis on worst-case scenarios exaggerates the risks and serves as a means of slowing competitors under the guise of safety concerns.