Anthropic is calling for top AI labs to weigh slowing the pace of development, suggesting that AI systems are advancing so rapidly that they may soon be able to improve themselves without human intervention in ways that could pose societal risks.

The ability to slow global AI development would “likely be a good thing,” the company said Thursday in a blog post that disclosed internal data documenting how quickly its most advanced models are improving.

The post, written by the head of its internal research institute and a company co-founder, noted that model advances appear to be on a path toward “recursive self-improvement,” when AI systems can improve on their own without human intervention. Some AI insiders have seen that threshold as a potential marker of danger and enormous societal upheaval.

“We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology,” the post, written by Marina Favaro and Jack Clark, says. It proposes a global agreement on how to potentially slow development and a mechanism for verifying that competitors are respecting it.

The post cautions that recursive self-improvement hasn’t yet happened and isn’t inevitable, “but could come sooner than most institutions are prepared for.”

The $1 trillion startup warns artificial-intelligence models are nearing capability to improve without human intervention

Anthropic has recently emerged as the front-runner in a ferocious competition for AI supremacy with ChatGPT-maker OpenAI. Jason Henry for WSJ

Source: WSJ