OpenAI Chief Scientist Warns of Imminent AI Self-Evolution, Urges Industry Slowdown

OpenAI Chief Scientist Jakub Pachocki has warned that artificial intelligence is rapidly advancing toward the ability to evolve without human oversight, a development he argues requires "extreme prudence" from the industry. In a lengthy post on Sunday, Pachocki called on AI labs to consider voluntarily slowing the pace of development until shared safety standards can be established.

The Dawn of Recursive Self-Improvement

Pachocki stated that AI models can already manipulate computers, collaborate with humans and other AIs, and conduct research. He anticipates that soon, AI will achieve recursive self-improvement, a process where it can enhance its own capabilities without human intervention, which is considered a key step toward artificial general intelligence (AGI) and could trigger an "intelligence explosion."

"This is a moment that calls for extreme prudence," Pachocki wrote. "I am concerned that no one is truly prepared for the consequences of a continued rapid rise in machine intelligence." He proposed that developers must either more closely align AI with human interests or be prepared to "slow down future development if necessary."

An Industry Torn Between Caution and Competition

Pachocki's warning echoes concerns from other industry leaders. Anthropic CEO Dario Amodei has cautioned that AI could replace a significant number of junior-level jobs, and OpenAI CEO Sam Altman has made similar predictions. The concept of recursive self-improvement is also moving from theory to practice.

Ad

• Anthropic published an article in June detailing how its Claude model is already deeply involved in AI research and development, stating the path to self-improvement is now visible and calling for a potential pause in frontier AI development.
• SoftBank Group CEO Masayoshi Son recently revised his forecast, suggesting superintelligence could arrive within two years after learning from OpenAI that its "models are designing future models."
• OpenAI itself published a research paper in June noting "early signs" of recursive self-improvement in its current systems, warning that society and governments are unprepared.

Investment Rushes Toward Advanced AI

Despite these high-level warnings, the tech industry continues to race toward more powerful models, and investors are pouring capital into the very technology causing concern. Nvidia CEO Jensen Huang recently posted on X that the "AGI era has arrived" in response to a new OpenAI model release.

Meanwhile, a growing number of startups are focused on creating self-improving AI, attracting significant venture capital. Inherent, founded by former DeepMind researchers, raised $50 million earlier this year, while AI lab Recursive Superintelligence, founded by scientist Richard Socher, has secured $650 million in funding. This push continues even as top labs withhold their most powerful creations; OpenAI has limited the release of its GPT-6 Astra model and Anthropic has not publicly released its Mythos model, both citing advanced and potentially risky cybersecurity capabilities.