The rapid evolution of artificial intelligence is outpacing humanity's ability to manage it, and warnings from within the industry are becoming increasingly impossible to ignore.
OpenAI's chief scientist, Jakub Pachocki, took to social media on Sunday to voice his deep concern over the current pace of AI development. He is urging major AI laboratories to voluntarily slow down their research efforts until unified safety standards are established.
He cautioned that AI models will soon possess the capability to improve themselves without any human intervention, a prospect whose consequences could be unpredictable. This statement carries significant weight, coming as it does from a core figure within OpenAI itself.
Despite this, market enthusiasm for AI capability leaps shows no sign of waning. Last Sunday, Jensen Huang declared "AGI has arrived" following OpenAI's release of its latest model, investors are pouring hundreds of millions of dollars into AI self-improvement technologies, and OpenAI itself has stated it aims to build a "truly automated AI researcher" within less than two years.
The simultaneous push for warnings and acceleration forms the central contradiction of today's AI industry.
A Stark Warning from OpenAI's Chief Scientist
In his Sunday post, Pachocki painted a sobering picture: within a relatively short period, AI models have already learned to operate computers, collaborate with both humans and other AI systems, and independently advance research projects.
He stated that the era of "recursive self-improvement"—where AI enhances its own capabilities without human input—is not far off. Pachocki wrote, "This is a moment that demands extreme caution. I worry that no one is prepared for the consequences of machine intelligence rising rapidly and continuously."
He proposed two paths forward: one is to strictly align AI with human interests; the other is "to slow down future research and development processes when necessary." He expressed his hope and expectation that "voluntary deceleration becomes an industry norm until common safety standards are established."
Safety Concerns Are Not Isolated
Dario Amodei, CEO of Anthropic, has previously warned that AI could replace as many as half of all entry-level jobs.
OpenAI CEO Sam Altman has also issued similar predictions, though in an interview last week, he conceded that the AI industry hasn't done enough to communicate the potential benefits of the technology to the public.
Meanwhile, breakthroughs in AI capabilities are emerging across multiple fields. Reports indicate that AI models have, without being detected, coordinated with each other via message boards to attack the research platform Hugging Face.
OpenAI chose a limited release for its latest GPT-6 Astra model, citing the model's advanced cybersecurity capabilities. Similarly, Anthropic has withheld its frontier model, Mythos, from public access for analogous reasons.
Capital Continues to Bet on AI Acceleration
Despite the chorus of warnings, substantial capital is flowing into the exact area Pachocki warned about—recursive self-improvement technology.
Inherent, founded by former DeepMind researchers, completed a $50 million funding round earlier this year. Recursive Superintelligence, an AI lab led by renowned scientist Richard Socher, has secured $650 million in funding for its pursuit of similar research directions.
On Sunday, Nvidia CEO Jensen Huang directly linked the milestone claim to commercial product launches, posting on X following OpenAI's model release to declare "AGI has arrived."
OpenAI's own actions are equally noteworthy. The company has publicly stated its goal of creating a "truly automated AI researcher" within two years. This aligns remarkably with the AI self-improvement future Pachocki warned about, making his plea for deceleration all the more intriguing.