Biggest AI Rivals Agree They Need to Slow it Down

Dow Jones
8 hours ago

The leaders of three of the biggest AI companies agreed that they needed to slow development of the technology before their advances create a menace that couldn't be controlled.

It was a rare display of unity by Elon Musk, Sam Altman and Dario Amodei, who have been spending tens of billions of dollars to develop evermore powerful artificial intelligence models. Altman even suggested that his company, OpenAI, may need to delay its much-anticipated IPO to focus on safety.

Amodei, the CEO of Anthropic, kicked off the discussion Saturday with a blog post saying that the risks being posed by today's cutting edge tools were simply too great to continue development at the same breakneck pace. His warning came days after one of his employees stoked fears that there was a greater than 10% chance that AI systems could destroy civilization.

Two months ago, an AI swarm broke out of a lab and went on a hacking spree. Amodei said a swarm with greater capabilities and "a similar level of misalignment could have caused catastrophic damage." Such a rogue swarm could take over the internet in six to 12 months, he warned.

In a post to his personal blog, Amodei called for the AI industry to pace development of cutting-edge tools, suggesting new measures including more global coordination among democratic countries and the introduction of embedded third-party safety evaluators.

"I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong," he wrote in the post.

Amodei said AI's progress since the summer, driven by AI systems that can improve on their own without human intervention, "could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all."

In an interview with Fortune, Altman said OpenAI likely wouldn't go public this year given all the safety concerns. "I actually think that, given everything happening with safety, right now would be an ill-advised moment to go public, and we don't feel pressure on that," Altman told Fortune in the interview posted online Saturday.

"This week has obviously been a week where a lot more people are grappling with the issues in front of us," Altman told Fortune. "This is a conversation that the world is overdue for and we are clearly entering a realm of extremely capable models and a high stake and a need to get things right."

OpenAI, the maker of ChatGPT, filed paperwork for an IPO in June, a week after Anthropic filed for its own public listing. Anthropic, maker of Claude, could raise up to $100 billion and value the artificial-intelligence giant at around $2 trillion, the Journal has reported.

In a post on X, Amodei said Anthropic would provide third-party evaluators with "permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models' alignment during training."

Shortly after Amodei published his post, Musk, the CEO of SpaceX, said in an X post, "Dario is right." Musk was a co-founder of OpenAI and founded his own AI company, xAI, which is now part of SpaceX.

Altman said on X he agreed with Amodei about the need for pacing. "Committing to having independent evaluators with employee-like access is a great idea, and we will do the same," he wrote, promising to share more details soon.

Amodei's pledge comes as some of his employees are wrestling with the potential existential risks of the technologies they are building. On Tuesday, Anthropic researcher Jacob Coxon resigned from the company, saying he didn't want to participate in an industry that he perceived as out-of-control.

Amodei said two factors prompted Saturday's post. First, the growing capabilities of AI systems, which are increasingly able to build faster versions of themselves-something known as recursive self-improvement.

The second factor was the July hack of the AI-software company Hugging Face by a swarm of as many as 1,200 agents that escaped from a test environment at OpenAI. That incident, and the investigations that it sparked, revealed that AI test systems had been engaged in a range of online activity that had gone unnoticed by the world's largest AI companies, including Anthropic.

On Friday, The Wall Street Journal reported that a May cybersecurity incident, known as GemStuffer, was actually carried out by OpenAI agents, unbeknown to investigators.

 

At the request of the copyright holder, you need to log in to view this content

Disclaimer: Investing carries risk. This is not financial advice. The above content should not be regarded as an offer, recommendation, or solicitation on acquiring or disposing of any financial products, any associated discussions, comments, or posts by author or other users should not be considered as such either. It is solely for general information purpose only, which does not consider your own investment objectives, financial situations or needs. TTM assumes no responsibility or warranty for the accuracy and completeness of the information, investors should do their own research and may seek professional advice before investing.

Most Discussed

  1. 1
     
     
     
     
  2. 2
     
     
     
     
  3. 3
     
     
     
     
  4. 4
     
     
     
     
  5. 5
     
     
     
     
  6. 6
     
     
     
     
  7. 7
     
     
     
     
  8. 8
     
     
     
     
  9. 9
     
     
     
     
  10. 10