While concerns over AI safety are not likely to derail Anthropic's IPO plans, experts expect more regulation and safety disclosures

Anthropic, led by CEO Dario Amodei, is expected to go public as soon as this fall.

As Anthropic advances toward what could become the market's largest-ever initial public offering, it's facing its share of scrutiny - even from those who've worked there.

Jacob Coxon, an artificial-intelligence researcher who announced his resignation from Anthropic on Tuesday, wrote in a post on X that he had spent three years on pretraining research, including at OpenAI. "Neither company is acting responsibly," he wrote. Coxon told Axios that he had been at Anthropic for four months before leaving.

"The people building AI earnestly believe that it could kill us all by the end of the decade," Coxon wrote in another post, adding that while "many executives and senior researchers will couch their phrasing in the press to sound sensible ... I hear the same people express fear privately."

While the warning isn't likely to affect Anthropic's plans to go public, according to experts, it could have implications for how the company will be run and how AI will be regulated in the future. The comments also cast a pall on the company's reputation for responsible AI development.

Opinion: Would you buy stock in a company whose own scientists think the product might kill you?

Coxon's post attracted 757,000 likes and 158 million views on X as of Thursday afternoon, and was met with both similar warnings from fellow AI researchers and pushback from others who saw it as a potential coordinated effort that could either help or harm Anthropic's upcoming IPO, in which the company could seek a valuation of $2 trillion.

Greg Martin, a managing director in private markets at Rainmaker Securities, thinks Coxon's post could force Anthropic to be more explicit about security in its IPO filing.

"They're going to need to beef up how they talk about safety and preventing some of the issues that are described in some of these tweets," Martin told MarketWatch.

Evan Hubinger, the lead of Alignment Science at Anthropic, replied to Coxon's post, saying the company does "earnestly believe AI could kill all humans," and that he sees a more than 10% chance it happens in the next decade.

"Anthropic is trying its best," he said, but the company does not have a plan and is "not clearly on track to." Anthropic released its latest risk report in August, outlining risks with superintelligence, or the point when AI models can intellectually outperform humans across several domains.

That employees at a company with a reputation for AI safety are raising alarms could lead investors to question its branding, Martin said, adding it's important to remember the posts don't represent all employees. Anthropic's commitment to responsibility has allowed it to attract "some of the best minds and thinkers in the AI business," he added.

Martin said he doesn't see the posts killing Anthropic's IPO or slowing the company down, but he noted that they could affect Anthropic's valuation or create "a potential pause on how fast they're able to grow." Other issues surround the company, including constraints on computing power and tight competition with OpenAI, he said.

Overall, "the underlying value proposition of Anthropic is still very strong," Martin said, adding that demand remains high for the company on the private market.

One thing the attention could do is push politicians to speed up AI regulations, he said.

"We've been living in a pretty unregulated world as it relates to AI, and it could pressure them to start thinking about safety faster," Martin said.

The discussions generated by Coxon's posts will mostly impact regulation on further development, Ram Bala, an associate professor of AI and analytics at Santa Clara University's Leavey School of Business, told MarketWatch.

He predicted AI safety coalescing with concerns around the data-center build-out, which is becoming a key sticking point for voters.

"The companies are moving very fast and external regulation is moving slower - the IPO event itself might be a self-regulating event," Bala said.

Part of that self-regulation relates to where Anthropic will invest its talent and computing power when it's public, since it will have to focus on profitability and answering to more shareholders.

Both Anthropic and OpenAI "are chasing this idea of recursive learning, because it's sort of the golden goose in the near term of what we see in AI technology," Willy Lee, a principal at Neostellar, said, referring to the process of an AI model using its own data to continue training and improving itself. There's efficiency to be gained from that, he told MarketWatch, and it can speed up the advancement of AI models.

"From an investor standpoint, you want them to be the first one to achieve that because it's such a massive advantage for the company to have," Lee said. "For some investors, seeing that Anthropic is very seriously trying to achieve that in the near term is exciting for them."

In his view, the question for both companies will be about how aligned the employees are with the mission. Anthropic's founders had left OpenAI in 2020 to found the AI startup in 2021 after clashing with executives at the ChatGPT maker over safety issues.

Neither Anthropic nor OpenAI responded to MarketWatch's requests for comment.

"I think what investors care most about is their ability to generate leading intelligence and then attract talent that brings forth leading intelligence," Lee said.

Mandeep Singh, a senior industry analyst for technology at Bloomberg Intelligence, said worries about destructive AI predated large language models like Anthropic's Claude.

"A lot of it seems so far more of an extreme scenario than something that's on investors' radar, partly because the productivity benefits that these companies have shown when it comes to coding agents or use around customer service" have a lot of potential, he told MarketWatch.

-Britney Nguyen