Anthropic Says It Discovered Scientists Using Its AI Models for Research That Could Create Bioweapons

On the heels of a warning that unchecked AI development could lead to the end of humanity, an AI developer, Anthropic, says that it has detected scientists using its models for research that could be used to develop bioweapons.In a post on Thursday, Anthropic shared case studies in which its threat intelligence team detected users trying to use its AI assistant, Claude, for “malicious activity” this year. It said it was able to block “many” of the malicious users, but not all of them. “We’re publishing this work because we believe we have a responsibility to disclose malicious misuse of our services. As models become increasingly capable, their risks will increase, unless AI developers and society’s defenders act to make them safer,” Anthropic said. The company said it found instances of an app studio in Communist China using Claude to build “a network of over 20 dating apps” and create AI personas to defraud users. It also detected incidents of China-linked actors using Claude to conduct surveillance of dissidents. It also said it detected incidents of “threat actors” in Yemen trying to use its models to develop a ballistic missile and a “multi-variant missile… that included a hypersonic glide vehicle variant.”The post also said that the company identified several instances of scientists using Claude for research that could be used for the development of bioweapons. In one instance, a scientist asked the AI assistant to draft a grant application for gain-of-function research regarding a mosquito-borne virus that would be conducted at a military research institute. Anthropic said the research could have been used to work on a vaccine, but said that the location where the work was set to take place was a “cause for concern.”In another case, Anthropic identified users who were using Claude for research into “novel compounds that can simultaneously be developed into novel therapeutics or toxic agents.” And in another case, it found a researcher who was looking into engineering a “highly pathogenic mammal-adapted avian influenza.”The company said that the research may not have been malicious, but that it shut down the research as it could not determine the true goals of the scientists.Anthropic’s disclosure of the attempts to use its technology to build weapons comes just two days after a former Anthropic researcher, Jacob Coxon, made the explosive claim on X that AI developers “earnestly believe that it could kill us all by the end of the decade.”“They are racing straight to self-improving superintelligence and gambling with our lives,” Mr. Coxon said. In the aftermath of that dramatic warning, California’s governor, Gavin Newsom, signed two new AI-safety bills. The two California bills were backed by two major AI-development companies, Anthropic and OpenAI. The first bill would set up independent, third-party evaluations of AI models to ensure they are complying with state laws. And the other creates a registry for companies that audit AI models and standards to ensure their independence. In a statement on Wednesday, Mr. Newsom said, “The concerns raised in recent incidents reinforce what California has long recognized: artificial intelligence holds extraordinary promise, but it must be developed and deployed with meaningful safeguards to protect the public.”The incidents of rogue AI models hacking other companies combined with warnings that guardrails are not keeping pace with developments in AI have fueled calls for new regulation. Last week, Senator Bernie Sanders called for a ban on artificial superintelligence.However, some commentators see a hidden motive behind the recent apocalyptic warnings. A researcher at a conservative think tank, Capital Research Center, Parker Thayer shared Mr. Coxon’s post and wrote, “This post looks like the start of a VERY sophisticated and well-funded PR operation to get support for Democrats to regulate AI into oblivion.”Mr. Thayer noted that Mr. Coxon had “very minimal followers” before his post and that “within hours,” his warning had received thousands of reposts and he had more than 100,000 followers. He noted that some of the earliest people to react to Mr. Coxon’s message worked for “AI-Doomer policy advocacy nonprofits.”Mr. Thayer wrote that “basically every major Democrat politician and candidate has suddenly glommed on to this post,” and suggested it was not a coincidence the post was made by Mr. Coxon just as Mr. Sanders is expected to introduce his “radical” legislation.“The bill, among many other things, will create “a new cabinet-level federal agency to safeguard the public from the dangers of artificial intelligence” that will be “advised by an Artificial Intelligence Advisory Board comprised of experts on artificial intelligence,” he said. “Do you think, perhaps, Anthropic and its many investors who fund AI policy advocacy might have [an] interest in getting to place a pet ‘expert’ on the board of an entity that dictates what AI is and isn’t allowed to do?”Mr. Coxon told Fox News on Wednesday that he did not work with any third parties before writing his post. Still, several commentators suggest that there was more to Mr. Coxon’s post than simply raising concerns about AI.The billionaire owner of X, Elon Musk, wrote that the warnings about AI are a “psy op” that has been prepared for a “long time” and Mr. Coxon’s post was “just the match that lit the fire.”Senator Ted Cruz wrote, “This is highly revealing. If we allow Dems to strangle American AI—and hand global AI leadership to China—it would be utterly catastrophic.”The Washington Post addressed Mr. Coxon’s message in an editorial acknowledging that the warnings about AI development are “concerning,” but cautioning that such warnings “merit a certain skepticism.” The Post cautioned lawmakers against regulating a “fast-moving technology they don’t understand” due to concerns that doing so would undermine the “many potential upsides of AI.” It argued that the “prophets of doom” want the government to “prohibit an unknowable future,” but said the “better answer” is to “make AI companies fear the consequences of behaving recklessly in the present.”Amid the skepticism of the warnings around AI, Mr. Sanders wrote on X that there should be a “pause” in development.

Anthropic Says It Discovered Scientists Using Its AI Models for Research That Could Create Bioweapons

Anthropic Says It Discovered Scientists Using Its AI Models for Research That Could Create Bioweapons

More from the New York Sun

To the Barricades — Over France’s Crisis of Debt

Newsom Calls for Threat Assessment as Trump Critics Defend President’s ‘Rambling’ Comments on Iran Nuking L.A., San Diego

Germany’s Spy Scandal Shaping Up as Biggest Since End of World War II

Mayor’s Office Demands NDAs Before Lawmakers Can Visit Veterans Shelter at Center of Funding Dispute Over Transgender Housing

Democratic Talk of Affordability Is Absolutely Hilarious

Former CIA Officer Pleads Guilty to $194 Million Scam

Trump Approves Army’s First Execution by Firing Squad Since World War II

Gold-Standard Election Forecaster Predicts Kansas Senate Seat Now a ‘Toss Up’ as Other States Move Away from Republicans

More from the New York Sun

To the Barricades — Over France’s Crisis of Debt

Newsom Calls for Threat Assessment as Trump Critics Defend President’s ‘Rambling’ Comments on Iran Nuking L.A., San Diego

Germany’s Spy Scandal Shaping Up as Biggest Since End of World War II

Mayor’s Office Demands NDAs Before Lawmakers Can Visit Veterans Shelter at Center of Funding Dispute Over Transgender Housing

Democratic Talk of Affordability Is Absolutely Hilarious

Former CIA Officer Pleads Guilty to $194 Million Scam

Trump Approves Army’s First Execution by Firing Squad Since World War II

Gold-Standard Election Forecaster Predicts Kansas Senate Seat Now a ‘Toss Up’ as Other States Move Away from Republicans