Anthropic Announces Fund for Public Good, Lobbies To Limit Guardrails Around Potentially Lethal AI Models

Anthropic’s attempt to burnish its credentials as a benevolent AI company through a fund “dedicated expressly to the public good” is running smack in the face of its efforts to see Congress pass legislation permitting the release of AI models if developers can reasonably assure that no more than 50 people may be killed by them. The limitation would apply to any model creating a biological weapon, evading the controls of its developer, or conducting a cyberattack. The bill, first reported by Politico, would allow the government to halt the release of the model if it could foreseeably kill more than 50 people or create a risk of more than $1 billion in damages. Models that pose a “foreseeable, imminent risk” of killing 50 people or fewer would be allowed to proceed.“In the opinion of Anthropic executives, it would be worth killing exactly 50 humans to roll out the next AI model, but anything more than 50 would be a tad excessive,” essayist and leftwing independent Australian journalist, Caitlin Johnstone, writes in her newsletter.“Anthropic places a monetary value on 50 human lives equivalent to one billion US dollars, or twenty million dollars per individual,” Ms. Johnstone adds. “Under Anthropic’s valuing system, the lives of your children are each worth less than a McLaren F1 sports car.”The legislation is the latest example of efforts by Anthropic representatives to place guardrails on AI models that could be developed for military or government purposes.The draft is similar to language enshrined in the first AI safety law in the nation, California’s SB 53, which Governor Gavin Newsom signed in September. It too holds the limit for collateral damage to more than 50 dead.“This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” an Anthropic spokesperson told Politico. “These are sensible policies we believe Congress should act on.”A former Oklahoma Democratic Congressman, Brad Carson, who works with the AI policy advocacy group Americans for Responsible Innovation, said “parts of the draft raise real concerns,” but the proposal contains several strong points.“It writes real obligations into law rather than leaving them to guidance,” he wrote on Monday. “As Congress debates AI policy, it’s worth recognizing these ideas.”Amid the lobbying, Anthropic has announced that it is going to create a public fund for good. The fund, subject to shareholder approval and initially funded with five percent of Anthropic shares, would be set up as a limited liability corporation, rather than a nonprofit. Anthropic would add an additional two percent for every 50 percent that the company’s value increases. “It is common practice for tech companies to offer escalating compensation awards to their executives,” Anthropic said in a statement. “We are instead directing that equity to the public benefit.”The announcement did not state what types of nonprofits or programs it would fund. It promises the equity in the plan will not be used for political contributions or lobbying.The Anthropic Public Benefit Fund “might explore large donations to traditional nonprofits or direct distributions to specific populations,” including tech and financial sector workers displaced from their jobs by artificial intelligence models. It’s unclear when the first payments from the public benefit fund would be made.“You’ve put 20% of your equity in an LLC,” the co-chairman of the President’s Council of Advisers on Science & Technology, David Sacks, wrote on social media of Anthropic’s proposal. “Whether it serves the public good remains to be seen.”General counsel for the AI regulatory advocacy group Encode, Nathan Calvin, warned that the draft legislation usurps the role of the states and makes it more difficult to get a court order blocking the development of a dangerous AI model. Mr. Calvin, who warned on Monday that Anthropic “is primarily trying to win the race to super intelligence” and puts other considerations, including safety, second, described himself as “viscerally angry” that the bill “would make us less protected against catastrophic risks than we are under existing law.”“The federal government’s powers to restrict the development, deployment, or internal use of a frontier model has to meet an absurdly high bar,” Mr. Calvin said, warning that the legislation puts the burden of proof on the government “to show why no alternative would prevent the imminent catastrophe,” and that a court is required to consider harms to the developer when making its decision to block a deployment.Concerns over AI’s independence were recently raised by a historian and board member of AI safety organization LawZero, Yuval Noah Harari, who predicts that with AI advancements, humans will become the new cows in the evolutionary pecking order. Humans are going to be the “cows of the future,” the bestselling author predicted, because they will not be born with the innate knowledge that AI has. As a result, AI will make decisions about human conduct, rather than humans making decisions for themselves.“Just as cows can’t understand the financial system that controls their lives today, we will not be able to understand the AI financial system that will control our lives in 10 or 20 years.” Mr. Harari said, adding to the analogy that “cows know humans exist, but cows don’t know money exists. Like I can sell you a cow and you can transfer money to my bank account: this is invisible to the cow.”Mr. Harari said that unlike humans, who have only been operating in a bureaucratic environment for about 10,000 years, AI was built on and is native to a task-oriented and organizational command system. “We were born in the forest, not in the office, so it’s alien to us,” he said.Congress is out of session and will not take up any legislation, AI or otherwise, until the lame-duck session after the midterms.

Anthropic Announces Fund for Public Good, Lobbies To Limit Guardrails Around Potentially Lethal AI Models

Anthropic Announces Fund for Public Good, Lobbies To Limit Guardrails Around Potentially Lethal AI Models

More from the New York Sun

Hakeem Jeffries Is on the Spot

Koreans and Americans Differ on the Crucial Question of Who Commands South Korean Forces If War Breaks Out

Prince Harry Scrambles To Walk Back ‘Swipe’ at His Family — as Prince William Carries On With Royal Duties

Estranged Mother of Democrat Running for Ohio Governor Labels Her a ‘Liar’ With a ‘Serious Problem’

In ‘The Ask,’ Liberals Clash Across Generational Lines

Supreme Court To Decide Legality of Civil War-Era Ban on Home Distilleries

JD Vance Sits Out Israel’s Election

As Europeans ‘Gorge’ on Putin’s Petroleum, Trump Is Pilloried for Allowing a Trickle of Russian Diesel to Flow

More from the New York Sun

Hakeem Jeffries Is on the Spot

Koreans and Americans Differ on the Crucial Question of Who Commands South Korean Forces If War Breaks Out

Prince Harry Scrambles To Walk Back ‘Swipe’ at His Family — as Prince William Carries On With Royal Duties

Estranged Mother of Democrat Running for Ohio Governor Labels Her a ‘Liar’ With a ‘Serious Problem’

In ‘The Ask,’ Liberals Clash Across Generational Lines

Supreme Court To Decide Legality of Civil War-Era Ban on Home Distilleries

JD Vance Sits Out Israel’s Election

As Europeans ‘Gorge’ on Putin’s Petroleum, Trump Is Pilloried for Allowing a Trickle of Russian Diesel to Flow