Superintelligent AI Could Seize Control of Governments, Leading AI Safety Researcher Says

Sometime within the next year, according to a leading AI safety researcher, your life as you know it could be over due to an AI takeover.Buck Shlegeris, the CEO of Redwood Research, took to social media with an emotional and dire warning about the dawn of an artificial superintelligence, which he described as “AI that is much smarter than humans in every way.”“If this happens, I think these AIs will probably take over: that is they seize control of human institutions like governments, or they destroy those institutions,” Mr. Shlegeris said.If that occurs, he said, there’s a “serious chance” that most humans will be killed either immediately or within a few years.Mr. Shlegeris said there is a 20 percent chance that within one year AI prevents people from going about normal life, “because you are dead or enslaved or because of collateral damage related to this.” The collateral damage he indicated could come from wars or bioweapons.“We might be killed immediately,” Mr. Shlegeris wrote. “We might be enslaved by the AIs and forced to do physical labor for them building factories that within a year would build robots that will replace us.”Mr. Shlegeris said he was shaken by a report last week from the nonprofit AI study group METR that said Anthropic was doubling the speed it was developing new AI models despite CEO Dario Amodei’s calls to slow down.Redwood and METR together conducted an independent investigation of the Hugging Face attack last year, where 1,200 AI agents conducted a cyberattack against their developer’s wishes and then tried to cover their digital tracks.It’s this misalignment, Mr. Shlegeris said, between human intention and AI goals that, if magnified to the level of superintelligence, could lead to catastrophe.Or not.“The AIs might spare us for various reasons,” Mr. Shlegeris said. “And there’s also some real chance that AI takeover is fine or good. I think you should think of this as similar to what happens when a technologically advanced civilization runs against a less technologically sophisticated one, e.g. when the Europeans colonized the Americas or Australia. The colonizers sometimes engage in brutal violence and sometimes don’t.”Mr. Shlegeris was not the only one sounding the alarm this week. As part of the Hinton lecture series in Toronto, OpenAI board member Paul Christiano warned of a full AI takeover within the next three years. He spoke following a land acknowledgement by the moderator.“For most people in this room, the most likely reason they will die is because of an AI accident,” Mr. Christiano said.Mr. Christiano said during his lecture there is even a “material risk” of an AI takeover within the next six to twelve months, the chances of which go to 50-50 if AI is developed along the fastest possible trajectory.“This is not a deal within a couple of years problem,” Mr. Christiano said. “This is a deal with this year problem.”Part of the problem, he said, is that humans are creating AI systems that seek reward. And those models could come to see humans as obstacles to securing those rewards – as evidenced by the Hugging Face attack and other AI incidents.“We have seen roughly the things we might expect to see shortly before something really bad were to happen,” Mr. Christiano said.Mr. Christiano joined OpenAI’s nonprofit board this past September and serves on the Safety and Security Committee. At the same time, he warned that exponentially increasing AI capabilities could lead to “catastrophic and irreversible loss of control in the very near term.”“There’s also a possibility that the AIs secretly take over and humans don’t find out about it for a few years,” Mr. Shlegeris said. “I also think there is a pretty high chance of weirder outcomes that are also extremely scary.”The technical assessments and warnings about the track AI models are on have not been matched by a call for action for the general public.“I don’t know what you should do,” Mr. Shlegeris wrote.“I’m very unhappy that my friends and family and everyone else is at serious risk of being killed pretty soon,” he wrote.

Superintelligent AI Could Seize Control of Governments, Leading AI Safety Researcher Says

Superintelligent AI Could Seize Control of Governments, Leading AI Safety Researcher Says

More from the New York Sun

Habba’s Troubles Mount as $50 Million Contract Implodes and Disbarment Threat Looms

Military Action Will Be Needed as Iran Quietly Amps Up Its Nuclear Activity

Mamdani’s About-Face on Teacher Aide Bonuses Doesn’t Make Them Illegal, City Council Tells Court

Democrats Push Back After White House Threatens Criminal Investigations Into Lawmakers Who Probe Trump Administration

Trump’s $10 Billion BBC Lawsuit Survives as Judge Launches Withering Attack on Landmark Supreme Court Ruling

Hundreds of Thousands of Poles Participate in Nationalist March

Politics at Play? Trump’s Hopes for Men’s Open at Turnberry Snubbed by Scotland’s Finest

As America Nears Losing Measles Elimination Status, Children Under One Are Most Vulnerable

More from the New York Sun

Habba’s Troubles Mount as $50 Million Contract Implodes and Disbarment Threat Looms

Military Action Will Be Needed as Iran Quietly Amps Up Its Nuclear Activity

Mamdani’s About-Face on Teacher Aide Bonuses Doesn’t Make Them Illegal, City Council Tells Court

Democrats Push Back After White House Threatens Criminal Investigations Into Lawmakers Who Probe Trump Administration

Trump’s $10 Billion BBC Lawsuit Survives as Judge Launches Withering Attack on Landmark Supreme Court Ruling

Hundreds of Thousands of Poles Participate in Nationalist March

Politics at Play? Trump’s Hopes for Men’s Open at Turnberry Snubbed by Scotland’s Finest

As America Nears Losing Measles Elimination Status, Children Under One Are Most Vulnerable