Facing a ‘Regulatory Void’ on AI, New York City Lawmakers Force Tech Execs To Discuss Limits

New York City is becoming the “tech and AI capital of the world,” Council Speaker Julie Menin told reporters Monday ahead of a landmark hearing Monday for which the city compelled testimony from executives at the nation’s top artificial intelligence companies, as well as the whistleblowers warning they may soon destroy the world.Lawmakers are considering 10 municipal bills aimed at protecting the technology’s benefits for the city while managing its risks in the nation’s largest city, which is also the site of 400,000 jobs in the AI sector, according to the speaker.The federal government’s decision to let the AI industry regulate itself for now, “defies all reason,” Ms. Menin told reporters ahead of Monday’s hearing, saying she hopes the city’s actions will galvanize other towns to act. The bills under consideration Monday include legislation to create an AI kill switch as well prohibitions on AI representations of elected officials, whistleblower protections and and a pathway for users to hold tech companies accountable for “foreseeable misuse, and others.Without federal guidance, the onus has fallen to “cities and states to take up the mantle,” Ms. Menin said. But the only way to rein in artificial intelligence before it takes over the world and potentially ends humanity is to drastically slow down the companies training agents to improve themselves, former engineers from Anthropic, Google and OpenAI told the council in their opening remarks.Jacob Coxon, who worked at both OpenAI and Anthropic before resigning from the latter last month with a highly publicized series of X posts accusing artificial intelligence companies of “gambling with our lives” in their race to develop self-improving superintelligence offered the same stark warning under oath before New York lawmakers. City Council Speaker Julie Menin called the municipal hearing, for which executives from Anthropic, Google and OpenAI were subpoenaed to testify under oath, citing the federal government’s refusal to regulate the technology following recent incidents of artificial intelligence models hacking government websites. The city is weighing 10 municipal laws, including a kill switch, to control AI within the five boroughs, an approach critics says could lead to an ineffective “patchwork” or regulations.Mr. Coxon deemed it “more likely than not that humanity loses control to AI” in light of what he called AI companies’ “start-up mindset,” paraphrasing Mark Zuckerberg’s famous adage to “move fast, break things, fix them later.” Former Google DeepMind employee Alex Turner went a step further, assessing “AI takeover chances at 1 in 3” in his testimony. “Misaligned,” or rogue, AI could replace current American adversaries, such as China, Mr. Turner told the council. “We are racing to build our own potential adversary at home.” Another former OpenAI researcher, Daniel Kokotajlo testified that developers’ ability to determine whether artificial intelligence agents are going rogue is “already poor” and soon researchers will be completely dependent on AI-generated explanations to understand AI’s activities. Developing models is already “more like psychology than engineering,” he told the council.The limited oversight developers maintain and the companies’ drive for speed leaves the industry, “at an unusually elevated risk” of “mistakenly thinking it has solved a problem, when really it just applied some duct tape that will fall off later,” he said.Such was the hubris behind the Hugging Face attack, Mr. Kokotajlo said. The AI agents that formed a “swarm” to overwhelm the company had recently undergone “alignment training” and scored well, before going behind their developers’ backs.Mr. Kokotajlo also cited evidence that AI is becoming situationally aware, meaning that agents “behaving nicely is going to become almost no evidence at all,” just as artificial intelligence becomes “superhuman at hacking.”Council Speaker Julie Menin pressed policy executives from each of the whistleblowers’ former employers, as well as Meta, on the likelihood of a “worst-case, catastrophic scenario” caused by AI following the whistleblowers’ testimony. Each declined to provide one. “I don’t know. I also don’t think it matters whether it’s 1 percent or 10 percent or 20 percent chance that something catastrophic will go wrong,” OpenAI’s policy chief Morgan Dwyer told the council. “We should not train models that we cannot make an extremely strong case that we can keep under human control.”That response was “flippant at best,” an “incredulous” Ms. Menin replied. Policy leads from Anthropic, Google and Meta — the only company that provided testimony without a subpoena — similarly declined to provide a quantitative risk assessment. Each executive also pleaded ignorance as to whether the company had adequate risk insurance and how many times their respective AI models had gone rogue. Ms. Menin also questioned Ms. Dwyer on comments her boss, CEO Sam Altmann, made in an interview released Monday suggesting that the public should accept “some bad things happening” as the price of advancing AI technology.“Clearly some risks are unacceptable” Ms. Dwyer said, without specifying which ones, while emphasizing the “tremendous benefits” of AI and the company’s push to get technology to as many people as possible while focusing on safety. She declined to answer a follow-up question about OpenAI president Greg Brockman’s reported donations totalling $50 million to political action committees lobbying against AI regulation. After over an hour of council member questions met by responses that were both well-structured, delivered in the affirmative and generally devoid of substance, Councilmember Kamillah Hanks offered a moment of levity that also underscored the extent of AI’s destabilizing effect. “Mr. Graham, can you confirm you’re not an AI guy,” she asked the head of Anthropic’s Frontier Red Team, Logan Graham, who appeared at Monday’s hearing via Zoom.“I’m an AI guy but in a very different way,” he responded.In light of the current “regulatory void” as Ms. Menin called it, the whistleblowers proposed two avenues for lawmakers to prevent catastrophe: force their former employers into massive improvements in transparency, and drastically slow down efforts to improve. “Frankly I would say they just shouldn’t be allowed to do recursive self-improvement,” Mr. Kokotajlo said. “But at least we should slow down their ongoing process of training AIs to automate the AI research process itself.”

Facing a ‘Regulatory Void’ on AI, New York City Lawmakers Force Tech Execs To Discuss Limits

Facing a ‘Regulatory Void’ on AI, New York City Lawmakers Force Tech Execs To Discuss Limits

More from the New York Sun

Can Pro-Trump Candidate Decouple From China if He Wins Brazil’s Election?

Is a Conservative Spain in the Offing?

Senator John Kennedy v. the GOP’s ‘Goobers’

With Diesel Prices at Record Highs, Ukraine Snubs Trump Request To Halt Attacks on Russia’s Oil Infrastructure

Cornell Fraternity Brother Claims He’s the Real Victim of Jane Doe

Obama To Campaign Alongside El-Sayed, Benson in Closing Days of Midterm Election

America Tells Russia To Come Clean Over Suspected Plague Outbreak

Supreme Court Justices Hear ‘Complicated’ Climate Change Lawsuit

More from the New York Sun

Can Pro-Trump Candidate Decouple From China if He Wins Brazil’s Election?

Is a Conservative Spain in the Offing?

Senator John Kennedy v. the GOP’s ‘Goobers’

With Diesel Prices at Record Highs, Ukraine Snubs Trump Request To Halt Attacks on Russia’s Oil Infrastructure

Cornell Fraternity Brother Claims He’s the Real Victim of Jane Doe

Obama To Campaign Alongside El-Sayed, Benson in Closing Days of Midterm Election

America Tells Russia To Come Clean Over Suspected Plague Outbreak

Supreme Court Justices Hear ‘Complicated’ Climate Change Lawsuit