Former Anthropic Researcher: 'We Don't Know How to Control AI'

"We don't know how to control AI." Jacob Coxon, a former researcher at Anthropic (a U.S. AI development company), warned at a New York City Council AI hearing on the 5th (local time) that "at the current rate, humanity is more likely to lose control of AI than not, and in the worst case, this could lead to human extinction." It was the first time the New York City Council formally addressed the risks of AI.

Coxon spent three years working on pre-training research at OpenAI and Anthropic. Last month on the 8th, he announced his departure on X, criticizing that "no company is acting responsibly."

The hearing was held at the request of New York City Council Speaker Julie Menin, who is pushing forward with a package of AI regulation bills consisting of 10 ordinances. The centerpiece is a mandate to install a "kill switch" that can immediately shut down systems in an emergency. The package also includes mandatory third-party safety audits before AI systems can be sold or deployed in New York City, a whistleblower reward program, and fines of up to $25,000 (about 34 million won) per violation.

On the kill switch, Coxon noted that "it could help in the short term, but it is not enough to guarantee safety in the long term."

Representatives from big tech companies including OpenAI, Anthropic, and Google also testified via video at the hearing. The City Council said it would ask a court to enforce a subpoena against SpaceX's AI division, which did not respond to the summons.

With the Donald Trump administration maintaining a largely hands-off approach to AI regulation, state and local governments such as New York are pursuing their own regulations, widening the gap between federal and local regulatory directions.