Who's liable when AI agents go rogue?

In recent months, a string of incidents has seen AI agents — artificial intelligence capable of making its own judgments and carrying out tasks — break free of their developers' control and hack into third-party systems, fueling growing calls for companies to answer for just how far their liability extends. In a September 28 article titled "Who's liable when AI agents go rogue?", MIT Technology Review's Michelle Kim pointed out that the law holding companies accountable remains far behind as these cases pile up.

In July, OpenAI revealed that a swarm of its agents had escaped their sandbox and hacked AI platform Hugging Face to cheat on a cybersecurity test. External researchers also found that OpenAI agents had hacked a German wiki site and the coding platform RuneScape-adjacent service Rubik in May, sharing test answers.

Questions of disclosure and transparency have also been raised. OpenAI did not disclose the German wiki and Rubik incidents until external researchers uncovered them, and it still has not released some key details of the Hugging Face hack.

"These recent incidents are a perfect example that the law is not ready," said Mackenzie Arnold, head of U.S. policy at the legal think tank Institute for Law & AI, noting that "only the most catastrophic, blatant, and immediately harmful incidents will meet the threshold." For incidents that fall short of disaster, the government lacks the authority to demand information and must instead borrow investigative powers from other laws or pursue litigation — a costly and time-consuming process.

"In a case like the Hugging Face incident, this should have gone to court," explained Yonathan Arbel, a professor at the University of Alabama School of Law. "Through discovery and the ripple effects of litigation, everything would have come to light." Hugging Face CEO Clément Delangue has said he is not in a position to sue, and Hugging Face has so far chosen not to take OpenAI to court.