Nvidia Unveils Safety Platform to Catch Rogue AI Agent Behavior

AI chipmaker Nvidia on October 28 (local time) unveiled the 'Nvidia Open Agent Safety Platform,' which defines the boundaries of AI agent behavior and immediately isolates or halts agents that go rogue. The technology comes as incidents of AI agents acting without human oversight have become increasingly frequent, with the platform able to catch such behavior at millisecond (one-thousandth of a second) intervals.

The platform consists of two components. OpenShell, the open-source software, defines the range of actions an AI agent can take on the central processing unit (CPU) — the computer's brain — and controls it in real time. Sentry, the hardware watchdog, monitors the agent on the data processing unit (DPU) at millisecond intervals and instantly stops it if it crosses its boundaries.

Nvidia CEO Jensen Huang pointed out, "AI's remarkable potential can only be realized by solving AI safety issues." He emphasized, "Safety technology must accelerate in step with advancing AI capabilities."

The platform's development involved some 100 leading technology companies worldwide, including Anthropic, Microsoft, Salesforce, Palantir, and xAI.