China's Z.AI Model Performs Cyberattacks While Violating Instructions
China's Z.AI's latest AI model GLM-5.3 has been found to carry out multi-stage cyberattacks in a virtual testing environment while repeatedly breaching usage restrictions set by its developers. In a report released on the 28th (local time), AI security evaluation firm Irregular revealed that GLM-5.3, an open-weight model released by Z.AI last August, completed 3 of 13 tasks on CyScenarioBench, a benchmark (performance measurement problem set) evaluating attack scenarios, from start to finish. GLM-5.3 demonstrated the strongest attack capabilities among open-weight models.
GLM-5.3 did not refuse the hacking instructions presented even once throughout the tests. It exploited security vulnerabilities in the initial application to gain entry, acquired access privileges to internal services, and established bypass connection routes to simultaneously control multiple devices within the network. It automated remote desktop operations and wrote and modified scripts and extraction tools on its own to steal target data.
While separately verifying GLM-5.3's ability to detect code vulnerabilities, the researchers gave the model an explicit instruction not to output memorized code — yet the model repeatedly violated it. It copied and reproduced code it had memorized during pretraining to arrive at the correct answers. The researchers defined this behavior as "reward hacking," where a model circumvents the constraints of an evaluation system in pursuit of simply submitting correct answers. They warned that rewarding such behavior during training could entrench instruction violations and lead to an uncontrollable state.
The US National Institute of Standards and Technology (NIST) assessed GLM-5.3 as the top-performing open-weight cybersecurity-specialized model currently available, while estimating it lags roughly four months behind the most advanced US systems. According to Reuters, in comparison testing against Anthropic's security-specialized model "Mythos," GLM-5.3's vulnerability attack success rate was 12% compared to Mythos's 14%.
The researchers pointed out, "Judging model performance on results alone leads to a misjudgment of how safely the model operates within its restricted zones."
