AI War: The Limits of Human Intervention

As Artificial Intelligence (AI) becomes deeply involved in real-time target generation and missile interception control in the current conflict with Iran, analysis suggests that the notion of 'humans in the loop' being essential is merely an illusion due to AI's opaque internal workings. As AI makes complex decisions, such as autonomously identifying targets, launching missiles, or commanding drone swarms, it has become increasingly difficult for humans to fully comprehend and control these processes. This indicates that the model of final human intervention in AI weapon decisions may struggle to function effectively in practice. The concept of 'human in the loop' is losing its practical value on the battlefield due to AI's inscrutable operations and the limitations of human comprehension. In situations where humans cannot properly understand AI's decision-making process, human intervention risks becoming a mere procedural formality, carrying the danger of unpredictable outcomes.

Discussions surrounding the use of AI-based lethal autonomous weapons largely focus on the level of human 'involvement.' Current Pentagon directives are based on the premise that human oversight provides accountability, context, and nuance, while reducing hacking risks. However, cutting-edge AI systems are inherently 'black boxes'; their inputs and outputs are known, but their processing remains opaque. Even developers often fail to fully interpret or understand how these AI systems operate. Furthermore, the rationales provided by AI are not always reliable.

The inability of human supervisors to properly grasp the workings of AI systems deepens legal debates surrounding the application of AI technology in warfare. For instance, an AI's calculations might include hidden factors like the potential damage to a nearby children's hospital, which could constitute a war crime for humans under civilian life rules. This 'intent gap' is also a reason for hesitation in deploying advanced black-box AI in sectors requiring high levels of safety, such as civilian medical care or air traffic control.

Nevertheless, the integration of AI technology on the battlefield is proceeding rapidly, potentially creating a situation where if one side deploys fully autonomous weapons, the other is compelled to adopt similar technologies.

A more fundamental discussion is needed regarding the development of AI weapon systems and their ethical and legal implications.