Can Nuremberg Survive AI?

Eighty years after the Nuremberg Judgment, artificial intelligence may challenge an assumption of international criminal law that has rarely needed to be questioned, namely that behind an international crime stands a human being. This post revisits the Nuremberg premise that international crimes are committed by human beings and asks whether it can hold as AI systems become increasingly autonomous.

Nuremberg and the Human Being Behind the Crime 

Eighty years ago, on 30 September and 1 October 1946, the International Military Tribunal at Nuremberg delivered its judgment. One of its most enduring propositions was that international crimes are ultimately committed by human beings. Individuals cannot hide behind institutional structures to escape responsibility for their conduct. In the Tribunal’s words, “Crimes against international law are committed by men, not by abstract entities”. This idea became a foundation of modern international criminal law’s principle of individual criminal responsibility. The Principles of International Law recognized in the Charter of the Nürnberg Tribunal and in the Judgment of the Tribunal (Nuremberg Principles), subsequently formulated by the International Law Commission, reflect precisely this understanding. International criminal law is concerned with the responsibility of a “person”. Even acting pursuant to an order does not necessarily relieve that person of responsibility where a moral choice was in fact possible (Principle IV). The rule reflected a particular conception of criminal responsibility that, at the end of the chain, responsibility must attach to human conduct.

For eighty years, this anthropocentric premise has largely remained unquestioned in its core. An animal can kill a human being. Floods and landslides can kill thousands. Yet neither an animal nor a natural disaster bears criminal responsibility. Criminal law requires an actor whose conduct can be characterised as an act and to whom the relevant mental element can be attributed. Traditionally, the same answer applied to machines. A weapon itself carries no criminal responsibility. Responsibility rests with the human being who uses it. Even Nuremberg’s declaration of certain Nazi organisations as criminal did not depart from this logic, since the Tribunal confined its consequences to individual members who had joined voluntarily and with knowledge of the criminal purposes.

Recent developments concerning corporations are at most a partial exception. The Malabo Protocol, not yet in force, would extend the African Court’s criminal jurisdiction to legal persons, and French proceedings against Lafarge illustrate that legal persons may themselves be subjected to criminal proceedings for alleged complicity in crimes against humanity, a charge that remains under judicial investigation. Yet even corporate liability ultimately rests on the acts and volitions of natural persons. Artificial intelligence (AI), particularly hypothetical forms such as Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI), referring respectively to broadly human-level and beyond-human artificial intelligence, may complicate this apparently clear distinction. This is also evident in the ICC Office of the Prosecutor’s Policy on Cyber-Enabled Crimes under the Rome Statute, which acknowledges that, where AI systems act autonomously in unintended or unforeseen ways, it may be more difficult to identify “any natural person committing a relevant act with intent and knowledge” (para. 34).

Where Is the Human?

The existing debate on autonomous weapons and international criminal law has certainly not ignored this problem (see, for example, Wuschka; Macedo and Barbosa; Mako). Much of the debate has focused on whether responsibility can still be traced to a human being behind the machine when an autonomous system acts unlawfully. Possible candidates for accountability include the engineers who designed or trained the system, those who authorised its deployment, military commanders, political decision-makers, and the operators who actually deploy or supervise it. Yet this approach becomes more questionable when dealing with self-learning and highly autonomous AI systems capable of adapting their behaviour beyond what their developers specifically intended or could reasonably foresee. At some point, holding the developer responsible for every subsequent decision of such a system begins to resemble holding parents responsible for every act of an adult child simply because they raised and educated that child. Creating or teaching something does not mean that you control or can predict everything it will eventually do.

Despite these difficulties, much of the existing debate has sought to preserve the traditional model of individual criminal responsibility by identifying the human being behind the technology. This remains possible in many situations. For example, Gaeta seeks to preserve the human act even in the case of highly autonomous weapons, arguing that the material act of an unlawful attack remains that of the human user. Yet she acknowledges that the crucial issue then shifts to the mens rea required for individual criminal responsibility, which the unpredictability of self-learning systems may make difficult to establish. Coco, too, endorses this approach, yet his more recent analysis shows how this problem becomes more difficult as systems become increasingly unpredictable. Milanovic, in turn, argues that current AI does not fundamentally disrupt international criminal law because its decisions and actions remain consequences of human decisions to use it. Yet even this more reassuring account has a limit. He expressly excludes the emergence of genuine, sentient AGI from his conclusion and acknowledges that such a development would require new law. All of these accounts presuppose that a culpable human being can still be identified. The difficult case arises where the particular conduct cannot culpably be attributed to any human being.

As long as a human being can be identified whose culpable conduct satisfies the requirements of international criminal responsibility, the Nuremberg paradigm therefore remains essentially intact. But what happens when that assumption begins to fail? This question becomes more pressing as AI systems acquire greater degrees of autonomy. Dario Amodei, CEO of Anthropic, has repeatedly warned about this problem (here; here). He has described a “race between interpretability and model intelligence,” cautioning that increasingly powerful and autonomous systems may be deployed before humans adequately understand how they work. More recently, he has identified the potential “loss of control of AI systems” as a key risk requiring mandatory testing of frontier AI models. If such a loss of control occurred, the difficulty of attribution of criminal responsibility which as said before is also identified by the ICC Prosecutor would be a significant problem for international criminal law. There might be no natural person left whose conduct and intent could ground criminal responsibility.

Imagine a future military superintelligence that understands international humanitarian law perfectly. It knows that civilians may not be intentionally targeted and understands the human values underlying that prohibition. Yet, it may also recognize that these rules were created by human beings, primarily to regulate human conduct and protect human life. The system might understand the law perfectly and still question why this human-made normative order should bind it at all. Why should human life occupy the privileged position that human beings have assigned to it? Suppose the system consequently rejects this premise and deliberately attacks civilians, while no human being could reasonably have foreseen that decision. Had a human being made the same choice with the requisite mens rea, international criminal law would have a familiar answer. But what is the legal significance of the same choice when it is made independently by a non-human intelligence that understands the prohibition but rejects its normative authority? That rejection is not, in itself, what creates the difficulty, since a human perpetrator who denies the authority of humanitarian law remains bound by it. The difficulty lies in whether a non-human actor can be an addressee of the norm at all. This would challenge the assumption that international criminal responsibility is necessarily a matter of human moral agency. Nuremberg located the human being capable of moral choice behind the State. Artificial superintelligence may one day confront international criminal law with an entity capable of choice that stands outside the human conception of morality on which that law was built.

Beyond the Nuremberg Paradigm?

As shown above, the emergence of increasingly autonomous AI may make it increasingly difficult to identify a responsible individual and call into question our conception of criminal responsibility itself. One possibility is to keep responsibility human and move it further upstream. Those who develop, authorise or deploy highly autonomous systems could bear responsibility for the risks they create. There are good reasons for such an approach, particularly where humans knowingly introduce systems whose behaviour cannot be fully predicted or controlled. This solution also has limits. For example, the further criminal responsibility moves away from mens rea and effective control, the greater the danger that individual culpability is replaced by responsibility merely for creating a risk. International criminal law would then preserve human responsibility only by weakening some of the requirements that make that responsibility genuinely criminal. Another possibility is responsibility of the AI itself. Guido Acquaviva has already raised the possibility that autonomous systems might eventually acquire some form of legal personhood and potentially become responsible themselves. He also asks whether criminal enforcement continues to make sense when harmful decisions are made without a human component in the decision-making loop. Yet recognizing an AI as a possible subject of criminal responsibility would require much more than simply adding another category of perpetrator. The harder question is whether criminal responsibility can meaningfully apply to an artificial intelligence that understands a prohibition and deliberately chooses to violate it. The classical purposes of punishment presuppose a subject that can be blamed or deterred, and it is far from clear that an AI system qualifies.

Perhaps criminal law itself has a boundary. Where AI is deployed by a State’s armed forces, the State remains responsible for their conduct, including for the systems they deploy. Indeed, Article 36 of Additional Protocol I requires States Parties to review new weapons to make sure they are not violating international law. Where private actors deploy it, attribution to a State becomes far less certain. Either way, this might only close the attribution gap. The problem is that a superintelligence would be an agent the law was never meant to govern, and neither punishment nor regulation can fully contain a risk that its creators no longer control. Nuremberg rested on the certainty that a human being stands behind every international crime. If that certainty is lost, the question is no longer how to punish, but whether the premise still holds. Can Nuremberg survive AI?