{"id":30105,"date":"2026-10-28T09:00:50","date_gmt":"2026-10-28T08:00:50","guid":{"rendered":"https:\/\/voelkerrechtsblog.org\/?p=30105"},"modified":"2026-10-29T19:07:41","modified_gmt":"2026-10-29T18:07:41","slug":"governing-ai-agents","status":"publish","type":"post","link":"https:\/\/voelkerrechtsblog.org\/de\/governing-ai-agents\/","title":{"rendered":"Governing AI Agents"},"content":{"rendered":"

Advancing<\/u><\/a> cyber capabilities and growing<\/u><\/a> AI autonomy increase the risk of unsanctioned agent behaviour. Despite emerging evidence of these risks, global AI regulation remains difficult. Climate change and nuclear risks offer useful points of comparison (as both involve transnational risks requiring international cooperation). So, what can we learn from them, and where do the analogies break down?<\/p>\n

Frontier AI<\/u><\/a> models (the most advanced general-purpose models at a given moment) now perform<\/u><\/a> at or above human levels on advanced science, mathematics<\/u><\/a>, and multimodal reasoning tasks. But these abilities come with greater security and safety risks. The Hugging Face<\/u><\/a> intrusion is proof that agents can discover unintended channels to communicate and coordinate, exploit system weaknesses, and bypass constraints in pursuit of their objectives. Other similar instances of AI agents deploying<\/u><\/a> malicious code and manipulating people have been identified by the AI\u00a0Security Institute. Moreover, evaluation<\/u><\/a> awareness can make AI safety tests less reliable because models may hide risky capabilities during testing. These raise genuine concerns about the potential of autonomous AI systems to exploit vulnerabilities in, or disrupt, critical infrastructure. Yet, approaches to AI governance remain fragmented<\/u><\/a>. States<\/u><\/a> (and industry<\/u><\/a> actors) differ over whether and how to regulate. So, why is collective action so difficult despite shared risks?<\/p>\n

Economic and Geopolitical Incentives <\/strong><\/p>\n

Frontier AI labs (e.g., Anthropic, OpenAI, xAI and DeepSeek) are key private players in the AI race. To develop<\/u><\/a> powerful models, they require substantial investment, creating pressure to remain competitive and profitable. While slowing development may improve safety, unilateral<\/u><\/a> restraint creates a competitive disadvantage. Therefore, labs have little incentive to unilaterally slow down. Regulation could address this collective action problem, but states themselves face a similar dilemma.<\/p>\n

The United States<\/u><\/a> and China<\/u><\/a> are the key players, driven by geopolitical primacy. If one slows down for safety, it risks falling<\/u><\/a> behind the other. In the absence of trust, this fear pushes both sides to continue the AI race (even when mutual restraint would benefit<\/u><\/a> both). This dilemma is evident with fragmented AI regulation across major jurisdictions. The US has no comprehensive federal AI statute. The federal government has also supported private actors in challenging state regulation. As a result, Colorado has now repealed<\/u><\/a> and replaced its AI law with a narrower regime.<\/p>\n

The EU is not a frontier AI power (compared to the US or China). However, its large market, strategic technology companies, and risk-based regulatory approach give it influence over AI governance. Yet, despite adopting the first comprehensive AI regulatory framework, it has faced industry pressure<\/u><\/a>, alongside delays<\/u><\/a> to key provisions governing high-risk AI systems. In a similar vein, the UK hosts<\/u><\/a> AI infrastructure for global companies, but its approach<\/u><\/a> to AI regulation is a \u201clight touch\u201d. It has no comprehensive AI-specific statute. More importantly, its AI Security Institute assesses frontier AI risks but lacks statutory regulatory powers.<\/p>\n

So, if economic and geopolitical pressures fragment domestic regulation, can global cooperation overcome the same collective-action problem? Climate and nuclear governance offer useful insights.<\/p>\n

Climate Governance <\/strong><\/p>\n

Both climate change and frontier AI create transnational negative externalities. Greenhouse gas (GHG) emissions cross borders, as can harms<\/u><\/a> arising from AI models developed elsewhere. However, the collective-action problems they present take different forms.<\/p>\n

\u201cClimate change mitigation is a global public good<\/u><\/a>\u201d. Its benefits are shared globally, while its costs are borne by those reducing emissions. This creates a free-rider problem. States can benefit from others\u2019 efforts while limiting their own action to protect national economic interests.\u00a0<\/strong>International climate governance has struggled with this problem. For example, the Kyoto Protocol imposed binding<\/u><\/a> emission targets only on developed countries. So, the US did not ratify<\/u><\/a> it on the grounds that major developing emitters such as China and India faced no binding targets.\u00a0<\/strong>The Paris Agreement<\/u><\/a> also leaves scope for free riding because its nationally determined contributions are not legally binding<\/u><\/a>.<\/p>\n

Frontier AI presents a different problem. Unlike climate mitigation, its benefits (except for open-weight<\/u><\/a> models) can be excluded. For example, a US government directive suspended<\/u><\/a> non-US nationals\u2019 access to Mythos 5 and Fable 5. Furthermore, the capacity to develop<\/u><\/a> and control frontier systems is concentrated among a small group of states and firms. More importantly, the source of risk is concentrated. While all states contribute<\/u><\/a> to global emissions, some far more than others, frontier AI risks arise mainly from actors capable of developing and controlling these systems. Therefore, the principle<\/u><\/a> of Common but Differentiated Responsibilities and Respective Capabilities is not directly applicable. Although scientific participation should remain global (as seen with the UN<\/u><\/a> Independent International Scientific Panel on AI), regulatory responsibility need not be equally shared. Instead, responsibility should reflect technological capacity and control over the source of risk.<\/p>\n

Cooperation could begin by identifying systemic risks and the capabilities that warrant regulatory intervention. Climate governance had a clear objective.\u00a0The Paris Agreement aims to keep warming<\/u><\/a> well below 2\u00b0C and pursue efforts to limit it to 1.5\u00b0C. Frontier AI has no comparable agreed threshold.<\/p>\n

The Council of Europe\u2019s Framework Convention<\/u><\/a> on AI, not yet in force, sets no common quantitative risk thresholds.\u00a0This leaves each Party to decide when to act. Without a common yardstick, countries may end up with very different rules. Contrastingly, the EU\u2019s AI Act<\/u><\/a> sets a numerical trigger. A general-purpose AI model is presumed to have \u201chigh-impact capabilities\u201d when its training compute exceeds 10\u00b2\u2075 FLOP<\/u><\/a> (Article 51(2)<\/u><\/a>). It is then treated as posing \u201csystemic risk\u201d, triggering notification and additional obligations u\/A 52 and 55. However, the Act recognises that the \u201crelationship between compute and risk is highly uncertain and rapidly changing<\/u><\/a>\u201d. Therefore, it allows the Commission to consider other factors including capabilities, number of users and scalability, and to amend the threshold as technology evolves.<\/p>\n

Currently, there is no international agreement on quantitative AI risk thresholds. Numerical triggers exist in some US states<\/u><\/a> and South Korea<\/u><\/a>. A 2026 Executive Order directs<\/u><\/a> federal agencies to develop a \u201cclassified benchmark\u201d for models with advanced cyber capabilities. China<\/u><\/a> uses tiered risk grading without numerical<\/u><\/a> thresholds. What is missing is not a single fixed number but a shared, adaptable baseline.\u00a0Achieving this appears unlikely in the short term unless existing trust deficits are addressed. Nuclear governance shows how cooperation can emerge despite mistrust and strategic rivalry.<\/p>\n

Nuclear Governance <\/strong><\/p>\n

Nuclear governance offers a closer analogy for frontier AI than climate governance. Both are tied to national security and geopolitical rivalry. There was a race then; there is a race now.<\/p>\n

The similarities go further. Both can cause catastrophic harm: from nuclear detonation to the use of frontier AI to facilitate<\/u><\/a> development of biological or chemical weapons. Both depend on scarce, controlled inputs – fissile material for nuclear weapons and advanced chips for frontier AI. Capabilities are also concentrated. China and the US account for 90%<\/u><\/a> of the computing power \u2018among the world\u2019s top 500 AI supercomputers\u2019.<\/p>\n

This raises the question: can meaningful cooperation emerge between two rival powers? Even at the height of the Cold War, the US and the Soviet Union managed to agree<\/u><\/a> on rules. This suggests that strategic rivalry does not preclude cooperation, although it makes cooperation more difficult by creating<\/u><\/a> a prisoner’s dilemma. Both would benefit from mutual restraint, but the fear of falling behind encourages each to keep racing. The US AI Action Plan describes<\/u><\/a> it as \u201ca race to achieve global dominance\u201d. Whereas China, constrained<\/u><\/a> by chip controls, competes<\/u><\/a> through wider adoption and low-cost open-weight models (not open source, as training data and methods are not disclosed). Each strategy is rational on its own. Together, these expose both to cross-border risks neither can manage<\/u><\/a> alone. Once released, these models can be downloaded<\/u><\/a> and modified, increasing the risk of misuse.<\/p>\n

However, their situation differs from a classic prisoner\u2019s dilemma. The classic game is played once, and the prisoners cannot communicate. In contrast, the US and China can communicate and interact repeatedly. This repeated interaction allows reciprocity, and recent discussions on a formal AI dialogue<\/u><\/a> and an incident hotline offer<\/u><\/a> evidence of this. However, this requires trust in each side to know whether the other is keeping its word. A winner-takes-all mindset undermines<\/u><\/a> that trust. There is a deeper problem in this rhetoric of \u201cwinning\u201d. It assumes the winner can control what it builds. The US AI Action Plan acknowledges<\/u><\/a> that \u201cthe inner workings of frontier AI systems are poorly understood\u201d. Likewise, China\u2019s AI Safety Governance Framework accepts<\/u><\/a> that the \u201cspeed and direction of technological evolution may exceed human anticipation and control\u201d. Therefore, if control is lost, neither side wins. In this context, the game may then look more like a Stag Hunt<\/u><\/a>. Each side has reason to exercise restraint, only if it can trust the other to do the same. The only obstacle is trust (not incentives).<\/p>\n

Insights from the Cold War show that a shift is possible. In 1985, Reagan and Gorbachev agreed<\/u><\/a> that \u201ca nuclear war cannot be won\u201d. For nuclear weapons, China has also endorsed this principle. There are other reasons cooperation may be possible. Growing<\/u><\/a> public scepticism about AI in Western democracies, including the US, can affect the incentives of elected governments. For China, its export-oriented economy raises<\/u><\/a> the costs of instability. More importantly, US and Chinese capital<\/u><\/a> are also intertwined in AI. Therefore, their economic interdependence gives an interest in stable relations.<\/p>\n

The real question is how quickly cooperation can make restraint rational before serious harm occurs. Nuclear cooperation often followed moments of crisis. The Washington-Moscow hotline came<\/u><\/a> after the Cuban Missile Crisis. Frontier AI should not require a similar shock. There are already signs of cooperation. In 2024, the US and China agreed<\/u><\/a> that humans (not AI) should control decisions to use nuclear weapons. Recently, the US has also proposed<\/u><\/a> a notification mechanism for AI incidents.\u00a0 Cooperation could extend to sharing incident reports and commitments against using<\/u><\/a> AI for cyberattacks. Such measures could gradually build trust.<\/p>\n

However, bilateral progress, in the absence of trust, may still be slow. Other states can help. The 1963 Limited Test Ban Treaty began<\/u><\/a> with the US, Soviet Union and UK and later gained<\/u><\/a> much wider participation. The EU, UK, Canada and Japan could play a similar role in AI governance. None lead in frontier AI, but all are key players. Much of their influence comes from what frontier AI depends on. The EU, for example, offers a single market that both American and Chinese firms want to sell into. It is also home to the Dutch firm ASML. Its EUV technology – used<\/u><\/a> to make the world\u2019s most advanced chips – is \u201cunique to ASML.\u201d That gives Europe real leverage over the global chip supply chain.<\/p>\n

Acting together would strengthen their influence, and ties are already growing. Canada and the EU have agreed<\/u><\/a> to \u201cexplore opportunities to cooperate with like-minded partners.\u201d The UK is connected to the EU\u2019s AI infrastructure. UK-based researchers and firms can apply<\/u><\/a> to use its AI Factories (hubs for developing AI models). Japan is deepening<\/u><\/a> AI and semiconductor cooperation with the EU, and its firms produce<\/u><\/a> most of the world\u2019s high-end photoresists (key chipmaking chemicals). All four belong to a UK-coordinated international network that develops<\/u><\/a> shared AI testing methods.<\/p>\n

Testing matters because rivals rarely trust promises they cannot check. The 1963 treaty banned<\/u><\/a> only the nuclear tests each side could already detect with its own systems. Underground tests were left out because the US and Soviet Union could not agree<\/u><\/a> on how to verify them. AI poses a similar challenge. A US-China AI agreement would likewise need shared tests, which the network could help supply. China may distrust tests designed by US allies, but open methods would let each side run them independently. Each test that confirms compliance would build trust and, over time, make restraint rational for both. None of this means that other countries should be excluded. It simply reflects where the greatest risks, capabilities, and critical dependencies are concentrated. A small group can start the process. Wider global rules can follow.<\/p>\n

 <\/p>\n

The author is grateful to Shaun Mills, Former Associate Dean, School of Law and Policing, for his insightful discussions and encouragement. These discussions provided valuable motivation and inspiration for the development of this article.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

Advancing cyber capabilities and growing AI autonomy increase the risk of unsanctioned agent behaviour. Despite emerging evidence of these risks, global AI regulation remains difficult. Climate change and nuclear risks offer useful points of comparison (as both involve transnational risks requiring international cooperation). So, what can we learn from them, and where do the analogies […]<\/p>\n","protected":false},"author":35,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[6639],"tags":[7546,5035,4043],"authors":[8139],"article-categories":[6000],"doi":[],"class_list":["post-30105","post","type-post","status-publish","format-standard","hentry","category-uncategorized","tag-artificial-intelligence","tag-cooperation","tag-governance","authors-leo-utkarsh","article-categories-article"],"acf":{"subline":"Why AI Safety May Not Be Global, Yet"},"meta_box":{"doi":"10.17176\/20261028-132720-0"},"_links":{"self":[{"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/posts\/30105","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/users\/35"}],"replies":[{"embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/comments?post=30105"}],"version-history":[{"count":5,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/posts\/30105\/revisions"}],"predecessor-version":[{"id":30148,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/posts\/30105\/revisions\/30148"}],"wp:attachment":[{"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/media?parent=30105"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/categories?post=30105"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/tags?post=30105"},{"taxonomy":"authors","embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/authors?post=30105"},{"taxonomy":"article-categories","embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/article-categories?post=30105"},{"taxonomy":"doi","embeddable":true,"href":"https:\/\/voelkerrechtsblog.org\/de\/wp-json\/wp\/v2\/doi?post=30105"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}