Abstract: Can effective international governance for artificial intelligence remain fragmented, or is there a need for a centralised international organisation for AI? We draw on the history of other international regimes to identify advantages and disadvantages in centralising AI governance. Some considerations, such as efficiency and political power, speak in favour of centralisation. Conversely, the risk of creating a slow and brittle institution speaks against it, as does the difficulty in securing participation while creating stringent rules. Other considerations depend on the specific design of a centralised institution. A well-designed body may be able to deter forum shopping and ensure policy coordination. However, forum shopping can be beneficial and a fragmented landscape of institutions can be self-organising. Centralisation entails trade-offs and the details matter. We conclude with two core recommendations. First, the outcome will depend on the exact design of a central institution. A well-designed centralised regime covering a set of coherent issues could be beneficial. But locking-in an inadequate structure may pose a fate worse than fragmentation. Second, for now fragmentation will likely persist. This should be closely monitored to see if it is self-organising or simply inadequate.

Should Artificial Intelligence Governance be Centralised? Design Lessons from History

INTRODUCTION In 2018, Canada and France proposed the International Panel on Artificial Intelligence (IPAI). After being rejected at the G7 in 2019, negotiations shifted to the OECD and are presently ongoing. As the field of AI continues to mature and spark public interest and legislative concern  [41] , the priority of such governance initiatives reflects the growing appreciation that AI has the potential to dramatically change the world for both good and ill  [9] . Research into AI governance needs to keep pace with policy-making and technological change. Choices made today may have long-lasting impacts on policymakers' ability to address numerous AI policy problems  [7] . Effective governance can promote safety, accountability, and responsible behaviour in the research, development, and deployment of AI systems. AI governance research to date has predominantly focused at the national and sub-national levels  [6, 16, 44] . Research into AI global governance remains relatively nascent (though see  [5] ). Kemp et al.  [24]  have called for specialised, centralised intergovernmental agencies to coordinate policy responses globally, and others have called for a centralised 'International Artificial Intelligence Organisation'  [14] . Others favour more decentralised arrangements based around 'Governance Coordinating Committees', global standards, or existing international law instruments  [8, 28, 47] . No one has taken a step back to inquire: what would the history of multilateralism suggest, given the state and trajectory of AI? Should AI governance be centralised or decentralised? 'Centralisation', in this case, refers to the degree to which the coordination, oversight and/or regulation of a set of AI policy issues or technologies are housed under a single (global) institution. This is not a binary choice; it exists across a spectrum. Trade is highly (but not entirely) centralised under the umbrella of the WTO. In contrast, environmental multilateralism is much more decentralised. In this paper, we seek to help the community of researchers, policymakers, and other stakeholders in AI governance understand the advantages and disadvantages of centralisation. This may help set terms and catalyse a much-needed debate to inform governance design decisions. We first outline the international governance challenges of AI, and review early proposed global responses. We then draw on existing literatures on regime fragmentation  [3]  and 'regime complexes'  [36]  to assess considerations in centralising the international governance of AI. We draw on the history of other international regimes  1  to identify considerations that speak in favour or against designing a centralised regime complex for AI. We conclude with two recommendations. First, many trade-offs are contingent on how well-designed a central body would be. An adaptable, powerful institution with a manageable mandate would be beneficial, but a poorly designed body could prove a fate worse than fragmentation. Second, for now there should be structured monitoring of existing efforts to see whether they are they are self-organising or insufficient.
 THE STATE OF AI GOVERNANCE There is debate as to whether AI is a single policy area or a diverse series of issues. Some claim that AI cannot be cohesively regulated as it is a collection of disparate technologies, with different risk profiles across different applications and industries  [45] . This is an important but not entirely convincing objection. The technical field has no settled definition for 'AI', 2 so it should be no surprise that defining a manageable scope for AI governance will be difficult. Yet this challenge is not unique to AI: definitional issues abound in areas such as environment and energy, but have not figured prominently in debates over centralisation. Indeed, energy and environment ministries are common at the domestic level, despite problems in setting the boundaries of natural systems and resources. We contend that there are numerous ways in which a centralised body could be designed for AI governance. For example, a centralised approach could carve out a subset of interlinked AI issues to cover. This could involve focusing on the potentially highrisk applications of AI systems, such as AI-enabled cyberwarfare, lethal autonomous weapons (LAWS), other advanced military applications, or high-level machine intelligence (HLMI).  3  Another approach could govern underlying hardware resources (e.g. largescale compute resources) or software libraries. We are agnostic on the specifics of how centralisation could or should be implemented, and instead focus on the costs and benefits of centralisation in the abstract. The exact advantages and disadvantages of centralisation are likely to vary depending on the institutional design. This is an important area of further study, particularly once more specific proposals are put forward. However, such work must be grounded in a higher-level investigation of trade-offs in centralising AI governance. It is this foundational analysis which we seek to offer. Numerous AI issues could benefit from international cooperation. These include the potentially catastrophic applications mentioned above. It also encompasses more quotidian uses, such as AI-enabled cybercrime; human health applications; safety and regulation of autonomous vehicles and drones; surveillance, privacy and datause; and labour automation. Multilateral coordination could also use AI to tackle other global problems such as climate change  [43] , or help meet the Sustainable Development Goals  [46] . This is an illustrative but not exhaustive list of international AI policy issues. Global regulation across these issues is currently nascent, fragmented, yet evolving. A wide range of UN institutions have begun to undertake some activities on AI  [20] . The bodies covering AI policy issues range across existing organisations including the International Labour Organisation (ILO), International Telecommunication Union (ITU), and UNESCO. This is complemented by budding regulations and working groups across the International Organisation for Standardisation (ISO), International Maritime Organisation (IMO), International Civil Aviation Organisation (ICAO), and other bodies, as well as treaty amendments, such as the updating of the Vienna Convention on Road Traffic to encompass autonomous vehicles  [28] , or the ongoing negotiations at the Convention on Certain Conventional Weapons (CCW) on LAWS. The UN System Chief Executives Board (CEB) for Coordination through the High-Level Committee on Programmes has been empowered to draft a system-wide AI capacity building strategy. The High-level Panel on Digital Cooperation has also sought to gather together common principles and ideas for AI relevant areas  [19] . Whether these initiatives bear fruit, however, remains questionable, as many of the involved international organisations have fragmented membership, were not originally created to address AI issues and lack effective enforcement or compliance mechanisms  [32, p.2] . The trajectory of these initiatives matters. How governance is initially organised can be central to its success. Debates over centralisation and fragmentation are long-lasting and prominent with good reason. How we structure international cooperation can be critical to its success, and most other debates often implicitly hinge on structural debates. Fragmentation and centralisation exist across a spectrum. In a world lacking a global government, some fragmentation will always prevail. But the degree to which it prevails is crucial. We define 'fragmentation' as a patchwork of international organisations and institutions which focus on a particular issue area, but differ in scope, membership and often rules  [3, p.16 ]. We define centralisation as an arrangement in which governance of a particular issue lies under the authority of a single umbrella body. A regime complex is a network of three or more international regimes on a common issue area. These should have overlapping membership and cause potentially problematic interactions  [36, p.29] . These definitions and terms are by nature normatively loaded. For example, some may find 'decentralisation' to be a positive framing, while others may see 'fragmentation' to possess negative connotations. Recognising this, we seek to use these terms in a primarily analytical manner. We will use findings from each of these theoretical areas to inform our discussion of the history of multilateral fragmentation and its implications for AI governance.
 CENTRALISATION CRITERIA: HISTORY OF GOVERNANCE TRADE-OFFS In the following discussion, we explore a series of considerations for AI governance. Political power and efficient participation support centralisation. The breadth vs. depth dilemma, as well as slowness and brittleness support decentralisation. Policy coordination and forum shopping considerations can cut both ways.
 Political Power Regimes embody power in their authority over rules, norms, and knowledge beyond states' exclusive control. A more centralised regime will see this power concentrated among fewer institutions. A centralised, powerful architecture is likely to be more influential against competing international organisations and with constituent states  [36, pp.36-7 ]. An absence of centralised authority to manage regime complexes has presented challenges in the past. Across the proliferation of Multilateral Environmental Agreements (MEAs) there is no requirement to cede responsibility to the UN Environmental Programme in the case of overlap or competition. This has led to turf wars, inefficiencies and even contradictory policies  [3] . One of the most notable examples is that of hydrofluorocarbons (HFCs). HFCs are potent greenhouse gases, and yet their use has been encouraged by the Montreal Protocol since 1987 as a replacement for ozonedepleting substances. This has only recently been resolved via the 2015 Kigali Amendment to the Montreal Protocol, which itself has a prolonged implementation period. Similarly, the internet governance regime complex is diffuse. Multiple venues and norms govern technical standards, cyber crime, human rights, and warfare  [35] . Although the UN Internet Governance Forum (IGF) discusses several cross-cutting issues, it does not have a mandate to consolidate even principles, let alone negotiate new formal agreements  [33] . In contrast, other centralised regimes have supported effective management. For example, under the umbrella of the WTO, norms such as the most-favoured-nation principle (equally treating all WTO member states) principle have become the bedrock of international trade. The power and track-record of the WTO is so formidable that it has created a chilling effect: the fear of colliding with WTO norms and rules has led environmental treaties to self-censor and actively avoid discussing or deploying trade-related measures  [12] . Both the chilling effect and the remarkably powerful application of common trade rules were not a marker of international trade until the establishment of the WTO. The power of these centralised body has stretched beyond influencing states in the domain of trade, to moulding related issues. Political power offers further benefits in governing emerging technologies that are inherently uncertain in both substance and policy impact. Uncertainty in technology and preferences has been associated with some increased centralisation in regimes  [25] . There may also be benefits to housing a foresight capacity within the regime complex, to allow for accelerated or even proactive efforts  [39] . Centralised AI governance would enable an empowered organisation to more effectively use foresight analyses to inform policy responses across the regime complex.
 Supporting Efficiency & Participation Decentralised AI governance may undermine efficiency and inhibit participation. States often create centralised regimes to reduce costs, for instance by eliminating duplicate efforts, yielding economies of scale within secretariats, and simplifying participation  [15] . Conversely, fragmented regimes may force states to spread resources and funding over many distinct institutions, particularly limiting the ability of less well-resourced states or parties to participate fully  [32, p.2] . Historically, decentralised regimes have presented cost and related participation concerns. Hundreds of related and sometimes overlapping international environmental agreements can create 'treaty congestion'  [1] . This complicates participation and implementation for both developed and developing nations  [15] . This includes costs associated with travel to different forums, monitoring and reporting for a range of different bodies, and duplication of effort by different secretariats (ibid.). Similar challenges are already being witnessed in AI governance. Simultaneous and globally distributed meetings pose burdensome participation costs for civil society. Fragmented organisations must duplicatively invest in high-demand machine learning subject matter experts to inform their activities. Centralisation would support institutional efficiency and participation.
 Slowness & Brittleness of Centralised Regimes One potential problem of centralisation lies in the relatively slow process of establishing centralised institutions, which may often be outpaced by the rate of technological change. Another challenge lies in centralised institutions' brittleness after they are established, i.e., their vulnerability to regulatory capture, or failure to react to changes in the problem landscape. Establishing new international institutions is often a slow process. For example, the Kyoto Protocol took three years of negotiations to create and then another eight to enter into force. This becomes even more onerous with higher participation and stakes. Under the GATT, negotiations for a 26% cut in tariffs between 19 countries took 8 months in 1947. The Uruguay round, beginning in 1986, took 91 months to achieve a tariff reduction of 38% between 125 parties  [31] . International law has been quick to respond to technological changes in some cases, and delayed in others  [42, p.184 ]. Decentralised efforts may prove quicker to respond to complex, 'transversal' issues, if they rely more on informal institutions with a smaller but like-minded membership  [32, pp.2-3] . Centralised AI governance may be particularly vulnerable to sparking lengthy negotiations, because progress on centralised regimes for new technologies tends to be hard if a few states hold clearly unequal stakes in the technology, or if there are significant differences in information and expertise among states or between states and private industry  [42, pp.187-94] . Both these conditions closely match the context of AI technology. Moreover, because AI technology develops rapidly, such slow implementation of rules and principles could lead to certain actors taking advantage by setting de facto arrangements or extant state practice. Even after its creation, a centralised regime can be brittle; the very qualities that provide it with political power may exacerbate the adverse effects of regulatory capture; the features that ensure institutional stability, may also mean that the institution cannot adapt quickly to unanticipated outside stressors outside its established mission. The regime might break before it bends. The first potential risk is regulatory capture. Given the high profile of AI issue areas, political independence is paramount. However, as illustrated by numerous cases, including undue corporate influence in the WHO during the 2009 H1N1 pandemic  [11] , no institution is fully immune to regime capture, and centralisation may reduce the costs of lobbying, making capture easier by providing a single locus of influence. On the other hand, a regime complex comprising many parallel institutions could find itself vulnerable to capture by powerful actors, who are better positioned than smaller parties to send representatives to every forum. Moreover, centralised regimes entail higher stakes. Many issues are in a single basket and thus failure is more likely to be severe if it does occur. International institutions can be notoriously pathdependent and thus fail to adjust to changing circumstances, as seen with the ILO's considerable difficulties in reforming its participation and rulemaking processes in the 1990s  [2] . The public failure of a flagship global AI institution or governance effort could have lasting political repercussions. It could strangle subsequent, more well-conceived proposals in the crib, by undermining confidence in multilateral governance generally or capable governance on AI issues specifically. By contrast, for a decentralized regime complex to similarly fail, all of its component institutions would need to simultaneously 'break' or fail to innovate at once.  4  A centralised institution that does not outright collapse, but which remains ineffective, may become a blockade against better efforts. Ultimately, brittleness is not an inherent weakness of centralisationand indeed depends far more on institutional design details. There may be strategies to 'innovation-proof'  [29]  governance regimes. Periodic renegotiation, modular expansion, 'principles based regulation', or sunset clauses can also support ongoing reform  [30, pp.29-30] . Such approaches have often proved successful historically, due partially to decentralisation but, importantly, also to particular designs.
 The Breadth vs. Depth Dilemma Pursuing centralisation may create an overly high threshold that limits participation. All multilateral agreements face a trade-off between having higher participation ('breadth') or stricter rules and greater ambition of commitments ('depth'). The dilemma is particularly evident for centralised institutions that are intended to be powerful and require strong commitments from states. However, the opposite dynamics of sacrificing depth for breadth can also pose risks. The 2015 Paris Agreement on Climate Change was significantly watered down to allow for the legal participation of the US. Anticipated difficulties in ratification through the Senate led to negotiators opting for a 'pledge and review' structure with few legal obligations. Thus, the US could join simply through the approval of the executive  [23] . In this case, inclusion of the US (which at any rate proved temporary) came at the cost of significant cutbacks on the demands which the regime sought to make of all parties. In contrast, decentralisation could allow for major powers to engage in relevant regulatory efforts where they would be deterred from signing up to a more comprehensive package. This has precedence in the history of climate governance. Some claim that the US-led Asia-Pacific Partnership on Clean Development and Climate helped, rather than hindered climate governance, as it bypassed UNFCCC deadlock and secured non-binding commitments from actors not bound by the Kyoto Protocol [49, pp.259-60].  4  We thank Nicolas Moës for this observation. This matters, as buy-in may prove a thorny issue for AI governance. The actors who lead in AI development include powerful states that are potentially most adverse to global regulation in this area. They have thus far proved recalcitrant in the global governance of security issues such as anti-personnel mines or cyberwarfare. In response, some have already recommended a critical-mass governance approach to the military uses of AI. Rather than seeking a comprehensive agreement, devolving and spinning off certain components into separate treaties (e.g. for LAWS testing standards; liability and responsibility; and limits to operational usage) could instead allow for the powerful to ratify and move forward at least a few of those options  [48] . The breadth vs. depth dilemma is a trade-off in multilateralism generally. However, it is a particularly pertinent challenge for centralisation. The key benefit of a centralised body would be to be a powerful anchor that ensures policy coordination and coherence, without suffering fragmentation in membership. This dilemma suggests it is unlikely to have both. It will likely need to restrict membership to have teeth, or lose its teeth to have wide participation. A critical mass approach may be able to deliver the best of both worlds. Nonetheless these dilemma poses a difficult knot for centralisation to unravel.
 Forum Shopping Forum shopping may help or hinder AI governance, depending on the particular circumstances. Fragmentation enables actors to choose where and how to engage. Such 'forum shopping' may take one of several forms: moving venues, abandoning one organisation, creating new venues, and working across multiple organisations to sew competition between them  [4] . Even when there is a natural venue for an issue, actors have reasons to forum-shop. For instance, states may look to maximise their influence, appease domestic pressure  [40]  and placate constituents by shifting to a toothless forum  [18] . The ability to successfully forum-shop depends on an actor's power. Most successful examples of forum-shifting have been led by the US  [4] . Intellectual property rights in trade, for example, was subject to prolonged, contentious forum shopping. Developed states resisted attempts of the UN Conference on Trade and Development (UNCTAD) to address intellectual property rights in trade by trying to push them onto the World Intellectual Property Organization (WIPO) (ibid., 566) and then subsequently to the WTO  [18] , overruling protests from developing states. Outcomes often reflect power, but weak states and non-state actors can also pursue forum shopping strategies in order to challenge the status-quo  [22] . Forum shopping may help or hurt governance. This is evident in current efforts to regulate LAWS. While the Group of Governmental Experts has made some progress, on the whole the CCW has taken slow deliberations on LAWS. In response, frustrated activists have threatened to shift to another forum, as happened with the Ottawa Treaty that banned landmines  [10] . This strategy could catalyse progress, but also brings risks of further forum shopping and weak or unimplemented agreements. Forum shopping may similarly delay, stall, or weaken regulation of time-sensitive AI policy issues, including potential future HLMI development. It is plausible that leading AI firms also have sway when they elect to participate in some venues but not others. The OECD Expert Group on AI included representatives from leading firms, whereas engagement at UN efforts, including the Internet Governance Forum (IGF), do not appear to be similarly prioritised. A decentralised regime will enable forum shopping, though further work is needed to determine whether this will help or hurt governance outcomes on the whole.
 Policy Coordination There are good reasons to believe that either centralisation or fragmentation could enhance coordination. A centralised regime can enable easier coordination both across and within policy issues, acting as a focal point for states. Others argue that this is not always the case, and that fragmentation can mutually supportive and even more creative institutions. Centralisation reduces the occurrence of conflicting mandates and enables communication. These are the ingredients for policy coherence. As noted previously, the WTO has been remarkably successful in ensuring coherent policy and principles across the realm of trade, and even into other areas such as the environment. However, fragmented regimes can often act as complex adaptive systems. Political requests and communication between secretariats often ensures bottom-up coordination even in the absence of centralisation. Multiple organisations have sought to reduce greenhouse gas emissions within their respective remits, often at the behest of the UNFCCC Conference of Parties. When effective, bottom-up coordination can slowly evolve into centralisation. Indeed, this was the case for the GATT and numerous regional, bilateral and sectoral trade treaties, which all coalesced together into the WTO. While this organic self-organisation has occurred, it has taken decades, with forum shopping and inaction prevailing for many years. Indeed, some have argued that decentralisation does not just deliver 'good enough' global governance  [38]  that reflects a demand for diverse principles in a multipolar world. Instead, they argue 'polycentric' governance approaches  [37]  may be more creative and legitimate than centrally coordinated regimes. Arguments in favour of polycentricity include the notion that it enables governance initiatives to begin having impacts at diverse scales, and that it enables experimentation with diverse policies and approaches, learning from experience and best practices (ibid., 552). Consequently, these scholars assume "that the invisible hand of a market of institutions leads to a better distribution of functions and effects"  [50, p.7] . It is unclear if the different bodies covering AI issues will selforganise or collide. Many of the issues are interdependent and will need to be addressed in tandem. Some particular policy-levers, such as regulating computing power or data, will impact almost all use areas, given that AI progress and use is closely tied to such inputs. Numerous initiatives on AI and robotics are displaying loose coordination  [28] , but it remains uncertain whether the virtues of a free market of governance will prevail here. Great powers can exercise monopsony-like influence in forum shopping, and the supply of both computing power and machine learning expertise are highly concentrated. In sum, centralisation can reduce competition and enhance coordination, but it may suffocate the creative selforganisation of more fragmented arrangements over time.
 DISCUSSION: WHAT WOULD HISTORY SUGGEST? 4.1 A Summary of Considerations The multilateral track record and peculiarities of AI yield suggestions and warnings for the future. A centralised regime could lower costs, support participation, and act as a powerful new linchpin within the international system. Yet centralisation presents risks for AI governance. It could simply produce a brittle dinosaur, of symbolic value but with little meaningful impact on underlying political or technological issues. A poorly executed attempt could lock-in a poorly designed centralised body: a fate worse than fragmentation. Accordingly, ongoing efforts at the UN, OECD, and elsewhere could benefit from addressing the considerations presented in this paper.
 The Limitations of 'Centralisation vs. Decentralisation' Debates Structure is not a panacea. Specific provisions such as agendas and decision-making procedures matter greatly, as do the surrounding politics. Underlying political will may be impacted by framing or connecting policy issues  [26, pp.770-1] . The success of a regime is not just a result of fragmentation, but of design details. Moreover, institutions can be dynamic and broaden over time by taking in new members, or deepen in strengthening commitments. Successful multilateral efforts, such as trade and ozone depletion, tend to do both. We are in the early days of global AI governance. Decisions taken early on will constrain and partially determine the future path. This dependency can even take place across regimes. The Kyoto Protocol was largely shaped by the targets and timetables approach of the Montreal Protocol, which in turn drew from the Convention on Long-range Transboundary Air Pollution. This targets and timetables approach continues today in the way that most countries frame their climate pledges to the Paris Agreement. The choices we make on governing short-term AI challenges will likely shape the management of other policy issues in the long term  [7] . On the other hand, committing to centralisation, even if successful, may amount to solving the wrong problem. The problem may not be structural, but geopolitical. Centralisation could even exacerbate the problem by diluting scarce political attention, incurring heavy transaction costs, and shifting discussions away from bodies which have accumulated experience and practice  [21] . For example, the Bretton Woods Institutions of the IMF and World Bank, joined later by the WTO, are centralised regimes that engender power. However, those institutions had the express support of the US and may have simply manifested state power in institutional form. Efforts to ban LAWS and create a cyberwarfare convention have been broadly opposed by states with an established technological superiority in these areas  [13] . A centralised regime may not unpick these power struggles, but just add a layer of complexity.
 LESSONS AND CONCLUSIONS Our framework provides a tool for policy-makers to inform their decisions of whether to join, create, or forgo new institutions that tackle AI policy problems. For instance, the recent choice of whether to support the creation of an independent IPAI involved these considerations. Following the US veto, ongoing negotiations for its replacement at the OECD may similarly benefit from their consideration. For now, it is worth closely monitoring the current landscape of AI governance to see if it exhibits enough policy coordination and political power to effectively deal with mounting AI policy problems. While there are promising initial signs  [28]  there are also already growing governance failures in LAWS, cyberwarfare, and elsewhere. We outline a suggested monitoring method in Table  1 . There are three key areas to monitor: conflict, coordination, and catalyst. First, conflict should measure the extent to which principles, rules, regulations and other outcomes from different bodies in the AI regime complex undermine or contradict each other or are in tension either in their principles or goals. Second, coordination seeks to measure the proactive steps that AI-related regimes take to work with each other. This includes liaison relationships, joint initiatives, as well as the extent to which their rules, outputs and principles tend to reinforce one another. Third, catalyst raises the important question of governance gaps: is the regime complex self-organising to proactively address international AI policy problems? Numerous AI policy problems currently have no clear coverage under international law, including AI-enabled cyber warfare and HLMI. Whether this changes is of vital importance. These areas require investigation through multiple methods. Qualitative surveys of relevant organisations and actors can yield data on expert perceptions of these questions. Surveys can be augmented with quantitative methods, including network analyses of the regime complex relations  [36, p.32] . Natural language processing could be used to examine contradictions and similarities between different regime outputs, e.g., statements, meeting minutes, and more. Monitoring the outcomes of fragmentation can help to determine whether centralisation is needed. One way forward would be to empower the OECD AI Policy Observatory or the UN CEB to regularly review the monitoring outcomes. This could inform a democratic discussion and decision of whether to centralise AI governance further. Our framework and discussion may also be useful for non-state actors. Researchers and leading AI firms can play an important role in sharing technical expertise and informing forecasts of new policy problems on the horizon. The considerations may benefit their decisions of where to engage. Civil society has a key role as participants, watch-dogs, and catalysts. For example, the Campaign to Stop Killer Robots has sought to boost engagement and support for a LAWS ban within the CCW. Given prolonged delays and a pessimistic outlook, some have articulated a strategy of creating an entirely new forum for the ban, inspired by the Ottawa Treaty which outlawed landmines. Our framework can help reveal the potential virtues (allowing for progress while avoiding high-threshold deadlocks) and vices (enabling forum shopping) of such an approach. It could even help inform the structure of a future international institution, such as allowing for a modular, flexible structure with 'critical mass' agreements. One cross-cutting consideration is clear: a fractured regime sees higher participation costs that may threaten to exclude many civil society organisations altogether. The international governance of AI is nascent and fragmented. Centralisation under a well-designed, modular, 'innovation-proof' framework organisation may be a desirable solution. However, such a move must be approached with caution. How to define its scope and mandate is one problem. Ensuring a politically-acceptable and well-designed body is perhaps a more daunting one. It risks cementing in place a fate worse than fragmentation. Monitoring conflict and coordination in the current AI regime complex, and whether governance gaps are filled, is a prudent way of knowing whether the existing structure can suffice. For now we should closely watch the trajectory of both AI technology and its governance initiatives to determine whether centralisation is worth the risk. Table 1 : 1 Regime complex monitoring suggestions Key theme Question
			 A regime is a set of 'implicit or explicit principles, norms, rules and decision-making procedures around which actors' expectations converge in a given area of international relations'[27, p.186]. 2  We define 'AI' as any machine system capable of functioning 'appropriately and with foresight in its environment'[34, p.13]; see too[9, p.5]. 3  'High-level machine intelligence' has been defined as 'unaided machines [that] can accomplish every task better and more cheaply than human workers'[17, p.1].


Abstract: Recent progress in artificial intelligence (AI) raises a wide array of ethical and societal concerns. Accordingly, an appropriate policy approach is urgently needed. While there has been a wave of scholarship in this field, the research community at times appears divided amongst those who emphasize 'near-term' concerns and those focusing on 'long-term' concerns and corresponding policy measures. In this paper, we seek to examine this alleged 'gap', with a view to understanding the practical space for inter-community collaboration on AI policy. We propose to make use of the principle of an 'incompletely theorized agreement' to bridge some underlying disagreements, in the name of important cooperation on addressing AI's urgent challenges. We propose that on certain issue areas, scholars working with near-term and long-term perspectives can converge and cooperate on selected mutually beneficial AI policy projects, while maintaining their distinct perspectives.

Bridging the gap: the case for an 'Incompletely Theorized Agreement' on AI policy

Introduction The prevailing uncertainty around the trajectory and impact of artificial intelligence (AI) makes it clear that appropriate technology policy approaches are urgently needed. The possible negative ethical and societal impacts of AI are considerable: from algorithmic bias to AI-enabled surveillance, and from lethal autonomous weapons systems to widespread technology-induced unemployment. Moreover, some forecast that continuing progress in AI capabilities will eventually make AI systems a 'general-purpose technology'  [1] , or may even enable the development of 'high-level machine intelligence' (HLMI)  [2]  or other 'transformative' capabilities  [3, 4] . Debate on these latter scenarios is diverse, and has at times focused on what some have referred to as 'Artificial General Intelligence' (AGI)  [5] . On the surface, those concerned with AI's impacts can appear divided between those who focus on discernible problems in the near term, and those who focus on more uncertain problems in the longer term  [6] [7] [8] [9] . This paper wants to investigate the dynamics and debates between these two communities, with an eye to fostering policy effectiveness through greater cooperation. In doing so, this paper seeks to take up the recent call to 'bridge the near-and long-term challenges of AI'  [9] . The focus therefore is not on the relative urgency of existing algorithmic threats (such as e.g. facial recognition or algorithmic bias), nor on the relative plausibility of various advanced AI scenarios (such as e.g. HLMI or AGI), nor do we mean to suggest that a long-term perspective is solely focused on or concerned with AGI  [10] [11] [12] . Rather, the paper proposes that even if some community divergence exists, each group's overarching intention to contribute to responsible and ethical AI policy 1 would benefit from cooperation within key domains to maximize policy effectiveness. The paper suggests that differences may be overstated, and proposes that even if one assume such differences, these are not practically insurmountable. Rather, it argues that the principle of an 'incompletely theorized agreement', originally derived from constitutional law, provides both philosophical foundations and historical precedent for a form of cooperation between divergent communities that enables progress on urgent shared issues, without compromising on their respective goals. The paper proceeds as follows: in Sect. 2, we provide a short rationale for our proposed intervention. We briefly lay out the landscape of AI policy concerns and the structure of the associated AI ethics and policy community. This is followed by a discussion, drawing on historical cases as well as the contemporary challenges facing AI policy scholars, of how fragmentation within an expert community might hinder progress on key and urgent policies. In Sect. 3, we explore potential sources which could contribute to community divergence. We touch on epistemic and methodological disagreements and normative disagreements, and home in on pragmatic disagreements around the tractability of formulating AI policy actions today which maintain long-term relevance. We briefly review how serious these disagreements are, arguing that these trade-offs are often exaggerated, or do not need to preclude collaboration. Finally, in Sect.  4 , we propose that one consolidating avenue to harness mutually beneficial cooperation for the purpose of effective AI policy could be anchored in the constitutional law principle of an 'incompletely theorized agreement'. This proposal works under the assumption that the influence of a community on policy making is significantly stronger if they act as a united front, rather than as scattered subgroups.
 AI policy: a house divided? Recent progress in AI has given rise to an array of ethical and societal concerns.  2  Accordingly, there have been calls for appropriate policy measures to address these. As an "omni-use technology"  [13] , AI has both potential for good  [14] [15] [16]  as well as for bad  [17] [18] [19]  applications. The latter include: various forms of pervasive algorithmic bias  [20, 21] , challenges around transparency and explainability  [22, 23] ; the safety of autonomous vehicles and other cyberphysical systems  [24] , or the potential of AI systems to be used in (or be susceptible to) malicious or criminal attacks  [25] [26] [27] ; the erosion of democracy through e.g. 'computational propaganda' or 'deep fakes'  [28] [29] [30] , and an array of threats to the full range of human rights  [31, 32] . The latter may eventually cumulate in the possible erosion of the global legal order by the comparative empowerment of authoritarian states  [33, 34] . Finally, some express concern that continued technological progress might eventually result in increasingly more 'transformative' AI capabilities  [3] , up to and including AGI. Indeed, a number of AI researchers expect some variation of 'high-level machine intelligence' to be achieved within the next five decades  [2] . Some have suggested that if those transformative capabilities are not handled with responsibility and care, such developments could well result in new and potential catastrophic risks to the welfare, autonomy, or even long-term survival of societies  [35, 36] . Looking at the current debate and scholarship involved in the aforementioned areas, we note, along with other scholars  [6, 8, 37] , that there appears to be a fuzzy split, along a temporal 'near-term'/'long-term' axis. This perception matters because, as in many other fields and contexts, a perceived or experienced distinction may eventually become a self-fulfilling prophecy  [38] . This holds true even if the perceived differences are based on misperceptions or undue simplification by popular scientific media  [39] [40] [41] . Of course, fragmentation between near-and longer-term considerations of AI's impact is only one way to explore the growing community, and it may not be the sole issue to overcome to maximize policy impact. However, for the purpose of this paper, our focus is on this specific gap.
 The policy advantages of collaboration: lessons from history The current AI ethics and policy community is a young one. Policy shifts, on the other hand, take time. As such, it is difficult to clearly outline what impact current dynamics have. We are, after all, still in the early stages of these developments. Nevertheless, historical examples of adjacent fields can help to demonstrate and forecast how fragmentation, or, conversely cooperation, on policy goals within the AI ethics and policy community could strengthen impact on technology policy. Why should potential fragmentation along an axis such as near-and longer-term concerns worry us? History shows that the structure of a field or community affects the ability of its members to shape and influence policy downstream. Importantly, it shows that there is significant benefit derived from collaboration. We put forward three historic examples of adjacent technology policy fields to AI, meaning those that tackled equally new and emerging technologies. We briefly highlight one case where fragmentation may have contributed to a negative impact on the overall policy impact of the community and two cases where a collaborative effort yielded a positive impact on policy formulation.
 Nanotechnology One community that arguably suffered from a public pursuit of fractious division was the nanotechnology community in the early 2000s  [42, 43] . Internal disagreements came to a head in the 2003 'Drexler-Smalley' debate  [44] , which cemented an oversimplified public caricature of the field. Scholars reviewing this incident have argued that 'para-scientific' media created "polarizing controversy that attracted audiences and influenced policy and scientific research agendas. […] bounding nanotechnology as a field-in-tension by structuring irreconcilable dichotomies out of an ambiguous set of uncertainties."  [38] . This showcases a missed opportunity within a fragmented community to come together to promote greater political engagement with the responsible development of the technology.
 Recombinant DNA In the 1970s, concerns arose over recombinant DNA (rDNA) technology. In particular, the ethical implications of the ability to reshape life, as well as fears over potential biohazards from new infectious diseases led the biotechnology community to come together at the 1975 Asilomar Conference on Recombinant DNA to set shared standards  [45] . The conference is widely considered a landmark in the field  [46] : the scientist's and lawyers' commitment to a forthright open and public discussion has been argued to have stimulated both public interest and grounded policymaker discussion about the social, political and environmental issues related to genetic biotechnology in medicine and agriculture  [47] .
 Ballistic missile defense arms control In the wake of the creation of the atom bomb, a number of scientists expressed dismay and sought to institutionalize global control of these weapons. Early efforts to this end, such as the 1946 Baruch Plan, proved unsuccessful  [48, 49] . However, by the 1950s-1960s, a new 'epistemic community' emerged, bringing together both technical and social scientists, specifically in opposition to the development of antiballistic missile (ABM) systems. This community proved able to develop and disseminate this new understanding of nuclear deterrence dynamics to policymakers  [50] . They achieved this by maintaining a high level of consensus on concrete policy goals, by framing public discourse on the ethical goals, and by fostering links to both policymakers as well as to Soviet scientists. This allowed them to persuade key administration figures and shift policymaker norms and perceptions at home and internationally. Ultimately, setting the stage for the 1972 ABM Treaty, the first arms control agreement of this kind  [50, 51] .
 The pitfalls of fragmented efforts in AI Policy While some of the historical context is surely different, a number of these historical dynamics may well transfer to the emerging AI ethics and policy community  [52] . Those concerned with AI policy could benefit from exploring such historical lessons. This should be done with urgency, for two reasons. First, there is a closing window of opportunity. The field of AI policy is a relatively new one, which offers a degree of flexibility in terms of problem framings, governance instrument choice and design, and community alignment. Going forwards, however, this field has a high likelihood of becoming progressively more rigid as framings, public perceptions, and stakeholder interests crystallize. Current dynamics could, therefore, have far-reaching impacts, given the potential to lock in a range of path-dependencies, for example through particular framings of the issues at hand  [53] . In this context, a divided community which potentially treats policymakers or public attention as a zero-sum good for competing policy projects may compromise the legitimacy of its individual efforts in front of these. This could undercut the leverage of policy initiatives today and in the future. Worse, public quarrels or contestation may 'poison the well'. Policymakers may begin to perceive and treat a divided community as a series of interest groups rather than an 'epistemic community' with a multi-faceted but coherent agenda for beneficial societal impact of AI. Finally, from a policy perspective, it is important to note that while current regulatory initiatives are and should not always be directly transferable to future issues, neither are they categorically irrelevant. As such, they can often provide the second-best tools for rapidly confronting new AI challenges. This has its own pitfalls, but is often superior to waiting out the slow and reactive formulation of new policies. Moreover, the risks are concrete and timely. It is plausible that political moods will shift within the coming years and decades, in ways that make policy progress much harder. Furthermore, it is possible that other epistemic communities may converge and mobilize faster to embed and institutionalize alternative, less broadly beneficial framings of AI. Indeed, public and global framings of AI in recent years have seemed to drift towards narratives of competition and 'arms races' [54-56, but see also 57]. An inflection point for how societies use and relate to AI may eventually be reached. Missing such a window of opportunity could mean that the relative influence of those concerned with making the impact of AI beneficial (whether in the near or longer term) will decline, right as their voices are needed most.
 3 Conversely, many gains secured today could have lasting benefits down the road.
 Examining potential grounds for divergence There are a range of factors that could contribute to the clustering into fuzzy 'near-' and 'long-term' communities, and different scholars may hold distinct and overlapping sets of beliefs on them [cf. 8]. In the following paragraphs, we provide our first attempt at mapping some of these factors.  3  Some part of the divergence may be due to varying epistemic or methodological commitments. These could reflect varying levels of tolerance regarding scientific uncertainty and distinct views on the threshold of probability required before far-reaching action or further investigation is warranted. This means that concerns surrounding AI may depend on qualitatively different conceptions of 'acceptable uncertainty' for each group of observers. This may well be hard to resolve. Moreover, epistemic differences over the implicit or explicit disagreements of the modal standards in these debates, for example, debates over what types of data or arguments are admissible in establishing or contesting the plausibility or probability of risk from AI may contribute to further divergence. This could even lead to differential interpretations of evidence that are available. For instance, do empirically observed failure modes of present-day architectures  [58] [59] [60] [61]  provide small-scale proof-of-concepts of the type of difficulties we might one day encounter in AI 'value alignment', or are such extrapolations unwarranted? For our purposes, however, the most salient factor may be essentially pragmatic. Different perceptions of the empirical dynamics and path-dependencies of governing AI can inform distinct theories-of-change. These are intertwined with one's expectations about the tractability and relevance of formulating useful and resilient policy action today. In this context, Prunkl and Whittlestone  [8]  have recently argued that a more accurate picture and more productive dialogue could be achieved if scholars differentiated amongst the four dimensions on which views vary, in terms of the capabilities, impacts, certainty or extremity of AI systems. They emphasize that views on each of these questions fall on a spectrum. Taking this point on board, there are additional ways to cash out possible divergences. One debate might concern the question, how long-lasting are the consequences of near-term AI issues? If those that care about the longer term are convinced that these issues will not have long-lasting consequences, or that they would eventually be swamped by the much larger trends and issues  [3] , then this could lead them to discount work on near-term AI problems. However, it is important to note that near-term issues are likely to considerably affect the degree to which society is vulnerable to longer-term dangers posed by future advanced AI systems. Short-term or medium-term issues  [7, 37]  can easily increase society's general turbulence  [62] , or lock in counterproductive framings of AI or our relation to it. In general, we might expect many nominally near-term effects of AI on society (such as in surveillance; job automation; military capabilities) to scale up and become more disruptive as AI capabilities gradually increase  [18, 37] . Indeed, some longer-term scholars have argued that advanced AI capabilities considerably below the level of HLMI might already suffice to achieve a 'prepotence' which could pose catastrophic risks  [10] . This would make mid-term impacts particularly important to handle, and collaboration between different groups on at least some generalizable projects crucial. Another pragmatic question or concern is over how much leverage we have today to meaningfully shape policies that will be applicable or relevant in the long term, especially if AI architectures or the broader political and regulatory environment change a lot in the meantime  [8] . Some scholars may hold that future AI systems will be technically so different from today's AI architectures that research into this question undertaken today will not be relevant, or they might hold that such advanced AI capabilities may be so remote that the regulatory environment will have changed too much for meaningful policy work to be conducted right now  [63] . These people might argue that we had better wait until things are clearer and we are in a better position to understand whether and what research is needed or meaningful. In practice, this critique does not appear to be a very common or deeply held position. Indeed, as a trade-off it may be overstated. It is plausible that there are diverse areas on which both communities can undertake valuable research today, because the shelf life of current policy and research efforts might be longer than is assumed. To be sure, there is still significant uncertainty over whether current AI approaches can at all be scaled up to very advanced performance  [64] [65] [66] . Nonetheless, research could certainly depart from a range of areas of overlap  [67]  and shared areas of concern  [68, 69] , as we will discuss shortly. Moreover, policy making is informed by a variety of aspects which range across different time spans. Starting with political agendas that often reflect the current status quo, policy making is equally shaped by shifting public discourse, societal dynamics and high-impact shocks. The latter factor has played a key role in AI policy, where highprofile incidents involving algorithmic discrimination, lack of transparency, or surveillance have driven policy shifts, as seen for example in the pushback on discriminatory algorithms used in the UK visa selection processes  [70] , the Artificial Intelligence Video Interview Act regulating the use of AI in employee interviews, or the California B.O.T. Law requiring AI systems to self-identify  [71, 72] . In sum, it is plausible that many perceived 'barriers' to inter-community cooperation on policy are not all that strong, and that many 'tradeoffs' are likewise overemphasized. However, does that mean there are also positive, mutually productive opportunities for both communities to work on with regard to policy in spite of outstanding disagreements? What would such an agreement look like?
 Towards 'incompletely theorized agreements' for AI policy Above, we have reviewed potential sources for divergence within the community. We will now discuss how even in the context of apparent disagreement, pragmatic agreements on shared policy goals and norms could be reached. We propose to adopt and adapt the legal principle of an 'incompletely theorized agreement' for this purpose. Legal scholarship in constitutional law and regulation has long theorized the legal, organizational and societal importance of such incompletely theorized agreements. Their key use is that they allow a given community to bypass or suspend  [73, 74]  any theoretical disagreement on matters where (1) the disagreement appears relatively intractable and (2) there is an urgent need to address certain shared practical issues. Disagreements are intractable in cases where either it simply does not appear as if the question will be decisively resolved one way or the other in the near term, or where there is limited time and capacity to reason through all underlying disagreements  [73] . Incompletely theorized agreements can therefore apply to deep philosophical and ethical questions as much as to contexts of pervasive scientific uncertainty. The latter is especially the case on questions where it still remains unclear where and how we might procure the information that allows definitive resolution. Incompletely theorized agreements are a fundamental component to well-functioning legal systems, societies, and organizations. They allow for stability and flexibility to get urgent things done  [75] . These agreements have long played a key role in constitutional and administrative law, and have made possible numerous landmark achievements of global governance, such as the establishment of the Universal Declaration of Human Rights  [75, 76] . The framework has also been extended to other domains, such as the collective development and analysis of health-care policies in the face of pluralism and conflicting views  [77] . Incompletely theorized agreements have broad similarities with the notion of an 'overlapping consensus', developed by John Rawls, which refers to the way adherents of different (and apparently inconsistent) normative doctrines can nonetheless converge on particular principles of justice to underwrite the shared political community  [78] . This concept has been read as a key mechanism in the field of bioethics, serving to enable agreement despite different fundamental outlooks  [79] . It also already plays a role in the existing literature on computer ethics  [80] , as well as in the field of intercultural information ethics  [81] . Indeed, overlapping consensus has been proposed as a mechanism on which to ground global cooperation on AI policy across cultural lines  [82] . If overlapping consensus can ground inter-cultural cooperation, incompletely theorized agreements might serve as a similar foundation for practical cooperation between nearand long-term perspectives. In a related context, Baum has suggested that policy interventions aimed at securing longterm resilience to various catastrophes can often involve significant co-benefits in the near term, and so do not narrowly depend on all parties agreeing on the deep reasons for the policies proposed  [83] . Could incompletely theorized agreements ground cooperation amongst AI policy communities? We suggest that they could.
 Incompletely theorized agreements in AI policy: examples and sketches There are a range of issue areas where both groups could likely locate joint questions they would want addressed, and shared goals for which particular AI policies should be implemented. This holds even if their underlying reasons for pursuing these are not fully aligned. Without intending to provide an exhaustive, in-depth or definitive overview, a brief survey might highlight various areas for cooperation. For one, gaining insight into-and leverage on the general levers of policy formation around AI  [52]  is a key priority. What are the steps in the policymaking process which determine what issues get raised to political agendas and eventually acted upon, and which might be derailed by other coalitions  [84] ? Given the above, research into underlying social and societal developments is fruitful to advance all groups' ability to navigate mutually agreeable policy goals across this policy making cycle  [85] . Likewise, research into when, where or why global AI governance institutions might become vulnerable to regulatory capture or institutional path dependency ought to be an important consideration, whatever one's AI concerns are  [86, 87] . On a more operational level, this can feed into joint investigation into the relative efficacy of various policy levers for AI governance. For example, insights into when and how AI research labs or individual researchers adopt, or alternately cut corners on, responsible and accountable AI, or the incentivization of shifts in workplace culture or employee norms, could shape the policy proposals the community might make. The question of how to promote prosocial norms in AI research environments is of core interest to both communities with an eye to technology policy  [88] . This might cover, e.g. whether to publicly name problematic performance (e.g. biased results; lack of safety) in commercial AI products results in tech companies actually correcting the systems  [89] ; or whether codes of ethics are effective at changing programmers' decision making on the working floor  [90, 91] . All of these could be fruitful areas of collaboration on eventual policy proposals for either community. More specifically, there are a range of particular AI policy programs that we expect could be the site of an incompletely theorized agreement. 1. Incompletely theorized agreements could shape norms and policy debates over the appropriate scientific culture for considering the impact and dissemination of AI research  [92, 93] , especially where it concerns AI applications with potentially salient misuses. The underlying reasons for such policies might differ. Some may be concerned over abuses vulnerable populations, new vectors for criminal exploitation or the implications of new language models for misinformation  [94, 95] ; and others over the long-term risks from the eventual open development of advanced AI systems  [96] . Accordingly, incompletely theorized agreements in this area could converge on policies to shape researcher norms around improved precaution or reflection around the impact or potential misuse of research [97]. 2. Another example might be found in the domain of the global regulation of military uses of AI. This area has already seen years of shared efforts and even collaboration amongst a coalition of activists, lawyers, and institutions departing from both a near-term as well as longer-term perspective, such as the Future of Life Institute  [52] .
 Incompletely theorized agreements could ground productive policy cooperation on policy interventions aimed at preserving the integrity of public discourse and informed decision-making in the face of AI systems. Policies aimed at combating AI-enabled disinformation would be a natural site for incompletely theorized collaboration, because a society's epistemic security  [98]  is relevant from both a near-term and long-term perspective alike. 4. Similarly, incompletely theorized agreements surrounding the promotion of policies aimed at securing citizens' (political) autonomy and independence from unaccountable perception control could be promising. After all, practices of opaque technological management  [99]  or perception control  [100]  can enable authorities to increasingly shape individuals' and societies' behaviour and values. Regulation to restrict the deployment of such tools, to facilitate privacy-preserving AI, or to ensure transparency and accountability of the principals of such tools, are important from a near-term perspective concerned with the role of 'hyper nudges'  [101] , algocracy  [102] , or surveillance capitalism  [103] . Simultaneously, such policies are also critical to avert long-term worries over a 'value lock-in', whereby one generation or party might someday "invent a technology that will enable the agents alive at that time to maintain their values indefinitely into the future, controlling the broad sweep of the entire rest of the future of civilisation"  [104] . Although the underlying motives for each group to pursue policies on the abovementioned domains may be partially distinct, these technical differences are arguably thwarted by the benefits derived from achieving impactful and effective policy measures together. In many of these cases, the practical benefits of an incompletely theorized agreement would be at least fourfold  (1)  to reduce public confusion around these topics; (2) to present policymakers with an epistemic community delivering integrated policy proposals; (3) to support the articulation of regulations or governance instruments for specific policy problems, which need not assume further advances in AI capabilities, but which are also not reliant on provisions or assumptions that are vulnerable to 'obsolescence' if or when such advances do occur  [105] [106] [107] , giving such policies a longer shelf life; (4) to improve engagement of particular AI policies with the steadily increasing cross-domain nature of AI, which could help inform regulatory responses across domains. This is especially relevant because different fields (such as content moderation, health-care, or the military) often confront different yet similar versions of underlying problems  [108] .
 Limitations of incompletely theorized agreements We do not wish to suggest that incompletely theorized agreements are an unambiguously valuable tool across all AI policy cases, or even a definite solution for any one policy case. Such agreements do suffer from a number of potential drawbacks or trade-offs, which both communities should consider before invoking them in any particular case.  4  First, depending on one's assumptions around the expected degree of change in AI or in its societal impacts, incompletely theorized agreements could prove brittle. Incompletely theorized agreements bypass a full examination of the underlying disagreements to facilitate pragmatic and swift action on particular policies on which both communities find themselves in practical alignment in a specific moment in time. This may create a lack of clarity over the boundary conditions of that practical agreement, along with opacity over whether, or where (i.e. for which future particular questions around AI policy) the practical agreement might suddenly break down for either or all parties. Second, an incompletely theorized agreement is, in an important sense, a 'stopgap' measure more than a general ideal or permanent fix. As above, an incompletely theorized agreement might be most suited to situations where (a) practical policy action is urgently needed, and (b) underlying theoretical agreement by stakeholders on all engaged principles or questions does not seem close. However, over longer timeframes, deeper inquiry and debate do appear necessary  [73] . In addition, there is always the possibility that agreement was not, in fact, intractable within the near term. As such, a premature leap by the community into an incompletely theorized agreement to achieve some policy X might inadvertently curb the very conversations amongst the communities which could have led both to eventually prefer policy Y instead, had their conversation been allowed to run its course. Moreover, there is a key related point here, on which we should reflect. By advocating for the adoption of incompletely theorized agreements on AI policy today, we ourselves are in a sense assuming or importing an implicit judgment about the urgency of AI issues today, and about the intractability of the underlying debates. Yet these are two positions which others might contest. For example, by arguing that 'AI issues do not today meet that threshold of urgency that the use of an incompletely theorized agreement is warranted'. We wish to make this assumption explicit. At the same time, we expect that it is an assumption widely shared by many scholars working on AI policy, many of whom may well share a sense that the coming years will be a sensitive and even critical time for AI policies. Third, a sloppily formulated incompletely theorized agreement on an AI policy issue may not actually reflect convergence on particular policies (e.g. 'certification scheme for AI products with safety tests X, Y, Z'). Instead, it might solidify on apparent agreement on vague mid-level principles or values (e.g. 'AI developers should ensure responsible AI development'). These may be so broad that they do not ground clear action at the level of actual policies. If this were to happen, incompletely theorized agreements might merely risk contributing to the already-abundant proliferation of broad AI principles or ethical frameworks on AI that have little direct policy impact. While the ecosystem of AI codes of ethics issued in recent years have certainly shown some convergence  [109] [110] [111] , they have been critiqued as being hard to operationalize, and for providing only the appearance of agreement while masking underlying tensions in the principles' interpretation, operationalization, or practical requirements  [93, 112] . Situations where an incompletely theorized agreement does not manage to root itself at the level of concrete policies but only mid-level principles would be a worst-of-both-worlds scenario: it would reduce the ability of actors to openly reflect upon and resolve inconsistencies amongst-or disagreements about high-level principles, while not even affording improvements at facilitating concrete policies or actions in particular AI domains. To mitigate this risk, incompletely theorized agreements should, therefore, remain closely grounded in concrete and clearly actionable policy goals or outputs. Nonetheless, while limitations such as these should be considered in greater detail, we argue that they do not categorically erode the case for implementing, or at least further examining the promise of this principle and tool for advancing responsible AI policy.
 Conclusion AI has raised multiple societal and ethical concerns. This highlights the urgent need for suitable and impactful policy measures in response. Nonetheless, there is at present an experienced fragmentation in the responsible AI policy community, amongst clusters of scholars focusing on 'nearterm' AI risks, and those focusing on 'longer-term' risks. This paper has sought to map the practical space for intercommunity collaboration, with a view towards the practical development of AI policy. As such, we briefly provided a rationale for such collaboration, by reviewing historical cases of scientific community conflict or collaboration, as well as the contemporary challenges facing AI policy. We argued that fragmentation within a given community can hinder progress on key and urgent policies. Consequently, we reviewed a number of potential (epistemic, normative or pragmatic) sources of disagreement in the AI ethics community, and argued that these trade-offs are often exaggerated, and at any rate do not need to preclude collaboration. On this basis, we presented the novel proposal for drawing on the constitutional law principle of an 'incompletely theorized agreement', for the communities to set aside or suspend these and other disagreements for the purpose of achieving higher-order AI policy goals of both communities in selected areas. We, therefore, non-exhaustively discussed a number of promising shared AI policy areas which could serve as the sites for such agreements, while also discussing some of the overall limits of this framework. This paper does not suggest that communities should fully merge or ignore differences whatever their source may be. To be sure, some policy projects will be relevant to one group within the community but not the other. Indeed, community heterogeneity and diversity is generally a good thing for a scientific paradigm. Instead, the paper proposes to question some possible reasons for conflicting dynamics which could stall positive progress for policy making, and suggests an avenue for a higher-order resolution. Most of all, the paper hopes to pragmatically encourage the exploration of opportunities for shared work and suggested that work on such opportunities, where it is found, can be well grounded through an incompletely theorized agreement. We invite scholars in the ethical AI community to explore the strengths and limits of this tool. Footnote 1 ( 1 continued) assumption that policy making can positively influence the development and deployment of AI technology.
			 This paper perceives AI's ethical and societal concerns to be closely intertwined, and as such refers to the broader set of these actual and potential concerns throughout.
			 It should be emphasized that this mapping is only an indicative sketch, and would be much enriched by further examination, for example through structured interviews or comprehensive opinion surveys.
			 We thank one reviewer for prompting this discussion of the drawbacks of incompletely theorized agreements.