# AI policy ideas: Reading list

Related: [Ideas for AI labs: Reading list](/posts/GCMMPTCmGagcP2Bhd/ideas-for-ai-labs-reading-list). See also: [AI labs' statements on governance](/posts/iFrefmWAct3wYG7vQ/ai-labs-statements-on-governance).

This document is about AI policy ideas. It's largely from an x-risk perspective. ~~Strikethrough~~ denotes sources that I expect are less useful for you to read.

Lists
-----

Lists of government (especially US government) AI policy ideas. I recommend carefully reading the lists in the first ~5 bullets in this list, noticing ideas to zoom in on, and skipping the rest of this section.

*   [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://arxiv.org/pdf/2004.07213.pdf#page=5) (Brundage et al. 2020)
    *   Followed up by [Filling gaps in trustworthy development of AI](https://arxiv.org/abs/2112.07773) (Avin et al. 2021)
*   [12 tentative ideas for US AI policy](https://www.openphilanthropy.org/research/12-tentative-ideas-for-us-ai-policy/) (Muehlhauser 2023) ([EAF](https://forum.effectivealtruism.org/posts/iiRGCydMX7aiEjvGm/12-tentative-ideas-for-us-ai-policy-luke-muehlhauser))
*   [Survey on intermediate goals in AI governance](https://forum.effectivealtruism.org/posts/g4fXhiJyj6tdBhuBK/survey-on-intermediate-goals-in-ai-governance) (Räuker and Aird 2023)
*   [Policymaking in the Pause](https://futureoflife.org/wp-content/uploads/2023/04/FLI_Policymaking_In_The_Pause.pdf#page=5) (FLI 2023) ([LW](/posts/ERsbTthnzWLmNDCb5/fli-report-policymaking-in-the-pause))
*   [Frontier AI Regulation: Managing Emerging Risks to Public Safety](https://arxiv.org/abs/2307.03718) (Anderljung et al. 2023)
*   "30 actions to reduce existential risk" in [Existential risk and rapid technological change](https://www.simoninstitute.ch/blog/post/existential-risk-and-rapid-technological-change-a-thematic-study-for-undrr/) (Stauffer et al. 2023)
*   [The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation](https://arxiv.org/pdf/1802.07228.pdf#page=51) (Brundage et al. 2018)
*   [How major governments can help with the most important century](https://www.cold-takes.com/how-governments-can-help-with-the-most-important-century/) (Karnofsky 2023) ([LW](/posts/xtJQcmv7THCE8Topa/how-major-governments-can-help-with-the-most-important))
*   [\[Chuck Schumer\]](https://www.axios.com/2023/04/13/congress-regulate-ai-tech) (2023)
*   [\[Dan Hendrycks\]](https://twitter.com/DanHendrycks/status/1646899553319403521) (2023)
*   ["Discussion" in "Verifying Rules on Large-Scale NN Training via Compute Monitoring"](https://arxiv.org/pdf/2303.11341.pdf#page=13) (Shavit 2023)
*   [Future Proof](https://www.longtermresilience.org/post/introducing-future-proof-a-roadmap-to-boost-the-uk-s-resilience-to-extreme-risks) (CLTR 2021)
*   [~~Year 1 Report~~](https://www.ai.gov/wp-content/uploads/2023/04/NAIAC-Report-Year1.pdf#page=12) ~~(National Artificial Intelligence Advisory Committee 2023)~~
*   [~~Final Report~~](https://www.nscai.gov/2021-final-report/) ~~(National Security Commission on Artificial Intelligence 2021)~~
*   [~~Challenges to U.S. National Security and Competitiveness Posed by AI~~](https://www.rand.org/pubs/testimonies/CTA2654-1.html) ~~(Matheny 2023)~~
*   [~~"Policy Options" in "Artificial Intelligence and Strategic Trade Controls"~~](https://www.jstor.org/stable/resrep25191.7) ~~(Viski et al. 2020)~~
*   [~~Existential and global catastrophic risk policy ideas database~~](https://www.gcrpolicy.com/ideas) ~~(filter for "Artificial intelligence") (Sepasspour et al. 2022)~~
*   Various private lists and works in progress

Levers
------

Some sources focus on policy levers rather than particular policy proposals.

*   ["Governmental levers" in "Literature Review of Transformative AI Governance"](https://docs.google.com/document/d/1zFio8mrknZXK1NAQZVyc1iT67pI3aLGNVGPm3LHSkP8/edit#heading=h.o4tazsvok05a) (Maas draft)
    *   This report is excellent
*   [AI Policy Levers: A Review of the U.S. Government's Tools to Shape AI Research, Development, and Deployment](https://www.governance.ai/research-paper/ai-policy-levers-a-review-of-the-u-s-governments-tools-to-shape-ai-research-development-and-deployment) (Fischer et al. 2021)
*   ["Affordances" in "Framing AI strategy"](/posts/AGrBjEb6vFaJBwjTj/framing-ai-strategy-1#Affordances) (Stein-Perlman 2023)
*   [Current UK government levers on AI development](https://forum.effectivealtruism.org/posts/BFBf5yPLoJMGozygE/current-uk-government-levers-on-ai-development) (Hadshar 2023)
*   Standards (largely a non-government lever)
    *   [How technical safety standards could promote TAI safety](https://forum.effectivealtruism.org/posts/zvbGXCxc5jBowCuNX/how-technical-safety-standards-could-promote-tai-safety) (O'Keefe et al. 2022)
    *   [Standards for AI Governance](https://www.governance.ai/research-paper/standards-for-ai-governance-international-standards-to-enable-global-coordination-in-ai-research-development) (Cihon 2019)
    *   [Actionable-guidance and roadmap recommendations for the NIST AI Risk Management Framework](https://arxiv.org/abs/2206.08966) (Barrett et al. 2022) ([LW](/posts/JNqXyEuKM4wbFZzpL/actionable-guidance-and-roadmap-recommendations-for-the-nist-1))
    *   [~~AI Risk Management Framework~~](https://www.nist.gov/itl/ai-risk-management-framework) ~~(NIST 2023)~~
    *   [~~Ethically Aligned Design~~](https://standards.ieee.org/news/ead_v2/) ~~(IEEE 2017)~~
    *   [~~Global AI Standards Repository~~](https://ethicsstandards.org/repository/) ~~(OCEANIS)~~

Other policy guidance
---------------------

*   [Five considerations to guide the regulation of "General Purpose AI" in the EU's AI Act](https://ainowinstitute.org/news/gpai-is-high-risk-should-not-be-excluded-from-eu-ai-act) (AI Now Institute 2023)
*   [~~AI Accountability Policy~~](https://www.federalregister.gov/documents/2023/04/13/2023-07776/ai-accountability-policy-request-for-comment) ~~(National Telecommunications and Information Administration 2023)~~

Desiderata
----------

Some sources focus on abstract desiderata rather than how to achieve them.

*   [Asilomar AI Principles](https://futureoflife.org/open-letter/ai-principles/) (FLI 2017)
*   [Blueprint for an AI Bill of Rights](https://www.whitehouse.gov/ostp/ai-bill-of-rights/) (OSTP 2022)
*   [OECD AI Principles](https://oecd.ai/en/ai-principles) (OECD 2019)
    *   I think there are other relevant OECD reports or recommendations
*   [Policy Desiderata for Superintelligent AI](https://www.governance.ai/research-paper/public-policy-and-superintelligent-ai-a-vector-field-approach) (Bostrom et al. 2018)
*   [Universal Guidelines for Artificial Intelligence](https://thepublicvoice.org/ai-universal-guidelines/) (The Public Voice 2018)
*   [AI Policy Challenges](https://futureoflife.org/resource/ai-policy-challenges-and-recommendations/) (FLI 2018)
*   [~~Artificial Intelligence: the global landscape of ethics guidelines~~](https://arxiv.org/abs/1906.11668) ~~(Jobin et al. 2019)~~

Ideas[^nxx31uey48a]
-------------------

*   [Verifying Rules on Large-Scale NN Training via Compute Monitoring](https://arxiv.org/abs/2303.11341) (Shavit 2023) ([LW](https://www.lesswrong.com/posts/ByCwWRgvTsSC6Wxst/what-would-a-compute-monitoring-plan-look-like-linkpost))
*   [Why and How Governments Should Monitor AI Development](https://arxiv.org/abs/2108.12427) (Whittlestone and Clark 2021)
*   [Regulatory Markets: The Future of AI Governance](https://arxiv.org/abs/2304.04914) (Hadfield and Clark 2023)
*   [Why we need a new agency to regulate advanced artificial intelligence: Lessons on AI control from the Facebook Files](https://www.brookings.edu/research/why-we-need-a-new-agency-to-regulate-advanced-artificial-intelligence-lessons-on-ai-control-from-the-facebook-files/) (Korinek 2021)
*   [Compute Funds and Pre-trained Models](https://www.governance.ai/post/compute-funds-and-pre-trained-models) (Anderljung 2022)
*   [AI & Antitrust: Reconciling Tensions Between Competition Law and Cooperative AI Development](https://yjolt.org/ai-antitrust-reconciling-tensions-between-competition-law-and-cooperative-ai-development) (Hua and Belfield 2021)
    *   See also [Antitrust-compliant AI industry self-regulation](https://www.legalpriorities.org/research/antitrust-self-regulation.html) (O'Keefe 2021)
    *   See also [AI & antitrust/competition law](https://docs.google.com/document/d/1BnPVuE_UF8IcPNNZy5mg-ht4fHPPP74DzQBpckd-2bE/edit) (Aird 2022)
*   [How We Can Regulate AI](https://asteriskmag.com/issues/03/how-we-can-regulate-ai) (Balwit 2023) \[compute governance\]
*   [Regulating artificial intelligence: Proposal for a global solution](https://www.sciencedirect.com/science/article/pii/S0740624X22000843) (Erdélyi and Goldsmith 2022)
*   [Immigration Policy and the Global Competition for AI Talent](https://cset.georgetown.edu/publication/immigration-policy-and-the-global-competition-for-ai-talent/) (Huang and Arnold 2020)
*   [AI & Global Governance: Why We Need an Intergovernmental Panel for Artificial Intelligence](https://cpr.unu.edu/publications/articles/ai-global-governance-why-we-need-an-intergovernmental-panel-for-artificial-intelligence.html) (Miailhe 2018)
*   Liability
    *   New: [LPP on liability](https://www.legalpriorities.org/research/ai-insight-forum)
    *   [Products liability law as a way to address AI harms](https://www.brookings.edu/research/products-liability-law-as-a-way-to-address-ai-harms/) (Villasenor 2019)
    *   [Liability for artificial intelligence and other emerging digital technologies](https://op.europa.eu/en/publication-detail/-/publication/1c5e30be-1197-11ea-8c1f-01aa75ed71a1/language-en/) (European Commission 2019).
    *   Footnote 16 of [Collective action on artificial intelligence: A primer and review](https://www.sciencedirect.com/science/article/pii/S0160791X2100124X) (de Neufville and Baum 2021) points to academic work on AI and liability
*   [Crafting Legislation to Prevent AI-Based Extinction](https://committees.parliament.uk/writtenevidence/119676/pdf/) (Cohen and Osborne 2023)
    *   This is technically a response to government request for comment, but is really a policy proposal
*   [An AI Policy Tool for Today: Ambitiously Invest in NIST](https://www.anthropic.com/index/an-ai-policy-tool-for-today-ambitiously-invest-in-nist) (Anthropic 2023)
    *   Or see the [policy memo](https://cdn2.assets-servd.host/anthropic-website/production/images/Anthropic_NIST_v3.pdf) version of this blogpost
*   [Sharing Powerful AI Models](https://www.governance.ai/post/sharing-powerful-ai-models) (Shevlane 2022)
*   [You Can't Regulate What You Don't Understand](https://www.oreilly.com/content/you-cant-regulate-what-you-dont-understand-2/) (O'Rielly 2023)
*   [Killer Robots Are Here—and We Need to Regulate Them](https://foreignpolicy.com/2022/05/11/killer-robots-lethal-autonomous-weapons-systems-ukraine-libya-regulation/) (Trager and Luca 2022)
*   Incident reporting & tracking
    *   See PAI's [AI Incidents Database](https://incidentdatabase.ai) ([arXiv](https://arxiv.org/abs/2011.08512), [blogpost](https://partnershiponai.org/aiincidentdatabase/))

Policy proposals in the mass media
----------------------------------

*   [AI's Gatekeepers Aren’t Prepared for What's Coming](https://foreignpolicy.com/2023/06/19/ai-regulation-development-us-china-competition-technology/ ) (FP: Scharre 2023)
    *   This piece is great
*   [The world needs an international agency for artificial intelligence, say two AI experts](https://www.economist.com/by-invitation/2023/04/18/the-world-needs-an-international-agency-for-artificial-intelligence-say-two-ai-experts) (Economist: Marcus and Reuel 2023)
*   [We Need a Manhattan Project for AI Safety](https://www.politico.com/news/magazine/2023/05/08/manhattan-project-for-ai-safety-00095779) (POLITICO: Hammond 2023)
*   [Does the world need an arms control treaty for AI?](https://cyberscoop.com/ai-danger-arm-control-nuclear-proliferation/) (CyberScoop: Groll 2023)
*   [AI Desperately Needs Global Oversight](https://www.wired.com/story/ai-desperately-needs-global-oversight/) (WIRED: Chowdhury 2023)
*   [The Surprising Thing A.I. Engineers Will Tell You if You Let Them](https://www.nytimes.com/2023/04/16/opinion/this-is-too-important-to-leave-to-microsoft-google-and-facebook.html) (NYT: Klein 2023)
*   [We Must Regulate A.I. Here’s How.](https://www.nytimes.com/2023/05/03/opinion/ai-lina-khan-ftc-technology.html) (NYT: Khan 2023)
*   [We Must Declare Jihad Against A.I.](https://compactmag.com/article/we-must-declare-jihad-against-a-i) (Compact: Cuenco 2023)

~~Responses to government requests for comment~~
------------------------------------------------

*   [~~Response to the NTIA AI Accountability Policy~~](https://www.governance.ai/research-paper/response-to-the-ntia-ai-accountability-policy) ~~(GovAI 2023)~~
*   [~~GovAI Response to the Future of Compute Review - Call for Evidence~~](https://www.governance.ai/research-paper/future-of-compute-review-call-for-evidence) ~~(GovAI 2022)~~
*   [~~Future of compute review - submission of evidence~~](https://www.longtermresilience.org/post/future-of-compute-review-submission-of-evidence) ~~(CLTR et al. 2022)~~
*   [~~Anthropic Comment Regarding "Study To Advance a More Productive Tech Economy"~~](https://www.regulations.gov/comment/NIST-2021-0007-0055) ~~(Anthropic 2022)~~
*   [~~National Security Addition to the NIST AI RMF~~](https://www.scsp.ai/wp-content/uploads/2023/04/National-Security-Addition-to-NIST-AI-RFM.docx-1.pdf) ~~(Special Competitive Studies Project 2023)~~
*   [~~Reconfiguring Resilience for Existential Risk~~](https://www.cser.ac.uk/media/uploads/files/Maas_et_al._-_2021_-_Reconfiguring_Resilience_for_Existential_Risk_Sub.pdf) ~~(CSER 2021)~~
*   [~~Response to the UK's Future of Compute Review~~](https://www.longtermresilience.org/post/response-to-the-uk-s-future-of-compute-review-a-missed-opportunity-to-lead-in-compute-governance) ~~(CLTR et al. 2023)~~
*   [~~Submission to the NIST AI Risk Management Framework~~](https://www.governance.ai/research-paper/submission-to-the-nist-ai-risk-management-framework) ~~(GovAI 2022)~~
*   [~~Submission of Feedback to the European Commission's Proposal for a Regulation laying down harmonised rules on artificial intelligence~~](https://www.cser.ac.uk/resources/feedback-european-regulation/) ~~(CSER 2021)~~
*   [~~Advice to UN High-level Panel on Digital Cooperation~~](https://www.cser.ac.uk/news/advice-un-high-level-panel-digital-cooperation/) ~~(CSER and GovAI 2019)~~
*   [~~Consultation on the European Commission's~~ *~~White Paper on Artificial Intelligence: a European approach to excellence and trust~~*](https://www.fhi.ox.ac.uk/wp-content/uploads/EU-White-Paper-Consultation-Submission-GovAI-Oxford.pdf) ~~(GovAI 2020)~~

See also
--------

Very non-exhaustive.

*   [AI Governance: A Research Agenda](https://www.governance.ai/research-paper/agenda) (Dafoe 2018)
*   [Slowing AI](https://www.lesswrong.com/s/xMdkfEJhDNCL2KweB) (Stein-Perlman 2023)
*   [Survey on intermediate goals in AI governance](https://forum.effectivealtruism.org/posts/g4fXhiJyj6tdBhuBK/survey-on-intermediate-goals-in-ai-governance) (Räuker and Aird 2023)
*   [FACT SHEET: Biden-⁠Harris Administration Announces New Actions to Promote Responsible AI Innovation that Protects Americans’ Rights and Safety](https://www.whitehouse.gov/briefing-room/statements-releases/2023/05/04/fact-sheet-biden-harris-administration-announces-new-actions-to-promote-responsible-ai-innovation-that-protects-americans-rights-and-safety/) (The White House 2023)
*   [~~OECD AI Policy Observatory~~](https://oecd.ai/en/)
*   [~~PAI~~](https://partnershiponai.org) ~~and the~~ [~~AI and Shared Prosperity Initiative~~](https://partnershiponai.org/workstream/shared-prosperity/)
*   [~~\[Mark Warner\]~~](https://www.warner.senate.gov/public/index.cfm/2023/4/warner-calls-on-ai-companies-to-prioritize-security-and-prevent-malicious-misuse) ~~(2023)~~
*   [~~EU AI Act~~](https://en.wikipedia.org/wiki/Artificial_Intelligence_Act) ~~(~~[~~EU~~](https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:52021PC0206)~~)~~
*   [~~CHIPS Act~~](https://en.wikipedia.org/wiki/CHIPS_and_Science_Act) ~~(~~[~~US~~](https://www.govinfo.gov/content/pkg/COMPS-17006/pdf/COMPS-17006.pdf)~~)~~
*   [~~A pro-innovation approach to AI regulation~~](https://www.gov.uk/government/publications/ai-regulation-a-pro-innovation-approach) ~~(UK Office for Artificial Intelligence 2023)~~

* * *

This post is largely missing policy levers beyond domestic policy: international relations,[^sbv48jz3pq] international organizations, and standards.

*Some sources are roughly sorted within sections by a combination of x-risk-relevance, quality, and influentialness– but sometimes I didn't bother to try to sort them, and I've haven't read all of them.*

*Please have a low bar to suggest additions, substitutions, rearrangements, etc.*

*Thanks to Jakub Kraus and Seán Ó hÉigeartaigh for some sources.*

*Last updated: 10 July 2023.*

[^nxx31uey48a]: - I think these sources each focus on a particular policy idea– I haven't even skimmed all of them- Very non-exhaustive- Thanks to Sepasspour et al. 2022 and a private list for some sources 

[^sbv48jz3pq]: International agreements seem particularly important and neglected.One source (not focused on policy ideas or levers): Nuclear Arms Control Verification and Lessons for AI Treaties (Baker 2023).Oliver Guest agrees that there are not amazing sources but mentions:- CSET on international security in the context of military AI- CNAS on international arms control and confidence-building measures for military AI