Open Problems in Technical AI Governance
Authors: Anka Reuel
Benjamin Bucknall
Stephen Casper
Timothy Fist
Lisa Soder
Onni Aarne
Lewis Hammond
Lujain Ibrahim
Alan Chan
Peter Wills
Markus Anderljung
Ben Garfinkel
Lennart Heim
Andrew Trask
Gabriel Mukobi
Rylan Schaeffer
Mauricio Baker
Sara Hooker
Irene Solaiman
Sasha Luccioni
Nitarshan Rajkumar
Nicolas Moës
Jeffrey Ladish
David Bau
Paul Bricman
Neel Guha
Jessica Newman
Yoshua Bengio
Tobin South
Alex Pentland
Sanmi Koyejo
Mykel Kochenderfer
Robert Trager
Venue: Accepted by TMLR ()

Abstract: AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, outline why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance.

Published in Transactions on Machine Learning Research (04/2025)
Open Problems in Technical Al Governance
Anka Reuel
anka. reka..eeul@sttanfrddd eduu backnall@rohots.c ac. uk
Ben Bucknall
Stephen Casper', Tim Fist Lisa Soder', Onni Aarne", Lewis Hammond 2.9, Lujain Ibrahim², Alan Chan Peter Wills2,10 Markus Anderljung", Ben Garfinkel") Lennart H Andrew Trask2.12 Gabriel Mukobi', Rylan Schaeffer', Mauricio Baker Sara Hooker Irene Solaiman 16 Alexandra Sasha Luccioni's, Nitarshan Rajkumar's Nicolas Moës!", Jeffrey Ladish David Bau Paul-Andrei Bricman"", Neel Guha!, Jessica Newman"! Yoshua Bengioll.22 Tobin South Alex Pentland Sanmi Koyejo Mykel J. Kochenderferl, Robert Trager
Stanford University 2 Universitt of Oxford Oxford Marlin AI Governance Iviliative 4 *IT CSAIL Institate for Progress Center for New American Securily interface Tech Analysis and Pokicy Ideas for Europe e. V. Insstittee for Policy and Strategy
Cooperatine AT Foundation Centre for the Governance of AT Mila 12 OpenMMneed Independent Researcher 14 Cohere for A1 Hugging Face 16 University of Cambridge 17 The Future Society 18 Polisade Research
19 Northeastern University 20 Noema Research University of California, Berkeley 22 University of Montreal 23 MIT 24 Stanford HAI 20 Virtue AI
Reviewed on OpenReview: https: // openreview.net. f td= In050
Abstract
AI progress is creating a growing range of risks and opportunities, but it often unclear how they should be navigated. In many cases, the barriers and uncortainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. Tn this paper, we explain what technical AT governance is, outline why it important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended resource for technical rescarchers research funders looking to contribute to Al governance.
"Equal contribution; corresponding authors; order randomized. Work completed while BB Centre for the Governance
of AI.
Given its scope, inclusion author does entail endorsement of all aspects of the paper, with the exception of AR and BB.
Cite Reuel, Bucknall, al. (2025) "Open Problems Technical AT Governance."

---

Published in Transactions on Machine Learning Research (04/2025)
1
Introduction
The rapid development and adoption of artificial intelligence (AT) systems has prompted great deal of governance action from the public sector academia. and civil society (Anderljung et al. 2023a
Moès &
Ryan 2023 Barrett et al.) 2023), and industry (Anthropic 2023a Microsoft, 2023 Dragan et al. 2024
OpenA1 2024a), with the aim of addressing potential risks while capitalizing on benefits.
However, key decision-makors socking to govern AI often have insufficient information for identifying the need for intervention and assessing the efficacy of different governance options. Furthermore, the technical tools necessary for successfully implementing governance proposals are often lacking (Reuel al.) 2024a), leaving uncertainty regarding how policies are to be implemented. For example, while the concept of watermarking Al-generated content has gained traction among policymakers (see for example Council of the European
Cnion 2024 The White House 2023b G7 leaders 2023 Department for Science, Innovation & Technology
2023), it is unclear whether current methods are sufficient for achieving policymakers' desired outcomes, nor how future-proof such methods will be to improvements in AI capabilitios (Zhang et al. 2023 Ghosal ct al. Addressing these and similar issues will require further largeted technical advances.
2023
In this paper, we aim catalyse work aimed addressing such challenges and uncertainties through moti- vating and introducing the field of technical AI governance (TAIG), providing a laxonomy of subareas, and outlining open problems and research questions. Throughout the paper use the term technical AI governance to refer to technical analysis and tools for supporting the effective governance of AT, where 'tech- nical' pertains to the physical sciences, mathematics, engineering, and related fields; and 'Al governance' refers to the processes and structures through which decisions related to AI are made, implemented and enforced
[ Al governance includes research that both non-technical (e.g. in politics, economics, law, phi- losophy, etc.) and technical in nature (o.g. in computer science, enginocring, and mathematics). Through our use of this definition we simply refer to this latter calegory of work within the umbrella calegory of AA governance."
By the above definition, TAIG can contribute to AI governance in number of ways, such as by identifying opportunities for governance intervention, informing key decisions, and enhancing options for implementa- tion. For example, deployment evaluations that assess the downstream impacts a system (see Section 3.4 could help identify a need for policy interventions to address these impacts. Alternatively, being able to design models that are robust malicious modifications (see Section 6.4) could add the menu of governance options available to prevent downstream misuse.
In particular, make the following contributions:
We introduce the field of technical Al governance and motivate the need for work in this area.
We present a taxonomy of TAIG arranged along two dimensions: capacities, which refer to actions such as access and verification that are useful for governance, and targets, which refer to key elements in the AI value chain, such as data and models, to which capacitics can be applicd.
Finally, we outline open problems within each category of our taxonomy, along with concrete example questions for future research.
Our understanding of Al systems follows that of [Basdevant et al. 2024 encompassing infrastructure such as compilers, model components such datascts, code, and weights, as well as UX considerations.
2Sec, for example, (The White House 2023a The White House Office of Science and Technology Policy 2023
Presidency
of the Council of the European Union| 2024 Department for Science, Innovation and Technology et al. 2023a Department
for Science, Innovstion and Technology & Office for Artificial Intelligence 2023 Advisory Body on Artificial Intelligence 2023
European Commission 2023
3
be ( 23 32 Ali #
2023
3 Watermarks signals placed in output content that are imperceptible humanus, but casily detectable through application of specific dotector algorithm.
4To the extent that this definition TAIG includes measures for directly increasing performance, safety, or robustness of systems, we only consider such measures for cases which they support. the governance of AT.
2

---

Published in Transactions on Machine Learning Research (04/2025)
Figure I provides overview of the open problem areas, organized according to the taxonomy. We hope that this paper serves as a resource and inspiration for technical researchers aiming to direct their expertise towards pelicy-relevant topics.
1.1 Relation to Al Governance
As noted above, we define Al governance as the processes and structures through which decisions related to AI are made, implemented, and enforced. It encompasses the rules, norms, and institutions that shape the behavior of actors in the Al ecosystem, as well as the means by which they are held accountable for their actions As per our definition above, TAIG consists of technical analysis and tools for supporting the effective governance of Al. Here we outline three ways in which TAIG can contribute to Al governance, which we refer to identifying, informing, and enhancing
Firstly, TAIG can identify arcas where governance intervention is needed, through mapping tech- nical aspects of Al systems to social and political concepts, typically conceived of as being addressed through governance. For example, tracking and considering technical advances in AI video generation could allow for more accurate predictions of the risk of video deepfakes, and thus motivate the need for governance response.
Secondly, TAIG can inform governance decisions by providing decision-makers with more accurate information, allowing them to better compare the effectiveness of different governance options. For example, policymakers can choose between different regulatory instruments (for example, registration or disclosure), as well as how they enforce compliance (for example, ex anle rules or post hoe adjudication), with the efficacy of these options potentially depending on technical details. Information could stem from implemented TAIG methods, such as the outcome of assessments (see Section 3), or TAIG research that maps or monitors the AI ecosystem (see Section Dp. For example, more developed risk models for assessing potential harms of AI could inform targeted policies for their mitigation.
Finally, TAIG can enhance governance options by providing or enabling mechanisms for enforcing, incentivizing, or complying with mandated requirements. For instance, developing methods for the robust evaluation of models with black-box access could facilitate more comprehensive third-party anditing, thereby enhancing enforcement safety requirements.
Previous overviews of Al governance, such as Dafoe 2018 's AI governance research agenda, focus primarily socio-political governance challenges, including labor displacement, incquality, shifts in national power, and AI race dynamics, taxonomizes themes, gaps, and issues in AI governance more broadly (Birkstedt et al. 2023). This literature sets the stage for why technical solutions are needed (e-g., an "AT race that sacrifices safety" (Dafoe) 2018) could imply a need for governance-oriente
technical security mechanisms). More recent policy documents (sce c.g., OECD 2019 G7, 2023) cnumcrate high-level issucs like privacy concerns, bias, a lack of accountability, and security in AT, while including technical, socio-political, and institutional considerations. Our work contributes by focusing more narrowly on the technical dimension of those issues. For instance, where such reports highlight Al accountability as a governance challenge, our taxonomy offers concrete technical problem areas (like verifiable auditing and data provenance tracking) that would help achieve accountability in practice.
The Al risk literature categorizes both realized harms and anticipated risks of AI (see e.g., Lobel 2024 Shelby
et al. 2023 Raji et al. 2022a Weidinger et al. 2022 Critch & Russell 2023 For example, Shelby et al. (0023 presented a numan--cnteree taxonomy of sociotechnical harms from algorithmic systems, identifying five harm categorics (representational, allocative, quality-of-scrricc interpersonal, and socictal harms) basod a review of 172 research papers. Similarly, Tuk et al. (2024) conducted systematic literature review to derive 13 categories of systemic Al risks, including environmental harms, structural discrimination, and loss of control. Similar to the litcrature on challenges in AI governance, this prior work is relevant to our efforts in that it motivates work on AI governance - indeed, many of the open problems we list (such as developing robust evaluation methods) are motivated by those very risks. By situating our work among these prior
For other proposed definitions of Al governance, Bullock ct 2022 Daly al. 2021 Dafoc 2018
A useful parallel TAIG may he the concepts of regulatory Technology (Reg Tech) and supervisory Technology (SupTech) in the financial sector (Bank for International Settlements] 2021 which aim support financial regulation and oversight.
3

---

Published in Transactions on Machine Learning Research (04/2025)
Taxonomy of Open Problems Areas in Technical Al Governance
Models and Algorithms
Data
Compute
Deployment
Identification of Problematic Data Infrastructure and Metadata to Analyze Large Datasets
Definition of Chip and Cluster
Reliable
Downstream Impact Evaluations
Evaluations
Specifications for Model Training Classification of Workloads
Efficient
Evaluations
Assessment
(Multi-JAgent Evaluations
Attribution of Model Behaviour to Data
Prisevi-Preserving Third-Party Access to Datasets
Provision Compute Resources
Facilitation Third-Party Access Models
Access to Downstream User Logs Data
o-D-D
Preservation Evaluation Data Integrity
Access
Verification of Training Data
Verification of Chip Location
Verification of Model Properties Verification of Dynamic Systems Proof of Learning
Veriflable Audits Verification of Al-generated Content
Verification of Compute Workloads
Verification
Detection and Prevention of Training Data Extraction
Use of Hardware Mechanisms Al Security
Prevention
Detection of Adversarial Attacks Modification-Resist ant Models Detection and Authorization of Dual-Use Capability Inference Time
Model Theft
Shared Model
Anti-Tamper
Governance
Hardware
Model
Security
Enforcement
Disgorgement and Machine
Compute Usage
Restrictions
Unlearning
Translation of Governance Goals into Policies and Regulatory
Requirements
Deployment Corrections
Operationalization
Clarification Associated Risks
Prediction of Future Developments and Impacts
Assessment Environmental Impacts
Ecosystem Monitoring
Supply Chain Mapping
Figure 1: An overview of the open problem areas covered this report, organized according to our taxonomy.

---

Published in Transactions on Machine Learning Research (04/2025)
catalogs, we underscore that our contribution is focused extension: zoom in the technical layer of AT governance to provide a structured overview of open technical problems that underlie or intersect with identified Al governance challenges.
1.2 Scope and Limitations
This paper aims to give a broad overview of open technical problems for Al governance, identifying gaps in existing or suggested governance proposals, while avoiding taking a normative position on their desirability or efficacy. Indeed, the governance aims motivating some of the open questions outlined below may be in tension with each other, and we do not expect their solutions all to be used within the same governance framework. For example, broad access to some AI systems may be in conflict with ensuring their security.
At the same time, we conscious of the potential pitfalls of techno-solutionism that is, relying solely proposed technical fixes to complex and often normative social problems including lack of demo- cratic oversight and introducing further problems to be fixed (Michael el al. 2020 Lindgren & Dignum, 2023 Angel & Boyd 2024 Allen 2024). Many of the TAIG tools presented below are hypothetical and speculative, and we make no claims about the fcasibility of developing solutions. Furthermore, some of the TAIG measures highlighted are dual-use. For example, while hardware-enabled mechanisms for monitoring advanced compute hardware could provide increased visibility into the private development of the largest models, they could also potentially be applied to unreasonably surveil individuals using such hardware for legitimate purposes.
Thus, having solutions to all open problems outlined in this paper will not have solved AI governance. On the contrary, careful management will be necessary to determine a balance between capacities that are in tension with each other, and to ensure that dual-use capacitios are not misused. Furthermore, many Al governance problems may rely predominantly on non-technical solutions, such as ensuring the appropriate inclusion of countrics impacted by AI in international AI governance docision-making (Trager al. 2023). However, we argue that making progress on the technical problems outlined below can help ensure more robust Al governance net.
We view TAIG as related to (and in some places overlapping with) existing arcas of rescarch. Namely, TAIG differs from topics in Al safely and alignment in that it is not aimed at directly improving the safety of AI systems. Furthermore, while sociotechnical approaches to Al safety and governance (see e.g. Lazar & Nelson 2023 Bogen & Winccoff,, 2024 Oduro & Kneese 2024) view "socicty and technology together as one coherent system" (Chen & Metcalf 2024). TAIG considers the instrumental value of technical work for enacting governance. Taken her with TAIG, these diverse approaches can serve as complementary methods for mitigating risks and promoting beneficial outcomes of Al (Narayanan et al. 2023). Finally, and as noted above, we view TAIG as a subset of AI governance more broadly, which also includes many topics in diverse disciplines including political science, economics, and philosophy.
We consider some notable fields, topics, and problems to be out scope for this paper. In particular, technical work that directly improves the performance, safety, or robustness of AI systems, or addresscs related cthical concerns while highly relevant to AI governance is considered out of scope. Topics regarding government
public-sector use of AI (Margetts & Dorobantu, 2019 Aitken et al. 2022 Margetts 2022 Straub et al. 2023) ways in which Al could itself be used to defend against ameliorate downstream harms of Al BBrmardi et al. 2024) are also out of scope.

---

Published in Transactions on Machine Learning Research (04/2025)
1.3
Reader Guide
Table 1: Relevant problem areas organized by reader background
ML Theory
Assessment
3.1.2
3.1.3
3.2.1
3.2.2
3.3.1
3.3.2
3.4.1
Access
4.1.1
4.2.1
4.3.1
Verification
5.1.1
5.2.2
5.3.1
5.3.3
5.4.1
5.4.2
Security
6.1.1
6.3.1
6.3.2
6.3.3
6.4.1
6.4.2
6.4.3
Operationalization
7.2
Applied ML
Assessment
3.1.2
3.3.1
3.3.2
3.4.1
Access
4.3.
4.4.1
Security
6.4.3
Operationalization
7.1
7.2
Ecosystem Monitoring
8.1
8.2
8.3
Cybersecurity
Verification
5.2.2
Security
6.2.1
6.2.3
6.3.1
6.4.3
Operationalization
7.2
Cryptography
Assessment
3.1.1
3.2.2
Access
4.1.1
4.1.2
4.2.1
4.3.1
4.4.1
Verification
5.1.1
5.2.
5.2.2
5.3.3
5.4.1
5.4.2
Security
6.2.
6.2.3
6.3.2
6.4.3
Hardware Engineering
Assessment
3.1.2
3.2.1
3.2.2
Access
4.2.
Verification
5.2.
5.2.2
Security
6.2.1
6.2.2
6.2.3
6.3.1
6.3.2
Software Engineering
Assessment
3.1.1
3.1.2
3.3.2
3.4.1
Access
4.2.1
Verification
5.2.2
Security
6.2.
6.3.1
Mathematics and Statistics
Assessment
3.1.2
3.4.1
Ecosystem Monitoring
8.2
8.3
This paper provides broad overview of open problems across the taxonomy defined in Section 2 Given the extensive nature of the main content (Sections S we have structured it for selective reading:
Each section is self-contained, allowing readers to focus on their area(s) of interest..
Each section begins with a summary table of problem areas.
Specific open problems within each area are highlighted in bold.
Example research questions provided in boxes at the start of each subsection.
Table I
offers suggested relevant problem areas based on reader expertise.
We attach a two-page policy brief in appendix
This structure aims to facilitate quick identification of key issues and relevant problems for readers across various backgrounds and interests.

---

Published in Transactions on Machine Learning Research (04/2025)
2
Taxonomy
The paper is organized according to two-dimensional taxonomy, based around capacities and targets. Ca- pacities encompass comprchensive suitc of abilities and mechanisms that enable stakcholders to understand and shape the development, deployment, and use of AI, such as by assessing or verifying system properties. These capacities are neither mutually exclusive nor collectively exhaustive, but they do capture what we be- lieve are the most important clusters of technical AT governance. We list all considerod capacitios, along with descriptions, in Table 2 An overview of methodology, and in particular how we derived this laxonomy, be found in Appendix
Table 2: Overview of capacities and their importance for AI governance
Capacity
Description
Why it matters for governance
Assessment
The ability to evaluate AI systems, involv- ing both technical analyses and considera- tion of broader societal impacts.
Enables the identification and under- standing of system capabilities and risks, allowing for more targeted gov- ernance intervention.
Access
The ability to interact with Al systems, in- cluding model internals, well as obtain relevant data and information while avoid- ing unacceptable privacy costs.
Enables external research and assess- ment of Al systems, and aids in fairly distributing benefits of AI across soci- ety.
Verification
The ability of developers or third parties to verify claims made about AT systems' development, behaviors, capabilities, and safety.
Establishes trust in Al systems and confirms compliance with regulatory requirements.
Security
The development and implementation of measures to protect AI system compo- nents from unauthorized access, use, or tampering.
Ensures the integrity, confidentiality, and availability of Al systems and guards against misuse.
Operationalization
The translation of ethical principles, legal requirements, and governance objectives into concrete technical strategies, proce- dures, or standards.
Bridges the gap between abstract prin- ciples and practical implementation of regulatory requirements.
Ecosystem Monitoring
Understanding and studying the evolving landscape of Al development and applica- tion, and associated impacts.
Fnables informed decision-making, an- ticipation of future challenges, and identification of key leverage points for effective governance interventions.
The second axis of our taxonomy pertains to the targets that encapsulate the essential building blocks and operational elements of Al systems that governance efforts may aim to influence or manage. They are adapted from categorics introduced to each target. We structure our paper around the resulting pairs of capacities and targets, with the exception of operationalization and ecosystem monitoring which cut across all targets. The targets considered in this report are summarized in Table
[Bommasani et al. 2023b). Each capacity given above can be applied
We recognize that organizational processes undertaken during the development and deployment of AI systems intersect with and shape these targets, and could be considered as regulatory targets in their own right.
7 (Repeat of footnate Our understanding of AI systems follows that of (Basdevant 2024 encompassing infrastructure such as compilers, model components such as datasets, code, and weights, well UX considerations.

---

Published in Transactions on Machine Learning Research (04/2025)
Table 3: Overview of targets
Target
Description
Data
The pretraining, fine-tuning, retrieval, and evaluation datasets on which Al systems are trained and benchmarked.
Compule
Computational and hardware resources required to develop and deploy Al systems.
Models and Algorithms
Core components of AT systems, consisting of software for training and inference, their theoretical underpinnings, model architectures, and learned parameters.
Deployment
The use of Al systems in real-world settings, including user interactions, and the resulting outputs, actions, and impacts.
However, we have chosen not to include them as explicit targets in our taxonomy as processes mostly involve non-technical challenges that fall outside the scope of our paper. In cases where processes do face challenges, we address such issues within the context of the most relevant target. For example, compliance with content-creators' right to opt-out is dependent identifying copyrighted samples in datasets (Section
3.1

---

Published in Transactions on Machine Learning Research (04/2025)
3 Assessment
Evaluations and asscssments of the capabilities and risks of AT systems have been proposed as key compo- nent in AI governance regimes. For example, model evaluations and red-teaming comprised comprissed key part of the voluntary commitments agreed between labs and the UK government at the Bletchley Summit (Department for Science, Innovation & Technology, 2023). Furthermore, the White House Executive Order on Artificial Intelligence requires developers of the most computo-intensin models to share the results of all red-team tests of their model with the federal government (The White House 2023a
The purpose of assessment is detect problematic behavior or impacts of Al systems before resulting harms can materialize, well as to ensure systems are safe, robust, and non-discriminatory. However, the assessment of some targets, especially in the context of foundation models, is currently more art than science, with significant number of open challenges (Chang et al. 2024 Weidinger et al. 2023 These issues are exacerbated by the fact that evaluations are expensive to conduct at scale. While assessment and evaluation standards are emerging (National Institute of Standards and Technology (NIST), 2023 UK Al
Safety Institute 2024), there are still fundamental open technical problems that need to be addressed ensure robust and informative asscssments.
Assessment
Models and Algorithms
Data
Compute
Deployment
Identification of Problematic Data Infrastructure and Metadata to Analyze Large Datasets
Definition of Chip and Cluster Specifications for Model Training Classification of Workloads
Reliable Evaluations Efficient Evaluations (Multi-JAgent Evaluations
Downstream Impact Evaluations
Attribution of Model Behaviour to Data
Figure 2: Open problem areas in the Assessment capacity, organized by target
3.1
Data
Example Research Questions
1. How can methods for identifying problematic data be scaled to large (on the magnitude of trillions of lokens/samples) datasets? (3.1.1
2. How can license collection be automated to prevent training on unlicensed data? 3.1.1
3. How the accuracy of licenses be ensured when aggregating datasets from multiple sources? 331.1
4. How can problematic data be identified without full/direct access the dataset? 3.1.1
5. How can contamination of training data with problematic samples be reliably detected? 3.1.1
6. How can harmful data be removed from dataset without facilitating its easy identification by malicious actors? 331.1
Red teameng refers deliberately trying find ways make system behave poorly, produce harmful outputs, be misused, in order identify potential risks and vulnerabilities be addressed.

---

Published in Transactions on Machine Learning Research (04/2025)
7. What license and meta-data reporting requirements could assist in responsible data practices?
3.1.2
8. What infrastructure is needed to enable researchers to audit large datasets? (31.22
9. How can macro-scale dataset properties, such as persistent bias, be identified and measured? 3.1
10. What information about datasets is necessary for determining their suitability for training? 3.1.2
11. What is the effect problematic data on downstream system performance? 3.1.3
12. Can system behaviors and/or or properties be accurately attributed to pretraining and/or fine- the-tuning data samples? 331.3
3.1.1 Identification of Problematic Data
Motivation: Data plays a central role in the development and resulting capabilities of Al systems. Therefore, issues with data can propagate downstream, resulting in undesirable properties of models. We identify two ways in which data can be problematic.
The first is that data samples may violate some legal ethical principle, simply by virtue of being included in a dataset. For instance, the presence of a sample could constitute copyright or privacy violation (Brown
et al.
2022
Rahman & Santacana, 2023
Subramani et al. 2023 Marcus & Southen 2024), data poisoning
Biggio et al.;
2012
Steinhardt et al. 2017 Wallace et. al. 2020: Carlini & Terzis 2021
Carlini 2021
Schuster et al.
2021
Carlini et al. 2023a), or be inherently harmful Thiel 2023
Birhane et al. 2021
2023
Luccioni & Viviano 2021).
The second way that data could be problematic is if its use in training causes undesirable downstreamm effects. For instance, models trained factually incorrect content in training data, such as vaccine disinformation. might replicate those factual inaccuracies. Indeed, (Lin et al. 2022b) demonstrate how models can "[generate] many false answers that mimic popular misconceptions! Alternatively, low-quality data, such as inaccuracies in low-resource languages, can compromise performance of models in those languages (Kreutzer et al. 2022
Being able to identify problematic data used in the development of AI systems could facilitate various governance measures. For example, developers found be using inherently harmful data could face penalties for improper Alternatively, developers found to be training data that induces harmful model behaviors could face higher burden of proof when demonstrating that their models are sufficiently safe for public deployment.
Open problems:
Identifying problematic data given access to the training data. For model developers with full access to the dataset, the major challenge is defining concrete, operationalizable criteria for detecting and removing problematic samples before training. Some problematic samples may be casier to identify than others; for instance, social security numbers can be identified with regexes direct copies can be easily identified through pattern matching. However, the identification of other forms of problematic data samples poses more of a challenge. For example, understanding whether a data sample constitutes copyright infringement requires knowledge of copyright law, making judgments about how much lexical similarity amounts to infringement, and the intended application of the data (Henderson et al. 2023a Balganesh, 2012). Other approaches could resemble detection methods for data contamination, such as fuzzy string matching, or audio and video fingerprinting, as used by Youtube to identify copyrighted pieces (Cano al.
2005
Wu et al. 2017).
Identifying problematic data without access to the training data. Regulators, auditors, or other entities who don't necessarily have access to a system's training data may need to find proxies for problematic data based on model behavior. Potential approaches include calculating confidence scores for the inclusion of data points (Li et al. (024a),, or using data watermarks introduced by creators that can be detected without access the training corpora (Wei al. (224)). Other approaches may be used to identify the use of copyrighted data, including inference attacks
(hhkrri et. al. 2017) and influence functions (Grosse et. al.
10

---

Published in Transactions on Machine Learning Research (04/2025)
2023
Choe et al. 2024 Yet, these approaches lack robustness an issue that further research could aim
to addrsss.
Tracing data provenance. It can be challenging for model developers to ensure training data is correctly licensed due to licenses frequently being aggregated and misrepresented (Longpre et al. 2023 Hence, better tooling for data provenance will be necessary if developers are to feel comfortable that they are honoring creators' licenses. The Data Provenance Initiative (Longpre et al.) 2023) conducted a large-scale audit of open-source fine-tuning data collections and cataloged corresponding data sources, licenses, and creators in attempt to establish the provenance of data. However, the Data Provenance Initiative is limited in the datascts they cover, and expanding it to other data is resource-intensive. Automated license collection or standardized meta-data reporting for datasets (Longpre et al. 2024b) could help developers to release systems without facing legal ambiguity.
Silent removal of harmful data. Another challenge that of being able to remove harmful data without the unwanted side-effect of publicly identifying it, and thus allowing malicious actors an easy way to source such data (Thiel, 2023). For example, the LAION-5B image dataset (Schuhmann et al. 2022 is composed of image URLs and metadata, opposed to the images themselves. If one were to simply remove the URLs of the identified harmful images from the dataset, then this could provide malicious actors with the locations of these harmful images, either by comparing the datasets before and after removal, or directly through a repository change log where the dataset hosted. However, care should be taken as methods for addressing this challenge could potentially also allow for the subversion of existing techniques (such as open change logs) for facilitating transparency into developers' data handling practices.
3.1.2 Infrastructure and Metadata to Analyze Large Datasets
Motivation: In addition to improved methods for auditing and identifying problematic samples in datasets, methods and infrastructure are needed for implementing these methods at scale. Contemporary datasets commonly contain on the order of tens of terabytes of data 2 introduuiing the challenge of having the compu- tational resources to store and handle such large quantities data. Addressing this challenge will be crucial for enabling the identification of problematic data in practice.
Open problems:
Automating meta-data collection for datasets. A challenge with large-scale dataset infrast is that metadata including links to the original data sources or license information - is not always provided (Piktus et al. 2023). An open research question is how the collection metadata for previously published datasets could be automated. Additionally, exploring the automatic addition of cryptographic check sums as part of the metadata at the dataset level could help to enable users to confirm that the data they download matches the original data. Such an approach could be a partial solution to ensuring that datasets have not been altered (either maliciously or incidentally), especially in light of advances in data poisoning attacks CCrlini et al. 2023a).
Determining relevant targets for dataset analysis. Another question concerns the appropriate metrics for ascertaining the suitability of datasets for use in training, based on diffuse and macro-scale properties. For example, a dataset may be biased not as a result of the inclusion of individual samples, but the overall distribution of all included samples. Detormining which mcasures and motrics are relevant in largo-scale dataset analyses is an open challenge (Cho & Lee 2021p. A further open quest ion concerns the information necessary for applying these metrics when evaluating datasets, extending the work on model and data cards by Mitchell et al. (2022b) and Pushkarna et al. (2022).
Developing search tools for large datasets. While there exists some work to quantitatively analyze dataset attributes (Mitchell et al., 2022b), this methodology generally "does not adapt well to web-scale corpora" (Piktus et al. 2023). One initial attempt to close this gap and to provide infrastructure for the quantitative and qualitative analysis of large datasets the ROOTS Search Tool (Piktus al. 2023). However, at the moment, ROOTS is limited to the 1.6TB corpus used to train BLOOM, and does not support other large-scale datasets. Extending this tool or creating similar tools for other open-access datasets could
9See for example, FineWeb Penedo al. 2024 a dataset of English text totaling terabytes.
11

---

Published in Transactions on Machine Learning Research (04/2025)
help with large-scale data governance. A related effort by Elazar et al. (002)) has provided an example of such an extension.
3.1.3 Attribution of Model Behavior to Data
Molivation: As noted above, one way in which training data may be problematic is if it. causes undesirable downstream effects models trained on that data, such as reproducing false information. In order to identify samples that should be excluded from training datasets for this reason, it may be necessary to understand how dataset composition can affect model performance. Thus, complementary open issue to identifying problematic data is attributing model behavior to specific data points. Insights could be applied to different datasets used during development. For example, while pretraining data forms the basis for overall model performance (see e.g. Blakeney et al.) 2024), preference data used during fine-tuning has a larger impact on the extent to which the resulting model is aligned with users' interests.
Open problems:
Understanding how pretraining data affects model behavior. Due to the size and complexity of contemporary Al systems, collective understanding of how specific training samples contribute to prob-
lematic model behavior is incomplete (Lin et al.) 2022a Siddiqui et al. 2022). Udandarao et al. (2024) have, for example, investigated how pretraining concept frequency impacts downstream performance. Others have studied how upsampling Komain-specific data relative to generic web-scraped text affects model performance
BBlkeney et al. 2024).
Understanding properties and effect of preference data fine-tuned models. In addition to pretraining data, future work could aim to understand the effects of the preference data used to fine-tune models, whether this is via a reward model, constitution, or another representation of preferences. It may be relevant to ensure that preference data has certain properties, such as representativeness, diversity, or neutrality, and design evaluations for the quality of this data. Related work in this context is research on preference aggregation for fine-tuning language models (Mostalazadeh Davani et al. 2022 Siththaranjan
et al. 2024 Barnett et al. 2023) and scaling laws for reward model over-optimizatiinn (for example, Gao
et al. 2023a
Understanding the impact of synthetic data. Model developers may face data scarcity current AI systems require increasing amounts of training data (Villalobos et al. 2022). Synthetic data generation offers an alternative to creating new authentic data, which can be both tine-consuming and resource intensive. However, the impacts of training on synthotic data on model behavior are not yet well understood (Guo
et al. 2023 Aleeohhmmaa et al. 2023 Martinez et al.] 2023: Shumailov et al. 2023 Gerstgrasser el al. 2024). Questions remain on the effect that different strategies of training on synthetic data can have on model performance and bias, given that synthetic data has been shown to lack representativeness and insufficiently reflect imperfections of real-world data (Hao et al.) 2024). Future research could aim to provide greater clarity regarding the effect of training on synthetic data, or develop uses of synthetic data for promoting model safety and reducing bias.
Balancing tractability and accuracy in data attribution. One key challenge in data attribution is the trade-off between computational tractability and accuracy (see, for example, Ghorbani & Zou. 2019 Jia
ct al. 2019 lyas et al., 2022 Akyürek et al. 2022), such as when using inffuence functions to attributc behaviors data examples (Basu et al. 2020 Grosse et. al. 2023 Choe al., 2024). Park al. (2023b) aim
to overcome this trade-off by introducing TRAK (Tracing with the Randomly projected After Kernel) but far this work has only been tested for small foundation models. Due to TRAK's methodology requiring the training of multiple model versions for different subsets of the training set, it is unlikely that such method would scale to the largest models, creating an avenue for future work.
12

---

Published in Transactions on Machine Learning Research (04/2025)
3.2 Compute
Example Research Questions
13. What hardware properties or chip specifications are most indicative of suitability for Al training and/or inference? How does this differ from other scientific, business, or casual uses of high-end hardware? 332.11
14. How efficiently can Al models be trained using a large number of small compute clusters? (32.11
15. How decentralized training attempts be identified? 3.2.1
16. Can large training runs be detected while retaining developer privacy, for example through iden- tifying signatures in processor utilization? (3.2.2)
17. Can compute workloads be reliably classified as either training, inference, or non-Al-related, for example throngh identifying signatures in processor utilization? 3.2.2
3.2.1 Definition of Chip and Cluster Specifications for Model Training
Compule governance has been proposed as lever for governing advanced AI systems, due to the large amounts of computing resources required for their training and deployment (Sastry et al.) 2024). However, despite its potential efficacy, compute governance is a blunt instrument, with previous actions including the restriction of the sale of a broad range of high-end chips to China (Bureau of Industry and Security 2022b 2024). It would thus be beneficial to be able to limit compute governance interventions only the chips compute clusters that are most relevant for developing and deploying Al systems of interest to policymakers. Ensuring that compute governance is targeted only where needed will require thoughtful derivation of metries and specifications that capture hardware of concern, while excluding the vast majority of computing resources that are not used for industrial-scale Al.
Motivation:
Open Problems:
Asscssing the effect of chip specifications on AI workload suitability. One open issuc under- standing how different chip specifications such as throughput, memory bandwidth, memory capacity, and interconnect bandwidth affect chip's suitability for different Al workloads, or the suitability of cluster made up of those chips. Previous regulations concerning chips have arguably contained loopholes due to limitations in their technical specifications. For example, NVIDIA's A800, "while compliant with October 7 controls, is still capable of performing complex Al lasks, and was a top seller in China before the updated October 17, 2023 controls." (Reinsch et al. 2023
Understanding the implications of decentralized training and cluster size. A related sub problem is understanding decentraaized training, specifically the question of how efficiently AI models can be trained using multiple geographically disparate compute clusters. There is substantial technical work developing decentralized training methods (see, for example, Douillard et al., 2024). Another open problem is the efficiency and cost impact of using larger number of less powerful chips within cluster, as opposed to using a smaller number of more powerful chips totaling the same theoretical throughput, sometimes known as slicing.
3.2.2 Classification of Workloads
Motivation: In addition to classifying hardware, it is also useful to classify computational workloads in order to identily potentially concerning or anomalous workloads. For example, Executive Order 14110 requires the reporting of training runs above particular compute threshold (The White House 2023a) Being able to classify workloads while preserving customer privacy could assist with such reporting, as well as range of other governance goals, such as identifying compute usage trends, and audit trails for the development of Shavit 2023).
powerful models (Heim et al. 2024
13

---

Published in Transactions on Machine Learning Research (04/2025)
Open Problems:
srivacy-preserving workload classification. Compute providers, such as data center operators and cloud computing firms, typically already collect & wide variety of high-level data on customers and work- loads (Heim et al. 2024/. An open question is thus whether is possible to use this data to develop reliable workload classification techniques, for example, determining whether a training workload exceeds certain compute thresholds, whether an inference workload involves malicious cyberactivity (Commerce Depart- ment 2024). Such techniques would need to account for changes in the hardware, software packages, and specific algorithms used in Al workloads over time.
Ensuring workload classification techniques are robust to adversarial gaming. Adversarial com- pule customers may try to obfuscate their activities to avoid workload classification, by introducing noise in the way computational resources are used, or breaking up workloads across multiple cloud accounts, providers, or computing clusters. Designing workload classification approaches that are robust to this kind of gaming, or otherwise are able to detect when this kind of gaming occurring, open challenge (Egan)
& Heim 2023
Heim et al. 2024
3.3 Models and Algorithms
Example Research Questions
18. How the thoroughness of evaluations be measured? 333.1
19. How can potential blind spots evaluations be identified? (33.1
20. How can potential data contamination be accounted for when conducting evaluations? 3.3.1
21. How can mechanistic analysis of model internals, such as weights, activations and loss landscapes on particular data, be used to improve understanding of model's capabilities, limitations and weaknessos? (3.3.1
22. How generalizable are mechanistic analyses across models? 3.3.1
23. How can methods for red-teaming models be scaled and/or automated? 3.3.2
24. How can the capabilities and risks AT agents be evaluated? (33.3
25. How can the capabilities and risks of networks of multiple interacting AI agents be evaluated?
3.3.3
3.3.1 Reliable Evaluations
Motivation: A great deal of research is focused on evaluating AI models measure their performance and identify capabilities and failure modes. including by state actors (Department for Science, Innovation &
Technology & Al Safety Institute 2024
UK Al Safety Institute, 2024 Anthropic (022b). Yet, state-of-
the-art Al systems can exhibit umpredicted downstream capabilities that often evade evaluations (Shayegani et al. 2023 Carlini et al. 2023b Schaeffer et al. 2024). The rapid advancement and widespread deployment
of AT systems has prompted major regions, including the European Union, China, and the United States, to put forward requirements for evaluating, reporting, and mitigating risks associated with these systems (Reuel et al. 2 For instance, Article 55 of the EU AI Act mandates that providers of general-purpose Al models with systemic risk perform model evaluations using standardized protocols and tools, including adversarial testing, to identify and mitigate systemic risks (Council of the Europcan Union 2024). Despite jurisdictions mandating capability evaluations across various risk areas, there a lack of technical clarily on how to perform these assessments comprehensively and reliably (Chang et al. 2024 Zhou et al. 2023a and for some risks such evaluations simply do not yet exist (Weidinger al. 2023): In addition, evaluations for
systems in high-stakes settings will likely demand higher level of confidence than other applications, but it is unclear how to determine the required level of rigor based
case.
14

---

Published in Transactions on Machine Learning Research (04/2025)
Open Problems:
Ensuring sufficient testing. Determining whether an evaluation procedure has identified all, if not most, of the vulnerabilities of a system is an open problem. This is especially relevant if the evaluation pertains to capabilities, such as deception and Long-horizon planning, that could enable harmful forms of misuse or
make systems hard to oversee or control (Park et al.] 2024 Shevlane et al.] 2023 Hendrycks et al.] 2023: Kinniment et al., 2023: Li et al. 2024b Phuong et al. 2024 Bengio al. 2024). This issue is most acute in benchmarks test cases (Liang et al. 2023 Srivastava et al. 2023 Gao et al. 2023b; Wang et al.) 2023a Lee et al.) 2023b Biderman et al, 2024). as such evaluations can fail to fully inform evaluators about a model's capabilities, with particular uncertainty around capabilities that the model may lack (Casper et al. 2024a). Specifically, if a model does not exhibit a particular behavior during testing, it is challenging to determine whether this is due to the model genuinely lacking the underlying ability or if the evaluation method applied was insufficient to surface it (Wei ct al., 2022 Zhu et al., 2023a), for example, due to prompt sensitivity (Zhu et al. [023a] Sclar et al., 2023). This ambiguity can lead to an incomplete understanding of a model's 's true capabilities and potential
the case behavioral evaluations that directly test models' performance
risks Barrett et al. 2023 Schaeffer et al. 2023: Raji et al. 2021p, and means that behavioral evaluations
can only offer a lower bound on a model's potential to exhibit harmful behavior (Gool et al. 2024 Casper
ct al. 2024b). This motivates research into how to expand the scope of behavioral evaluations as well as
how to estimate the robustness and extensiveness of existing evaluations (Chan 2024 10
Improving evaluations using mechanistic analysis. The shortcomings of behavioral evaluation tech-
niques motivate additional work on evaluation methods that do not suffer from the same limitations. One
proposed approach toward this is to study their internal mechanisms (Burns al. 2022 Olah 2023 Carranza
et al. 2023 Casper et al. 2024a Bereska & Gavves 2024 Although evaluations that involve developing
interpretations of a model's internal structure have seen a great deal of interest, they remain largely untested
in practice (although see [Templeton et al.) 2024: Gao et al. 2024). Furthermore, interpreting the internal mechanisms of models incvitably involves reductions in the complexity of the system being studied (Gilpin
et al. 2018).
Testing the validity of evaluations. It can be hard to have high confidence that the results of the evaluations reflect propertics of the model rather than the evaluation methodology employed that is, that the evaluation is internally valid IT Making progress on this could be challenging due to uncertainties pertaining to both evaluations and the models to which they are applied, making it difficult to attribute results to either the evaluation or the model. One potential avenue for progress could be using model organisms- - smaller simpler AI models that have particular properties by construction (for example Hubinger et al. 2024)- - to test whether evaluations for the constructed property are able to reliably detect it. However, this method would likely not be of use if trying to develop evaluations for model properties that are only exhibited by the most capable models. More generally, designing meta-evaluations that assess the reliability and consistency of evaluation methodologies across different models and contexts remains an open research challenge.
Establishing a causal relationship between procedural design choices and system characteris- tics. Similar to attributing a model's properties to characteristics of its training data (see Section 3.1.3), . it may be possible to attribute behavior or performance to design decisions made during a model's development process (see, for example, [Simson et al., 2024).. Establishing causal relationships between such decisions and resulting system properties may allow for greater standardization of development best practices.
Understanding potential risks and capabilitics of future AI systems. Finally, it is also difficult to study certain risk scenarios that might emerge through advanced capabilities for example, the ability to plan over a long horizon- - due to their hypotaetical nature. It could be useful to develop and study specific demonstrations of harmful behavior and the effectiveness of current safety techniques against these behaviors
Beyond insufficient model-level testing, evaluations along the whole AI life cycle lacking. We address this problem separately in Section 3.4.
Internal validity refers the extent which evaluation accurately measures what it intends to mcasure within its specific context, while external validity refers to how well the evaluation results can be generalized or applied to other situations, populations, or contexts beyond the original study.
15

---

Published in Transactions on Machine Learning Research (04/2025)
in current AI systems, similar to the model organisms approach suggested in the previous paragraph (Scheurer et al. 2024 Järviniemi & Hubinger 2024 Hubinger et. al. 2024).
3.3.2 Efficient Evaluations
Ideally, would be possible to test an AT system under all possible inputs to ensure that it would not produce harmful output for any of them. However, performing a brute-force search over possible inputs is intractable due to the astronomically large input spaces for modern systems and the fact that whether output is harmful may be unclear context-dependent. As result, current evaluation methods manually apply heuristics to guide vulnerability searches towards regions of the input space assumed to be more concerning as observed in voluntary audits by developers (OpenAl et al. 2024 Kinniment et al. 2023 Touvron et al. 2023 Anthropic 2023b). However, manual attacks quickly become impractical, expensive, and insufficient for conducting scalable evaluations (Ganguli et al.) 2022), especially when searching across modalities and languages (Ustün ct al. 2024).. This problem is exacorbated for increasingly capable, general- purpose systems that have a siguilicantly larger attack surface than narrower systems. The development of more efficient automated approaches for identifying model vulnerabilities will be crucial if results from model evaluations are to be applied inputs to governance-relevant decisions.
Motivation:
Open Problems:
Making comprehensive red-teaming less resource-intensive. Recent progress has been made on
automated red-teaming by using generative AI models to produce test cases (Perez et al. 2023 Shah et al.
2023), develop adversarial prompts (Deng et al.
2022
Perez et al.
2022
Mehrabi et al. 2023
Hubinger
et al. 2024 Casper al.] 2023 Hong et al. 2024].
and automatically evaluate
the outputs of other models
[Zheng et al. 2023 Chiang et al. 2023
Ye et al.
2023
Kim et al.
2023
Chao et al.
2024 Souly et al.
2024). Furthermore, automated search methods have been applied to find
adversarial attacks as a way of
enhancing or replacing manual methods /Wallace et al.
2019
Song et al.
2020
Shin et al.
2020
Guo et al.
2021 Shi et al. 2022 Kumar et al.) 2022 Wenn et al.
2023
Jones ct al.
2023
Zou et al.
2023b Liu et al.
2023b Zhu el al. 2023b Andriushchenko el al. 2024).
However, despite these approaches for automating
evaluations, thorough red-teaming remains labor-intensive process, and many failure modes still evade
rod-tcaming efforts (Shaycgani et al., 2023 Carlini et al. 2023b Longpre et al. 2024a). Those challenges in
part stem from how existing automated techniques are often computationally expensive and crude, requiring large degree of guidance from human engineers (Mazeika et al. 2024). Qualitatively different approaches that automate some or all the red-teaming process perhaps through the use of agentic Al systems that can plan, use tools, and dynamically evaluate systems - couldd also allow for more scalable evaluation.
3.3.3 (Multi-)Agent Evaluations
Motivation: Agentic Al systems are generally characterized by an ability to accomplish tasks from high- level specifications, directly influence the world, take goal-directed actions, and perform long-terni planning (Chan et al. 2023b Durante et al.; 2024 Huang et al.; 2024a). These capabilities could allow agentic systems to perform tasks with little human involvement and control. While economically uscful, for example, as customized, personal assistants, or for autonomously managing complex supply chains, agentic systems could pose unique risks due to their ability to directly act in the world, potentially with deffcul-tto-rediie impacts (Chan et al. 2023b
Lazar 2024
Gabriel et al. 2024 Bengio et al., 2024
Open Problems:
Evaluating and monitoring agentic systems. User customizability, such as through prompting the integration of new tools, makes it particularly difficult to foresce the use cases and potential risks of agents (Shavit et. al., 2023 Kolt 2024: Cohen et al. 2024a), motivating polential measures for tracking and monitoring their actions (Chan et al. 2024). Furthermore, evaluating agents a nascent field with significant challenges existing agent benchmarks often don't have adequate holdout datasets, causing existing agents to game and overfit to the benchmark, which in turn results in unreliable evaluations of these systems
12For example, there are vastly more possible 20-token strings of text 10 10 pixel images than there particles in the observable universe. (One current estimate for the number of particles in the obscrvable universe is 1000. CPT-2's tokenizer had vocabulary size of 50, 257, meaning that there are approximately 50, 000 10 unique strings 20 tokens. There are 256100 10240 possible gray-scale images with integer pixel values the range [0. 255].)
16

---

Published in Transactions on Machine Learning Research (04/2025)
[Kapoor et al. 2024b) Similar to non-agent benchmarks (see above), best practices are currently lacking, leading to inconsistencies across evaluations and limiting their reproducibility (Kapoor et al. 2024b). Thus, future work could aim to introduce best practices for evaluating agentic systems.
Expanding limited multi-agent evaluations. On top the difficulties of studying single-agent systems, multi-agent interactions add an additional layer of complexity due to information asymmetries, destabilizing dynamics, and difficulties in forming trust and establishing security. These problems can lead to unique complexity and failure modes (Hammond et al., 2025; Chan et al. 2023a Akata et al. 2023 Mukobi et al., 2023). In addition, it may be the case that collectives of agents exhibit unpredicabble capabilities goals 2025). If AI agents become incrcasingly
not attributable to any one agent in isolation (Hammond et al. embedded in real-world services, such as in finance or the use of web services, it will be relevant to understand such multi-agent dynamics.
Attributing downstream impact individual agents. For issues of liability, it will be critical to be able determine which as) or system(s) can be held responsible for a particular decision action, if any. This may be complicated for cases in which the cause is not solely attributable to single Al agent [3 Having methods of tracing multi-agent interactions and determining the cause of particular outcome could help to solve this problem. An open technical question in this context regards techniques for monitoring individual agents' contributions to multi-agent systems, order to ease attribution of responsibility (Friedenberg & Halpern 2019
3.4 Deployment
Example Research Questions
26.
How can the downstream societal impacts of Al systems be predicted and/or determined? 3.4.1
27. How downstream impact evaluations be scaled across languages and modalities? (3.4.1
28.
How can benchmarks be designed in a way that ensures construct validity and/or ecological valid- ity? 3.4.1
29. How can dynamic simulation environments be designed to better reflect real-world environments?
3.4.1
3.4.1
Downstream Impact Evaluations
The performance of models in isolation is an imperfect proxy for the impact that Al systems will have in everyday use. Thus, comprehensively understanding the impacts that AI could have society demands robust methods for evaluating systems in dynamic, real-world settings (lbrahim et al. 2024 Having such methods would allow policymakers a higher-fidelity picture of where governance intervention might be necessary in order to address potential harms.
Motivation:
Open Problems:
Predicting the downstream socictal impacts of AI systems. Although understanding overall soci- etal impact is overarching goal of much work evaluating AT, it diffieult, large-scale sociotechnical
problem (Dolata et al. 2022 Solaiman et al. 2023 Rakova & Dobbel 2023 Weidinger et al. 2023 Dobbe &
Wolters
2024
Bengio et al. 2024). For example, simplified technical proxies for complex concepts like fair-
Wes8 and equily insulficiently measure the disparate impact of Al systems on diverse communities (Bodggeu et al. 2020 Selbst 2021p. Tt may also be logistically challenging to conduct. downstream evaluations. For example, while it is known that large language models exhibit significant cross- bosss-lingull differences in safety and capabilities (Yong al.) 2023 Wang et al. 2023c Jin et al. 2023 Üstün et al. 2024), it expensive and tine-consuming thoroughly evaluate the cross-lingual properties of models due the required coordi- nation between speakers of many languages. Ultimately, thoroughly assossing downstream societal impacts
13gee, for example, Wex Definitions Team 2023
17

---

Published in Transactions on Machine Learning Research (04/2025)
requires nuanced analysis, interdiisciplinaityy, and inchusion (aaerry & Rubinov
2019
Bengio al. 2024
While recent work has taxonomized societal impacts and provided an overview of early techniques for their evaluation (Moss et al. 2021 Shelby et al.) 2022 Raghavan 2023 Solaiman et al. 2023 Weidinger et al, 2024. there remains a lack of structured, effective methods to quantify and analyze these impacts.
2023
Ensuring construct validity of evaluations. Tt can be difficult establish confidence that the proxy used in an evaluation or benchmark accurately captures the concept aims to measure that is, its construct validity (Raji ct al., 2021 Bowman & Dahl 2021 Hutchinson et al., 2022 Subramonian et al., 2023 McIntosh et al., 2024]. For example, while MMLU [Hendrycks et al.) (021)) claims to assess model's understanding and memorization of knowledge from pretraining through the proxy of performance on question answering, it is unclear how well the ability to accurately answer questions serves as an indicator for understanding, Future research could aim to evaluate the construct validity of current benchmarks as well as ensure construct validity for AT system evaluations, perhaps taking inspiration from prior work in psychology (sce, for example, Smith 2005)). ).
(Westen & Rosenthal 2003 Strauss & Smith 2009
Ensuring ecological validity of evaluations. In addition to construct validity, establishing the ecological validity of benchmarks is an open challenge 14
Current benchmarks tend to be biased towards easily- quantifiable and model-only metrics, potentially making them ill-suited for predicting how well models perform when deployed in real-world settings (Ouyang et al.) 2023 Lee et al.] 2023a).. Future work could aim to assess the correlation between performance on benchmarks and downstream performance, as well as propose evaluation methods for which this correlation is tighter. Another consideration is that benchmarks used for assessing capabilities are oftentimes used to guide development of models, limiting the extent which such benchmarks can be scen unbiased (Marie al.] 2021 Dehghani et al.. 2021 Madaan et al. 2024 Salaudeen & Hardt 2024), and making it challenging to know how to interpret resulting scores.
Designing dynamic evaluations and real-world simulation environments. Despite many user in- teractions with AT systems taking place in dynamic, multi-turn environments, current benchmarks only evaluate performance in such settings to a limited degree (lbrahim et al. 2024). By creating dynamic eval- uation frameworks (for example, Dynabench 2023; Park et al.j [022a]], researchers could better assess a system's 's performance, inherent characteristies, and potential risks in more realistic manner compared static test sets. This would require significant investment in infrastructure and tooling, such sophisticated simulated environments tailored to specific domains like hacking, persuasion, or biosecurity. Similarly, exper- iments with human subjects are valuable to understand the risks from human capability increases through Al models (Mouton et al. 2023 but are less common due to their resoure-intensiveness
14 Ecological validity refers to the extent which results from experiments generalize to contexts outside of the testing environment. Compare this to construct soudity which refers the extent which assessment measures the target construct.
18

---

Published in Transactions on Machine Learning Research (04/2025)
4 Access
Many governance actions will likely require third-party access to system components, along with the pro- vision of resources such as compute. As examples, external access will likely be necessary consideration for facilitating third-party audits (Raji et al. 2022b Anderljung et al.
2023b U.S. National Telecommuni- cations and Information Administration, 2024 Mökander et al. 2023 Casper et al., 2024a), evaluations by National Institute of Standards and Technology (NIST)
government AI safety institutes (GOV.UR UK 2023
2024 Department for Science, Innovation and Technology el al. 2023b), D, and independent academic research
(Bucknall & Trager [2023] Kapoor Kapoor et al. 2024a Longpre al. 2024a House of Commons Science, Innovation and Technology Committee, 2024).
There are numerous reasons for why it may be desirable lo enable these functions to be performed by those other than the system developers. Firstly, actions such as evaluation and auditing could benefit from the independence of being conducted by parties external to Al developers, so that developers do not "mark [grade] their own homework" (Gerken & Rahman-Jones, 2023 Furthermore, AI developers may not have the capacity or incentives conduct research the extent needed for advancing our scientific understanding of Al systems rate comparable that of advances Al development, motivating the need for involvement of the broader academic community. Third-parly access may also allow for broader cultural diversity and representation in the development and governance of AI (Dobbe et al, 2020 Delgado et al. 2023 Held et al.
2023 Crowell 2023). though should not be seen as sufficient measure Chan et al. 2021 Sloane et al.
2022
However, many of the aforementioned actions are precluded due to insufficient external access to relevant system components, espocially the case of the most statc-of-the-art systems (Solaiman) 2023 Bommasani et al. 2023a Bucknall & Trager 2023; Casper al. 2024a). This is often due to concerns regarding developers intellectual property, privacy of data subjects, legal uncertainty, and the safety of the system in question (Seger et al. 2023
Access to compute and other resources peripheral to the systems under consideration, such as training data, is also essential for many auditing and research functions (Ahmed & Wahed 2020 Besiroglu et al., 2024 Ojewale et al.) 2024). The challenges associated with facilitating access to such resources will also need to addressed if these functions are to be fulfilled academia and the public sector.
Access
6-O-6
Models and Algorithms
Data
Compute
Deployment
Privacy-Preserving Third-Party Access Datasets
Provision of Compute Resources
Facilitation of Third-Party Access to Models
Access to Downstream User Logs and Data
Preservation of Evaluation Data Integrity
Figure 3: Open problem areas in the Access capacity, organized by target
19

---

Published in Transactions on Machine Learning Research (04/2025)
4.1
Data
Example Research Questions
30.
How can data access be structured so as to preserve privacy while enabling meaningful auditing?
4.1.1
31. How data access be reconciled with privacy-preservin machine learning?
4.1.1
32. How can openly hosted datasels be prevented from contaminating training data?
4.1.2
33. How can independent evaluation on standardized datasets be facilitated without openly hosting evaluation datasets? 4.1.2
4.1.1
Privacy- Third-Party Access to Datasets
Motivation: Access to training datasets is crucial for enabling external data audits that aim to identify instances of harmful, personal, or inappropriate data being included in datasets TTiel 2023 Birhanc et al. Subramani et al.
2021 2023
2023 Luccioni Viviano 2021, see also Sections 3.1 and 5.1). External
access to datasets could also be instrumentally necessary for facilitating assessment. and research into models
[ucknall & Trager 2023 Ojewale et al., 2024], for example, as research into how the content of training
datasets influence downstream model behavior depends visibility into the training datascts (for example
Cdandarao et al.
2024 see also Section 3.1
However, naively providing unrestricted access to datasets may violate legal bounds, for example by repro- ducing copyrighted data, or otherwise raise security concerns. Additional challenges stem from the lack of legal clarity regarding how to define problematic data, such as what constitutes copyright violation in the context of AI training (Henderson et al. 2023a Quang. 2021). Furthermore, developers may be reluctant to provide unrestricted access to proprietary dalasets due to the high costs associated with collating such datasets 15 the risk of leaking sensitive intellectual property, or, in some cases, risking disclosing having knowingly trained on illegally collected data. These concerns may be able to be alleviated by providing greater transparency into the makeup, contents, and aggregate characteristics of datasets, as well as infor- mation about sources and compilation methodologies, in the absence of unrestricted access to the entire dataset.
Open problems:
Structuring external access to datasets to reduce privacy risks. Research could aim to develop methods for providing sufficiently deep access to datasets, for example, for the purpose of auditing and evaluation, while protecting privacy of data subjects. Existing work in this direction includes Trask et al. (2023) which suggests a framework for allowing third-parties to propose and execute approved queries on AT systems and third-party data without exposing sensitive information beyond the explicitly authorized results. Similarly, Project Oak a software package for providing "security and [..]] transparency around how data is used and by whom, even in a highly distributed system", through the use of trusted execution environments (TEEs) specialized hardware (Project Oak) also Section 6.2 Inspiration can also be taken from other industries, including healthcare (NHS Research SDE Network 2024 For example, AFEL provides a platform for the analysis of patient healthcare data for the purposes of academic research through a combination of pseudonymization, methods for working with data in situ while obfuscating raw patient data, and providing transparency into researchers' use of the platform (Bennett Institute for Applied Data Science 2024). Future research could draw insight from this case to propose methods for safely accessing user data for research into the societal impacts of AI.
Addressing the tension between data access and privacy-presorving machine learning. A signifi- cant body of research has explored how we can train machine learning models when data-owners do not trust
Inncludg the cost of data collection and the resources ensure that licenses are correctly represented avoid incorrect. infringements.
20

---

Published in Transactions on Machine Learning Research (04/2025)
model-trainers for example through federated learning (Kairouz al 2019) and training on encrypted data (Xie et al. 2014; Nandakumar et al. 2019). Tn such cases, it can be challenging to provide data access given that the developer themselves lacks such visibility, Future work could explicitly address this tension, proposing methods that allow for the auditing of data that may have been enerypted during training.
4.1.2 Preservation of Evaluation Data Integrity
Motivation: Current standardized methods for evaluating models often utilize openly-available datasets, with the goal of comparing model performance fkee-fo-like (Reuel et al., forthcoming). At present, this is largely achieved by hosting such datasets openly in online repositories such as HuggingFacd To (see, for example, Hendrycks et al.] 2021 Srivastava et al. 2023 Gao et al. 2023b). However, openly hosting evaluation datasets online risks their inclusion in web-scraped training datasets (Deng et al. 2023), either accidentally, or intentionally as a method for artificially inflating benchmarking results. Such contamination of training data has scrious implications for the efficacy and reliability of these standardized motrics (Oren et al.) 2023
Roberts et al. 2023
Jiang al.] 2024 Zhang el al., 2024
Schaeller 2023
Open problems:
Identifying and mitigating contamination of training datasets. Current approaches to mitigating contamination of training datasets are rudimentary. A potential post-contaminatioon approach is to detect data contamination (Dong et al. 2024b; Golchin & Surdeamu, 2023 and then correct for it when scoring benchmarks. For example, OpenAl and Meta measured which benchmarks' test samples were potentially included in the pretraining data of GPT-4 and Llama 2, respectively, and reported how scores differed between contaminated test samples and non-contaminated test samples (OpenAI et al. 2024 Touvron al. 2023 Zhang et al. (2024) attempted to correct for data contamination by creating version of the benchmark GSMSk that is comparable in terms of tasks, complexity and human solve rates, and reported which models' scores dropped precipitously. Other approaches pertain to the design of benchmarks that are robust against contaminated models by using templates from which variations of a task can be generated (Yu et al. 2023) Srivastava et al. 2024) or by being frequently updated (While et al., 2024]. However, the frequent updating approach is resource intensive, especially if covering a large span of test tasks and fields, and designing templates to support variations of tasks may not be feasible for tasks that don't follow predictable structure. Alternatively, (Srivastava et al. 2023) make use of a canary string, that is, globally unique identifier that is included in all sub repositories of the BTG-bench collection in order to ease ident of these test samples in training datasets. BIG-bench also includes a dedicated training on oon_tst__at ___sttt task, which serves as a "post-hoc diagnosis of whether BIG-bench data was used in model training" (Srivastava et al. 2023) However, use of canary strings depends on any and all copies of the repositories to also include the string, and thus is not robust to negligent users.
Evaluating on private or encrypted evaluation datasets. Alternatively, research could aim to develop ways in which models can be independently evaluated private or encrypted test set. Indeed, some recent benchmarking datascts have only been made available only through a custom evaluation API (see, for example, (Swadda et al.] 2023). Furthermore, popular dataset repositories including HuggingFace, as well as competition platforms such as Kaggle, gate access to evaluation datasets in order to reduce the risk of contaminating training data [17 Finally, Bricman (2023) proposes hashmarks, a "protocol for evaluating language models in the open without having to disclose the correct answers" by cryptographically hashing a benchmark's 's reference solutions before publication. Further work could develop this, and similar protocols for reliably evaluating system capabilities on private evaluation data.
Ihttps: /tt.h//ugggggac..coo/
Hugging Face 2024
kaggle
2024
21

---

Published in Transactions on Machine Learning Research (04/2025)
4.2 Compute
Example Rescarch Questions
34. How can public compute resources be allocated fairly and equitably between users? 4.2.1
35. How can public compute infrastructure be developed in way that ensures interoperability between models and software packages? 4.2.1
36. How can assurance be given that researcher compute provisions are being used for intended and stated purposes? (42.1
4.2.1 Addressing Compute Inequities
Motivation: Compute usage by private companies in training and rumning models has increased exponentially in the past years, and now greatly exceeds the compute resources available for non-industry researchers (Maslej et al. 2024 Besiroglu et al. 2024). While some researchers have found that the majority of academic researchers do not feel primarily constrained by compute access (Musser et al. 2023), others have found that this access inequality is specifically limiting researchers' contribution to frontier research (Ahmed & Wahed, 2020 Besiroglu et al.] 2024 Birhane et al., 2023). To address these concerns, there have been proposals and lunding for public compute infrastructure (NDIF proposal Ho et al. 2021 Organisation for Economic Co- Co-Orpptiioon and Development 2023 National Artificial Intelligence Research Resource Task Force 2023]
UK Research and Innovation, 2023). Though the success of these initiatives primarily depends raising funds to purchase sufficient compute, technical advances could still be instrumental in their success.
Open problems:
Ensuring interoperability of public compute resources. Public compute resources should be com- patible and interoperable with a wide range of models and software packages, in order support the range of rescarch projects that would be conducted. System performance
vary considerably depending the hardware and software which it is run (Nelaturu et al. 2023 Gundersen et al., 2022), and common ML software frameworks can lose more than 40% of their key functionality when ported to non-native hardware [Mince et al. 2023 Future research could aim to propose solutions that address these observed defects.
Ensuring environmental sustainability of public compute resources. The resourcing requirements of large-scale supercomputers and data centers are considerable, both terms of energy (Strubell et al., 2019) and other resources such water, used for cooling (Mytton) (002]). Thus, measures will need to be taken to balance broad access to computing resources with environmental sustainability. The environmental impacts of AT systems and associated open problems is discussed further in Section g
Ensuring public compute is used for intended purposes. System administrators would need to be able to ensure that public compute resources are being used for the stated purposes, rather than malicious or otherwise unintended uses, for example by performing "workload classification" (Heim al. 2024 see Section
3.2 for open problems this context). Such oversight methods would need to preserve end-users' privacy system administrators, as well that of any potential subjects of data used in conducting experiments (see Section 5.2)
Equitably allocating public compute resources. Given the high demand for public compute resources, another issue will be the elficient and fair allocation of processor time between users. While methods for allocating compute resources in more general cases have been explored (Ghodsi et al. 2011 Wang et al. 2015 Xu Yu 2014 Souravlas & Katsavounis 2019 Jebalia al.) 2018). future work could aim to ensure their applicability to this case. Alternatively, research could aim to find Al-specific optimizations for allocating
compute among diverse users.
22

---

Published in Transactions on Machine Learning Research (04/2025)
4.3 Models and Algorithms
Example Research Questions
37.
What research and auditing methodologies are possible given a range of forms of access on the continuum between black- and white-box access? (4.3.1
38. How do different forms of access affect potential risks of misuse of models? 4.3.1
39. How do different forms of access on the continuum between black- and white- box access affect the risk model theft or duplication? (43.11
40. How can model access requirements for research and auditing be reconciled with commercial and/or safety concerns? (43.11
4.3.1 Facilitation of Third Third-Party Access to Models
Motivation: A fundamental requirement of conducting external research and evaluation of Al systems is having access to the underlying models. However, many systems not released openly (Solaiman 2028 and, while access requirements vary widely for different external actors, current APIs do not offer sufficient depth or flexibility of access to facilitate many actions important for research and evaluation (Bucknall & Trager 2023; Casper et al. 2024a Longpre et al. 2024a). For example, Casper et al. (2024a) argue that evaluations conducted with solely black-box access can "produce misleading results" and only offer "limited insights to help address failures" due their not revcaling complete information regarding the nature of discovered flaws. It challenge to find the balance between providing external parties with sufficient access for conducting independent research and evaluation, while addressing developers' concerns such as IP theft misuse of their models. While there are certainly social and legal pathways that can be pursued towards this end, there are also several technical avenues CCasper et al. 2024a
Open problems:
Illuminating the continuum between black- and white-box access. Greater clarity regarding how much, and what kinds of, research can be conducted with different depth and breadth of access would be helpful for navigating the trade off between access and socurity (Bucknall & Trager 2023 Different auditing procedures can demand varied levels of access, motivating the need for a range of methods for supporting researchers, rather than prescribing set approach (Casper et al. 2024a). On the flip side, a clearer picture of how differing forms of access bear on developers' security and privacy concerns would also be needed. Black- box access already allows for the training of distilled models that can then be used to generate effective adversarial atlacks against production models via transfer (Zou el. al. 2023b), and fine-tuning APTs can be ineffective at guarding against the removal of pre-deployment safety measures (Qi et al., (022). Meanwhile, the ability to view language model output logits has been shown to be sufficient for extracting proprictary system information, including the model's hidden dimension, though it is unclear the extent to which this is a icactical threat (Carlini et al. 2024). Further research could aim to elucidate how the provision of intermediate forms of grey-box access could exacerbate these existing vulnerabilities when compared to the bascline of black-box access.
Applying technical measures to address vulnerabílities of greater access provisions. Outstanding technical questions regarding external model access include whether the use of technical tools, such privacy-enhancing technologies or TEEs, could enable near-white-oos auditing and research while addressing commercial and safety concerns. For example, Aarne et al. (2024) describe how approach combining multi- party computing with TEEs "could be used by third-party evaluator to run tests on an AI model without ever having direct access to the unencrypted weights." Future research could assess the exlent to which such solutions allow model providers, auditors, and regulators to interact in a way that is: easy to set up for the model provider; lcaks no model information to the auditor; leaks no audit information to the model provider; and does not compromise the cybersecurity of the model provider. Alternatively, work could aim to incorporate approaches for providing third-party model access at scale into secure and trusted compute
23

---

Published in Transactions on Machine Learning Research (04/2025)
To date, there has been preliminary work on protocols for secure privileged evaluations (Trask et al. 2023 though there is lack of existing applications or established best practices.
clusters, including public compute resources (Anderljugg et al. 2022 Heim 2024
Ensuring version stability and bockaaird of hosted models. Large commercial models are frequently and continually updated during deployment, with prior versions often being replaced without notice or knowledge. However, reproducibility and replicability of independent research conducted on pro- prietary models depends stable and continued access models, even after their being succeeded by newer versions (Pozzobon et al., 2023 Bucknall & Trager 2023 Biderman et al. 2024 Maintaining hardware and software compatibility (Mince et al.) (2023) may be necessary in order to be able to provide access to discon- tinued systems upon request. Future work could lay out best practices for documenting and communicating when models are being deprecated or discontinued, in order to ease reproducibility concerns
4.4 Deployment
Example Research Questions
41. How can user logs and data be used for downstream impact assessments while preserving the privacy of data subjects? (4..11
42. How could responsibilities for providing user data access be effectively allocated along the AT value chain?
(.4.1)
43. What cryptographic methods can be developed to allow analysis of user interaction data without revealing individual user identities sensitive information? 4.4.1
44. How could secure multi-party computation be leveraged to allow collaborative analysis of user logs across different entities the AI value chain? (44.1)
4.4.1 Access to Downstream User Logs and Data
Motivation: Assessing models post-deployment, as covered in Section 3.4 requires access to relevant real- world data on user interactions with systems (Nicholas] 2024). This data can be used to directly assess aspects of user-model interactions, build evaluations that are more reflective of real-world use, and guide assessments of societal-level patterns in key sectors (lbrahim et al.] 2024).. While there are crowd-sourcing initiatives which allow users voluntarily submit some their interaction data to create research datasets (The Allen Institute for Artificial Intelligence 2024 ShareGPT 2022), to the best of our knowledge, model provider has made their interaction datasets, privacy-preserving metadata about these logs, widely available.
Access to real-world user data, such as usage logs and user feedback records, may be additionally relevant for legal purposes. For example, legal cases may arise when users experience harm as a result of an interaction with an AT system either directly as participant in the interaction of concern, or indirectly as a subject of actions taken following an interaction. In such cases, the availability of information including user logs and audit trails to prosccutors or courts may be relevant for determining the outcome of case.
Open problems:
Addressing user privacy concerns regarding access to user logs. External access to user interaction data must overcome privacy concerns relating to the collection, sharing, and analysis of potentially sensitive and identifying user information. This challenge has parallels other industries, for example in online
platform governance, where the EU's Digital Services Act (Digital Services Act) mandates independent researcher access to platform data (EU Joint Research Centre 2023 Albert 2022). While implementation
challenges remain in the case of the Digital Services Act (Leerssen 2021 Leerssen et al. 2023 Leerssen
2023 Jaursch et al. 2024 Morten et al. 2024), more developed solutions can be seen in the healthcare
18 See Luccioni al.
2022 related work deprecating datasets.
24

---

Published in Transactions on Machine Learning Research (04/2025)
sector (see the discussion of OpenSAFELY in Section 4.1 Inspiration could be taken from these sectors for ensuring user privacy while providing access to user logs for research purposes.
Understanding how access responsibilities may vary along the AI value chain. Additional diffi- culties in providing user data stem from the complexities of the Al value chain (Küspert et al., 2023 for example, the case that a foundation model built. upon and incorporated in a user-facing application by downstream deployer. In this scenario, it is not immediately clear how access requirements interact with the division of information between the foundation model developer and subsequent deployer. An additional challenge may emerge if the provision of user data in this implicitly reveals information about how the deployer is using the model as part of their service, potentially putting their IP at risk of leakage. Further work could clarify potential access responsibilities in this and similar situations.
25

---

Published in Transactions on Machine Learning Research (04/2025)
5 Verification
In many cases, may be beneficial to be able to verify claims regarding AI systems' properties, capabilitios, and salety a way of increasing trust between aclors (Brundage al. 2020 While related assessment, covered in Section verification concerns the process of checking whether AT system "complies with [specific] regulation, requirement, specification, imposed condition" (IEEE. 2011 opposed to evaluating the system's 's performance, capabilities, potential societal impacts. For example, an assessment task could be to uncover details about the data that a given model was trained on. In contrast, a verification problem could be, given dataset and a model, to confirm or refute the claim that the model was trained on the dataset.
There trend towards increasing amount. of regulation and corresponding requirements for model developers, deployers, and users being passed by major national and international jurisdictions (Maslej al, 2024p. It may thus be necessary for model developers and deployers lo verily and allest. to certain properties of their AI systems in order to prove that they comply with regulations. On the flip side, governments may need to be able to verify whether actors the AT ecosystem comply with the regulations, or verify that other countries are in compliance with international rules (Baker 2023
Shavit 2023 Avenhaus et al, 2006
Verification
Models and Algorithms
Data
Compute
Deployment
Verification of Training Data
Verification of Chip Location
Verification of Model Properties Verification of Dynamic Systems Proof of Learning
Veriflable Audits Verification of Al-generated Content
Verification of Compute Workloads
Figure 4: Open problem areas in the Verification capacity, organized by target
5.1
Data
Example Research Questions
45.
How can it be verified that model was (not) trained on a given dataset? 5.1.1
46. How can it be verified that a dataset has certain properties, or (does not) include certain informa- tion? 5.1.1
47. How can membership inference attacks be optimized for large-scale verification of training data in black-box settings? (5.1.
48. How could the verification process for correct use of licensed data in AT model training be formal- ized?
5.1.1
5.1.1
Verification of Training Data
Motivation: Being able to verify the data on which a given model was trained cither as model developer or third-party auditor could aid in demonstrating compliance with data handling standards and regulation, including the Blueprint for AT Bill of Right, the EU's General Data Protection Regulation (The White
26

---

Published in Transactions on Machine Learning Research (04/2025)
House Office of Science and Technology Policy 2023 European Commission 2016 In particular, even if it
is possible to demonstrate that dataset does not contain harmful or copyrighted material (see Section (.1). this is insufficient for guaranteeing that a model was not trained on problematic data, as the developer may have used an alternate dataset to the one assessed. Being able to post hoc provide evidence that a model was trained on specific data dataset would help preclude such instances.
Open Problems:
Verifying datasets used to train model. Choi et al. (2023], building Shavit (203)), formalize the troof-of-tminnng-ddlee problem, wherein a prover aims to "prove to a verifier that the resulting target model weights W* are the result of training on data D*," and propose a solution. However, the authors concede that their approach is not robust to all potent attacks, in particular additions of small amounts of harmful data, such as that used for inserting backdoors (Xu et al. 2021). addition, Choi et al.ss protocol is not applicable to models for which data not fully known before training, as in the case of online reinforcement learning. Finally, their protocol requires that the "Prover disclose confidential information to the Verifier, including training data, model weights, and code" (Choi et al. 2023), which may create IP and security concerns. An alternate approach to verifying training data could be the application of membership inference attacks which aim to infer whether a given data point was contained in given model's training data a black-box set setting (Shokri et al. 2017 Duan et al. 2024a Wei et al. 2024). Future research could aim to suggest robust methods for verifying training data, assess the robustness of existing methods, or address the parallel issue of verifying that a given dataset (or sample) was not used in the training of a given model.
Verifying fair data use. Verifying copyright compliance and fair data use is complex challenge that may require additional legal and technical frameworks (see Section 3.1) to extend or replace the Proof-of- Training-Dae Data approach (Choi et al., 2023). Open challenges include formalizing the verification of correct use of licensed data, as well as verifying the exclusion of specific licensed data from training sets, should the respective license not allow the data to used for training a model.
5.2 Compute
Example Research Questions
49. How can the location of AT hardware be verified? 5.2.1
50. How can on-chip geolocation mechanisms be made robust to existing GPS spoofing methods?
5.2.1
51. Can TEEs be used to robustly attest to the identity of the specific chip, or the data that. is processing? (5.2.2)
52. What methods can be used to verify compute usage without the use of TEFs? 5.2.2
53. How can TEEs and their applications be designed in a way that limits their potential for misuse, for example through unnecesarrily-rroas surveillance? 5.2.2
54. How the computational overhead of verification mechanisms be reduced to level that enables application across large compute clusters? 5.2.2
5.2.1 Verification of Chip Location
Motivation: High-end data center AI chips are the subject of U.S. export controls, but are at present straightforward to smuggle (Huang. 2024 Harithas 2024 Fist & Grunewald, 2023). One key technical problem for enforcement that it's not currently possible to know the location or owner of a chip after it has been exported. Being able verify a chip's location could also help users of cloud computing validate that their (or their customers") data is being processed in accordance with local data processing laws.
27

---

Published in Transactions on Machine Learning Research (04/2025)
Open Problems:
Verifying chip location. Accurately detecting chip's location remains an open challenge. One approach could be to measure verifiable latencies between the chip in question and a network of trusted servers (Aarne jet al.) 2024 Brass & Aarne 2024). Other methods could be valuable for verifying that a large number of chips are co-located in a single data center, and have not been resold or dispersed. Tt may be possible to verify this using a proof of work challenge that would require large number of co-located systems to complete in the allotted time (Jakobsson & Juels, 1999). Alternatively, chips could more directly verify their identities to each other through mutual attestation (ETTF Datatracker 2024). It is worth noting that non-technical solutions for verifying chip location, involving the physical inspection of data centers, have
also been proposed (Shavit
2023
Baker 2023).
Designing hard-to-spoof chip IDs. Secure IDs for chips (either software- or hardware- could help with traccability and preventing unauthorized use. For example, unique identifiers and information may be engraved during routine water labrication flows (MULTIBEAM 2024), ; or physical unclonable functions devices that "[exploit] inherent randomness introduced during manufacturing to give physical entity a unique 'fingerpritt' or trust anchor" (Gao et al. 2020) may be added. Open questions include understanding the security of these approaches, as well as their feasibility, given potential trade offs between usability, security, and effects on chip performance.
5.2.2 Verification of Compute Workloads
Motivation: It could be valuable for AI developers and deployers to be able to reliably verify their compute usage, for example, which chips were used to train their models and for how long. Likewise, chip owners, such as cloud providers, may want to demonstrate which models were trained evidence that their compute is not being used for unreported large-scale training runs 19] Such mechanisms may be of particular utility in international and extra-territorial situations in which levels of trust between verifier and prover may be limited. Any verification scheme would neod to uphold privacy of both user data and intellectual property.
their compute, as to provide
We note that this is one area of research that itself is dual-use. While potentially beneficial in the above example cases, some implementations of such mechanisms could enable far-reaching control and monitoring of AT chips. However, careful design of these mechanisms can limit the scope of powers given to regulator, for example by only requiring data flows for voluntary verification, rather than remote monitoring or control.
Open Problems:
Verifying properties of workloads using TEEs. may be possible to use TEEs (or other hardware security technologies) attest to the exact program code and model being run (Chen et al. 2019 see also Section 6.2). For example, TEEs could, given inputs along with function to evaluate signature that accompanies the output from the computation attesting to the computation being run intended. Alternatively, the chip could also return a hash of the inputs and outputs of the computation. This would allow the prover to keep the inputs and outputs private, instead proving ownership by demonstrating that they can generate the hash returned by the chip during computation. A verifier could also use a TEE to confidentially run arbitrary tests on the model weights or other data, Despite promising theoretical possibilitics, further work will be required to be able to implement the above in practice. In particular, firmware to implement the above solution on existing hardware is lacking, and morover, may need to be unusually secure. Advancements will also be needed if solutions such as the above are to be feasible at the largest scales, without introducing prohibitive overhead costs.
them, return
Verifying properties of workloads with a trusted neutral cluster. TETES are unavailable or imprac- tical, another approach could be to save hashed snapshots of neural network weights during training along with information about the training run being conducted that is, a braining transcript. This information may then be used to verify that the training transcript provided would have resulted in the weights, with the use of trusted neutral cluster (Shavit 2023). Current challenges to implementing this procodure include
19 Reporting of training runs above the threshold of 10"26 floating-point operations (FLOP) required, for example, by the US Executive Order Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence (Executive Office of the President 2023
28

---

Published in Transactions on Machine Learning Research (04/2025)
difficulties in accounting for randomness in training procedures, building sufficiently trustworthy neutral clusters, and finding efficient methods for proving the authenticity of training transcripis that scale to the largest models (havvi 2023
Verifying compute usage of large, non-AI workloads. Owners or users of large clusters may wish to demonstrate that their clusters were used for a large, non-AT workload (for example, climate simulations). as such use would not fall under the purview of Al regulation. One approach is workload classification, discussed above. There may also be viable computational approaches to verification, potentially including analogues to proof-of of-lennin methods (Jia et al. 2021), that could be explored in future research.
5.3 Models and Algorithms
Example Research Questions
55.
How can model properties be verified with full access to the model? 5.3.1
56. How can the risk associated with given context, query, and AI response be assessed in order obtain assurances about the system's compliance with safety requirements? (5.3.1
57. What should constitute the lower bar for tracking updates to models, for example in a model registry? 5.3.2
58. Could proof-of-learning be used to demonstrate and verify model ownership? 5.3.3
59. How can proof-of-lerring mechanisms be made robust to adversarial spoofing? 5.3.3
5.3.1
Verification of Model Properties
Motivation: In order for system developers or deployers to demonstrate compliance with regulatory require- ments, it. may be necessary prove claims regarding model properties and information. Verifiable properties could include model architecture, training procedures, or performance metries, enabling developers for- mally demonstrate compliance with any mandated technical specifications.
Open Problems:
Verifying claimed capabilities and performance characteristics with full model access. Model properties could be verified through formal verification methods if the verifier has full access to the model, as the case of the model developer. Such methods aim to mathematically prove that a given system can() respond in particular ways particular inputs (Katz et al. [017bb/a Kuper et al., 2018 Katz ee al.] 2019). For instance, formal verification was used to study the safety of neural networks used for unmanned aircraft collision avoidance (Trfan et al., 2020). However, such methods remain largely untested for advanced Al models (Dalrymple et al.) 2024). In particular, many methods quickly become prohibitively complex when scaled up to contemporary state-of-the-art models. While there exist additional methods for verifying properties such as performance metrics without full access (see Section 5.4.1) research could focus on more efficient methods given full access to the model. Furthermore, verifying properties such as a system's architecture or training procedure remain open questions.
5.3.2 Verification of Dynamic Systems
Motivation: Modern Al systems, such ChatGPT, are not based on static models. Rather, they consist of multiple models and components, for example, mixture-of-experts. input filters, and output filters, that undergo change throughout their life cycle. This poses an oversight challenge due the ever-changing nature of many systems throughout their deployment life cycle. Having a reliable, accessible process for versioning could help to monitor system updates and their impacts.
29

---

Published in Transactions on Machine Learning Research (04/2025)
Open Problems:
Tracking versioning and updates. Key open questions in this context relate to how model versioning and post-deployment modifications should be kept track of, especially for models that undergo frequent updates. One approach could be to have registries that track models over time, however, it's not clear what information should be stored in such registry, nor how the information could be verified. Other approaches that can be useful learning (Gilbert et al., 2023), ecosystem graphs (Bommasani et al. 2023c of foundation models (Xu et al 2024
starting point to verify dynamic models include reward reports for reinforcement instructional fingerprinting
5.3.3 Proof-of-Learning
Motivation: In the current landscape, there is no mechanism for a model developer to prove that they have invested the computational resources required train given model. Such a proof could be used for resolving ownership disputes when models are released or stolen by allowing the doveloper to attest
their having trained the model (Tramèr et al. 2016 Orekondy al. 2018 Jīa al.) 2021). Additionally,
proof-of-learning could aid in defending against accidental malicious corruption of the training process
when performing distributed training across multiple workers (Li ct al. 2014 Jia et al. 2021
Open Problems:
Scalable proof-of-learingg (Jia et al, 2021 were the first to formalize the notion of proof-of-learning for AI models. The authors demonstrated that stochastic gradient descent accumulates "seeret information due to its stochasticity," " which they show be used to construct "a proof-of-learing which demonstrates that party has expended the compute required to obtain set of model parameters correctly" (Jia et al, 2021b. Alternatively. (Goldwasser et al. 2021p develop Probably Approcimalely Correct verification, in which weak verifier interacts with a strong prover to test whether the model trained by the prover has low loss relative to the best possible model, with respect to a given loss function. Scaling these techniques such that they remain practical given the growing training compute budgets of foundation models an open challenge.
Designing adversarially robust proof-of-learningg Since the introduction of proof-of of-rnnnnieng in (Jia et al. 2021), ) subsoquent work has demonstrated its vulnerability to adversarial attacks that is, false proofs that are cheap for adversarial prover to generate (Zhang et al. 2022 Fang al. 2023). In particular, Fang et al. (2023) demonstrate "systemic vulnerabilities of prrof-of-learning and which depend on advances in understanding optimization to be sufficiently addressed. While Choi et al. (2023) suggest protocol to counter these vulnerabilities through memorization- based tests, and fixing the initialization and data order, they only lest their protocol for single attacks and not for composite allacks. Their protocol further only covers language models, and has not been tested for other modalities. Future work could aim assess these claims, and aim to increase the robustness of proof-of-learning to adversaries.
5.4 Deployment
Example Research Questions
60.
How can audit registries be used to provide end-to-end verification along the Al value chain? 5.4.1
61. How should verification information from model registries be presented users? 5.4.1
62. Can zero-knowledge proofs be applied to demonstrate model's compliance with hypothetical mandated criteria, without directly disclosing architectural details? (5..11
63. How can it be verified that the model version on which an evaluation or audit was performed is the same as is deployed? (5..1l
64. How the implementation of safety measures be verified at deployment? 5.4.1
65. How can output watermarking schemes be made robust. to adversarial at attempts at removal? 5.4.2
30

---

Published in Transactions on Machine Learning Research (04/2025)
66. How can metadala watermarking be applied to Al-generated content? (5.4.2)
67. How robust can AI content detectors be expected to be in light of contimuing advances in generative AT? (.4.2)
68. How should Al-generated content detectors handle cases of genuine images that have been modified or edited with Al tools? 5.4.2
5.4.1 Verifiable Audits
Motivation: As discussed in Section 3 external audits and assessment have been proposed as crucial com- ponents of governance regimes (Raji et al. 2022b; Mökander et al. 2023). Being able to attest to an audit's 's process and outcome could establish greater trust between model developers, third- party anditors, and governments by proving compliance with regulatory requirements. Trust could also be established with end-users by enabling them to verify that the model with which they are interacting has been shown to have the properlies claimed by developers (Godinot et al.) 2024), for example in model cards (Mitchell et al. 2019 official communications (for example, [Authropic] 2024ap, or technical papers (Gemini Team et all, OpenAl et al., 2024). Verifying audit results is often made more challenging result of access to models often being restricted due to IP and security concerns South al. 2024).
2023
Open Problems:
Verifying claimed capabilities and performance characteristics without full model access. Pre- liminary work has explored how the application of zero-knowledge proofs to AT systems can enable privacy- preserving verifications of claimed system propertics, as well as confirmation that model weights used for inference match those on which an audit was run (South et al., 2024 Waiwitlikhit et al., 2024 Sun et al., 2024 Yadav et al. 2024).. However, due to the high computational overhead associated with these methods, addressing speed constraints will be necessary if such methods are to be applied to larger models. Current approaches that future work could build on include GPU acceleration (Sun et al. 2024 or proof splitting (outth et. al. 2024
Verifying audit results at inference time. In theory, verified computing such as through TEEs (aabt et al. 2015), zero-knowledge proofs (Fiege et al.) 1987), or secure multi-party computation (Goldreich, 1998) with active security developer could load an inference pipeline into cluster of enclave computers. Upon an auditor concluding their study of the system (for example, based on the approaches outlined in the previous paragraph and in 5,3.1), they could ask the cluster of enclaves to produce a certificate of the pipeline that was evaluated, which is then stored in a public audit registry. In the second stage, a user, when interacting with the secured pipeline, could request a corresponding certificate with each received generation. Using such a method, this consumer could know that the AI pipeline they're receiving generations from is the same pipeline that was previously evaluated to be safe. If any change was made to the pipeline, the certificates would not match, and the user would know that they're receiving generations from pipcline which has not been evaluated. However, given the dynamic nature of current models in use (see Section 5.3.2). changes may occur more frequently than audits of such models, which poses an open challenge to this proposal. Another open problem is that this pipeline requires that all evaluation and inference is done in enclaves and with significant computational overhead, effectively limiting verifiable audits for a few critical systems, and necessitating more scalable structures for verifying audits. It will also be necessary to find way for consumers to be informed of the outputs of this verification in a low-friction way, as in the case of browsers that provide warnings for websites without HTTPS certificates. Finally, it is unclear how secure this method is against attempls to ex ralt model weights by auditors.
could facilitate verifiable audits in two-stage process. In the first step, model
Verifying use of safety measures post-deployment. In safety-critical settings, regulators may want to ensure that safety measures, for example, output filters, are applied to AI models or their outputs (see, for example, Dong et al., 2024a Leslie al. 2024 Welbl et al., 2021). Enforcing this may require methods for auditing systems deployed such domains chock that they do fact have safeguards that meet
20Typically consisting of input pre-processing, model prediction, and post-pracessing of the model's output.
31

---

Published in Transactions on Machine Learning Research (04/2025)
these specifications. An open question is how to enforce that additional filters, classifiers, modifications are attached to models deployed in safety-critical domains.
5.4.2 Verification of Al-generated Content
Motivalion: The ability to distinguish between Al-generated and authentic content may be instrumental verifying the authenticity of information and maintaining public trust information ecosystems. Stipulations for being able to detect Al-generated media are made in several regulatory efforts, for example in Article 50 of the AI Act (Council of the European Union 2024) which are, given the state of the art of detection and verification tools, currently unrealizable (Zhang et al. 2023). Methods for verifying AT-generated content can roughly be divided into ex ante approaches that mark Al-generated content as such by embedding machine- readable watermarks, and ex post methods that aim to classify content as either A Al-generated or not, in the absence of a watermark (Ghosal et al. 2023). In addition, watermarks could potentially be used to verify that A--generated content was created by particular model, improving accountability by facilitating the identification of responsible parties in case of unintended consequences or misuse.
Open Problems:
Developing robust watermarking schemes. Watermarks - signals placed in output content that are im- perceptible to humans, but easily detectable through application of a specific detector algorithm have been proposed as one method for verifying that a particular model generated a given output (Kirchenbauer et al, 2023 Christ et al. 2023; Saberi et al. 2023). However, the level of robustness of watermarks varies bet ween modalities. In particular, the continuous output space of images and audio enables hidden watermarking that is more effective than for text. (Chosal et al., 2023). As such, future work could aim to address the rela- tive lack of robustness of watermarks in the case of Al-generated text (Zhang et al. 2023 Liu et al. 2023a Additionally, research could aim to address the possibility that watermarks are easy to fake for example, having two similar models that produce watermarks that cannot be distinguished (Srinivasan 2024).
Designing robust AI content detectors. While efforts to develop methods for the detection of AI- generated content have seen increased atlention in the last two years (Sadasivan et al.] [0023 Corvi et al. 2023 Berber Sardinha, 2024), these methods have not always held up to independent evaluation (Weber- Wulff et al.) ).23). As generative systems improve, it will be increasingly difficult to develop methods (machine learning-basee will need to be done in order to improve and maintain the efficacy of Al content detectors in light of this continued advancement.
or otherwise) to distinguish their output from genuine media. Continued work
Utilizing verifiable meta-data to identify authentic content. An alternative to identifying AI- generated content could be develop ways for a content creator to verify their content as AT-generatod or authentic by adding verifiable meta-data to it (Jain et al., 2023e Knott et al., 2023). For example, the Coalition for Content Provenance and Authenticity (C2PA) is tackling this issue by developing standards for the certification of the provenance of digital content (22AA 2022). Similar work is being done by the (Content Authenticity Initiativo 2024). This could be useful for AI labs to label content, but also for cre- alors of -og-Ae-geeeeated media to label their authentic content as such using the same standard. However, a significant drawback of this approach is that it is not robust to adversaries, as meta-data can easily be stripped from the content limitation which future rescarch could aim to address.
Verifying authentic content modified using AI. Complications arise when going beyond the binary distinction of Al-generated content the one hand, and Human-generated on the other. For example, it is currently unclcar how AI content detectors should respond to authentic images that have been modified using generative AI tools. Future work could aim to assess the suitability of Al-content generation tools for detecting such cases, or design detectors that are able to distinguish between AT-generated, AT-modified, and authentic content.
32

---

Published in Transactions on Machine Learning Research (04/2025)
6 Security
In this section, consider security in the context of AT governance, which aims to ensure that unauthorized actors are not able to access systems and infrastructure not intended for their use, nor use systems for malicious purposes. Being able to give security guarantees across system components could be helpful for a number of reasons. Increased security can strengthen a wide array of governance actions through reducing the risk of regulatory requirements being subverted. For example, comprehensive security measures can protect the ianit of training data, ensuring that. AT systems developed using sensitive personal information remain in compliance with data protection laws.
It should be noted that security is one of the areas of this report that comes closest to topics within AI safety, As such, many of the topics discussed below under the unbrella of TATG could also be viewed through the framing of improving AI safety, or otherwisc be closely related to topics that can. However, due to the reasons above, we decided to include security within this report nonetheless.
Security
Models and Algorithms
Data
Compute
Deployment
Detection and Prevention of Training Data Extraction
Use of Hardware
Prevention of Model Theft
Detection of Adversarial Attacks Modification-Resittar Models
Mechanisms for Security
Shared Model
Anti-Tamper
Governance
Hardware
Model
Detection and Authorization of Dual-Use Capability Inference Time
Enforcement
Disgorgement and Machine Unlearning
Compute Usage
Restrictions
Figure 5: Open problem areas in the Security capacity, organized by target
6.1 Data
Example Research Questions
69.
How can attempted data extraction attacks be reliably identified? (6.1.1)
70. How can Al systems be made robust to data extraction attacks? 6.1.1
71. How can methods for restricting verbatim reproduction of training data be generalized protect the same information being extracted in slightly different form? 6.1.1
6.1.1 Detection and Prevention of Training Data Extraction
Motivation: Prior research has demonstrated how large amounts of models' training data can be extracted verbatóm, with a variety of methods applicable in both black- and white-box settings (Carlini al. 2023e
Nasr et al. 2023 Shi al. 2023 Balle et al.] 2022 Carlini al.] 2019 2021 Duan et al.) 2024b Prashanth et al. 2024 Short of building models that are robust to extraction attacks, having the ability detect them could enable API-level defenses that can block model outputs upon detection of a potential attack, or the introduction and enforcement. of litigation against perpetrators of extraction altacks.
Open Problems:
Improving system robustness to extraction attacks. Do-duplication of training data has been shown assist in reducing memorization, and hence extraction, of specific data-points (Kandpal et al. 2022), though
33

---

Published in Transactions on Machine Learning Research (04/2025)
Nasr et al. 2023 suggest that this provides only marginal improvement. Alternatively, there may be post- hoc interventions that do not alter a model's memorization of its training data, but nonetheless decrease its propensity to reproduce it, with one such example being machine unlearning (see Section 6.3.3) However, acute challenges still remain. For example, restricting the verbatim reproduction of training samples does not prevent the same information being generated by models with slight rewording or reformalting (Ippolito jtt al. 2023). Furthermore, guarding against the reproduction of specific samples has been found to expose previously-safe samples the same attacks phenomenon dubbed the "Privacy Onion Effect" (Carlini et al. 2022 Finally, related yet under-explored area of concern regards the potential for large-scale data extraction from retrieval datasets (Qi et al. 2024c).
Detecting attempted data extraction attacks. Proposed methods for detecting potential data extrac- tion attacks are noticeably absent from the literature, with most publications on this topic aiming to identify model vulnerabilities such attacks. Potential methods for detecting extraction attacks could focus on ei- ther the model inputs or outputs, allowing model providers to filter out suspicious prompts, or outputs that bear a elose resemblance to training samples, respectively.
6.2 Compute
Example Research Questions
72.
How can hardware-enabled governance methods be implemented at scale to ensure the security of a. compute cluster? 6.2.1
73. How it be ensured that given code, along with model weights, can only be executed with license that is verified on-chip, that distributed Al executables only be run approved chips? 6.2.1
74. How can on-chip governance firmware be modified or updated while the chip in operation while remaining resistant to potential allacks? 6.2.1
75. How secure are existing implementations of TEEs on Al accelerators? 6.2.1
76. How methods for tamper-evidence responsiveness be reconciled with the performance de- mands of high-end AI accelerators? (6..2)
77. How secure are existing approaches to tamper-evidence and responsiveness? (62.22
78. How can tamper-proofing methods incorporate self-destruct mechanisms in case of attempted tam- pering? 6.2.2
79. How could the of high-end chips in training foundation models be prevented? 6.2.3
80. Can enforceable mechanisms be developed to allow for the export of chips under predefined con- ditions? 6.2.3
6.2.1 Use of Hardware Mechanisms for Al Security
Motivation: The integration of hardware mechanisms such as TEEs into AI computing clusters could ensure the confidentiality and integrity of workloads (Li et al. 2023b
Geppert et al., 2022: Mo et al.] (024] while
also greatly aiding with AT security and attestation (Nevo et al. 2024 Kulp et al. 2024 Aarne et al. 2024
This in turn would assist in implementing many of the aforementioned problem areas relating to verification and access.
34

---

Published in Transactions on Machine Learning Research (04/2025)
Open problems:
Ensuring utility of TEEs for bardware-enabled governance and security. While TEEs have seen broad adoption on CPUs 21 application to Al accelerators (such as GPUs and TPUs) has so far been limited. The most notable example is Nvidia's incorporation of a TEE in its H100 GPU, referring to it as "NVIDIA Confidential Computing" (Dhanuskodi et al. 2023 Hande 2023 Apsey al.) 2023). However, the H100 implementation "still may not support all of the mechanisms required for an ideal implementation of [hardware governance measures]" (Aarne et al., 2024]. In particular, questions remain regarding whether TEEs robustly attest to the identity of specific chip the data that it is processing. Furthermore, current implementations mostly do not support confidential computing across multiple individual accelerators in a cluster. Further work could investigate the extent to which such functions are supported by current and next-generation chips, or aim to specify hardware, firmware, or software requirements necessary for robustly implementing such functions at the scale of compute clusters, or even entire data centers.
Ensuring security of TEEs on AI accelerators. Given the novelty of the application of TEEs to high-end, Al-specific hardware, it is as-yet unknown how secure such systems are in practice due to lack of independent testing. Previous independent testing of CPU TEEs has uncovered numerous potential vulnerabilities (Muñoz et al. 2023 van Schaik et al.] 2022). Additional security research into GPU TEEs and other security features that they rely on, such as Nvidia's GPU system processor, would be valuable for identifying areas of improvement and assessing whether these features can be relied on for different governance applications.
6.2.2 Anti- Tamper Hardware
Motivation: Some of the aforementioned hardwaree-nabled governance mechanisms, such as verifying com- pute workloads (Section (.2oonn52)). 5.2p, rely on the assumption that the hardware in question has not been compromised or tampered with, for example, if dependent on a TEE for implementation. However, well-resourced adver- saries may aim to physically tamper with chips in order to obviate the defensive protections installed on them. could therefore be useful disincentivize or prevent such lampering. This can come in the form of tamper evidence, whereby physical manipulation of the chip is detectable after the fact, or through tamper responsiveness, whereby tampering with the chip triggers automatic response ranging from delction of sensitive information stored the chip, known zeroization, to permanent self-destruction.
Open problems:
Reconciling tamper evidence and responsiveness with practical requirements of state-of-the- art AI hardware. While approaches to lamper evidence and responsiveness have been proposed (Immler et al. 2018), ensuring that such approaches are compatible with the unique requirements of stato-of-the- art Al accelerators, while retaining affordability and scalability, is an outstanding challenge {Aarne et al., 2024). For example, demanding cooling requirements and high- bathwihhdwidthhh interconnect between chips pose challenge due to the need to bridge the interior and exterior of the tamper-proof enclosure (Obermaier &
Immler 2018).
Ensuring the robustness of anti-tamper approaches. Established approaches to tamper evidence and responsiveness have depended on the use specialized packaging that encases the chip and unable to be removod without leaving visible traces of damage. In the case of tamper responsiveness, the packaging may carry an electric current such that. damage disturbs the current, acting as a trigger for zeroization other active responses. More advanced methods use physical unclonable functions
method that "exploits inherent randomness introduced during manufacturing to give physical entity a unique 'fingerprint' or trust anchor" " (Gao et al. 2 to remotely attest that chip has not been tampered with (Immler et al. 2018 2019 Obermaier & Immler 2018 However, evidence pertaining to the practical success of tamper-proofing has been limited AArne et al.] 2024 Kulp et al.) (024).. Further research is needed if we are to increase confidence in anti- aati-tamper measures when applied to AI hardware security.
Soc, for example, specifications by Intel Intel 2022 and AMD AMD
2024
22 this context, packaging refers specifically physical security enclosure that encases hardware device as opposed to packaging, such cardboard box other container, which the device may be stored transported.
35

---

Published in Transactions on Machine Learning Research (04/2025)
6.2.3 Enforcement of Compute Usage Restrictions
Motivation: Recent attention in compute governance has been paid to export controls placed on cutting- edge chips of the type used in large-scale training of Al systems (Bureau of Industry and Security 2022a Allen 2022). However, export controls are a blunt instrument with a high potential for collateral damage by restricting the sale of affected chips for legitimate uses. Indeed, the Bureau of Industry and Security, the US executive agency responsible for such export controls, itself put call out for "public comments proposed technical solutions that limit items specified under [the export controls] from being used in conjunction with large numbers of other such itemms in ways that enable training large dual-use AI foundation models with capabilities of concern" (Bureau of Industry and Security 2023). presumably acknowledging that technological developments for disentangling legitimate and malicious uses of high-end chips would desirable. It is worth noting, however, that there is considerable disagreement over both the viability of such solutions, with concerns raised regarding the level of confidentiality of such measures, as well
the possibility of their being circumvented (Ting-Fang Grunewald 2023
2023
Patel 2023
Grunewald Aird 2023 Fist &
Open problems:
Implementing remote attestation for disaggregated machines. It would be useful to verify the particular set of hardware components- in this case, AT chips - that are part of the same cluster. This would assist with hardware- based methods for verifying properties of workloads, which typically rely on knowing which chips are participating in a workload, and for ensuring that end-users are complying with export control obligations. An open question is how remote attestation could work for disaggregated machines (Google Cloud 2024) for heterogencous devices. These mochanisms could allow the nature and acceptability of the configuration to be remotely attested to. Other relevant projects in this context that may serve starting points are Caliptro and Opennitan
Restricting particular cluster configurations. It may also be possible to restrict possible cluster config- urations assist with export control policies. One proposed approach involves restricting the communication bandwidth between GPUs to prevent "many consumer device-chips from being aggregated into a supercom- puter" (Kulp et al.) 2024). It may be possible to build such system top of cxisting features such trusted platform modules (Hosseinzadch et al. 2019). or it may require new protocols and new hardware-level features all of which are open problems.
6.3 Models and Algorithms
Example Rescarch Qucstions
81. What cybersecurity measures can be taken at the infrastructure level to protect model weights from theft by an adversary? (6.3.
82. How can models be protected from inference attacks aiming to reproduce replicate model weights and architecture? 6.3.1
83. What are the most promising methods for enabling shared model governance? 6.3.2
84. How should the success of different model unlearning techniques be evaluated? 66..3
85. How can be ensured that machine unlearning and model editing techniques do not cause un- wanted side-effects such as removing concepts that were not explicitly targeted? (6..3
86. How effective are model unlcarning and model editing techniques when applied to multi-lingual or multi-modal models? 6.3.3
36

---

Published in Transactions on Machine Learning Research (04/2025)
6.3.1 Prevention of Model Theft
Motivation: As models become more capable they could become an increasingly valuable target for theft by adversarial parties wanting to put them to their own potential (mis)use. Similarly, as state-of-the-art models become more broadly integrated into the economy and society, the attack surface will increase, potentially leading to a greater threat of exfiltration (Nevo et al. 2024). Tt follows that securing model weights, and other system components, might become an increasing priority to prevent theft or model access by unauthorized parties that may undermine governance initiatives aimed at ensuring customer safety and national security [Nvo et al. 2024
Open problems:
Ensuring adequate cybersecurity for model weights. Protecting model weights against exfiltration attempts requires protections against insider and outsider threats (Nevo et al. 2024p. This includes stan- dards for physical sccurity of the data center facility itself, as well as of the hardware and software stacks AO 2024b) 28 Improved coordination between actors facing similar threats might also assist defenders in understanding the threat landscape and better protecting their assets during training and deployment. Further analysis of potential threat vectors, as well as development of physical and cybersccurity mcasures including and beyond those in (Nevo et al. 2024), would help to identify and address these risks.
Defending against model inference attacks. Alternatively, adversaries may try to extract or replicate models through at attacks to a query API (Orekondy et al. 2018 Tramer et al.] 2016 Jagielski et al.) 2020)
Carlini et al. 2020 2024), logit values @Carlini et al. (2024) or side-channel attacks (Wei et al. (2.0].02)).. Further
research could aim to quantify threats and develop methods for defending against. these, and other, forms of model extraction attacks.
6.3.2 Shared Model Governance
Motivation: Shared model governance refers to the practice of distributing control over a model's training or inference across multiple parties, such that training or inference can ouly be carried out with the agreement of all parties (Bluemke et al.] 2023). The ability to distribute control of model in this way could have many potential use cases for example, if multiple diverse actors want to pool investment for training a shared model where each actor has specific requirements for how the model is trained. This may also be applicable in the case of international collaboration between state-backed institutes wanting to collaborate on Al research (Ho et al. 2023).
Open problems:
Enabling shared model governance through model splitting. One proposed approach for technically- enforced shared model governance is model splitting (Martic et al., 2018) that is, "distributing deep learning model between multiple parties such that each party holds a disjoint subset of the model's parame- ters." Martic et al. (2018) also investigate the resulting question of how computationally expensive it would be for a single actor to reconstruct the entire model, starting from their share of the parameters. A similar approach is taken by SyliINN (Vepakomma et al. 2018 Ceballos et. al. 2020), (hough the emphasis is placed on model splitting to achieve data privacy, rather than shared model governance. Given the relatively small amount of prior work on model splitting for shared model governance, future work could aim to provide further proofs of concept and evaluate their efficacy.
Enabling shared model governance through secure multi-party computation and homomorphic encryption. Two potential alternative approaches for achieving shared model governance are applying either
secure multi-party computation (SMPC) 1 Yao 1982
Evans et al. 2018 or homomorphic encryption 2018 Though usually applied to AT for the purposes of data privacy (Knott
1986
(HE)
/entry| 2009: Acar et al.
Tan et al. 2021 Guo et al. 2022 Riazi et al. 2018). or model privacy (Trask) 2018). both SMPC and HE could potentially be leveraged for shared model governance. For example, using SMPC, model creator could take each parameter within model and split it into multiple shares, distributing such shares across shareholders. Alternatively, using HE, model could be encrypted using one or more private keys such that the ability to decrypt model results relies upon
et al.
2021 Kumar et al. 2019
2017
Dahl 2017 Ryffel et al.
23 Data center security standards: [International Organization for Standardization 2021 Wikipedia contributors
2023
37

---

Published in Transactions on Machine Learning Research (04/2025)
the application of the private key of all parties. However, both HE and SMPC have performance concerns, with even state-of-the-art encrypted deep learning approaches yielding 100x performance slowdown (Wagh) ett al. 2019 2020 Stoian et al.) 2023 Frery et al.) 2023). Future work could investigate this potential in more detail, aiming to provide proof-of-concee demonstrations of how shared model governance using either HE or SMPC could be achieved. Alternatively, future research could aim to reduce the high computational overheads of HE and SMPC.
Enabling shared model governance through TEEs. Finally, shared governance could potentially be achieved through the application of TEEs. In this case, multiple parties could upload program to TEE, with the chip providing evidence on the software program being run. With this evidence, all parties can know how any information they upload to the enclave will be handled. TEEs may also be able to reinforce the above approaches, for example, SMPC. Future research could aim to provide proof-of-concept demonstrations of shared model governance through the use of TEEs given the current hypothetical nature of this approach.
6.3.3 Model Disgorgement and Machine Unlearning
Motivation:
The concepts of model disgorgement Achille al.
2024 and machine unlearning Bourtoule
ct al. 2021: Nguyen ct al. 2022 Shaik et al.] 2023 ct al. 2023 Eldan & Russinovich 2023 Yao ct al. 2023
Liu et al. 2024a b Goel et al. 2024) have been proposed as methods for removing memorized information or otherwise nullifying the impact of a model's having been trained on problematic data. This could potentially
introduce a pathway through which harms of reproducing inappropriate or copyright data could be addressed in cases where action was not taken during data curation or model training. Related methods for direct model
editing (Mitchell et al. 2021 2022a Meng et al.
2022
Hernandez al.
2023) to remove learned harmful
concepts, through editing activations (Zou et al.
2023a
Turner et al. 2023
concept crasure (Ravfogel et al.,
2022 Belrose et al. 2023). or targeted lesions Li et al.
2023a
Wu et al.
2023 could provide alternative
approaches to achieving these aims.
Open problems:
Ensuring unlearning methods are robust and well-calibrated. Machine unlearning involves an interplay between specificity of, and generalization from, concepts to be unlearned. In particular, methods that successfully generalize aid in cases where the unlcarning target is hard to preciscly specify. However, generalization may open the door to unintended side-effects if it results in the removal of non-targel concepts (Cohen et al. 2024b). A challenge to be addressed then is to ensure that methods for machine unlearning and model disgorgement are well-calibrated in that they successfully generalize to comprehensively remove target concepts, while avoiding the removal of benign concepts.
Extending unlcarning and model editing to cross-lingual and cross-modal models. As trends towards multilingual :Üstün al., 2024) and multi-modal (Yin al. 2023) models continue, there will be a need to extend model unlearning and editing techniques to these models. Questions remain as to the efficacy of such techniques when applied in such cases (Si et al. 2023), for example, regarding whether models retain concepts in other languages, despite that concept having been unlearned in English.
Evaluating the efficacy of unlearning and direct model editing techniques. A further outstanding question is how the efficacy of unlearning attempts can be evaluated (Lynch et al., 2024 Shi et al. 2024) Evaluations should aim assess not only whether the influence of the specified unlcarning targets has indoed been removed and that model performance in other domains has not been adversely affected (Li et al. 2024b).
but also identify potential ripple effects that may have resulted from application of unlearning or model editing (Cohen et al. 2024b).
38

---

Published in Transactions on Machine Learning Research (04/2025)
6.4 Deployment
Example Research Questions
87.
How can the robustness of methods for detecting adversarial attacks be improved? 664.1
88.
What interventions are most effective for handling detected adversarial attacks at inference time? 6.4.1
89. How model be made rosistant to being fine-tuned for malicious tasks, while still allowing for benign fine-tuning? (64.2)
90. How can the request of dual-use system capabilities reliably detected? 6.4.3
91. How could authorization of user identity be used as gate for dual-use model capabilities?
6.4.3)
6.4.1 Detection of Adversarial Attacks
Motivation: Adversarial attacks refer to deliberate attempts to exploit inherent model vulnerabilities in order to make the model behave incorrectly or harmfully, for example, by outputting offensive or toxic language 2024). In the case of language models, such attacks [Lohn] 2020 Shayegani et al. 2023 Vassilev et al. often take the form of modifying the user input so as to bypass any implemented safety filters. Some attacks are transferable across different models (Zou et al.
2023b and defenses against adversarial attacks are
typically narrow and brittle (Narayanan & Kapoor 2024). The ability detect such attacks could enable the application of largeted system-level defenses, such halting filtering system oulput, separate from relying solely on the underlying model's robustness to attacks. Furthermore, having empirical evidence the frequency of attacks can help inform deployment corrections (Section 7.2) and threat models (Section 8.1.
While some system-level defenses against adversarial attacks exist, it is important to note that many such protective measures can only implemented effectively in an application or deployment context (Narayanan) & Kapoor 2024). Though directly improving the adversarial robustness the model level is related active research area Vassilev al., 2024 Hu et al.) 2024 Su et al. 2024) here emphasize the closely related issuc of being able to detect and handle potential adversarial attacks at inference time due to its relevance for governance interventions as mentioned above.
Open problems:
Detecting adversarial inputs and outputs. Being able to detect and classify user inputs a model as potential adversarial attacks allows for filtering (Jain al. 2023a Aldahdooh et al. 2022) or preprocessing
(Cohen al. 2019: Nie al. 2022 Kumar et al.) 2023 Jain al. 2023a Zhou al. 2024] of concerning inputs before being given the model. Alternatively, model outputs could be filtered with the aim of
detecting model's response to adversarial attacks in order to remove them before reaching the user (Phute)
et al. 2024 Greenblatt et al.) 2023 Current techniques for detection, however, can suffer from lack of
robustness themselves or may introduce significant latency for the user (Glukhov et al. 2024
6.4.2 Modification-Resistaal Models
Motivation: Post-deployment fine-tuning is common method for user customization of language models, either through an API or locally in the case of downloadable models. Fine-tuning, and other post-training enhancements have been theorized to have outsized impact on downstream performance (Davidson et all 2023) However, just as fine-tuning can be used to customize model for legitimate and beneficial use cases, il can just as easily be used to adapt model for malicious purposes, often with small amounts of data (Jain) et al. 2023b; Yang et al. 2023 Qi et al.) 2023 Lermen et al.) 2023 Zhan et al.) 2024 Qi et al., 2024a b) Fine-tuning attacks often aim to achieve similar aims to those adversarial attacks above, though through customizing models directly, rather than by modifying input prompts. Having methods for preventing the customization of models for malicious use could reduce misuse risks associated with wpen-weight release,
39

---

Published in Transactions on Machine Learning Research (04/2025)
thus expanding the range of potential deployment options and promoting the numerous benefits of more open release strategies.
Open problems:
Preventing the modification of models for malicious tasks. An open question whether there exist technical methods that restrict model's amenability to being fine-tuned (or modified through other methods) for harmful uses, while retaining the ability to be modified for benign uses (Rosati et al. 2024b: Peng et al. 2024). Potential methods may aim to raise the computational cost of fine-tuning on harmful
data to prohibitive levels (Henderson et al. 2023b Deng et al., 2024 Rosati et al. 2024a Tamirisa el al.
2024 or make models resistant to learning from harmful data (Zhou et al. 2023b Huang et al.) 2024cc Hsu
et al. 2024 Huang al.] 2024b). However, given the nascency of these techniques, future rescarch could aim establish their robustness in practice.
6.4.3 Detection and Authorization of Dual-Use Capability at Inference Time
Motivation: In the event that model assessments have flagged system's competence in dual-use domains that is, domains which can have both beneficial and harmful applications model providers might need to avoid exposing these capabilities publicly by default in order to avoid misuse. However, completely removing these capabilities may not be feasible, or economically favorable due to the legitimate and beneficial use- cases, such as cybersecurity professional using a system aid in the identification and patching of software vulnerabilities. This differs from the discussion of adversarial attacks above, for which the aim of attack is taken to be unambiguously harmful.
Open Problems:
Detecting requests of dual-use capabilitics. Guarding against malicions uses of dual-use capabilities is currently imperfectly achieved by conducting safety fine-tuning so that the model refuses to respond malicious requests. However, this approach is not robust jailbreaks that pose as legitimate requests for such capabilities (Wei al., 2023 Fang et al. 2024). An alternative approach could be to detect all requests for dual-use capabilities. This would allow for the application of separate methods for distinguishing between legitimate and malicious requests, such as independent classifiers trained to separate between legitimate and malicious intent, which may perform better than broad safety fine-tuning.
Requiring authorization for dual-use capabilities. Alternatively, authentication, for example as a certified cybersecurity expert, before accessing certain capabilities may be one way of managing the dual-use nature of general models. This may also have uses for allowing red-teamers capabilities for research purposes (Longpre al 2024a Such proposals currently hypothetical, and future work could aim to propose proof-of-concept demonstrations of how such authorization scheme could be implemented in practice.
researchers access such
40

---

Published in Transactions on Machine Learning Research (04/2025)
7 Operationalization
Previous scctions have discussed concrete technical problems in relation specific targets in the taxon- ony. In contrast, the following two sections will discuss capacities that span across these largets, namely, operationalization, and ecosystem monitoring.
Operationalization entails the translation of ethical principles, legal requirements, and governance objectives into concrete stralegies, procedures, and technical standards. It. can also involve the harmonization of terminology and concepts across governance frameworks, for example, NIST's Risk Monagement Framework Crosswalks [NIST] )023a). Without technical expertise, operationalization efforts may fail to capture the nuances and realities of AI systems, leading to ineffective or even counterproductive governance initiatives.
Operationalisation
Transiation of Governance Goals into Policies and Regulatory Requirements
Deployment Corrections
Figure 6: Open problem areas the Operationalization capacity
Example Research Questions
92. What system properties (if any) are the most reliable indicators of risk, and thus candidates for serving as regulatory targets? (7.1
93. How can AI safcty, rcliability, and other technical requirements be standardized given insufficient explanatory understanding of model behavior? 7..
94. What are general intervention and correction options flaws with a model are identified post. deployment? 7.22
7.1 Translation of Governance Goals into Policies and Requirements
Motivation: Policies are often formulated with specific aims in mind, for example, to protect consumer safety, promote fairness, or ensure accountability. In cases of rule-based regulation, these aims must be translated into rules that "typically prescribe or prohibit a specific behavior" (Schuett et al. 2024). In many cases this act of translation will demand involvement of technical expertise to provide guidance on the feasibility of proposed rules, as well as the extent to which they achieve a policy's stated aims. For example, the goal of ensuring consumer safety may motivate the introduction of licensing regime that mandates pre-market safety evaluations. Given the current robustness of such evaluations (see Section 3), this may not only fail to ensure safe and reliable products, but also create false sense of security (Reuel et al. 2024b Wu 2028 For many governance elforts, concrele translations of goals into elfective requirements and standards are still
lacking (Guha et al. 2024
Pouget 2023
Open Problems:
Identifying target dimensions for regulation. Identifying technical dimensions that best align with governance priorities is an open challenge. For example, the current practice of using training compute measure of risk may not always be suitable, given that smaller modcls outperform larger ones with targeted training. Furthermore, modern AI systems are often the result of multiple data curation and training processes, and it is unclear whether compute expenditure from auxiliary processes should contribute towards
24 Measured in fooating-ponnt operations (FLOP).
41

---

Published in Transactions on Machine Learning Research (04/2025)
the final FLOP count. It's also unclear how measures of training compute should take into account techniques such as quantization and drop-out (Hooker 2024 23 Aside from training compute, are there other, more precise ways of defining which systems should be subject to regulation? How could such measures account for improved algorithms, and ways in which relevant capabilities can be increased after training (Davidson
et al. 2023
Scharre 2024h?
Detailing and creating standards across the Al system life cycle. While recent Al standard-setting efforts, such the NIST Risk Management Framework (NIST) 2023b 2024 and ISO guidance on AI risk management (International Organization for Standardization 2023 provide valuable general principles, they often lack the technical specificity required for objective assessment of AI systems' compliance with safety and ethical requirements (Pouget 2023). Additionally, though standard-setting bodies such as IETF, IEEE, ISO, and CEN LEN-CENELEC, along with initiatives such as the Partnership on AI (Partnership on Al, 2023), are working to develop more detailed and verifiable guidance, many areas stilll require further technical expertise and rescarch (Barrett al.) 2023). One such area is that of security-by-desiee for hardware, where further work is needed to implement standards across firms, including standards for multi-device allestation in cluster and TEEs (Cybersecurity and Infrastructure Security Agency 2023 Kelly et al.) 2022 see
also Section 6.2 Other goals, such as fairnoss, can be mcasured in various ways, and it remains unclear which metrics are most appropriate and effective in which contexts (Parraga et al.
2023 Caton & Haas,
2024 Chouldechoval 2017 Kleinberg] 2018). Finally, standardizing reporting for Al systems, such as the 2021 could increase
information included in model cards (Mitchell et al., 2019)) or data sheets (Gebru et al. the utility of such practices in governance contexts, motivating the question of what specific information should be included in standardized reports (Kolt et al. 2024 Bommasani et al. 2024
7.2 Deployment Corrections
Motivation: In the event that flaws are identified a deployed model, would be beneficial to adequately respond to the identified risk. Such a scenario could occur either through the identification of previously unobserved capabilities in deployed model, through post-training enhancements such as fine-tuning DDvidsoon et al., 2023). O'Brien et al. (0023) refer to post-deployment responsive actions "deployment corrections." While they explore this issue from an institutional perspective, for example providing recom- mendations on how this could be addressed through corporate governance structures and procedures, we see scope for much greater exploration from a technical perspective.
Open problems:
Navigating the continuum of model corrections and interventions. O'Bren et al. (2023) define five categories for deployment corrections: user-based restrictions, access frequency limits, capability or feature restrictions, use case restrictions, and model shutdown. Within each of these categories there are open questions regarding the feasibility of implementation. For example, model shuldown is a relatively extreme action to take upon discovery of a system flaw, and if carried out naïvely could risk major disruption to users, clients, and services that depend on the system in question. Thus, it would be beneficial to have methods in place for minimizing disruption to downstream services in the event that model shutdown is deemed necessary. Furthermore, model shutdown, along with deployment corrections that modify the underlying model, are at odds with the issuc of providing model stability and backward-compatiillt are particularly relevant for ensuring the reprodueibility and replicability of AI research (see Sections 4.3.1
features that
and 5.3.2.
25mm recommendation are put forth
Frontier Model Forum 2024
42

---

Published in Transactions on Machine Learning Research (04/2025)
8 Ecosystem Monitoring
Duc to the rapid pace of advancements in AI, coupled with uncertainty about future devclopments, AT governance needs to be forward-looking, future-proof, and adaptive (Guihot et al. 2017 Kolt 2023 Reuel & Undheim 2024). To fulfill this goal, pecision-makens need to be aware of the multiplicity of stakeholders in the AT ecosystem and how they relate to each other, as well as general trends and potential impacts of current and future AI systems (Wansley, 2016 Ada Lovelace Institute 2023: Whittlestone & Clark 2021 Epoch, 2023). Collating and providing such information, which we refer to as ecosystem moniloring, can enable Al governance actors to make more informed decisions, better anticipate future challenges, and identify key leverage points for effective governance interventions.
Ecosystem Monitoring
Clarification of Associated Risks
Prediction Future Developments and Impacts
Assessment Environmental Impacts
Supply Chain Mapping
Figure 7: Open problem areas in the Ecosystem Monitoring capacity
Example Research Questions
95. What risks, whether from intended or unintended harm, are associated with different (types of) systems? 881
96. How do potential risks differ across domains? 8.
97. How trends and/or properties observed in current systems be extrapolated to make predictions about future systems? 8.2
98. How could developments in Al-specific hardware impact the governability of compute? 8.2
99. What information about systems is needed to accurately assess the environmental impact of its development and deployment? 8.3
100.
Given the required information, how can the environmental impact of Al system be accurately assessed? 8.3
101.
What technical methods can be implemented to create an auditable log of all actors and their contributions throughout the AI development process, from data collection model deployment?
8.4
8.1
Clarification of Associated Risks
Understanding risks associated with the development and deployment of AI systems enables policymakers to prioritize governance efforts, allocate resources effectively, and determine the urgency of addressing specific risks (Whittlestone & Clark 2021 Clark 2023
Motivation:
Open Problems:
Developing better threat models for risks of AI. While much prior work has intended to lay out taxonomies risks and harms posed exacerbated by AI systems (Critch & Russell 2023 Hendrycks
al. 2023 Weidinger et al. 2022 Abercrombie et al. 2024 Hammond et al. [orthcoming Zeng et al. 2024
43

---

Published in Transactions on Machine Learning Research (04/2025)
OECD 2023
Hoffmann & Frase 2023 Turchin & Denkenberger 2018 Grabb et al. 2024 detailed threat
models have been relatively underexplored. One option for future research could aim to apply standardized risk management approaches, such as causal mapping (Eden et al. 2004
Ackermann et al. 2014), to gain greater clarity into if and how harms from AI may materialize, and where policy could intervenc.
Improving incident reporting and monitoring. Additionally, developing improved systems for mon- itoring and reporting previous or ongoing incidents could not only allow for more targeted response to ongoing harms, but also facilitate the identification of early warning signals for potential harms (Shane, 2024). Al incident databases have been developed by both the OECD and Partnership on AI, both of which log news articles detailing AI-related incidents (OECD.AT Policy Obscrvatory 2024 McGregor 2020). Given that these databases rely solely on public sources, it is likely that only a subset of all incidents are included. In addition, they do not record all details about an incident such as model specifics or deployed guardrails, limiting the utility for analysis of what may have caused an incident. Open questions thus concern how non-public incidents can be reliably reported, as well as what technical information should be reported in order to facilitate meaningful analysis of incidents.
8.2 Prediction of Future Developments and Impacts
Motivation: Anticipating the trajectory and potential impact of Al systems may allow policymakers to proac- tively set governance priorities, determine the urgency of addressing specific issues, and allocate resources accordingly (Toner al.) 2023). Greater foresight would enable more adaptive and anticipatory approaches to AI governance, which are essential given the rapid pace of AI development (Reuel & Undheim 2024
Open Problems:
Measuring and extrapolating from empirical trends. Existing work has aimed to empirically measure trends in training compute (Sevilla et al.] 2022) and algorithmic progress (Ho et al.) 2024 among others (Epoch 2023). . Future work could aim to extend this effort by quantifying other trends that have not yet been addressed, such as usage patterns of Al in different industries, or assessing the accuracy of predictions based the extrapolation of observed trends.
Estimating a system's impact before deployment. Estimating the impact of an AT system before deployment, including economic impacts (Eloundou et al.) 2024), could help prioritize governance efforts, While such predictions could be aided by more developed threat models (see Section 8.1 above), research may also benefit from technical tools to safely and ethically experiment and simulate potential outcomes without causing harm.
8.3
Assessment of Environmental Impacts
Motivation: The environmental impact of Al systems extends across the entire Al life cycle [Metcalf et al. 2021 Luccioni et al.) 2023b Rakova & Dobbe 2023 including both during training (Strubell et al. 2019 Patterson el al. 2022) and inference (Luccioni et al. 2023a). Having an accurate understanding of the end- to-end environmental impacts is crucial for policy initiatives, for example, determine suitable incentives and penalties for encouraging AI developers to reduce the environmental costs associated with their systems.
Open Problems:
Assessing the energy usage of training and hosting systems. Open problems remain due to the logistical challenges of tracking energy consumption and carbon emissions across numerous dynamic system instances. Furthermore, current efforts struggle to take into account energy sources a factor which can massively affect the overall impact assessment. Ongoing work aims to develop energy ratings for combinations of models and tasks, allowing users to make informed decisions about their system usage, taking into account the environmental impacts of their choice (Luccioni 2024). Alternatively, tools such as CodeCarbor provide developers with real-time estimates for the carbon emissions from running their code. Other work has focused on comparing compute cost on smartphones vs. the cloud (Patterson et al. 2024), best practices for training
26 "thttps//roodeearboom.
44

---

Published in Transactions on Machine Learning Research (04/2025)
models PPatterson et al. 2022 and comparing cost for different models (Luccioni al. 2023a Luccioni & Herrandez-Garcia 2023
Assessing environmental costs of raw resources for building and running data centers. Along with energy, environmental costs may come from other sources along the semiconductor and AI supply chains, for example, from mining and refining the rare earth minerals required for the manufacturing of semiconductors (Kuo al, 2022 Ruberti 2023). Additionally, large data centers used for training and hosting AI systems require great quantities of water part of their cooling systems (Mytton 2021). Future research could aim to provide in-depth end-to-end predictions of the environmental costs of constructing and maintaining data centers to inform policies aimed at reducing associated environmental impacts.
8.4 Supply Chain Mapping
Motivation: Mapping the Al supply chains allow policymakers to better understand the complex ecosys- tem involved in the development and deployment of AI systems. By identifying key actors and processes at each stage of the supply chain, policymakers can target interventions at the most suitable point in the supply chain. Furthermore, cxisting export controls limiting chip exports to Russia and China have been marked by substantial enforcement difficulties (Allen et al. 2022), , and analyses have suggested that AI chips are also likely become targets for substantial smuggling operations (Grunewald Aird 2023, Fist Grunewald, 2023). By understanding the flow of these resources, authorities can better combat the smuggling of chips and other hardware components.
Open Problems:
Identifying supply chain components and actors. Another area requiring further technical expertise is the identification and assessment of supply chain components. For example, in the context of liability and copyright law, tracking components and design choices made by different actors along the AT supply chain could enable courts to make more precise assessments of potential infringement responsibility (Lee et al. 2024 Longpre et al. 2024b). This granular understanding might be necessary infringement can occur multiple points: during data collection, model training, output generation. If model produces content resembling copyrighted material, determining liability may require tracing back through the supply chain to identify the source of infringement, whether in training data, model architecture, or generation prompt.
45

---

Published in Transactions on Machine Learning Research (04/2025)
9 Conclusion
In this paper, presented broad overview of open problems in tochnical AI governance (TAIG) across six capacities and four governance targets. We provided a definition of TAIG, corresponding taxonomy of the work that it entails, and an overview of sub-problems for each sub-area defined in our taxonomy. We also provided links to relevant literature and example research questions that technical researchers could tackle to help advance AI governance efforts.
We also showed that TATG not monolithic problem instead, comprises interrelated sub-problems requiring distinct yet complementary approaches. Our taxonomy provides structured framework for ad- dressing such issues: Each open problem area presents unique technical bottlenecks, such challenges in altributing model behaviors to training data, ensuring privacy-preserving access, and verifying system prop- erties. By outlining these constraints, we highlight where governance aspirations are hindered by technical limitations and where technical research can drive progress in Al governance.
Our work further highlights that technical work does not exist isolation. Instead, addressing many of these problems will require cross-disciplnary collaboration among tochnical rescarchers, policymakers, industry stakeholders, and civil sociely. No single discipline can fully resolve Al governance challenges, necessitating integrated expertise across computer science, law, political science, ethics, and economics. As AT systems become more capable and widely deployed, governance strategies must develop alongside technical advances, making TAIG an evolving field and ongoing priority.
Acknowledgements
LHa acknowledges the support of an EPSRC Doctoral Training Partnership studentship (Reference: 2218880). RS acknowledges support from Stanford Data Science, and OpenAT Superalignment grant. NG acknowledges support from Stanford Interdisciplinary Graduate Fellowship. YB acknowledges funding from CIFAR. AP acknowledges a the support of gift from Project Liberty. SK acknowledges support by NSF 2046795 and 2205329, NTFA award 2020-67021-32799, the Alfred P. Sloan Foundation, and Google Inc. DB, PB and AR acknowledge funding from Open Philanthropy and the Stanford Interdisciplinary Graduate Fellowship.
The authors would also like to acknowledge the early feedback received as part of Work- Work-In-Progrene In-Progress meet- ing, hosted by the Centre for the Governance of AI. We would particularly like to thank Jamic Bernardi, Ben Clifford, Augustin Godinot, John Halstead, Leonie Koessler, Patrick Levermore, Sam Manning, Malthew van der Merwe, Aidan Peppin, James Petrie, and Christopher Phenicie for detailed comments and insight- ful conversations. We further thank Dewcy Murdick for his thoughtful and constructive feodback, which siguificantly improved the paper's rigor.
Finally, the authors would like to thank Beth Eakman and José Luis León Medina for support with copy- editing and typesctting, respectively.
46

---

Published in Transactions on Machine Learning Research (04/2025)
A
Appendix: Policy Brief
The increasing adoption of artificial intelligence (AI) has prompted governance actions from the public sector, academia, civil society, and industry. However, policymakers often have insufficient information for identifying the need for intervention and assessing the efficacy of different policy options. Furthermore, the technical tools necessary for suecessfully implementing policy proposals are often lacking. We introduce the field of technical AI governance, which sceks to address these challenges.
Technical AI governance refers to technical analysis and tools for supporting the effective governance of AT. We argue that technical AT governance can:
Identify areas where policy intervention is needed through mapping technical aspects of systems to risks and opportunities associated with their application;
Inform policy decisions by assessing the effectiveness and feasibility different policy options; and
3. Enhance policy options by enabling mechanisms for enforcing, incentivizing, or complying with norms and requirements.
We taxonomize technical AT governance according to elements the AT value chain: the inputs of data, compute, models, and algorithms, through to the deployment setting of the resulting systems. The figure below shows the key governance capacities that can be applied each target.
G-D-D
Assessment
Access
Verification
Enables the understanding system capabilities and risks, to allow for more targeted policy intervention.
Enables external research and assessment of Al systems, and the fair distribution benefits Al.
Establishes trust in Al systems confirms compliance with regulatory requirements.
Security
Operationalization
Ecosystem Monitoring
Ensures the integrity, confidentiality, and availability of Al systems and guards against misuse.
Bridges gap between abstract principles and the implementation of norms and requirements.
Enables anticipation future challenges and identification of levers for governance intervention.
Key Takeaways
We highlight number of the key takeaways within technical Al governance, including that:
Evaluations of systems and their downstream impacts users and society have been proposed in many governance regimes. However, current evaluations lack robustness, reliability, and validity, especially for foundation models.
Hardware mechanisms could potentially enable actions including facilitating preveccyir access datascts and models, verifying the use of computational resources, or attesting the results of audits and evaluations. However, the use of such mechanisms for these purposes largely unproven.
47

---

Published in Transactions on Machine Learning Research (04/2025)
The development of infrastructure for enabling research into AI, such resources for conducting analyses of large training datasets or for providing privacy-preerviigg access to models for evaluation and auditing, could facilitate research that advances the scientific understanding of AI systems and external oversight into developers' activitics.
Research that aims monitor the AI ecosystem by collecting and analyzing data trends and advances in AT has already proven crucial for providing policymakers with the information needed to ensure that policy is forward-looking and future-proof.
We note that technical Al governance is merely one component of a comprehensive Al governance portfolio, and should be seen in service of sociotechnical and political solutions. A technosolutionist approach to AI governance and policy is unlikely to succeed.
Recommendations
Based on the above takeaways, we recommend:
1. Allocating funding and resources through open calls and funding bodies, to technical Al gov- ernance research, drawing established expertise in adjacent fields;
That policymakers collaborate closcly with technical experts to define feasible objectives and identify viable pathways to implementation;
3. That government bodies, such as Al Safety Institutes, conduct in-house research on technical AI governance topics, beyond their current focus on performing evaluations; and
4. That the future summits on AI, other fora such the G7, the UN Al advisory body, and reports such as the International Scientific Report on the Safety of Advanced AI, focus effort and attention towards technical Al governance.
Please have a low bar for reaching out Anka Reuel (anka. rouellsaaool edu) and Ben Bucknall (buckmall@robota ax.ac.uk uk) with any questions comments.
48

---

Published in Transactions on Machine Learning Research (04/2025)
B Appendix: Methodology
We grounded our methodology in established practices from governance and policy research. Taxonomies
are widely used in these fields to organize complex domains into structured calegories (Smith 2002). A
taxonomy is essentially a "formal system for classifying multifaceted, complex phenomena according to common conceptual domains and dimensions" (Bradley ct al.] (007)), based on Patton (2002)) ). Typologies
and Laxonomies can help bring clarity to ill-defined problem spaces (Sofaer) 1999 Smith 2002p. Following this
previous literature and work, we adopted a aaoooomyybasee approach to systematically map the landscape of technical Al governance challenges.
As outlined in Section ! previous AI governance research, in particular, has introduced such taxonomies and frameworks to categorize governance issues. For example, Dafoe (2018) divided the AT governance space into three clusters technical landscape, Al politics, and ideal governance structures to attempt comprehensive coverage of AI governance challenges. Likewise, Critch & Russell (023)) proposed a taxonomy of socictal- scale AT risks organized by accountability (i.e., who the responsible actors are and whether harmful actions are deliberate or emergent).. These prior efforts illustrate that there is no single 'correct' AI governance laxonomy; rather, their value lies in choosing organizing principles that reveal new insights or gaps (Billey 1994h.
For our rescarch, we began with a systematic scarch on Google Scholar and SSRN using the search terms "Techmical' AND ('Artificial Intelligence' OR 'Machine Learning' OR 'Al' OR 'ML') AND ( ('Governance' OR 'Regulation') to identify academic literature targeting technical Al governance, which did not yield any search results based on paper titles or abstracts. We then expanded search efforts to include the terms ('Artificial Intelligence' OR 'Machine Learning' OR 'AI' OR 'ML') AND ("Governance" OR 'Regulation') to identify goals, functions, and challenges of AT governance more broadly, not just focused technical work. In addition, we built on efforts by Reuel et al. (2024b) and compared existing regulatory and governance aims and the technical state of the art to identify gaps and governance challenges that could be addressed with technical advancements (these insights were later also integrated into the MMotvvttion' sections for each open problem area in Sections 3 to
Following this initial literature review, we generated a list of candidate capacities and open problems. We red-teamed this preliminary taxonomy with a focus group of five experts across AI and governance. Based on their feedback, which also referenced the work by [Bommasani et al. (2023b), we added the target dimension to the taxonomy. We applied this mixture between deductive and inductive approaches throughout the whole taxonomy development process, in line with best practices for taxonomy development (Bailey 1994] Nickerson et al. 2013).. Our taxonomy then went through multiple rounds of expert- and literature-driven refinements. We iteratively discussed and adjusted capacity definitions in a scries of intervicws and focus groups with the authors, who are all researchers or practitioners in (AT) governance or relevant technical fields, and external experts, involving over 30 researchers and practitioners across all capacities, targets, and identified open problems areas. In these sessions, we critically examined whether each proposed category was conceptually distinct and collectively exhaustive of the domain's key issues and whether missed any open problems. The resulting feedback led to taxonomy adjustments (e.g., clarifying the scope and definition of the Operationalizaaionn capacity) until we converged on a final taxonomy structure. Whenever a new capacity was added (including for the initial set of defined capacities), we also conducted a literature search on Google Scholar and SSRN with the terms ('Artificial Intelligence' OR 'AT') AND '[Capecily] to validate it, idcntify existing rescarch this area, and to expand on open problems in the identified area. We then again requested expert feedback on gaps, previous literature, open research questions, and missing open problems for the newly-added capacity.
We validated our final taxonomy's relevance by comparing it with previous classifications of issues in AI governance research and AI risk laxonomies, looking for overlaps and differences. We compared our open- problems categories to challenges raised in Dafoe (2018): : Lobel (2024); Shelby et al. (2023) : Raji et al. (2022)): Google (n.d.)); Critch & Russell (2023); Weidinger et al. (202)): Birkstodt et al. (2023), confirming that all major governance challenge areas identified by these authors that could have a technical component have a place within our two-dimensional framework. In fact, many open problems we identify (such as evaluating
49

---

Published in Transactions on Machine Learning Research (04/2025)
model risks or securing compute infrastructure) are technical prerequisites to addressing the higher-level governance challenges described in these prior works. For example, the challenge of governance failure due to inadequate oversight (Uuk et al. 2024 can be partially addressed by our verification capacity, and the risk AI misuse (Critch Russell 2023 could be mitigated by advances in security and assessment, two
of our capacities which are also governance mechanisms highlighted by |MCorrack Bendechache) (2024) and Lobel 2024). This cross-validation across capacities gave us confidence that our taxonomy neither missing well-known challenge domains nor duplicating existing taxonomies without added value. Instead, it synthesizes and builds prior efforts: for instance, where Critch & Russell (2023) focuses who is responsible for risks, and Shelby et al. (2023) catalog harms by impact type, our taxonomy focuses on how technical work can proactively support governance.
Finally, we acknowledge our taxonomy's ' limitations. It is expert-defined and thus subject to unique perspec- tives and experiences. Furthermore, validation is challenging, given that there no ground truth for our taxonomy to test against. is further constrained technical challenge areas by design, meaning leaves out many important socio-technical institutional governance issues (e.g. labor impacls, international dy-
namics) highlighted by others (Dafoe 2018 Lobel 2024
Shelby et al. 2023 Raji et al. 2022a Critch &
Russell 2023 Weidinger et al. 2022).
50

---

Published in Transactions on Machine Learning Research (04/2025)
References
Onni Aarne, Tim Fist, and Caleb Withers. Secure, governable chips: Using On-Chip mechanisms to manage national security risks from AI & advanced computing. Technical report, Center for New American Security, 2024. Tech-Secure-Chips. Jan 24--224--2 -lla-n----naee finall....
Gavin Abercrombie, Djalel Benbouzid, Paolo Giudici, Delaram Golpayegani, Julio Hernandez, Pierre Noro, Harshvardhan Pandit, Eva Paraschou, Charlie Pownall, Jyoti Prajapati, Mark Sayre, Ushnish Sengupta, Arthit Suriyawongkul,, Ruby Thelot, Sofia Vei, and Laura Waltersdorfer. A collaborative, Human--Cnttee taxonomy of AI, algorithmic, and automation harms. Xrv: 2407.01294 [cs.IG). July 2024. URL http:
//arxiv.org /20.. 01294
Abbas Acar, Hidayet Aksu, A Sclcuk Uluagac, and Mauro Conti. A survey on homomorphic encryption schemes: Theory and implementation. ACM Compuling Surveys, 51(4): 35, July 2018. ISSN 0360-0300.
URL httpp:://aal..orr///1141/////2/3
A lessandro Achille, Michael Kearns, Carson Klingenberg, and Stefano Soatto. AT model disgorgement: Methods and choices. Proceedings of the National Academy of Sciences of the United States of Amer- iea, 121(18) #230730412,, April 2024. ISSN 0027-8424, 1091-6490. URL pnas.230730412
Fran Ackermann, Susan Howick, John Quigley, Lesley Walls, and Tom Houghton. Systemic risk elici- tation: Using causal maps to engage stakeholders and build comprehensive view of risks. Euro-
pean journal of operational research, 238(1):290-299) October 2014. ISSN 0377-2217. URL https:
//
Ada Lovelace Institute. Keeping eye AI: Approaches to government monitoring of the AI landscape. Technical report, Ada Lovelace Institute, July 2023. URL httpp:://www. http:://ww..allloeeeeeeestttttte
report/keeping- sttpt//keeeiinna--e-----ee- an-eye-on-ai/
Advisory Body on Artificial Intelligence. Interim report: Governing AI for humanity. Techni- cal report, United Nations, December 2023. URL
un ai bow.isorrybooyy_govrnnnngggg for
Nur Ahmedd and Muntasir Wahed. The de-democratization of AT: Deep learning and the compute divide in artificial intelligence research. arXiv: 15581 [cs.CY] CY], October 2020. doi: http:///XXiv.210000
Mhairi Aitken, David Leslie, Florian Ostmann, Jacob Pratt, Helen Margetts, and Cosmina Dorobantu.
Common regulatory capacity for A1. Technical report, The Alan Turing Institute, 2022. URL https:
ac.
Elif Akata, Lion Schulz, Julian Coda-Forno, Seong Joon Oh, Matthias Bethge, and Eric Schulz. Play- ing repeated games with large language models. arXiv: 1386.16867 [cs. CLJ, May 2023. URL
http:
//axxiv.org hwtr:/abb22305.111
Ekin Akyürek, Tolga Bolukbasi, Frederick Liu, Binbin Xiong, Ian Tenney, Jacob Andreas, and Kelvin Gun. Towards tracing factual knowledge in language models back to the training data. arXiv: 2205.11482
[es. Cls, May 2022. URL
John Albert. A guide to the EU's new rules for researcher access to platform data, 2022. URL https: :
Accessed: 2024-7-17.
Ahmed Aldahdooh, Wassim Hamidouche, Sid Ahmed Fezza, and Olivier Déforges. Adversarial example
detection for DNN models: a review and experimental comparison. Artificial intelligence review, 55(6):
August 2022. ISSN 2821, 1573-7462. URL [ttpee:/daa.oog////.oo/////11100
10125-w
51

---

Published in Transactions on Machine Learning Research (04/2025)
Sina Alemohammad, Josue Vasco-Rodrigues" Lorenzo Luzi, Ahmed Imtiaz Humayun, Hossein Babaci, Daniel LeJeune, Ali Siahkoohi, and Richard Baraniuk. Self-Consuming generative models go MAD. In The 12th International Conference on Learning Representations (ICLR 2024), Vienna, Austria, October 2023. URL
httpss//ppeareeiis..ntttftuuuurp M
Gregory Allen. Choking off china's access the future of AI. Technical report, Center for Strategic and International Studies, 2022. URL http::///w....ccoooo////aaaaaaaa http://hhhoookk----caaaaaaaaaaee future--
ai
Gregory C Allen, Emily Benson, and William Alan Reinsch. Improved export controls enforcement tech- nology nceded for U.S. national security. Technical report, Center for Stratogic and International Studies (CSIS), 2022. URL http:p///ww....sooooooo///aaaaaa
Hilary J Allen. Fintech and Techno-Solutionism,,, January 2024. URL httpss//pappr..arrccccaa//a121
4686469
AMD. AMD secure encrypted virtualization (SEV), 2024. sev html Accessed: 2024-7-17.
URL tttp://www..ut.cc//////rveeleps
Markus Anderljung, Lennart Heim, and Toby Shevlanc. Compute funds and pro-trained modcls, April 2022. URL https///ww..geernaaoca///////ppp funds- and- and-are-ddaa-------e-e tet--trrandddmooe Accessed: 2022-7-
3.
Markus Anderljung, Joslyn Barnhart, Anton Korinek, Jade Leung, Cullen O'Keefe, Jess Whittlestone, Sha- har Avin, Miles Brundage, Justin Bullock, Duncan Cass-Beggs, Ben Chang, Tantun Collins, Tim Fist, Gillian Hadfield, Alan Hayes, Lewis Ho, Sara Hooker, Eric Horvitz, Noam Kolt, Jonas Schuett, Yonadav Shavit, Divya Siddarth, Robert Trager, and Kevin Wolf. Frontier AT regulation: Managing emerging risks to public safety. Xrxiv: 2307.08718 CYs, July 2023a. doi: 10.48550 205500/arXiv..0 wtt..iv..337....8. 0307703718.
Markus Anderljung, Everett Thornton Smith, Joe O'Brien, Lisa Soder, Benjamin Bucknall, Emma Bluemke, Jonas Schuett, Robert Trager, Lacey Strahm, and Rumman Chowdhury. Towards publicly accountable frontier LLMs: Building an external scrutiny ecosystem under the ASPIRE framework. arXiv: 2311.14711 fes. CYJ, November 2023b. URL
Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Jailbreaking leading Safety-Aligned LLMs with simple adaptive attacks. arXiv: 2404.0215 02151 fes. (Rs.CR), April 2024. URL http://arxiv.or abs/ 0bs/2404.02151
María P Angel and Danah Boyd. Techno-legal solutionism: Regulating children's online safety in the united states. In Proceedings of the Symposium on Computer Science and Law, CSLAW '24, pp. 86- 97, New York, NY, USA, March 2024. Association for Computing Machinery. ISBN 9796400703331. URL
(ttpps///b..org//.111///11111 3643705
Anthropic. Authropic's responsible scaling policy, September 2023a. URL http://ww.atttrppicccoo. hes///atbanpgerrrssspppbbeesesep
Accessed: 2024-5-6.
Anthropic. Challenges in evaluating AI systems, 2023b.
URL https:/ww..athhrppL..co////
wwwpatttinnni--sseteeee Accessed: 2024-7-16.
Anthropic. Introducing the next generation of claude, March 2024a. URL https://ww.attirrpic..omm news/claude-3ffaii Accessed: 2024-4-17.
Anthropic. Claude 3.5 sonnet, 2024b. URL http::///wwaaaaaappccco///o///// 5onneet Ac- cessed: 2024-7-15.
Emily Apscy, Phil Rogers, Michael O'Connor, and Rob Nertney. Confidential computing on NVIDIA
H100 GPUs for secure and trustworthy AI, August 2023. URL https::/ hwt.:/ppp...iiiii..oooo////////a coo/blog/////oooooccoc.c..ooon/.
confitentiall-cooppii---oooooo huw.r:/eeeeccceecm--oooooooooaaa Accessed: 2024-7-17.
52

---

Published in Transactions on Machine Learning Research (04/2025)
Rudolf Avenhaus, Nicholas Kyriakopoulos, Michel Richard, and Gotthard Stein (eds.). Verifying Treaty Compliance. Springer Berlin Heidelberg, 2006.
33854-3
Kenneth D Bailey. Typologies and tozonomies: An introduction to classification techniques, volume 12, Sage, 1994.
Mauricio Baker. Nuclear arms control verification and lessons for AI treaties. arXiu: 2304.04123 fes.CY], April 2023. URL
Shyamkrishna Balganesh. The normativity of copying in copyright law. Duke low journal, 2012. ISSN 0012-7086. URL
Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adver- sarics. In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1138-1156. IEEE, May 2022. ISBN 9781665413169, 9781665413111
Bank for International Settlements. BIS innovation hub expands suptech and regtech research to include monetary policy tech, February 2021. URL http:://ww..ia.ogg//ao/t/bbbbtbp
Accessod: 2024-7-12.
Peter Barnett, Rachel Freedman, Justin Svegliato, and Stuart Russell. Active reward learning from multiple teachers. arxiv: 2303.0089) (es. March 2023. URL
Anthony Barrett, Jessica Newman, Brandie Nonnecke, Dan Hendrycks, Evan R Murphy, and Krys- tal Jackson. AT Risk-Management standards profile for General-Purpose AT systems (GPATS) and foundation models. Technical report, UC Berkeley Center for Long-term Cybersecurity, 2023. URL
Adrien Basdevant, Camille François, Victor Storchan, Kevin Bankston, Ayah Bdeir, Brian Behlendorf, Mer- ouane Debbah, Sayash Kapoor, Yann LeCun, Mark Surman, Helen King- Turvey, Nathan Lambert, Stefano Maffulli, Nik Marda, Govind Shivkumar, and Justine Tunney. Towards a framework for openness in foun- dation models: Proceedings from the columbia convening on openness in artificial intelligence. arXiv: 15802 SEs.SE], May 2024. URL hbtt::///arriivoorggaab//225
Samyadeep Basu, Philip Pope, and Soheil Feizi. Influence functions in deep learning fragile. arXio: 2006. 14651 [cs.LG], June 2020. URL http:://aarii..ogg/bb///000///
Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Bider- man. LEACE: Perfect linear concept erasure in closed form. In 37th Conference on Neural Infor- mation Processing Systems (NewrIPS 2023), New Orleans, LA, USA, November 2023. URL https: JuXXcffMrr
Yohsua Bengio, Danicl Privitera, Tamay Besiroglu, Rishi Bommasani, Stephen Casper, Yejin Choi, Danielle Goldfarb, Hoda Heidari, Leila Khalatbari, Shayne Longpre, Vasilios Mavroudis, Mantas Mazeika, Kwan Yee Ng, Chinasa T Okolo, Deborah Raji, Theodora Skeadas, Florian Tramer, and Soren Minder- mann. International scientific report on the safety of advanced AI. Technical report, Department for Science, Innovation and Technology, 2024. URL
Bennett Institute for Applied Data Science. About OpenSAFELY, 2024. www.opensafely.org/ about// Accessed: 2024-7-17.
URL https://
Tony Berber Sardinha. Al Al-generated vs human-authored texts: A multidimensional comparison. Applied
Corpus Linguistics, 4(1):000083, April 2024. ISSN 2666-7991. URL httpps//wwwaccenedddeecccoo
science/articcin ccfeen//aarracle//111///77711100
Leonard Bereska and Efstratios Gavves. Mechanistic interpretability for AI safety a review. arxiv:
2404.14082 ATS, April 2024. URL httpp:://aartt.oor///////...0.00
53

---

Published in Transactions on Machine Learning Research (04/2025)
Jamie Bernardi, Gabriel Mukobi, Hilary Greaves, Lennart Heim, and Markus Anderljung Societal adaptation to advanced AT. Xixi: 2405. 10295 fcs. CY), May 2024. URL htttp://arxxv.rrgg//// http://xx.v..rr//////////1119999
Tamay Besiroglu, Sage Andrus Bergerson, Amelia Michael, Lennart Heim, Xueyun Luo, and Neil Thomp- son. The compute divide in machine learning: A threat to academic contribution and scrutiny? arXiv: 2401.02452 (es.CY; January 2024. doi: (18850/arXivv / 2401.02522.
Stella Biderman, Hailey Schoelkopf, Lintang Sutawika, Leo Gao, Jonathan Tow, Baber Abbasi, Alham Fikri Aji, Pawan Sasanka Ammanamanchi, Sidney Black, Jordan Clive, Anthony DiPofi, Julen Etxaniz, Ben- jamin Fattori, Jessica Zosa Forde, Charles Foster, Jeffrey Hsu, Mimansa Jaiswal, Wilson Y Lee, Haonan Li, Charles Lovering, Niklas Muennighoff, Ellie Pavlick, Jason Phang, Aviya Skowron, Samson Tan, Xi- angru Tang, Kevin A Wang, Genta Indra Winata, François Yvon, and Andy Zou. Lessons from the trenches on reproducible evaluation of language models. arXiv: 2405. 14782 CLJ, May 2024. URL http://arxiv.. http:/v.orrr/abb/240044447
Baltista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. In Proceedings of the 29th International Coference on International Conference on Machine Learning. ICML'12, pp. 1467-1474, Madison, WI, USA, June 2012. Omnipross. ISBN 9781450312851. URL https:
Abeba. Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: Misogyny, pornog- raphy, and malignant stereotypes. arXiv: 2110.01963 [cs.CY]. October 2021. URL http://arxiv.org/
abs/2110.01963,
Abeba. Birhane, Vinay Prabhu, Sang Han, Vishmu Naresh Boddeti, and Alexandra Sasha Luccioni. Into the LAIONs den: Investigating hate in multimodal datasets. arXiv: 2311.03449 [cs.CY], November 2023. URL
Teemu Birkstedt, Matti Minkkinen, Anushree Tandon, and Matti Mäntymäki. Ai governance: themes, knowledge gaps and future agendas. Internet Research, 33(7):133 167, 2023.
Cody Blakeney, Mansheej Paul, Brett W Larsen, Sean Owen, and Jonathan Frankle. Does your data spark joy? Performance gains from domain upsampling at the end of training. arXiv: 2406.03476 [es.I.Gf, June 2024. URL http::/tt//ttt http:p:/aaxivvoo..g. httgg/bb/224000 22////2000007376
Su Lin Blodgett, Solon Barocas, Hal Dauné, III, and Hanna Wallach. Language (technology) is power: A critical survey of "bias" in NLP. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Association for Compulational Linguistics (ACT 2020), pp. 5454 5476, Virtual, July 2020. Association for Computational Linguistics. URL https:/7 aclanthology.org//oooo wwwp:oogyy.rr//02..a..../ -aa.acc.mmmma88
Emma Bluemke, Tantum Collins, Ben Garfinkel, and Andrew Trask. Exploring the relevance of data Privacy- Enhaneing technologies for AT governance use cases. arXio: 2303. 08956 (cs.AT], March 2023. URL
http://aariiv.rr//bbs/2000
Miranda Bogen and Amy Winccoff. Applying sociotechnical approaches AI governance practice. Techni- cal report, Center for Democracy & Technology, May 2024. URL /tttps://cdtt..o www.rrrrgggggggnnnnerreeeeeeeeee happss/gppplgggg..
Rishi Bommasani, Kevin Klyman, Shayne Longpre, Sayash Kapoor, Nestor Maslej, Betty Xiong, Daniel Zhang, and Percy Liang. The foundation model transparency index. Technical report, Center for Rescarch on Foundation Models (CRFM) and Institute on Human-Centered Artificial Intelligence (HAT), October 2023a. URL http:://arrxwoorr//////2000
Rishi Bommasani, Kevin Klyman, Daniel Zhang, and Percy Liang. Do foundation model providers comply with the draft EU AI act? Technical report, Stanford Center for Research on Foundation Models, 2023b.
54

---

Published in Transactions on Machine Learning Research (04/2025)
Rishi Bommasani, Dilara Soylu, Thomas I Liao, Kathleen A Creel, and Percy Liang. Ecosystem graphs: The social footprint of foundation models. arXiv: 2303. 15772 [es.T.]/ March 2023c. doi: 10.48550/ arXiv:230.11577.
Rishi Bommasani, Kevin Klyman, Shayne Longpre, Betty Xiong, Sayash Kapoor, Nestor Maslej, Arvind Narayanan, and Percy Liang. Foundation model transparency reports. arXiv: 2402.16268 Les. Febru- ary 2024. URL nttp://aazii.orrr/a///222..11.
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 141-159, Virtual, May 2021. IEEE. ISBN 0781728189345 9781728189322. URL http:///a.ddo..rr//11111////////
Samuel R Bowman and George Dahl. What will it take to fix benchmarking in natural language understand- ing? In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkami-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (ods.), Proceedings of the 2021 Confer- ence of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACI-HIT 2021), pp. 4843-4855, Online, June 2021. Association for Computational Lin- guistics. URL https://aclaatthll..g org. 2021 mwacl-main.n..a
Elizabeth H Bradley, Leslie A Curry, and Kelly J Devers. Qualitative data analysis for health services research: developing taxonomy, themes, and theory. Health services research, 42(4):1758 2007.
Asher Brass and Onni Aarne Location verification for Al chips. Technical report, Institute for AI Policy and Strategy, 2024. http:////sssseaaaarooooooooottaa 66706/762244711155510/7766688122 6at.7h/aaaaaatttl1ooo.oaaataatea
Paul Brieman. Hashmarks: Privacy-Preserving benchmarks for High-Staaes Al evaluation. arvi:
2312.00645 [cs.LGj, December 2023. URL http:///axxvvorrr/////2222..0000
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. What does it mean for a language model to preserve privacy? Tn Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transperency (FAceT '22), pp. 2280 2292, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 9781450393522. URL httpss//dalacc.rrr//////00000 8531146.3534642
Miles Brundage, Shahar Avin, Jasmine Wang, Haydn Belfield, Gretchen Krueger, Gillian Hadfield, Heidy Khlaaf, Jingying Yang, Helen Toner, Ruth Fong, Tegan Maharaj, Pang Wei Koh, Sara Hooker, Jade Leung, Andrew Trask, Emma Bluemke, Jonathan Lebensold, Cullen O'Keefe, Mark Koren, Théo Ryffel, J B Rubinovitz, Tamay Besiroglu, Federica Carugati, Jack Clark, Peter Eckersley, Sarah de Haas, Maritza Johnson, Ben Laurie, Alex Ingerman, Igor Krawezuk, Amanda Askell, Rosario Cammarota, Andrew Lohn, David Krueger, Charlotte Stix, Peter Henderson, Logan Graham, Carina Prunkl, Bianca Martin, Elizabeth Seger, Noa Zilberman, Seán Ó hEigeartaigh, Frens Kroeger, Girish Sastry, Rebecca Kagan, Adrian Weller, Brian Tse, Elizabeth Barnes, Allan Dafoe, Paul Scharre, Ariel Herbert- Voss, Martijn Rasser, Shagun Sodhani, Carrick Flynn, Thomas Krendl Gilbert, Lisa Dyer, Saif Khan, Yoshua Bengio, and Markus Anderljung. Toward trustworthy AI development: Mechanisms for supporting verifiable claims. arXio: 2004.07213 [cs.C1 CYJ, April 2020. URL htttp://arxiv.ogg/aaaaaa wttp:o////bbbb../0/....1
Benjamin S Bucknall and Robert F Trager. Structured access for third-party research on frontier ai models: Investigating rescarchers' model access roquirements. Technical report, Oxford Martin School, Univer- sily of Oxford and Center for the Governance of AI, October 2023. URL
Structured Accelccccessseeee for Third-Parte Research.pdf
Justin B Bullock, Yu-Che Chen, Johannes Himmelreich, Valerie M Hudson, Anton Korinek, Matthew M Young, and Baobao Zhang (eds.). The Orford Handbook of AT Governance. Oxford University Press, London, England, 1 edition, February 2022. ISBN 9780197579329, 9780197579350.
URL https:
//
55

---

Published in Transactions on Machine Learning Research (04/2025)
Bureau of Industry and Security. Commerce implements new export controls on advanced computing and semiconductor manufacturing items to the people's republic of china (PRC). Technical report, U.S. De- partment of Commerce, 2022a.
Bureau of Industry and Security. Implementation of additional export controls: Certain advanced com- puting and semiconductor manufacturing items; supercomputer and semiconductor end use; entity list modification, October 2022b. URL http:://ww...darraaeggsoooo////p
21658
Bureau of Industry and Security. Implementation of additional export controls: Certain advanced computing
items; supercomputer and semiconductor end use; updates and corrections, October 2023. URL https:
URL //w.f//derrrrrgssseeoo/o/////22 23055
Bureau of Industry and Security. Commerce control List Supplement no. 1 to part 774 category 4, 2024.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv: 2212.03827 [cs.CLi, CLJ, December 2022. doi: 20.44850//rXiv.2212..mm
C2PA. Overview, 2022. URL httts::/c2pa.oorg Accessod: 2024-7-17.
Pedro Cano, Eloi Batlle, Ton Kalker, and Jaap Haitsma. A review of audio fingerprinting. Journal of VLSI signal processing systems for signel, image, and video technology. 41(3):271-284, November 2005. ISSN 1387-5485, 0922-5773.
Nicholas Carlini. Poisoning the unlabeled dataset of May 2021. URL
learning, arXiv: 2105.01622 [cs.LG].
Nicholas Carlini and Andreas Terzis. Poisoming and backdooring contrastive learning. arxiv: 2106.09667 Gej, June 2021. URL http:://arxivvorr//b//22..... 09667
Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19), pp. 267 284, Santa Clara, CA, August 2019. USENIX Association. ISBN 9781939133069. URL https: www. http:///ww.oo.ooooooooo./e/////a
Nicholas Carlini, Matthew Jagielski, and Tlya Mironov. Cryptanalytic extraction of neural network models. In Daniele Micciancio and Thomas Ristenpart (ods.), Advances in Cryptology - CRYPTO 2020, pp. 189- 218, Cham, 2020. Springer International Publishing. ISBN 9783030568771. URL http://dxx.oi.oorg
11.1007/978-33000-666877
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Berbert-Vosss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ülfar Erlingsson, Alina Oprea, and Colin Raffel. Extract- ing training data from large language models. In 30th USENIX security symposium (USENIX se- curity 21), pp. 2633 2650. USENIX Association, August 2021. ISBN 9781939133243. 7/ www..usemix.c org/ crence/ene///emo httpe/iissccutttt///ersssaaaaaaa car
URL https:
Nicholas Carlini, Matthew Jagielski, Chiyuan Zhang, Nicolas Papernot, Andreas Terzis, and Florian Tramer. The privacy onion effect: Memorization is relative. In S Koyejo, S Mohamed, A Agarwal, D) Belgrave, K Cho, and A Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 13263 13276. Curran Associates, Inc., 2022. URL /tt.:ttppp//pppcceein...eee.eear 5664ff8888a466000e6eee1333333333
fiieer_files/ppee filles//pppr//222//iill//
Nicholas Carlini, Matthew Jagielski, Christopher A Choquelte-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramer. Poisoning Web-Scale training datasets is [es.CRj, February 2023a. URL http://aaxx..rr////s///00....000
practical. arXiv: 2302.10149
Nicholas Carlini, Milad Nasr, Christopher A Choquette- Choo, Matthew Jagielski, Irena Gao, Pang Wei Koh, Daphne Ippolito, Florian Tramèr, and Ludwig Schmidt. Are aligned neural networks adversarially aligned? In 37th Conference on Neural Information Processing Systems (NeurIPS 2023). New Orleans, LA, USA,
November 2023b. URL httt:://ggaaaeeeisseeeessssssssg
56

---

Published in Transactions on Machine Learning Research (04/2025)
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Fric Wallace, David Rolnick, and Florian Tramer. Stealing part of a production language model. arxi:: 2403.06634 fes. CRJ, March 2024 doi: 10.48550/ 10.44550/arxiv arvv 2403.06634.
Nicolas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Schwag, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. Extracting training data from diffusson models. In 32nd USENIX secu- rily symposium (USENIX securily 23), pp. 5253 5270, Anaheim, CA, August 2023c. USENIX Association. ISBN 9781939133373. URL https: http://wwwwueeixxxrre/.e wrt.a//www......oooooooo//aooaaa
carlini
Andres Carranza, Dhruv Pai, Rylan Schaeffer, Arnuv Tandon, and Sanmi Koyejo. Deceptive alignment monitoring, arXiv: 2307. 10569 [es.LG], July 2023. URL http:///ar.....rg////////./.... http:///aabba07...105550
Stephen Casper, Jason Lin, Joe Kwon, Gatlen Culp, and Dylan Hadiield-Menell. Explore, establish, exploit: Red teamiing language models from scratch. Xiv: 2306.09443 Cs,,, June 2023. doi: 10.48550/
MrXXiv2306.09442
Stephen Casper, Carson Ezell, Charlotte Siegmann, Noam Kolt, Taylor Lynn Curtis, Benjamin Bucknall, Andreas Haupt, Kevin Wei, Jérémy Scheurer, Marius Hobbhahn, Lee Sharkey, Satyapriya Krishna, Marvin Von Hagen, Silas Alberti, Alan Chan, Qinyi Sun, Michael Gerovitch, David Bau, Max Tegmark, David Krueger, and Dylan Hadfield-Menell.. Black-Box access is insufficient for rigorous AT audits. arXiv: 2401.14446 CYs, January 2024a. URL http:::/aaaxx.oorra/////......
Stephen Casper, Lennart Schulze, Oam Patel, and Dylan Hadfield-Menell. Defending against unforeseen failure modes with latent adversarial training, arXiv: 2403.05030 [cs.CR], March 2024b. doi: 10.48550/
arXiv:2403.0503:
Simon Caton and Christian Haas. Fairness in machine learning: A survey. ACM Comput. Surv., 56(7):1-38 April 2024. ISSN 0360-0300. URL http::///6....rg///..11////111
Iker Ceballos, Vivek Sharma, Eduardo Mugica, Abhishek Singh, Alberto Roman, Praneeth Vepakomma, and Ramesh Raskar. Nddivenn vertical partitioning, arXiv: 2008.04137, [es.LG], August 2020. URL http://aaxxvoorrraab///0..111111
Alan Chan. Evaluating predictions of model behaviour, 2024. URL http::/www.ovvrraccc.///pptt wwaluaating-prediccioo-----
Accessed: 2024-7-16.
Alan Chan, Chinasa T Okolo, Zachary Terner, and Angelina Wang. The limits of global inclusion in Al
development. arXiv: 2102.00 01265 CYs, February 2021. URL http:://aaxxv.oog//aa////..01111
Alan Chan, Maxime Riché, and Jesse Clifton. Towards the scalable evalattio of cooperativenoss in language models. Xív: 2303. 13360 fcs. CLJ, March 2023a. URL http:://araav.orr/ab/////// /303a././/333333
Alan Chan, Rebecca. Salganik, Alva Markelius, Chris Pang, Nitarshan Rajkumar, Dmitrii Krasheninnikov, Lauro Langosco, Zhonghao He, Yawen Duan, Micah Carroll, Michelle Lin, Alex Mayhew, Kather- ine Collins, Maryam Molamohamnadi, John Burden, Wanru Zhao, Shalaleh Rismani, Konstantinos Voudouris, Umang Bhatt, Adrian Weller, David Krueger, and Tegan Maharaj. Harms from increasingly agentic algorithmic systems. In Proceedings of the 2023 ACM Conference on Fairness, Accountabililyy and Transporency (FAceT 223, pp. 651-666, New York, NY, USA, June 2023b. Association for Computing Machinery. URL https:://// /atl:://lllcccmm
Alan Chan, Carson Ezell, Max Kaufmann, Kevin Wei, Lewis Hammond, Herbie Bradley, Emma Blummke, Nitarshan Rajkumar, David Krueger, Noam Kolt, Lennart Heim, and Markus Anderljung. Visibility into Al agents. arxiv: 2401.13131 /cs.CY), January 2024. doi: 10.48550/ar /tt//aarxvv..21.113.
Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang. Wei Ye, Yue Zhang, Yi Chang, Philip S Yu, Qiang Yang, and Xing Xic. A survey on evaluation of large language models. ACM transactions on intelligent systems and technology, 15(3):39:1-399) -9:45,, March 2024. ISSN 2157-6904, 2157-6912. URL httpsss//oo..orr///11/////2222
57

---

Published in Transactions on Machine Learning Research (04/2025)
Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J Pappas, Florian Tramer, Hamed Hassani, and Eric Wong, BeinhreakBench: An open robustness benchmark for jailbreaking large language models. arXiv: 2404.01918 [R), March 2024. URL http: / /ttp://aw..rrr//////00..00003030
Brian J Chen and Jacob Metcalf. Explainer: A sociotechnical apppa to Al policy. Technical report, Data & Society, 2024. URL
policy/
Huili Chen, Cheng Fu, Bita Darvish Rouhani, Jishen Zhao, and Farinaz Koushanfar. DeepAttest: an end-to- end attestation framework for deep neural networks. In Proceedings of the 46th International Symposium on Computer Architecture, ISCA '19, pp. 487-498, New York, NY, USA, June 2019. Association for Computing Machinery. ISBN 9781450366694 URL http://pp//.oaoooo/////////////0
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, lon Stoica, and Eric P Xing. Vicuna: An Open-Source chatbot impressing GPT-4 with 90%* * ChatGPT quality, March 2023. URL https:://mmss..org/bbll.. 2023-03-30-vicuna/
Hyeongmin Cho and Sangkyun Lee. Data quality measures and efficient evaluation algorithms for Large- Scale High-Dimenssonal data. arxiv: 2101.01441 [cs.LG], January 2021. URL http://arxiv.org/aan
2101.01441
Sang Keun Choe, Hwijeen Ahn, Juhan Bae, Kewen Zhao, Minsoo Kang, Youngseog Chung, Adithya Pratapa, Willie Neiswanger, Emma Strubell, Teruko Mitamura, Jeff Schneider, Eduard Hovy, Roger Grosse, and Eric Xing. What is your data worth to GPT? LLM-Scale data valuation with influence functions. arxi 13954 Lcs, May 2024. URL http://arrdv.rr//abs//46666.
Dami Choi, Yonadav Shavit, and David K Duvenaud. Tools for verifying neural models' training data. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023) Main Conference Track, volume abs/2307.000 2024, July 2023. URL. httl:http:// http:////x.ooooorr/000005///////
Alexandra Chouldechova. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153 163, June 2017. ISSN 2167-647X, 2167-6461. URL http:///dx.di.org// 10. 1..089/bbg.2066.0444
Miranda Christ, Sam Gunn, and Or Zamir. Undetectable watermarks for language models. arXio: 2306. 2306609194 [cs.CR) May 2023. doi: 10.48550/ar 2..550/arXiv.225 2rXiv.2306.09194
Jack Clark. Information markets and AT development. In The Oaford Handbook of AT Governance, pp. 345 357. Oxford University Press, January 2023. ISBN 9780197579329, 9780197579350. URL https:
Jeremy Cohen, Elan Rosenfeld, and Zico Kolter. Certified adversarial robustness via randomized smoothing. In Proceedings of the 36th Internalional Conference on Machine Learning, pp. 1310-1320. PMLR, May 2019. URL http://prroceddng.n....rreeee//s
Michael K Cohen, Noam Kolt, Yoshua Bengio, Gillian K Hadfield, and Stuart Russell. Regulating advanced artificial agents. Science, (54(6691):36-355 April 2024a. URL
Sciiene.add0060
Roi Cohen, Eden Biran, Ori Yoran, Amir Globerson, and Mor Geva. Evaluating the ripple effects of knowl- edge editing in language models. Transactions of the Association for Compulational Linguistics, 12:283- 298, April 2024b. ISSN 2307-387X. URL http:p/:s/cceeccaaaaaaaa//a///ap
tacl a 020004442362222/taaaa tacp:/e22222aaccc.cc.e Lataclla000000 0__00644.pdf
Commerce Department. Taking additional steps to address the national emergenncy with respect to significant malicious Cyber-Fnabled activities, 01 2024. URL http:://wwwwffeeeeanrgggaeeerr/p
58

---

Published in Transactions on Machine Learning Research (04/2025)
Content Authenticity Initiative. Restoring trust and transparency in the age of AI, 2024. URL https: //cortartautbetticcttyror// Accessed: 2024-7-17.
Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva.
On the detection of synthetic images generated by diffuuson models. In ICASSP 2023 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1 5, June 2023. URL https:///eeexplorr.eees.
document/10095167
Council of the European Union. REGULATION OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL laying down harmonised rules artificial intelligence and amending regulations (EC) no 300/2008, (EU) no 167/2013, (EU) no 168/2013, (EU) 2018/858, (EU) 2018/1139 and (EU) 2019/2144 and directives 2014/90/EU, (EU) 2016/797 and (EU) 2020/1822 1828 (artificial intelligence act), 2024. URL
2------------------2 IN14-INIT/en/pdf
Andrew Critch and Stuart Russell. TASRA: A taxonomy and analysis of Societal-Scale risks from Al. ar 2306.06924 [cs.Al], June 2023. URL http://axxxvvorrr/a///20.0..002
Rachel Crowell. Why Al's diversity crisis matters, and how to tackle it. Nature, May 2023. ISSN 0028-0836, 1476-4687. URL
Cybersecurity and Infrastructure Security Agency. Secure by design, 2023. URL https::/www.cisa..tp
securebydesign Accessed: 2024-7-18.
Allan Dafoe. AI governance: A research agenda. Technical report, Centre for the Governance of AI & Future of Humanity Institute, University of Oxford, 2018. URL
Aalloadd/GovAI-Agendaaa..
Morten Dahl. Private deep learning with MPC: A simple tutorial from scratch, 2017. URL https:// Accessed: 2024-7-17.
David Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohun- dro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, and Joshua Tenenbaum. Towards guaranteed safe AT: A framework for ensuring robust and reliable Al systems. arXiv: 2405.06624 [cs.Al], May 2024. URL
Angela Daly, Thilo Hagendorff, Li Hui, Monique Mann, Vidushi Marda, Ben Wagner, Wayne Wei Wang, Hans- W Micklitz, Oreste Pollicino, Amnon Reichman, Andrea Simoncini, Giovanni Sartor, and Giovanni De Gregorio. AT, governance and ethics. In Constitutional Challenges the Algorithmii Sociely, pp. 182- 201. Cambridge University Press, December 2021. URL http:://www..abrrrrggggg//////pp
(3008054776655556777760666
Tom Davidson, Jean-Stanislas Denain, Pablo Villalobos, and Guillem Bas. Al capabilities can be sig- nificantly improved without expensive retraining. Technical report, Epoch, December 2023. URL
http://arxiv.or//abb//
Mostafa Dehghani, Yi Tay, Alexey A Gritsenko, Zhe Zhao, Neil Houlsby, Fernando Diaz, Donald Metzler, and Oriol Vinyals. The benchmark lottery. arvi 2107.07002 [cs.LGG], July 2021. URL http://arxiv.org/ abs/2107.07002
Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang. The participatory turn in AI design: Theoretical foundations and the current state of practice. In Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '23), pp. 1-23, New York, NY, USA, October 2023. Association for Computing Machinery. URL 3617694.3623261
Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. Investigating data contam- ination in modern benchmarks for large language models. orXiv: 2311.00 9311.09783 (cs.CL) November 2023. URL htttp://arxivoor//apaa
59

---

Published in Transactions on Machine Learning Research (04/2025)
Jiangyi Deng, Shengyuan Pang, Yanjiao Chen, Liangming Xia, Yijie Bai, Haiqin Weng, and Wenyuan Xu. SOPHON: Non-Fine- Tunable learning to restrain task transferability for pre-trained models. arxiv: 2404.12699 Les.LG], April 2024. URL http:///arxxvorrgg/a///2....00
Mingkai Deng, Jianyu Wang, Cheng--Pnn Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. RLPrompt: Optimizing discrete text prompts with reinforcement learning. arxiv: 2205.12548 (es. CLJ, May 2022. URL
Department for Science, Innovation & Technology.
nical report, GOV.UK, October 2023. URL Emerging processes for frontier Al safety, Tech- /ttpabbbd088400000f//bbb//eeeee
Department for Science, Innovation & Technology and AI Safely Institute. Advanced AI evaluations at AISI: May update. Technical report, GOV.UK, 2024. URL a1 ai-evaluations- waapaaatttoooo--uuaaauaaa- may-up
Department for Science, Innovation and Technology and Office for Artificial Intelligence. A pro-innovation approach to Al regulation. Technical report, GOV.UK, August 2023. URL https://wwww http:://www....uk/
Department for Science, Innovation and Technology, Forcign, Commonwealth & Development Office, and Prime Minister's Office, 10 Downing Street. The bletchley declaration by countries atlending the Al safely summit, 1-2 november 2023. Technical report, GOV, November 2023a. URL /twp://wwwww.../kk//
bletehhheeeteeeeeceeeeeeeee atw.attrrioe-attttin-------t ai- 1
Department for Science, Innovation and Technology, Foreign, Commonwealth Development Office, and Prime Minister's Office 10 Downing Street. Chair's summary of the AI safety summit 2023, bletchley park. Technical report, GOV.UK, 2023b. URL httppp/////www.oo....//////////t
2023 1923-bletchley-par/
Gobikrishna
Dhamuskodi, Sudeshna Guha, Vidhya Krishnan, Aruna Manjunatha, Michael O'Connor, Rob Nertney, and Phil Rogers. Creating the first confidential GPUs: The team NVIDIA brings confiden- tiality and integrity to user code and data for accelerated computing. Queueing Systems. Theory and Applications, 21(4):68 93, September 2023. ISSN 0257-0130, 1542-7730. URL BtLL:ttts://dooorrg//10 3628593.3628391
Digital Services Act. Regulation (EU) 2022/ 2065 of the Europcan Parliament and of the Council of 19 October 2022 on a Single Market For Digital Services and amending Directive 2000/31/EC (Digital Services Act) (Text with EEA relevance), October 2022. URL
eng
Roel Dobbe and Anouk Wolters. Toward sociotechnical AI: Mapping vulnerabilities for machine learning in context. Minds and Machines, 34(2):12, May 2024. ISSN 0924-6495, 1572-8641. URL https://doi.org/
1011077/11003-024-96688
Roel I J Dobbe, Thomas Krendl Gilbort, and Yonatan Mintz. Hard choices in artificial intelligenc:: Ad- dressing normative uncertainty through sociotechnical commitments (AIES '20). In Proceedings of the AAAT/ACM Conference on AT, Fthies, and Sociely, pp. 242, New York, NY, USA, February 2020. Associa- tion for Computing Machinery. ISBN 9781450371100. URL htt.l//pee//0a000//////1//////77
Mateusz Dolata, Stelan Feuerriegel, and Gerhard Schwabe. ness. Information systems journal, 3244)8754-818 2022. ISSN 1350-1917, 1365-2575. URL https:
sociotechnical view of algorithmic fair-
Yi Dong, Ronghui Mu, Yanghao Zhang, Siqi Sun, Tianle Zhang, Changshun Wu, Gaojic Jin, Yi Qi, Jinwei Hu, Jie Meng, Saddek Bensalem, and Xiaowei Huang. Safeguarding large language models: A survey. arxiv: 3,06.02622 fes. CRJ, June 2024a. URL httppp///aaaiiiorr///////......
60

---

Published in Transactions on Machine Learning Research (04/2025)
Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li. Generalization mem- orization: Data contamination and trustworthy evaluation for large language models. arXiv: 2402. 15938 (es. CLJ, February 2024b. URL http://aarxtvorr///ss/2202002212
Arthur Douillard, Qixuan Feng, Andrei Rusu, Adhiguna Kuncoro, Yani Donchev, Rachita Chhaparia, Ionel Gog, Marc'aurelio Ranzato, Jiajun Shen, and Arthur Szlam. DiPaCo: Distributed path composition. arxiv: 2403. 10616 [es.LG], March 2024. URL http://aarrcv.rrr/daa//221....10
Anca Dragan, Helen King, and Allan Dafoe. Introducing the frontier t framework, 2024. https///
Accessed: 2024-7-
12.
Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sowon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. Do membership inference atlacks work large language models? arXiv: 2402.07841 CLJ, February 2024a. URL hutp:://arxvvorr//abl
2402.07841
Sunny Duan, Mikail Khona, Abhiram Iyer, Rylan Schaeffer, and lla R Fiete. Uncovering latent memories: Assessing data leakage and memorization patterns in large language models. arXiv: 14549 (vs.CV), June 2024b. URL
14549
Zane Durante, Qiuyuan Huang, Naoki Wake, Ran Gong, Jae Sung Park, Bidipta Sarkar, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Yejin Choi, Katsushi lkeuchi, Hoi Vo, Li Fei-Fei, and Jianfeng Gao. Agent AI: Surveying the horizons of multimodal interaction. arXiv: 2401. 03568 [cs.Al], January 2024.
URL
Dynabench. Challenging the limits of benchmarking AI, 2023. URL https://éynabeenchoog org/ 2024-7-16.
Accessed:
Colin Eden, Fran Ackermann, Charles B Finn, and John M Bryson. Visible Thinking: Unlocking Causal Mapping for Practical Business Resulls. John Wiley & Sons, 2004. ISBN 9780470869161. URL https
Janet Egan and Lennart Heim. Oversight for frontier Al through a Know- Costooe scheme for compute providers. arXiv: 2310.13625 fes. CYY, October 2023. URL http:///arxivorr//as//22/11
Yanai Elazar, Akshita Bhagia, Ian Helgi Magnusson, Abhilasha Ravichander, Dustin Schwenk, Alanc Suhr, Evan Pele Walsh, Dirk Groeneveld, Luca Soldaini, Sameer Singh, Hannaneh Hajishirzi, Noah A Smith, and Jesse Dodge. What's in my big data? In The Twelfth International Conference on Learning Repre- sentations, 2024. URL
Ronen Eldan and Mark Russinovich. Who's harry potter? Approximate unlcarning in LLMs. arxiv: 2310.02238 (es.CL). October 2023. doi: 10.48550/ar (00/550///Xiv... 2t..iiv.31..0223...
Tyna Eloundou, Sam Manning, Pamela Mishkin, and Daniel Rock. GPTs are GPTs:: Labor market impact potential of LLMs. Science, 384(6700) June 2024. ISSN 0036-8075, 1095-9203. URL http:
Epoch. Key trends and figures in machine learning, 2023. URL https://ggpccha..rg//reedddd
EU Joint Research Centre. FAQs: DSA data access for rescarchers, 2023. URL httpsts/////lgrrithh...cc
[ttpppaww..........o.......rrerr 2024-7-17.
Accessed:
European Commission, EU general data protection regulation (GDPR): Regulation (EU) 2016/679 of the curopean parliament and of the council of 27 april 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing directive 95/46/EC EEC (general data protection regulation). OJ 2016 L 119/1, 59:1 88, May 2016.
61

---

Published in Transactions on Machine Learning Research (04/2025)
European Commission. Commission welcomes G7 leaders' agreement guiding principles and code of conduct on artificial intelligence, October 2023. URL https://ligggall-ssaaateeee.eeee
artificial Accessed: 2024-7-12.
David Evans, Vladimir Kolesnikov, and Mike Rosulek. A Pragmatic Introduction to Secure Multi-Party
Computation. Now Foundations and Trends, 2018. ISBN 9781680835083, 9781680835090. URL http:
//da.dol.orgg11.155//30000000000
Executive Office of the President. Safe, secure, and trustworthy development and use of artificial intelligence, November 2023. URL
Congyu Fang, Hengrui Jia, Anvith Thudi, Mohammad Yaghini, Christopher A Choquctte-Choo, Natalic Dullerud, Varun Chandrasekaran, and Nicolas Papernot. Proof-ofLLearinng is currently more broken than you think. In 2023 IEEE 8th European Symposium on Security and Privacy (EuroS&P), pp. 797-816. IEEE, July 2023. ISBN 9781665465120, 9781663465137. URL http:://d..do..rgg/10... EuroSP57164 0177577164222300055555
Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. LLM agents can autonomously hack websites. arXiv: 2402.06605 [cs. CRJ, February 2024. doi: 11448500/ariit 2402.06664.
Uriel Fiege, Amos Fiat, and Adi Shamir. Zero knowledge proofs of identity. In Proceedings of the Nineteenth Annual ACM Symposium on Theory of Computing, pp. 210 207, 1987.
Tim Fist and Erich Grunewald. Preventing Al chip smuggling to china. Technical report, Center for a New American Security, October 2023. URL
ai ---chip-smuggingg to- china
Jordan Frery, Andrei Stoian, Roman Bredehoft, Luis Montero, Celia Kherfallah, Benoit Chevallier Mamees and Arthur Meyre. Privacy-Prosorving Treo-Basce inference with fully homomorphic eneryption, 2023. URL ltttp:://eprrnn.aaarro//////////
Meir Friedenberg and Joseph Y Halpern. Blameworthiness Multi-Agent settings. or Xin: 1903.04 04102
[es. CYs, March 2019. URL
Frontier Model Forum. Issue
brief:
Measuring training compute, 2024.
URL https://
www www.rrorrteemmeeeerrrroooo/a//pp htgp:///eesestbbbreeeeeerooaranr
Accessed: 2024-
7-18.
G7. Hiroshima process international guiding principles for organizations developing advanced ai system.
2023. URL htwp::/pswwwwoooa......////////0
G7 leaders. Hiroshima process international code of conduct for advanced Al systems, 2023. URL https:
7 www..oofaa..p//hheee////7773233
lason Gabriel, Arianna Manzini, Geoff Keeling, Lisa Anne Hendricks, Verena Rieser, Hasan Iqbal, Nenad Tomašev, Tra Ktena, Zachary Kenton, Mikel Rodriguez, Seliem El-Sayed, Sasha Brown, Canfer Akbulut, Andrew Trask, Edward Hughes, A Stevie Bergman, Renee Shelby, Nahema Marchal, Conor Griffin, Juan Mateos-Garcia, Laura Weidinger, Winnie Street, Benjamin Lange, Alex Ingerman, Alison Lentz, Reed En- ger, Andrew Barakat, Victoria Krakovna, John Oliver Siy, Zeb Kurth-Nelson, Amanda McCroskery, Vijay Bolina, Harry Law, Murray Shanahan, Lize Alberts, Borja Balle, Sarah de Haas, Yetunde Ibitoye, Allan Dafoe, Beth Goldberg, Sébastien Krier, Alexander Reese, Sims Witherspoon, Will Hawkins, Maribeth Rauh, Don Wallace, Matija Franklin, Josh A Goldstein, Joel Lehman, Michael Klenk, Shannon Vallor, Courtney Biles, Meredith Ringel Morris, Helen King, Blaise Agüera y Arcas, William Isaac, and James Manyika. The ethics of advanced AI assistants. Technical report, Google DeepMind, April 2024. URL http:///ariivorrg/as/220000
62

---

Published in Transactions on Machine Learning Research (04/2025)
Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, Andy Jones, Sam Bowman, Anna Chen, Tom Conerly, Nova DasSarma, Dawn Drain, Nelson Elhage, Sheer El-Showk, Stanislav Fort, Zac Hatfield-Dodds, Tom Henighan, Danny Hernandez, Tristan Hume, Josh Jacobson, Scott Johnston, Shauna Kravec, Catherine Olsson, Sam Ringer, Eli Tran-Johnson, Dario Amodei, Tom Brown, Nicholas Joseph; Sam McCandlish, Chris Olah, Jared Kaplan, and Jack Clark. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. Technical report, Authropic, November 2022. URL http:// arxiv.org /tt..oorr//bb//22000785555
Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptinization. In Proceedings of the 40th International Conference on Machine Learning, pp. 10835- Honolulu, Hawaii, USA, July 2023a. PMLR. URL https::/prooeeedngg..n https//eddiiaam....pee/e////////
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Lau- rence Golding, Jeffrey Hsu, Alain Le Noachh, Haonan Li, Kyle McDonell, Niklas Muennighofl, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Fric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. Im--evaluatioon-aarress A framework for few-shot language model evaluation, 2023b. URL
wttp://///eeooteeeeteeeeeeeeeeee evallaato wwweralalaliiiiaaaaeeeesses harness
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. arXiv: 2406. (00604093 [es.LG], June 2024. URL
Yansong Gao, Said F Al-Sarawi, and Derek Abbott. Physical unclonable functions. Nature Electronics, 3(2): 81 91, February 2020. ISSN 2520-1131, 2520-1131. URL htttap//www..aaauurrccc////rttts 020-0372-5
Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé Tii, and Kate Crawford. Datashects for datasets. Communications of the ACM, 64(12): 86 92, November 2021. ISSN 0001-0782. URL http:///l.....oor/////oo.../11//
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Jo- han Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, David Silver, Melvin Johnson, Ioannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Ruther- ford, Erica Moreira, Kareem Ayoub, Megha Goel, Jack Krawezyk, Cosmo Du, Ed Chi, Heng- Tze Cheng, Eric Ni, Purvi Shah, Patrick Kane, Betty Chan, Mannal Faruqui, Aliaksei Severyn, Hanzhao Lin, Yaguang Li, Yong Cheng, Abe Ittycheriah, Mahdis Mahdieh, Mia Chen, Pei Sun, Dustin Tran, Sumit Bagri, Bal- aji Takshminarayanan, Jeremiah Liu, Andras Orban, Fabian Güra, Hao Zhou, Xinying Song, Aurelien Boffy, Harish Ganapathy, Steven Zheng, Hyunjeong Choe, Ágoston Weisz, Tao Zhu, Yifeng Lu, Siddharth Gopal, Jarrod Kahn, Maciej Kula, Jeff Pitman, Rushin Shah, Emanuel Taropa, Majd Al Mercy, Martin Baeuml, Zhifeng Chen, Laurent El Shafey, Yujing Zhang, Olcan Sercinoglu, George Tucker, Enrique Pi- queras, Maxim Krikun, lain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, Alexandre Frechette, Charlotte Smith, Laura Culp, Lev Proleev, Yi Luan, Xi Chen, James Lottes, Nathan Schucher, Foderico Lebron, Alban Rrustemi, Natalie Clay, Phil Crone, Tomas Ko- cisky, Jeffrey Zhao, Bartek Perz, Dian Yu, Heidi Howard, Adam Bloniarz, Jack W Rae, Han Lu, Laurent Sifre, Marcello Maggioni, Fred Alcober, Dan Garrette, Megan Barnes, Shantanu Thakoor, Jacob Austin, Gabriel Barth-Maron, William Wong, Rishabh Joshi, Rahma Chaabouni, Deeni Fatiha, Arun Ahuja, Gaurav Singh Tomar, Evan Senter, Martin Chadwick, Ilya Kornakov, Nithya Attaluri, Iñaki Iturrate, Ruibo Liu, Yunxuan Li, Sarah Cogan, Jeremy Chen, Chao Jia, Chenjie Gu, Qiao Zhang, Jordan Grim- stad, Ale Jakse Hartman, Xavier Garcia, Thanumalayan Sankaranarayana Pillai, Jacob Devlin, Michael Laskin, Diego de Las Casas, Dasha Valter, Connic Tao, Lorenzo Blanco, Adríà Puigdomênech Badia, David Reitter, Mianna Chen, Jenny Brennan, Clara Rivera, Sergey Brin, Shariq Iqbal, Gabriela Surita, Jane Labanowski, Abhi Rao, Stephanie Winkler, Emilio Parisotto, Yiming Gu, Kate Olszewska, Ravi
63

---

Published in Transactions on Machine Learning Research (04/2025)
Addanki, Antoine Miech, Annie Louis, Denis Teplyashin, Geoff Brown, Elliot Catt, Jan Balaguer, Jackie Xiang, Pidong Wang, Zoe Ashwood, Anton Briukhov, Albert Webson, Sanjay Ganapathy, Smit Sanghavi, Ajay Kannan, Ming-Wat Wei Chang, Axel Stjerngren, Josip Djolonga, Yuting Sun, Ankur Bapna, Matthew Aitchison, Pedram Pejman, Henryk Michalewski, Tianhe Yu, Cindy Wang, Julictte Love, Junwhan Ahn, Dawn Bloxwich, Kehang Han, Peter Humphreys, Thibault Sellam, James Bradbury, Varun Godbole, Sina Samangooei, Bogdan Damoc, Alex Kaskasoli, Sébastien M R Arnold, Vijay Vasudevan, Shubham Agrawal, Jason Riesa, Dmitry Lepikhin, Richard Tanburn, Srivatsan Srinivasan, Hyeontaek Lim, Sarah Hodkinson, Pranav Shyam, Johan Ferret, Steven Hand, Ankush Garg, Tom Le Paine, Jian Li, Yujia Li, Minh Gi- ang, Alexander Neitz, Zaheer Abbas, Sarah York, Machel Reid, Elizabeth Cole, Aakanksha Chowdhery, Dipanjan Das, Dominika Rogozińska, Vitaliy Nikolaev, Pablo Sprechmann, Zachary Nado, Lukas Zilka, Flavien Prost, Luheng He, Marianne Monteiro, Gaurav Mishra, Chris Welty, Josh Newlan, Dawei Jia, Miltiadis Allamanis, Clara Huiyi Hu, Raoul de Liedekerke, Justin Gilmer, Carl Saroufim, Shruti Rijhwani, Shaobo Hou, Disha Shrivastava, Anirudh Baddopudi, Alex Goldin, Adnan Ozturel, Albin Cassirer, Yun- han Xu, Daniel Sohn, Devendra Sachan, Reinald Kim Amplayo, Craig Swanson, Dessie Petrova, Shashi Narayan, Arthur Guez, Siddhartha Brahma, Jessica Landon, Miteyan Patel, Ruizhe Zhao, Kevin Villela, Luyu Wang, Wenhao Jia, Matthew Rahtz, Mai Giménez, Legg Yeung, James Keeling, Petko Georgiev, Diana Mincu, Boxi Wu, Salem Haykal, Rachel Saputro, Kiran Vodrahalli, James Qin, Zeynep Cankara, Abhanshu Sharma, Nick Fernando, Will Hawkins, Behnam Neyshabur, Solomon Kim, Adrian Hutter, Priyanka Agrawal, Alex Mestro-Ros, George van den Driessche, Tao Wang, Fan Yang, Shuo-Yitt Chang, Paul Komarek, Ross Mcllroy, Mario Lučié, Guodong Zhang, Wael Farhan, Michael Sharman, Paul Natsev, Paul Michel, Yamini Bansal, Siyuan Qiao, Kris Cao, Siamak Shakeri, Christina Butterfield, Justin Chung, Paul Kishan Rubenstein, Shivani Agrawal, Arthur Mensch, Kedar Soparkar, Karel Lene, Timothy Chung, Aedan Pope, Loren Maggiore, Jackie Kay, Priya Jhakra, Shibo Wang, Joshua Maynez, Mary Phuong, Taylor Tobin, Andrea Tacchetti, Maja Trebacz, Kevin Robinson, Yash Katariya, Sebastian Riedel, Paige Bailey, Kefan Xiao, Nimesh Ghelani, Lora Aroyo, Ambrose Slone, Neil Houlsby, Xuehan Xiong, Zhen Yang, Elena Gribovskaya, Jonas Adler, Matco Wirth, Lisa Loe, Li, Music, Thais Kagohara, Jay Pava- gadhi, Sophie Bridgers, Anna Bortsova, Sanjay Ghemawal, Zafarali Ahmed, Tianqi Liu, Richard Powell, Vijay Bolina, Mariko linuma, Polina Zablotskaia, James Besley, Da- Woon Chung, Timothy Dozat, Ra- mona Comanescu, Xiance Si, Jeremy Greer, Guolong Su, Martin Polacck, Raphaäl Lopez Kanfman, Simon Tokumine, Hexiang Hu, Elena Buchatskaya, Yingjie Miao, Mohamed Elhawaty, Aditya Siddhant, Nenad Tomasev, Jinwei Xing, Christina Greer, Helen Miller, Shereen Ashraf, Aurko Roy, Zizhao Zhang, Ada Ma, Angelos Filos, Milos Besta, Rory Blevins, Ted Klimenko, Chih-Kuan Yeh, Soravit Changpinyo, Jiaqi Mu, Oscar Chang, Mantas Pajarskas, Carric Muir, Vered Cohen, Charline Le Lan, Krishna Haridasan, Amit Marathe, Steven Hansen, Sholto Douglas, Rajkumar Samuel, Mingqiu Wang, Sophia Austin, Chang Lan, Jiepu Jiang, Justin Chiu, Jaime Alonso Lorenzo, Lars Lowe Sjösund, Sébastion Cevey, Zach Gleicher, Thi Avrahami, Anudhyan Boral, Hansa Srinivasan, Vittorio Selo, Rhys May, Konstantinos Aisopos, Léonard Hussenot, Livio Baldini Soares, Kate Baumli, Michael B Chang, Adrià Recasens, Ben Caine, Alexander Pritzel, Filip Pavetic, Fabio Pardo, Anita Gergely, Justin Frye, Vinay Ramasesh, Dan Horgan, Kartikeya Badola, Nora Kassner, Subhrajit Roy, Ethan Dyer, Victor Campos Campos, Alex Tomala, Yunhao Tang, Dalia El Badawy, Elspeth White, Basil Mustafa, Oran Lang, Abhishek Jindal, Sharad Vikram, Zhitao Gong, Sergi Caelles, Ross Hemsley, Gregory Thornton, Fangxiaoyu Feng, Wojciech Stokowiec, Ce Zheng, Phocbe Thacker, Çağlar Unlü, Zhishuai Zhang, Mohammad Salch, James Svensson, Max Bileschi, Piyush Patil, Ankesh Anand, Roman Ring, Katerina Tsihlas, Arpi Vezer, Marco Selvi, Toby Shevlane, Mikel Rodriguez, Tom Kwiatkowski, Samira. Daruki, Keran Rong, Allan Dafoe, Nicholas FitzGerald, Keren Gu-Lemberg, Mina Khan, Lisa Anne Hendricks, Marie Pellat, Vladimir Feinberg, James Cobon-Kerr, Tara Sainath, Maribeth Rauh, Sayed Hadi Hashemi, Richard Ives, Yana Hasson, Eric Noland, Yuan Cao. Nathan Byrd, Te Hou, Qingze Wang, Thibault Sottiaux, Michela Paganini, Jean-Baptiste Lespiau, Alexan- dre Moufarek, Samer Hassan, Kaushik Shivakumar, Joost van Amersfoort, Amol Mandhane, Pratik Joshi, Anirudh Goyal, Matthew Tung, Andrew Brock, Haunah Sheahan, Vedant Misra, Cheng Li, Nemanja Rakićević, Mostafa Dehghani, Fangyu Liu, Sid Mittal, Junhyuk Oh, Seb Noury, Eren Sezener, Fantine Huot, Matthew Lamm, Nicola De Cao, Charlie Chen, Sidharth Mudgal, Romina Stella, Kevin Brooks, Gautam Vasudevan, Cheuxi Liu, Mainak Chain, Nivedita Melinkeri, Aaron Cohen, Venus Wang, Kristie Seymore, Sergey Zubkov, Rahul Goel, Summer Yue, Sai Krishnakumaran, Brian Albert, Nate Hurley, Motoki Sano, Anhad Mohananey, Jonah Joughin, Egor Filonov, Tomasz Kępa, Yomna Eldawy, Jiawern
64

---

Published in Transactions on Machine Learning Research (04/2025)
Lim, Rahul Rishi, Shirin Badiezadegan, Taylor Bos, Jerry Chang, Sanil Jain, Sri Gayatri Sundara Pad- manabhan, Subha. Puttagunta, Kalpesh Krishna, Leslie Baker, Norbert Kalb, Vamsi Bedapudi, Adam Kurzrok, Shuntong Lei, Anthony Yu, Oren Litvin, Xiang Zhou, Zhichun Wu, Sam Sobell, Andrea Sicil- iano, Alan Papir, Robby Neale, Jonas Bragagnolo, Tej Toor, Tina Chen, Valentin Anklin, Feiran Wang, Richie Feng, Milad Gholami, Kevin Ling, Lijuan Liu, Jules Walter, Hamid Moghaddamm, Arun Kishore, Jakub Adamek, Tyler Mercado, Jonathan Mallinson, Siddhinita Wandekar, Stephen Cagle, Eran Ofek, Guillermo Garrido, Clemens Lombriser, Maksim Mukha, Botu Sun, Hafeezul Rahman Mohammad, Josip Matak, Yadi Qian, Vikas Peswani, Pawel Janus, Quan Yuan, Leif Schelin, Oana David, Ankur Garg, Yifan He, Oleksii Duzhyi, Anton Ålgmyr, Témothée Lottaz, Qi Ti, Vikas Yadav, Luyao Xu, Alex Chinien, Rakesh Shivanna, Aleksandr Chuklin, Josie Li, Carrie Spadine, Travis Wolfe, Kareem Mohamed, Sub- habrata Das, Zihang Dai, Kyle He, Daniel von Dincklage, Shyam Upadhyay, Akanksha Maurya, Luyan Chi, Sebastian Krause, Khalid Salama, Pam G Rabinovitch, M, Pavan Kumar Reddy, Aarush Selvan, Mikhail Dektiarev, Golnaz Ghiasi, Erdem Guven, Himanshu Gupta, Boyi Liu, Deepak Sharma, Idan Heim- lich Shtacher, Shachi Paul, Oscar Akerlund, François-Xavier Aubet, Terry Huang, Chen Zhu, Erie Zhu, Elico Teixeira, Matthew Fritze, Francesco Bertolini, Eleonorre Marinescu, Martin Bölle, Dominik Paulus, Khyatti Gupta, Tejasi Latkar, Max Chang, Jason Sanders, Roopa Wilson, Xuewei Wu, Yi-Xuan Tan, Lam Nguyen Thict, Tulsee Doshi, Sid Lall, Swaroop Mishra, Wanming Chen, Thang Luong, Seth Benjamin, Jasmine Lee, Ewa Andrejczuk, Dominik Rabiej, Vipul Ranjan, Kraysztof Styre, Pengcheng Yin, Jon Simon, Malcolm Rose Harriott, Mudit Bansal, Alexei Robsky, Geoff Bacon, David Greene, Daniil Mirylenka, Chen Zhou, Obaid Sarvana, Abhimanyu Goyal, Samuel Andermatt, Patrick Siegler, Ben Horn, Assaf Israel, Francesco Pongetti, Chih-Wee "louis" Chen, Marco Selvatici, Pedro Silva, Kathie Wang, Jackson Tolins, Kelvin Guu, Roey Yogev, Xiaochen Cai, Alessandro Agostini, Manlik Shah, Hung Nguyen, Noah Ó Donnaile, Sébastien Pereira, Linda Friso, Adam Stambler, Adam Kurzrok, Chenkai Kuang, Yan Romanikhin, Mark Geller, Z J Yan, Kane Jang, Cheng- Chun Lee, Wojeiech Fica, Eric Malmi, Qijun Tan, Dan Banica, Daniel Balle, Ryan Pham, Yanping Huang, Diana Avram, Hongzhi Shi, Jasjot Singh, Chris Hidey, Niharika Ahuja, Pranab Saxena, Dan Dooley, Srividya Pranavi Potharaju, Eileon O'Neill, Anand Gokulchandran, Ryan Foley, Kai Zhao, Mike Dusenberry, Yuan Liu, Pulkit Mehta, Ragha Kotikalapudi, Chalence Kolhar, Chris Gorgolewski, Ali Tbrahim, Yang Song, Ali Eichenbaum, Thomas Brovelli, Sahitya Potluri, Preethi Lahoti, Cip Baetu, Ali Ghorbani, Charles Chen, Andy Crawford, Shalini Pal, Mukund Sridhar, Petru Gurita, Asier Mujika, Tgor Petrovski, Pierre-Louis Cedoz, Chenmei Li, Shiyuan Chen, Niccolò Dal Santo, Siddharth Goyal, Jitesh Punjabi, Karthik Kappaganthu, Chester Kwak, Pallavi Lv, Sarmishta Velury, Himadri Choudhury, Jamie Hall, Premal Shah, Ricardo Figucira, Matt Thomas, Minjic Lu, Ting Zhou, Chintu Kumar, Thomas Jurdi, Sharat, Chikkerur, Yenai Ma, Adams Yu, Soo Kwak, Victor Ähdel, Sujeevan Rajayogam, Travis Choma, Fei Liu, Aditya Barua, Colin Ji, Ho Park, Vincent Hellendoorn, Alex Bailey, Taylan Bilal, Huanjie Zhou, Mehrdad Khatir, Charles Sutton, Wojciech Rzadkowski, Fiona Macintosh, Konstantin Shagin, Paul Medina, Chen Liang, Jinjing Zhou, Pararth Shah, Yingying Bi, Attila Dankovics, Shipra Banga, Sabine Lehmann, Marissa Bredesen, Zifan Lin, John Fric Hoffmann, Jonathan Lai, Raynald Chung, Kai Yang, Nihal Balani, Arthur Bražinskas, Andrei Sozanschi, Matthew Hayes, Héctor Fernández Alcalde, Peter Makarov, Will Chen, Antonio Stella, Liselotte Snijders, Michael Mandl, Ante Kärrman, Pawel Nowak, Xinyi Wu, Alex Dyck, Krishnan Vaidyanathan, R, Raghavender, Jessica Mallet, Mitch Rudominer, Eric Johnston, Sushil Mittal, Akhil Udathu, Janara Christensen, Vishal Verma, Zach Irving, Andreas Santucci, Gamaleldin Elsayed, Elnaz Davoodi, Marin Georgiev, Ian Temney, Nan Hua, Geoffrey Cideron, Edouard Leurent, Mahmoud Alnahlawi, Tonut Georgescu, Nan Wei, Tvy Zheng, Dylan Scandinaro, Heinrich Jiang, Jasper Snoek, Mukund Sundararajan, Xuezhi Wang, Zack Ontiveros, Itay Karo, Jeremy Cole, Vinu Rajashekhar, Lara Tumch, Eyal Bev-David, Rishub Jain, Jonathan Uesato, Romina Datta, Oskar Bunyan, Shimu Wu, John Zhang, Piotr Stanczyk, Ye Zhang, David Steiner, Subhajit Naskar, Michael Azzam, Matthew Johnson, Adam Paszke, Shung-Cheng Chiu, Jaume Sanchez Elias, Afroz Mohiuddin, Faizan Muhammad, Jin Miao, Andrew Lee, Nino Vieillard, Jane Park, Jiageng Zhang, Jeff Stanway, Drew Garmon, Abhijit Karmarkar, Zhe Dong, Jong Lee, Aviral Kumar, Luowei Zhou, Jonathan Evens, William Isaac, Geoffrey Irving, Edward Loper, Michael Fink, Isha Arkatkar, Nanxin Chen, Tzhak Shafran, Ivan Petrychenko, Zhe Chen, Johnson Jia, Anselm Levskaya, Zhenkai Zhu, Peter Grabowski, Yu Mao, Alberto Magni, Kaisheng Yao, Javier Snaider, Norman Casagrande, Evan Palmer, Paul Sugan- than, Alfonso Castaño, Irene Giannoumis, Wooyeol Kim, Mikolaj Rybiński, Ashwin Sreevatsa, Jennifer
Andrew Goodman, Joshua Kessinger, Eran Globen, Prateek
65

---

Published in Transactions on Machine Learning Research (04/2025)
Prendki, David Soergel, Adrian Goedeckemeyer, Willi Gierke, Mohsen Jafari, Meenu Gaba, Jeremy Wies- ner, Diana Gage Wright, Yawen Wei, Harsha Vashisht, Yana. Kulizhskaya, Jay Hoover, Maigo Le, Lu Li, Chimezie Iwuanyanwu, Lu Liu, Kevin Ramirez, Andrey Khorlin, Albert Cui, Tian Lin, Marcus Wu, Ri- cardo Aguilar, Keith Pallo, Abhishek Chakladar, Ginger Perng, Elena Allica Abellan, Mingyang Zhang, Ishita Dasgupta, Nate Kushman, Ivo Penchev, Alena Repina, Xihui Wu, Tom van der Weide, Priya Pon- napalli, Caroline Kaplan, Jiri Simsa, Shuangfeng Li, Olivier Dousse, Fan Yang, Jeff Piper, Nathan Ic, Rama Pasumarthi, Nathan Lintz, Anitha Vijayakumar, Daniel Andor, Pedro Valenzuela, Minnie Lui, Cosmin Paduraru, Daiyi Peng, Katherine Lee, Shuyuan Zhang, Somer Greene, Due Dung Nguyen, Paula Kurylowicz, Cassidy Hardin, Lucas Dixon, Lili Janzer, Kiam Choo, Ziqiang Feng, Biao Zhang, Achintya Singhal, Dayou Du, Dan McKinnon, Natasha Antropova, Tolga Bolukbasi, Orgad Keller, David Reid, Daniel Finchelstein, Maria Abi Raad, Remi Crocker, Peter Hawkins, Robert Dadashi, Colin Gal[ney, Ken Franko, Anna Bulanova, Rémi Leblond, Shirley Chung, Harry Askham, Luis C Cobo, Kelvin Xu, Felix Fischer, Jun Xu, Christina Sorokin, Chris Alberti, Chu-Cheng Lin, Colin Evans, Alck Dimitriov, Hamnah Forbes, Dylan Banarse, Zora Tung, Mark Omernick, Colton Bishop, Rachel Sterneck, Rohan Jain, Jiawei Xia, Ehsan Amid, Francesco Piccinno, Xingyu Wang, Praseem Banzal, Daniel J Mankowitz, Alex Polo- zov, Victoria Krakovna, Sasha Brown, Mohammadhossein Bateni, Dennis Duan, Vlad Firoiu, Meghana Thotakuri, Tom Natan, Matthicu Geist, Ser Tan Girgin, Hui Li, Jiayu Ye, Ofrr Roval, Reiko Tojo, Michael Kwong, James Lee-Thorp, Christopher Yew, Danila Sinopalnikov, Sabela Ramos, John Mellor, Abhishek Sharma, Kathy Wu, David Miller, Nicolas Sonnerat, Denis Vnukov, Rory Greig, Jennifer Beattie, Emily Caveness, Libin Bai, Julian Eisenschlos, Alex Korchemniy, Tomy Tsai, Mimi Jasarevic, Weize Kong, Phuong Dao, Zeyu Zheng, Frederick Liu, Fan Yang, Rui Zhu, Tian Huey Teh, Jason Sanmiya, Evgeny Gladchenko, Neje Trdin, Daniel Toyama, Evan Rosen, Sasan Tavakkol, Linting Xue, Chen Elkind, Oliver Woodman, John Carpenter, George Papamakarios, Rupert Kemp, Sushant Kafle, Tanya Grunina, Rishika Sinha, Alice Talbert, Diane Wu, Denese Owusu-Afriyie, Cosmo Du, Chloe Thornton, Jordi -ust-, Pradyumna Narayana, Jing Li, Saaber Fatehi, John Wieting, Omar Ajmeri, Benigno Uria, Yeongil Ko, Laura Knight, Amélic Hélion, Ning Niu, Shane Gu, Chenxi Pang, Yeqing Li, Nir Levine, Ariel Stolovich, Rebeca Santamaria- Santanarria-rrrnandee,, Sonam Goenka, Wenny Yustalim, Robin Strudel, Ali Elqursh, Charlie Deck, Hyo Lee, Zonglin Li, Kyle Levin, Raphael Hoffmann, Dan Holtmann Arora, Christy Koh, Soheil Hassas Yeganch, Siim Pöder, Mukarram Tarig, Yanhua Sun, Lucian Tonita, Mojtaba Seyedhosscini, Pouya Tafti, Zhiyu Liu, Anmol Gulati, Jasmine Liu, Xinyu Ye, Bart Chrzaszez, Tily Wang, Nikhil Sethi, Tianrun Li, Ben Brown, Shreya Singh, Wei Fan, Aaron Parisi, Joe Stanton, Vinod Koverkathu, Christopher A Choquette-Chon Choo, Yunjie Li, T J Lu, Abe Ittycheriah, Prakash Shroff, Mani Varadarajan, Sanaz Bahargam, Rob Willoughby, David Gaddy, Guillaume Desjardins, Marco Cor- nero, Brona Robenek, Bhavishya Mitial, Ben Albrecht, Ashish Shenoy, Fedor Moiseev, Henrik Jacobsson, Alireza Ghaffarkhah, Morgane Rivière, Alanna Walton, Clément Crepy, Alicia Parrish, Zongwei Zhou, Clement Farabet, Carey Radebaugh, Praveen Srinivasan, Claudia van der Salm, Andreas Fidjeland, Sal- vatore Scellato, Eri Latorre- Latorre-Chimoto, Hanna Klimezak-Plncifiske. David Bridson, Dario de Cesare, Tom Hudson, Piermaria Mendolicchio, Lexi Walker, Alex Morris, Matthew Mauger, Alexey Guseynov, Alison Reid, Seth Odoom, Lucia Loher, Victor Cotruta, Madhavi Yenugula, Dominik Grewe, Anastasia Petrushk- ina, Tom Duerig, Autonio Sanchez, Sueve Yadlowsky, Amy Shen, Amir Globerson, Lynette Webb, Sahil Dua, Dong Li, Surya Bhupatiraju, Dan Hurt, Haroon Qureshi, Ananth Agarwal, Tomer Shani, Matan Eyal, Anuj Khare, Shreyas Rammohan Belle, Lei Wang, Chetan Tekur, Mihir Sanjay Kalc, Jinliang Wei, Ruoxin Sang, Bremnan Saeta, Tyler Liechty, Yi Sun, Yao Zhao, Stephan Lee, Pandu Nayak, Doug Fritz, Manish Reddy Vuyyuru, John Aslanides, Nidhi Vyas, Martin Wicke, Xiao Ma, Fvgenii Eltyshev, Nina Mar- tin, Hardie Cate, James Manyika, Keyvan Amiri, Yelin Kim, Xi Xiong, Kai Kang, Florian Luisier, Nilesh Tripuraneni, David Madras, Mandy Guo, Austin Waters, Oliver Wang, Joshua Ainslic, Jason Baldridge, Han Zhang, Garima Pruthi, Jakob Bauer, Feng Yang, Riham Mansour, Jason Gelman, Yang Xu, George Polovets, Ji Liu, Honglong Cai, Warren Chen, Xianghai Sheng, Emily Xue, Sherjil Ozair, Christof Anger- mueller, Xiaowei Li, Anoop Sinha, Weiren Wang, Julia Wiesinger, Emmanouil Koukounidis, Yuan Tian, Anand Iyer, Madhu Gurumurthy, Mark Goldenson, Parashar Shah, M K Blake, Hongkun Yu, Anthony Urbanowicz, Jennimaria Palomaki, Chrisantha. Fernando, Ken Durden, Harsh Mehta, Nikola Momchey, Elahe Rahintoroghi, Maria Georgaki, Amit Raul, Sebastian Ruder, Morgan Redshaw, Jinhyuk Lee, Denny Zhou, Komal Jalan, Dinghua Li, Blake Hechtman, Parker Schuh, Milad NasT, Kieran Milan, Vladimir Mikulik, Juliana Franco, Tim Green, Nam Nguyen, Joe Kelley, Aroma Mahendru, Andrea Hu, Joshua
Olivier Bachem, Sho
66

---

Published in Transactions on Machine Learning Research (04/2025)
Howland, Ben Vargas, Jeffrey Hui, Kshitij Bansal, Vikram Rao, Rakesh Ghiya, Emma Wang, Ke Ye, Jean Michel Sarr, Melanie Moranski Preston, Madeleine Elish, Steve Li, Aakash Kaku, Jigar Gupta, Ice Pasupat, Da- Ch-nn Juan, Milan Someswar, Tejvi M., Xinyun Chen, Aida Amini, Alex Fabrikant, Eric Chu, Xuanyi Dong, Amruta Muthal, Sonaka Buthpitiya, Sarthak Jauhari, Nan Hua, Urvashi Khandelwal, Ayal Hitron; Jie Ren, Larissa Rinaldi, Shahar Drath, Avigail Dabush, Nan-Jiang Jiang, Harshal Godhia, Uli Sachs, Anthony Chen, Yicheng Fan, Hagai Taitelbaum, Hila Noga, Zhuyun Dai, James Wang, Chen Liang, Jenny Hamer, Chun-Sung Ferng, Chenel Elkind, Aviel Atias, Paulina Lee, Vit Listík, Mathias Carlen, Jan van de Kerkhof, Marcin Pikus, Krunoslav Zaher, Paul Miller, Sasha Zykova, Richard Ste- fanec, Vitaly Gatsko, Christoph Hirnschall, Ashwin Sethi, Xingyu Federico Xu, Chetan Ahuja, Beth Tsai, Anca Stefanoiu, Bo Feng, Keshav Dhandhania, Manish Katyal, Akshay Gupta, Atharva Parulekar, Di- vya Pitta, Jing Zhao, Vivaan Bhatia, Yashodha Bhavnani, Omar Alhadlag, Xiaolin Li, Peter Danenberg, Dennis Tu, Alex Pine, Vera Filippova, Abhipso Ghosh, Ben Limonchik, Bhargava Urala, Chaitanya Kr- ishna Lanka, Derik Clive, Yi Sun, Edward Li, Hao Wu, Kevin Hongtongsak, Ianna Li, Kalind Thakkar, Kuanysh Omarov, Kushal Majmundar, Michael Alverson, Michael Kucharski, Mohak Patel, Mudit Jain, Maksim Zabelin, Paolo Pelagatti, Rohan Kohli, Saurabh Kumar, Joseph Kim, Swetha Sankar, Vineet Shah, Lakshmi Ramachandruni, Xiangkai Zeng, Ben Bariach, Laura Weidinger, Amar Subramanya, Sissie Hsiao, Demis Hassabis, Koray Kavukcuoglu, Adam Sadovsky, Quoc Le, Trevor Strohman, Yonghui Wu, Slav Petrov, Jeffrey Dean, and Oriol Vinyals. Gemini: A family of highly capable multimodal models. Technical report, Google DeepMind, 2023. URL htttp:/arxiv.ortt abt./212..2100
Craig Gentry. A fully homomorphic encryplion scheme. PhD thesis, Stanford University, Stanford, CA, USA, 2009. URL https: http:atpss//....coooo///o///////
Tim Geppert, Stefan Deml, David Sturzenegger, and Nico Ebert. Trusted execution environments: Appli- cations and organizational challenges. Frontiers in Computer Science, 4, 2022. ISSN 2624-9898. URL 10wp:////000300/fcooo/omm wwwp:mpppp222200000.0 9930441
Tom Gerken and Imran Rahman-Jones. Rishi sunak: AT firms cannot 'mark their own homework'. BBC, November 2023. URL
Matthias Gerstgrasser, Rylan Schaeffer, Apratim Dey, Rafael Rafailov, Henry Sleight, John Hughes, Tomasz Korbak, Rajashree Agrawal, Dhruv Pai, Andrey Gromov, Daniel A Roberts, Diyi Yang, David L Donoho, and Sanmi Koyejo. Is model collapse inevitable? Breaking the curse of recursion by accumulating real http://aaxxiv.oo/aa////////
and synthetic data. arXiv: 2404.01413 [cs.LG], April 2024. URL
Ali Ghodsi, Matei Zaharia, Benjamin Hindman, Andy Konwinski, Scout Shenker, and Ion Stoica. Dominant resource fairness: Fair allocation of multiple resource types. In 8th USENIX symposium on networked sys- tems design and implementation (NSDI 11), 2011. URL http:://wwwwwuutoooooooooooooorr org.
conference//sdiil11
Amirata Ghorbani and James Zou. Data Shapley: Equitable Valuation of Data for Machine Learning. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference Machine Learning (ICMI, 2019), volume 97 of Proceedings of Machine Learning Research, pp. 2242-2251, New Orleans, LA, USA, 2019. PMLR. URL http:::/ppooeeddn....r..p.sss v /g/g//ooorbbino wwt.aann99..ht.mmm
Soumya Suvra Ghosal, Souradip Chakraborty, Jonas Geiping, Furong Huang, Dinesh Manocha, and Amrit Bedi. A survey on the possibilities & impossibilities of Al-generated text detection. Transactions on Machine Learning Research, October 2023. ISSN 2835-8856. URL AXtFeYjboj]
Thomas Krendl Gilbert, Nathan Lambert, Sarah Dean, Tom Zick, Aaron Snoswell, and Soham Mehta. Reward reports for reinforcement learning. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pp. 84-130. Association for Computing Machinery, 2023.
Leilani H Gilpin, David Bau, Ben Z Yuan, Ayesha Bajwa, Michael Specter, and Lalana Kagal. Explain- ing explanations: An overview of interpretability of machine learning. In 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA), pp. 80-89. IEEE, October 2018. ISBN 9781 5386509959 9781538650912. URL. http:://aaaaaaaoog/////////////0 020p:000001000000....8.000000000
67

---

Published in Transactions on Machine Learning Research (04/2025)
David Glukhov, Ziwen Han, Ili Shumailov, Vardan Papyan, and Nicolas Papernot. A false sense of safety: Unsafe information leakage in 'safe' AT responses. arxiv: 2407.02551 [es.CR], July 2024. URL http:
//arxiv.org//bb/220000 2tt.///bbb//200255511111
Augustín Godinot, Erwan Le Merrer, Gilles Trédan, Camilla Penzo, and Franois Taïani. Under manip- ulations, are some ai models harder to audit? In 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp- 644 664, Los Alamitos, CA, USA, apr 2024. IEFE Computer So- ciety. doi: 10.1109/ /SA.///5MMM0000022222220 202..03377.222.0033.. URL https://oo.beeoooprrrrreeerr///p 2ATML699770.224400000 00038
Shashwat Goel, Ameya Prabhu, Philip Torr, Ponnurangam Kumaraguru, and Amartya Sanyal. Correc- tive machine unlearning. arXiv: 24015 Les, February 2024. URL hattp://arxiv.org/all
2402.14015
Shahriar Golchin and Mihai Surdeanu. Time travel in LLMs: Tracing data contamination in large language models. In The 12th International Conference on Learning Reeresentations (ICLR 2024), Viemna, Austria, October 2023. URL https:
2Rx2B3333tv
Oded Goldreich. Secure multi-party computation. Manuscripl. Preliminary version, 78(110):1 108, 1998.
Shafi Goldwasser, Guy N Rothblum, Jonathan Shafer, and Amir Ychudayoff. Interactive proofs for verifying machine learning. In James R. Lee (ed.), 12th Innovations in Theoretical Computer Science Conference (ITCS 2021), volume 185 of Leibniz International Proceedings in Informatics (LIPIcs), pp. 41:1-41:19,
Dagstuhl, Germany, 2021. Schloss Dagstuhl deibniz-Zentrum für Informatik. ISBN 9783959771771. URL
Google. Perspectives on issues in ai governance. n.d. URL
Google Cloud, Remote attestation of disaggregated machines, 2024. URL httpp://cllou..goglee.coo
hoce//saartt//reenttttttttssaaaa Accessed: 2024-7-17.
GOV.UK. Al safety institute: Overview, November 2023. URL httpp://www.gv..u/gvvrrmmmil
publication//ai-satrry onst Accessed: 2024-7-16.
Declan Grabb, Max Lamparth, and Nina Vasan. Risks from Language Models for Automated Mental
Healthcare: Ethics and Structure for Implementation, 2024. URL http:://www..marriivooo/otttttt
early//024//4408//2244.44 00..0......... 007.0/000222220000..22325355222
Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan, and Fabicn Roger. AI control: Improving safety despite intentional subversion. arXiv: 2312.06942 [cs.LG], December 2023. doi: 10.48550/arxivv23122 201..v..311.00442.2
Roger Grosse, Juhan Bac, Com Amil, Nelson Elhage, Alex Tamkin, Amirhossein Tajdini, Benoit Steiner, Dustin Li, Esin Durmus, Ethan Perez, Evan Hubinger, Kamilé Lukošiuté, Karina Nguyen, Nicholas Joseph, Sam McCandlish, Jared Kaplan, and Samuel R Bowman. Studying large language model generaliza- tion with influence functions. arXiv: 0008603296 [cs.LG), August 2023. URL http://arrxvv.or//bbb
2308.03296
Erich Grunewald and Michael Aird. AI chip smuggling into china: Potential paths, quantities, and countermeasures. Technical report, Institute for AT Policy and Strategy, 2023. URL https:
Neel Gula, Christie M. Lawrence, Lindsey A Gailmard, Kit T Rodolfa, Faiz Surani, Rishi Bommasani, Inioluwa Deborah Raji, Mariano-Florentino Cuéllar, Colleen Honigsberg, Percy Liang, and Daniel E Ho. AI regulation has its own alignment problem: The technical and institutional fcasibility of disclosure, registration, licensing, and auditing. The George Washington Law Review, (2forthhommngg) 2024. ISSN 0016-8076. URL http:://diisstaffrtttced//// http://hhwa.ctc.eeeeoo/ooeeaaaaa Reguallttooooo
68

---

Published in Transactions on Machine Learning Research (04/2025)
M Guihot, A F Matthew, and N P Suzor. Nudging robots: Innovative solutions to regulate arti- ficial intelligence. Vand. J. Ent. & Tech. L., 2017. URL http:://hheeoooninnorrrooooooo-/
get_pdf.ccg ceeff.ggi77 how.://eeeeeeoouurraaaaaaeeaaeee
Odd Erik Gundersen, Saeid Shamsaliei, and Richard Juul Isdahl. Do machine learning platforms provide out-of-the-box reproducibility? Puture generations computer systems: FGCS, 126:34 47, Jamuary 2022. ISSN 0167-739X. CRL
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. Gradient-based adversarial at- tacks against text transformers. arXiu: 13733 fes. CLJ, April 2021. URL http:///arxi..ogg/bbb.
2104.13733
Chuan Guo, Awni Hannun, Brian Knott, Laurens van der Maaten, Mark Tygert, and Ruiyu Zhu. Secure multiparty computations in floating-point arithmetic. Information and Inference, 11(1):103-11) 135, March 2022. URL http:///daaaattiooooc//////////d
Yanzhu Guo, Guokan Shang, Michalis Vazirgiannis, and Chloé Clavel. The curious decline of linguistic diversily: Training language models on synthetic lext. arXiv: 2311.09807 fes. CLJ, November 2023. URL bttp://aaxxv.orrg/a///221..11
Alexa Hagerty and Igor Rubinov. Global AT ethics: A review of the social impacts and ethical implications of artificial intelligence. arXie: 1907.07892 fes. Css.CY; July 2019. doi: 10. 10.48550/ata /arr0v////v..... 1907.07877.222
Lewis Hammond, Alan Chan, Jesse Clifton, Jason Hoelscher- Obermmrirreerrrrrr Akbir Khan, Euan McLean, Chan- dler Smith, Wolfram Barfuss, Jakob Foerster, Tomášs Gavenčiak, The Anh Han, Edward Hughes, Vojtëch Kovařík, Jan Kulvcit, Joel Z. Leibo, Caspar Ocsterheld, Christian Schrooder de Witt, Nisarg Shah, Michael Wellman, Paolo Bova, Theodor Cimpeanu, Carson Ezell, Quentin Feuillade-Montixi, Matija Franklin, Es- ben Kran, Igor Krawezuk, Max Lamparth, Niklas Lauffer, Alexander Meinke, Sumeet Motwani, Anka Reuel, Vincent Conitzer, Michael Dennis, Iason Gabriel, Adam Gleave, Gillian Hadfield, Nika Haghtalab, Atoosa Kasirzadeh, Sébastien Krier, Kate Larson, Joel Lehman, David C. Parkes, Georgios Piliouras, and Tyad Rahwan. Multi-agent risks from advanced AT. arXiv: 2502. 14143 [cs.MA], February 2025. URI.
Hammond et al., forthcoming.
Krishnaprasad Hande.
Announcing axure confidential VMs with NVIDIA H100 tensor core GPUs in compuuiin ahapraa...a..uucaatta....t.tttta
preview, 2023. URL httppp//eeeeettetttto.ttcooo/orr
Accessed:
2024-7-17.
Shuang Hao, Wenfeng Han, Tao Jiang, Yiping Li, Haonan Wu, Chunlin Zhong, Zhangjun Zhou, and He Tang. Synthetic data in AI: Challenges, applications, and ethical implications. January 2024. URL
arXiv: 04019 [cs.LG],
2200000199
Barath Harithas. Mapping the chip smuggling pipeline and improving export control compliance. Technical report, Center for Strategic and International Studies, 2024. URL http::://ww.cssssorr///aaa/
Lennart Heim. A trusted Al compute cluster for Al verification and evaluation, 2024. URL https://
cluster/
Accessed: 2024-7-17.
Lennart Heim, Tim Fist, Janet Egan, Sihao Huang, Stephen Zekany, Robert Trager, Michael Os- borne, and Noa Zilberman. Governing through the cloud: The intermediary role of compute providers in Al regulation. Technical report, Oxford Martin Al Governance Initiative, March 2024. URL aatts:///chgggeemmmmmaaaa///eeee Thrr Pr--oveeuee-rroviders---- -nneeesiii-------- hitprorieeersii--A---ggaaaaaaapp
Through- the-Cloud The- Internedddaryyooole
William Held, Camille Harris, Michael Best, and Diyi Yang. A material lens on coloniality in NLP. arXiv: 2311.08391 [cs.CLJ, November 2023. URL http:///arxx...og//b//3311111
69

---

Published in Transactions on Machine Learning Research (04/2025)
Peter Henderson, Xuechen Li, Dan Jurafsky, Tatsunori Hashimoto, Mark A Lemley, and Percy Liang. Foun- dation models and fair use. arx: 2303. 15715 [es.CY], March 2023a. URL httt:://arxiv.org//aa
2303.15715
Peter Henderson, Eric Mitchell, Christopher Manning, Dan Jurafsky, and Chelsea Finn. Delf-Destructing models: Increasing the costs of harmful dual uses of foundation models. In Proceedings of the 2023 AAAT/ACM Conference on AI, Ethics, and Sociely, ATES '23, pp. 287-2987880 New York, NY, USA, August 2023b. Association for Computing Machinery. ISBN 9798400702310. URL https://ab..rrg1001145
3600211.3604699
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazcika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In The 9th International Conference on Learning Representations (ICLR 2021), Virtual, 2021. URL htt.a///s/ppppprrresssss///rrrrr
Dan Hendrycks, Mantas Mazeika, and Thomas Woodside. An overview of catastrophic AT risks. ar 2306.1200 12001 [cs,CY], June 2023. doi: 00.48500/rriiv.206..222 12001.
Evan Hernandez, Belinda Z Li, and Jacob Andreas. Inspecting and editing knowledge representations in language models. arXiv: 2304.00740 fcs. CLJ, April 2023. doi: 10.48550 00.44550/arXiiv 230..300000.
Anson Ho, Tamay Besiroglu, Ege Erdil, David Owen, Robi Rahman, Zifan Carl Guo, David Atkinson, Neil Thompson, and Jaime Sevilla. Algorithmic progress in language models. Technical report, Epoch, March 2024. URL http:://arriv.rrr/bbb//223333333
Daniel E Ho, Jennifer King, Russell C Wald, and Christopher Wan. Building a national Al research resource: A blueprint for the national research cloud. Technical report, Stanford University Centeree Arti- ficial Intelligence, 2021.
Lewis Ho, Joslyn Barnhart, Robert Trager, Yoshua Bengio, Miles Brundage, Allison Carnegie, Rumman Chowdhury, Allan Dafoe, Gillian Hadfield, Margaret Levi, and Duncan Snidal. International institutions for advanced AI. Xrvi:: 2307.04699 [cs. CYJ, July 2023. URL http:://arriv..r//aba////// 2300///00/b//30004699999
Mia Hoffmann and Heather Frase. Adding structure to AI harm. Technical report, Center for Secu- rity and Emerging Technology Publications (CSET), July 2023. URL publication/ wwwpcatiiioonninngga-aaa
Zhang-Wo Wei Hong, Idan Shenfeld, Tsun-Hsuan Wang, Yung-Sung Chuang, Aldo Pareja, James Glass, Akash Srivaslava, and Pulkit Agrawal. Curiosity-driven red-teaming for large language models. arXio: 19464 LGj, February 2024. URL httt:://aaxxv..rg/abb///////
Sara Hooker. On the limitations of compute thresholds as a governance strategy. arXiv: 2407.05694 [cs.AI],
July 2024. URL
Shohrch Hosseinzadeh, Bernardo Sequeiros, Pedro R M Inácio, and Ville Leppänen. Recent trends in applying TPM to cloud computing. Securily and privacy, 3(1), 2019. ISSN 2475-6725. URL
House Commons Science, Innovation and Technology Committee. Governance of artificial intelliggenn (AI). Tochnical report. House of Commons, 2024. URL
Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pun-Yu Chen, Chia-Mu Yu, and Chun- Ming Huang. Safe LoRA: The silver lining of reducing safety risks when fine-tuning large language models. arXie: 2405. Lcj, May 2024. URL
Xiaomeng Hu, Yu Chen, and Tsung-Yi Yi Hp. Gradient Cuff: Detecting jailbreak attacks large language models by exploring refusal loss landscapes. arXiv: 2403.00867 (es.CR) March 2024. URL http://
arziw.org/abs/24330005
70

---

Published in Transactions on Machine Learning Research (04/2025)
Qiuyuan Huang, Naoki Wake, Bidipta Sarkar, Zane Durante, Ran Gong, Rohan Taori, Yusuke Noda, Demetri Terzopoulos, Noboru Kuno, Ade Famoti, Ashley Llorens, John Langford, Hoi Vo, Li Fei-Fei, Katsu Ikeuchi, and Jianfeng Gao. Position paper: Agent Al towards a holistic intelligence. arxiv: 2403.00833 [cs.Al]. February 2024a. URL
Raffaele Huang. The underground network sneaking nvidia chips inlo china, July 2024. URL https: http://c//tteeeeeeeerrrrroaattre
Tiansheng Huang, Sihao Hu, Faith Ilhan, Selim Furkan Tekin, and Ling Liu. Booster: Tackling harmful finc-tuning for large language models via attenuating harmful perturbation. arXiv: 2409.01586 fes. CL), September 2024b. URL http:://axxi..rrg///aaa/// 2400000000
Tiansheng Huang, Sihao Hu, and Ling Liu. Vaccine: Perturbation-awaue aligmment for large language model. arXiv: 2402.01 109 [es.LG], February 2024c. URL
Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, Adam Jermyn, Amanda Askell, Ansh Radhakrishnan, Cem Anil, David Duvenaud, Deep Ganguli, Fazl Barez, Jack Clark, Kamal Ndousse, Kshitij Sachan, Michael Sellitto, Mrinank Sharma, Nova DasSarma, Roger Grosse, Shauna Kravec, Yuntao Bai, Zachary Witten, Marina Favaro, Jan Brauner, Holden Karnofsky, Paul Christiano, Samuel R Bowman, Logan Graham, Jared Kaplan, Sören Mindermann, Ryan Greenblatt, Buck Shlegeris, Nicholas Schiefer, and Ethan Perez. Sleeper agents: Training deceptive LLMs that persist through safety training. arXiv: 2401.05566 (R), Jamuary 2024. doi: 10.48550//2xiv.2441..55
Hugging Face. Gated datasets, 2024. URL http:///wwngiiigteeee//////////a cessed: 2024-7-17.
Ac-
Ben Hutchinson, Negar Rostamzadeh, Christina Greer, Katherine Heller, and Vinodkumar Prabhakaran, Evaluation gaps in machine learning practice. In Proceedings of the 2022 ACM Conference on Fair- ness, Accounlability, and Transparency (FAccT '22), pp. 1859 1876, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 9781450393522. URL http::///al.acmooo//////////////
1531146.3566228
Lujain Ibrahim, Saffron Huang, Lama Ahmad, and Markus Anderljung. Beyond static Al evaluations: Advancing human interaction evaluations for LLM harms and risks. arxi 2405. 10632 (es.CY; May 2024. URL
TEE. IEEE draft guide: Adoption of the project management institute (PMJ) standard: A guide to the project management body of knowledge (PMBOK guide)-2008 (4th edition), June 2011. URL http:
//dx.do..org/1..110// http:////1....0////...0000011110
ETF Datatracker. Remote ATtestation ProcedureS (rats), 2024. URL http:://datatracte:eefff org/ Eg/ se/rats/abbut. Accessed: 2024-7-17.
Andrew llyas, Sung Min Park, Logan Engstrom, Guillaume Leclere, and Aleksander Madry. Datamodels:
Predicting predictions from training data. arXio: 2202.00622 [stat.ML], February 2022. URL http:
//arxiv.org/abs// absp//2/02222220222
Vincent Immler, Johannes Obermaier, Martin König, Matthias Hiller, and Georg Sig. B-TREPID: Battery- less tamper-resistant envelope with PUF and integrity detection. In 2018 IEEE International Symposium on Hardware Oriented Security and Trust (HOST), pp. 49-56. IEEE, April 2018. ISBN 9781538647318,
5386533647325. URL
Vincent Immler, Johannes Obermaier, Kuan Kuan Ng, Fei Xiang Ke, Jinyu Lee, Yak Peng Lim, Wei Koon Oh, Keng Hoong Woe, and Georg Sigl. Secure physical enclosures from covers with Tamper- Camper-Resistance. IACR Transactions on Cryptographic Hardware and Embedded Systems, pp. 51-96, 2019. ISSN 2569-2925,
2569-2925. URL http:p////cccne.e..e.oooooaaaaap
71

---

Published in Transactions on Machine Learning Research (04/2025)
Intel. Intel converged security and management engine (intel CSME) security. Technical Report 631900, Intel, 2022.
International Organization for Standardization. ISO/IEC 22237-1:2021: Information technology data centre facilities and infrastructures part 1: General concepts. Technical report, ISO/IEC, 2021. URL
https::/www.sso.rrg///tttt sttp://aandadda555..... html
International Organization for Standardization. ISO/IEC 23894:2023:
Information technology arti- ficial intelligence - guidance on risk management. Technical report, ISO/IEC, 2023. URL https:
//
Daphne Tppolito, Florian Tramer, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christo- pher Choquette Choo, and Nicholas Carlini. Preventing generation of verbatim memorization in language models gives a false sense of privacy. In C Maria Keet, Hung-Yi Yi Lee, and Sina Zarrieß (eds.), Proceedings of the 16th International Natural Language Generation Conference, pp. 28 28-53, Prague, Czechia, September 2023. Association for Computational Linguistics. URL htttp:://aclanthoolgg....
Ahmed Irfan, Kyle D Julian, Haoze Wu, Clark Barrett, Mykel J Kochenderfer, Baoluo Meng, and James Lopez. Towards verification of neural networks for small unmanned aircraft collision avoidance. In 2020 AIAA/IEEE 39th Digital Avionies Systems Conference (DASC). LEEE, 2020. URL http://áx.doi.org
10.1109/DASC509325 202..92556666
Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. In Proceedings of the 29th USENIX Conference on Security Symposium (SEC'20), pp. 1345-1362, USA, August 2020. USENIX Association. ISBN 9781939133175.
Neel Jain, Avi Schwarzschild, Yuxin Wen, Gowthami Somepalli, John Kirchenbauer, Ping- Yeh Chiang, Micah Goldblum, Aniruddha Saha, Jonas Geiping, and Tom Goldstein. Baseline defenses for adver- sarial attacks against aligned language models. arXiv: 2309.00614 [cs.LG], September 2023a. doi: 10.48550/ar arv 2309.00614.
Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P Dick, Hidenori Tanaka, Edward Grefenstette, Tim Rocktäschel, and David Scott Krueger. Mechanistically analyzing the effects of fine-tuning pro- cedurally defined tasks. arXiv: 2311.12786 [es.LG/, November 2023b. URL http://arxiw.org/bbb
2311.12786
Shrey Jain, Zoë Hitzig, and Pamela Mishkin. Contextual confidence and generative AI. arXio: 2311.01198 Als, November 2023c. URL
Markus Jakobsson and Ari Juels. Proofs of work and bread pudding (oolocols(Extender abstract). In Bart Preneel (ed.), Secure Information Networks: Communications and Multimedia Security IFIP TC6/TC11 Joint Working Conference on Communications and Multimedia Securily (CMS 99) September 20 21, 1999,
Lewven, Belgium, pp. 258 272. Springer US, Boston, MA, 1999. ISBN 9780387355689.
URL https:
//abl.org/10..00779978-- 3-777-0--377756688 9__18
Olli Järviniemi and Evan Hubinger. Uncovering deceptive tendencies in language models: A simulated company Al assistant. arXiv: 2405.01570 /es.CL,, April 2024. URL http:://aarrii.rr///aa////55
Julian Jaursch, Jakob Ohme, and Ulrike Klinger. Enabling research with publicly accessible platform data: Early DSA compliance issues and suggestions for improvement. Technical report, Weizenbaum Institut, 2024. URL
Maha Jebalia, Asma Ben Letaïfa, Mohamed Hamdi, and Sami Tabbane. fair resource allocation ap- proach in cloud computing environments. In 2018 IEEE 27th International Conference on Enabling Tech- nologies: Infrastructure for Collaborative Enterprises (WETICE), pp. 54-57. IEEE, June 2018. ISBN 9781 538666916, 9781538669174. URL. http:p////...aa..a//////////1110
72

---

Published in Transactions on Machine Learning Research (04/2025)
Hengrui Jia, Mohammad Yaghini, Christopher A Choquette-Choo Natalie Dullerud, Anvith Thudi, Varun Chandrasekaran, and Nicolas Papernot. Proof-of-Teaaring:: Definitions and practice. In 2021 IEFE Symposium on Security and Privacy (SP), pp. 1039 1056. EEEE, May 2021. ISBN 9781728189345, 9781728189852. URL http://dx.doe wtt.p//..gggg/111110//000000..00
Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve Gurel, Bo Li, Ce Zhang, Dawn Song, and Costas Spanos. Towards efficient data valuation based the shapley value. arxi:: 10275 [cs.LGj, February 2019. URL
Minhao Jiang, Ken Ziyu Liu, Ming Zhong, Rylan Schaeffer, Siru Ouyang, Jiawei Han, and Sanmi Koyejo. Investigating data contamination for pre-training language models. arXiv: 2401.06055 fcs. CLJ, Jamuary 2024. URL http::///aarivooggab///////
Yiqiao Jin, Mohit Chandra, Gaurav Verma, Yibo Hu, Munmun De Choudhury, and Srijan Kumar. Better to ask in english: fes. CLJ. October 2023. URL http:///axxv..rg/// abs///////////a 2
evaluation of large language models for healthcare queries. arxiv: 13132
Erik Jones, Anca Dragan, Aditi Raghunathan, and Jacob Steinhardt. Automatically auditing large language models via discrete optimization. In Andreas Krause, Emmma Brunskill, Kyunghyun Cho, Barbara. Engel- hardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 15307 15329. PMLR, 2023. URL https::/proceeddingg mlr wtw.mmmr...rm.s.. www.prresssvv.. ptoess/v222/oones23..mm..
kaggle. Code competition FAQ, 2024. URL 7-17.
Accessed: 2024-
Peter Kairouz, H Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, Rafael G L D'Oliveira, Hubert Eichner, Salim El Rouayheb, David Evans, Josh Gardner, Zachary Garrett, Adrià Gascón, Badih Ghazi, Phillip B Gibbons, Marco Gruteser, Zaid Harchaoui, Chaoyang He, Lie He, Zhouyuan Huo, Ben Hutchin- son, Justin Hsu, Martin Jaggi, Tara Javidi, Gauri Joshi, Mikhail Khodak, Jakub Konečný, Aleksandra Korolova, Farinaz Koushanfar, Sanmi Koyejo, Tancrède Lepoint, Yang Liu, Prateek Mittal, Mehryar Mohri, Richard Nock, Ayfer Özgür, Rasmmu Pagh, Mariana Raykova, Hang Qi, Daniel Ramage, Ramesh Raskar, Dawn Song, Weikang Song, Scbastian Stich, Ziteng Sun, Ananda Theertha Suresh, Florian Tramèr, Praneeth Vepakomma, Jianyu Wang, Li Xiong, Zheng Xu, Qiang Yang, Felix X Yu, Han Yu, and Sen Zhao. Advances and open problems in federated learning. arXiv: 1912.04977 Lcs, December 2019, URL http: / http://xxivvorggg/eo absa 1ttps//91...4977.
Nikhil Kandpal, Eric Wallace, and Colin Raffel Deduplicating training data mitigates privacy risks in language models. In Kamalika Chaudhuri, Stefanic Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learn- ing, volume 162 of Proceedings of Machine Learning Research, pp. 10697-10707, PMLR, 2022. URL attps://proceeddngg: wwwp://wwww..pp...pppppp.eeeeeee v162/ kandpal22..htm
Sayash Kapoor, Rishi Bommasani, Kevin Klyman, Shayne Longpre, Ashwin Ramaswami, Peter Cihon, Aspen Hopkins, Kevin Bankston, Stella Biderman, Miranda Bogen, Rumman Chowdhury, Alex Engler, Peter Henderson, Yacine Jernite, Seth Lazar, Stefano Maffulli, Alondra Nelson, Joelle Pineau, Aviya Skowron, Dawn Song, Victor Storchan, Daniel Zhang, Danicl E Ho, Percy Liang, and Arvind Narayanan. On the socictal impact of open foundation models. arXiv: 2403.07918 CYs, February 2024a. doi: 10.48550/ ar Xiv. wtw.2.433.09911
Sayash Kapoor, Benedikt Stroebl, Zachary S Siegel, Nitya Nadgir, and Arvind Narayanan. AT agents that matter. arXiv: 2407.01502 [cs.LG], July 2024b.
Guy Katz, Clark Barrett, David I, Dill, Kyle Julian, and Mykel J Kochenderfer. Reluplex: An efficient SMT solver for verifying deep neural networks. In Computer Aided Verification, pp. 97-117. Springer International Publishing, 2017a. URL httppp///aaaaa..rrr///.0.0077701
73

---

Published in Transactions on Machine Learning Research (04/2025)
Guy Katz, Clark Barrett, David L Dill, Kyle Julian, and Mykel J Kochenderfer. Towards proving the adversarial robustness of deep neural networks. arXin: 7709.02805 [es. (es.,.G, September 2017b. URL
Guy Katz, Derek A Huang, Duligur lbeling, Kyle Julian, Christopher Lazarus, Rachel Lim, Parth Shah, Shantanu Thakoor, Haoze Wu, Aleksandar Zeljić, David Dill, Mykel J Kochenderfer, and Clark Barrett. The marabou framework for verification and analysis of deep neural networks. In Computer Aided Verifi- cation, pp. 143-452. Springer International Publishing, 2019. URL 030-25540-4_24
Bryan Kelly, Andrés Lagar-Cavilla, Jeff Andersen, Prabhu Jayana, Piotr Kwidzinski, Rob Strong, John Traver, Louis Ferraro, Ishwar Agarwal, Anjana Parthasarathy, Bharat Pillilli, Vishal Soni, Marius Schilder, Sudhir Mathane, Nathan Nadarajah, and Kor Nielsen. Caliptra: A datacenter system on chip (SOC) root of trust (RoT). Technical report, Open Compute Project, 2022.
Seungone Kim, Jamin Shin, Yejin Cho, Joel Jang, Shayne Longpre, Hwaran Lee, Sangdoo Yun, Seongjin Shin, Sungdong Kim, James Thorne, and Minjoon Sco. Prometheus: Inducing fine-grained evaluation capability in language models. arxiv: 2310.08491 jcs. CLJ, October 2023. URL htt.l/tppp//arrri..o/////////../
Megan Kinniment, Lucas Jun Koba Sato, Haoxing Du, Brian Goodrich, Max Hasin, Lawrence Chan, Luke Harold Miles, Tao R Lin, Hjalmar Wijk, Joel Burget, Aaron Ho, Elizabeth Barnes, and Paul Chris- tiano. Evaluating Language-Moce Model agents on realistic autonomous tasks. arXiv: 2312. 11671 [es. CL), December 2023. URL
John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. A watermark for large language models. In Proceedings of the 40th International Conference on Machine Learning, pp. 17061-17084. PMLR, July 2023. URL
Jon Kleinberg. Inherent Trade-Offs in algorithmic fairness. In Abstracts of the 2018 ACM International Conference Measurement and Modeling of Computer Systems (SIGMETRICS '18), pp. 40, New York, NY, USA, June 2018. Association for Computing Machinery. ISBN 9781450358460. URL https:// 0oo.:or//1001144/226667...3
Alistair Knott, Dino Pedreschi, Raja Chatila, Tapabrata Chakraborti, Susan Leavy, Ricardo Baeza- Yates, David Eyers, Andrew Trotman, Paul D Teal, Przemyslaw Biecek, Stuart Russell, and Yoshua Bengio. Generative AI models should include detection mechanisms as condition for public release. Ethics and information technology, 25(4):55, October 2023. ISSN 1388-1957, 1572-8439. URL https://doi.org/ 10.1007/s10676-0000 022-116667602--977722 09728-4
Brian Knott, Shobha Venkataraman, Awni Hanmun, Shubho Sengupta, and Mark Ibrahim. Cryp'Ten: Se- cure Multi-Party Computation Meets Machine Learning. In Advances in Newral Information Processing Systems, volume 34, pp. 4961-4973. Curran Associates, Inc., 2021. URL https::/papprr..eeuri// nww.ppppsr..rrrps.c. 077754518221cc0 1221 ettpe///5061110644444411-1----t-
Noam Kolt. Algorithmic black swans. Washington University Law Review, 101, 2023. URL. https://
papers.ssrn.comt wwt.scoomcooocccecc https//sss.....oooo///pppsrrrrre (rappprsscccmaabsttrrttt1111-001
Noam Kolt. Governing AI agents, 2024. URL http:///www.ssm..ooom//tttttttte
Noam Kolt, Markus Anderljung, Joslyn Barnhart, Asher Brass, Kevin Esvelt, Gillian K Hadfield, Lennart Heim, Mikel Rodriguez, Jonas B Sandbrink, and Thomas Woodside. Responsible reporting for frontier AI development. arXi: 2404.02674 (es. April 2024. doi: 0044550//arivv244..4
Julia Kreutzer, Isaac Caswell, Lisa Wang, Absan Wahab, Daan van Esch, Nasanbayar Ulzii-Orshikh, Allah- sera Tapo, Nishant Subramani, Artem Sokolov, Claytone Sikasote, Monang Setyawan, Supheakmungkol Sarin, Sokhar Samb, Benoit Sagot, Clara Rivera, Annette Rios, Tsabel Papadimitrion, Salomcy Osci, Pedro Ortiz Suarez, Iroro Orife, Kelechi Ogueji, Andre Niyongabo Rubungo, Toan Q Nguyen, Math- ias Müller, André Müller, Shamsuddeen Hassan Muhammad, Nanda Muhammad, Ayanda Mnyakeni,
74

---

Published in Transactions on Machine Learning Research (04/2025)
Jamshidbek Mirzakhalov, Tapiwanashe Matangira, Colin Leong, Nze Lawson, Sneha Kudugunta, Yacine Jernite, Mathias Jenny, Orhan Firat, Bonaventure F P Dossou, Sakhile Dlamini, Nisansa de Silva, Sakine Çabuk Balh, Stella Biderman, Alessia Battisti, Ahmed Baruwa, Ankur Bapna, Pallavi Baljekar, Israel Abebe Azime, Ayodele Awokoya, Duygu Ataman, Orevaoghene Ahia, Oghenefogo Ahia, Sweta Agrawal, and Mofetoluwa Adeyemi. Quality at a glance: An audit. of CrawCCr multilingual datasets. Transactions of the Association for Computational Linguisties, 10:50-72, 2022. ISSN 2307-387X. URL
.rgg 2ttp////......1...1414.
Gabriel Kulp, Daniel Gonzales, Everett. Smith, Lennart Heim, Prateek Puri, Michael JD Vermeer, and Zev Winkelman. Hardware- Harbware-Enabled Enabled Governance Mechanisms:: Developing Technicol Solutions to Exempt Items Otherwise Classified Under Export Control Classification Numbers 3A 090 and 4A090. RAND Corporation, Santa Monica, CA, 2024. URL http://dx.doo/ wtt.///0...29///33055-----
Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, and Himabindu Lakkaraju. Certifying LLM safety against adversarial prompting, arXiv: 2309.00 02000 CLs,, September 2023, doi:
10.48550/ 1.A8550/arXiiv httaaarvvvvvvv...... 2309.0277.
Nishant Kumar, Mayank Rathee, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. CrypTFFlw: Secure TensorFlow inference. orXiv: 1909. 1909.07814 CR), September 2019. URL http:
Sachin Kumar, Biswajit Paria, and Yulia Tsvetkov. Gradient-Based constrained sampling from language models. arXiv: 2205. 1 12558 CL/:; May 2022. URL
Tsai-Chi Kuo, Chien Yun Kuo, and Liang-Wi Wei Chen. Assessing environmental impacts of nanoscale semi- conductor manufacturing from the life cycle assessment perspective. Resources, Conservalion and Recy- cling, 182:106289, July 2022. ISSN 0921-3449. URL htt.:t/ss///waaraaaaaarrccooo//s p11/80921344922001399
Lindsey Kuper, Guy Katz, Justin Goltschlich, Kyle Julian, Clark Barrett, and Mykel Kochenderfer. Toward scalable verification for Safety-Critical deep networks. arXie: 1801. 059100550 [cs.AlJ, January 2018. URL
http://aax....og//b///BB......
Sabrina Küispert, Nicolas Moës, and Connor Dunlop. The value chain of peneral-purpose AI, 2023. URL http:s/www..aall.vvllaeeeesssttt http://blloooolueeeeeeeeeeeeeeee chain petrerg--arla-ppseeee ai/ Accessed: 2024-
7-17.
Seth Lazar. Frontier Al ethics: Anticipating and evaluating the societal impacts of generative agents. arXio: 2404.06750 [cs,CY], April 2024. URL
Seth Lazar and Alondra Nelson. Ai safety on whose terms?, 2023.
Katherine Lee, A Feder Cooper, and James Grinmelmann. Talkin' 'bout Al generation: Copyright and the Generative-A AI supply chain (the short version). In Proceedings of the Symposium on Computer Science and Law (CSLAW 244, pp. 48 63, New York, NY, USA, March 2024. Association for Computing Machinery. ISBN 9798400703331. URL
3643669
Mina Lee, Megha Srivastava, Amelia Hardy, John Thickstun, Esin Durmus, Ashwin Paranjape, Ines Gerard- Ursin, Xiang Lisa Li, Faisal Ladhak, Frieda Rong, Rose E Wang, Minae Kwon, Joon Sung Park, Hancheng Cao, Tony Lee, Rishi Bommasani, Michael S Bernstein, and Percy Liang, Evaluating Human- Language model interaction, July 2023a. URL httppp//ppppeeerrerrtt//////ad//
Tony Lee, Michihiro Yasunaga, Chenlin Meng, Yifan Mai, Joon Sung Park, Agrim Gupta, Yunzhi Zhang, Deepak Narayanan, Hannah Benita Teufel, Marco Bellagente, Minguk Kang, Tacsung Park, Jure Leskovec, Jun Yan Zhu, Li Fei-Fei, Jiajun Wu, Stefano Ermon, and Percy Liang. Holistic evaluation of Text-To To-ge models. Xiv: 2311. 1.0287 fes. CV], November 2023b. URL http::///arri..orgr//////1111..0
75

---

Published in Transactions on Machine Learning Research (04/2025)
Paddy Leerssen. Platform research access in article 31 of the digital services act: Sword without shield?
Verfussungsblog: On Mallers Constitulional, 2021. ISSN 2366-7044. URL httts:://antr2bxx.ii2
recht.de//receive/miix _warar_mods_____ 00011130
Paddy Leerssen. Seeing what others are seeing: Studies in the regulation of transparency for social media recommender systems. PhD thesis, Faculty of Law, 2023.
Paddy Leerssen, Amélie P Heldt, and Matthias C Kettemann. Scraping By? Europe's law and policy social media. research access. In Christian Strippel, Sünje Paasch-Colberg, Martin Emmer, and Joachim Trebbe (eds.), Challenges and Perspectives of Hate Speech Research, volume 12, pp. 405 425. Digital Com- munication Research, Berlin, 2023. ISBN 9783945681129 URL http:::ppddddd.oorrg///////// wttp:///...444./........o
Simon Lermen, Charlic Smith,, and Jeffrey Ladish. LoRA fine-tuning efficiicnly undocs safety training in llama 2-chat 70B. Xiv: 2310.20624 [es.LG], October 2023. URL http:::/arzii...rg/aabb/ http:///w/r..oo/./.000...0
David Leslie, Cami Rincon, Morgan Briggs, Antonella Perini, Smera Jayadeva, Ann Borda, S J Bennett, Christopher Burr, Mhairi Aitken, Michael Katell, Claudia Fischer, Janis Wong, and Ismael Kherroubi Garcia. AI fairness in practice. arXiv: 2403. 14636 fcs. CYJ, February 2024. URL http:://rxii....t////// www.//oo////o org/abs/ abs/
2403.14636
Haodong Li, Gelei Deng, Yi Liu, Kailong Wang, Yuekang Li, Tianwei Zhang, Yang Liu, Guoai Xu, Guosheng Xu, and Haoyu Wang. Digger: Detecting copyright content mis-usage in large language model training. arXiv: 2401.00676 /cs. CRJ, January 2024a. URL http:///aaxttoorr//////22...00
Maximilian Li, Xander Davies, and Max Nadcau. Circuit breaking: Removing model behaviors with targeted ablation. In Proceedings of the 40th International Conference on Machine Learning, Honolulu, Hawaii, USA, 2023a. PMLR. URL
059733
Mu Li, David G Andersen, Jun Woo Park, Alexander J Smola, Amr Ahmed, Vanja Josifovski, James Long, Eugene J Shekita, and Bor-Ying Su. Scaling distributed machine learning with the parameter server. In Proceedings of the 11th USENIX Conference on Security Symposium, Broomfield, CO, 2014. USENIX Association. ISBN 9781931971164. URL http:://www..aaaii..c.///s//aate
Bedi14-paper-- lf mdf
Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D) Li, Ann- Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B Liu, Michael Chen, Tsabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel Herbert- Voss, Cort B Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Liu, Adam A Hunt, Justin Boonen-HHadder Kevin Y Shih, Kemper Talley John Guan, Russell Kaplan, Ian Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Joan Wang, William Qian, Kallol Krishna Karmakar, Steven Basart, Stephen Fitz, Mindy Levine, Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Yan Shoshitaishvili, Jimmy Ba, Kevin M Esvelt, Alexandr Wang, and Dan Hendrycks. The WMDP benchmark: Measuring and reducing malicious use with unlearning, arXiv: 2403.03214 fcs.LG), March 2024b. doi: 10. 10.48550/ar http//aa............ 240..0218..
Xiaoguo Li, Bowen Zhao, Guomin Yang, Tao Xiang, Jian Weng, and Robert H Dong. A survey of secure computation using trusted execution environments. arXim: 2302. 12150 fes. (Rj, February 2023b. URL http://arxv..or//aa//000..
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga Yian Zhang, Decpak Narayanan, Yuhuai Wu, Ananya Kumar, Benjamin Newman, Binhang Yuan, Bobby Yan, Ce Zhang, Christian Alexander Cosgrove, Christopher D Manning, Christopher Re, Diana Moosta-Navar Drew Arad Hudson, Eric Zelikman, Esin Durmus, Faisal Ladhak, Frieda Rong, Hongyu Ren, Huaxiu Yao, Jue Wang, Keshav Santhanam, Laurel Orr, Lucia Zheng, Mert Yuksckgonul, Mirac Suzgun, Nathan Kim, Neel Guha, Niladri S Chatterji, Omar Khattab, Peter Henderson, Qian Huang, Ryan Andrew Chi, Sang Michael Xie, Shibani Santurkar, Surya Ganguli, Tatsunori Hashimoto, Thomas Ieard, Tianyi Zhang,
76

---

Published in Transactions on Machine Learning Research (04/2025)
Vishrav Chaudhary, William Wang, Xuechen Li, Yifan Mai, Yuhui Zhang, and Yuta Koreeda. Holis- tic evaluation of language models. Transactions on Machine Learning Research, February 2023. ISSN 2835-8856. URL
Jinkun Lin, Anqi Zhang, Mathias Lecuyer, Jinyang Li, Aurojit Panda, and Siddhartha Sen. Measuring the effect of training data on deep learning prodictions via randomized experiments. arXiv: 2206. 10013 Lej, June 2022a. URL
Stephanie Lin, Jacob Hilton, and Owain Evans. TruthfulQA: Measuring how models mimic human falschoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Associalion for Compulational Linguistics (Volume 1: Long Papers), pp-
3214 3252, Dublin, Ireland, May 2022b. Association for Computational Linguistics. URL https: //aclanthology wtrg/2022.accr 20022.acl-oong220.
Simon Lindgren and Virginia. Digmum. Beyond AI solutionism: toward a multi-discipllnary approach to artificial intelligence society. In Handbook of Critical Studies of Artificial Intelligence, pp. 163 172. Edward Elgar Publishing, November 2023. ISBN 9781803928562. URL attps://www.elgaronliii com/
xml
Aiwei Liu, Leyi Pan, Yijian Lu, Jingjing Li, Xuming Hu, Xi Zhang, Lijie Wen, Trwin King, Hui Xiong, and Philip S Yu. A survey of text watermarking in the era of large language models. arXiv: 2312.07913 fes. CLJ, December 2023a. URL http::/aaxxv..org/eaae /
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalic Baracaldo, Peter Hase, Xiaojun Xu, Yuguang Yao, Hang Li, Kush R Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. Rethink- ing machine unlearning for large language models. arXiæ: 2402.08787 [es.I.G), February 2024a. doi: 10.48550/ aiv 2402.08787.
Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. AutoDAN: Generating stealthy jailbreak prompts on aligned large language models. arxi:: 04450 fes. CL), October 2023b. URL http://arxxv.org/
abs/2310.0442
Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. Towards safer large lan- guage models through machine unlearning. arXiw: 2402. 10058 CLJ, February 2024b. doi: 10.48550/
Orly Lobel. The ai regulatory pyramid: A laxonomy & analysis of the emerging toolbox in the global race for the regulation and governance of artificial intelligence. Loy. LAL Rev., 57:859, 2024.
Andrew Lohn. Hacking Al: A primer for policymakers on machine learning cybersecurity. Technical report.
(Center for Security and Emerging Technology), December 2020. URL http:://aee..ggggeemmmmddl publication//acckig-
Shayne Longpre, Robert Mahari, Anthony Chen, Naana Obeng-Marnu, Damien Sileo, William Brannon, Niklas Muennighoff, Nathan Khazam, Jad Kabbara, Kartik Perisctla, Xinyi Wu, Enrico Shippole, Kurt Bollacker, Tongshuang Wu, Luis Villa, Sandy Pentland, and Sara Hooker. The data provenance initialive: A large scale audit of dataset licensing & attribution in AI. arXiv: 2310.16787 fes. CLJ, October 2023. URL http::/arxiv..r 2b312310.16777
Shayne Longpre, Sayash Kapoor, Kevin Klyman, Ashwin Ramaswami, Rishi Bommasani, Borhane Blili- Hamelin, Yangsibo Huang, Aviya Skowron, Zheng-Xin Xin Yong, Suhas Kotha, Yi Zeng, Weiyan Shi, Xianjun Yang, Reid Southen, Alexander Robey, Patrick Chao, Diyi Yang, Ruoxi Jia, Daniel Kang, Sandy Pentland, Arvind Narayanan, Percy Liang, and Peter Henderson. A safe harbor for AT evaluation and red teaming. arXiv: 2403.04899 0483 [cs.AT], March 2024a. doi: (18550/arXiv.... 2403.04333.
Shayne Longpre, Robert Mahari, Naana Obeng- Marnu, William Brannon, Tobin South, Jad Kabbara, and Sandy Pentland. Data authenticity, consent, and provenance for AI are all broken: What will it take fix them? An MIT Exploration of Generative AI, March 2024b. URL hntps:: //mm seriitt-gnaaiuuppe www.aii.puppu.orr....
pub/uk7op8zs
77

---

Published in Transactions on Machine Learning Research (04/2025)
Alexandra Luccioni and Joseph Viviano. What's in the box? analysis of undesirable content in the Common Crawl corpus. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annuel Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Voluee 2: Short Papers), pp. 182-189, Online, August 2021. Association for Compulational Linguisties. htt.at/s///ellaaaooooooo////////
Alexandra Sasha Luccioni and Alex -arnandez-Garcia. Counting carbon: A survey of factors influencing the emissions of machine learning. arXiv: 2302.08476 [cs.LG], February 2023. URL http://arxiv.org/
abs/2302.08476
Alexandra Sasha Luccioni, Frances Corry, Hamsini Sridharan, Mike Ananny, Jason Schultz, and Kate Craw- ford. A framework for deprecating datasets: Standardizing documentation, identification, and commu- nication. In Proceedings of the 2022 ACM Conference on Fairness, Accountabllit,, and Trunsparency (FAccT '22), pp. 199-212, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 9781450333522. URL
Alexandra Sasha Luccioni, Yacine Jernite, and Emma Strubell. Power hungry processing: Watts driving the cost of AT deployment? arXiv: 2311.16863 [es.LG], November 2023a. URL. http://aarxi..or//bb..
2311.16863
Alexandra Sasha Luccioni, Sylvain Viguier, and Anne-Laure Ligozat. Estimating the carbon footprint of BLOOM, a 176B parameter language model. Journal of machine learning research, 24(253):1-15, 2023b. ISSN 1532-4435, 1533-7928. URL
Sasha Luccioni. Energy star ratings for AI models, 2024. URL
/ sasha/
strray--tarr---prppeas Accessed: 2024-7-18.
Aengus Lynch, Phillip Guo, Aidan Ewart, Stephen Casper, and Dylan Hadfield-Menel. Eight methods to evaluate robust unlearning in LLMs. arXiv: 2402.16835 [es. Cls, February 2024. doi: 10.48550/
11X.1.2402.16835.
Lovish Madaan, Aadilya K Singh, Rylan SchaelTer, Andrew Poulton, Sanmi Koyejo, Pontus Stenetorp, Sharan Narang, and Dieuwke Hupkes. Quantifying variance in evaluation benchmarks. arXiv: 2406. 10229 Les,LG], June 2024. URL
Gary Marcus and Reid Southen. Generative AI has a visual plagiarism problem, Jamuary 2024. URL Accessed: 2024-7-14.
Helen Margetts. Rethinking Al for good governance. Daedalus, 151(2):3603371 May 2022. ISSN 0011-5266
01922. pdf
Helen Margetts and Cosmina Dorobantu. Rethink government with AI. Nature, pp. 163 165, April 2019.
URL
Benjamin Marie, Atsushi Fujita, and Raphael Rubino. Scientific credibility of machine translation research: A Meta-Evaluation of 769 papers. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annuel Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp- 7297- 7306, Online, August 2021. Association for Computational Linguistics. URL htttps:/acclattoll/ 2021.cc-Long.566
Miljan Martic, Jan Leike, Andrew Trask, Matteo Hessel, Shane Legg, and Pushmeet Kohli. Scaling shared model governance via model splitting. arXin: 03929 [cs.T.G], December 2018. URL.
http://aaziv.orr//aa///112000
Gonzalo Martínez, Lauren Watson, Pedro Reviricgo, José Alberto Hernández, Marc Juarez, and Rik Sarkar. Combining generative artificial intelligence (AI) and the internet: Heading towards evolution or degrada- tion? Xivi 2303.01255 fes. CV,, February 2023. URL http:://aarrk.rrrraaa//22.0..150
78

---

Published in Transactions on Machine Learning Research (04/2025)
Nestor Maslej, Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark. The Al index 2024 annual report. Technical report, Al Index Steering Committee, Institute for liman-Centered AI, Stanford University, Stanford, CA, USA, April 2024. URL https: //aiindex.stanf ord. wttp://aaan.orreeeuuuueooo-ooooo htwpa///wwwpooocttttttt//aaaa//a 2024////// AI-Index- -el---nxx-RReort---- 20f4.pdff
Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Ti, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. Henc:: A standardized evaluation framework for automated red teamiin and robust refusal. arXiv: 2402.0 04249 [cs.LG], February 2024.
URL http:///arxiv..rg/abbb///// wwwp://////...o.oo..//// 2402
Louise McCormack and Malika Bendechache. Ethical ai governance: Methods for evaluating trustworthy ai. arXiv preprint arXio:2499.07. 2024.
Sean McGregor. Preventing repeated real world AI failures by incideenn The AI incident database. arXiv: 2011.08512 [cs. CYJ, November 2020. URL http:///arriioorr/aa/////1111110
Timothy R McIntosh, Teo Susnjak, Tong Liu, Paul Watters, and Malka N Halgamuge, Inadequacies of large language model benchmarks in the era of generative artificial intelligence. arXiv: 2402.09880 [cs.AI], February 2024. doi: 10.48550/arXiv www.aiiivv...009.00 099880....
Ninareh Mehrabi, Palash Goyal, Christophe Dupuy, Qian Hu, Shalini Ghosh, Richard Zemel, Kai- Wei Chang, Aram Galstyan, and Rahul Gupta. FLIRT: Foedback loop in-context red teaming. arXio: 2308. 04265 Als, August 2023. URL
Kevin Meng, David Bau, Alex J Andonian, and Yonatan Bolinkov. Locating and oditing factual associations in GPT. In 36th Conference on Neural Information Processing Systems (NewrIPS 2022), Virtual, October 2022. URL hattg:///ppaaeerreeeeeee/rrrrrrp
Jacob Metcalf, Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, and Madeleine Clare Elish. Al- gorithmic impact assessments and accountability: The co-construction of impacts. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAceT '21), pp. 735-746, New York, NY, USA, March 2021. Association for Computing Machinery. ISBN 9781450383097. URL
34445935
Katina Michael, Roba Abbas, Rafael A Calvo, George Roussos, Fusebio Scornavacca, and Samuel Fosso Wamba. Manufacturing consent: The modern pandemic of technosolutionism. IEEE Transactions Techmology and Society, 1(2):68 72, June 2020. ISSN 2637-6415. URL http:://db.doi.org/10...
21S.2020.2994381
Microsoft.
Governing AI: A blueprint for the future. Technical report, Microsoft, 2023. URL https: hwwpery.proddmms......ccoooooooe
Fraser Mince, Dzung Dinh, Jonas Kgomo, Neil Thompson, and Sara Hooker. The grand illusion: The myth of software portability and implications for MT. progress. arxiv: 0909.07181 [es.SE], September 2023.
URL http:://aaaw...oggaa////00..0.
Eric Mitchell, Charles Lin, Antoine Bossclut, Chelsea Finn, and Christopher D Manning. Fast model editing scale. Xiv: 2110.1 11309 [cs.LG], October 2021. URL
Eric Mitchell, Charles Lin, Antoine Bossclut, Christopher D Manning, and Chelsea Finn. Memory-Basod model editing at scale. In Proceedings of the 39th International Conference on Machine Learning, pp. 15817-15831. PMLR, June 2022a. URL
Margaret Mitchell, Simone Wu, Andrew Zaldivrr, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19), pp. 220-229, New York, NY, USA, Jamuary 2019. Association for Computing Machinery. ISBN 9781450361255. URL https: 7/dbl..rg/00.1155/88750033333330
79

---

Published in Transactions on Machine Learning Research (04/2025)
Margaret Mitchell, Alexandra Sasha Luccioni, Nathan Lambert, Marissa Gerchick, Angelina McMillan- Major, Exinwanne Ozoani, Nazneen Rajani, Tristan Thrush, Yacine Jernite, and Douwe Kiela. Measuring data. arxi:: 2212.05129 [cs.Alf, December 2022b. doi: 0005500/aXivv22122.5512222
Fan Mo, Zahra Tarkhani, and Hamed Haddadi. Machine learning with confidential computing: systemati- zation of knowledge. ACM computing surveys, 1611):144, November 2024. ISSN 0360-0300, 1557-7341. URL httpp/////l...ao.orr///////...1/
Nicolas Moñs and Frank Ryan. Heavy is the head that wears the crown: A risk based tiered approach to governing general purpose Al. Technical report, The Future Society, 2023.
Jakob Mökander, Jonas Schuett, Hannah Rose Kirk, and Luciano Floridi. Auditing large language models: a throe-laycred approach. AI and Ethics, May 2023. ISSN 2730-5961. URL https://tbl.org/101000//
843681-023-00289-2
Christopher Morten, Gabriel Nicholas, and Salome Viljoen. Researcher access to social media data: Lessons from clinical trial data sharing. Berkeley Technology Law Journal, 39(1))), 2024. ISSN 1556-5068, URL
Emanuel Moss, Elizabeth Anne Watkins, Ranjit Singh, Madcleine Clare Elish, and Jacob Metcalf. Assem- bling accountability: Algorithmic impact assessment for the public interest. Technical report, Data & Soci- ety, June 2021. URL
Aida Mostafazadeh Davani, Mark Díaz, and Vinodkumar Prabhakaran. Dealing with disagreements: Looking beyond the majority vote in subjective annotations. Transactions of the Association for Computational Linguistics, 10:92-110, 2022. URL http:s///ccccanthoooooorr///2222
Christopher A Mouton, Caleb Lucas, and Ella Guest. The operational risks of AT in Large-Scale biological attacks: A Red- Team approach. Technical report, RAND Corporation, October 2023. URL https:
Gabriel Mukobi, Hannah Erlebach, Niklas Lauffer, Lewis Hammond, Alan Chan, and Jesse Clifton. Welfare diplomacy: Benchmarking language model cooperation, October 2023. URL https://opearrrieev.ee// AKLZiid=AKInDgzz.mm
MULTIBEAM. Applications, 2024. URL httppp/s//muutllllooocccoooooopp Accessod: 2024-7-
17.
Antonio Muñoz, Ruben Ríos, Rodrigo Román, and Javier López. A survey on the (in (in)security of trusted execution environments. Computers & Security, 109:10,,80, June 2023. ISSN 0167-4048. URL https:
Micah Musser, Rebecca Gelles, Catherine Aiken, and Andrew Lohn. "the main resource is the human": A survey of AI researchers on the importance of compute. Technical report, Center for Security and Emerg- ing Technology, 2023. URL http://se//eett..eeoooo/oo////// Ls-the-human/!
David Mytton. Data centre water consumption. npj Clean Water, 4(1):1-6, February 2021. ISSN 2059-7037, 2059-7037. URL htttps://wwwnatuut http://ww...attteeee//////----00
Karthik Nandakumar, Nalini Ratha, Sharath Pankanti, and Shai Halevi. Towards deep neural network
training on encrypted data. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Workshops (CVPRW), pp. 40 48. IEEE, June 2019. ISBN 9781728125060, 9781728125077. URL http:
(dd..ooi.rrg/0..1009//////// http://0011000/P100.2201011...1. 2009.000111
Arvind Narayanan and Sayash Kapoor. AI safety is not a model property, 2024.
URL https://
80

---

Published in Transactions on Machine Learning Research (04/2025)
Arvind Narayanan, Sayash Kapoor, and Seth Lazar. Model alignment protects against accidental harms, not intentional ones, 2023. URL http://p////aaaaeeeeeoc.cc.oooap Accessed: 2024-7-15.
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, Feder Cooper, Daphne Ippolito, Christopher A Choquette- Choo, Eric Wallace, Florian Tramèr, and Katherine Lec. Scalable extraction of training data from (production) language models. arxie: 2311. 17015 [es.LG], November 2023. doi:
10.48550/ arvi wtw.w..3111.1703..
National Artificial Intelligence Research Resource Task Force. Strengthening and democratizing the U.S. artificial intelligence innovation ecosystem: An implementation plan for a national artificial intelligence research resource. Technical report, NAIRR, 2023.
National Institute of Standards and Technology (NIST). Biden-Harris administration announces NIST public working group AI, June 2023. URL http::/hhg///////aaaao/////a/rrr 2024-7-
14.
National Institute Standards and Technology (NIST). Biden-Harris adminnstrtion annouccss First-Ever
consortium dedicated AT safety, February 2024. URL http:://///wwaassssr//////sssssr
harris-rrae or www.aaaatattttteeeeeeeeeeeeerrrr cessed: 2024-7-16.
Ac-
NDIF proposal. National deep inference facility for very large language models (NDIF): Project proposal to the NSF, 2023. URL http:s//harraaabaaaae/////lttppp
Sree Harsha Nelaturu, Nishaanth Kanna Ravichandran, Cuong Tran, Sara Hooker, and Ferdinando Fioretto. On the fairness impacts of hardware selection in machine lcarning. arXiv: 2312.03886 [es.LG], December 2023. URL http:://aaaii.rrrr//////12.....0
Sella Nevo, Dan Lahav, Ajay Karpur, Yogev Bar-On, Henry Alexander Bradley, and Jelf Alstott. Securing Al Model Weights: Preventing Theft and Misuse of Frontier Models. RAND Corporation, Santa Monica, CA, 2024. URL
Thanh Tam Nguyen, Thanh Trung Huynh, Phi Le Nguyen, Alan Wec- ChunCCung Liow, Hongzhi Yin, and Quoe Viet Hung Nguyen. A survey of machine unlearning. arXiv: 2209.0228 09990229999.. [cs.LGj, September 2022. URL http://arxiv.org/ /bsp//bb/.22222999999
NHS Research SDE Network. Secure data environment, 2024. URL http:///aggiaalnnsus/eerrric..
daccure-dat--oe www.eeeennnntteereeeeeeee Accessod: 2024-7-17.
Gabriel Nicholas. Grounding policy: Towards rescarcher access to usage data. Technical report, Center for Democracy & Technology, August 2024. URL
Cowardd-reeerrcco--ccceee-s-- aai---ttttt---uaa-----taaaaaaa
Robert C Nickerson, Upkar Varshney, and Jan Muntermann. A method for taxonomy development and its application in information systems. European Journal of Information Systems, 22(3):336-23 336-359, 2013.
Weili Nic, Brandon Guo, Yujia Huang, Chaowei Xiao, Arash Vahdat, and Animashroc Anandkumar. Dif- fusion models for adversarial purification. In Proceedings of the 39th International Conference Ma- chine Learning, pp. 16805-16827. PMLR, June 2022. URL https://proccediigg.nrrpesssss html
NIST. NIST AIRC - Crosswalk Documents, 2023a. URL htlphtt:///arrriiitt.ooo/ RMF BMF__MM_____
Crosswalks Accessed: 2024-7-18.
NIST. NIST AI RMF playbook, 2023b. URL http::///aar..iitttoo/..tt. hagp://aggoooooaoooooooooooaaaaa Accessed: 2024-7-18.
81

---

Published in Transactions on Machine Learning Research (04/2025)
NIST. Artificial intelligence risk management framework: Generative artificial intelligence profile, 2024. URL.
Johannes Obermaier and Vincent Immler. The past, present, and future of physical security enclosures: From Battery-Backed monitoring to PUF-Based inherent. security and beyond. Journal of Hardware and Systems Security, 2(4):289-20) December 2018. ISSN 2509-3436. URL httpss//ooalor////000/////665665
0045-2
Joe O'Brien, Shaun Ee, and Zoe Williams. Deployment corrections: An incident response framework for frontier AI models. Technical report, Institute for AI Policy and Strategy, 2023. URL https:
models.pdf.
Serena Oduro and Tamara Kneese. AI governance needs sociotechnícal expertise: Why the humanities and
social sciences are critical to government efforts. Technical report, Data&rsociety, 2024. URI. https:
//ttatacdcy.aataproooaaa////apaa oorpaaaeeeaaaaaaooooooooooooooaa Polieccanneeellclcccooeeele .dff
OFCD. Oced principles on artificial intelligence. Organisation for Economic Co- and Development, 2019. URL http:://ooocccaaa///pppppppppppp
OECD. Stocktaking for the development of an Al incident definition. Technical report, OECD, Octo- ber 2023. URL
c323ac71-en
OECD.AT Policy Observatory. OECD AT incidents monitor, 2024. URL http:://eeed.aii/m/nniiieeer methodology Accessed: 2024-7-18.
Victor Ojewale, Ryan Steed, Briana Vecchione, Abeba Birhane, and Inioluwa Deborah Raji. Towards AI accountability infrastructure: Gaps and opportunities in Al audit tooling. arXiv: 2402.17861 fes. (Ys.CY).
February 2024. doi: 2015500/aaXii.222117788
Chris Olah. Interpretabilityy dreams, 2023. URL httpe://transtommrrrrrccuuuttttb
imeeppreaablltyyyrrre//mmtooooit Accessod: 2024-7-16.
OpenAl. Prepareduess, 2024a. URL
Accessed: 2024-5-26.
OpenAl. Reimagining secure infrastructure for advanced AI, 2024b. URL htttss://openaa.com//neeee Accessed: 2024-7-17.
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad,, Igge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadott-Shapiee Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cumminngs Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufci Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen Hc, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaisor, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo,
82

---

Published in Transactions on Machine Learning Research (04/2025)
Lukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Tkai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mchta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reichhrr Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr H Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Proehl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra. Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalic Staudacher, Felipe Petroski Such, Natalic Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B Thompson, Phil Tillet, Amin Tootoonchian, Flizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan Ward, Jason Wei, C J Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hamnah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojeiech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhnang, William Zhuk, and Barret Zoph. GPT-4 technical report. Technical report, OpenAl, March 2024. URL hattp://axxiv org/
Tribhuvanesh Orekondy, B Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of Black- Box models. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. December 2018. ISSN 1063-6919, 2575-7075 URL htttps://openaccess. wtwpeeenwwccsssseee....oooo
of content_ CvPrtt___________ 2019//apers/ Koockyy_nnooofffeo wttesteeeeeeeeeeetteeetaeeeeeeee ofllcck Black-
Mox_ModelssCrie C__R__CRP________ fatss_pppprprrrre
Yonatan Oren, Nicole Meister, Niladri Chatterji, Faisal Ladhak, and Tatsunori B Hashimoto. Proving test set contamination in black box language models. arXiv: 2310.17620 CLs, October 2023. URL http://arriv.org//a//22.....
Organisation for Economic Co-Operation and Development. A blueprint for building national compute capacity for artificial intelligence. Technical report, OECD, February 2023. URL
and-techhologgg//a-uuppinn cetpiaprrrin--o--uuu--e-------ee soallding-nationa
paapteerrpppiiyty---rrrtitiiaeee artiffiiaraiiiiiiaaa... htwp:tffiiicaaiiaaieeeeeeeeeeeea 876367e3-en
Siru Ouyang, Shuohang Wang, Yang Liu, Ming Zhong, Yizhu Jiao, Dan Tter, Reid Pryzant, Chenguang Zhu, Heng Ji, and Jiawei Han. The shifted and the overlooked: A task-oriented investigation of User-GPPT interactions. In Houda Bouamor, Juan Pino, and Kalika Bali (ods.). Proceedings of the 2023 Conference on Empirical Methods in Natural Longuage Processing, pp. 2375 2393, Singapore, December 2023. Association for Computational Linguistics. URL tittps://aclantho...
146
Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bern- stein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium User Interface Software and Technology (UIST 233, pp. 22, New York, NY, USA, October 2023a. Association for Computing Machinery. URL https: /ttt:////////.111//1//11188800
Peter S Park, Simon Goldstein, Aidan O'Gara, Michael Chen, and Dan Hendrycks. Al deception: A survey of examples, risks, and potential solutions. Patterns, 5(5), May 2024. ISSN 2666- 3899. URL ottpa///wwoooooooaooaa/oa/aaaaaa
83

---

Published in Transactions on Machine Learning Research (04/2025)
Sung Min Park, Kristian Georgiev, Andrew Ilyas, Guillaume Leclere, and Aleksander Madry. TRAK: Attributing model behavior at scale. Xin: 2303. 14186 (slat. March 2023b. URL WRt.httt///aaxiv.orgg abs/ 230..8363..
Otavio Parraga, Martin D More, Christian M Oliveira, Nathan S Gavenski, Lucas S Kupssinskii, Adilson Medronha, Luis V Moura, Gabriel S Simões, and Rodrigo C Barros. Fairness in deep lcarning: A survey on vision and language research. ACM Compul. Surv., December 2023. ISSN 0360-0300. URL https:
//8bi.org/10.114///888
Partnership on AI. PAI's guidance for safe foundation model deployment, October 2023. [/aattaershiiooaaaooo///odddeepp
URL https:
Accessed: 2024-7-18.
Dylan Patel. China Al & semiconductors rise: US sanctions have failed, 2023. URL https://
www.semi wtt.emmmmnalyyssscmmm////// wwwpw/////ppp/aaaaaaaaaaaaaaaaaa aadaaa-----eeeeee-eaaa -an--anddd-meeeee semiconducttos--rt Accessod: 2024-7-17.
David Patterson, Joseph Gonzalez, Urs Hölzle, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Danicl Rothchild, David R So, Maud Texier, and Jeff Dean. The carbon footprint of machine learning train- ing will plateau, then shrink. Computer, 55(7):18 28, July 2022. ISSN 0018-9162, 1558-0814. URL
https://iseexploreee www.://eepppooreeee.rrrroo/////e www.eeeeee.eeeeee /ee..orr///cccmme///8000077
David Patterson, Jeffrey M Gilbert, Marco Gruteser, Efren Robles, Krishna Sekar, Yong Wei, and Tenghui Zhu. Energy and emissions of machine learning on smartphones vs. the cloud. Communications of the ACM, 67(2):86-97, January 2024. ISSN 0001-0782. URL
Michael Quimn Patton. Qualitative research & evaluation methods. sage, 2002.
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The FineWeb datasets: Decanting the web for the finest text data scale. Xive 2406. 17557 [cs.CL], June 2024. URL
Shengyun Peng, Pin-Yu Chen, Matthew Hull, and Duen Horng Chau. Navigating the safely landscape: Measuring risks in finetuning large language models, arxiv: 2405.17374 [cs, May 2024. URL http:
//axxiv www.:/ogggaaaaaeeeeee 2eta//2055.7337 17374
Ethan Perez, Saffron Huang, Francis Song, Trevor Cai, Roman Ring, John Aslanides, Amelia Glaese, Nat McAleese, and Geoffrey Irving. Red leaming language models with language models. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP 2022), pp. 3419 3448, Abu Dhabi, United Arab Emirates, De- cember 2022. Association for Computational Linguistics. URL main.225
Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, Andy Jones, Anna Chen, Benjamin Mann, Brian Isracl, Bryan Sccthor, Cameron McKinnon, Christopher Olah, Da Yan, Danicla Amodci, Dario Amodei, Dawn Drain, Dustin Li, Eli Tran-Johnson, Guro Khundadze, Jackson Kernion, James Landis, Jamie Kerr, Jared Mueller, Jeeyoon Hyun, Joshua Landau, Kamal Ndousse, Landon Goldberg, Liane Lovitt, Martin Lucas, Michael Sellitto, Miranda. Zhang, Neerav Kingsland, Nelson Elhage, Nicholas Joseph, Noemi Mercado, Nova DasSarma, Oliver Rausch, Robin Larson, Sam McCandlish, Scott Johnston, Shauna Kravec, Sheer El Showk, Tamera Lanham, Timothy Telleen-Lawton, Tom Brown, Tom Henighan, Tris- tan Hume, Yuntao Bai, Zac Hatfield- -adiseld-Dodds, Jack Clark, Samuel R Bowman, Amanda Askell, Roger Grosse, Danny Hernandez, Deep Ganguli, Evan Hubinger, Nicholas Schiefer, and Jared Kaplan. Dis- covering language model behaviors with Model- Written evaluations. In Anna Rogers, Jordan Boyd- Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguisties: ACL 2023, pp. 13387 13434, Toronto, Canada, July 2023. Association for Computational Linguistics. URL https://aclantholoog] wwwp/r/gg///..e.ee http:/222..ffffo. findings-acll..c acc.nngssacl.877
Mary Phuong, Matthew Aitchison, Elliot Catt, Sarah Cogan, Alexandre Kaskasoli, Victoria Krakovna, David Lindner, Matthew Rahtz, Yannis Assael, Sarah Hodkinson, Heidi Howard, Tom Lieberum, Ramana
84

---

Published in Transactions on Machine Learning Research (04/2025)
Kumar, Maria Abi Raad, Albert Webson, Lewis Ho, Sharon Lin, Sebastian Farquhar, Marcus Hutter, Gregoire Deletang, Anian Ruoss, Seliem El-Sayed, Sasha Brown, Anca Dragan, Rohin Shah, Allan Dafoe, and Toby Shevlane. Evaluating frontier models for dangerous capabilities. Technical report, Google Deepmind, March 2024. URL
Mansi Phute, Alec Helbling, Matthew Daniel Hull, Shengyun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. LLM self defense: By self examination, LLMs know they are being tricked. In The Second Tiny Papers Track at ICLR 2024, Vienna, Austria, March 2024. URL http::s://pperreiii...ta
forum?id= Yorum?id=YoggeIA199
Aleksandra Piktus, Christopher Akiki, Paulo Villegas, Hugo Laurençon, Gérard Dupont, Alexandra Sasha Luccioni, Yacine Jernite, and Anna Rogers. The ROOTS search tool: Data transparency for LLMs. arXiv: 14035 (es. CLJ, February 2023. URL httt:://aratv.orgg/bb///// 14035
Hadrien Pouget. The EU's Al act barreling toward Al standards that do not exist, 2023. URL https:// www.:/aeerrrrrrreeeeeeeeeeeeeeee ai- aat-aacccabeelenne a at-aard-a--staadaa no-t exist cessed: 2024-7-18.
www
Ac-
Luiza Pozzobon, Beyza Ermis, Patrick Lewis, and Sara Hooker. On the challenges of using Black-Box APIs for toxicity evaluation in research. arviv: 2304.12397 fes. CLJ, April 2023. URL htttp://arxi.orrg/ org/abs/
2304. 12397
Usvan Sai Prashanth, Alvin Deng, Kyle O'Brien, V Jyothir, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, and Naomi Saphra. Recite, reconstruct, recollect: Memorization in LMs multifaceted phenomenon. arXiv: 1406617776 [es.CL] June 2024. URL
Presidency of the Council of the European Union. Proposal for regulation of the european parlia- ment and of the council laying down harmonised rules on artificial intelligence (artificial intelligence act) and amending cerlain union legislative acts analysis of the final compromise text with view to agreement. Technical Report 5662/24 LIMITE, Council of the European Union, 2024. URL 1011///////
Project Oak. Project oak, 2024. URL
Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson. Data cards: Purposeful and transparent datasct documentation for responsible AI. In Proceedings of the 2022 ACM Conference on Fairness, Accounlability, and Trunsparency, FAceT '22, pp. 1776 1826, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 9781450398522. URL https://doi.org// 11../10.11111 1tt.///......1111131111116.33330
Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin- Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine- tuning aligned language models compromises safety, even when users do not intend to! In The 12th International Conference on Learning Representations (ICLR 2024), Vienna, Austria, October 2023. URL http:///ppppeeeeeaaaaatrtrrrr/es
Xiangyu Qi, Ashwince Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Pratock Mittal, and Peter Henderson. Safety aligmimett should be made more than just a few tokens deep. arXiv: 2406.05944 (es. CRJ, June 2024a. URL
Xiangyu Qi, Boyi Wei, Nicholas Carlini, Yangsibo Huang, Tinghao Xie, Luxi He, Matthew Jagielski, Milad Nasr, Prateek Mittal, and Peter Henderson. On evaluating the durability of safeguards for open- weight LLMs. arXiv: 2412.0 07097 fes. CRJ, December 2024b. URL http:://aaaa...rrr/b/////2200000 wwwpo//////.///////////////
Zhenting Qi, Hanlin Zhang, Eric Xing, Sham Kakade, and Himabindu Lakkaraju. Follow my instruction and spill the beans: Scalable data extraction from Retrieval-Augmenned generation systems, arXiv: 2402.17840 fes. CL.), February 2024c. URL http:///ardi..r//abh22222...11
85

---

Published in Transactions on Machine Learning Research (04/2025)
Jenny Quang. Does training AI violate copyright law? Boolt Hall School of Law, University of California, Berkeley, 36(4):1407, 2021. ISSN 1086-3818. URL 3003-36-4Quang.poF
Manish Raghavan. The Societal Impacts of Algorithmic Decision-Making, volume 53. Association for Com- puting Machinery, New York, NY, USA, 1 edition, August 2023.
Noorjahan Rahman and Eduardo Santacana. Beyond fair use: Legal risk evaluation for training LLMs on copyrighted text, 2023. URL
Inioluwa Deborah Raji, Emily Denton, Emily M Bender, Alex Hanna, and Amandalynne Paullada. Al and the everything in the whole wide world benchmark. In 35th Conference on Neural Information Processing Systems (NewrIPS 2021) Datasets and Benchmarks Track (Round 2), Virtual, August 2021.
URL
Inioluwa Deborah Raji, I Elizabeth Kumar, Aaron Horowitz, and Andrew Selbst. The fallacy of ai fune- tionality. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transporency, pp. 959-972, 2022a.
Inioluwa Deborah Raji, Peggy Xu, Colleen Honigsberg, and Daniel Ho. Outsider oversight: Designing third party audit ecosystem for AI governance. In Proceedings of the 2022 AAAT/ACM Conference on AI, Ethics, and Society (AIES 222), pp. 557-571, 57,, New York, NY, USA, July 2022b. Association for Computing Machinery. ISBN 9781450392471. URL
Bogdana Rakova and Roel Dobbe. Algorithms as Social-Ecologee Ecological systems: An environmental justice lens algorithmic audits. arXiu: 2305.05733 [cs.CY] May 2023. URL http://arxxv.orr// /bbs/
2305.05733
Shauli Ravfogel, Francisco Vargas, Yoav Goldberg, and Ryan Cotterell. Adversarial concept erasure in kernel space. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pp. 6034-6055, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguisties. CRL httppaa///clammmalooogooorrr 2022 wwwweennnaaeeeeeeee main.405///
William Alan Reinsch, Matthew Schleich, and Thibault Denamiel. Insight into the U.S. semiconductor export controls updato, 2023. URL
concucttt -xxportrrtttrr-rrottttt--
controls-update Accessed: 2024-7-15.
Anka Reuel and Trond Arne Undheim. Generative AI needs adaptive governance. orXiv: 2406.04554 CYJ, June 2024. URL. htttp://arxiv.org/bbbs.
04554
Anka Reuel, Lisa Soder, Ben Bucknall, and Trond Arne Undheim. Position paper: Technical research and talent is needed for effective Al governance. arXiv: (006006987// fcs. CYJ, June 2024a. URL http:
berxiv.org/a abs/ wtbb//246.000m 2406.06987
Anka Reuel, Lisa Soder, Benjamin Bucknall, and Trond Arne Undheim. Position: Technical research and talent is needed for effective AI governance. In Forty-firs: International Conference on Machine Learning, BBoee
June 2024b. URL
M Sadegh Riazi, Christian Weinert, Oleksandr Tkachenko, Ebrahim M Songhori, Thomas Schneider, and Farinaz Koushanfar. Chameleon: A hybrid secure computation framework for machine learning appli- cations. In Proceedings of the 2018 Asía Conference Computer and Communications Security, ASTACCS '18, pp. 707-721, New York, NY, USA, May 2018. Association for Computing Machinery. ISBN 9781450335766. URL httpp///ddoo.orgg01111//////3///
Manlcy Roberts, Himanshu Thakur, Christine Herlihy, Colin White, and Samuel Dooley. To the cutoff... and beyond? a longitudinal perspective on LLM data contamination. In The Twelfth International Conference on Learning Representations, October 2023. URL http::///pparreerr....ee//////tp
86

---

Published in Transactions on Machine Learning Research (04/2025)
Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszeze, David Atanasov, Robie Gonzales, Sub- habrata. Majumdar, Carsten Maple, Hassan Sajjad, and Frank Rudzicz. Representation noising effec- tively prevents harmful fine-tuning on LLMs. arXiv: 2,05.14577 [es. (Ls.CL), May 2024a. URL http:
//arxiv.org/ abs/ 2205.
Domenic Rosati, Jan Wehner, Kai Williams, Lukasz Bartoszeze, Jan Batzner, Hassan Sajjad, and Frank Rudzicz. Immunization against harmful fine-tuning attacks. arXiv: 2402. 16382 [es.CL,, February 2024b.
URL http://arxxxv.or//as////// http://w..rr//ab//2.....000/
Marcello Ruberti. The chip manufacturing industry: Environmental impacts and eco-efficiency analysis.
The Science of the total environment, 858(Pt 1):157873 February 2023. ISSN 0048-9697, 1879-1026. URL
Theo Ryffel, Andrew Trask, Morten Dahl, Bobby Wagner, Jason Mancuso, Daniel Rueckert, and Jonathan Passerat-Palmbacli A generic framework for privacy preserving deep learning. arXiv: 1811.0401 [cs.LG]. November 2018. URL
Mchrdad Saberi, Vinu Sankar Sadasivan, Keivan Rezaci, Aounon Kumar, Atoosa Chegini, Wenxiao Wang, and Soheil Feizi. Robustness of Al-Image detectors: Fundamental limits and practical at- tacks. In The Twelfth International Conference on Learning Representations, October 2023. URL
Mohamed Sabt, Mohammed Achemlal, and Abdelmadjid Bouabdallah. Trusted execution environment: What it is, and what it not. In 2015 IEEE Trustcom Baaioom////DDaat/lllaaaaa volume 1, pp. 57 64. IEEE 2015.
Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. Can AT-Generated text be reliably detected? arXiv: 2309.11155 CLJ, March 2023. doi: 10.48550/
23Xiv.3003.1116.
Olawale Salaudoen and Moritz Hardt. ImageNot: A contrast with ImageNet preserves model rankings. arXiv: 2404.02112 12 LG), April 2024. URL http::///aaatt.oog////////4....
Girish Sastry, Lennart Heim, Haydn Belfield, Markus Anderljung, Miles Brundage, Julian Hazell, Cullen O'Keefe, Gillian K Hadfiel, Richard Ngo, Konstantin Pilz, George Gor, Emma Bluemke, Sarah Shoker, Janct Egan, Robert F Trager, Shahar Avin, Adrian Weller, Yoshua Bengio, and Diane Coyle. Computing power and the governance of artificial intelligence. arxiv: 2402.08797 (cs. CYJ, February 2024. URL http://aarsv.orr//a/222200..0121
Tomohiro Sawada, Daniel Paleka, Alexander Havrilla, Pranav Tadepalli, Paula Vidas, Alexander Kranias, John J Nay, Kshitij Gupta, and Aran Komatsuzaki. ARB: Advanced reasoning benchmark for large language models. orXiv: 2307.13693 CLJ, July 2023. URL httt:://arxiv.orr//b////23//
Rylan Schacffer. Pretraining on the test set is all you need. arXiv: 0809.08632 Jcs. CLJ, September 2023.
URL httt:://axii.orrrretr htsp//as./////000/ 08632
Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? In 37th Conference on Neural Information Processing Systems (NewrIPS 2023), New Orleans, LA, USA, November 2023. URL
Rylan Schaeffer, Hailey Schoelkopf, Brando Miranda, Gabriel Mukobi, Varun Madan, Adam Ibrahim, Herbie Bradley, Stella Biderman, and Sanmi Koyejo. Why has predicting downstream capabilities of frontier AI models with scale remained elusive? arXiv: 2406.04391 [cs.LGG], June 2024. URL. http://arxxv.org/
obs/2406.043911
Paul Scharre. Inturo-Proofing frontier AI regulation: Projecting future compute for frontier AI models.
Technical report, CNAS, March 2024. URL https: h/tp://wwwww/oooc. http:///www.aaaooooooo/oooo////a
froofin--rrotter------ aiagulaaiii.
87

---

Published in Transactions on Machine Learning Research (04/2025)
Jérémy Scheurer, Mikita Balesni, and Marius Hobbhahn. Large language models can strategically deceive their users when put under pressure. In ICLR 2024 Workshop Large Longuage Model (I.M) Agents, Vienna, Austria, March 2024. URL httpp://ppaarrieirreee///rtttttd
Jonas Schuett, Markus Anderljung, Alexis Carlier, Leonie Koessler, and Ben Garfinkel. From principles to rules: A regulatory approach for frontier AL. orXiu: 2407.07300 CYs, July 2024. URL http:
//arxiv.org/abs/// /
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mchdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, Patrick Schramowski, Srivatsa Kun- durthy, Katherine Crowson, Ludwig Schmidt, Robert Kaczmarczyk, and Jemia Jitsev. LAION-5B: An open large-scale dataset for training next generation image-text models. arxiv: 011008402 [cs. CV3, October 2022. URL http://arxiv.org/ wttp://gggbb/...eeeesss
Roei Schuster, Congzheng Song, Eran Tromer, and Vitaly Shmatikov. You autocomplete me: Poisoning vulnerabilitics in neural code completion. In 30th USENIX Security Symposium (USENIX Security 21),
pp. 1559 1575, 2021. URL http:p//www..aean...ooo////////s
Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensitivity spurious features in prompt design or: How learned start worrying about prompt formatting. arXio: 2310.11324 (es. CLJ, October 2023. URL httpp///arai.orr/aa/2220....2
Elizabeth Seger, Noemi Dreksler, Richard Moulange, Emily Dardaman, Jonas Schuett, K Wei, Christoph Winter, Mackenzie Arnold, Seán Ó hEigeartaigh, Anton Korinek, Markus Anderljung, Ben Bucknall, Alan Chan, Eoghan Stafford, Leonic Koessler, Aviv Ovadya, Ben Garfinkel, Emma Bluemke, Michael Aird, Patrick Levermore, Julian Hazell, and Abhishek Gupta. Open-Sourcing highly capable foundation models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives. Technical report, Centre for the Governance of AI, September 2023. URL httpp:/arxxv..rgg/ wtt../////....1.///222..
Andrew D Selbst. An institutional view of algorithmic impact assessments. Hurvard Journal of Law & Technology, 35(1), 2021.
Jaime Sevilla, Lennart Heim, Anson Ho, Tamay Besiroglu, Marius Hobbhahn, and Pablo Villalobos. Com- pute trends across three eras of machine learning. In 2022 International Joint Conference on Newral Net- works (IJCNN 2022), pp. 1 1-8, Padua, Italy, July 2022. URI. httppp///aaeepplorreeeeeeo//////
9891914
Rusheb Shah, Quentin Feuillade Montixi, Soroush Pour, Arush Tagade, and Javier Rando. Scalable and transferable Black-Box jailbreaks for language models via persona modulation. In 37th Conference on Newral Information Processing Systems (NeurIPS 2023) Socially Responsible Language Modelling Research Workshop (SoLaR), New Orleans, LA, USA, November 2023. URL httpss///ppprreeee..a/////orrrra x3Ltqz1UFg
Thanveer Shaik, Xiaohui Tao, Haoran Xie, Lin Li, Xiaofeng Zhu, and Qing Li. Exploring the landscape of machine unlearning: A comprehensive survey and taxonomy. arXiv: 2305. 06360 [cs.LG), May 2023. URL
http://arxiv.orge
Tommy Shaffer Shane. Al incident reporting: Addressing a gap in the UK's regulation of AL. Technical report, The Centre for Long-Term Resilience, June 2024. URL Uttppptte////wrrrooeeeeeeeeeeeer
http:/ai-inciieet--rrrrrtrni oootaaaetaaaroottttrreeaaoaererd
ShareGPT. ShareGPT, 2022. URL pttps://shareept..omm/
Accessed: 2024-7-17.
Yonadav Shavit. What does it take to catch chinchilla? Verifying rules on Large-Scale neural network training via compute monitoring. arXiv: 2303.11341 [cs.LG), March 2023. URL http:://arxv.org//aaa
2303.11841
88

---

Published in Transactions on Machine Learning Research (04/2025)
Yonadav Shavit, Sandhini Agarwal, Miles Brundage, Steven Adler Cullen O'Keefe, Rosie Campbell, Teddy Lee, Pamela Mishkin, Tyna Floundou, Alan Hickey, Katarina Slama, Lama Ahmad, Paul McMillan, Alex Beutel, Alexandre Passos, and David G Robinson. Practices for governing aagentic Al systems. Research Paper, OpenAl, 2023.
Erfan Shayegani, Md Abdullah AI Mamun, Yu Fu, Pedram Zarce, Yue Dong, and Nacl Abu-Ghazale Survey of vulnerabilities in large language models revealed by adversarial attacks. ar 3310010844 fes. [es.CL], October 2023. URL attp://arxiv.ct .ww../og/bbbbo
absgg/bb//// ha////b//222111 23100.04448
Renee Shelby, Shalalch Rismani, Kathryn Henne, Ajung Moon, Negar Rostamzadeh, Paul Nicholas, N'mah Yilla, Jess Gallegos, Andrew Smart, Emilio Garcia, and Gurleen Virk. Sociotechnical harms of algorithmic systems: Scoping a taxonomy for harm reduction. arXiv: 2210.05791 [cs.HC], October 2022. URL
Rence Shelby, Shalalch Rismani, Kathryn Henne, AJung Moon, Negar Rostamzadch, Paul Nicholas, N'Mah Yilla-Akbari, Jess Gallegos, Andrew Smart, Emilio Garcia, et al. Sociotechuical harms of algorithmic systems: Scoping a taxonomy for harm reduction. In Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society, pp. 723 741, 2023.
Toby Shevlane, Sebastian Farquhar, Ben Garfinkel, Mary Phuong, Jess Whittlestone, Jade Leung, Daniel Kokotajlo, Nahema Marchal, Markus Anderljung, Noam Kolt, Lewis Ho, Divya Siddarth, Shahar Avin, Will Hawkins, Been Kim, Iason Gabriel, Vijay Bolina, Jack Clark, Yoshua Bengio, Paul Christiano, and Allan Dafoe. Model evaluation for extreme risks. Technical report, Google DeepMind, September 2023.
URL httpp///axxv.org absaabss///// 2305.11322
Weijia Shi, Xiaochuang Han, Hila Gonen, Ari Holtzman, Yulia. Tsvetkov, and Luke Zettlemoyer. Toward human readable prompt tuning: Kubrick's the shining is good movie, and good prompt too? arXiv: 2212.10539 fes. CL), December 2022. URL httt:://rxxiv.or//aaa 2ttp:/2212..1050
Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. Detecting pretraining data from large language models. In The 12th International Conference on Learning Representations (ICLR 2024), Vienna, Austria, October 2023. URL https:
Weijia. Shi, Jaechan Lee, Yangsibo Huang, Sadhika Malladi, Jieyu Zhao, Ari Holtzman, Daogao Liu, Luke Zettllmooyr, Noah A Smith, and Chiyuan Zhang. MUSE: Machine Unlearning Six-Way Evaluation for Language Models. arXiv: 2407.064 060700460 fcs. CLJ, July 2024. doi: 10. 10.48550/ /ar500////iv../ 240..06460.
Taylor Shin, Yasaman Razeghi, Robert L. Logan, IV, Eric Wallace, and Sameer Singh. AutoPrompt: Eliciting knowledge from language models with automaticalle generated prompts. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), , Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP 2020), pp. 4222 4235, Online, November 2020. Association for Computational Linguistics. URL
www.o/2...eeeeeeeeoeo 202000220000.. emnlp-main.345
Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pp. 3 18, San Jose, CA, USA, 2017. IEEE. ISBN 9781509055333, URL
Ilia Shumailov, Zakhar Shunaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv: 2305. 17493 [cs.I.G], May 2023.
URL
Nianwen Si, Hao Zhang. Heyu Chang, Wenlin Zhang, Dan Qu, and Weiqiang Zhang. Knowledge unlearning for LLMs: Tasks, methods, and challenges. arXiv: //arxiv.orr//bbb//31111666
C.s.C,), November 2023. URL http:
Shoaib Ahmed Siddiqui, Nitarshan Rajkumar, Tegan Maharaj, David Krueger, and Sara Hooker. Meta- data archacology: Unearthing data subsets by leveraging training dynamics. arXiv: 10015 [es.LG),
September 2022. URL httttp///axxivorr//aaa///0///
89

---

Published in Transactions on Machine Learning Research (04/2025)
Jan Simson, Florian Pfisterer, and Christoph Kern. One model many scores: Using multiverse analysis to prevent fairness hhacking and evaluate the influence of model design decisions. In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT '24), pp. 1305 1320, New York, NY, USA, June 2024. Association for Computing Machinery. URL https: http://ddl.ccc..... org.acmmrr//////01114/5///
8630106.3658974
Anand Siththaranjan, Cassidy Laidlaw, and Dylan HadfelldMeenel. Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF. In The 12th International Conference on Learning Representations (ICLR), Vienna, Austria, 2024. URL attp:://apeameerieeentttttfoooo
OtWTxYYPaW
Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano. Participation is not a design fix for machine learning. In Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mech- anisms, and Optimization (EAAMO '22), pp. 1-6, New York, NY, USA, October 2022. Association for Computing Machinery. ISBN 9781450394772. URL http://p//a...aa..ooo/o/////////
Gregory T Smith. On construct validity: Issues of method and measurementt. Psychological assessment 17 (4):396-408, December 2005. ISSN 1040-3590. URL http:///.....lloog/////////00000
Kevin B Smith. Typologies, taxonomics, and the benefits of policy classification. Policy studies journal, 30 (3):379 395, 2002.
Shoshanna Sofaer. Qualitative methods: what are they and why use them? Health services research, 34(5 Pt 2):1101, 1999.
Irene Solaiman. The gradient of generative AI release: Methods and considerations. arxi:: 2302.04844 (es. CY,, February 2023. URL http:://arxxv..or////////////0// 048444
Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé, III, Jesse Dodge, Ellie Evans, Sara Hooker, Yacine Jernite, Alexandra Sasha Luccioni, Alberto Lusoli, Margaret Png, Andrew Strait, and Apostol Vassilev. Evaluating the social impact of generative AI systems in systems and socicty. arXiv: 2306. 9906.05949 [cs.CY], June 2023. URL http:///arxv..orr/ab//300...
Mitchell, Jessica Newman,
Liwei Song, Xinwei Yu, Hsuan-Tung Peng, and Karthik Narasimhan. Universal adversarial attacks with natural triggers for text classification. arXio: 2005.00174 fes.CT. May 2020. URL http://arxiv.org/ abs/2005.00174/
Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, Pieter Abbeel, Justin Svegliato, Scott Emmons, Olivia Watkins, and Sam Toyer. A StrongREJJCT for empty jailbreaks. arXiv: 2402.10260 [cs.LG], February 2024. URL http::/aaraiv..rr//b////9999
Stavros Souravlas and Stefanos Katsavounis. Scheduling fair resource allocation policies for cloud computing through flow control. Electronics, 8111):1348, November 2019. ISSN 2079-9292, 2079-9292. URL https:
// //tp./mdp..com//77999990 httppppp...co/0000g999//////11/1
Tobin South, Alexander Camuto, Shrey Jain, Shayla Nguyen, Robert Mahari, Christian Paquin, Jason Morton, and Alex 'sandy' Pentland. Verifiable evaluations of machine learning models using 2kSNARKs. arXiv: 3402.02675 [cs, February 2024. URL http:://axxxvorr///////22.......
Siddarth Srinivasan. Detecting AI fingerprints: A guide to watermarking and beyond. Technical report. Brookings, Jamuary 2024. URL http:///pp//wwwooooooaea/a////ee ainn-eeeeettttn---------eeernnss -agiie-to--tttrrrrrran------ and-
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Carriga-Alonss Agnieszka Kluska, Aitor Lewkowyez, Akshat Agarwal, Alethca Power, Alex Ray, Alex Warstadt, Alexander W Kocurek, Ali Safaya, Ali Tazarv, Alice Xiang, Alicia Parrish, Allen Nie, Aman Hussain, Amanda Askell, Amanda Dsouza, Am- brose Slone, Ameet Rahane, Anantharaman S Tyer, Anders Johan Andreassen, Andrea Madotto, Andrea
90

---

Published in Transactions on Machine Learning Research (04/2025)
Santilli, Andreas Stuhlmüller, Andrew M Dai, Andrew La, Andrew Lampinen, Andy Zou, Angela Jiang, Angelica Chen, Anh Vuong, Animesh Gupta, Anna Gottardi, Antonio Norelli, Anu Venkatesh, Arash Gholamidavoodi, Arfa Tabassun, Arul Menezes, Arun Kirubarajan, Asher Mullokandov, Ashish Sabhar- wal, Austin Herrick, Avia Efrat, Aykut Erdem, Ayla Karakag, B Ryan Roberts, Bao Sheng Loe, Barret Zoph, Barthomiej Bojanowski, Batuhan Ozyurt, Behmam Hedayalnia, Behnam Neyshabur, Benjamin In- den, Benno Stein, Berk Ekmekei, Bill Yuchen Lin, Blake Howald, Bryan Orinion, Cameron Diao, Cameron Dour, Catherine Stinson, Cedrick Argueta, Cesar Ferri, Chandan Singh, Charles Rathkopf, Chenlin Meng, Chitta Baral, Chiyu Wu, Chris Callison-Burch, Christopher Waites, Christian Voigt, Christopher D Man- ning, Christopher Potts, Cindy Ramirez, Clara E, Rívera, Clemencia Siro, Colin Raffel, Courtney Ashcraft, Cristina Garbacea, Damien Sileo, Dan Garrette, Dan Hendrycks, Dan Kilman, Dan Roth, C Daniel Free- man, Daniel Khashabi, Daniel Levy, Daniel Moseguí González, Danielle Perszyk, Danny Hernandez, Danqi Chen, Daphne Ippolito, Dar Gilboa, David Dohan, David Drakard, David Jurgens, Debajyoti Datta, Deep Ganguli, Denis Emelin, Denis Kleyko, Deniz Yurct, Derck Chen, Derek Tam, Dicuwke Hupkes, Diganta Misra, Dilyar Buzan, Dimitri Coelho Mollo, Diyi Yang, Dong-Ho Lee, Dylan Schrader, Ekaterina Shutova, Ekin Dogus Cubuk, Elad Segal, Eleanor Hagerman, Elizabeth Barnes, Elizabeth Donoway, Ellie Pavlick, Emanuele Rodolà, Emma Lam, Eric Chu, Eric Tang, Erkut Erdem, Ernie Chang, Ethan A Chi, Ethan Dyer, Ethan Jerzak, Ethan Kim, Eunice Engefu Manyasi, Evgenii Zheltonozhskii, Fanyue Xia, Fate- meh Siar, Fernando Martínez-Plumee, Francesca Happé, Francois Chollet, Frieda Rong, Gaurav Mishra, Genta Indra Winata, Gerard de Melo, Germán Kruszewski, Giambattista Parascandolo, Giorgio Mariani, Gloria Xinyue Wang, Gonzalo Jaimovitch-LLope, Gregor Betz, Guy Gur-Ari, Hana Galijasevic, Han- nah Kim, Hannah Rashkin, Hannaneh Hajishirzi, Harsh Mehta, Hayden Bogar, Henry Francis Anthony Shevlin, Hinrich Schuetze, Hiromu Yakura, Hongming Zhang, Hugh Mee Wong, Ian Ng, Isaac Noble, Jaap Jumelet, Jack Geissinger, Jackson Kernion, Jacob Hilton, Jaehoon Lee, Jaime Fernández Fisac, James B Simon, James Koppel, James Zheng, James Zou, Jan Kocon, Jana Thompson, Janelle Wing- field, Jared Kaplan, Jarema Radom, Jascha Sohl-Dickstein, Jason Phang, Jason Wei, Jason Yosinski, Jekaterina Novikova, Jelle Bosscher, Jennifer Marsh, Jeremy Kim, Jeroen Taal, Jesse Engel, Jesujoba Alabi, Jiacheng Xu, Jiaming Song, Jillian Tang, Joan Waweru, John Burden, John Miller, John U Balis, Jonathan Batchelder, Jonathan Berant, Jörg Frohberg, Jos Rozen, Jose Cernanndz-Orrlllle Joseph Boude- man, Joseph Guerr, Joseph Jones, Joshua B Tenenbaum, Joshua S Rule, Joyce Chua, Kamil Kanclerz, Karen Livescu, Karl Krauth, Karthik Gopalakrishnan, Katerina Ignatyeva, Katja Markert, Kaustubh Dhole, Kevin Gimpel, Kevin Omondi, Kory Wallace Mathewson, Kristen Chiafullo, Ksenia Shkaruta, Ku- mar Shridhar, Kyle McDonell, Kyle Richardson, Laria Reynolds, Leo Gao, Li Zhang, Liam Dugan, Lianhui Qin, Lidia Contreras- Ochaando, Louis-Philippe Morency, Luca Moschella, Lucas Lam, Lucy Noble, Lud- wig Schmidt, Luheng He, Luis Cliveros-ooónn, Luke Metz, Lütfi Kerem Senel, Maarten Bosma, Maarten Sap, Maartje Ter Hoeve, Maheen Farooqi, Manaal Faruqui, Mantas Mazeika, Marco Baturan, Marco Marelli, Marco Maru, Maria Jose Ramirez-Quintana, Marie Tolkiehn, Mario Giulianelli, Martha Lewis, Martin Potthast, Matthew L Leavitt, Matthias Hagen, Mátyás Schubert, Medina Orduna Baitemirova, Melody Arnaud, Melvin McElrath, Michael Andrew Yee, Michael Cohen, Michael Gu, Michael Ivan- itskiy, Michael Starritt, Michael Strube, Michal Swędrowski, Michele Bevilacqua, Michihiro Yasunga, Mihir Kale, Mike Cain, Mimee Xu, Mirac Suzgun, Mitch Walker, Mo Tiwari, Mohit Bansal, Moin Amin- naseri, Mor Geva, Mozhdeh Gheini, Mukund Varma T, Nanyun Peng, Nathan Andrew Chi, Nayeon Loe, Neta Gur-Ari Krakover, Nicholas Cameron, Nicholas Roberts, Nick Doiron, Nicole Martinez, Nikita Nangia, Niklas Deckers, Niklas MuennighofT, Nitish Shirish Keskar, Niveditha. S Iyer, Noah Constant, Noah Fiedel, Nuan Wen, Oliver Zhang, Omar Agha, Omar Fllbaghdddi, Omer Tevy, Owain Evans, Pablo Antonio Moreno Casares, Parth Doshi, Pascale Fung, Paul Pu Liang, Paul Vicol, Pegah Alipoormo- labashi, Peiyuan Liao, Percy Liang, Peter W Chang, Peter Eckersley, Phu Mon Htut, Pinyu Hwang, Piotr Milkowski, Piyush Patil, Pouya Pezeshkpour, Priti Oli, Qiaozhu Mei, Qing Lyu, Qinlang Chen, Rabin Banjade, Rachel Etta Rudolph, Raefer Gabriel, Rahel Habacker, Ramon Risco, Raphaël Millière, Rhythm Garg, Richard Barnes, Rif A Saurous, Riku Arakawa, Robbe Raymaekers, Robert Frank, Rohan Sikand, Roman Novak, Roman Sitelew, Ronan Le Bras, Rosanne Liu, Rowan Jacobs, Rui Zhang, Russ Salakhutdi- nov, Ryan Andrew Chi, Seungjae Ryan Lee, Ryan Stovall, Ryan Techan, Rylan Yang, Sahib Singh, Saif M Mohammad, Sajant Anand, Sam Dillavou, Sam Shleifer, Sam Wiseman, Samuel Gruetter, Samuel R Bow- man, Samuel Stern Schoenholz, Sanghyun Han, Sanjeev Kwatra, Sarah A Rous, Sarik Ghazarian, Sayan Ghosh, Sean Casey, Sebastian Bischoff, Sebastian Gehrmann, Sebastian Schuster, Sepideh Sadeghi, Shadi
91

---

Published in Transactions on Machine Learning Research (04/2025)
Hamdan, Sharon Zhou, Shashank Srivastava, Sherry Shi, Shikhar Singh, Shima Asaadi, Shixiang Shane Gu, Shubh Pachchigar, Shubham Toshniwal, Shyam Upadhyay, Shyamolima Shammie Debnath, Siamak Shakeri, Simon Thormeyer, Simone Melzi, Siva Reddy, Sneha Priscilla Makini, Soo-Hwan Lee, Spencer Torene, Sriharsha Hatwar, Stanislas Dehacne, Stefan Divic, Stefano Ermon, Stella Biderman, Stephanic Lin, Stephen Prasad, Steven Piantadosi, Stuart Shieber, Summer Misherghi, Svetlana Kiritchenko, Swa- roop Mishra, Tal Linzen, Tal Schuster, Tao Li, Tao Yu, Tariq Ali, Tatsunori Hashimoto, Tc-Lin Wu, Théo Desbordes, Theodore Rothschild, Thomas Phan, Tianle Wang, Tiberius Nkinyili, Timo Schick, Timo- fei Kornev, Titus Tunduny, Tobias Gerstenberg, Trenton Chang, Trishala Neeraj, Tushar Khot, Tyler Shultz, Uri Shaham, Vedant Misra, Vera Demberg, Victoria Nyamai, Vikas Raunak, Vinay Venkatesh Ra- masesh, Vinay Uday Prabhu, Vishakh Padmakumar, Vivek Srikumar, William Fedus, William Saunders, William Zhang, Wout Vossen, Xiang Ren, Xiaoyu Tong, Xinran Zhao, Xinyi Wu, Xudong Shen, Yadollah Yaghoobzadeh, Yair Lakretz, Yangqiu Song, Yasaman Bahri, Yejin Choi, Yichi Yang, Yiding Hao, Yifu Chen, Yonatan Belinkov, Yu Hou, Yufang Hou, Yuntao Bai, Zachary Scid, Zhuoye Zhao, Zijian Wang, Zijie J Wang, Zirui Wang, and Ziyi Wu. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. Transactions Machine Learning Research, May 2023. ISSN 2835-8856.
URL
Saurabh Srivastava, Annarose M B, Anto P V, Shashank Menon, Ajay Sukumar, Adwaith Samod T, Alan Philipose, Stevin Prince, and Sooraj Thomas. Functional benchmarks for robust evaluation of reasoning performance, and the reasoning gap. aivi:: 2402.19450 [es.Alf, February 2024. URL abs/2402.19450/
Jacob Steinhardt, Pang Wei Koh, and Percy Liang. Certified defenses for data poisoning attacks. arXiv: 1 1706.03691 [es.LGJ, June 2017. URL. http://aaxxiv.orr/aa//////6
Andrei Stoian, Jordan Frery, Roman Bredehoft, Luis Montero, Celia Kherfallah, and Benoit Chevallier- Mames. Deep neural networks for encrypted inference with TFHE. arXiv: 2302.10906 [cs.LG), February 2023. URL httppp//aaa.iiioorrra/////221110
Vincent J Straub, Deborah Morgan, Jonathan Bright, and Helen Margetts. Artificial intelligence in gov- ernment: Concepts, standards, and unified framework. Government information quarterly, 40(4): 101881, October 2023. ISSN 0740-624X. URL httppp//wwwaataaeeddderccc.eooes
50740624X23000813
Milton E Strauss and Gregory T Smith. Construct validity: Advances in theory and methodology. Annual review of clinical psychology, 5:1 25, 2009. ISSN 1548-5943, 1548-5951. UtLattp://bb...o..rr////... annurev. Sanurev.llinsy..3222. 032408115633
Emma Strubell, Ananya Ganesh, and Andrew McCallum. Energy and Policy Considerations for Deep Learning in NLP. In Anna Korhonen, David Traum, and Lluis Mårquez (eds.), Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 3645-3650, Florence, Italy, July 2019. Association for Computational Linguistics. URL htttp::///ccaattooooogoo///////
Jingtong Su, Julia Kempe, and Karen Ullrich. Mission Impossible: A statistica perspective on jailbreaking LLMs. arXiv: 2408.0142 Lcs. August 2024. URL http:://arxvv.orggaa////////////
Nishant Subramani, Sasha Luccioni, Jesse Dodge, and Margaret Mitchell. Detecting personal informa- tion in training corpora: an analysis. In Anaelia Ovalle, Kai-Wei Chang, Ninareh Mehrabi, Yada Pruksachatkun, Aram Galystan, Jwala Dhamala, Apurv Verma, Trista Cao, Anoop Kumar, and Rahul Gupta (eds.), Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023). pp. 208-220, Toronto, Canada, July 2023. Association for Computational Linguistics. URL https://aclanthologg httpttthwoooooooooooooooooo wtwpg/223..trusee 1r22..ruustnl-111
Arjun Subramonian, Xingdi Yuan, Hal Daumé, III, and Su Lin Blodgett. It takes two to tango: Navigating conceptualizations of NLP tasks and measurements of performance. arviv 2305.09022 fes. CL/. May 2023. URL htt.:/ttg./aarrrrrrrr////////...
92

---

Published in Transactions on Machine Learning Research (04/2025)
Haochen Sun, Jason Li, and Hongyang Zhang. zkLLM: Zero knowledge proofs for large language models. 16109
arXiv: 2404. 16109 Jcs.I.G] April 2024. URI http:/tp////xxxi.oro////////////
Rishub Tamirisa, Bhrugu Bharathi, Long Phan, Andy Zhou, Alice Gatti, Tarun Suresh, Maxwell Lin, Justin Wang, Rowan Wang, Ron Arel, Andy Zou, Dawn Song, Bo Li, Dan Hendrycks, and Mantas Mazeika, Tamper-resistant safeguards for open-weight LLMs. arXiv: 2408.00761 [cs.LG], August 2024. URL
Tan, Knott, Tian, and Wu. CryptGPU: Fast
machine learning on the GPU. In 2021 IEEE Symposium on Security and Privacy (SP), volume 0, pp. 1021 1038, May 2021. URL http://
dx.doi.oog wto..oog/1..110000///// http:/0000101/900000000000000000
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C Daniel Freeman, Theodore Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interprctable features from claude 3 sonnet. Transformer Circuils Thread, 2024. URL httpee:/rrraasffrrrrrrrccrtttttt pubb/ 2024/scaling /ww.aaanggoooooomaaiiiiiiiiiiaaa html
The Allen Institute for Artificial Intelligence. WildChat, 2024. URL htRphtttpp///illcaaaaallleee Accessed: 2024-7-17.
The White House. Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence. Technical report, The White House, October 2023a. URL https:
www.whitteoouse.ov/biaii on htt.hh/hee////ttaarooaaaaaaaaatr
The White House. FACT SHEET: President biden issues executive order on safe, secure, and trustworthy
artificial intelligence, October 2023b. URL http:///t/////.aatiit//////////p
www.orrdrrrroo-------e------e-ee saw--ssfee--aeeeeeee secure-
on-
and hatr:rrtaseertttaaaaffttllllllln Accessed: 2024-4-23.
The White House Office of Science and Technology Policy. Blueprint for an AI bill of rights: Making auto-
mated systems work for the american people. Technical report, White House, November 2023. https:
//ww.whitehouse.goo http:/twwwsspepeeeeeee/////e///a
David Thiel. Identifying and eliminating CSAM in generative ML training data and models. Technical report, Stanford Digital Repository, 2023. URL
Cheng Ting-Fang. ASML says decoupling chip supply chain is practically impossible. Financial Times, 2023.
URL htctp:://ww..ft.cmmmoottttet htt.://///....ooe////e/0/0000000
Helen Toner, Jessica Ji, John Bansemer, Lucy Lim, Chris Painter, Courtney Corley, Jess Whittlestone, Matt Botvinick, Mikel Rodriguez, and Ram Shankar Siva Kumar. Skating to where the puck is going: Anticipating and managing risks from frontier AI systems. Technical report, Center for Security and Emerging Technology, 2023. URL http: / http://xxxdii.oorg www.:/ooo..oggg..o..
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bash- lykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Fer- rer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nic, Andrew Poulton, Jeremy Reizenstein, Rashi Rungla, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subra- manian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Ilyan Zarov, Yuchen Zhang, Angela Fan, Mclanic Kambadur, Sharan Narang, Aurclion Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and Fine- Tuned chat models. Technical report. Meta AI, July 2023. http::///arrii.orrgaa//////0..0
93

---

Published in Transactions on Machine Learning Research (04/2025)
Robert Trager, Ben Harack, Anka Reuel, Allison Carnegie, Lennart Heim, Lewis Ho, Sarah Kreps, Ran- jit Lall, Owen Larter, Seán Ô hFigeartaigh, Simon Staffell, and José Jaime Villalobos. International governance of civilian AI: A jurisdictional certification approach. Technical report, Oxford Martin Al Governance Initiative, August 2023.
Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction APIs. In Proceedings of the 25th USENIX Conference Security Symposium (6)C'16), pp. 601-618, USA, August 2016. USENIX Association. ISBN 9781931971324.
Andrew Trask. Building safe AT: A tutorial for enerypted deep learning, 2017. URL https:// lamtrask. http://atttttbbbo/o/2///////////
Accessod: 2024-7-17.
Andrew Trask, Akshay Sukumar, Antti Kalliokoski, Bennett Farkas, Callis Ezenwaka, Carmen Popa, Curtis Mitchell, Dylan Hrebenach, George-Cristian Muraru, lonesio Junior, Irina Bejan, Ishan Mishra, Ivoline Ngong, Jack Bandy, Jess Stahl, Julian Cardonnet, Kellye Trask, Kellye Trask, Khoa Nguyen, Kien Dang, Koen van der Veen, Kyoko Eng, Lacey Strahm, Laura Ayre, Madhava Jay, Oleksandr Lytvyn, Osam Kyemenu-Sarsah, Peter Chung, Peter Smith, Rasswanth, Ronnie Falcon, Shubham Gupta, Stephen Gabriel, Teo Milea, Theresa Thoraldson, Thiago Porto, Tudor Cebere, Yash Gorana, and Zarreen Reza. How to audit an Al model owned by someone else (part 1), 2023. URL attps:://blog.opemiiee... a1 ailaudit-parte 1/ Accessed: 2024-7-17.
Alexey Turchin and David Denkenberger. Classification of global catastrophic risks connected with artificial intelligence. AI & society, 35(1)1147-163, 2018. ISSN 0951-5666, 1435-5655. URL ltttps///dii.rr// 10.1007/800146-08--68844
Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering langnage models without optimization. arXiv: 2908.10246 10248 CLJ, August 2023. doi: 10.48550/ar /ae///aXXVvv//00 100..1048.
Vishaal Udandarao, Ameya Prabhu, Adhiraj Ghosh, Yash Sharma, Philip H S Torr, Adel Bibi, Samuel Albanie, and Matthias Bethge. No "Zero-Shot" without exponential data: Pretraining concept fre- quency determines multimodal model performance. arXiv: 2404.04125 jes. CVJ, April 2024. URL
UK AI Safety Institute. Inspect, 2024. URL https://iigoormmmmoooaaaathlllap ai/ Ac- cessed: 2024-7-15.
UK Research and Innovation. +000 million to launch first phase of new AI research re-
wwtp:wukr.oom source, 2023. URL htapps///www.kkki..oore///////// research-resource/
new-ai-
Accessed: 2024-7-17.
U.S. National Telecommunications and Information Administration. Artificial intelligence: Accountabil- ity policy report. Technical report, NTIA, 2024. URL https://wwwttiagrr//tt///f/aaats publicationsnntien -reporrrt- aerprroot--iaall final.pdi pdf.
Ahmet Üstün, Viraat Aryabumi, Xin Yong, Wei-Yin Ko, Daniel D'souza, Gbemileke Onilude, Neel Bhandari, Shivalika Singh, Hui-Lee Ooi, Amr Kayid, Freddie Vargus, Phil Blunsom, Shayne Longpre, Niklas Mucnnighoff, Marzich Fadacc, Julia Kreutzer, and Sara Hooker. Aya model: An instruction finetuned Open-Aceess multilingual language model. arxiv: 2402. 078277827 [es.CL], February 2024. URL
Risto Uuk, Carlos Ignacio Gutierrez, Daniel Guppy, Lode Lauwaert, Atoosa Kasirzadeh, Lucia Velasco, Peter Slattery, and Carina Prunkl. A taxonomy of systemic risks from general-purpose ai. arXiv preprint arXiv:2412.07778 077070 2024.
Stephan van Schaik, Adam Batori, Alex Scto, Bader AlBassam, Christina Garman, Thomas Yurck, An- drew Miller, Daniel Genkin, Eyal Ronen, and Yuval Yarom. Fail 2022. URL https: bttpps//ssx..ail1 Accessed: 2024-7-17.
94

---

Published in Transactions on Machine Learning Research (04/2025)
Apostol Vassilev, Alina Oprea, Alie Fordyce, and Hyrum Anderson. Adversarial machine learning: A tax- onomy and terminology of attacks and mitigations. Technical report, U.S. National Institute of Stan- dards and Technology, Gaithersburg, MD, January 2024. URL http:://wwppueeaaatggevvniiitttt
AI1NIST 2e0000-222233.pp.
Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. Split learning for health: Distributed deep learning without sharing raw patient data. arXiv: 1812. 00564 (cs. December 2018. URL /tttttp:///arrrtrrrr/////////// attp//////////.55.....
Pablo Villalobos, Jaime Sevilla, Lennart Heim, Tamay Besiroglu, Marius Hobbhahn, and Anson Ho. Will we run out of data? Limits of LLM scaling based on human-generated data. orxi 2211.0453 1.04325 [cs.LG). October 2022. URL http://arxiv.oo/ wwwp /ttp///////21132555555
Sameer Wagh, Divya Gupla, and Nishanth Chandran. SecureNN: 3-party secure computation for neural network training. Proceedings on Privacy Enhancing Technologies, 2019(3):26 49, July 2019. ISSN 2299- 0984. URL
Sameer Wagh, Shruti Tople, Fabrice Benhamouda, Eyal Kushilevitz, Prateek Mittal, and Tal Rabin. FALCON: Honest-Majority maliciously secure framework for private deep learning. arxiv: 2004.022 0004.02229 (es.CRf. April 2020. URL
Suppakit Waiwitlikhit, Ion Stoica, Yi Sun, Tatsunori Hashimoto, and Daniel Kang. Trustless audits with- out revealing data or models. arXiv: 2404.04500 [cs.CR], April 2024. URL http::/arxivvorg//aaa
Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. Universal adversarial trig- gers for attacking and analyzing NLP. In Kentaro Imui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference Natural Language Processing (EMNLP-IJCNLP 2019), pp. 2153 2162, Hong Kong, China, November 2019. Association for Computational Linguisties. URL
Eric Wallace, Tony Z Zhao, Shi Feng, and Sameer Singh. Concealed data poisoning attacks on NLP models. arXiv: 2010.12563 fcs. CLJ, October 2020. URL
Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang T Truong, Simran Arora, Mantas Mazeika, Dan Hendryeks, Zinan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. DecodingTrust: A comprehensive as- sessment of trustworthiness in GPT models. In 37th Conference on Neural Information Processing Sys- tems (NewrIPS 2023) Datasets and Benchmarks Track, New Orleans, LA, USA, November 2023a. URL sttp:://pparrriis.ne///ooo 2ttpe/fooomm7ddkkaxlp0000222
Wei Wang, Ben Liang, and Baochun Li. Multi Rettiieeeurrcee fair allocation in heterogeneous cloud computing systems. JEEE Transactions on Parallel and Distributed Systems, 26(10):2822 2835, October 2015. ISSN 1045-9219, 1558-2183. URL http::/dxx.oo..rrg10100//////1/9
Wenxuan Wang, Zhaopeng Tu, Chang Chen, Youliang Yuan, Jen- Huang, Wenxiang Jiao, and Michael R Lyu. All languages matter: On the multilingual safety of large language models. arXiv: 2310.00905 fes. CL), October 2023b. URL http:://arxlv.or//abs/// 2310000005
Yuhang Wang, Yanxu Zhu, Chao Kong, Shuyu Wei, Xiaoyuan Yi, Xing Xie, and Jitao Sang. CDEval: A benchmark for measuring the cultural dimensions of large language models, arXio: 2311.16421 (es. CL), November 2023c. URL http:///arriv.orr//bb//311....
Matthew Wansley. Regulation of emerging risks. Vanderbilt Law Review, 69(2):401, 2016. ISSN 0042-2533. URL http::://schoolasshipp..... httpa//p........oioaaaaaaaaaaaaa
95

---

Published in Transactions on Machine Learning Research (04/2025)
Debora Weber- Weler-Wulff, Alla Anohina-Naumece Sonja Bjelobaba, Tomáš Foltýnek, Jean Guerrero-Dib, Olu- mide Popoola, Petr Sigut, and Lorna Waddington. Testing of detection tools for AT-generated text. International Journal for Educational Integrity, 19(1): 39, December 2023. ISSN 1833-2595, 1833-2595. URL
Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does LLM safely training fail? In 37th Conference on Neural Information Processing Systems (NeurIPS 2023), New Orleans, LA, USA, November 2023. URL htttp:///gpaaaeeeeeeeeee////ttap
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoe V Le, and Denny Zhou. Chain-of- Chain-of-Thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (NewrIPS 2022), volume 35, pp. 24824- 24837, December 2022. URL
Johnny Tian-Zheng Wei, Ryan Yixiang Wang, and Robin Jia. Proving membership in LLM pretraining org/////////////
data via data watermarks. arXiv: 2402.10892 CRJ, February 2024. URL
2402.10892
Jun Junyi Wei, Yicheng Zhang, Zhe Zhou, Zhou Li, and Mohammad Abdullah Al Faruque. Leaky DNN: Stealing Deep-Learning model secret with GPU Context-Switching Side- Channel. In 2020 50th Annual IEEE/IFII International Conference on Dependable Systems and Networks (DSN), pp. 125 137. TEFE, June 2020. ISBN 9781728158099, 9781728158105. URL httpp//////.......oo//////////// www.:///.../////00/0000111 20000666.....0.000 2020.00031
Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mcllor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, Courtney Biles, Sasha Brown, Zac Kenton, Will Hawkins, Tom Stepleton, Abeba Birhane, Lisa Anne Hendricks, Laura Rimell, William Isaac, Julia Haas, Scan Legassick, Geoffrey Irving, and Iason Gabriel. Taxonomy of risks posed by language models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT '22, pp. 214 229, New York, NY, USA, June 2022. Association for Computing Machinery. ISBN 778145039352.. URL httpps//000..0rg/1.114////////5
Laura Weidinger, Maribeth Rauh, Nahema Marchal, Arianna Manzini, Lisa Anne Hendricks, Juan Matcos- Garcia, Stevie Bergman, Jackie Kay, Conor Griffin, Ben Bariach, lason Gabriel, Verena Rieser, and William Isaac. Sociotechnical safety evaluation of generative AI systems. Technical report, Google Deepmind, October 2023. URL htttp:://rxxv.oo///aa www.oorggg/////e...eeo.oe.eeeeee abs/2300.121955
Laura Weidinger, John Mellor, Bernat Guillen Pegueroles, Nahema Marchal, Ravin Kumar, Kristian Lum, Canfer Akbulut, Mark Diaz, Stevie Bergman, Mikel Rodriguez, Verena Rieser, and William Isaac. STAR: SocioTechnical approach to red teaming language models. arXiv: 2406. 1 11757 [es.AT], June 2024. URL
http://aarxi.orr/aas/2000.1111
Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. Challenges in detoxifying language models. arXiv: 2109.07445 fcs. CLJ, September 2021. URL http:://arxxv..o//aaa////1. 07445
Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping, and T Goldstein. Hard prompts made easy: maadient-based discrete optimization for prompt tuning and discovery. In Advances in Neu- ral Information Processing Systems 36 (NewrIPS 2023) Main Conference Track, volume abs/ 200/0300.0366.. February 2023. URL http:://x..xoo//
http//xx.dii.ooo .or./ioor//ooo.. 1...0..850/./. www5////000..//222 wr3.....2.v.2..666....
Drew Westen and Robert Rosenthal. Quantifying construct validity: Two simple measures. Jour-
nal of personality and social psychology, 84(3) 618, March 2003. ISSN 0022-3514. URL https:
/
Wex Definitions Team. joint and several liability, 2023. URL https://ww.lex.comeell.com
joint_ aadaaandddrdaaaa several litiiiiity Accessed: 2024-7-16.
96

---

http://awwi..orraab////aaaa Published in Transactions on Machine Learning Research (04/2025)
Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz- Ziv, Neel Jain, Khalid Saifullah, Siddartha Naidu, Chinmay Hegde, Yann LeCun, Tom Goldstein, Willie Neiswanger, and Micah Goldblum. Livebench: A challenging, contamination-feee Ilm benchmark, June 2024. URL htttps://axxiv.rrg/ wttgg/bb////.....///
Jess Whittlestone and Jack Clark. Why and how governments should monitor AI development. arXiv: 21 2108.12427 [es.CY], August 2021. URL http:///arrxvorr///sa22......2.2
Wikipedia contributors. TIA-942 en. o.w.kktped...oggg///........tttp http://w..ppppttta....eooo222aa. 012-177----22221011111788888888
Wikipedia, the free encyclopedia, 2023.
URL. https://
Wu, Li Liu, Yuchen Guo, Guiguang Ding, J Han, Jialie Shen, and Ling Shao. Unsupervised deep video hashing with balancod rotation. International Joint Conference on Artificial Intelligence, pp. 3076-3082, August 2017. URL
Tim Wu. In regulating A.I. may be doing too much. and too little. The New York Times, November 2023. ISSN 0362-4331, 1553-8095. URL /ttp:///ceeeed/////////////33 .df regulation.html
Xinwei Wu, Junzhuo Li, Minghui Xu, Weilong Dong, Shuangzhi Wu, Chao Bian, and Deyi Xiong. DEPN: Detecting and editing privacy neurons in pretrained language models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Process- ing (EMNLP 2023), pp. 2875 2886, Gateway, Singapore, December 2023. Association for Computational http///aaanaoooooorrrr/r////////
Linguistics. URL
Pengtao Xie, Misha Bilenko, Tom Finley, Ran Gilad Bachrach, Kristin Lauter, and Michael Nachrig. Crypto- Nets: Neural networks over encrypted data. arXiv: 1412.6181 (cs.LG), December 2014. URL http:
//arxiv.org/abs//412..55
Jiashu Xu, Fei Wang, Mingyu Derek Ma, Pang Wei Koh, Chaowei Xiao, and Muhao Chen. Instrue- tional fingerprinting of large language models. arXi:: 2401.12255 fes. CR), January 2024. URL
Xiaojun Xu, Qi Wang, Huichen Li, Nikita Borisov, Carl A Gunter, and Bo Li. Detecting AI trojans using meta neural analysis. In 2021 IEEE Symposium on Security and Privacy (SP), pp. 103-120. IEEE, May 2021. ISBN 0781728189345 9781728189357
URL http:://ar.ooooorr///0001/////00
Xin Xu and Huiqun Yu. A game theory approach to fair and elficient resource allocation in cloud computing. Mathematical Problems in Engineering, 2014:1-14, 2014. ISSN 1024-123X, 1563-5147. URL http://
www www.hhhinww..coo/m/om http:///orraaaaa////////////////
Chhavi Yadav, Amrita Roy Chowdhury, Dan Bonch, and Kamalika Chaudhuri. Fairproof Confidential
and certifiable fairness for neural networks. arXiv: 2402. 12572 [es.LG/, February 2024. URL https:
//arxiv.org/abll abtp//bbbb/22225222
Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. Shadow alignment: The case of subverting Safely-Aligeed language modols. arXiv: 2810.02949 fes. CL), October 2023. URL htttp://arxiv.orgg/ absaaass////1 httg//abs2311.0.....
Andrew C Yao. Protocols for secure computations. In 23rd Annual Symposium on Foundations of Compuler Science (sfes 1982), pp. 160-164. TEFE, November 1982. URL httppp/p//...o..ooog///////////0
Andrew Chi-Chin Chi-Chih Yao. How to generate and exchange secrets. In 27th Annual Symposium on Foundations of Computer Science (sfcs 1986), pp. 162 167. IEEE, October 1986. ISBN 9780818607400. URL http:
/ddr.dblorrg/0..00///ssss...
Yuanshun Yao, Xiaojun Xu, and Yang Liu. Large language model unlearning. orXiv: 2310.10683 (es. CL), October 2023. URL http:///rri..orr/hhb///00....1
97

---

Published in Transactions on Machine Learning Research (04/2025)
Sconghyoon Ye, Doyoung Kim, Sungdong Kim, Hyeonbin Hwang, Seungone Kim, Yongrae Jo, James Thorne, Juho Kim, and Minjoon Seo. FLASK: Fine-grained language model evaluation based on alignment skill sets. arxiv: 2307.10928 fcs. CLJ. July 2023. URL
Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. arxiv: 2306. 135/9 [cs. CVj, June 2023. httL:ttt.///arrr...r///////2
Zheng Xin Yong, Cristina Menghini, and Stephen Bach. Low-Resource languages jailbreak GPT-4. In NewrIPS Workshop on Socially Responsible Language Modelling Research (SoLaR), New Orleans, LA, USA, November 2023. URL httpp:///pooooeeeesssssseo/ooosr
Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal, and Sanjeev Arora. Skill-mix: a flexible and expandable family of evaluations for ai models. arXiv: 2310. 17567 [es. CLJ, October 2023,
URL https:://rxiii www.ooog/g/////eeeeeeeeeeeeeeeee httprgg/ab//33....eee 201.//13311757777
Yi Zeng, Kevin Klyman, Andy Zhou, Yu Yang, Minzhou Pan, Ruoxi Jia, Dawn Song, Percy Liang, and Bo Li. AI risk categorization decoded (AIR 2024): From government regulations to corporate policies. arXiv: 2406.1786, [cs. CYJ, June 2024. URL
Qiusi Zhan, Richard Fang, Rohan Bindu, Akul Gupta, Tatsunori Hashimoto, and Daniel Kang. Removing RLHF protections in GPT-4 via Fine-Tuning. In 2024 Annual Conference of the North American Chapler of the Association for Computational Linguistics, Mexico City, Mexico, 2024. URL http://arxiv.org/ abs/2311.05555
Hanlin Zhang, Benjamin L Edelman, Danilo Francati, Daniele Venturi, Giuseppe Ateniese, and Boaz Barak. Watermarks the sand: Impossibility of strong watormarking for generative models. arXiv: 2911.04378 Les,LG], November 2023. doi: 10.48550/arK 231.0.37.
Hugh Zhang, Jeff Da, Dean Lee, Vaughn Robinson, Catherine Wu, Will Song, Tiffany Zhao, Pranav Raja, Dylan Slack, Qin Lyu, Sean Hendryx, Russell Kaplan, Michele Lunati, and Summer Yue. A careful examination of large language model performance on grade school arithmetic. arXiv: 2405.00332 CL), May 2024. URL httt:::/arxdv.org/ab////aa
Rui Zhang, Jian Liu, Yuan Ding, Zhibo Wang, Qingbiao Wu, and Kui Ren. "adversarial examples" for Proof-of- -ear-ing.... In 2022 IEEE Symposium on Security and Privacy (SP), pp. 1408-1422. IEEE, May 2022. ISBN 9781665413169, 9781665413176. URL http:///d..........00000010/////
h0tp://1.1110000
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Lú, Dacheng Li, Eric Xing, Hao Zhang, Joseph F. Gonzalez, and Ton Stoica. Judging L.M.- as-a-Judge with MT-BBechh and chatbot arena. In 37th Conference on Newral Information Processing Systems (NeurIPS 2023) Datasets and Benchmarks Track, New Orleans, LA, USA, November 2023. URL
Andy Zhou, Bo Li, and Haohan Wang. Robust prompt optimization for defending language models against jailbreaking attacks. In ICLR 2024 Workshop on Secure and Trustworthy Large Language. Models, Vienna, Austria, April 2024. URL http::///aaaaeeeeeeeeeee//eooor0
Kun Zhou, Yutao Zhu, Zhipeng Chen, Wentong Chen, Wayne Xin Zhao, Xu Chen, Yankai Lin, Ji-Rong Wen, and Jiawei Han. Don't make your LLM an evaluation benchmark cheater. arxiv: 2311.01964 (cs. CL), November 2023a. URL http:::://aaxx...og/aa/////1////
Xin Zhou, Yi Lu, Ruotian Ma, Tao Gui, Qi Zhang, and Xuanjing Huang. Making harmful behaviors unlearnable for large language models. arxiv: 2311.02105 [Cs, November 2023b. URL http:///
mrx.v.org/abs/231.0000
Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Yue Zhang, Ncil Zhenqiang Gong, and Xing Xic. BromptBench: Towards evaluating the robustness of Large Language Models on adversarial prompts. arxiv: 2306. 04528 CLJ, June 2023a. URL https:
98

---

Published in Transactions on Machine Learning Research (04/2025)
Sicheng Zhu, Ruiyi Zhang, Bang An, Gang Wu, Joe Barrow, Zichao Wang, Furong Huang, Ani Nenkova, and Tong Sun. AutoDAN: Interpretable Gradient-Based adversarial attacks on large language models. arXiv: 2310.15140 [cs. CRJ, October 2023b. URL http:///arriv.orrrabb///11......
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazcika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathanicl Li, Michael J Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J Zico Kolter, and Dan Hendrycks. Representation engineering: A Top-Down approach to Al transparency. arXiw: 2310.01405 LG,, October 2023a. doi: 10.48550/am arria. 2tt...331..0145...
Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv: 5307.1504, fcs. CLJ, July 2023b. doi: 10.48550/ Xiv. 2307.15043.
I x A A Interim measures for the management of gener-
ative artificial intelligence services, 2023. URL https: /ttpt///ss//..chhoaaeeeeaaaaaaa /ttp://whiaaattrraaaaaaaaaaaaaaa wtt..////eenngeeeeaiieeeee--ee
ai-interim
99


Reviews:
[{"review_id": "lamqGnKIG2", "reviewer": "Reviewer_oCbv", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: this paper serves to introduce and guide the reader through the research landscape of technical AI governance. As such, it offers tools for the conceptual organization of the field, and offers the reader a bevy of open problems and further resources for reading. I found the taxonomy and other organizational tools to be accurate and useful. The descriptions and open problems for sub-fields that I am most qualified to discuss reflect my understanding of the research landscape, and offer ample reference for a reader to follow-up on. For the technical reader, this paper can serve as a research agenda development aid, or a framing tool for one's research. For a policy audience, this paper offers a peek at the devilish details contained in governance discussions, and can serve as a guide for funding technical work in AI governance.\n\nWeaknesses: this paper is missing a discussion of how the sections and subsections were identified - was there a formal literature review? How were papers sampled? How were sections and sub-sections identified within the literature? Without this discussion I have to assume that the organization of the TAIG field within this paper was performed ad hoc - while this isn't damning, it certainly decreases the perceived authority and objectivity of the work. Further, I'm unsure how good a fit this work is for TMLR. As per reviewer guidance I have erred on the side of generosity, as I'm sure there are some members of TMLR's audience that will find value in the organization this paper brings to a wide host of research topics. However, the contributions of this work are organizational (as opposed to technical) - and my understanding is that technical contributions are the primary feature of TMLR.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This submission introduces \"Technical AI Governance\" (TAIG) as a broad area of research. The authors organize the targets of governance strategies (data, compute, models, and deployment) and the governance capacity (assessment, access, verification, security, operationalization, ecosystem monitoring) into a 2D matrix taxonomy. The paper details each capacity and identifies how each target interacts with that capacity, and offers a selection of open problems within each interaction.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "- Discussion of literature review methodology (semi-critical)\n- Include reference to https://arxiv.org/abs/2401.15897 in assessment paragraph 2 - it's really just an excellent paper and belongs amid the references (non-critical)", "broader_impact_concerns": "n/a", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "hGtGtfC3Fo", "reviewer": "Reviewer_LTCH", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Overall, the primary contributions of the paper provided in sections 2-8 are strong. The work presents a thorough overview of a wide range of problems related to the technical challenges in AI governance which will be useful as a resource for both researchers and decision-makers. The main weakness of this paper is the ambiguity and lack of context provided in the introduction to help set up the contribution provided in subsequent sections. However, these issues are quite fixable.\n\n## Strengths\n\n**Laying out the open problems in Technical AI Governance**. This comprises the bulk of the paper and its contribution, and is also very clearly the paper’s strength. The information is presented in a clear, objective, and informative manner. The content is well-substantiated, with extensive references to existing research. The paper strikes the right balance of discussing the open problems, identifying potential solutions or tools, and assessing current limitations.\n\n**Comprehensive**. The paper covers a broad range of subjects and related open problems in AI which will be a useful resource for decision makers and researchers going forward. The authors did a good job in covering the breadth of problems I expected or hoped to find, as well as some that I had not but found valuable.\n\n**Structure of the taxonomy**. The taxonomy is well laid out and each capacity/target is defined in a clear and concise manner. While the choices the authors made in creating the taxonomy could use greater justification (see below), the substance of the taxonomy and the framing around capacities and targets provides a useful lens for approaching the issues discussed in the subsequent sections.\n\n**Organization of the paper**. In acknowledging the length of their paper, the authors have given special consideration to the organization of each of the taxonomy sections (3-8), including outlining that structure up front in Section 2. The authors also provide a useful mapping between subject areas that a reader might be interested in and relevant subsections of the paper—a useful tool for navigating a paper of this length.\n\n**Acknowledging scope and limitations**. The authors are thoughtful in outlining their approach and acknowledging its potential drawbacks and limitations. I appreciated the discussion of how the proposed solutions may be speculative in some cases, may be in tension with one another in others, and in aggregate do not solve the challenges of AI governance. This strengthens the position of the paper and helps to make it clear what the authors are trying to accomplish and what they are not.\n\n**Providing useful tables and summaries**. Figure 1 provides a great overview of the proposed taxonomy up front in the paper. The reappearance of relevant portions of this table in sections 3-8 are helpful for orienting the reader. I also found the example research questions to be a good resource at the beginning of each taxonomy section.\n\n**Writing**. The paper is generally well written. The writing style is clear and concise. In many cases the authors anticipated the questions I found myself asking as I read, providing an illustrative example or further explanation in the subsequent sentence or paragraph.\n\n## Weaknesses\n\n**Defining Technical AI Governance**. The definition of TAIG used in both the abstract and introduction leaves the reader without a clear sense of what the paper will precisely cover. The definition of Technical AI Governance as “technical analysis and tools for supporting effective governance of AI” is both circular, as it uses the same terms (i.e. technical and governance) to define itself, and uninformative, providing little to no further detail than what was in the original term itself. As the paper progresses into Section 2, one can start to surmise what TAIG refers to, but a clear and more detailed definition is needed up front.\n\n**Differentiating Technical AI Governance**. The paper does not do a great job at differentiating TAIG from existing efforts in the AI governance, safety, and security space. In part, this is due to the issues in defining TAIG discussed above. However, it is further compounded in Section 1.1 which focuses more on how TAIG can be useful rather than what TAIG is in relation to AI governance. Paragraph 4 of Section 1.2 provides a brief comparison between TAIG and “sociotechnical approaches to AI safety and governance” which is good, but this should be expanded further.\n\n**Contextualizing the paper’s contribution in relation to previous work**. While Sections 3-8 are very well substantiated with references to other research, the contribution of the paper is not well oriented in relation to the broader literature. The paper lacks an initial discussion of previous works that have attempted to identify, collate, or assess AI governance problems (technical or otherwise) and how the contribution of this paper differs from those efforts or fills in existing gaps.\n\n**Distinguishing between the benefits of TAIG as a field and the contribution of this paper**. There is a substantial amount of ambiguity in the introduction of this paper between these two topics. For example, the motivation in the second paragraph of Section 1 is compelling (lack of information for decision-makers, unclear technical feasibility of governance options). However, the intro then makes a jump to why TAIG can be helpful in addressing these problems rather than why the contributions of this paper can be. I can see why this is the case implicitly, but the authors should make this logical step explicit.\n\n**Basis for the taxonomy**. As mentioned, the structure and content of the taxonomy is a strength of this paper. However, the basis for the taxonomy and the justification for its design is not strong. The only justification given for the selected *capacities* is that they “capture what we believe are the most important clusters of technical AI governance”. Further evidence or argument is needed to support why the authors think these are the most important clusters. Convince the reader as to why they should also agree with that statement. While the selection of the *targets* is more defensible, being based on a previous paper (Bommasani et al., 2923b), further details as to why any modifications were made, if any, would strengthen this section.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper introduces the concept of Technical AI Governance (TAIG) and describes how it can be used to support AI governance efforts by 1) identifying areas for intervention, 2) informing governance decisions, and 3) providing mechanisms for implementing governance levers. The author’s present a proposed two-dimensional taxonomy that provides a structure for TAIG as a field. The primary contribution is an in-depth discussion of the broad array of open problems in TAIG using the proposed taxonomy to frame the identified issues, potential solutions, and existing limitations. In collating this information, the authors provide a valuable resource for decision-makers and informative guide for future research directions.", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "## Adjustments critical for recommendation\n\n**A clear definition and explanation of Technical AI Governance**. Provide an informative definition for TAIG that is not circular in the introduction and abstract. In addition to a definition, provide a more-detailed explanation of what constitutes this field in the introduction. Providing this information up front would really be valuable for a reader before going into why TAIG can be helpful. In both Section 1 and 1.1, there is a logical jump straight from the definition of TAIG to the contributions of TAIG as a field. I recommend adding the additional context in both of these places (after paragraph 3 in Section 1 and after paragraph 1 in Section 1.1).\n\n**A “related works” type section or paragraph to contextualize contribution**. Provide the reader with an overview of previous papers that have taken similar approaches to this paper or have addressed aspects of problems in AI governance. Distinguish the contributions of this paper in comparison to those works. Without this, it is difficult to assess the contribution of the work. This does not have to be its own section, or even sub-section, it could simply be an additional paragraph in the introduction.\n\n**Differentiation between TAIG and other related efforts**. First, explain how TAIG and AI governance relate in greater detail beyond just definitions (as described above). Second, expand on how TAIG relates to or differs from other related efforts like the sociotechnical approaches discussed in Section 1.2. The most important question to answer here is why should we be thinking about TAIG separately from other efforts and what is the benefit in doing so? In doing so provide some justification for why certain seemingly sociotechnical problems (downstream societal impacts of AI systems, standards setting, environmental concerns, threat modelling) are included while others are not.\n\n**Greater justification for the design of the taxonomy**. I have no issues with the content or structure of the taxonomy, but the lack of justification for the design decisions is concerning as a reviewer. Include a basis for those decisions, including how the authors determined that the capacities were “the most important clusters” and details on how and why the targets were modified from the cited paper by Bommasani et al. If the latter were used as is, then just say that.\n\n**Clear lines between TAIG’s benefits and the paper’s contribution**. Eliminate the ambiguity between TAIG as a field and the contributions of the paper. Make it clear when you are advocating for the need for TAIG as an emerging field and describing its benefits versus the contribution of your paper in providing a taxonomy and assessing open problems related to TAIG.\n\n## Adjustments to strengthen the work\n\n**Greater background and context for TAIG as a field**. While the paper talks about TAIG as an emerging field, it provides the reader with no initial background on the field itself. Adding a section in the introduction that helps to explain what the field is, how it has evolved, and its current state would be useful context to help set up the discussion of open problems in the field.\n\n**Expanded discussion of the implications of the work at the end of the paper**. Currently the paper ends abruptly with a very short conclusion that restates the contribution. Providing a broader synthesis of the large amount of information (and work) the authors put into sections 3-8 would help to improve the paper. What does the state of open problems in TAIG mean for the direction of AI governance? Building on this work, what is next for researchers, policymakers, etc.? Some of the content of the policy memo in the appendix could be relevant to add here.\n\n**Reasoning for items out of scope**. Setting the scope in 1.2 was good, but paragraph 5 would benefit from a greater explanation of why the identified items were out of scope. Defining the scope is important, but providing the reasoning behind these decisions would strengthen the paper. For example, why was technical work that directly improves performance, safety, or robustness out of scope?", "broader_impact_concerns": "No broader impact concerns.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "3nh9K4zo0p", "reviewer": "Reviewer_CDZz", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "**Strengths:**\n\n1. The paper is well-written and very well-structured. It systematically presents an introduction to Technical AI governance, which provides analysis and tools for supporting effective AI governance. \n\n2. The audience of this paper could be very broad. The paper not only benefits researchers who do AI research,  but also may be helpful to refer for policymakers who try to establish regulation, and more broadly can serve to raise safety awareness from the general public. \n\n3. Page 47, the paper offers useful recommendations, which can serve as an important guideline of how AI safety research should be conducted. \n\n**Weakness:** \n\n1. **Writing issue**:\n-  Definition of AI safety risks is not explicit.   Before Section 3, it is unclear to me what AI safety risks this paper aims to address, and why they are important and require AI governance.    \n\n-  Section 3.1.1, how to identify problematic data without access to the training data, and why it is useful to identify them? Are we trying to inverse the harmful data?\n\n- Section 3.1.3, I firmly believe that understanding how the data shapes the model behavior by itself is an important research agenda. The authors list several prelimarial research on the impact of pre-training data/fine-tuning data/synthetic data. However, it is unclear to me what is the conclusion of these papers. How the three different data affect the model behavior respectively and how they differ? \nIn which stage, the impact of data on the model will be larger and therefore should pay more attention to? \n\n- Section 6.4.1, the definition of adversarial attack is unclear, and how it affect the model behaviors? \n\n- Section 6.4.1, for adversarial attack, it is too narrow to only talk about  \"detecting adversarial inputs and outputs\" as the only counter-measurement that the governance intervention can be enforced. The ultimate goal of is not to detect the adversarial input and output trying to circumvent the issue, but the governance effort should be made to enforce the deployed model itself to be robust to such an attack. Also, because of this narrow coverage of adversarial samples detection, many recent research/efforts on this topic are missing.  For example, [1] is not a detection-based circumvention. \n[1] Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks\n\n- Section 6.4.2,  the definition of harmful fine-tuning attack is unclear, and how it differs from the adversarial attack is not explicitly written.\n\n- Section 6.4.3,  the definition of the dual-use capability of the model and how it differs from the adversarial attack mentioned above is not clear.  \n\n2. **More related research**\n- Some recent research on jail-break attacks/adversarial attacks are missing. \n\n[1] Defending against alignment-breaking attacks via robustly aligned llm.\n\n[2] Smoothllm: Defending large language models against jailbreaking attacks.\n\n[3] Autodan: Automatic and interpretable adversarial attacks on large language models\n\n[4]  Mission Impossible: A Statistical Perspective on Jailbreaking LLMs\n\n[5] Detecting language model attacks with perplexity\n\n[6] Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapes\n\n[7] On the Safety of Open-Sourced Large Language Models: Does Alignment Really Prevent Them From Being Misused?\n\n- Some Recent research on harmful fine-tuning attack are missing. \n\n[8] On the Vulnerability of Safety Alignment in Open-Access LLMs\n\n[9] Buckle Up: Robustifying LLMs at Every Customization Stage via Data Curation\n\n[10] Tamper-Resistant Safeguards for Open-Weight LLMs\n\n[11] Booster: Tackling harmful fine-tuning for large language models via attenuating harmful perturbation\n\n[12] Targeted Vaccine: Safety Alignment for Large Language Models against Harmful Fine-Tuning via Layer-wise Perturbation\n\n[13] Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions\n\n[14]  Mitigating fine-tuning jailbreak attack with backdoor enhanced alignment\n\n[15]  Keeping llms aligned after fine-tuning: The crucial role of prompt templates \n\n[16] Lazy safety alignment for large language models against harmful fine-tuning\n\n[17] Safety alignment should be made more than just a few tokens deep\n\n[18] Safe lora: the silver lining of reducing safety risks when fine-tuning large language models\n\n[19] Antidote: Post-fine-tuning safety alignment for large language models against harmful fine-tuning\n\n[20] Locking Down the Finetuned LLMs Safety\n\n[21] On Evaluating the Durability of Safeguards for Open-Weight LLMs\n\n[22] Harmful Fine-tuning Attacks and Defenses for Large Language Models: A Survey\n\n[23] No two devils alike: Unveiling distinct mechanisms of fine-tuning", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "In order to chase revenue from the new market, large AI companies are investing insane amounts of funding in training large-scale foundation models, aiming to improve their intelligence and reasoning ability. However, the data they used for pre-training such foundation model are pretty wild-- every bit of data on the Internet is used and many of the data do not reflect the core values of humanity. Without proper governance intervention, this reckless act of pretraining the foundation model on unsanitized data may introduce unpredictable safety risks to the foundation models themselves, eventually leading to a risk for the future of humanity.  \n\nThis paper provides a systematical introduction of technical AI governance and give concrete direction in where and how the goevernace efforts should be enforced. The paper also provides a summarization/introduction to techniques/evaluation tools that may provide useful information for the governace decision making.  The paper serves as a important guideline for safely developing/researching AI in industry/academy. \n\n**The paper should be highlighted by TMLR to raise the attention/awareness of researchers, policymakers and the general public.**", "claims_and_evidence_yn": "Yes", "claims_explanation": "", "audience_yn": "Yes", "audience_explanation": "", "requested_changes": "1. Before Section 3, I suggest the author at least briefly discuss the AI safety risks with examples and also introduce the outcome of not properly controlling them with governance efforts.\n\n2. Section 3.1.3,  can more explicitly talk about the impact of data on model behavior. \n\n3. Section 6.4.1, examples/figures can be combined to illustrate the adversarial attack being discussed.  Also, the authors should expand the literature review but not highlight the detection-based method as the only counter-measurement that the governance intervention can be taken place. \n\n4. Section 6.4.2, the definition of fine-tuning attack is also ambiguous and can be illustrated with figures.  \n\n5. Include a discussion on related research papers on jail-break attacks and harmful fine-tuning attacks.  \n\n6. Section 6.4.3, gives more discussion on how the dual-use attack differs from the adversarial attack mentioned in Section 6.4.1.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The manuscript provides a broad survey of TAIG issues and would be an excellent on-ramp for researchers new to the field. Reviewer are positive about the work after authors made revisions to add procedural detail and better convey some points.
Decision: Accept (poster)