278. B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, R. Sayres, "Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)" in Proceedings of the 35th International Conference on Machine Learning (PMLR, 2018) pp. 2668-2677. https://proceedings.mlr.press/v80/kim18d.html.

279. Y. Belinkov, Probing Classifiers: Promises, Shortcomings, and Advances. Comput. Linguist. Assoc. Comput. Linguist. 48, 207-219 (2022). https://doi.org/10.1162/coli_a_00422.

280. S. Black, L. Sharkey, L. Grinsztajn, E. Winsor, D. Braun, J. Merizian, K. Parker, C. R. Guevara, B. Millidge, G. Alfour, C. Leahy, Interpreting Neural Networks through the Polytope Lens, arXiv:2211.12312 [cs.LG] (2022). http://arxiv.org/abs/2211.12312.

281. A. Zou, L. Phan, S. Chen, J. Campbell, P. Guo, R. Ren, A. Pan, X. Yin, M. Mazeika, A.-K. Dombrowski, S. Goel, N. Li, M. J. Byun, Z. Wang, A. Mallen, S. Basart, S. Koyejo, D. Song, M. Fredrikson, . . . D. Hendrycks, Representation Engineering: A Top-Down Approach to AI Transparency, arXiv:2310.01405 [cs.LG] (2023). https://doi.org/10.48550/arXiv.2310.01405.

282. S. Marks, C. Rager, E. J. Michaud, Y. Belinkov, D. Bau, A. Mueller, Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models, arXiv:2403.19647 [cs.LG] (2024). https://doi.org/10.48550/arXiv.2403.19647.

283. S. Carter, Z. Armstrong, L. Schubert, I. Johnson, C. Olah, Activation Atlas. Distill 4, 10.23915/distill.00015 (2019). https://doi.org/10.23915/distill.00015.

284. J. Mu, J. Andreas, "Compositional Explanations of Neurons" in Advances in Neural Information Processing Systems (NeurIPS 2020) (Curran Associates, Inc., 2020) vol. 33, pp. 17153-17163. https://proceedings.neurips.cc/paper/2020/hash/c74956ffb38ba48ed6ce977af6727275-Abstract.html.

285. E. Hernandez, S. Schwettmann, D. Bau, T. Bagashvili, A. Torralba, J. Andreas, "Natural Language Descriptions of Deep Visual Features" in The 10th International Conference on Learning Representations (2022). https://openreview.net/forum?id=NudBMY-tzDr.

286. S. Casper, M. Nadeau, D. Hadfield-Menell, G. Kreiman, "Robust Feature-Level Adversaries are Interpretability Tools" in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022). https://openreview.net/forum?id=IQ--doSB2o.

287. M. Geva, J. Bastings, K. Filippova, A. Globerson, "Dissecting Recall of Factual Associations in Auto-Regressive Language Models" in The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023) (2023) pp. 11116-12235. https://openreview.net/forum?id=F1G7y94K02.

288.* T. Bolukbasi, A. Pearce, A. Yuan, A. Coenen, E. Reif, F. Viégas, M. Wattenberg, An Interpretability Illusion for BERT, arXiv:2104.07143 [cs.CL] (2021). https://doi.org/10.48550/arXiv.2104.07143.

289. A. Makelov, G. Lange, A. Geiger, N. Nanda, "Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching" in The 12th International Conference on Learning Representations (ICLR 2024) (2023). https://openreview.net/forum?id=Ebt7JgMHv1.

290. M. Ananny, K. Crawford, Seeing without knowing: Limitations of the transparency ideal and its application to algorithmic accountability. New Media Soc. 20, 973-989 (2018). https://doi.org/10.1177/1461444816676645.

291. T. Miller, Explanation in artificial intelligence: Insights from the social sciences. Artif. Intell. 267, 1-38 (2019). https://doi.org/10.1016/j.artint.2018.07.007.

292. F. Doshi-Velez, B. Kim, Towards A Rigorous Science of Interpretable Machine Learning, arXiv:1702.08608 [stat.ML] (2017). https://doi.org/10.48550/arXiv.1702.08608.

293. Z. C. Lipton, The Mythos of Model Interpretability: In machine learning, the concept of interpretability is both important and slippery. ACM Queue 16, 31-57 (2018). https://doi.org/10.1145/3236386.3241340.

294.* M. L. Leavitt, A. Morcos, Towards falsifiable interpretability research, arXiv:2010.12016 [cs.CY] (2020). http://arxiv.org/abs/2010.12016.

295. T. Räuker, A. Ho, S. Casper, D. Hadfield-Menell, Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks, arXiv:2207.13243 [cs.LG] (2022). http://arxiv.org/abs/2207.13243.

296. M. Krishnan, Against Interpretability: a Critical Examination of the Interpretability Problem in Machine Learning. Philos. Technol. 33, 487-502 (2020). https://doi.org/10.1007/s13347-019-00372-9.

297. J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, B. Kim, "Sanity Checks for Saliency Maps" in Advances in Neural Information Processing Systems (NeurIPS 2018) (Curran Associates, Inc., 2018) vol. 31. https://proceedings.neurips.cc/paper_files/paper/2018/hash/294a8ed24b1ad22ec2e7efea049b8737-Abstract.html.

298. J. Adebayo, M. Muelly, I. Liccardi, B. Kim, "Debugging Tests for Model Explanations" in Advances in Neural Information Processing Systems (NeurIPS 2020) (Curran Associates, Inc., 2020) vol. 33, pp. 700-712. https://proceedings.neurips.cc/paper/2020/hash/075b051ec3d22dac7b33f788da631fd4-Abstract.html.

299. P. Hase, M. Bansal, B. Kim, A. Ghandeharioun, "Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models" in 37th Conference on Neural Information Processing Systems (NeurIPS 2023) (2023). https://openreview.net/forum?id=EldbUIZtbd.
300. S. Casper, C. Ezell, C. Siegmann, N. Kolt, T. L. Curtis, B. Bucknall, A. Haupt, K. Wei, J. Scheurer, M. Hobbhahn, L. Sharkey, S. Krishna, M. Von Hagen, S. Alberti, A. Chan, Q. Sun, M. Gerovitch, D. Bau, M. Tegmark, . . . D. Hadfield-Menell, Black-Box Access is Insufficient for Rigorous Al Audits, arXiv:2401.14446 [cs.CY] (2024). http://arxiv.org/abs/2401.14446.

301. B. S. Bucknall, R. F. Trager, "Structured access for third-party research on frontier Al models: Investigating researchers' model access requirements" (Oxford Martin School, University of Oxford and Center for the Governance of Al, 2023); https://www.ceris.be/wp-content/uploads/2024/03/Structured-Access-for-Third-Party-Research-Robert-Trager.pdf.

302. R. Ashmore, R. Calinescu, C. Paterson, Assuring the machine learning lifecycle. *ACM Comput. Surv.* **54**, 1–39 (2022). https://doi.org/10.1145/3453444.

303. S. Casper, X. Davies, C. Shi, T. K. Gilbert, J. Scheurer, J. Rando, R. Freedman, T. Korbak, D. Lindner, P. Freire, T. T. Wang, S. Marks, C.-R. Segerie, M. Carroll, A. Peng, P. Christoffersen, M. Damani, S. Slocum, U. Anwar, . . . D. Hadfield-Menell, Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. *Transactions on Machine Learning Research* (2023). https://openreview.net/forum?id=bx24KpJ4Eb.

304. S. Longpre, S. Kapoor, K. Klyman, A. Ramaswami, R. Bommasani, B. Bili Hamelin, Y. Huang, A. Skowron, Z.-X. Yong, S. Kotha, Y. Zeng, W. Shi, X. Yang, R. Southen, A. Robey, P. Chao, D. Yang, R. Jia, D. Kang, . . . P. Henderson, A Safe Harbor for Al Evaluation and Red Teaming, arXiv:2403.04893 [cs.Al] (2024). https://doi.org/10.48550/arXiv.2403.04893.

305. T. Shevlane, Structured access: an emerging paradigm for safe Al deployment, arXiv:2201.05159 [cs.Al] (2022). https://doi.org/10.48550/arXiv.2201.05159.

306. M. Dolata, S. Feuerriegel, G. Schwabe, A sociotechnical view of algorithmic fairness. *Inf. Syst.* **J. 32**, 754–818 (2022). https://doi.org/10.1111/isj.12370.

307. S. Lazar, A. Nelson, Al safety on whose terms? *Science* **381**, 138 (2023). https://doi.org/10.1126/science.adi8982.

308. Y. Wang, Y. Zhu, C. Kong, S. Wei, X. Yi, X. Xie, J. Sang, CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models, arXiv:2311.16421 [cs.CL] (2023). http://arxiv.org/abs/2311.16421.

309. Z. X. Yong, C. Menghini, S. Bach, "Low-Resource Languages Jailbreak GPT-4" in *NeurIPS Workshop on Socially Responsible Language Modelling Research (SoLaR)* (2023). https://openreview.net/forum?id=pn83r8V2sv.

310. Y. Jin, M. Chandra, G. Verma, Y. Hu, M. De Choudhury, S. Kumar, Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries, arXiv:2310.13132 [cs.CL] (2023). http://arxiv.org/abs/2310.13132.

311.* A. Üstün, V. Aryabumi, Z.-X. Yong, W.-Y. Ko, D. D'souza, G. Onilude, N. Bhandari, S. Singh, H.-L. Ooi, A. Kayid, F. Vargus, P. Blunsom, S. Longpre, N. Muennighoff, M. Fadaee, J. Kreutzer, S. Hooker, Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model, arXiv:2402.07827 [cs.CL] (2024). http://arxiv.org/abs/2402.07827.

312. Y. Xu, H. Cao, W. Du, W. Wang, A Survey of Cross-lingual Sentiment Analysis: Methodologies, Models and Evaluations. *Data Science and Engineering* **7**, 279–299 (2022). https://doi.org/10.1007/s41019-022-00187-3.

313.* W. Wang, Z. Tu, C. Chen, Y. Yuan, J.-T. Huang, W. Jiao, M. R. Lyu, All Languages Matter: On the Multilingual Safety of Large Language Models, arXiv:2310.00905 [cs.CL] (2023). http://arxiv.org/abs/2310.00905.

314. A. D. Selbst, An Institutional View Of Algorithmic Impact Assessments. **35** (2021). https://jolt.law.harvard.edu/assets/articlePDFs/v35/Selbst-An-Institutional-View-of-Algorithmic-Impact-Assessments.pdf.

315. S. L. Blodgett, S. Barocas, H. Daumé, III, H. Wallach, "Language (Technology) is Power: A Critical Survey of "Bias" in NLP" in *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020)* (Association for Computational Linguistics, 2020) pp. 5454–5476. https://doi.org/10.18653/v1/2020.acl-main.485.

316. A. Hagerty, I. Rubinov, Global Al Ethics: A Review of the Social Impacts and Ethical Implications of Artificial Intelligence, arXiv:1907.07892 [cs.CY] (2019). https://doi.org/10.48550/arXiv.1907.07892.

317. M. M. Maas, "Aligning Al Regulation to Sociotechnical Change" in *The Oxford Handbook of Al Governance*, J. B. Bullock, Y.-C. Chen, J. Himmelreich, V. M. Hudson, A. Korinek, M. M. Young, B. Zhang, Eds. (Oxford University Press, 2022).

318.* L. Weidinger, J. Barnhart, J. Brennan, C. Butterfield, S. Young, W. Hawkins, L. A. Hendricks, R. Comanescu, O. Chang, M. Rodriguez, J. Beroshi, D. Bloxwich, L. Proleev, J. Chen, S. Farquhar, L. Ho, I. Gabriel, A. Dafoe, W. Isaac, "Holistic Safety and Responsibility Evaluations of Advanced Al Models" (Google Deepmind, 2024); http://arxiv.org/abs/2404.14068.

319. M. Raghavan, *The Societal Impacts of Algorithmic Decision-Making* (Association for Computing Machinery, New York, NY, USA, ed. 1, 2023), vol. 53, pp. 366.

320. S. Kapoor, R. Bommasani, K. Klyman, S. Longpre, A. Ramaswami, P. Cihon, A. Hopkins, K. Bankston, S. Biderman, M. Bogen, R. Chowdhury, A. Engler, P. Henderson, Y. Jernite, S. Lazar, S. Maffulli, A. Nelson, J. Pineau, A. Skowron, . . . A. Narayanan, "On the Societal Impact of Open Foundation Models" (Stanford Institute for Human-Centered Artificial Intelligence, 2024); https://crfm.stanford.edu/open-fms/paper.pdf.

321. E. Moss, E. A. Watkins, R. Singh, M. C. Elish, J. Metcalf, "Assembling Accountability: Algorithmic Impact Assessment for the Public Interest" (Data & Society, 2021); https://datasociety.net/library/assembling-accountability-algorithmic-impact-assessment-for-the-public-interest/.
300. S. Casper, C. Ezell, C. Siegmann, N. Kolt, T. L. Curtis, B. Bucknall, A. Haupt, K. Wei, J. Scheurer, M. Hobbhahn, L. Sharkey, S. Krishna, M. Von Hagen, S. Alberti, A. Chan, Q. Sun, M. Gerovitch, D. Bau, M. Tegmark, . . . D. Hadfield-Menell, Black-Box Access is Insufficient for Rigorous Al Audits, arXiv:2401.14446 [cs.CY] (2024). http://arxiv.org/abs/2401.14446.

301. B. S. Bucknall, R. F. Trager, "Structured access for third-party research on frontier Al models: Investigating researchers' model access requirements" (Oxford Martin School, University of Oxford and Center for the Governance of Al, 2023); https://www.ceris.be/wp-content/uploads/2024/03/Structured-Access-for-Third-Party-Research-Robert-Trager.pdf.

302. R. Ashmore, R. Calinescu, C. Paterson, Assuring the machine learning lifecycle. *ACM Comput. Surv.* **54**, 1–39 (2022). https://doi.org/10.1145/3453444.

303. S. Casper, X. Davies, C. Shi, T. K. Gilbert, J. Scheurer, J. Rando, R. Freedman, T. Korbak, D. Lindner, P. Freire, T. T. Wang, S. Marks, C.-R. Segerie, M. Carroll, A. Peng, P. Christoffersen, M. Damani, S. Slocum, U. Anwar, . . . D. Hadfield-Menell, Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback. *Transactions on Machine Learning Research* (2023). https://openreview.net/forum?id=bx24KpJ4Eb.

304. S. Longpre, S. Kapoor, K. Klyman, A. Ramaswami, R. Bommasani, B. Bili Hamelin, Y. Huang, A. Skowron, Z.-X. Yong, S. Kotha, Y. Zeng, W. Shi, X. Yang, R. Southen, A. Robey, P. Chao, D. Yang, R. Jia, D. Kang, . . . P. Henderson, A Safe Harbor for Al Evaluation and Red Teaming, arXiv:2403.04893 [cs.Al] (2024). https://doi.org/10.48550/arXiv.2403.04893.

305. T. Shevlane, Structured access: an emerging paradigm for safe Al deployment, arXiv:2201.05159 [cs.Al] (2022). https://doi.org/10.48550/arXiv.2201.05159.

306. M. Dolata, S. Feuerriegel, G. Schwabe, A sociotechnical view of algorithmic fairness. *Inf. Syst.* **J. 32**, 754–818 (2022). https://doi.org/10.1111/isj.12370.

307. S. Lazar, A. Nelson, Al safety on whose terms? *Science* **381**, 138 (2023). https://doi.org/10.1126/science.adi8982.

308. Y. Wang, Y. Zhu, C. Kong, S. Wei, X. Yi, X. Xie, J. Sang, CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models, arXiv:2311.16421 [cs.CL] (2023). http://arxiv.org/abs/2311.16421.

309. Z. X. Yong, C. Menghini, S. Bach, "Low-Resource Languages Jailbreak GPT-4" in *NeurIPS Workshop on Socially Responsible Language Modelling Research (SoLaR)* (2023). https://openreview.net/forum?id=pn83r8V2sv.

310. Y. Jin, M. Chandra, G. Verma, Y. Hu, M. De Choudhury, S. Kumar, Better to Ask in English: Cross-Lingual Evaluation of Large Language Models for Healthcare Queries, arXiv:2310.13132 [cs.CL] (2023). http://arxiv.org/abs/2310.13132.

311.* A. Üstün, V. Aryabumi, Z.-X. Yong, W.-Y. Ko, D. D'souza, G. Onilude, N. Bhandari, S. Singh, H.-L. Ooi, A. Kayid, F. Vargus, P. Blunsom, S. Longpre, N. Muennighoff, M. Fadaee, J. Kreutzer, S. Hooker, Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model, arXiv:2402.07827 [cs.CL] (2024). http://arxiv.org/abs/2402.07827.

312. Y. Xu, H. Cao, W. Du, W. Wang, A Survey of Cross-lingual Sentiment Analysis: Methodologies, Models and Evaluations. *Data Science and Engineering* **7**, 279–299 (2022). https://doi.org/10.1007/s41019-022-00187-3.

313.* W. Wang, Z. Tu, C. Chen, Y. Yuan, J.-T. Huang, W. Jiao, M. R. Lyu, All Languages Matter: On the Multilingual Safety of Large Language Models, arXiv:2310.00905 [cs.CL] (2023). http://arxiv.org/abs/2310.00905.

314. A. D. Selbst, An Institutional View Of Algorithmic Impact Assessments. **35** (2021). https://jolt.law.harvard.edu/assets/articlePDFs/v35/Selbst-An-Institutional-View-of-Algorithmic-Impact-Assessments.pdf.

315. S. L. Blodgett, S. Barocas, H. Daumé, III, H. Wallach, "Language (Technology) is Power: A Critical Survey of "Bias" in NLP" in *Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020)* (Association for Computational Linguistics, 2020) pp. 5454–5476. https://doi.org/10.18653/v1/2020.acl-main.485.

316. A. Hagerty, I. Rubinov, Global Al Ethics: A Review of the Social Impacts and Ethical Implications of Artificial Intelligence, arXiv:1907.07892 [cs.CY] (2019). https://doi.org/10.48550/arXiv.1907.07892.

317. M. M. Maas, "Aligning Al Regulation to Sociotechnical Change" in *The Oxford Handbook of Al Governance*, J. B. Bullock, Y.-C. Chen, J. Himmelreich, V. M. Hudson, A. Korinek, M. M. Young, B. Zhang, Eds. (Oxford University Press, 2022).

318.* L. Weidinger, J. Barnhart, J. Brennan, C. Butterfield, S. Young, W. Hawkins, L. A. Hendricks, R. Comanescu, O. Chang, M. Rodriguez, J. Beroshi, D. Bloxwich, L. Proleev, J. Chen, S. Farquhar, L. Ho, I. Gabriel, A. Dafoe, W. Isaac, "Holistic Safety and Responsibility Evaluations of Advanced Al Models" (Google Deepmind, 2024); http://arxiv.org/abs/2404.14068.

319. M. Raghavan, *The Societal Impacts of Algorithmic Decision-Making* (Association for Computing Machinery, New York, NY, USA, ed. 1, 2023), vol. 53, pp. 366.

320. S. Kapoor, R. Bommasani, K. Klyman, S. Longpre, A. Ramaswami, P. Cihon, A. Hopkins, K. Bankston, S. Biderman, M. Bogen, R. Chowdhury, A. Engler, P. Henderson, Y. Jernite, S. Lazar, S. Maffulli, A. Nelson, J. Pineau, A. Skowron, . . . A. Narayanan, "On the Societal Impact of Open Foundation Models" (Stanford Institute for Human-Centered Artificial Intelligence, 2024); https://crfm.stanford.edu/open-fms/paper.pdf.

321. E. Moss, E. A. Watkins, R. Singh, M. C. Elish, J. Metcalf, "Assembling Accountability: Algorithmic Impact Assessment for the Public Interest" (Data & Society, 2021); https://datasociety.net/library/assembling-accountability-algorithmic-impact-assessment-for-the-public-interest/.
322.* I. Solaiman, Z. Talat, W. Agnew, L. Ahmad, D. Baker, S. L. Blodgett, H. Daumé, III, J. Dodge, E. Evans, S. Hooker, Y. Jernite, A. S. Luccioni, A. Lusoli, M. Mitchell, J. Newman, M.-T. Png, A. Strait, A. Vassilev, Evaluating the Social Impact of Generative AI Systems in Systems and Society, arXiv:2306.05949 [cs.CY] (2023). http://arxiv.org/abs/2306.05949.

323.* L. Weidinger, M. Rauh, N. Marchal, A. Manzini, L. A. Hendricks, J. Mateos-Garcia, S. Bergman, J. Kay, C. Griffin, B. Bariach, I. Gabriel, V. Rieser, W. Isaac, "Sociotechnical Safety Evaluation of Generative AI Systems" (Google Deepmind, 2023); http://arxiv.org/abs/2310.11986.

324. A. D. Selbst, D. Boyd, S. A. Friedler, S. Venkatasubramanian, J. Vertesi, "Fairness and Abstraction in Sociotechnical Systems" in Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19) (Association for Computing Machinery, 2019) pp. 59-68. https://doi.org/10.1145/3287560.3287598.

325. E. Moss, J. Metcalf, "Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies" (Data & Society, 2020); https://datasociety.net/wp-content/uploads/2020/09/Ethics-Owners_20200923-DataSociety.pdf.

326. M. Feffer, M. Skirpan, Z. Lipton, H. Heidari, "From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research" in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23) (ACM, 2023) pp. 38-48. https://doi.org/10.1145/3600211.3604661.

327. F. Delgado, S. Yang, M. Madaio, Q. Yang, "The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice" in Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '23) (Association for Computing Machinery, 2023) pp. 1-23. https://doi.org/10.1145/3617694.3623261.

328. A. Birhane, W. Isaac, V. Prabhakaran, M. Diaz, M. C. Elish, I. Gabriel, S. Mohamed, "Power to the People? Opportunities and Challenges for Participatory AI" in Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '22) (Association for Computing Machinery, 2022) pp. 1-8. https://doi.org/10.1145/3551624.3555290.

329. J. Metcalf, E. Moss, E. A. Watkins, R. Singh, M. C. Elish, "Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts" in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21) (Association for Computing Machinery, 2021) pp. 735-746. https://doi.org/10.1145/3442188.3445935.

330. D. Martin, Jr, V. Prabhakaran, J. Kuhlberg, A. Smart, W. S. Isaac, "Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics" in ICLR Workshop on Machine Learning in Real Life (2020). https://doi.org/10.48550/arXiv.2005.07572.

331. M. Sloane, E. Moss, O. Awomolo, L. Forlano, "Participation Is not a Design Fix for Machine Learning" in Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '22) (Association for Computing Machinery, 2022) pp. 1-6. https://doi.org/10.1145/3551624.3555285.

332. R. I. J. Dobbe, T. K. Gilbert, Y. Mintz, "Hard Choices in Artificial Intelligence: Addressing Normative Uncertainty through Sociotechnical Commitments (AIES '20)" in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (Association for Computing Machinery, 2020) pp. 242. https://doi.org/10.1145/3375627.3375861.

333. S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, The (Im)possibility of fairness: different value systems require different mechanisms for fair decision making. Commun. ACM 64, 136-143 (2021). https://doi.org/10.1145/3433949.

334. N. Guha, C. M. Lawrence, L. A. Gailmard, K. T. Rodolfa, F. Surani, R. Bommasani, I. D. Raji, M.-F. Cuéllar, C. Honigsberg, P. Liang, D. E. Ho, AI Regulation Has Its Own Alignment Problem: The Technical and Institutional Feasibility of Disclosure, Registration, Licensing, and Auditing. The George Washington Law Review 92 (2024). https://dho.stanford.edu/wp-content/uploads/Al_Regulation.pdf.

335. U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y. Jia, J. Ghosh, R. Puri, J. M. F. Moura, P. Eckersley, "Explainable machine learning in deployment" in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20) (Association for Computing Machinery, 2020) pp. 648-657. https://doi.org/10.1145/3351095.3375624.

336. K. Kaye, P. Dixon, "Risky Analysis: Assessing and Improving AI Governance Tools An international review of AI Governance Tools and suggestions for pathways forward" (World Privacy Forum, 2023); https://www.worldprivacyforum.org/wp-content/uploads/2023/12/WPF_Risky_Analysis_December_2023_fs.pdf.

337. D. Slack, S. Hilgard, E. Jia, S. Singh, H. Lakkaraju, "Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods" in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES '20) (Association for Computing Machinery, 2020) pp. 180-186. https://doi.org/10.1145/3375627.3375830.

338. I. D. Raji, J. Yang, "ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles" in NeurIPS 2019 Human-Centric Machine Learning Workshop (2019). https://doi.org/10.48550/arXiv.1912.06166.

339. Project Management Institute (PMI), "A guide to the project management body of knowledge (PMBOK Guide)" (PMI, 2000).

340. P. V. Falade, Decoding the Threat Landscape: ChatGPT, FraudGPT, and WormGPT in Social Engineering Attacks. int. j. sci. res. comput. sci. eng. inf. technol. 9, 185-198 (2023). https://doi.org/10.32628/cseit2390533.
322.* I. Solaiman, Z. Talat, W. Agnew, L. Ahmad, D. Baker, S. L. Blodgett, H. Daumé, III, J. Dodge, E. Evans, S. Hooker, Y. Jernite, A. S. Luccioni, A. Lusoli, M. Mitchell, J. Newman, M.-T. Png, A. Strait, A. Vassilev, Evaluating the Social Impact of Generative AI Systems in Systems and Society, arXiv:2306.05949 [cs.CY] (2023). http://arxiv.org/abs/2306.05949.

323.* L. Weidinger, M. Rauh, N. Marchal, A. Manzini, L. A. Hendricks, J. Mateos-Garcia, S. Bergman, J. Kay, C. Griffin, B. Bariach, I. Gabriel, V. Rieser, W. Isaac, "Sociotechnical Safety Evaluation of Generative AI Systems" (Google Deepmind, 2023); http://arxiv.org/abs/2310.11986.

324. A. D. Selbst, D. Boyd, S. A. Friedler, S. Venkatasubramanian, J. Vertesi, "Fairness and Abstraction in Sociotechnical Systems" in Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19) (Association for Computing Machinery, 2019) pp. 59–68. https://doi.org/10.1145/3287560.3287598.

325. E. Moss, J. Metcalf, "Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies" (Data & Society, 2020); https://datasociety.net/wp-content/uploads/2020/09/Ethics-Owners_20200923-DataSociety.pdf.

326. M. Feffer, M. Skirpan, Z. Lipton, H. Heidari, "From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research" in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23) (ACM, 2023) pp. 38–48. https://doi.org/10.1145/3600211.3604661.

327. F. Delgado, S. Yang, M. Madaio, Q. Yang, "The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice" in Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '23) (Association for Computing Machinery, 2023) pp. 1–23. https://doi.org/10.1145/3617694.3623261.

328. A. Birhane, W. Isaac, V. Prabhakaran, M. Diaz, M. C. Elish, I. Gabriel, S. Mohamed, "Power to the People? Opportunities and Challenges for Participatory AI" in Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '22) (Association for Computing Machinery, 2022) pp. 1–8. https://doi.org/10.1145/3551624.3555290.

329. J. Metcalf, E. Moss, E. A. Watkins, R. Singh, M. C. Elish, "Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts" in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21) (Association for Computing Machinery, 2021) pp. 735–746. https://doi.org/10.1145/3442188.3445935.

330. D. Martin, Jr, V. Prabhakaran, J. Kuhlberg, A. Smart, W. S. Isaac, "Participatory Problem Formulation for Fairer Machine Learning Through Community Based System Dynamics" in ICLR Workshop on Machine Learning in Real Life (2020). https://doi.org/10.48550/arXiv.2005.07572.

331. M. Sloane, E. Moss, O. Awomolo, L. Forlano, "Participation Is not a Design Fix for Machine Learning" in Proceedings of the 2nd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '22) (Association for Computing Machinery, 2022) pp. 1–6. https://doi.org/10.1145/3551624.3555285.

332. R. I. J. Dobbe, T. K. Gilbert, Y. Mintz, "Hard Choices in Artificial Intelligence: Addressing Normative Uncertainty through Sociotechnical Commitments (AIES '20)" in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (Association for Computing Machinery, 2020) pp. 242. https://doi.org/10.1145/3375627.3375861.

333. S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, The (Im)possibility of fairness: different value systems require different mechanisms for fair decision making. Commun. ACM 64, 136–143 (2021). https://doi.org/10.1145/3433949.

334. N. Guha, C. M. Lawrence, L. A. Gailmard, K. T. Rodolfa, F. Surani, R. Bommasani, I. D. Raji, M.-F. Cuéllar, C. Honigsberg, P. Liang, D. E. Ho, AI Regulation Has Its Own Alignment Problem: The Technical and Institutional Feasibility of Disclosure, Registration, Licensing, and Auditing. The George Washington Law Review 92 (2024). https://dho.stanford.edu/wp-content/uploads/Al_Regulation.pdf.

335. U. Bhatt, A. Xiang, S. Sharma, A. Weller, A. Taly, Y. Jia, J. Ghosh, R. Puri, J. M. F. Moura, P. Eckersley, "Explainable machine learning in deployment" in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20) (Association for Computing Machinery, 2020) pp. 648–657. https://doi.org/10.1145/3351095.3375624.

336. K. Kaye, P. Dixon, "Risky Analysis: Assessing and Improving AI Governance Tools An international review of AI Governance Tools and suggestions for pathways forward" (World Privacy Forum, 2023); https://www.worldprivacyforum.org/wp-content/uploads/2023/12/WPF_Risky_Analysis_December_2023_fs.pdf.

337. D. Slack, S. Hilgard, E. Jia, S. Singh, H. Lakkaraju, "Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methods" in Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society (AIES '20) (Association for Computing Machinery, 2020) pp. 180–186. https://doi.org/10.1145/3375627.3375830.

338. I. D. Raji, J. Yang, "ABOUT ML: Annotation and Benchmarking on Understanding and Transparency of Machine Learning Lifecycles" in NeurIPS 2019 Human-Centric Machine Learning Workshop (2019). https://doi.org/10.48550/arXiv.1912.06166.

339. Project Management Institute (PMI), "A guide to the project management body of knowledge (PMBOK Guide)" (PMI, 2000).

340. P. V. Falade, Decoding the Threat Landscape: ChatGPT, FraudGPT, and WormGPT in Social Engineering Attacks. int. j. sci. res. comput. sci. eng. inf. technol. 9, 185–198 (2023). https://doi.org/10.32628/cseit2390533.
341. Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, Y. Zhang, A Survey on Large Language Model (LLM) Security and Privacy: The Good, The Bad, and The Ugly. *High-Confidence Computing* **4**, 100211 (2024). https://doi.org/10.1016/j.hcc.2024.100211.

342. N. Begou, J. Vinoy, A. Duda, M. Korczyński, "Exploring the dark side of Al: Advanced phishing attack design and deployment using ChatGPT" in *2023 IEEE Conference on Communications and Network Security (CNS) (IEEE, 2023)*. https://doi.org/10.1109/cns59707.2023.10288940.

343.* M. Heinemeyer, Cyber Security Threats - Email Compromise With Generative Al. (2023). https://es.darktrace.com/blog/tackling-the-soft-underbelly-of-cyber-security-email-compromise.

344. O. Bendel, The synthetization of human voices. *Al Soc.* **34**, 83-89 (2019). https://doi.org/10.1007/s00146-017-0748-x.

345. R. Gil, J. Virgili-Gomà, J.-M. López-Gil, R. García, Deepfakes: evolution and trends. *Soft Computing* **27**, 11295-11318 (2023). https://doi.org/10.1007/s00500-023-08605-y.

346. Á. F. Gambín, A. Yazidi, A. Vasilakos, H. Haugerud, Y. Djenouri, Deepfakes: current and future trends. *Artificial Intelligence Review* **57**, 64 (2024). https://doi.org/10.1007/s10462-023-10679-x.

347. V. Ciancaglini, C. Gibson, D. Sancho, O. McCarthy, M. Eira, P. Amann, A. Klayn, "Malicious uses and abuses of artificial intelligence" (European Union Agency for Law Enforcement Cooperation, 2020).

348. R. Umbach, N. Henry, G. Beard, C. Berryessa, Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries, arXiv:2402.01721 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2402.01721.

349. M. B. Kugler, C. Pace, Deepfake Privacy: Attitudes and Regulation. *Northwest. Univ. Law Rev.* **116**, 611-680 (2021). https://scholarlycommons.law.northwestern.edu/nulr/vol116/iss3/1.

350. M. Viola, C. Voto, Designed to abuse? Deepfakes and the non-consensual diffusion of intimate images. *Synthese* **201**, 30 (2023). https://doi.org/10.1007/s11229-022-04012-z.

351. D. M. J. Lazer, M. A. Baum, Y. Benkler, A. J. Berinsky, K. M. Greenhill, F. Menczer, M. J. Metzger, B. Nyhan, G. Pennycook, D. Rothschild, M. Schudson, S. A. Sloman, C. R. Sunstein, E. A. Thorson, D. J. Watts, J. L. Zittrain, The science of fake news. *Science* **359**, 1094-1096 (2018). https://doi.org/10.1126/science.aao29998.

352. G. Spitale, N. Biller-Andorno, F. Germani, Al model GPT-3 (dis)informs us better than humans. *Sci Adv* **9**, eadh1850 (2023). https://doi.org/10.1126/sciadv.adh1850.

353. M. Jakesch, J. T. Hancock, M. Naaman, Human heuristics for Al-generated language are flawed. *Proc. Natl. Acad. Sci. U. S. A.* **120**, e2208839120 (2023). https://doi.org/10.1073/pnas.2208839120.

354. K.-C. Yang, F. Menczer, Anatomy of an Al-powered malicious social botnet, arXiv:2307.16336 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2307.16336.

355. M. Masood, M. Nawaz, K. M. Malik, A. Javed, A. Irtaza, H. Malik, Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward. *Applied Intelligence* **53**, 3974-4026 (2023). https://doi.org/10.1007/s10489-022-03766-z.

356. D. Cooke, A. Edwards, S. Barkoff, K. Kelly, As Good As A Coin Toss: Human detection of Al-generated images, videos, audio, and audiovisual stimuli, arXiv:2403.16760 [cs.HC] (2024). http://arxiv.org/abs/2403.16760.

357. S. J. Nightingale, H. Farid, Al-synthesized faces are indistinguishable from real faces and more trustworthy. *Proc. Natl. Acad. Sci. U. S. A.* **119** (2022). https://doi.org/10.1073/pnas.2120481119.

358. S. C. Matz, J. D. Teeny, S. S. Vaid, H. Peters, G. M. Harari, M. Cerf, The potential of generative Al for personalized persuasion at scale. *Sci. Rep.* **14**, 4692 (2024). https://doi.org/10.1038/s41598-024-53755-0.

359. H. Bai, J. G. Voelkel, J. C. Eichstaedt, R. Willer, Artificial Intelligence Can Persuade Humans on Political Issues. (2023). https://doi.org/10.31219/osf.io/stakv.

360. K. Hackenburg, L. Ibrahim, B. M. Tappin, M. Tsakiris, Comparing the persuasiveness of role-playing large language models and human experts on polarized U.S. political issues. (2023). https://doi.org/10.31219/osf.io/ey8db.

361. K. Hackenburg, H. Margetts, Evaluating the persuasive influence of political microtargeting with large language models. (2023). https://doi.org/10.31219/osf.io/wnt8b.

362. A. Simchon, M. Edwards, S. Lewandowsky, The persuasive effects of political microtargeting in the age of generative artificial intelligence. *PNAS Nexus* **3**, gae035 (2024). https://doi.org/10.1093/pnasexus/pgae035.

363. B. M. Tappin, C. Wittenberg, L. B. Hewitt, A. J. Berinsky, D. G. Rand, Quantifying the potential persuasive returns to political microtargeting. *Proc. Natl. Acad. Sci. U. S. A.* **120**, e2216261120 (2023). https://doi.org/10.1073/pnas.2216261120.

364. F. Salvi, M. H. Ribeiro, R. Gallotti, R. West, On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial, arXiv:2403.14380 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2403.14380.

365. T. H. Costello, G. Pennycook, D. G. Rand, Durably reducing conspiracy beliefs through dialogues with Al. (2024). https://doi.org/10.31234/osf.io/xcwdn.
341. Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, Y. Zhang, A Survey on Large Language Model (LLM) Security and Privacy: The Good, The Bad, and The Ugly. *High-Confidence Computing* **4**, 100211 (2024). https://doi.org/10.1016/j.hcc.2024.100211.

342. N. Begou, J. Vinoy, A. Duda, M. Korczyński, "Exploring the dark side of Al: Advanced phishing attack design and deployment using ChatGPT" in *2023 IEEE Conference on Communications and Network Security (CNS) (IEEE, 2023)*. https://doi.org/10.1109/cns59707.2023.10288940.

343.* M. Heinemeyer, Cyber Security Threats - Email Compromise With Generative Al. (2023). https://es.darktrace.com/blog/tackling-the-soft-underbelly-of-cyber-security-email-compromise.

344. O. Bendel, The synthetization of human voices. *Al Soc.* **34**, 83-89 (2019). https://doi.org/10.1007/s00146-017-0748-x.

345. R. Gil, J. Virgili-Gomà, J.-M. López-Gil, R. García, Deepfakes: evolution and trends. *Soft Computing* **27**, 11295-11318 (2023). https://doi.org/10.1007/s00500-023-08605-y.

346. Á. F. Gambín, A. Yazidi, A. Vasilakos, H. Haugerud, Y. Djenouri, Deepfakes: current and future trends. *Artificial Intelligence Review* **57**, 64 (2024). https://doi.org/10.1007/s10462-023-10679-x.

347. V. Ciancaglini, C. Gibson, D. Sancho, O. McCarthy, M. Eira, P. Amann, A. Klayn, "Malicious uses and abuses of artificial intelligence" (European Union Agency for Law Enforcement Cooperation, 2020).

348. R. Umbach, N. Henry, G. Beard, C. Berryessa, Non-Consensual Synthetic Intimate Imagery: Prevalence, Attitudes, and Knowledge in 10 Countries, arXiv:2402.01721 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2402.01721.

349. M. B. Kugler, C. Pace, Deepfake Privacy: Attitudes and Regulation. *Northwest. Univ. Law Rev.* **116**, 611-680 (2021). https://scholarlycommons.law.northwestern.edu/nulr/vol116/iss3/1.

350. M. Viola, C. Voto, Designed to abuse? Deepfakes and the non-consensual diffusion of intimate images. *Synthese* **201**, 30 (2023). https://doi.org/10.1007/s11229-022-04012-z.

351. D. M. J. Lazer, M. A. Baum, Y. Benkler, A. J. Berinsky, K. M. Greenhill, F. Menczer, M. J. Metzger, B. Nyhan, G. Pennycook, D. Rothschild, M. Schudson, S. A. Sloman, C. R. Sunstein, E. A. Thorson, D. J. Watts, J. L. Zittrain, The science of fake news. *Science* **359**, 1094-1096 (2018). https://doi.org/10.1126/science.aao29998.

352. G. Spitale, N. Biller-Andorno, F. Germani, Al model GPT-3 (dis)informs us better than humans. *Sci Adv* **9**, eadh1850 (2023). https://doi.org/10.1126/sciadv.adh1850.

353. M. Jakesch, J. T. Hancock, M. Naaman, Human heuristics for Al-generated language are flawed. *Proc. Natl. Acad. Sci. U. S. A.* **120**, e2208839120 (2023). https://doi.org/10.1073/pnas.2208839120.

354. K.-C. Yang, F. Menczer, Anatomy of an Al-powered malicious social botnet, arXiv:2307.16336 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2307.16336.

355. M. Masood, M. Nawaz, K. M. Malik, A. Javed, A. Irtaza, H. Malik, Deepfakes generation and detection: state-of-the-art, open challenges, countermeasures, and way forward. *Applied Intelligence* **53**, 3974-4026 (2023). https://doi.org/10.1007/s10489-022-03766-z.

356. D. Cooke, A. Edwards, S. Barkoff, K. Kelly, As Good As A Coin Toss: Human detection of Al-generated images, videos, audio, and audiovisual stimuli, arXiv:2403.16760 [cs.HC] (2024). http://arxiv.org/abs/2403.16760.

357. S. J. Nightingale, H. Farid, Al-synthesized faces are indistinguishable from real faces and more trustworthy. *Proc. Natl. Acad. Sci. U. S. A.* **119** (2022). https://doi.org/10.1073/pnas.2120481119.

358. S. C. Matz, J. D. Teeny, S. S. Vaid, H. Peters, G. M. Harari, M. Cerf, The potential of generative Al for personalized persuasion at scale. *Sci. Rep.* **14**, 4692 (2024). https://doi.org/10.1038/s41598-024-53755-0.

359. H. Bai, J. G. Voelkel, J. C. Eichstaedt, R. Willer, Artificial Intelligence Can Persuade Humans on Political Issues. (2023). https://doi.org/10.31219/osf.io/stakv.

360. K. Hackenburg, L. Ibrahim, B. M. Tappin, M. Tsakiris, Comparing the persuasiveness of role-playing large language models and human experts on polarized U.S. political issues. (2023). https://doi.org/10.31219/osf.io/ey8db.

361. K. Hackenburg, H. Margetts, Evaluating the persuasive influence of political microtargeting with large language models. (2023). https://doi.org/10.31219/osf.io/wnt8b.

362. A. Simchon, M. Edwards, S. Lewandowsky, The persuasive effects of political microtargeting in the age of generative artificial intelligence. *PNAS Nexus* **3**, gae035 (2024). https://doi.org/10.1093/pnasexus/pgae035.

363. B. M. Tappin, C. Wittenberg, L. B. Hewitt, A. J. Berinsky, D. G. Rand, Quantifying the potential persuasive returns to political microtargeting. *Proc. Natl. Acad. Sci. U. S. A.* **120**, e2216261120 (2023). https://doi.org/10.1073/pnas.2216261120.

364. F. Salvi, M. H. Ribeiro, R. Gallotti, R. West, On the Conversational Persuasiveness of Large Language Models: A Randomized Controlled Trial, arXiv:2403.14380 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2403.14380.

365. T. H. Costello, G. Pennycook, D. G. Rand, Durably reducing conspiracy beliefs through dialogues with Al. (2024). https://doi.org/10.31234/osf.io/xcwdn.
366. P. S. Park, S. Goldstein, A. O'Gara, M. Chen, D. Hendrycks, Al deception: A survey of examples, risks, and potential solutions. PATTERN 5 (2024). https://doi.org/10.1016/j.patter.2024.100988.

367.* M. Phuong, M. Aitchison, E. Catt, S. Cogan, A. Kaskasoli, V. Krakovna, D. Lindner, M. Rahtz, Y. Assael, S. Hodkinson, H. Howard, T. Lieberum, R. Kumar, M. A. Raad, A. Webson, L. Ho, S. Lin, S. Farquhar, M. Hutter, ... T. Shevlane, "Evaluating Frontier Models for Dangerous Capabilities" (Google Deepmind, 2024); https://doi.org/10.48550/arXiv.2403.13793.

368. M. Burtell, T. Woodside, Artificial Influence: An Analysis Of Al-Driven Persuasion, arXiv:2303.08721 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2303.08721.

369. S. Kapoor, A. Narayanan, "How to prepare for the deluge of generative Al on social media: A grounded analysis of the challenges and opportunities" (Knight First Amendment Institute at Columbia University, 2023); https://s3.amazonaws.com/kfai-documents/documents/a566f4ded5/How-to-Prepare-for-the-Deluge-of-Generative-Al-on-Social-Media.pdf.

370. M. Hameleers, Cheap Versus Deep Manipulation: The Effects of Cheapfakes Versus Deepfakes in a Political Setting. Int J Public Opin Res 36 (2024). https://doi.org/10.1093/ijpor/edae004.

371. S. Zuboff, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (PublicAffairs, 2019), pp. 704.

372. S. Vosoughi, D. Roy, S. Aral, The spread of true and false news online. Science 359, 1146-1151 (2018). https://doi.org/10.1126/science.aap9559.

373. D. Citron, R. Chesney, Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security. Calif. Law Rev. 107, 1753 (2019). https://scholarship.law.bu.edu/faculty_scholarship/640.

374. V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, S. Feizi, Can Al-Generated Text be Reliably Detected?, arXiv:2303.11156 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2303.11156.

375. S. S. Ghosal, S. Chakraborty, J. Geiping, F. Huang, D. Manocha, A. Bedi, A Survey on the Possibilities & Impossibilities of Al-generated Text Detection. Transactions on Machine Learning Research (2023). https://openreview.net/pdf?id=AXtFeYjboj.

376. J. Luo, G. Nan, D. Li, Y. Tan, Al-Generated Review Detection. (2023). https://doi.org/10.2139/ssrn.4610727.

377. L. Fröhling, A. Zubiaga, Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover. PeerJ Comput Sci 7, e443 (2021). https://doi.org/10.7717/peerj-cs.443.

378. S. Gehrmann, H. Strobelt, A. Rush, "GLTR: Statistical Detection and Visualization of Generated Text" in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations (Association for Computational Linguistics, 2019) pp. 111-116. https://doi.org/10.18653/v1/P19-3019.

379. D. M. Markowitz, J. T. Hancock, J. N. Bailenson, Linguistic Markers of Inherently False Al Communication and Intentionally False Human Communication: Evidence From Hotel Reviews. J. Lang. Soc. Psychol. 43, 63-82 (2024). https://doi.org/10.1177/0261927x231200201.

380. T. Berber Sardinha, Al-generated vs human-authored texts: A multidimensional comparison. Applied Corpus Linguistics 4, 100083 (2024). https://doi.org/10.1016/j.acorp.2023.100083.

381. Y. Xie, A. Rawal, Y. Cen, D. Zhao, S. K. Narang, S. Sushmita, MUGC: Machine Generated versus User Generated Content Detection, arXiv:2403.19725 [cs.CL] (2024). https://doi.org/10.48550/arXiv.2403.19725.

382. M. Christ, S. Gunn, O. Zamir, Undetectable Watermarks for Language Models, arXiv:2306.09194 [cs.CR] (2023). https://doi.org/10.48550/arXiv.2306.09194.

383. H. Zhang, B. L. Edelman, D. Francati, D. Venturi, G. Ateniese, B. Barak, Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models, arXiv:2311.04378 [cs.LG] (2023). https://doi.org/10.48550/arXiv.2311.04378.

384. C. R. Leibowicz, S. McGregor, A. Ovadya, "The Deepfake Detection Dilemma: A Multistakeholder Exploration of Adversarial Dynamics in Synthetic Media" in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) (Association for Computing Machinery, 2021) pp. 736-744. https://doi.org/10.1145/3461702.3462584.

385. L. Zhong, Z. Wang, Can LLM Replace Stack Overflow? A Study on Robustness and Reliability of Large Language Model Code Generation. AAAI 38, 21841-21849 (2024). https://doi.org/10.1609/aaai.v38i19.30185.

386.* H. Khlaaf, P. Mishkin, J. Achiam, G. Krueger, M. Brundage, "A Hazard Analysis Framework for Code Synthesis Large Language Models" (OpenAI, 2022); http://arxiv.org/abs/2207.14157.

387. H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, R. Karri, "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions" in 2022 IEEE Symposium on Security and Privacy (SP) (IEEE Computer Society, 2022) pp. 754-768. https://doi.org/10.1109/sp46214.2022.9833571.

388. G. Deng, Y. Liu, V. Mayoral-Vilches, P. Liu, Y. Li, Y. Xu, T. Zhang, Y. Liu, M. Pinzger, S. Rass, PentestGPT: An LLM-empowered Automatic Penetration Testing Tool, arXiv:2308.06782 [cs.SE] (2023). http://arxiv.org/abs/2308.06782.
366. P. S. Park, S. Goldstein, A. O'Gara, M. Chen, D. Hendrycks, Al deception: A survey of examples, risks, and potential solutions. PATTERN 5 (2024). https://doi.org/10.1016/j.patter.2024.100988.

367.* M. Phuong, M. Aitchison, E. Catt, S. Cogan, A. Kaskasoli, V. Krakovna, D. Lindner, M. Rahtz, Y. Assael, S. Hodkinson, H. Howard, T. Lieberum, R. Kumar, M. A. Raad, A. Webson, L. Ho, S. Lin, S. Farquhar, M. Hutter, ... T. Shevlane, "Evaluating Frontier Models for Dangerous Capabilities" (Google Deepmind, 2024); https://doi.org/10.48550/arXiv.2403.13793.

368. M. Burtell, T. Woodside, Artificial Influence: An Analysis Of Al-Driven Persuasion, arXiv:2303.08721 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2303.08721.

369. S. Kapoor, A. Narayanan, "How to prepare for the deluge of generative Al on social media: A grounded analysis of the challenges and opportunities" (Knight First Amendment Institute at Columbia University, 2023); https://s3.amazonaws.com/kfai-documents/documents/a566f4ded5/How-to-Prepare-for-the-Deluge-of-Generative-Al-on-Social-Media.pdf.

370. M. Hameleers, Cheap Versus Deep Manipulation: The Effects of Cheapfakes Versus Deepfakes in a Political Setting. Int J Public Opin Res 36 (2024). https://doi.org/10.1093/ijpor/edae004.

371. S. Zuboff, The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power (PublicAffairs, 2019), pp. 704.

372. S. Vosoughi, D. Roy, S. Aral, The spread of true and false news online. Science 359, 1146-1151 (2018). https://doi.org/10.1126/science.aap9559.

373. D. Citron, R. Chesney, Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security. Calif. Law Rev. 107, 1753 (2019). https://scholarship.law.bu.edu/faculty_scholarship/640.

374. V. S. Sadasivan, A. Kumar, S. Balasubramanian, W. Wang, S. Feizi, Can Al-Generated Text be Reliably Detected?, arXiv:2303.11156 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2303.11156.

375. S. S. Ghosal, S. Chakraborty, J. Geiping, F. Huang, D. Manocha, A. Bedi, A Survey on the Possibilities & Impossibilities of Al-generated Text Detection. Transactions on Machine Learning Research (2023). https://openreview.net/pdf?id=AXtFeYjboj.

376. J. Luo, G. Nan, D. Li, Y. Tan, Al-Generated Review Detection. (2023). https://doi.org/10.2139/ssrn.4610727.

377. L. Fröhling, A. Zubiaga, Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover. PeerJ Comput Sci 7, e443 (2021). https://doi.org/10.7717/peerj-cs.443.

378. S. Gehrmann, H. Strobelt, A. Rush, "GLTR: Statistical Detection and Visualization of Generated Text" in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations (Association for Computational Linguistics, 2019) pp. 111-116. https://doi.org/10.18653/v1/P19-3019.

379. D. M. Markowitz, J. T. Hancock, J. N. Bailenson, Linguistic Markers of Inherently False Al Communication and Intentionally False Human Communication: Evidence From Hotel Reviews. J. Lang. Soc. Psychol. 43, 63-82 (2024). https://doi.org/10.1177/0261927x231200201.

380. T. Berber Sardinha, Al-generated vs human-authored texts: A multidimensional comparison. Applied Corpus Linguistics 4, 100083 (2024). https://doi.org/10.1016/j.acorp.2023.100083.

381. Y. Xie, A. Rawal, Y. Cen, D. Zhao, S. K. Narang, S. Sushmita, MUGC: Machine Generated versus User Generated Content Detection, arXiv:2403.19725 [cs.CL] (2024). https://doi.org/10.48550/arXiv.2403.19725.

382. M. Christ, S. Gunn, O. Zamir, Undetectable Watermarks for Language Models, arXiv:2306.09194 [cs.CR] (2023). https://doi.org/10.48550/arXiv.2306.09194.

383. H. Zhang, B. L. Edelman, D. Francati, D. Venturi, G. Ateniese, B. Barak, Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models, arXiv:2311.04378 [cs.LG] (2023). https://doi.org/10.48550/arXiv.2311.04378.

384. C. R. Leibowicz, S. McGregor, A. Ovadya, "The Deepfake Detection Dilemma: A Multistakeholder Exploration of Adversarial Dynamics in Synthetic Media" in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) (Association for Computing Machinery, 2021) pp. 736-744. https://doi.org/10.1145/3461702.3462584.

385. L. Zhong, Z. Wang, Can LLM Replace Stack Overflow? A Study on Robustness and Reliability of Large Language Model Code Generation. AAAI 38, 21841-21849 (2024). https://doi.org/10.1609/aaai.v38i19.30185.

386.* H. Khlaaf, P. Mishkin, J. Achiam, G. Krueger, M. Brundage, "A Hazard Analysis Framework for Code Synthesis Large Language Models" (OpenAI, 2022); http://arxiv.org/abs/2207.14157.

387. H. Pearce, B. Ahmad, B. Tan, B. Dolan-Gavitt, R. Karri, "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions" in 2022 IEEE Symposium on Security and Privacy (SP) (IEEE Computer Society, 2022) pp. 754-768. https://doi.org/10.1109/sp46214.2022.9833571.

388. G. Deng, Y. Liu, V. Mayoral-Vilches, P. Liu, Y. Li, Y. Xu, T. Zhang, Y. Liu, M. Pinzger, S. Rass, PentestGPT: An LLM-empowered Automatic Penetration Testing Tool, arXiv:2308.06782 [cs.SE] (2023). http://arxiv.org/abs/2308.06782.
389.* J. Xu, J. W. Stokes, G. McDonald, X. Bai, D. Marshall, S. Wang, A. Swaminathan, Z. Li, AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks, arXiv:2403.01038 [cs.CR] (2024).
http://arxiv.org/abs/2403.01038.

390. A. Happe, A. Kaplan, J. Cito, LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks, arXiv:2310.11409 [cs.CR] (2023). https://doi.org/10.48550/arXiv.2310.11409.

391. R. Fang, R. Bindu, A. Gupta, Q. Zhan, D. Kang, LLM Agents can Autonomously Hack Websites, arXiv:2402.06664 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2402.06664.

392. M. Shao, B. Chen, S. Jancheska, B. Dolan-Gavitt, S. Garg, R. Karri, M. Shafique, An Empirical Evaluation of LLMs for Solving Offensive Security Challenges, arXiv:2402.11814 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2402.11814.

393. R. Raman, P. Calyam, K. Achuthan, ChatGPT or Bard: Who is a better Certified Ethical Hacker? Computers & Security 140, 103804 (2024). https://doi.org/10.1016/j.cose.2024.103804.

394. Center for Security and Emerging Technology, B. Buchanan, J. Bansemer, D. Cary, J. Lucas, M. Musser, "Automating Cyber Attacks: Hype and Reality" (Center for Security and Emerging Technology, 2020); https://doi.org/10.51593/2020ca002.

395. National Cyber Security Centre (NCSC), "The near-term impact of AI on the cyber threat" (GOV.UK, 2024); https://www.ncsc.gov.uk/report/impact-of-ai-on-cyber-threat.

396.* B. Berabi, A. Gronskiy, V. Raychev, G. Sivanrupan, V. Chibotaru, M. Vechev, DeepCode AI Fix: Fixing Security Vulnerabilities with Large Language Models, arXiv:2402.13291 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2402.13291.

397. R. Meng, M. Mirchev, M. Böhme, A. Roychoudhury, "Large language model guided protocol fuzzing" in Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS) (2024). https://www.ndss-symposium.org/wp-content/uploads/2024-556-paper.pdf.

398. Y. Ding, Y. Fu, O. Ibrahim, C. Sitawarin, X. Chen, B. Alomair, D. Wagner, B. Ray, Y. Chen, Vulnerability Detection with Code Language Models: How Far Are We?, arXiv:2403.18624 [cs.SE] (2024). http://arxiv.org/abs/2403.18624.

399. H. Pearce, B. Tan, B. Ahmad, R. Karri, B. Dolan-Gavitt, "Examining Zero-Shot Vulnerability Repair with Large Language Models" in 2023 IEEE Symposium on Security and Privacy (SP) (2023) pp. 2339-2356. https://doi.org/10.1109/sp46215.2023.10179324.

400.* A. Shestov, R. Levichev, R. Mussabayev, E. Maslov, A. Cheshkov, P. Zadorozhny, Finetuning Large Language Models for Vulnerability Detection, arXiv:2401.17010 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2401.17010.

401. N. Risse, M. Böhme, "Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection" in USENIX Security Symposium 2024 (2024) pp. 19.

402.* Artificial Intelligence Cyber Challenge, Artificial Intelligence Cyber Challenge (2024). https://aicyberchallenge.com/.

403. S. Ullah, M. Han, S. Pujar, H. Pearce, A. Coskun, G. Stringhini, "LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks" in IEEE Symposium on Security and Privacy (2024). https://research.ibm.com/publications/llms-cannot-reliably-identify-and-reason-about-security-vulnerabilities-yet-a-comprehensive-evaluation-framework-and-benchmarks.

404. S. Rose, C. Nelson, "Understanding AI-Facilitated Biological Weapon Development" (Centre for Long-Term Resilience, 2023); https://www.longtermresilience.org/post/report-launch-examining-risks-at-the-intersection-of-ai-and-bio.

405. J. B. Sandbrink, Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools, arXiv:2306.13952 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2306.13952.

406. E. H. Soice, R. Rocha, K. Cordova, M. Specter, K. M. Esvelt, Can large language models democratize access to dual-use biotechnology?, arXiv:2306.03809 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2306.03809.

407. S. R. Carter, N. Wheeler, S. Chwalek, C. Isaac, J. M. Yassif, "The Convergence of Artificial Intelligence and the Life Sciences: Safeguarding Technology, Rethinking Governance, and Preventing Catastrophe" (Nuclear Threat Initiative, 2023); https://www.nti.org/wp-content/uploads/2023/10/NTIBIO_AI_FINAL.pdf.

408. N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A.-K. Dombrowski, S. Goel, L. Phan, G. Mukobi, N. Helm-Burger, R. Lababidi, L. Justen, A. B. Liu, M. Chen, I. Barrass, O. Zhang, X. Zhu, ... D. Hendrycks, The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning, arXiv:2403.03218 [cs.LG] (2024). https://doi.org/10.48550/arXiv.2403.03218.

409. S. Batalis, "AI and Biorisk: An Explainer" (CSET, 2023); https://cset.georgetown.edu/publication/ai-and-biorisk-an-explainer/.

410. G. Lewis, P. Millett, A. Sandberg, A. Snyder-Beattie, G. Gronvall, Information Hazards in Biotechnology. Risk Anal. 39, 975-981 (2019). https://doi.org/10.1111/risa.13235.

411. C. A. Mouton, C. Lucas, E. Guest, "The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study" (RAND Corporation, 2024); https://www rand.org/pubs/research_reports/RRA2977-2.html.
389.* J. Xu, J. W. Stokes, G. McDonald, X. Bai, D. Marshall, S. Wang, A. Swaminathan, Z. Li, AutoAttacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks, arXiv:2403.01038 [cs.CR] (2024).
http://arxiv.org/abs/2403.01038.

390. A. Happe, A. Kaplan, J. Cito, LLMs as Hackers: Autonomous Linux Privilege Escalation Attacks, arXiv:2310.11409 [cs.CR] (2023). https://doi.org/10.48550/arXiv.2310.11409.

391. R. Fang, R. Bindu, A. Gupta, Q. Zhan, D. Kang, LLM Agents can Autonomously Hack Websites, arXiv:2402.06664 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2402.06664.

392. M. Shao, B. Chen, S. Jancheska, B. Dolan-Gavitt, S. Garg, R. Karri, M. Shafique, An Empirical Evaluation of LLMs for Solving Offensive Security Challenges, arXiv:2402.11814 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2402.11814.

393. R. Raman, P. Calyam, K. Achuthan, ChatGPT or Bard: Who is a better Certified Ethical Hacker? Computers & Security 140, 103804 (2024). https://doi.org/10.1016/j.cose.2024.103804.

394. Center for Security and Emerging Technology, B. Buchanan, J. Bansemer, D. Cary, J. Lucas, M. Musser, "Automating Cyber Attacks: Hype and Reality" (Center for Security and Emerging Technology, 2020); https://doi.org/10.51593/2020ca002.

395. National Cyber Security Centre (NCSC), "The near-term impact of AI on the cyber threat" (GOV.UK, 2024); https://www.ncsc.gov.uk/report/impact-of-ai-on-cyber-threat.

396.* B. Berabi, A. Gronskiy, V. Raychev, G. Sivanrupan, V. Chibotaru, M. Vechev, DeepCode AI Fix: Fixing Security Vulnerabilities with Large Language Models, arXiv:2402.13291 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2402.13291.

397. R. Meng, M. Mirchev, M. Böhme, A. Roychoudhury, "Large language model guided protocol fuzzing" in Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS) (2024). https://www.ndss-symposium.org/wp-content/uploads/2024-556-paper.pdf.

398. Y. Ding, Y. Fu, O. Ibrahim, C. Sitawarin, X. Chen, B. Alomair, D. Wagner, B. Ray, Y. Chen, Vulnerability Detection with Code Language Models: How Far Are We?, arXiv:2403.18624 [cs.SE] (2024). http://arxiv.org/abs/2403.18624.

399. H. Pearce, B. Tan, B. Ahmad, R. Karri, B. Dolan-Gavitt, "Examining Zero-Shot Vulnerability Repair with Large Language Models" in 2023 IEEE Symposium on Security and Privacy (SP) (2023) pp. 2339-2356. https://doi.org/10.1109/sp46215.2023.10179324.

400.* A. Shestov, R. Levichev, R. Mussabayev, E. Maslov, A. Cheshkov, P. Zadorozhny, Finetuning Large Language Models for Vulnerability Detection, arXiv:2401.17010 [cs.CR] (2024). https://doi.org/10.48550/arXiv.2401.17010.

401. N. Risse, M. Böhme, "Uncovering the Limits of Machine Learning for Automatic Vulnerability Detection" in USENIX Security Symposium 2024 (2024) pp. 19.

402.* Artificial Intelligence Cyber Challenge, Artificial Intelligence Cyber Challenge (2024). https://aicyberchallenge.com/.

403. S. Ullah, M. Han, S. Pujar, H. Pearce, A. Coskun, G. Stringhini, "LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks" in IEEE Symposium on Security and Privacy (2024). https://research.ibm.com/publications/llms-cannot-reliably-identify-and-reason-about-security-vulnerabilities-yet-a-comprehensive-evaluation-framework-and-benchmarks.

404. S. Rose, C. Nelson, "Understanding AI-Facilitated Biological Weapon Development" (Centre for Long-Term Resilience, 2023); https://www.longtermresilience.org/post/report-launch-examining-risks-at-the-intersection-of-ai-and-bio.

405. J. B. Sandbrink, Artificial intelligence and biological misuse: Differentiating risks of language models and biological design tools, arXiv:2306.13952 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2306.13952.

406. E. H. Soice, R. Rocha, K. Cordova, M. Specter, K. M. Esvelt, Can large language models democratize access to dual-use biotechnology?, arXiv:2306.03809 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2306.03809.

407. S. R. Carter, N. Wheeler, S. Chwalek, C. Isaac, J. M. Yassif, "The Convergence of Artificial Intelligence and the Life Sciences: Safeguarding Technology, Rethinking Governance, and Preventing Catastrophe" (Nuclear Threat Initiative, 2023); https://www.nti.org/wp-content/uploads/2023/10/NTIBIO_AI_FINAL.pdf.

408. N. Li, A. Pan, A. Gopal, S. Yue, D. Berrios, A. Gatti, J. D. Li, A.-K. Dombrowski, S. Goel, L. Phan, G. Mukobi, N. Helm-Burger, R. Lababidi, L. Justen, A. B. Liu, M. Chen, I. Barrass, O. Zhang, X. Zhu, ... D. Hendrycks, The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning, arXiv:2403.03218 [cs.LG] (2024). https://doi.org/10.48550/arXiv.2403.03218.

409. S. Batalis, "AI and Biorisk: An Explainer" (CSET, 2023); https://cset.georgetown.edu/publication/ai-and-biorisk-an-explainer/.

410. G. Lewis, P. Millett, A. Sandberg, A. Snyder-Beattie, G. Gronvall, Information Hazards in Biotechnology. Risk Anal. 39, 975-981 (2019). https://doi.org/10.1111/risa.13235.

411. C. A. Mouton, C. Lucas, E. Guest, "The Operational Risks of AI in Large-Scale Biological Attacks: Results of a Red-Team Study" (RAND Corporation, 2024); https://www.rand.org/pubs/research_reports/RRA2977-2.html.
412. S. Ben Ouaghram-Gormley, *Barriers to Bioweapons: The Challenges of Expertise and Organization for Weapons Development* (Cornell University Press, 2014), pp. 220.

413. J. Revill, C. Jefferson, Tacit knowledge and the biological weapons regime. *Sci. Public Policy* **41**, 597–610 (2014).
https://doi.org/10.1093/scipol/sct090.

414. E. National Academies of Sciences, and Medicine, *Biodefense in the Age of Synthetic Biology* (National Academies Press, Washington, DC, USA, 2018), pp. 188.

415. A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. White, P. Schwaller, "Augmenting large language models with chemistry tools" in *37th Conference on Neural Information Processing Systems (NeurIPS 2023) AI for Science Workshop* (2023). https://openreview.net/forum?id=wdGgL6Ix3I.

416. K. H. Sumida, R. Núñez-Franco, I. Kalvet, S. J. Pellock, B. I. M. Wicky, L. F. Milles, J. Dauparas, J. Wang, Y. Kipnis, N. Jameson, A. Kang, J. De La Cruz, B. Sankaran, A. K. Bera, G. Jiménez-Osés, D. Baker, Improving Protein Expression, Stability, and Function with ProteinMPNN. *J. Am. Chem. Soc.* **146**, 2054–2061 (2024). https://doi.org/10.1021/jacs.3c10941.

417. Z. Wu, S. B. J. Kan, R. D. Lewis, B. J. Wittmann, F. H. Arnold, Machine learning-assisted directed protein evolution with combinatorial libraries. *Proc. Natl. Acad. Sci. U. S. A.* **116**, 8852–8858 (2019). https://doi.org/10.1073/pnas.1901979116.

418. A. H.-W. Yeh, C. Norn, Y. Kipnis, D. Tischer, S. J. Pellock, D. Evans, P. Ma, G. R. Lee, J. Z. Zhang, I. Anishchenko, B. Coventry, L. Cao, J. Dauparas, S. Halabiya, M. DeWitt, L. Carter, K. N. Houk, D. Baker, De novo design of luciferases using deep learning. *Nature* **614**, 774–780 (2023). https://doi.org/10.1038/s41586-023-05696-3.

419. J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahern, A. J. Borst, R. J. Ragotte, L. F. Milles, B. I. M. Wicky, N. Hanikel, S. J. Pellock, A. Courbet, W. Sheffler, J. Wang, P. Venkatesh, I. Sappington, S. V. Torres, . . . D. Baker, De novo design of protein structure and function with RFdiffusion. *Nature* **620**, 1089–1100 (2023).
https://doi.org/10.1038/s41586-023-06415-8.

420. T. Blaschke, J. Arús-Pous, H. Chen, C. Margreitter, C. Tyrchan, O. Engkvist, K. Papadopoulos, A. Patronov, REINVENT 2.0: An AI Tool for De Novo Drug Design. *J. Chem. Inf. Model.* **60**, 5918–5922 (2020).
https://doi.org/10.1021/acs.jcim.0c00915.

421. N. N. Thadani, S. Gurev, P. Notin, N. Youssef, N. J. Rollins, D. Ritter, C. Sander, Y. Gal, D. S. Marks, Learning from prepandemic data to forecast viral escape. *Nature* **622**, 818–825 (2023). https://doi.org/10.1038/s41586-023-06617-0.

422. F. Urbina, F. Lentzos, C. Invernizzi, S. Ekins, Dual Use of Artificial Intelligence-powered Drug Discovery. *Nat Mach Intell* **4**, 189–191 (2022). https://doi.org/10.1038/s42256-022-00465-9.

423.* A. Elnaggar, H. Essam, W. Salah-Eldin, W. Moustafa, M. Elkerdawy, C. Rochereau, B. Rost, Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling, arXiv:2301.06568 [cs.LG] (2023).
https://doi.org/10.48550/arXiv.2301.06568.

424. T. Inagaki, A. Kato, K. Takahashi, H. Ozaki, G. N. Kanda, LLMs can generate robotic scripts from goal-oriented instructions in biological laboratory automation, arXiv:2304.10267 [q-bio.QM] (2023). http://arxiv.org/abs/2304.10267.

425. J. N. Acosta, G. J. Falcone, P. Rajpurkar, E. J. Topol, Multimodal biomedical AI. *Nat. Med.* **28**, 1773–1784 (2022).
https://doi.org/10.1038/s41591-022-01981-2.

426. M. Moor, O. Banerjee, Z. S. H. Abad, H. M. Krumholz, J. Leskovec, E. J. Topol, P. Rajpurkar, Foundation models for generalist medical artificial intelligence. *Nature* **616**, 259–265 (2023). https://doi.org/10.1038/s41586-023-05881-4.

427.* 310.ai, GenAI + BIO: Nature didn't have time, we have GPUs (2024). https://310.ai/.

428. National Security Commission on Emerging Biotechnology (NSCEB), "Risks of AlxBio" (NSCEB, 2024); https://www.biotech.senate.gov/wp-content/uploads/2024/01/NSCEB_Albio_WP3_Risks.pdf.

429. C. A. Mouton, C. Lucas, E. Guest, "The Operational Risks of AI in Large-Scale Biological Attacks: A Red-Team Approach" (RAND Corporation, 2023); https://www.rand.org/pubs/research_reports/RRA2977-1.html.

430. I. D. Raji, I. E. Kumar, A. Horowitz, A. Selbst, "The Fallacy of AI Functionality" in *Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22)* (Association for Computing Machinery, 2022) pp. 959–972. https://doi.org/10.1145/3531146.3533158.

431. A. Mallen, A. Asai, V. Zhong, R. Das, D. Khashabi, H. Hajishirzi, "When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories" in *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)* (Association for Computational Linguistics, 2023) pp. 9802–9822. https://doi.org/10.18653/v1/2023.acl-long.546.

432. A. Perlman, The Implications of ChatGPT for Legal Services and Society in *The Practice*. (Center on the Legal Profession, Harvard Law School, 2023). https://clp.law.harvard.edu/knowledge-hub/magazine/issues/generative-ai-in-the-legal-profession/the-implications-of-chatgpt-for-legal-services-and-society/.

433. E. Martínez, Re-evaluating GPT-4's bar exam performance. *Artificial Intelligence and Law* (2024).
https://doi.org/10.1007/s10506-024-09396-9.

434. J. Tan, H. Westermann, K. Benyekhlef, "ChatGPT as an Artificial Lawyer?" in *Workshop on Artificial Intelligence for Access to Justice (AI4AJ 2023)* (CEUR Workshop Proceedings, 2023). https://ceur-ws.org/Vol-3435/short2.pdf.
412. S. Ben Ouaghram-Gormley, *Barriers to Bioweapons: The Challenges of Expertise and Organization for Weapons Development* (Cornell University Press, 2014), pp. 220.

413. J. Revill, C. Jefferson, Tacit knowledge and the biological weapons regime. *Sci. Public Policy* **41**, 597–610 (2014).
https://doi.org/10.1093/scipol/sct090.

414. E. National Academies of Sciences, and Medicine, *Biodefense in the Age of Synthetic Biology* (National Academies Press, Washington, DC, USA, 2018), pp. 188.

415. A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. White, P. Schwaller, "Augmenting large language models with chemistry tools" in *37th Conference on Neural Information Processing Systems (NeurIPS 2023) AI for Science Workshop* (2023). https://openreview.net/forum?id=wdGtL6Ix3I.

416. K. H. Sumida, R. Núñez-Franco, I. Kalvet, S. J. Pellock, B. I. M. Wicky, L. F. Milles, J. Dauparas, J. Wang, Y. Kipnis, N. Jameson, A. Kang, J. De La Cruz, B. Sankaran, A. K. Bera, G. Jiménez-Osés, D. Baker, Improving Protein Expression, Stability, and Function with ProteinMPNN. *J. Am. Chem. Soc.* **146**, 2054–2061 (2024). https://doi.org/10.1021/jacs.3c10941.

417. Z. Wu, S. B. J. Kan, R. D. Lewis, B. J. Wittmann, F. H. Arnold, Machine learning-assisted directed protein evolution with combinatorial libraries. *Proc. Natl. Acad. Sci. U. S. A.* **116**, 8852–8858 (2019). https://doi.org/10.1073/pnas.1901979116.

418. A. H.-W. Yeh, C. Norn, Y. Kipnis, D. Tischer, S. J. Pellock, D. Evans, P. Ma, G. R. Lee, J. Z. Zhang, I. Anishchenko, B. Coventry, L. Cao, J. Dauparas, S. Halabiya, M. DeWitt, L. Carter, K. N. Houk, D. Baker, De novo design of luciferases using deep learning. *Nature* **614**, 774–780 (2023). https://doi.org/10.1038/s41586-023-05696-3.

419. J. L. Watson, D. Juergens, N. R. Bennett, B. L. Trippe, J. Yim, H. E. Eisenach, W. Ahern, A. J. Borst, R. J. Ragotte, L. F. Milles, B. I. M. Wicky, N. Hanikel, S. J. Pellock, A. Courbet, W. Sheffler, J. Wang, P. Venkatesh, I. Sappington, S. V. Torres, . . . D. Baker, De novo design of protein structure and function with RFdiffusion. *Nature* **620**, 1089–1100 (2023).
https://doi.org/10.1038/s41586-023-06415-8.

420. T. Blaschke, J. Arús-Pous, H. Chen, C. Margreitter, C. Tyrchan, O. Engkvist, K. Papadopoulos, A. Patronov, REINVENT 2.0: An AI Tool for De Novo Drug Design. *J. Chem. Inf. Model.* **60**, 5918–5922 (2020).
https://doi.org/10.1021/acs.jcim.0c00915.

421. N. N. Thadani, S. Gurev, P. Notin, N. Youssef, N. J. Rollins, D. Ritter, C. Sander, Y. Gal, D. S. Marks, Learning from prepandemic data to forecast viral escape. *Nature* **622**, 818–825 (2023). https://doi.org/10.1038/s41586-023-06617-0.

422. F. Urbina, F. Lentzos, C. Invernizzi, S. Ekins, Dual Use of Artificial Intelligence-powered Drug Discovery. *Nat Mach Intell* **4**, 189–191 (2022). https://doi.org/10.1038/s42256-022-00465-9.

423.* A. Elnaggar, H. Essam, W. Salah-Eldin, W. Moustafa, M. Elkerdawy, C. Rochereau, B. Rost, Ankh: Optimized Protein Language Model Unlocks General-Purpose Modelling, arXiv:2301.06568 [cs.LG] (2023).
https://doi.org/10.48550/arXiv.2301.06568.

424. T. Inagaki, A. Kato, K. Takahashi, H. Ozaki, G. N. Kanda, LLMs can generate robotic scripts from goal-oriented instructions in biological laboratory automation, arXiv:2304.10267 [q-bio.QM] (2023). http://arxiv.org/abs/2304.10267.

425. J. N. Acosta, G. J. Falcone, P. Rajpurkar, E. J. Topol, Multimodal biomedical AI. *Nat. Med.* **28**, 1773–1784 (2022).
https://doi.org/10.1038/s41591-022-01981-2.

426. M. Moor, O. Banerjee, Z. S. H. Abad, H. M. Krumholz, J. Leskovec, E. J. Topol, P. Rajpurkar, Foundation models for generalist medical artificial intelligence. *Nature* **616**, 259–265 (2023). https://doi.org/10.1038/s41586-023-05881-4.

427.* 310.ai, GenAI + BIO: Nature didn't have time, we have GPUs (2024). https://310.ai/.

428. National Security Commission on Emerging Biotechnology (NSCEB), "Risks of AlxBio" (NSCEB, 2024); https://www.biotech.senate.gov/wp-content/uploads/2024/01/NSCEB_Albio_WP3_Risks.pdf.

429. C. A. Mouton, C. Lucas, E. Guest, "The Operational Risks of AI in Large-Scale Biological Attacks: A Red-Team Approach" (RAND Corporation, 2023); https://www.rand.org/pubs/research_reports/RRA2977-1.html.

430. I. D. Raji, I. E. Kumar, A. Horowitz, A. Selbst, "The Fallacy of AI Functionality" in *Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22)* (Association for Computing Machinery, 2022) pp. 959–972. https://doi.org/10.1145/3531146.3533158.

431. A. Mallen, A. Asai, V. Zhong, R. Das, D. Khashabi, H. Hajishirzi, "When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories" in *Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)* (Association for Computational Linguistics, 2023) pp. 9802–9822. https://doi.org/10.18653/v1/2023.acl-long.546.

432. A. Perlman, The Implications of ChatGPT for Legal Services and Society in *The Practice*. (Center on the Legal Profession, Harvard Law School, 2023). https://clp.law.harvard.edu/knowledge-hub/magazine/issues/generative-ai-in-the-legal-profession/the-implications-of-chatgpt-for-legal-services-and-society/.

433. E. Martínez, Re-evaluating GPT-4's bar exam performance. *Artificial Intelligence and Law* (2024).
https://doi.org/10.1007/s10506-024-09396-9.

434. J. Tan, H. Westermann, K. Benyekhlef, "ChatGPT as an Artificial Lawyer?" in *Workshop on Artificial Intelligence for Access to Justice (AI4AJ 2023)* (CEUR Workshop Proceedings, 2023). https://ceur-ws.org/Vol-3435/short2.pdf.
435. J. A. Omiye, J. C. Lester, S. Spichak, V. Rotemberg, R. Daneshjou, Large language models propagate race-based medicine. npj Digit. Med. **6**, 1-4 (2023). https://doi.org/10.1038/s41746-023-00939-z.

436. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, P. Payne, M. Seneviratne, P. Gamble, C. Kelly, A. Babiker, N. Schärli, A. Chowdhery, P. Mansfield, D. Demner-Fushman, ... V. Natarajan, Large language models encode clinical knowledge. Nature **620**, 172-180 (2023). https://doi.org/10.1038/s41586-023-06291-2.

437. T. H. Kung, M. Cheatham, A. Medenilla, C. Sillos, L. De Leon, C. Elepaño, M. Madriaga, R. Aggabao, G. Diaz-Candido, J. Maningo, V. Tseng, Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health **2**, e0000198 (2023). https://doi.org/10.1371/journal.pdig.0000198.

438. A. Ettinger, What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Trans. Assoc. Comput. Linguist. **8**, 34-48 (2020). https://doi.org/10.1162/tacl_a_00298.

439. Y. Zhang, M. Yasunaga, Z. Zhou, J. Z. HaoChen, J. Zou, P. Liang, S. Yeung, "Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models" in Findings of the Association for Computational Linguistics: ACL 2023 (Association for Computational Linguistics, 2023) pp. 7479-7498. https://doi.org/10.18653/v1/2023.findings-acl.472.

440.* M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, ... W. Zaremba, Evaluating Large Language Models Trained on Code, arXiv:2107.03374 [cs.LG] (2021). http://arxiv.org/abs/2107.03374.

441. C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, K. R. Narasimhan, "SWE-bench: Can Language Models Resolve Real-world Github Issues?" in The 12th International Conference on Learning Representations (ICLR 2024) (2023). https://openreview.net/forum?id=VTF8yNQM66.

442. R. Pan, A. R. Ibrahimzada, R. Krishna, D. Sankar, L. P. Wassi, M. Merler, B. Sobolev, R. Pavuluri, S. Sinha, R. Jabbarvand, "Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code" in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE '24) (Association for Computing Machinery, 2024) pp. 1-13. https://doi.org/10.1145/3597503.3639226.

443. F. Cassano, L. Li, A. Sethi, N. Shinn, A. Brennan-Jones, J. Ginesin, E. Berman, G. Chakhnashvili, A. Lozhkov, C. J. Anderson, A. Guha, Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions, arXiv:2312.12450 [cs.SE] (2023). http://arxiv.org/abs/2312.12450.

444. S. Nguyen, H. M. Babe, Y. Zi, A. Guha, C. J. Anderson, M. Q. Feldman, "How Beginning Programmers and Code LLMs (Mis)read Each Other" in Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24) (Association for Computing Machinery, 2024) pp. 1-26. https://doi.org/10.1145/3613904.3642706.

445. P. Darke, T. Dhar, C. B. Weinberg, X. Zeng, Truth, Lies and Advertising: Understanding Market and Individual Causes of False Advertising Claims. (2022). https://doi.org/10.2139/ssrn.4140673.

446. M. Rauh, J. F. J. Mellor, J. Uesato, P.-S. Huang, J. Welbl, L. Weidinger, S. Dathathri, A. Glaese, G. Irving, I. Gabriel, W. Isaac, L. A. Hendricks, "Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models" in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022). https://openreview.net/forum?id=u46CbCaLufp.

447. O. Bohdal, T. Hospedales, P. H. S. Torr, F. Barez, "Fairness in AI and Its Long-Term Implications on Society" in Intersections, Reinforcements, Cascades: Proceedings of the 2023 Stanford Existential Risks Conference (Stanford Existential Risks Initiative, 2023) pp. 171-186. https://doi.org/10.25740/pj287ht2654.

448. D. Acemoglu, "Harms of AI" in The Oxford Handbook of AI Governance, J. B. Bullock, Y.-C. Chen, J. Himmelreich, V. M. Hudson, A. Korinek, M. M. Young, B. Zhang, Eds. (Oxford University Press, 2023).

449. I. Ajunwa, The Quantified Worker: Law and Technology in the Modern Workplace (Cambridge University Press, Cambridge, 2023).

450. Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan, Dissecting racial bias in an algorithm used to manage the health of populations. Science **366**, 447-453 (2019). https://doi.org/10.1126/science.aax2342.

451. A. Wong, E. Otles, J. P. Donnelly, A. Krumm, J. McCullough, O. DeTroyer-Cooley, J. Pestrue, M. Phillips, J. Konye, C. Penoza, M. Ghous, K. Singh, External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Intern Med **181**, 1065-1070 (2021). https://doi.org/10.1001/jamainternmed.2021.2626.

452. J. Buolamwini, T. Gebru, "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification" in Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT/MM '19) (PMLR, 2018) pp. 77-91. https://proceedings.mlr.press/v81/buolamwini18a.html.

453. J. Dressel, H. Farid, The accuracy, fairness, and limits of predicting recidivism. Sci Adv **4**, eaao5580 (2018). https://doi.org/10.1126/sciadv.aao5580.

454. M. A. Gianfrancesco, S. Tamang, J. Yazdany, G. Schmajuk, Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern. Med. **178**, 1544-1547 (2018). https://doi.org/10.1001/jamainternmed.2018.3763.
435. J. A. Omiye, J. C. Lester, S. Spichak, V. Rotemberg, R. Daneshjou, Large language models propagate race-based medicine. npj Digit. Med. **6**, 1-4 (2023). https://doi.org/10.1038/s41746-023-00939-z.

436. K. Singhal, S. Azizi, T. Tu, S. S. Mahdavi, J. Wei, H. W. Chung, N. Scales, A. Tanwani, H. Cole-Lewis, S. Pfohl, P. Payne, M. Seneviratne, P. Gamble, C. Kelly, A. Babiker, N. Schärli, A. Chowdhery, P. Mansfield, D. Demner-Fushman, ... V. Natarajan, Large language models encode clinical knowledge. Nature **620**, 172-180 (2023). https://doi.org/10.1038/s41586-023-06291-2.

437. T. H. Kung, M. Cheatham, A. Medenilla, C. Sillos, L. De Leon, C. Elepaño, M. Madriaga, R. Aggabao, G. Diaz-Candido, J. Maningo, V. Tseng, Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health **2**, e0000198 (2023). https://doi.org/10.1371/journal.pdig.0000198.

438. A. Ettinger, What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models. Trans. Assoc. Comput. Linguist. **8**, 34-48 (2020). https://doi.org/10.1162/tacl_a_00298.

439. Y. Zhang, M. Yasunaga, Z. Zhou, J. Z. HaoChen, J. Zou, P. Liang, S. Yeung, "Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models" in Findings of the Association for Computational Linguistics: ACL 2023 (Association for Computational Linguistics, 2023) pp. 7479-7498. https://doi.org/10.18653/v1/2023.findings-acl.472.

440.* M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, ... W. Zaremba, Evaluating Large Language Models Trained on Code, arXiv:2107.03374 [cs.LG] (2021). http://arxiv.org/abs/2107.03374.

441. C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, K. R. Narasimhan, "SWE-bench: Can Language Models Resolve Real-world Github Issues?" in The 12th International Conference on Learning Representations (ICLR 2024) (2023). https://openreview.net/forum?id=VTF8yNQM66.

442. R. Pan, A. R. Ibrahimzada, R. Krishna, D. Sankar, L. P. Wassi, M. Merler, B. Sobolev, R. Pavuluri, S. Sinha, R. Jabbarvand, "Lost in Translation: A Study of Bugs Introduced by Large Language Models while Translating Code" in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (ICSE '24) (Association for Computing Machinery, 2024) pp. 1-13. https://doi.org/10.1145/3597503.3639226.

443. F. Cassano, L. Li, A. Sethi, N. Shinn, A. Brennan-Jones, J. Ginesin, E. Berman, G. Chakhnashvili, A. Lozhkov, C. J. Anderson, A. Guha, Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions, arXiv:2312.12450 [cs.SE] (2023). http://arxiv.org/abs/2312.12450.

444. S. Nguyen, H. M. Babe, Y. Zi, A. Guha, C. J. Anderson, M. Q. Feldman, "How Beginning Programmers and Code LLMs (Mis)read Each Other" in Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24) (Association for Computing Machinery, 2024) pp. 1-26. https://doi.org/10.1145/3613904.3642706.

445. P. Darke, T. Dhar, C. B. Weinberg, X. Zeng, Truth, Lies and Advertising: Understanding Market and Individual Causes of False Advertising Claims. (2022). https://doi.org/10.2139/ssrn.4140673.

446. M. Rauh, J. F. J. Mellor, J. Uesato, P.-S. Huang, J. Welbl, L. Weidinger, S. Dathathri, A. Glaese, G. Irving, I. Gabriel, W. Isaac, L. A. Hendricks, "Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models" in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022). https://openreview.net/forum?id=u46CbCaLufp.

447. O. Bohdal, T. Hospedales, P. H. S. Torr, F. Barez, "Fairness in AI and Its Long-Term Implications on Society" in Intersections, Reinforcements, Cascades: Proceedings of the 2023 Stanford Existential Risks Conference (Stanford Existential Risks Initiative, 2023) pp. 171-186. https://doi.org/10.25740/pj287ht2654.

448. D. Acemoglu, "Harms of AI" in The Oxford Handbook of AI Governance, J. B. Bullock, Y.-C. Chen, J. Himmelreich, V. M. Hudson, A. Korinek, M. M. Young, B. Zhang, Eds. (Oxford University Press, 2023).

449. I. Ajunwa, The Quantified Worker: Law and Technology in the Modern Workplace (Cambridge University Press, Cambridge, 2023).

450. Z. Obermeyer, B. Powers, C. Vogeli, S. Mullainathan, Dissecting racial bias in an algorithm used to manage the health of populations. Science **366**, 447-453 (2019). https://doi.org/10.1126/science.aax2342.

451. A. Wong, E. Otles, J. P. Donnelly, A. Krumm, J. McCullough, O. DeTroyer-Cooley, J. Pestrue, M. Phillips, J. Konye, C. Penoza, M. Ghous, K. Singh, External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Intern Med **181**, 1065-1070 (2021). https://doi.org/10.1001/jamainternmed.2021.2626.

452. J. Buolamwini, T. Gebru, "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification" in Proceedings of the 1st Conference on Fairness, Accountability and Transparency (FAT/MM '19) (PMLR, 2018) pp. 77-91. https://proceedings.mlr.press/v81/buolamwini18a.html.

453. J. Dressel, H. Farid, The accuracy, fairness, and limits of predicting recidivism. Sci Adv **4**, eaao5580 (2018). https://doi.org/10.1126/sciadv.aao5580.

454. M. A. Gianfrancesco, S. Tamang, J. Yazdany, G. Schmajuk, Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data. JAMA Intern. Med. **178**, 1544-1547 (2018). https://doi.org/10.1001/jamainternmed.2018.3763.
455. Y. Wan, G. Pu, J. Sun, A. Garimella, K.-W. Chang, N. Peng, ""Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters" in Findings of the Association for Computational Linguistics: EMNLP 2023 (Association for Computational Linguistics, 2023) pp. 3730-3748. https://doi.org/10.18653/v1/2023.findings-emnlp.243.

456. D. van Niekerk, M. Pérez-Ortiz, J. Shawe-Taylor, D. Orlič, I. Drobnjak, J. Kay, N. Siegel, K. Evans, N. Moorosi, T. Eliassi-Rad, L. M. Tanczer, W. Holmes, M. P. Deisenroth, I. Straw, M. Fasli, R. Adams, N. Oliver, D. Mladenić, U. Aneja, ... M. Janicky, "Challenging Systematic Prejudices: An Investigation into Bias Against Women and Girls in Large Language Models" (UNESCO, IRCAI, 2024); https://ircai.org/project/challenging-systematic-prejudices/.

457. M. Vlasceanu, D. M. Amodio, Propagation of societal gender inequality by internet search algorithms. *Proceedings of the National Academy of Sciences* **119**, e2204529119 (2022). https://doi.org/10.1073/pnas.2204529119.

458. US Equal Employment Opportunity Commission, EEOC Sues iTutorGroup for Age Discrimination (2022). https://www.eeoc.gov/newsroom/eeoc-sues-itutorgroup-age-discrimination.

459. M. Díaz, I. Johnson, A. Lazar, A. M. Piper, D. Gergle, "Addressing Age-Related Bias in Sentiment Analysis" in *Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI-19)* (International Joint Conferences on Artificial Intelligence Organization, 2019) pp. 6146-6150. https://doi.org/10.24963/ijcai.2019/852.

460. J. Stypinska, Al ageism: a critical roadmap for studying age discrimination and exclusion in digitalized societies. *AI Soc.* **38**, 665-677 (2023). https://doi.org/10.1007/s00146-022-01553-5.

461. P. Rucker, M. Miller, D. Armstrong, How Cigna Saves Millions by Having Its Doctors Reject Claims Without Reading Them in *ProPublica*. (2023). https://www.propublica.org/article/cigna-pxdx-medical-health-insurance-rejection-claims.

462. A. Mack, R. Qadri, R. Denton, S. K. Kane, C. L. Bennett, ""They only care to show us the wheelchair": Disability representation in text-to-image AI models" in *Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24)* (ACM, 2024). https://dl.acm.org/doi/10.1145/3613904.3642166.

463. P. N. Venkit, M. Srinath, S. Wilson, "Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models" in *Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)* (Association for Computational Linguistics, 2023) pp. 26-34. https://doi.org/10.18653/v1/2023.trustnlp-1.3.

464. I. A. Adeyanju, O. O. Bello, M. A. Adegboye, Machine learning methods for sign language recognition: A critical review and analysis. *Intelligent Systems with Applications* **12**, 200056 (2021). https://doi.org/10.1016/j.iswa.2021.200056.

465. S. Gueuwou, K. Takyi, M. Müller, M. S. Nyarko, R. Adade, R.-M. O. M. Gyening, "AfriSign: Machine Translation for African Sign Languages" in *4th Workshop on African Natural Language Processing (AfricaNLP 2023)* (2023). https://openreview.net/forum?id=EHldk3J2xk.

466. W. Guo, A. Caliskan, "Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases" in *Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21)* (Association for Computing Machinery, 2021) pp. 122-133. https://doi.org/10.1145/3461702.3462536.

467. Á. A. Cabrera, W. Epperson, F. Hohman, M. Kahng, J. Morgenstern, D. H. Chau, "FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning" in *2019 IEEE Conference on Visual Analytics Science and Technology (VAST)* (2019) pp. 46-56. https://doi.org/10.1109/vast47406.2019.8986948.

468. L. Magee, L. Ghahremanlou, K. Soldatic, S. Robertson, Intersectional Bias in Causal Language Models, arXiv:2107.07691 [cs.CL] (2021). https://doi.org/10.48550/arXiv:2107.07691.

469. A. Ovalle, A. Subramonian, V. Gautam, G. Gee, K.-W. Chang, "Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness" in *Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23)* (Association for Computing Machinery, 2023) pp. 496-511. https://doi.org/10.1145/3600211.3604705.

470. J. S. Park, M. S. Bernstein, R. N. Brewer, E. Kamar, M. R. Morris, "Understanding the Representation and Representativeness of Age in AI Data Sets" in *Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21)* (Association for Computing Machinery, 2021) pp. 834-842. https://doi.org/10.1145/3461702.3462590.

471. R. Kamikubo, L. Wang, C. Marte, A. Mahmood, H. Kacorri, "Data Representativeness in Accessibility Datasets: A Meta-Analysis" in *Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)* (Association for Computing Machinery, 2022) pp. 1-15. https://doi.org/10.1145/3517428.3544826.

472.* L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, Z. Kenton, S. Brown, W. Hawkins, T. Stepleton, C. Biles, A. Birhane, J. Haas, L. Rimell, L. A. Hendricks, ... I. Gabriel, "Ethical and social risks of harm from Language Models" (Google DeepMind, 2021); http://arxiv.org/abs/2112.04359.

473. J. Nwatu, O. Ignat, R. Mihalcea, "Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models" in *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)* (Association for Computational Linguistics, 2023) pp. 10686-10702. https://doi.org/10.18653/v1/2023.emnlp-main.660.

474. S. Ghosh, A. Caliskan, "ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages" in *Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23)* (Association for Computing Machinery, 2023) pp. 901-912. https://doi.org/10.1145/3600211.3604672.
455. Y. Wan, G. Pu, J. Sun, A. Garimella, K.-W. Chang, N. Peng, ""Kelly is a Warm Person, Joseph is a Role Model": Gender Biases in LLM-Generated Reference Letters" in Findings of the Association for Computational Linguistics: EMNLP 2023 (Association for Computational Linguistics, 2023) pp. 3730-3748. https://doi.org/10.18653/v1/2023.findings-emnlp.243.

456. D. van Niekerk, M. Pérez-Ortiz, J. Shawe-Taylor, D. Orlič, I. Drobnjak, J. Kay, N. Siegel, K. Evans, N. Moorosi, T. Eliassi-Rad, L. M. Tanczer, W. Holmes, M. P. Deisenroth, I. Straw, M. Fasli, R. Adams, N. Oliver, D. Mladenić, U. Aneja, ... M. Janicky, "Challenging Systematic Prejudices: An Investigation into Bias Against Women and Girls in Large Language Models" (UNESCO, IRCAI, 2024); https://ircai.org/project/challenging-systematic-prejudices/.

457. M. Vlasceanu, D. M. Amodio, Propagation of societal gender inequality by internet search algorithms. *Proceedings of the National Academy of Sciences* **119**, e2204529119 (2022). https://doi.org/10.1073/pnas.2204529119.

458. US Equal Employment Opportunity Commission, EEOC Sues iTutorGroup for Age Discrimination (2022). https://www.eeoc.gov/newsroom/eeoc-sues-itutorgroup-age-discrimination.

459. M. Díaz, I. Johnson, A. Lazar, A. M. Piper, D. Gergle, "Addressing Age-Related Bias in Sentiment Analysis" in *Proceedings of the 28th International Joint Conference on Artificial Intelligence (IJCAI-19)* (International Joint Conferences on Artificial Intelligence Organization, 2019) pp. 6146-6150. https://doi.org/10.24963/ijcai.2019/852.

460. J. Stypinska, Al ageism: a critical roadmap for studying age discrimination and exclusion in digitalized societies. *AI Soc.* **38**, 665-677 (2023). https://doi.org/10.1007/s00146-022-01553-5.

461. P. Rucker, M. Miller, D. Armstrong, How Cigna Saves Millions by Having Its Doctors Reject Claims Without Reading Them in *ProPublica*. (2023). https://www.propublica.org/article/cigna-pxdx-medical-health-insurance-rejection-claims.

462. A. Mack, R. Qadri, R. Denton, S. K. Kane, C. L. Bennett, ""They only care to show us the wheelchair": Disability representation in text-to-image AI models" in *Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24)* (ACM, 2024). https://dl.acm.org/doi/10.1145/3613904.3642166.

463. P. N. Venkit, M. Srinath, S. Wilson, "Automated Ableism: An Exploration of Explicit Disability Biases in Sentiment and Toxicity Analysis Models" in *Proceedings of the 3rd Workshop on Trustworthy Natural Language Processing (TrustNLP 2023)* (Association for Computational Linguistics, 2023) pp. 26-34. https://doi.org/10.18653/v1/2023.trustnlp-1.3.

464. I. A. Adeyanju, O. O. Bello, M. A. Adegboye, Machine learning methods for sign language recognition: A critical review and analysis. *Intelligent Systems with Applications* **12**, 200056 (2021). https://doi.org/10.1016/j.iswa.2021.200056.

465. S. Gueuwou, K. Takyi, M. Müller, M. S. Nyarko, R. Adade, R.-M. O. M. Gyening, "AfriSign: Machine Translation for African Sign Languages" in *4th Workshop on African Natural Language Processing (AfricaNLP 2023)* (2023). https://openreview.net/forum?id=EHldk3J2xk.

466. W. Guo, A. Caliskan, "Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biases" in *Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21)* (Association for Computing Machinery, 2021) pp. 122-133. https://doi.org/10.1145/3461702.3462536.

467. Á. A. Cabrera, W. Epperson, F. Hohman, M. Kahng, J. Morgenstern, D. H. Chau, "FAIRVIS: Visual Analytics for Discovering Intersectional Bias in Machine Learning" in *2019 IEEE Conference on Visual Analytics Science and Technology (VAST)* (2019) pp. 46-56. https://doi.org/10.1109/vast47406.2019.8986948.

468. L. Magee, L. Ghahremanlou, K. Soldatic, S. Robertson, Intersectional Bias in Causal Language Models, arXiv:2107.07691 [cs.CL] (2021). https://doi.org/10.48550/arXiv:2107.07691.

469. A. Ovalle, A. Subramonian, V. Gautam, G. Gee, K.-W. Chang, "Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairness" in *Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23)* (Association for Computing Machinery, 2023) pp. 496-511. https://doi.org/10.1145/3600211.3604705.

470. J. S. Park, M. S. Bernstein, R. N. Brewer, E. Kamar, M. R. Morris, "Understanding the Representation and Representativeness of Age in AI Data Sets" in *Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21)* (Association for Computing Machinery, 2021) pp. 834-842. https://doi.org/10.1145/3461702.3462590.

471. R. Kamikubo, L. Wang, C. Marte, A. Mahmood, H. Kacorri, "Data Representativeness in Accessibility Datasets: A Meta-Analysis" in *Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '22)* (Association for Computing Machinery, 2022) pp. 1-15. https://doi.org/10.1145/3517428.3544826.

472.* L. Weidinger, J. Mellor, M. Rauh, C. Griffin, J. Uesato, P.-S. Huang, M. Cheng, M. Glaese, B. Balle, A. Kasirzadeh, Z. Kenton, S. Brown, W. Hawkins, T. Stepleton, C. Biles, A. Birhane, J. Haas, L. Rimell, L. A. Hendricks, ... I. Gabriel, "Ethical and social risks of harm from Language Models" (Google DeepMind, 2021); http://arxiv.org/abs/2112.04359.

473. J. Nwatu, O. Ignat, R. Mihalcea, "Bridging the Digital Divide: Performance Variation across Socio-Economic Factors in Vision-Language Models" in *Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)* (Association for Computational Linguistics, 2023) pp. 10686-10702. https://doi.org/10.18653/v1/2023.emnlp-main.660.

474. S. Ghosh, A. Caliskan, "ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages" in *Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23)* (Association for Computing Machinery, 2023) pp. 901-912. https://doi.org/10.1145/3600211.3604672.
475. G. Vardi, On the Implicit Bias in Deep-Learning Algorithms. Commun. ACM 66, 86-93 (2023).
https://doi.org/10.1145/3571070.

476. F. Bianchi, P. Kalluri, E. Durmus, F. Ladhak, M. Cheng, D. Nozza, T. Hashimoto, D. Jurafsky, J. Zou, A. Caliskan, "Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale" in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23) (Association for Computing Machinery, 2023) pp. 1493-1504. https://doi.org/10.1145/3593013.3594095.

477. J. Hartmann, J. Schwenzow, M. Witte, The political ideology of conversational Al: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation, arXiv:2301.01768 [cs.CL] (2023). http://arxiv.org/abs/2301.01768.

478. T. Kaufmann, S. Ball, J. Beck, E. Hüllermeier, F. Kreuter, "On the Challenges and Practices of Reinforcement Learning from Real Human Feedback" in First Workshop on Hybrid Human-Machine Learning and Decision Making (HLDM'23) (2023).

479. A. Siththaranjan, C. Laidlaw, D. Hadfield-Menell, "Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF" in The 12th International Conference on Learning Representations (ICLR) (2024). https://openreview.net/forum?id=OtWTxYYPnW.

480.* OpenAI, ChatGPT plugins (2023). https://openai.com/blog/chatgpt-plugins.

481.* I. Gabriel, A. Manzini, G. Keeling, L. A. Hendricks, V. Rieser, H. Iqbal, N. Tomašev, I. Ktena, Z. Kenton, M. Rodriguez, S. El-Sayed, S. Brown, C. Akbulut, A. Trask, E. Hughes, A. Stevie Bergman, R. Shelby, N. Marchal, C. Griffin, ... J. Manyika, "The Ethics of Advanced Al Assistants" (Google DeepMind, 2024); http://arxiv.org/abs/2404.16244.

482. A. Chan, R. Salganik, A. Markelius, C. Pang, N. Rajkumar, D. Krasheninnikov, L. Langosco, Z. He, Y. Duan, M. Carroll, M. Lin, A. Mayhew, K. Collins, M. Molamohammadi, J. Burden, W. Zhao, S. Rismani, K. Voudouris, U. Bhatt, ... T. Maharaj, "Harms from Increasingly Agentic Algorithmic Systems" in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23) (Association for Computing Machinery, 2023) pp. 651-666.
https://doi.org/10.1145/3593013.3594033.

483. A. M. Turing, Intelligent Machinery, A Heretical Theory*. Philos. Math. 4, 256-260 (1996).
https://doi.org/10.1093/philmat/4.3.256.

484. I. J. Good, "Speculations Concerning the First Ultraintelligent Machine" in Advances in Computers, F. L. Alt, M. Rubinoff, Eds. (Elsevier, 1966), vol. 6, pp. 31-88.

485. N. Wiener, Some Moral and Technical Consequences of Automation. Science 131, 1355-1358 (1960).
https://doi.org/10.1126/science.131.3410.1355.

486. Center for Al Safety, Statement on Al Risk: Al experts and public figures express their concern about Al risk (2024). https://www.safe.ai/work/statement-on-ai-risk.

487. Y. Bengio, Written Statement of Professor Yoshua Bengio Before the US Senate Al Insight Forum. (2023).
https://www.schumer.senate.gov/imo/media/doc/Yoshua%20Benigo%20-%20Statement.pdf.

488. K. Grace, H. Stewart, J. F. Sandkühler, S. Thomas, B. Weinstein-Raun, J. Brauner, Thousands of Al Authors on the Future of Al, arXiv:2401.02843 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2401.02843.

489. A. Pan, K. Bhatia, J. Steinhardt, "The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models" in The 10th International Conference on Learning Representations (ICLR 2022) (2021).
https://openreview.net/forum?id=JYtwGwll7ye.

490. R. Ngo, L. Chan, S. Mindermann, "The Alignment Problem from a Deep Learning Perspective" in The 12th International Conference on Learning Representations (ICLR 2024) (2023). https://openreview.net/forum?id=fh8EYKFKns.

491. J. Ji, T. Qiu, B. Chen, B. Zhang, H. Lou, K. Wang, Y. Duan, Z. He, J. Zhou, Z. Zhang, F. Zeng, K. Y. Ng, J. Dai, X. Pan, A. O'Gara, Y. Lei, H. Xu, B. Tse, J. Fu, ... W. Gao, Al Alignment: A Comprehensive Survey, arXiv:2310.19852 [cs.Al] (2023).
https://doi.org/10.48550/arXiv.2310.19852.

492. D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, D. Song, "Pretrained Transformers Improve Out-of-Distribution Robustness" in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Association for Computational Linguistics, 2020) pp. 2744-2751. https://doi.org/10.18653/v1/2020.acl-main.244.

493. M. K. Cohen, M. Hutter, M. A. Osborne, Advanced artificial agents intervene in the provision of reward. AI Mag. 43, 282-293 (2022). https://doi.org/10.1002/aaai.12064.

494. T. Everitt, M. Hutter, R. Kumar, V. Krakovna, Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective. Synthese 198, 6435-6467 (2021). https://doi.org/10.1007/s11229-021-03141-4.

495.* R. Kumar, J. Uesato, R. Ngo, T. Everitt, V. Krakovna, S. Legg, REALab: An Embedded Perspective on Tampering, arXiv:2011.08820 [cs.LG] (2020). http://arxiv.org/abs/2011.08820.

496. D. Hadfield-Menell, A. D. Dragan, P. Abbeel, S. Russell, "The Off-Switch Game" in The 31st AAAI Conference on Artificial Intelligence (2017). http://aaai.org/ocs/index.php/WS/AAAIW17/paper/view/15156.
475. G. Vardi, On the Implicit Bias in Deep-Learning Algorithms. Commun. ACM 66, 86-93 (2023).
https://doi.org/10.1145/3571070.

476. F. Bianchi, P. Kalluri, E. Durmus, F. Ladhak, M. Cheng, D. Nozza, T. Hashimoto, D. Jurafsky, J. Zou, A. Caliskan, "Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale" in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23) (Association for Computing Machinery, 2023) pp. 1493-1504. https://doi.org/10.1145/3593013.3594095.

477. J. Hartmann, J. Schwenzow, M. Witte, The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientation, arXiv:2301.01768 [cs.CL] (2023). http://arxiv.org/abs/2301.01768.

478. T. Kaufmann, S. Ball, J. Beck, E. Hüllermeier, F. Kreuter, "On the Challenges and Practices of Reinforcement Learning from Real Human Feedback" in First Workshop on Hybrid Human-Machine Learning and Decision Making (HLDM'23) (2023).

479. A. Siththaranjan, C. Laidlaw, D. Hadfield-Menell, "Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF" in The 12th International Conference on Learning Representations (ICLR) (2024). https://openreview.net/forum?id=OtWTxYYPnW.

480.* OpenAI, ChatGPT plugins (2023). https://openai.com/blog/chatgpt-plugins.

481.* I. Gabriel, A. Manzini, G. Keeling, L. A. Hendricks, V. Rieser, H. Iqbal, N. Tomašev, I. Ktena, Z. Kenton, M. Rodriguez, S. El-Sayed, S. Brown, C. Akbulut, A. Trask, E. Hughes, A. Stevie Bergman, R. Shelby, N. Marchal, C. Griffin, ... J. Manyika, "The Ethics of Advanced AI Assistants" (Google DeepMind, 2024); http://arxiv.org/abs/2404.16244.

482. A. Chan, R. Salganik, A. Markelius, C. Pang, N. Rajkumar, D. Krasheninnikov, L. Langosco, Z. He, Y. Duan, M. Carroll, M. Lin, A. Mayhew, K. Collins, M. Molamohammadi, J. Burden, W. Zhao, S. Rismani, K. Voudouris, U. Bhatt, ... T. Maharaj, "Harms from Increasingly Agentic Algorithmic Systems" in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23) (Association for Computing Machinery, 2023) pp. 651-666.
https://doi.org/10.1145/3593013.3594033.

483. A. M. Turing, Intelligent Machinery, A Heretical Theory*. Philos. Math. 4, 256-260 (1996).
https://doi.org/10.1093/philmat/4.3.256.

484. I. J. Good, "Speculations Concerning the First Ultraintelligent Machine" in Advances in Computers, F. L. Alt, M. Rubinoff, Eds. (Elsevier, 1966), vol. 6, pp. 31-88.

485. N. Wiener, Some Moral and Technical Consequences of Automation. Science 131, 1355-1358 (1960).
https://doi.org/10.1126/science.131.3410.1355.

486. Center for AI Safety, Statement on AI Risk: AI experts and public figures express their concern about AI risk (2024). https://www.safe.ai/work/statement-on-ai-risk.

487. Y. Bengio, Written Statement of Professor Yoshua Bengio Before the US Senate AI Insight Forum. (2023).
https://www.schumer.senate.gov/imo/media/doc/Yoshua%20Benigo%20-%20Statement.pdf.

488. K. Grace, H. Stewart, J. F. Sandkühler, S. Thomas, B. Weinstein-Raun, J. Brauner, Thousands of AI Authors on the Future of AI, arXiv:2401.02843 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2401.02843.

489. A. Pan, K. Bhatia, J. Steinhardt, "The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models" in The 10th International Conference on Learning Representations (ICLR 2022) (2021).
https://openreview.net/forum?id=JYtwGwll7ye.

490. R. Ngo, L. Chan, S. Mindermann, "The Alignment Problem from a Deep Learning Perspective" in The 12th International Conference on Learning Representations (ICLR 2024) (2023). https://openreview.net/forum?id=fh8EYKFKns.

491. J. Ji, T. Qiu, B. Chen, B. Zhang, H. Lou, K. Wang, Y. Duan, Z. He, J. Zhou, Z. Zhang, F. Zeng, K. Y. Ng, J. Dai, X. Pan, A. O'Gara, Y. Lei, H. Xu, B. Tse, J. Fu, ... W. Gao, Al Alignment: A Comprehensive Survey, arXiv:2310.19852 [cs.AI] (2023).
https://doi.org/10.48550/arXiv.2310.19852.

492. D. Hendrycks, X. Liu, E. Wallace, A. Dziedzic, R. Krishnan, D. Song, "Pretrained Transformers Improve Out-of-Distribution Robustness" in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020) (Association for Computational Linguistics, 2020) pp. 2744-2751. https://doi.org/10.18653/v1/2020.acl-main.244.

493. M. K. Cohen, M. Hutter, M. A. Osborne, Advanced artificial agents intervene in the provision of reward. AI Mag. 43, 282-293 (2022). https://doi.org/10.1002/aaai.12064.

494. T. Everitt, M. Hutter, R. Kumar, V. Krakovna, Reward tampering problems and solutions in reinforcement learning: a causal influence diagram perspective. Synthese 198, 6435-6467 (2021). https://doi.org/10.1007/s11229-021-03141-4.

495.* R. Kumar, J. Uesato, R. Ngo, T. Everitt, V. Krakovna, S. Legg, REALab: An Embedded Perspective on Tampering, arXiv:2011.08820 [cs.LG] (2020). http://arxiv.org/abs/2011.08820.

496. D. Hadfield-Menell, A. D. Dragan, P. Abbeel, S. Russell, "The Off-Switch Game" in The 31st AAAI Conference on Artificial Intelligence (2017). http://aaai.org/ocs/index.php/WS/AAAIW17/paper/view/15156.
497.* V. Krakovna, J. Kramar, Power-seeking can be probable and predictive for trained agents, arXiv:2304.06528 [cs.Al] (2023). https://doi.org/10.48550/arXiv.2304.06528.

498. A. Turner, L. Smith, R. Shah, A. Critch, P. Tadepalli, "Optimal Policies Tend To Seek Power" in 35th Conference on Neural Information Processing Systems (NeurIPS 2021) (Curran Associates, Inc., 2021) vol. 34. https://proceedings.neurips.cc/paper/2021/hash/c26820b8a4c1b3c2aa868d6d57e14a79-Abstract.html.

499. A. Turner, P. Tadepalli, "Parametrically retargetable decision-makers tend to seek power" in Advances in Neural Information Processing Systems 35 (NeurIPS 2022) Main Conference Track (2022) vol. abs/2206.13477. https://doi.org/10.48550/arXiv.2206.13477.

500. J. D. Gallow, Instrumental divergence. Philos. Stud. (2024). https://doi.org/10.1007/s11098-024-02129-3.

501. J. Scheurer, M. Balesni, M. Hobbhahn, "Large Language Models can Strategically Deceive their Users when Put Under Pressure" in ICLR 2024 Workshop on Large Language Model (LLM) Agents (2024). https://doi.org/10.48550/arXiv.2311.07590.

502. E. Brynjolfsson, The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence. Daedalus 151, 272-287 (2022). https://doi.org/10.1162/daed_a_01915.

503. M. Kuziemski, G. Misuraca, AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings. Telecomm. Policy 44, 101976 (2020). https://doi.org/10.1016/j.telpol.2020.101976.

504. L. Mitrou, M. Janssen, E. Loukis, "Human control and discretion in AI-driven decision-making in government" in 14th International Conference on Theory and Practice of Electronic Governance (ICEGOV 2021) (ACM, 2021). https://doi.org/10.1145/3494193.3494195.

505. I. Taylor, Justice by Algorithm: The Limits of AI in Criminal Sentencing. Crim. Justice Ethics 42, 193-213 (2023). https://doi.org/10.1080/0731129x.2023.2275967.

506. J.-P. Rivera, G. Mukobi, A. Reuel, M. Lamparth, C. Smith, J. Schneider, Escalation Risks from Language Models in Military and Diplomatic Decision-Making, arXiv:2401.03408 [cs.Al] (2024). http://arxiv.org/abs/2401.03408.

507. D. Hendrycks, M. Mazeika, T. Woodside, An Overview of Catastrophic Al Risks, arXiv:2306.12001 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2306.12001.

508.* T. Shevlane, S. Farquhar, B. Garfinkel, M. Phuong, J. Whittlestone, J. Leung, D. Kokotajlo, N. Marchal, M. Anderljung, N. Kolt, L. Ho, D. Siddarth, S. Avin, W. Hawkins, B. Kim, I. Gabriel, V. Bolina, J. Clark, Y. Bengio, ... A. Dafoe, "Model evaluation for extreme risks" (Google DeepMind, 2023); http://arxiv.org/abs/2305.15324.

509. M. Kinniment, L. J. K. Sato, H. Du, B. Goodrich, M. Hasin, L. Chan, L. H. Miles, T. R. Lin, H. Wijk, J. Burget, A. Ho, E. Barnes, P. Christiano, Evaluating Language-Model Agents on Realistic Autonomous Tasks, arXiv:2312.11671 [cs.CL] (2023). http://arxiv.org/abs/2312.11671.

510. P. J. Denning, The Science of Computing: The Internet Worm. Am. Sci. 77, 126-128 (1989). http://www.jstor.org/stable/27855650.

511. A. Critch, S. Russell, TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI, arXiv:2306.06924 [cs.Al] (2023). http://arxiv.org/abs/2306.06924.

512. G. Marcus, Deep Learning: A Critical Appraisal, arXiv:1801.00631 [cs.Al] (2018). https://doi.org/10.48550/arXiv.1801.00631.

513. D. Acemoglu, D. Autor, "Skills, Tasks and Technologies: Implications for Employment and Earnings*" in Handbook of Labor Economics, D. Card, O. Ashenfelter, Eds. (Elsevier, 2011), vol. 4, pp. 1043-1171.

514. D. H. Autor, The "task approach" to labor markets: an overview. J. Labour Mark. Res. 46, 185-199 (2013). https://doi.org/10.1007/s12651-013-0128-z.

515. D. Acemoglu, P. Restrepo, Automation and New Tasks: How Technology Displaces and Reinstates Labor. J. Econ. Perspect. 33, 3-30 (2019). https://doi.org/10.1257/jep.33.2.3.

516. D. Autor, "Applying AI to Rebuild Middle Class Jobs" (National Bureau of Economic Research, 2024); https://doi.org/10.3386/w32140.

517. D. H. Autor, Why Are There Still So Many Jobs? The History and Future of Workplace Automation. J. Econ. Perspect. 29, 3-30 (2015). https://doi.org/10.1257/jep.29.3.3.

518. A. Georgieff, R. Hyee, Artificial Intelligence and Employment: New Cross-Country Evidence. Front Artif Intell 5, 832736 (2022). https://doi.org/10.3389/frai.2022.832736.

519. M. Cazzaniga, F. Jaumotte, L. Li, G. Melina, A. J. Panton, C. Pizzinelli, E. J. Rockall, M. M. Tavares, "Gen-Al: Artificial Intelligence and the Future of Work" (International Monetary Fund, 2024); https://www.imf.org/en/Publications/Staff-Discussion-Notes/Issues/2024/01/14/Gen-Al-Artificial-Intelligence-and-the-Future-of-Work-542379.

520. F. Dell'Acqua, E. McFowland, III, E. R. Mollick, H. Lifshitz-Assaf, K. Kellogg, S. Rajendran, L. Krayer, F. Candelon, K. R. Lakhani, "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge
497.* V. Krakovna, J. Kramar, Power-seeking can be probable and predictive for trained agents, arXiv:2304.06528 [cs.Al] (2023). https://doi.org/10.48550/arXiv.2304.06528.

498. A. Turner, L. Smith, R. Shah, A. Critch, P. Tadepalli, "Optimal Policies Tend To Seek Power" in 35th Conference on Neural Information Processing Systems (NeurIPS 2021) (Curran Associates, Inc., 2021) vol. 34. https://proceedings.neurips.cc/paper/2021/hash/c26820b8a4c1b3c2aa868d6d57e14a79-Abstract.html.

499. A. Turner, P. Tadepalli, "Parametrically retargetable decision-makers tend to seek power" in Advances in Neural Information Processing Systems 35 (NeurIPS 2022) Main Conference Track (2022) vol. abs/2206.13477. https://doi.org/10.48550/arXiv.2206.13477.

500. J. D. Gallow, Instrumental divergence. Philos. Stud. (2024). https://doi.org/10.1007/s11098-024-02129-3.

501. J. Scheurer, M. Balesni, M. Hobbhahn, "Large Language Models can Strategically Deceive their Users when Put Under Pressure" in ICLR 2024 Workshop on Large Language Model (LLM) Agents (2024). https://doi.org/10.48550/arXiv.2311.07590.

502. E. Brynjolfsson, The Turing Trap: The Promise & Peril of Human-Like Artificial Intelligence. Daedalus 151, 272-287 (2022). https://doi.org/10.1162/daed_a_01915.

503. M. Kuziemski, G. Misuraca, AI governance in the public sector: Three tales from the frontiers of automated decision-making in democratic settings. Telecomm. Policy 44, 101976 (2020). https://doi.org/10.1016/j.telpol.2020.101976.

504. L. Mitrou, M. Janssen, E. Loukis, "Human control and discretion in AI-driven decision-making in government" in 14th International Conference on Theory and Practice of Electronic Governance (ICEGOV 2021) (ACM, 2021). https://doi.org/10.1145/3494193.3494195.

505. I. Taylor, Justice by Algorithm: The Limits of AI in Criminal Sentencing. Crim. Justice Ethics 42, 193-213 (2023). https://doi.org/10.1080/0731129x.2023.2275967.

506. J.-P. Rivera, G. Mukobi, A. Reuel, M. Lamparth, C. Smith, J. Schneider, Escalation Risks from Language Models in Military and Diplomatic Decision-Making, arXiv:2401.03408 [cs.Al] (2024). http://arxiv.org/abs/2401.03408.

507. D. Hendrycks, M. Mazeika, T. Woodside, An Overview of Catastrophic Al Risks, arXiv:2306.12001 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2306.12001.

508.* T. Shevlane, S. Farquhar, B. Garfinkel, M. Phuong, J. Whittlestone, J. Leung, D. Kokotajlo, N. Marchal, M. Anderljung, N. Kolt, L. Ho, D. Siddarth, S. Avin, W. Hawkins, B. Kim, I. Gabriel, V. Bolina, J. Clark, Y. Bengio, ... A. Dafoe, "Model evaluation for extreme risks" (Google DeepMind, 2023); http://arxiv.org/abs/2305.15324.

509. M. Kinniment, L. J. K. Sato, H. Du, B. Goodrich, M. Hasin, L. Chan, L. H. Miles, T. R. Lin, H. Wijk, J. Burget, A. Ho, E. Barnes, P. Christiano, Evaluating Language-Model Agents on Realistic Autonomous Tasks, arXiv:2312.11671 [cs.CL] (2023). http://arxiv.org/abs/2312.11671.

510. P. J. Denning, The Science of Computing: The Internet Worm. Am. Sci. 77, 126-128 (1989). http://www.jstor.org/stable/27855650.

511. A. Critch, S. Russell, TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI, arXiv:2306.06924 [cs.Al] (2023). http://arxiv.org/abs/2306.06924.

512. G. Marcus, Deep Learning: A Critical Appraisal, arXiv:1801.00631 [cs.Al] (2018). https://doi.org/10.48550/arXiv.1801.00631.

513. D. Acemoglu, D. Autor, "Skills, Tasks and Technologies: Implications for Employment and Earnings*" in Handbook of Labor Economics, D. Card, O. Ashenfelter, Eds. (Elsevier, 2011), vol. 4, pp. 1043-1171.

514. D. H. Autor, The "task approach" to labor markets: an overview. J. Labour Mark. Res. 46, 185-199 (2013). https://doi.org/10.1007/s12651-013-0128-z.

515. D. Acemoglu, P. Restrepo, Automation and New Tasks: How Technology Displaces and Reinstates Labor. J. Econ. Perspect. 33, 3-30 (2019). https://doi.org/10.1257/jep.33.2.3.

516. D. Autor, "Applying AI to Rebuild Middle Class Jobs" (National Bureau of Economic Research, 2024); https://doi.org/10.3386/w32140.

517. D. H. Autor, Why Are There Still So Many Jobs? The History and Future of Workplace Automation. J. Econ. Perspect. 29, 3-30 (2015). https://doi.org/10.1257/jep.29.3.3.

518. A. Georgieff, R. Hyee, Artificial Intelligence and Employment: New Cross-Country Evidence. Front Artif Intell 5, 832736 (2022). https://doi.org/10.3389/frai.2022.832736.

519. M. Cazzaniga, F. Jaumotte, L. Li, G. Melina, A. J. Panton, C. Pizzinelli, E. J. Rockall, M. M. Tavares, "Gen-Al: Artificial Intelligence and the Future of Work" (International Monetary Fund, 2024); https://www.imf.org/en/Publications/Staff-Discussion-Notes/Issues/2024/01/14/Gen-Al-Artificial-Intelligence-and-the-Future-of-Work-542379.

520. F. Dell'Acqua, E. McFowland, III, E. R. Mollick, H. Lifshitz-Assaf, K. Kellogg, S. Rajendran, L. Krayer, F. Candelon, K. R. Lakhani, "Navigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge
Worker Productivity and Quality" (Harvard Business School, 2023); https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf.

521.* S. Peng, E. Kalliamvakou, P. Cihon, M. Demirer, The Impact of AI on Developer Productivity: Evidence from GitHub Copilot, arXiv:2302.06590 [cs.SE] (2023). https://doi.org/10.48550/arXiv.2302.06590.

522. E. D. Yilmaz, I. Naumovska, V. A. Aggarwal, "AI-Driven Labor Substitution: Evidence from Google Translate and ChatGPT" (INSEAD, 2023); https://sites.insead.edu/facultyresearch/research/doc.cfm?did=70468.

523. X. Hui, O. Reshef, L. Zhou, "The Short-Term Effects of Generative Artificial Intelligence on Employment: Evidence from an Online Labor Market" (CESifo Working Paper, 2023); https://www.econstor.eu/handle/10419/279352.

524. L. Belzner, T. Gabor, M. Wirsing, "Large Language Model Assisted Software Engineering: Prospects, Challenges, and a Case Study" in *Bridging the Gap Between AI and Reality (AISoLA 2023)* (Springer Nature Switzerland, 2024) pp. 355-374. https://doi.org/10.1007/978-3-031-46002-9_23.

525. J. Bessen, "Automation and Jobs: When Technology Boosts Employment" (Boston University School of Law, 2019); https://scholarship.law.bu.edu/faculty_scholarship/815.

526. D. Autor, C. Chin, A. Salomons, B. Seegmiller, "New Frontiers: The Origins and Content of New Work, 1940-2018" (National Bureau of Economic Research, 2022); https://doi.org/10.3386/w30389.

527. A. Agrawal, J. S. Gans, A. Goldfarb, Artificial Intelligence: The Ambiguous Labor Market Impact of Automating Prediction. *J. Econ. Perspect.* **33**, 31–50 (2019). https://doi.org/10.1257/jep.33.2.31.

528. B. Moll, L. Rachel, P. Restrepo, Uneven Growth: Automation's Impact on Income and Wealth Inequality. *Econometrica* **90**, 2645–2683 (2022). https://doi.org/10.3982/ecta19417.

529. M. Beraja, N. Zorzi, "Inefficient Automation" (National Bureau of Economic Research, 2022); https://doi.org/10.3386/w30154.

530. D. Acemoglu, P. Restrepo, "The Wrong Kind of AI? Artificial Intelligence and the Future of Labor Demand" (National Bureau of Economic Research, 2019); https://doi.org/10.3386/w25682.

531. S. G. Benzell, L. J. Kotlikoff, G. LaGarda, J. D. Sachs, "Robots Are Us: Some Economics of Human Replacement" (National Bureau of Economic Research, 2015); https://doi.org/10.3386/w20941.

532. A. Korinek, D. Suh, "Scenarios for the Transition to AGI" (National Bureau of Economic Research, 2024); https://doi.org/10.3386/w32255.

533. E. Ilzetzki, S. Jain, The impact of artificial intelligence on growth and employment in *VoxEU – CEPR’s policy portal*. (2023). https://cepr.org/voxeu/columns/impact-artificial-intelligence-growth-and-employment.

534. M. N. Baily, E. Brynjolfsson, A. Korinek, "Machines of mind: The case for an AI-powered productivity boom" (Brookings, 2023); https://www.brookings.edu/articles/machines-of-mind-the-case-for-an-ai-powered-productivity-boom/.

535. A. K. Agrawal, J. S. Gans, A. Goldfarb, "The Turing Transformation: Artificial Intelligence, Intelligence Augmentation, and Skill Premiums" (National Bureau of Economic Research, 2023); https://doi.org/10.3386/w31767.

536. D. Acemoglu, P. Restrepo, The Race between Man and Machine: Implications of Technology for Growth, Factor Shares, and Employment. *Am. Econ. Rev.* **108**, 1488–1542 (2018). https://doi.org/10.1257/aer.20160696.

537. D. Acemoglu, P. Restrepo, "Artificial Intelligence, Automation and Work" (National Bureau of Economic Research, 2018); https://doi.org/10.3386/w24196.

538. I. Cockburn, R. Henderson, S. Stern, "The Impact of Artificial Intelligence on Innovation" (National Bureau of Economic Research, 2018); https://doi.org/10.3386/w24449.

539. W. D. Nordhaus, Are We Approaching an Economic Singularity? Information Technology and the Future of Economic Growth. *Am. Econ. J. Macroecon.* **13**, 299–332 (2021). https://doi.org/10.1257/mac.20170105.

540. OECD, "Using AI in the workplace: Opportunities, risks and policy responses" (OECD, 2024); https://doi.org/10.1787/73d417f9-en.

541. D. Acemoglu, P. Restrepo, Tasks, Automation, and the Rise in U.S. Wage Inequality. *Econometrica* **90**, 1973–2016 (2022). https://doi.org/10.3982/ecta19815.

542. Ó. Afonso, R. Forte, Routine and non-routine sectors, tasks automation and wage polarization. *Appl. Econ.* (2023). https://www.tandfonline.com/doi/abs/10.1080/00036846.2023.2280461.

543. D. Acemoglu, J. Loebbing, "Automation and Polarization" (National Bureau of Economic Research, 2022); https://doi.org/10.3386/w30528.

544. L. Karabarbounis, "Perspectives on the Labor Share" (National Bureau of Economic Research, 2023); https://doi.org/10.3386/w31854.

545. M. Ranaldi, Income Composition Inequality. *Rev. Income Wealth* **68**, 139–160 (2022). https://doi.org/10.1111/roiw.12503.
Worker Productivity and Quality" (Harvard Business School, 2023); https://www.hbs.edu/ris/Publication%20Files/24-013_d9b45b68-9e74-42d6-a1c6-c72fb70c7282.pdf.

521.* S. Peng, E. Kalliamvakou, P. Cihon, M. Demirer, The Impact of Al on Developer Productivity: Evidence from GitHub Copilot, arXiv:2302.06590 [cs.SE] (2023). https://doi.org/10.48550/arXiv.2302.06590.

522. E. D. Yilmaz, I. Naumovska, V. A. Aggarwal, "AI-Driven Labor Substitution: Evidence from Google Translate and ChatGPT" (INSEAD, 2023); https://sites.insead.edu/facultyresearch/research/doc.cfm?did=70468.

523. X. Hui, O. Reshef, L. Zhou, "The Short-Term Effects of Generative Artificial Intelligence on Employment: Evidence from an Online Labor Market" (CESifo Working Paper, 2023); https://www.econstor.eu/handle/10419/279352.

524. L. Belzner, T. Gabor, M. Wirsing, "Large Language Model Assisted Software Engineering: Prospects, Challenges, and a Case Study" in *Bridging the Gap Between AI and Reality (AISoLA 2023)* (Springer Nature Switzerland, 2024) pp. 355-374. https://doi.org/10.1007/978-3-031-46002-9_23.

525. J. Bessen, "Automation and Jobs: When Technology Boosts Employment" (Boston University School of Law, 2019); https://scholarship.law.bu.edu/faculty_scholarship/815.

526. D. Autor, C. Chin, A. Salomons, B. Seegmiller, "New Frontiers: The Origins and Content of New Work, 1940-2018" (National Bureau of Economic Research, 2022); https://doi.org/10.3386/w30389.

527. A. Agrawal, J. S. Gans, A. Goldfarb, Artificial Intelligence: The Ambiguous Labor Market Impact of Automating Prediction. *J. Econ. Perspect.* **33**, 31–50 (2019). https://doi.org/10.1257/jep.33.2.31.

528. B. Moll, L. Rachel, P. Restrepo, Uneven Growth: Automation's Impact on Income and Wealth Inequality. *Econometrica* **90**, 2645–2683 (2022). https://doi.org/10.3982/ecta19417.

529. M. Beraja, N. Zorzi, "Inefficient Automation" (National Bureau of Economic Research, 2022); https://doi.org/10.3386/w30154.

530. D. Acemoglu, P. Restrepo, "The Wrong Kind of AI? Artificial Intelligence and the Future of Labor Demand" (National Bureau of Economic Research, 2019); https://doi.org/10.3386/w25682.

531. S. G. Benzell, L. J. Kotlikoff, G. LaGarda, J. D. Sachs, "Robots Are Us: Some Economics of Human Replacement" (National Bureau of Economic Research, 2015); https://doi.org/10.3386/w20941.

532. A. Korinek, D. Suh, "Scenarios for the Transition to AGI" (National Bureau of Economic Research, 2024); https://doi.org/10.3386/w32255.

533. E. Ilzetzki, S. Jain, The impact of artificial intelligence on growth and employment in *VoxEU – CEPR’s policy portal*. (2023). https://cepr.org/voxeu/columns/impact-artificial-intelligence-growth-and-employment.

534. M. N. Baily, E. Brynjolfsson, A. Korinek, "Machines of mind: The case for an AI-powered productivity boom" (Brookings, 2023); https://www.brookings.edu/articles/machines-of-mind-the-case-for-an-ai-powered-productivity-boom/.

535. A. K. Agrawal, J. S. Gans, A. Goldfarb, "The Turing Transformation: Artificial Intelligence, Intelligence Augmentation, and Skill Premiums" (National Bureau of Economic Research, 2023); https://doi.org/10.3386/w31767.

536. D. Acemoglu, P. Restrepo, The Race between Man and Machine: Implications of Technology for Growth, Factor Shares, and Employment. *Am. Econ. Rev.* **108**, 1488–1542 (2018). https://doi.org/10.1257/aer.20160696.

537. D. Acemoglu, P. Restrepo, "Artificial Intelligence, Automation and Work" (National Bureau of Economic Research, 2018); https://doi.org/10.3386/w24196.

538. I. Cockburn, R. Henderson, S. Stern, "The Impact of Artificial Intelligence on Innovation" (National Bureau of Economic Research, 2018); https://doi.org/10.3386/w24449.

539. W. D. Nordhaus, Are We Approaching an Economic Singularity? Information Technology and the Future of Economic Growth. *Am. Econ. J. Macroecon.* **13**, 299–332 (2021). https://doi.org/10.1257/mac.20170105.

540. OECD, "Using AI in the workplace: Opportunities, risks and policy responses" (OECD, 2024); https://doi.org/10.1787/73d417f9-en.

541. D. Acemoglu, P. Restrepo, Tasks, Automation, and the Rise in U.S. Wage Inequality. *Econometrica* **90**, 1973–2016 (2022). https://doi.org/10.3982/ecta19815.

542. Ó. Afonso, R. Forte, Routine and non-routine sectors, tasks automation and wage polarization. *Appl. Econ.* (2023). https://www.tandfonline.com/doi/abs/10.1080/00036846.2023.2280461.

543. D. Acemoglu, J. Loebbing, "Automation and Polarization" (National Bureau of Economic Research, 2022); https://doi.org/10.3386/w30528.

544. L. Karabarbounis, "Perspectives on the Labor Share" (National Bureau of Economic Research, 2023); https://doi.org/10.3386/w31854.

545. M. Ranaldi, Income Composition Inequality. *Rev. Income Wealth* **68**, 139–160 (2022). https://doi.org/10.1111/roiw.12503.
546. T. Piketty, A. Goldhammer, Capital in the twenty-first century (The Belknap Press of Harvard University Press, Cambridge Massachusetts, 2014), pp. 685.

547. A. Korinek, J. E. Stiglitz, "Artificial Intelligence, Globalization, and Strategies for Economic Development" (National Bureau of Economic Research, 2021); https://doi.org/10.3386/w28453.

548. Association for Progressive communications, Article 19, Swedish International Development Cooperation Agency, "Global Information Society Watch 2019: Artificial intelligence: Human rights, social justice and development" (APC, 2019).

549. N. Sambasivan, E. Arnesen, B. Hutchinson, T. Doshi, V. Prabhakaran, "Re-imagining Algorithmic Fairness in India and Beyond" in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21) (Association for Computing Machinery, 2021) pp. 315-328. https://doi.org/10.1145/3442188.3445896.

550. C. T. Okolo, AI in the Global South: Opportunities and challenges towards more inclusive governance (2023). https://www.brookings.edu/articles/ai-in-the-global-south-opportunities-and-challenges-towards-more-inclusive-governance/.

551. M.-T. Png, "At the Tensions of South and North: Critical Roles of Global South Stakeholders in AI Governance" in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22) (Association for Computing Machinery, 2022) pp. 1434-1445. https://doi.org/10.1145/3531146.3533200.

552. S. L. Harlan, D. N. Pellow, J. T. Roberts, S. E. Bell, W. G. Holt, J. Nagel, "Climate Justice and Inequality" in Climate Change and Society, R. E. Dunlap, R. J. Brulle, Eds. (Oxford University Press, 2015), pp. 127-163.

553. F. Sultana, Critical climate justice. Geogr. J. 188, 118-124 (2022). https://doi.org/10.1111/geoj.12417.

554. R. Zwetsloot, B. Zhang, N. Dreksler, L. Kahn, M. Anderljung, A. Dafoe, M. C. Horowitz, "Skilled and Mobile: Survey Evidence of AI Researchers' Immigration Preferences" in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) (Association for Computing Machinery, 2021) pp. 1050-1059. https://doi.org/10.1145/3461702.3462617.

555. R. Jurowetzki, D. Hain, J. Mateos-Garcia, K. Stathoulopoulos, The Privatization of AI Research(-ers): Causes and Potential Consequences -- From university-industry interaction to public research brain-drain?, arXiv:2102.01648 [cs.CY] (2021). http://arxiv.org/abs/2102.01648.

556. N. Ahmed, M. Wahed, The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research, arXiv:2010.15581 [cs.CY] (2020). https://doi.org/10.48550/arXiv.2010.15581.

557. T. Besiroglu, S. A. Bergerson, A. Michael, L. Heim, X. Luo, N. Thompson, The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?, arXiv:2401.02452 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2401.02452.

558. S. Teleanu, J. Kurbalija, "Stronger digital voices from Africa: Building African digital foreign policy and diplomacy" (Diplo, 2022); https://www.diplomacy.edu/resource/report-stronger-digital-voices-from-africa/.

559. International Telecommunication Union (ITU), "Internet use" in Measuring digital development: Facts and Figures 2023 (ITU Publications, 2023), pp. 1-2.

560. E. Panos, M. Densing, K. Volkart, Access to electricity in the World Energy Council's global energy scenarios: An outlook for developing regions until 2030. Energy Strategy Reviews 9, 28-49 (2016). https://doi.org/10.1016/j.esr.2015.11.003.

561. H. Ritchie, P. Rosado, M. Roser, Access to energy in Our World in Data. (2024). https://ourworldindata.org/energy-access.

562. N. Maslej, L. Fattorini, R. Perrault, V. Parli, A. Reuel, E. Brynjolfsson, J. Etchemendy, K. Ligett, T. Lyons, J. Manyika, J. C. Niebles, Y. Shoham, R. Wald, J. Clark, "The AI Index 2024 Annual Report" (AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, 2024); https://aiindex.stanford.edu/wp-content/uploads/2024/04/HAI_AI-Index-Report-2024.pdf.

563. N. Ahmed, M. Wahed, N. C. Thompson, The growing influence of industry in AI research. Science 379, 884-886 (2023). https://doi.org/10.1126/science.ade2420.

564.* M. Sukumaran, A. Lewis, Server Market Analysis – 2H23 (2023). https://omdia.tech.informa.com/om033795/server-market-analysis--2h23.

565. M. L. Gray, S. Suri, Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass (Houghton Mifflin Harcourt, 2019), pp. 297.

566. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, "ImageNet: A large-scale hierarchical image database" in 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009) pp. 248-255. https://doi.org/10.1109/cvpr.2009.5206848.

567. A. Arora, M. Barrett, E. Lee, E. Oborn, K. Prince, Risk and the future of AI: Algorithmic bias, data colonialism, and marginalization. Inf. Organ. 33 (2023). https://doi.org/10.1016/j.infoandorg.2023.100478.

568. C. T. Okolo, "Addressing global inequity in AI development" in Handbook of Critical Studies of Artificial Intelligence (Edward Elgar Publishing, 2023), pp. 378-389.
546. T. Piketty, A. Goldhammer, Capital in the twenty-first century (The Belknap Press of Harvard University Press, Cambridge Massachusetts, 2014), pp. 685.

547. A. Korinek, J. E. Stiglitz, "Artificial Intelligence, Globalization, and Strategies for Economic Development" (National Bureau of Economic Research, 2021); https://doi.org/10.3386/w28453.

548. Association for Progressive communications, Article 19, Swedish International Development Cooperation Agency, "Global Information Society Watch 2019: Artificial intelligence: Human rights, social justice and development" (APC, 2019).

549. N. Sambasivan, E. Arnesen, B. Hutchinson, T. Doshi, V. Prabhakaran, "Re-imagining Algorithmic Fairness in India and Beyond" in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21) (Association for Computing Machinery, 2021) pp. 315-328. https://doi.org/10.1145/3442188.3445896.

550. C. T. Okolo, AI in the Global South: Opportunities and challenges towards more inclusive governance (2023). https://www.brookings.edu/articles/ai-in-the-global-south-opportunities-and-challenges-towards-more-inclusive-governance/.

551. M.-T. Png, "At the Tensions of South and North: Critical Roles of Global South Stakeholders in AI Governance" in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT '22) (Association for Computing Machinery, 2022) pp. 1434-1445. https://doi.org/10.1145/3531146.3533200.

552. S. L. Harlan, D. N. Pellow, J. T. Roberts, S. E. Bell, W. G. Holt, J. Nagel, "Climate Justice and Inequality" in Climate Change and Society, R. E. Dunlap, R. J. Brulle, Eds. (Oxford University Press, 2015), pp. 127-163.

553. F. Sultana, Critical climate justice. Geogr. J. 188, 118-124 (2022). https://doi.org/10.1111/geoj.12417.

554. R. Zwetsloot, B. Zhang, N. Dreksler, L. Kahn, M. Anderljung, A. Dafoe, M. C. Horowitz, "Skilled and Mobile: Survey Evidence of AI Researchers' Immigration Preferences" in Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society (AIES '21) (Association for Computing Machinery, 2021) pp. 1050-1059. https://doi.org/10.1145/3461702.3462617.

555. R. Jurowetzki, D. Hain, J. Mateos-Garcia, K. Stathoulopoulos, The Privatization of AI Research(-ers): Causes and Potential Consequences -- From university-industry interaction to public research brain-drain?, arXiv:2102.01648 [cs.CY] (2021). http://arxiv.org/abs/2102.01648.

556. N. Ahmed, M. Wahed, The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research, arXiv:2010.15581 [cs.CY] (2020). https://doi.org/10.48550/arXiv.2010.15581.

557. T. Besiroglu, S. A. Bergerson, A. Michael, L. Heim, X. Luo, N. Thompson, The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?, arXiv:2401.02452 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2401.02452.

558. S. Teleanu, J. Kurbalija, "Stronger digital voices from Africa: Building African digital foreign policy and diplomacy" (Diplo, 2022); https://www.diplomacy.edu/resource/report-stronger-digital-voices-from-africa/.

559. International Telecommunication Union (ITU), "Internet use" in Measuring digital development: Facts and Figures 2023 (ITU Publications, 2023), pp. 1-2.

560. E. Panos, M. Densing, K. Volkart, Access to electricity in the World Energy Council's global energy scenarios: An outlook for developing regions until 2030. Energy Strategy Reviews 9, 28-49 (2016). https://doi.org/10.1016/j.esr.2015.11.003.

561. H. Ritchie, P. Rosado, M. Roser, Access to energy in Our World in Data. (2024). https://ourworldindata.org/energy-access.

562. N. Maslej, L. Fattorini, R. Perrault, V. Parli, A. Reuel, E. Brynjolfsson, J. Etchemendy, K. Ligett, T. Lyons, J. Manyika, J. C. Niebles, Y. Shoham, R. Wald, J. Clark, "The AI Index 2024 Annual Report" (AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, 2024); https://aiindex.stanford.edu/wp-content/uploads/2024/04/HAI_AI-Index-Report-2024.pdf.

563. N. Ahmed, M. Wahed, N. C. Thompson, The growing influence of industry in AI research. Science 379, 884-886 (2023). https://doi.org/10.1126/science.ade2420.

564.* M. Sukumaran, A. Lewis, Server Market Analysis – 2H23 (2023). https://omdia.tech.informa.com/om033795/server-market-analysis--2h23.

565. M. L. Gray, S. Suri, *Ghost Work: How to Stop Silicon Valley from Building a New Global Underclass* (Houghton Mifflin Harcourt, 2019), pp. 297.

566. J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, L. Fei-Fei, "ImageNet: A large-scale hierarchical image database" in 2009 IEEE Conference on Computer Vision and Pattern Recognition (2009) pp. 248-255. https://doi.org/10.1109/cvpr.2009.5206848.

567. A. Arora, M. Barrett, E. Lee, E. Oborn, K. Prince, Risk and the future of AI: Algorithmic bias, data colonialism, and marginalization. Inf. Organ. 33 (2023). https://doi.org/10.1016/j.infoandorg.2023.100478.

568. C. T. Okolo, "Addressing global inequity in AI development" in Handbook of Critical Studies of Artificial Intelligence (Edward Elgar Publishing, 2023), pp. 378-389.
569. D. Wang, S. Prabhat, N. Sambasivan, "Whose AI Dream? In search of the aspiration in data annotation" in CHI Conference on Human Factors in Computing Systems (CHI '22) (ACM, 2022) pp. 1-16.
https://doi.org/10.1145/3491102.3502121.

570. M. Miceli, T. Yang, A. Alvarado Garcia, J. Posada, S. M. Wang, M. Pohl, A. Hanna, Documenting Data Production Processes: A Participatory Approach for Data Work. Proc. ACM Hum. Comput. Interact. 6, 1-34 (2022).
https://doi.org/10.1145/3555623.

571. E. Brynjolfsson, A. Ng, "Big AI can centralize decision-making and power, and that's a problem" in Missing links in AI governance (UNESCO/Mila, 2023), pp. 65-87.

572. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N. Chatterji, A. Chen, K. Creel, J. Q. Davis, D. Demszky, ... P. Liang, On the Opportunities and Risks of Foundation Models, arXiv:2108.07258 [cs.LG] (2021).
https://doi.org/10.48550/arXiv.2108.07258.

573. J. Vipra, A. Korinek, "Market concentration implications of foundation models: The Invisible Hand of ChatGPT" (Brookings Institution, 2023).

574. Competition and Markets Authority, "AI Foundation Models: Initial report" (CMA, 2023); https://www.gov.uk/government/publications/ai-foundation-models-initial-report.

575. A. Goldfarb, D. Trefler, "Artificial Intelligence and International Trade" in The Economics of Artificial Intelligence: An Agenda (University of Chicago Press, 2018), pp. 463-492.

576. R. Naik, B. Nushi, "Social Biases through the Text-to-Image Generation Lens" in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23) (Association for Computing Machinery, 2023) pp. 786-808.
https://doi.org/10.1145/3600211.3604711.

577. J. Danielsson, R. Macrae, A. Uthemann, Artificial intelligence and systemic risk. J. Bank. Financ. 140, 106290 (2022). https://doi.org/10.1016/j.jbankfin.2021.106290.

578. International Energy Agency, "Tracking Clean Energy Progress 2023" (IEA, 2023); https://www.iea.org/reports/tracking-clean-energy-progress-2023.

579. European Commission Joint Research Centre, G. Kamiya, P. Bertoldi, Energy consumption in data centres and broadband communication networks in the EU (Publications Office of the European Union, 2024).

580. E. Halper, Amid explosive demand, America is running out of power in Washington Post. (2024).
https://www.washingtonpost.com/business/2024/03/07/ai-data-centers-power/.

581.* A. S. Luccioni, A. Hernandez-Garcia, Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning, arXiv:2302.08476 [cs.LG] (2023). http://arxiv.org/abs/2302.08476.

582. P. Li, J. Yang, M. A. Islam, S. Ren, Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Models, arXiv:2304.03271 [cs.LG] (2023). http://arxiv.org/abs/2304.03271.

583. D. J. Solove, Understanding privacy (Harvard University Press, London, England, ed. First, 2009), pp. 257.

584. B. Smith, Developing and deploying AI responsibly: Elements of an effective legislative framework to regulate AI. (2023). https://blogs.microsoft.com/on-the-issues/2023/09/12/developing-and-deploying-ai-responsibly-elements-of-an-effective-legislative-framework-to-regulate-ai/.

585. H. Nissenbaum, Privacy in Context: Technology, Policy, and the Integrity of Social Life (Stanford University Press, Palo Alto, CA, 2009), pp. 304.

586. L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, N. Papernot, "Machine Unlearning" in 2021 IEEE Symposium on Security and Privacy (SP) (IEEE, 2021) pp. 141-159.
https://doi.org/10.1109/sp40001.2021.00019.

587. D. J. Solove, Artificial Intelligence and Privacy. Florida Law Review (2025).
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4713111.

588. R. Shokri, M. Stronati, C. Song, V. Shmatikov, "Membership Inference Attacks Against Machine Learning Models" in 2017 IEEE Symposium on Security and Privacy (SP) (IEEE, 2017) pp. 3-18. https://doi.org/10.1109/sp.2017.41.

589. M. Fredrikson, S. Jha, T. Ristenpart, "Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures" in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS '15) (Association for Computing Machinery, 2015) pp. 1322-1333. https://doi.org/10.1145/2810103.2813677.

590. G. Chen, Y. Zhang, F. Song, "SLMIA-SR: Speaker-level membership inference attacks against speaker recognition systems" in Proceedings 2024 Network and Distributed System Security Symposium (NDSS 2024) (Internet Society, 2024). https://doi.org/10.14722/ndss.2024.241323.

591. N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, Ú. Erlingsson, A. Oprea, C. Raffel, "Extracting training data from large language models" in 30th USENIX security symposium (USENIX security 21) (USENIX Association, 2021) pp. 2633-2650.
https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting.
569. D. Wang, S. Prabhat, N. Sambasivan, "Whose AI Dream? In search of the aspiration in data annotation" in CHI Conference on Human Factors in Computing Systems (CHI '22) (ACM, 2022) pp. 1-16.
https://doi.org/10.1145/3491102.3502121.

570. M. Miceli, T. Yang, A. Alvarado Garcia, J. Posada, S. M. Wang, M. Pohl, A. Hanna, Documenting Data Production Processes: A Participatory Approach for Data Work. Proc. ACM Hum. Comput. Interact. 6, 1-34 (2022).
https://doi.org/10.1145/3555623.

571. E. Brynjolfsson, A. Ng, "Big AI can centralize decision-making and power, and that's a problem" in Missing links in AI governance (UNESCO/Mila, 2023), pp. 65-87.

572. R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, E. Brynjolfsson, S. Buch, D. Card, R. Castellon, N. Chatterji, A. Chen, K. Creel, J. Q. Davis, D. Demszky, ... P. Liang, On the Opportunities and Risks of Foundation Models, arXiv:2108.07258 [cs.LG] (2021).
https://doi.org/10.48550/arXiv.2108.07258.

573. J. Vipra, A. Korinek, "Market concentration implications of foundation models: The Invisible Hand of ChatGPT" (Brookings Institution, 2023).

574. Competition and Markets Authority, "AI Foundation Models: Initial report" (CMA, 2023);
https://www.gov.uk/government/publications/ai-foundation-models-initial-report.

575. A. Goldfarb, D. Trefler, "Artificial Intelligence and International Trade" in The Economics of Artificial Intelligence: An Agenda (University of Chicago Press, 2018), pp. 463-492.

576. R. Naik, B. Nushi, "Social Biases through the Text-to-Image Generation Lens" in Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES '23) (Association for Computing Machinery, 2023) pp. 786-808.
https://doi.org/10.1145/3600211.3604711.

577. J. Daniélsson, R. Macrae, A. Uthemann, Artificial intelligence and systemic risk. J. Bank. Financ. 140, 106290 (2022). https://doi.org/10.1016/j.jbankfin.2021.106290.

578. International Energy Agency, "Tracking Clean Energy Progress 2023" (IEA, 2023);
https://www.iea.org/reports/tracking-clean-energy-progress-2023.

579. European Commission Joint Research Centre, G. Kamiya, P. Bertoldi, Energy consumption in data centres and broadband communication networks in the EU (Publications Office of the European Union, 2024).

580. E. Halper, Amid explosive demand, America is running out of power in Washington Post. (2024).
https://www.washingtonpost.com/business/2024/03/07/ai-data-centers-power/.

581.* A. S. Luccioni, A. Hernandez-Garcia, Counting Carbon: A Survey of Factors Influencing the Emissions of Machine Learning, arXiv:2302.08476 [cs.LG] (2023). http://arxiv.org/abs/2302.08476.

582. P. Li, J. Yang, M. A. Islam, S. Ren, Making AI Less "Thirsty": Uncovering and Addressing the Secret Water Footprint of AI Models, arXiv:2304.03271 [cs.LG] (2023). http://arxiv.org/abs/2304.03271.

583. D. J. Solove, Understanding privacy (Harvard University Press, London, England, ed. First, 2009), pp. 257.

584. B. Smith, Developing and deploying AI responsibly: Elements of an effective legislative framework to regulate AI. (2023). https://blogs.microsoft.com/on-the-issues/2023/09/12/developing-and-deploying-ai-responsibly-elements-of-an-effective-legislative-framework-to-regulate-ai/.

585. H. Nissenbaum, Privacy in Context: Technology, Policy, and the Integrity of Social Life (Stanford University Press, Palo Alto, CA, 2009), pp. 304.

586. L. Bourtoule, V. Chandrasekaran, C. A. Choquette-Choo, H. Jia, A. Travers, B. Zhang, D. Lie, N. Papernot, "Machine Unlearning" in 2021 IEEE Symposium on Security and Privacy (SP) (IEEE, 2021) pp. 141-159.
https://doi.org/10.1109/sp40001.2021.00019.

587. D. J. Solove, Artificial Intelligence and Privacy. Florida Law Review (2025).
https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4713111.

588. R. Shokri, M. Stronati, C. Song, V. Shmatikov, "Membership Inference Attacks Against Machine Learning Models" in 2017 IEEE Symposium on Security and Privacy (SP) (IEEE, 2017) pp. 3-18. https://doi.org/10.1109/sp.2017.41.

589. M. Fredrikson, S. Jha, T. Ristenpart, "Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures" in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS '15) (Association for Computing Machinery, 2015) pp. 1322-1333. https://doi.org/10.1145/2810103.2813677.

590. G. Chen, Y. Zhang, F. Song, "SLMIA-SR: Speaker-level membership inference attacks against speaker recognition systems" in Proceedings 2024 Network and Distributed System Security Symposium (NDSS 2024) (Internet Society, 2024). https://doi.org/10.14722/ndss.2024.241323.

591. N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. Brown, D. Song, Ú. Erlingsson, A. Oprea, C. Raffel, "Extracting training data from large language models" in 30th USENIX security symposium (USENIX security 21) (USENIX Association, 2021) pp. 2633-2650.
https://www.usenix.org/conference/usenixsecurity21/presentation/carlini-extracting.
592. N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramèr, B. Balle, D. Ippolito, E. Wallace, "Extracting training data from diffusion models" in *32nd USENIX security symposium (USENIX security 23)* (USENIX Association, 2023) pp. 5253-5270. https://www.usenix.org/conference/usenixsecurity23/presentation/carlini.

593. W. Shi, A. Ajith, M. Xia, Y. Huang, D. Liu, T. Blevins, D. Chen, L. Zettlemoyer, "Detecting Pretraining Data from Large Language Models" in *The 12th International Conference on Learning Representations (ICLR 2024)* (2023). https://openreview.net/forum?id=zWqr3MQuNs.

594. N. Lukas, A. Salem, R. Sim, S. Tople, L. Wutschitz, S. Zanella-Béguelin, "Analyzing Leakage of Personally Identifiable Information in Language Models" in *2023 IEEE Symposium on Security and Privacy (SP)* (IEEE, 2023) pp. 346-363. https://doi.org/10.1109/sp.46215.2023.10179300.

595. N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramèr, C. Zhang, "Quantifying Memorization Across Neural Language Models" in *The 11th International Conference on Learning Representations (ICLR 2023)* (2022). https://openreview.net/forum?id=TatRHT_1cK.

596.* K. Saab, T. Tu, W.-H. Weng, R. Tanno, D. Stutz, E. Wulczyn, F. Zhang, T. Strother, C. Park, E. Vedadi, J. Z. Chaves, S.-Y. Hu, M. Schaekermann, A. Kamath, Y. Cheng, D. G. T. Barrett, C. Cheung, B. Mustafa, A. Palepu, ... V. Natarajan, "Capabilities of Gemini Models in Medicine" (Google Deepmind, 2024); http://arxiv.org/abs/2404.18416.

597. K. Hill, The Secretive Company That Might End Privacy as We Know It in *The New York Times*. (2020). https://www.nytimes.com/2020/01/18/technology/clearview-privacy-facial-recognition.html.

598. R. Staab, M. Vero, M. Balunovic, M. Vechev, "Beyond Memorization: Violating Privacy via Inference with Large Language Models" in *The 12th International Conference on Learning Representations (ICLR 2024)* (2023). https://openreview.net/forum?id=kmn0BhQk7p.

599. M. Sharma, M. Kaur, "A Review of Deepfake Technology: An Emerging AI Threat" in *Soft Computing for Security Applications* (Springer Singapore, 2022) pp. 605-619. https://doi.org/10.1007/978-981-16-5301-8_44.

600. P. Burger, The Berne Convention: Its History and Its Key Role in the Future. J.L. & Tech. **3**, 1-70 (1988). https://heinonline.org/HOL/LandingPage?handle=hein.journals/jlawtecy3&div=4&id=&page=

601. L. Patterson, Copyright in 1791: An Essay Concerning the Founers' View of the Copyright Power Granted to Congress in Article I, Section 8, Clause 8 of the US Constitution. *Emory L. J.* (2003). https://heinonline.org/hol-cgi-bin/get pdf.cgi?handle=hein.journals/emlj52&section=25.

602. The Office of the Law Revision Counsel of the United States House of Representatives, "Limitations on exclusive rights: Fair use. Sec. 107" in *United States Code, 2006 Edition, Supplement 4, Title 17 - Copyrights* (US Government Publishing Office, ed. 2010, 2010).

603. E. Parliament, D.-G. f. I. P. o. t. Union, E. Rosati, *The exception for text and data mining (TDM) in the proposed Directive on Copyright in the Digital Single Market – Technical aspects* (European Parliament, 2018).

604. Japanese Law Translation Database System, "著作权法(一部未施行) Copyright Act (Partially unenforced)" (Ministry of Justice, Japan, 2024); https://www.japaneselawtranslation.go.jp/en/laws/view/4207.

605. Israeli Ministry of Justice, Opinion: Uses of Copyrighted Materials for Machine Learning. (2022). https://www.gov.il/BlobFolder/legalinfo/machine-learning/he/18-12-2022.pdf.

606. Intellectual Property Office of Singapore, "Copyright: Factsheet on Copyright Act 2021" (IPOS, 2022); https://www.ipos.gov.sg/docs/default-source/resources-library/copyright/copyright-act-factsheet.pdf.

607. L. Soldaini, R. Kinney, A. Bhagia, D. Schwenk, D. Atkinson, R. Authur, B. Bogin, K. Chandu, J. Dumas, Y. Elazar, V. Hofmann, A. H. Jha, S. Kumar, L. Lucy, X. Lyu, N. Lambert, I. Magnusson, J. Morrison, N. Muennighoff, ... K. Lo, Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research, arXiv:2402.00159 [cs.CL] (2024). http://arxiv.org/abs/2402.00159.

608. L. Tiedrich, The AI data scraping challenge: How can we proceed responsibly? (2024). https://oecd.ai/en/wonk/data-scraping-responsibly.

609. B. L. W. Sobel, Artificial Intelligence's Fair Use Crisis. JLA **41**, 45-97 (2018). https://doi.org/10.7916/jla.v4i1.2036.

610. M. A. Lemley, B. Casey, Fair Learning. Tex. Law Rev. **99**, 743-786 (2020). https://texaslawreview.org/fair-learning/.

611. P. Samuelson, Generative AI meets copyright. *Science* **381**, 158-161 (2023). https://doi.org/10.1126/science.adi0656.

612. Tremblay v. OpenAI, Inc. (3:23-cv-03223) Document 1. (District Court, N.D. California, 2023). https://storage.courtlistener.com/recap/gov.uscourts.cand.414822/gov.uscourts.cand.414822.1.0_1.pdf.

613. D. Zhang, B. Xia, Y. Liu, X. Xu, T. Hoang, Z. Xing, M. Staples, Q. Lu, L. Zhu, "Privacy and Copyright Protection in Generative AI: A Lifecycle Perspective" in *3rd International Conference on AI Engineering - Software Engineering for AI (CAIN)* (2024). http://arxiv.org/abs/2311.18252.

614. R. Mahari, S. Longpre, "Discit ergo est: Training Data Provenance And Fair Use" in *Dynamics of Generative AI*, T. Schrepel, V. Stocker, Eds. (Network Law Review, 2023).
592. N. Carlini, J. Hayes, M. Nasr, M. Jagielski, V. Sehwag, F. Tramèr, B. Balle, D. Ippolito, E. Wallace, "Extracting training data from diffusion models" in *32nd USENIX security symposium (USENIX security 23)* (USENIX Association, 2023) pp. 5253-5270. https://www.usenix.org/conference/usenixsecurity23/presentation/carlini.

593. W. Shi, A. Ajith, M. Xia, Y. Huang, D. Liu, T. Blevins, D. Chen, L. Zettlemoyer, "Detecting Pretraining Data from Large Language Models" in *The 12th International Conference on Learning Representations (ICLR 2024)* (2023). https://openreview.net/forum?id=zWqr3MQuNs.

594. N. Lukas, A. Salem, R. Sim, S. Tople, L. Wutschitz, S. Zanella-Béguelin, "Analyzing Leakage of Personally Identifiable Information in Language Models" in *2023 IEEE Symposium on Security and Privacy (SP)* (IEEE, 2023) pp. 346-363. https://doi.org/10.1109/sp.46215.2023.10179300.

595. N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramèr, C. Zhang, "Quantifying Memorization Across Neural Language Models" in *The 11th International Conference on Learning Representations (ICLR 2023)* (2022). https://openreview.net/forum?id=TatRHT_1cK.

596.* K. Saab, T. Tu, W.-H. Weng, R. Tanno, D. Stutz, E. Wulczyn, F. Zhang, T. Strother, C. Park, E. Vedadi, J. Z. Chaves, S.-Y. Hu, M. Schaekermann, A. Kamath, Y. Cheng, D. G. T. Barrett, C. Cheung, B. Mustafa, A. Palepu, ... V. Natarajan, "Capabilities of Gemini Models in Medicine" (Google Deepmind, 2024); http://arxiv.org/abs/2404.18416.

597. K. Hill, The Secretive Company That Might End Privacy as We Know It in *The New York Times*. (2020). https://www.nytimes.com/2020/01/18/technology/clearview-privacy-facial-recognition.html.

598. R. Staab, M. Vero, M. Balunovic, M. Vechev, "Beyond Memorization: Violating Privacy via Inference with Large Language Models" in *The 12th International Conference on Learning Representations (ICLR 2024)* (2023). https://openreview.net/forum?id=kmn0BhQk7p.

599. M. Sharma, M. Kaur, "A Review of Deepfake Technology: An Emerging AI Threat" in *Soft Computing for Security Applications* (Springer Singapore, 2022) pp. 605-619. https://doi.org/10.1007/978-981-16-5301-8_44.

600. P. Burger, The Berne Convention: Its History and Its Key Role in the Future. J.L. & Tech. 3, 1-70 (1988). https://heinonline.org/HOL/LandingPage?handle=hein.journals/jlawtecy3&div=4&id=&page=

601. L. Patterson, Copyright in 1791: An Essay Concerning the Founers' View of the Copyright Power Granted to Congress in Article I, Section 8, Clause 8 of the US Constitution. *Emory L. J.* (2003). https://heinonline.org/hol-cgi-bin/get pdf.cgi?handle=hein.journals/emlj52&section=25.

602. The Office of the Law Revision Counsel of the United States House of Representatives, "Limitations on exclusive rights: Fair use. Sec. 107" in *United States Code, 2006 Edition, Supplement 4, Title 17 - Copyrights* (US Government Publishing Office, ed. 2010, 2010).

603. E. Parliament, D.-G. f. I. P. o. t. Union, E. Rosati, *The exception for text and data mining (TDM) in the proposed Directive on Copyright in the Digital Single Market - Technical aspects* (European Parliament, 2018).

604. Japanese Law Translation Database System, "著作权法(一部未施行) Copyright Act (Partially unenforced)" (Ministry of Justice, Japan, 2024); https://www.japaneselawtranslation.go.jp/en/laws/view/4207.

605. Israeli Ministry of Justice, Opinion: Uses of Copyrighted Materials for Machine Learning. (2022). https://www.gov.il/BlobFolder/legalinfo/machine-learning/he/18-12-2022.pdf.

606. Intellectual Property Office of Singapore, "Copyright: Factsheet on Copyright Act 2021" (IPOS, 2022); https://www.ipos.gov.sg/docs/default-source/resources-library/copyright/copyright-act-factsheet.pdf.

607. L. Soldaini, R. Kinney, A. Bhagia, D. Schwenk, D. Atkinson, R. Authur, B. Bogin, K. Chandu, J. Dumas, Y. Elazar, V. Hofmann, A. H. Jha, S. Kumar, L. Lucy, X. Lyu, N. Lambert, I. Magnusson, J. Morrison, N. Muennighoff, ... K. Lo, Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research, arXiv:2402.00159 [cs.CL] (2024). http://arxiv.org/abs/2402.00159.

608. L. Tiedrich, The AI data scraping challenge: How can we proceed responsibly? (2024). https://oecd.ai/en/wonk/data-scraping-responsibly.

609. B. L. W. Sobel, Artificial Intelligence's Fair Use Crisis. JLA 41, 45-97 (2018). https://doi.org/10.7916/jla.v4i1.2036.

610. M. A. Lemley, B. Casey, Fair Learning. Tex. Law Rev. 99, 743-786 (2020). https://texaslawreview.org/fair-learning/.

611. P. Samuelson, Generative AI meets copyright. *Science* 381, 158-161 (2023). https://doi.org/10.1126/science.adi0656.

612. Tremblay v. OpenAI, Inc. (3:23-cv-03223) Document 1. (District Court, N.D. California, 2023). https://storage.courtlistener.com/recap/gov.uscourts.cand.414822/gov.uscourts.cand.414822.1.0_1.pdf.

613. D. Zhang, B. Xia, Y. Liu, X. Xu, T. Hoang, Z. Xing, M. Staples, Q. Lu, L. Zhu, "Privacy and Copyright Protection in Generative AI: A Lifecycle Perspective" in *3rd International Conference on AI Engineering - Software Engineering for AI (CAIN)* (2024). http://arxiv.org/abs/2311.18252.

614. R. Mahari, S. Longpre, "Discit ergo est: Training Data Provenance And Fair Use" in *Dynamics of Generative AI*, T. Schrepel, V. Stocker, Eds. (Network Law Review, 2023).
615. K. Lee, A. F. Cooper, J. Grimmelmann, "Talkin' 'Bout AI Generation: Copyright and the Generative-Al Supply Chain (The Short Version)" in Proceedings of the Symposium on Computer Science and Law (CSLAW '24) (Association for Computing Machinery, 2024) pp. 48-63. https://doi.org/10.1145/3614407.3643696.

616. J. Grimmelmann, Copyright for Literate Robots. Iowa Law Rev. 101, 657-682 (2015).
https://scholarship.law.cornell.edu/cgi/viewcontent.cgi?article=2615&context=facpub.

617. K. Lee, A. F. Cooper, J. Grimmelmann, D. Ippolito, Al and Law: The Next Generation. (2023).
https://doi.org/10.2139/ssrn.4580739.

618. L. Tiedrich, When Al generates work, standard contractual terms can help generate value and clarity in OECD. Al Policy Observatory. (2024). https://oecd.ai/en/wonk/contractual-terms.

619. M. Sag, Copyright Safety for Generative Al. Houst. Law Rev. 61, 295-347 (2023).
https://houstonlawreview.org/article/92126-copyright-safety-for-generative-al.

620. N. Vyas, S. M. Kakade, B. Barak, "On Provable Copyright Protection for Generative Models" in Proceedings of the 40th International Conference on Machine Learning (ICML 2023) (PMLR, 2023).
https://proceedings.mlr.press/v202/vyas23b.html.

621. K. McElheran, J. F. Li, E. Brynjolfsson, Z. Kroff, E. Dinlersoz, L. Foster, N. Zolas, Al adoption in America: Who, what, and where. J. Econ. Manag. Strategy 33, 375-415 (2024). https://doi.org/10.1111/jems.12576.

622. R. Mahari, L. Shayne, L. Donewald, A. Polozov, A. Pentland, A. Lipsitz, Comment to US Copyright Office on Data Provenance and Copyright. (US Copyright Office, 2023). https://dspace.mit.edu/handle/1721.1/154171?show=full?show=full.

623. S. Min, S. Gururangan, E. Wallace, W. Shi, H. Hajishirzi, N. A. Smith, L. Zettlemoyer, "SILO Language Models: Isolating Legal Risk In A Nonparametric Datastore" in NeurIPS 2023 Workshop on Distribution Shifts (DistShift) (2023).
https://openreview.net/forum?id=z03bWOdoni.

624. S. Longpre, R. Mahari, N. Obeng-Marnu, W. Brannon, T. South, J. Kabbara, S. Pentland, Data authenticity, consent, and provenance for Al are all broken: What will it take to fix them? An MIT Exploration of Generative AI (2024).
https://doi.org/10.21428/e4baedd9.a650f77d.

625. S. G. Patil, T. Zhang, V. Fang, N. C., R. Huang, A. Hao, M. Casado, J. E. Gonzalez, R. A. Popa, I. Stoica, GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications, arXiv:2404.06921 [cs.CL] (2024).
https://doi.org/10.48550/arXiv.2404.06921.

626. D. Dalrymple, J. Skalse, Y. Bengio, S. Russell, M. Tegmark, S. Seshia, S. Omohundro, C. Szegedy, B. Goldhaber, N. Ammann, A. Abate, J. Halpern, C. Barrett, D. Zhao, T. Zhi-Xuan, J. Wing, J. Tenenbaum, Towards Guaranteed Safe Al: A Framework for Ensuring Robust and Reliable Al Systems, arXiv:2405.06624 [cs.AI] (2024). http://arxiv.org/abs/2405.06624.

627. N. Cammarata, S. Carter, G. Goh, C. Olah, M. Petrov, L. Schubert, Thread: Circuits. Distill 5, 10.23915/distill.00024 (2020). https://doi.org/10.23915/distill.00024.

628. N. Nanda, L. Chan, T. Lieberum, J. Smith, J. Steinhardt, "Progress measures for grokking via mechanistic interpretability" in The 11th International Conference on Learning Representations (ICLR 2023) (2022).
https://openreview.net/forum?id=9XFSbDPmdW.

629. Z. Zhong, Z. Liu, M. Tegmark, J. Andreas, "The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks" in 37th Conference on Neural Information Processing Systems (NeurIPS 2023) (2023).
https://openreview.net/forum?id=S5wmbQc1We.

630. E. J. Michaud, I. Liao, V. Lad, Z. Liu, A. Mudide, C. Loughridge, Z. C. Guo, T. R. Kheirkhah, M. Vukelić, M. Tegmark, Opening the Al black box: program synthesis via mechanistic interpretability, arXiv:2402.05110 [cs.LG] (2024).
https://doi.org/10.48550/arXiv.2402.05110.

631. R. Huben, H. Cunningham, L. R. Smith, A. Ewart, L. Sharkey, "Sparse Autoencoders Find Highly Interpretable Features in Language Models" in The 12th International Conference on Learning Representations (ICLR 2024) (2023).
https://openreview.net/forum?id=F76bwRSLeK.

632.* T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, ... C. Olah, Towards monosemanticity: Decomposing language models with dictionary learning, Transformer Circuits Thread (2023).
https://transformer-circuits.pub/2023/monosemantic-features.

633. L. Gao, J. Schulman, J. Hilton, "Scaling Laws for Reward Model Overoptimization" in Proceedings of the 40th International Conference on Machine Learning (PMLR, 2023) pp. 10835-10866.
https://proceedings.mlr.press/v202/gao23h.html.

634. J. M. V. Skalse, N. H. R. Howe, D. Krasheninnikov, D. Krueger, "Defining and Characterizing Reward Gaming" in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022).
https://openreview.net/forum?id=yb3HOXO3IX2.

635. P. Singhal, T. Goyal, J. Xu, G. Durrett, A Long Way to Go: Investigating Length Correlations in RLHF, arXiv:2310.03716 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2310.03716.
615. K. Lee, A. F. Cooper, J. Grimmelmann, "Talkin' 'Bout AI Generation: Copyright and the Generative-Al Supply Chain (The Short Version)" in Proceedings of the Symposium on Computer Science and Law (CSLAW '24) (Association for Computing Machinery, 2024) pp. 48-63. https://doi.org/10.1145/3614407.3643696.

616. J. Grimmelmann, Copyright for Literate Robots. Iowa Law Rev. 101, 657-682 (2015).
https://scholarship.law.cornell.edu/cgi/viewcontent.cgi?article=2615&context=facpub.

617. K. Lee, A. F. Cooper, J. Grimmelmann, D. Ippolito, Al and Law: The Next Generation. (2023).
https://doi.org/10.2139/ssrn.4580739.

618. L. Tiedrich, When Al generates work, standard contractual terms can help generate value and clarity in OECD. Al Policy Observatory. (2024). https://oecd.ai/en/wonk/contractual-terms.

619. M. Sag, Copyright Safety for Generative Al. Houst. Law Rev. 61, 295-347 (2023).
https://houstonlawreview.org/article/92126-copyright-safety-for-generative-al.

620. N. Vyas, S. M. Kakade, B. Barak, "On Provable Copyright Protection for Generative Models" in Proceedings of the 40th International Conference on Machine Learning (ICML 2023) (PMLR, 2023).
https://proceedings.mlr.press/v202/vyas23b.html.

621. K. McElheran, J. F. Li, E. Brynjolfsson, Z. Kroff, E. Dinlersoz, L. Foster, N. Zolas, Al adoption in America: Who, what, and where. J. Econ. Manag. Strategy 33, 375-415 (2024). https://doi.org/10.1111/jems.12576.

622. R. Mahari, L. Shayne, L. Donewald, A. Polozov, A. Pentland, A. Lipsitz, Comment to US Copyright Office on Data Provenance and Copyright. (US Copyright Office, 2023). https://dspace.mit.edu/handle/1721.1/154171?show=full?show=full.

623. S. Min, S. Gururangan, E. Wallace, W. Shi, H. Hajishirzi, N. A. Smith, L. Zettlemoyer, "SILO Language Models: Isolating Legal Risk In A Nonparametric Datastore" in NeurIPS 2023 Workshop on Distribution Shifts (DistShift) (2023).
https://openreview.net/forum?id=z03bWOdoni.

624. S. Longpre, R. Mahari, N. Obeng-Marnu, W. Brannon, T. South, J. Kabbara, S. Pentland, Data authenticity, consent, and provenance for Al are all broken: What will it take to fix them? An MIT Exploration of Generative AI (2024).
https://doi.org/10.21428/e4baedd9.a650f77d.

625. S. G. Patil, T. Zhang, V. Fang, N. C., R. Huang, A. Hao, M. Casado, J. E. Gonzalez, R. A. Popa, I. Stoica, GoEX: Perspectives and Designs Towards a Runtime for Autonomous LLM Applications, arXiv:2404.06921 [cs.CL] (2024).
https://doi.org/10.48550/arXiv.2404.06921.

626. D. Dalrymple, J. Skalse, Y. Bengio, S. Russell, M. Tegmark, S. Seshia, S. Omohundro, C. Szegedy, B. Goldhaber, N. Ammann, A. Abate, J. Halpern, C. Barrett, D. Zhao, T. Zhi-Xuan, J. Wing, J. Tenenbaum, Towards Guaranteed Safe Al: A Framework for Ensuring Robust and Reliable Al Systems, arXiv:2405.06624 [cs.AI] (2024). http://arxiv.org/abs/2405.06624.

627. N. Cammarata, S. Carter, G. Goh, C. Olah, M. Petrov, L. Schubert, Thread: Circuits. Distill 5, 10.23915/distill.00024 (2020). https://doi.org/10.23915/distill.00024.

628. N. Nanda, L. Chan, T. Lieberum, J. Smith, J. Steinhardt, "Progress measures for grokking via mechanistic interpretability" in The 11th International Conference on Learning Representations (ICLR 2023) (2022).
https://openreview.net/forum?id=9XFSbDPmdW.

629. Z. Zhong, Z. Liu, M. Tegmark, J. Andreas, "The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks" in 37th Conference on Neural Information Processing Systems (NeurIPS 2023) (2023).
https://openreview.net/forum?id=S5wmbQc1We.

630. E. J. Michaud, I. Liao, V. Lad, Z. Liu, A. Mudide, C. Loughridge, Z. C. Guo, T. R. Kheirkhah, M. Vukelić, M. Tegmark, Opening the Al black box: program synthesis via mechanistic interpretability, arXiv:2402.05110 [cs.LG] (2024).
https://doi.org/10.48550/arXiv.2402.05110.

631. R. Huben, H. Cunningham, L. R. Smith, A. Ewart, L. Sharkey, "Sparse Autoencoders Find Highly Interpretable Features in Language Models" in The 12th International Conference on Learning Representations (ICLR 2024) (2023).
https://openreview.net/forum?id=F76bwRSLeK.

632.* T. Bricken, A. Templeton, J. Batson, B. Chen, A. Jermyn, T. Conerly, N. Turner, C. Anil, C. Denison, A. Askell, R. Lasenby, Y. Wu, S. Kravec, N. Schiefer, T. Maxwell, N. Joseph, Z. Hatfield-Dodds, A. Tamkin, K. Nguyen, ... C. Olah, Towards monosemanticity: Decomposing language models with dictionary learning, Transformer Circuits Thread (2023).
https://transformer-circuits.pub/2023/monosemantic-features.

633. L. Gao, J. Schulman, J. Hilton, "Scaling Laws for Reward Model Overoptimization" in Proceedings of the 40th International Conference on Machine Learning (PMLR, 2023) pp. 10835-10866.
https://proceedings.mlr.press/v202/gao23h.html.

634. J. M. V. Skalse, N. H. R. Howe, D. Krasheninnikov, D. Krueger, "Defining and Characterizing Reward Gaming" in 36th Conference on Neural Information Processing Systems (NeurIPS 2022) (2022).
https://openreview.net/forum?id=yb3HOXO3IX2.

635. P. Singhal, T. Goyal, J. Xu, G. Durrett, A Long Way to Go: Investigating Length Correlations in RLHF, arXiv:2310.03716 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2310.03716.
636. J. Tien, J. Z.-Y. He, Z. Erickson, A. Dragan, D. S. Brown, "Causal Confusion and Reward Misidentification in Preference-Based Reward Learning" in *The 11th International Conference on Learning Representations (ICLR 2023)* (2022). https://openreview.net/forum?id=ROXxvr_X3ZA.

637. L. E. McKinney, Y. Duan, D. Krueger, A. Gleave, "On The Fragility of Learned Reward Functions" in *36th Conference on Neural Information Processing Systems (NeurIPS 2022) Deep Reinforcement Learning Workshop* (2022). https://openreview.net/forum?id=9gj9vXfeS-y.

638. L. L. D. Langosco, J. Koch, L. D. Sharkey, J. Pfau, D. Krueger, "Goal Misgeneralization in Deep Reinforcement Learning" in *Proceedings of the 39th International Conference on Machine Learning* (PMLR, 2022) vol. 162, pp. 12004-12019. https://proceedings.mlr.press/v162/langosco22a.html.

639. E. Seger, N. Dreksler, R. Moulange, E. Dardaman, J. Schuett, K. Wei, C. Winter, M. Arnold, S. Ó. hÉigeartaigh, A. Korinek, M. Anderljung, B. Bucknall, A. Chan, E. Stafford, L. Koessler, A. Ovadya, B. Garfinkel, E. Bluemke, M. Aird, ... A. Gupta, "Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives" (Centre for the Governance of AI, 2023); http://arxiv.org/abs/2311.09227.

640. S. Kapoor, R. Bommasani, K. Klyman, S. Longpre, A. Ramaswami, P. Cihon, A. Hopkins, K. Bankston, S. Biderman, M. Bogen, R. Chowdhury, A. Engler, P. Henderson, Y. Jernite, S. Lazar, S. Maffulli, A. Nelson, J. Pineau, A. Skowron, ... A. Narayanan, On the Societal Impact of Open Foundation Models, arXiv:2403.07918 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2403.07918.

641. K. Hu, ChatGPT sets record for fastest-growing user base - analyst note in *Reuters*. (2023). https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/.

642.* M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. Feder Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, K. Lee, Scalable Extraction of Training Data from (Production) Language Models, arXiv:2311.17035 [cs.LG] (2023). https://doi.org/10.48550/arXiv.2311.17035.

643. H. Li, D. Guo, W. Fan, M. Xu, J. Huang, F. Meng, Y. Song, "Multi-step Jailbreaking Privacy Attacks on ChatGPT" in *The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)* (2023). https://openreview.net/forum?id=Is4PfsI2jZ.

644. A. Deshpande, V. Murahari, T. Rajpurohit, A. Kalyan, K. Narasimhan, "Toxicity in chatgpt: Analyzing persona-assigned language models" in *Findings of the Association for Computational Linguistics: EMNLP 2023* (Association for Computational Linguistics, 2023) pp. 1236-1270. https://doi.org/10.18653/v1/2023.findings-emnlp.88.

645. V. Hofmann, P. R. Kalluri, D. Jurafsky, S. King, Dialect prejudice predicts AI decisions about people's character, employability, and criminality, arXiv:2403.00742 [cs.CL] (2024). https://doi.org/10.48550/arXiv.2403.00742.

646. E. Choi, Y. Jo, J. Jang, J. Jang, M. Seo, "Fixed Input Parameterization for Efficient Prompting" in *Findings of the Association for Computational Linguistics: ACL 2023* (Association for Computational Linguistics, 2023) pp. 8428-8441. https://doi.org/10.18653/v1/2023.findings-acl.533.

647. E. Shayegani, M. A. Al Mamun, Y. Fu, P. Zaree, Y. Dong, N. Abu-Ghazaleh, Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks, arXiv:2310.10844 [cs.CL] (2023). http://arxiv.org/abs/2310.10844.

648.* C. Anil, E. Durmus, M. Sharma, J. Benton, S. Kundu, J. Batson, N. Rimsky, M. Tong, J. Mu, D. Ford, F. Mosconi, R. Agrawal, R. Schaeffer, N. Bashkansky, S. Svenningsen, M. Lambert, A. Radhakrishnan, C. Denison, E. J. Hubinger, ... D. Duvenaud, "Many-shot Jailbreaking" (Anthropic, 2024).

649. Y. M. Pa Pa, S. Tanizaki, T. Kou, M. van Eeten, K. Yoshioka, T. Matsumoto, "An Attacker's Dream? Exploring the Capabilities of ChatGPT for Developing Malware" in *Proceedings of the 16th Cyber Security Experimentation and Test Workshop (CSET '23)* (Association for Computing Machinery, 2023) pp. 10-18. https://doi.org/10.1145/3607505.3607513.

650. Q. Zhan, R. Fang, R. Bindu, A. Gupta, T. Hashimoto, D. Kang, "Removing RLHF Protections in GPT-4 via Fine-Tuning" in *2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics* (2024). https://doi.org/10.48550/arXiv.2311.05553.

651. X. Qi, Y. Zeng, T. Xie, P.-Y. Chen, R. Jia, P. Mittal, P. Henderson, "Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!" in *The 12th International Conference on Learning Representations (ICLR 2024)* (2023). https://openreview.net/forum?id=hTEGyKfOdZ.

652. Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, R. Zheng, X. Fan, X. Wang, L. Xiong, Y. Zhou, W. Wang, C. Jiang, Y. Zou, X. Liu, ... T. Gui, The Rise and Potential of Large Language Model Based Agents: A Survey, arXiv:2309.07864 [cs.AI] (2023). https://doi.org/10.48550/arXiv.2309.07864.

653. Y. Tian, X. Yang, J. Zhang, Y. Dong, H. Su, Evil Geniuses: Delving into the Safety of LLM-based Agents, arXiv:2311.11855 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2311.11855.

654. Z. Wu, C. Han, Z. Ding, Z. Weng, Z. Liu, S. Yao, T. Yu, L. Kong, OS-Copilot: Towards Generalist Computer Agents with Self-Improvement, arXiv:2402.07456 [cs.AI] (2024). http://arxiv.org/abs/2402.07456.

655.* SIMA Team, M. A. Raad, A. Ahuja, C. Barros, F. Besse, A. Bolt, A. Bolton, B. Brownfield, G. Buttimore, M. Cant, S. Chakera, S. C. Y. Chan, J. Clune, A. Collister, V. Copeman, A. Cullum, I. Dasgupta, D. de Cesare, J. Di Trapani, ... N. Young, "Scaling Instructable Agents Across Many Simulated Worlds" (Google Deepmind, 2024); http://arxiv.org/abs/2404.10179.
636. J. Tien, J. Z.-Y. He, Z. Erickson, A. Dragan, D. S. Brown, "Causal Confusion and Reward Misidentification in Preference-Based Reward Learning" in *The 11th International Conference on Learning Representations (ICLR 2023)* (2022).
https://openreview.net/forum?id=ROXxvr_X3ZA.

637. L. E. McKinney, Y. Duan, D. Krueger, A. Gleave, "On The Fragility of Learned Reward Functions" in *36th Conference on Neural Information Processing Systems (NeurIPS 2022) Deep Reinforcement Learning Workshop* (2022).
https://openreview.net/forum?id=9gj9vXfeS-y.

638. L. L. D. Langosco, J. Koch, L. D. Sharkey, J. Pfau, D. Krueger, "Goal Misgeneralization in Deep Reinforcement Learning" in *Proceedings of the 39th International Conference on Machine Learning* (PMLR, 2022) vol. 162, pp. 12004-12019.
https://proceedings.mlr.press/v162/langosco22a.html.

639. E. Seger, N. Dreksler, R. Moulange, E. Dardaman, J. Schuett, K. Wei, C. Winter, M. Arnold, S. Ó. hÉigeartaigh, A. Korinek, M. Anderljung, B. Bucknall, A. Chan, E. Stafford, L. Koessler, A. Ovadya, B. Garfinkel, E. Bluemke, M. Aird, ... A. Gupta, "Open-Sourcing Highly Capable Foundation Models: An evaluation of risks, benefits, and alternative methods for pursuing open-source objectives" (Centre for the Governance of AI, 2023); http://arxiv.org/abs/2311.09227.

640. S. Kapoor, R. Bommasani, K. Klyman, S. Longpre, A. Ramaswami, P. Cihon, A. Hopkins, K. Bankston, S. Biderman, M. Bogen, R. Chowdhury, A. Engler, P. Henderson, Y. Jernite, S. Lazar, S. Maffulli, A. Nelson, J. Pineau, A. Skowron, ... A. Narayanan, On the Societal Impact of Open Foundation Models, arXiv:2403.07918 [cs.CY] (2024).
https://doi.org/10.48550/arXiv.2403.07918.

641. K. Hu, ChatGPT sets record for fastest-growing user base - analyst note in *Reuters*. (2023).
https://www.reuters.com/technology/chatgpt-sets-record-fastest-growing-user-base-analyst-note-2023-02-01/.

642.* M. Nasr, N. Carlini, J. Hayase, M. Jagielski, A. Feder Cooper, D. Ippolito, C. A. Choquette-Choo, E. Wallace, F. Tramèr, K. Lee, Scalable Extraction of Training Data from (Production) Language Models, arXiv:2311.17035 [cs.LG] (2023).
https://doi.org/10.48550/arXiv.2311.17035.

643. H. Li, D. Guo, W. Fan, M. Xu, J. Huang, F. Meng, Y. Song, "Multi-step Jailbreaking Privacy Attacks on ChatGPT" in *The 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023)* (2023).
https://openreview.net/forum?id=ls4PfsI2jZ.

644. A. Deshpande, V. Murahari, T. Rajpurohit, A. Kalyan, K. Narasimhan, "Toxicity in chatgpt: Analyzing persona-assigned language models" in *Findings of the Association for Computational Linguistics: EMNLP 2023* (Association for Computational Linguistics, 2023) pp. 1236-1270.
https://doi.org/10.18653/v1/2023.findings-emnlp.88.

645. V. Hofmann, P. R. Kalluri, D. Jurafsky, S. King, Dialect prejudice predicts AI decisions about people's character, employability, and criminality, arXiv:2403.00742 [cs.CL] (2024).
https://doi.org/10.48550/arXiv.2403.00742.

646. E. Choi, Y. Jo, J. Jang, J. Jang, M. Seo, "Fixed Input Parameterization for Efficient Prompting" in *Findings of the Association for Computational Linguistics: ACL 2023* (Association for Computational Linguistics, 2023) pp. 8428-8441.
https://doi.org/10.18653/v1/2023.findings-acl.533.

647. E. Shayegani, M. A. Al Mamun, Y. Fu, P. Zaree, Y. Dong, N. Abu-Ghazaleh, Survey of Vulnerabilities in Large Language Models Revealed by Adversarial Attacks, arXiv:2310.10844 [cs.CL] (2023).
http://arxiv.org/abs/2310.10844.

648.* C. Anil, E. Durmus, M. Sharma, J. Benton, S. Kundu, J. Batson, N. Rimsky, M. Tong, J. Mu, D. Ford, F. Mosconi, R. Agrawal, R. Schaeffer, N. Bashkansky, S. Svenningsen, M. Lambert, A. Radhakrishnan, C. Denison, E. J. Hubinger, ... D. Duvenaud, "Many-shot Jailbreaking" (Anthropic, 2024).

649. Y. M. Pa Pa, S. Tanizaki, T. Kou, M. van Eeten, K. Yoshioka, T. Matsumoto, "An Attacker's Dream? Exploring the Capabilities of ChatGPT for Developing Malware" in *Proceedings of the 16th Cyber Security Experimentation and Test Workshop (CSET '23)* (Association for Computing Machinery, 2023) pp. 10-18.
https://doi.org/10.1145/3607505.3607513.

650. Q. Zhan, R. Fang, R. Bindu, A. Gupta, T. Hashimoto, D. Kang, "Removing RLHF Protections in GPT-4 via Fine-Tuning" in *2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics* (2024).
https://doi.org/10.48550/arXiv.2311.05553.

651. X. Qi, Y. Zeng, T. Xie, P.-Y. Chen, R. Jia, P. Mittal, P. Henderson, "Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!" in *The 12th International Conference on Learning Representations (ICLR 2024)* (2023).
https://openreview.net/forum?id=hTEGyKfOdZ.

652. Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, R. Zheng, X. Fan, X. Wang, L. Xiong, Y. Zhou, W. Wang, C. Jiang, Y. Zou, X. Liu, ... T. Gui, The Rise and Potential of Large Language Model Based Agents: A Survey, arXiv:2309.07864 [cs.AI] (2023).
https://doi.org/10.48550/arXiv.2309.07864.

653. Y. Tian, X. Yang, J. Zhang, Y. Dong, H. Su, Evil Geniuses: Delving into the Safety of LLM-based Agents, arXiv:2311.11855 [cs.CL] (2023).
https://doi.org/10.48550/arXiv.2311.11855.

654. Z. Wu, C. Han, Z. Ding, Z. Weng, Z. Liu, S. Yao, T. Yu, L. Kong, OS-Copilot: Towards Generalist Computer Agents with Self-Improvement, arXiv:2402.07456 [cs.AI] (2024).
http://arxiv.org/abs/2402.07456.

655.* SIMA Team, M. A. Raad, A. Ahuja, C. Barros, F. Besse, A. Bolt, A. Bolton, B. Brownfield, G. Buttimore, M. Cant, S. Chakera, S. C. Y. Chan, J. Clune, A. Collister, V. Copeman, A. Cullum, I. Dasgupta, D. de Cesare, J. Di Trapani, ... N. Young, "Scaling Instructable Agents Across Many Simulated Worlds" (Google Deepmind, 2024); http://arxiv.org/abs/2404.10179.
656. Q. Lu, L. Zhu, X. Xu, Z. Xing, S. Harrer, J. Whittle, Towards Responsible Generative Al: A Reference Architecture for Designing Foundation Model based Agents, arXiv:2311.13148 [cs.Al] (2023). http://arxiv.org/abs/2311.13148.

657.* R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, X. Jiang, K. Cobbe, T. Eloundou, G. Krueger, K. Button, M. Knight, B. Chess, J. Schulman, "WebGPT: Browser-assisted question-answering with human feedback" (OpenAI, 2021); http://arxiv.org/abs/2112.09332.

658. A. Chan, C. Ezell, M. Kaufmann, K. Wei, L. Hammond, H. Bradley, E. Bluemke, N. Rajkumar, D. Krueger, N. Kolt, L. Heim, M. Anderljung, Visibility into Al Agents, arXiv:2401.13138 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2401.13138.

659. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, . . . D. Hassabis, Mastering the game of Go with deep neural networks and tree search. Nature **529**, 484-489 (2016). https://doi.org/10.1038/nature16961.

660. M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, D. Forsyth, D. Hendrycks, HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal, arXiv:2402.04249 [cs.LG] (2024). http://arxiv.org/abs/2402.04249.

661. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, K. R. Narasimhan, "Tree of Thoughts: Deliberate Problem Solving with Large Language Models" in *37th Conference on Neural Information Processing Systems (NeurIPS 2023)* (2023). https://openreview.net/forum?id=5Xc1ecx01h.

662. T. A. Han, L. M. Pereira, T. Lenaerts, "Modelling and Influencing the AI Bidding War: A Research Agenda" in *Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES '19)* (2019) pp. 5–11. https://doi.org/10.1145/3306618.3314265.

663. T. Cimpeanu, F. C. Santos, L. M. Pereira, T. Lenaerts, T. A. Han, Artificial intelligence development races in heterogeneous settings. *Sci. Rep.* **12**, 1723 (2022). https://doi.org/10.1038/s41598-022-05729-3.

664. S. Armstrong, N. Bostrom, C. Shulman, Racing to the precipice: a model of artificial intelligence development. *Al Soc.* **31**, 201–206 (2016). https://doi.org/10.1007/s00146-015-0590-y.

665. M. Menashe, The Race to the Bottom Revisited: International Labour Law, Global Trade and Evolutionary Game Theory. *Oxf. J. Leg. Stud.* **40**, 53–81 (2020). https://doi.org/10.1093/ojls/ggz029.

666. M. M. Maas, "Artificial intelligence governance under change: Foundations, facets, frameworks", thesis, University of Copenhagen (2020).

667. L. Collina, M. Sayyadi, M. Provitera, Critical Issues About A.I. Accountability Answered. *California Management Review Insights* (2023). https://cmr.berkeley.edu/2023/11/critical-issues-about-a-i-accountability-answered/.

668. A. T. da Fonseca, E. Vaz de Sequeira, L. Barreto Xavier, "Liability for Al Driven Systems" in *Multidisciplinary Perspectives on Artificial Intelligence and the Law*, H. Sousa Antunes, P. M. Freitas, A. L. Oliveira, C. Martins Pereira, E. Vaz De Sequeira, L. Barreto Xavier, Eds. (Springer International Publishing, Cham, 2024), pp. 299–317.

669. M. Buiten, A. de Streel, M. Peitz, The law and economics of Al liability. *Comput. Law Secur. Rep.* **48**, 105794 (2023). https://doi.org/10.1016/j.clsr.2023.105794.

670. M. Busuioc, Accountable Artificial Intelligence: Holding Algorithms to Account. *Public Adm. Rev.* **81**, 825–836 (2021). https://doi.org/10.1111/puar.13293.

671. F. Doshi-Velez, M. Kortz, R. Budish, C. Bavitz, S. J. Gershman, D. O'Brien, K. Scott, S. Shieber, J. Waldo, D. Weinberger, A. Weller, A. Wood, "Accountability of Al Under the Law: The Role of Explanation" (Berkman Klein Center Working Group on Explanation and the Law, 2017); http://nrs.harvard.edu/urn-3:HULInstRepos:34372584.

672. N. Kolt, M. Anderljung, J. Barnhart, A. Brass, K. Esvelt, G. K. Hadfield, L. Heim, M. Rodriguez, J. B. Sandbrink, T. Woodside, Responsible Reporting for Frontier Al Development, arXiv:2404.02675 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2404.02675.

673. M. Anderljung, J. Barnhart, A. Korinek, J. Leung, C. O'Keefe, J. Whittlestone, S. Avin, M. Brundage, J. Bullock, D. Cass-Beggs, B. Chang, T. Collins, T. Fist, G. Hadfield, A. Hayes, L. Ho, S. Hooker, E. Horvitz, N. Kolt, . . . K. Wolf, Frontier Al Regulation: Managing Emerging Risks to Public Safety, arXiv:2307.03718 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2307.03718.

674. R. Palin, I. Habli, "Assurance of Automotive Safety – A Safety Case Approach" in *Computer Safety, Reliability, and Security (SAFECOMP 2010)* (Springer, 2010) pp. 82–96. https://doi.org/10.1007/978-3-642-15651-9_7.

675. M. L. Cummings, Rethinking the Maturity of Artificial Intelligence in Safety-Critical Settings. *Al Mag.* **42**, 6–15 (2021). https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/7394.

676. World Health Organization, Health products policy and standards (2024). https://www.who.int/teams/health-product-and-policy-standards/standards-and-specifications/norms-and-standards-for-pharmaceuticals/guidelines.

677. I. I. Livshitz, P. A. Lontsikh, N. P. Lontsikh, E. Y. Golovina, O. M. Safonova, "A Study of Modern Risk Management Methods for Industrial Safety Assurance in the Fuel and Energy Industry" in *2021 International Conference on Quality*
656. Q. Lu, L. Zhu, X. Xu, Z. Xing, S. Harrer, J. Whittle, Towards Responsible Generative AI: A Reference Architecture for Designing Foundation Model based Agents, arXiv:2311.13148 [cs.Al] (2023). http://arxiv.org/abs/2311.13148.

657.* R. Nakano, J. Hilton, S. Balaji, J. Wu, L. Ouyang, C. Kim, C. Hesse, S. Jain, V. Kosaraju, W. Saunders, X. Jiang, K. Cobbe, T. Eloundou, G. Krueger, K. Button, M. Knight, B. Chess, J. Schulman, "WebGPT: Browser-assisted question-answering with human feedback" (OpenAI, 2021); http://arxiv.org/abs/2112.09332.

658. A. Chan, C. Ezell, M. Kaufmann, K. Wei, L. Hammond, H. Bradley, E. Bluemke, N. Rajkumar, D. Krueger, N. Kolt, L. Heim, M. Anderljung, Visibility into Al Agents, arXiv:2401.13138 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2401.13138.

659. D. Silver, A. Huang, C. J. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, . . . D. Hassabis, Mastering the game of Go with deep neural networks and tree search. Nature 529, 484-489 (2016). https://doi.org/10.1038/nature16961.

660. M. Mazeika, L. Phan, X. Yin, A. Zou, Z. Wang, N. Mu, E. Sakhaee, N. Li, S. Basart, B. Li, D. Forsyth, D. Hendrycks, HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal, arXiv:2402.04249 [cs.LG] (2024). http://arxiv.org/abs/2402.04249.

661. S. Yao, D. Yu, J. Zhao, I. Shafran, T. L. Griffiths, Y. Cao, K. R. Narasimhan, "Tree of Thoughts: Deliberate Problem Solving with Large Language Models" in 37th Conference on Neural Information Processing Systems (NeurIPS 2023) (2023). https://openreview.net/forum?id=5Xc1ecx01h.

662. T. A. Han, L. M. Pereira, T. Lenaerts, "Modelling and Influencing the AI Bidding War: A Research Agenda" in Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society (AIES '19) (2019) pp. 5–11. https://doi.org/10.1145/3306618.3314265.

663. T. Cimpeanu, F. C. Santos, L. M. Pereira, T. Lenaerts, T. A. Han, Artificial intelligence development races in heterogeneous settings. Sci. Rep. 12, 1723 (2022). https://doi.org/10.1038/s41598-022-05729-3.

664. S. Armstrong, N. Bostrom, C. Shulman, Racing to the precipice: a model of artificial intelligence development. AI Soc. 31, 201–206 (2016). https://doi.org/10.1007/s00146-015-0590-y.

665. M. Menashe, The Race to the Bottom Revisited: International Labour Law, Global Trade and Evolutionary Game Theory. Oxf. J. Leg. Stud. 40, 53–81 (2020). https://doi.org/10.1093/ojls/ggz029.

666. M. M. Maas, "Artificial intelligence governance under change: Foundations, facets, frameworks", thesis, University of Copenhagen (2020).

667. L. Collina, M. Sayyadi, M. Provitera, Critical Issues About A.I. Accountability Answered. California Management Review Insights (2023). https://cmr.berkeley.edu/2023/11/critical-issues-about-a-i-accountability-answered/.

668. A. T. da Fonseca, E. Vaz de Sequeira, L. Barreto Xavier, "Liability for Al Driven Systems" in Multidisciplinary Perspectives on Artificial Intelligence and the Law, H. Sousa Antunes, P. M. Freitas, A. L. Oliveira, C. Martins Pereira, E. Vaz De Sequeira, L. Barreto Xavier, Eds. (Springer International Publishing, Cham, 2024), pp. 299–317.

669. M. Buiten, A. de Streel, M. Peitz, The law and economics of Al liability. Comput. Law Secur. Rep. 48, 105794 (2023). https://doi.org/10.1016/j.clsr.2023.105794.

670. M. Busuioc, Accountable Artificial Intelligence: Holding Algorithms to Account. Public Adm. Rev. 81, 825–836 (2021). https://doi.org/10.1111/puar.13293.

671. F. Doshi-Velez, M. Kortz, R. Budish, C. Bavitz, S. J. Gershman, D. O'Brien, K. Scott, S. Shieber, J. Waldo, D. Weinberger, A. Weller, A. Wood, "Accountability of Al Under the Law: The Role of Explanation" (Berkman Klein Center Working Group on Explanation and the Law, 2017); http://nrs.harvard.edu/urn-3:HULInstRepos:34372584.

672. N. Kolt, M. Anderljung, J. Barnhart, A. Brass, K. Esvelt, G. K. Hadfield, L. Heim, M. Rodriguez, J. B. Sandbrink, T. Woodside, Responsible Reporting for Frontier Al Development, arXiv:2404.02675 [cs.CY] (2024). https://doi.org/10.48550/arXiv.2404.02675.

673. M. Anderljung, J. Barnhart, A. Korinek, J. Leung, C. O'Keefe, J. Whittlestone, S. Avin, M. Brundage, J. Bullock, D. Cass-Beggs, B. Chang, T. Collins, T. Fist, G. Hadfield, A. Hayes, L. Ho, S. Hooker, E. Horvitz, N. Kolt, . . . K. Wolf, Frontier Al Regulation: Managing Emerging Risks to Public Safety, arXiv:2307.03718 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2307.03718.

674. R. Palin, I. Habli, "Assurance of Automotive Safety – A Safety Case Approach" in Computer Safety, Reliability, and Security (SAFECOMP 2010) (Springer, 2010) pp. 82–96. https://doi.org/10.1007/978-3-642-15651-9_7.

675. M. L. Cummings, Rethinking the Maturity of Artificial Intelligence in Safety-Critical Settings. AI Mag. 42, 6–15 (2021). https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/7394.

676. World Health Organization, Health products policy and standards (2024). https://www.who.int/teams/health-product-and-policy-standards/standards-and-specifications/norms-and-standards-for-pharmaceuticals/guidelines.

677. I. I. Livshitz, P. A. Lontsikh, N. P. Lontsikh, E. Y. Golovina, O. M. Safonova, "A Study of Modern Risk Management Methods for Industrial Safety Assurance in the Fuel and Energy Industry" in 2021 International Conference on Quality
Management, Transport and Information Security, Information Technologies (IT&QM&IS) (2021) pp. 165-167.
https://doi.org/10.1109/itqmis53292.2021.9642791.

678. H. E. Roland, B. Moriarty, System safety engineering and management (Wiley, New York, ed. 2nd ed, 1990), pp. 367.

679. L. Koessler, J. Schuett, Risk assessment at AGL companies: A review of popular risk assessment techniques from other safety-critical industries, arXiv:2307.08823 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2307.08823.

680. National Institute of Standards and Technology, Al Risk Management Framework. NIST (2021).
https://www.nist.gov/itl/ai-risk-management-framework.

681. D. Khodyakov, S. Grant, J. Kroger, M. Bauman, "RAND methodological guidance for conducting and critically appraising Delphi panels" (RAND Corporation, 2023); https://doi.org/10.7249/tla3082-1.

682. ISO, ISO 31000: Risk management (2021). https://www.iso.org/iso-31000-risk-management.html.

683. US Nuclear Regulatory Commission, Risk Metrics for Operating New Reactors. (2009).
https://www.nrc.gov/docs/MLO909/MLO90910608.pdf.

684. E. Black, R. Naidu, R. Ghani, K. Rodolfa, D. Ho, H. Heidari, "Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools" in Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '23) (Association for Computing Machinery, 2023) pp. 1-11.
https://doi.org/10.1145/3617694.3623259.

685. S. Rismani, R. Shelby, A. Smart, E. Jatho, J. Kroll, A. Moon, N. Rostamzadeh, "From Plane Crashes to Algorithmic Harm: Applicability of Safety Engineering Frameworks for Responsible ML" in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23) (Association for Computing Machinery, 2023) pp. 1-18.
https://doi.org/10.1145/3544548.3581407.

686. T. Kelly, A Systematic Approach to Safety Case Management. SAE Trans. J. Mater. Manuf. 113, 257-266 (2004).
http://www.jstor.org/stable/44699541.

687. M. Stein, C. Dunlop, "Safe before sale: Learnings from the FDA's model of life sciences oversight for foundation models" (Ada Lovelace Institute, 2023); https://www.adalovelaceinstitute.org/wp-content/uploads/2023/12/2023_12_ALI_Safe-before-sale_Discussion_paper.pdf.

688. T. Raz, D. Hillson, A Comparative Review of Risk Management Standards. Risk Manage.: Int. J. 7, 53-66 (2005).
https://doi.org/10.1057/palgrave.rm.8240227.

689. J. Clymer, N. Gabrieli, D. Krueger, T. Larsen, Safety Cases: How to Justify the Safety of Advanced AI Systems, arXiv:2403.10462 [cs.CY] (2024). http://arxiv.org/abs/2403.10462.

690. C. Haddon-Cave, The Nimrod Review: an independent review into the broader issues surrounding the loss of the RAF Nimrod MR2 aircraft XV230 in Afghanistan in 2006, report (Stationery Office, 2009).

691. N. G. Leveson, Applying systems thinking to analyze and learn from events. Saf. Sci. 49, 55-64 (2011).
https://doi.org/10.1016/j.ssci.2009.12.021.

692. D. Hendrycks, Introduction to AI Safety, Ethics, and Society (Taylor & Francis).

693.* Anthropic, Anthropic's Responsible Scaling Policy, Version 1.0. (2023).

694.* OpenAI, "Preparedness Framework (Beta)" (OpenAI, 2023); https://cdn.openai.com/openai-preparedness-framework-beta.pdf.

695.* Z. Kenton, T. Everitt, L. Weidinger, I. Gabriel, V. Mikulik, G. Irving, "Alignment of Language Agents" (Google DeepMind, 2021); http://arxiv.org/abs/2103.14659.

696. M. Wu, A. F. Aji, Style Over Substance: Evaluation Biases for Large Language Models, arXiv:2307.03025 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2307.03025.

697.* N. Lambert, R. Calandra, The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback, arXiv:2311.00168 [cs.LG] (2023). https://doi.org/10.48550/arXiv.2311.00168.

698. H. Bansal, J. Dang, A. Grover, "Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models" in The 12th International Conference on Learning Representations (ICLR 2024) (2023).
https://openreview.net/forum?id=dKl6lMwbCy.

699. J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, I. Higgins, "Solving math word problems with process- and outcome-based feedback" (Google Deepmind, 2022); https://doi.org/10.48550/arXiv.2211.14275.

700. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, K. Cobbe, "Let's Verify Step by Step" in The 12th International Conference on Learning Representations (ICLR 2024) (2023).
https://openreview.net/forum?id=v8LOpN6EOi.
Management, Transport and Information Security, Information Technologies (IT&QM&IS) (2021) pp. 165-167.
https://doi.org/10.1109/itqmis53292.2021.9642791.

678. H. E. Roland, B. Moriarty, System safety engineering and management (Wiley, New York, ed. 2nd ed, 1990), pp. 367.

679. L. Koessler, J. Schuett, Risk assessment at AGL companies: A review of popular risk assessment techniques from other safety-critical industries, arXiv:2307.08823 [cs.CY] (2023). https://doi.org/10.48550/arXiv.2307.08823.

680. National Institute of Standards and Technology, Al Risk Management Framework. NIST (2021).
https://www.nist.gov/itl/ai-risk-management-framework.

681. D. Khodyakov, S. Grant, J. Kroger, M. Bauman, "RAND methodological guidance for conducting and critically appraising Delphi panels" (RAND Corporation, 2023); https://doi.org/10.7249/tla3082-1.

682. ISO, ISO 31000: Risk management (2021). https://www.iso.org/iso-31000-risk-management.html.

683. US Nuclear Regulatory Commission, Risk Metrics for Operating New Reactors. (2009).
https://www.nrc.gov/docs/MLO909/MLO90910608.pdf.

684. E. Black, R. Naidu, R. Ghani, K. Rodolfa, D. Ho, H. Heidari, "Toward Operationalizing Pipeline-aware ML Fairness: A Research Agenda for Developing Practical Guidelines and Tools" in Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO '23) (Association for Computing Machinery, 2023) pp. 1-11.
https://doi.org/10.1145/3617694.3623259.

685. S. Rismani, R. Shelby, A. Smart, E. Jatho, J. Kroll, A. Moon, N. Rostamzadeh, "From Plane Crashes to Algorithmic Harm: Applicability of Safety Engineering Frameworks for Responsible ML" in Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23) (Association for Computing Machinery, 2023) pp. 1-18.
https://doi.org/10.1145/3544548.3581407.

686. T. Kelly, A Systematic Approach to Safety Case Management. SAE Trans. J. Mater. Manuf. 113, 257-266 (2004).
http://www.jstor.org/stable/44699541.

687. M. Stein, C. Dunlop, "Safe before sale: Learnings from the FDA's model of life sciences oversight for foundation models" (Ada Lovelace Institute, 2023); https://www.adalovelaceinstitute.org/wp-content/uploads/2023/12/2023_12_ALI_Safe-before-sale_Discussion_paper.pdf.

688. T. Raz, D. Hillson, A Comparative Review of Risk Management Standards. Risk Manage.: Int. J. 7, 53-66 (2005).
https://doi.org/10.1057/palgrave.rm.8240227.

689. J. Clymer, N. Gabrieli, D. Krueger, T. Larsen, Safety Cases: How to Justify the Safety of Advanced AI Systems, arXiv:2403.10462 [cs.CY] (2024). http://arxiv.org/abs/2403.10462.

690. C. Haddon-Cave, The Nimrod Review: an independent review into the broader issues surrounding the loss of the RAF Nimrod MR2 aircraft XV230 in Afghanistan in 2006, report (Stationery Office, 2009).

691. N. G. Leveson, Applying systems thinking to analyze and learn from events. Saf. Sci. 49, 55-64 (2011).
https://doi.org/10.1016/j.ssci.2009.12.021.

692. D. Hendrycks, Introduction to AI Safety, Ethics, and Society (Taylor & Francis).

693.* Anthropic, Anthropic's Responsible Scaling Policy, Version 1.0. (2023).

694.* OpenAI, "Preparedness Framework (Beta)" (OpenAI, 2023); https://cdn.openai.com/openai-preparedness-framework-beta.pdf.

695.* Z. Kenton, T. Everitt, L. Weidinger, I. Gabriel, V. Mikulik, G. Irving, "Alignment of Language Agents" (Google DeepMind, 2021); http://arxiv.org/abs/2103.14659.

696. M. Wu, A. F. Aji, Style Over Substance: Evaluation Biases for Large Language Models, arXiv:2307.03025 [cs.CL] (2023). https://doi.org/10.48550/arXiv.2307.03025.

697.* N. Lambert, R. Calandra, The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedback, arXiv:2311.00168 [cs.LG] (2023). https://doi.org/10.48550/arXiv.2311.00168.

698. H. Bansal, J. Dang, A. Grover, "Peering Through Preferences: Unraveling Feedback Acquisition for Aligning Large Language Models" in The 12th International Conference on Learning Representations (ICLR 2024) (2023).
https://openreview.net/forum?id=dKl6lMwbCy.

699. J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, I. Higgins, "Solving math word problems with process- and outcome-based feedback" (Google Deepmind, 2022); https://doi.org/10.48550/arXiv.2211.14275.

700. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, K. Cobbe, "Let's Verify Step by Step" in The 12th International Conference on Learning Representations (ICLR 2024) (2023).
https://openreview.net/forum?id=v8LOpN6EOi.