[146] Dan Hendrycks and Mantas Mazeika. X-risk analysis for AI research. arXiv preprint, 2022. (p. 10)
[147] Dan Hendrycks. Natural selection favors AIs over humans. arXiv preprint, 2023. (p. 10)
[148] Leo Gao, John Schulman, and Jacob Hilton. Scaling laws for reward model overoptimization. arXiv preprint, 2022. (p. 10)
[149] Joar Max Viktor Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. In NeurIPS, 2022. (p. 10)
[150] Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, and Zac Kenton. Goal misgeneralization: Why correct specifications aren't enough for correct goals. arXiv preprint, 2022. (p. 10)
[151] Lauro Langosco Di Langosco, Jack Koch, Lee D Sharkey, Jacob Pfau, and David Krueger. Goal misgeneralization in deep reinforcement learning. In ICML, 2022. (p. 10)
[152] Ben Smith. A brief review of the reasons multi-objective RL could be important in AI Safety Research. https://wwwalignmentforum.org/posts/i5dLfi6m6FCexReK9/a-brief-review-of-the-reasons-multi-objective-rl-could-be, 2021. (p. 10)
[153] Yotam Wolf, Noam Wies, Yoav Levine, and Amnon Shashua. Fundamental limitations of alignment in large language models. arXiv preprint, 2023. (p. 10)
[154] Manel Rodriguez-Soto, Maite Lopez-Sanchez, and Juan A Rodríguez-Aguilar. Guaranteeing the learning of ethical behaviour through multi-objective reinforcement learning. In AAMAS, 2021. (p. 10)
[155] Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization. arXiv preprint, 2023. (pp. 10 and 22)
[156] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In AISTATS, 2017. (pp. 10 and 23)
[157] Philip E Tetlock. A value pluralism model of ideological reasoning. JPSP, 1986. (p. 22)
[158] Umer Siddique, Paul Weng, and Matthieu Zimmer. Learning fair policies in multi-objective (deep) reinforcement learning with average and discounted rewards. In ICML, 2020. (p. 22)
[159] Jason Gabriel and Vafa Ghazavi. The challenge of value alignment: From fairer algorithms to AI safety. arXiv preprint, 2021. (p. 22)
[160] Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In ACM SIGSAC, 2016. (p. 22)
[161] Kristof Van Moffaert, Tim Brys, Arjun Chandra, Lukas Esterle, Peter R Lewis, and Ann Nowé. A novel adaptive weight selection algorithm for multi-objective multi-agent reinforcement learning. In IJCNN, 2014. (p. 23)
[162] Zafir Stojanovski, Karsten Roth, and Zeynep Akata. Momentum-based weight interpolation of strong zero-shot models for continual learning. In NeurIPS Interpolate Workshop, 2022. (p. 23)
[163] Steven Vander Eeckt et al. Weight averaging: A simple yet effective method to overcome catastrophic forgetting in automatic speech recognition. arXiv preprint, 2022. (p. 23)
[164] Margaret Li, Suchin Gururangan, Tim Dettmers, Mike Lewis, Tim Althoff, Noah A Smith, and Luke Zettlemoyer. Branch-Train-Merge: Embarrassingly parallel training of expert language models. arXiv preprint, 2022. (p. 23)
[165] Colin Raffel. A Call to Build Models Like We Build Open-Source Software. https://colinraffel.com/blog/a-call-to-build-models-like-we-build-open-source-software.html, 2021. (p. 23)