AIアライメント
引用文献

↑  MacAskill, William (2022). What we owe the future. New York, NY. ISBN 978-1-5416-1862-6. OCLC 1314633519.  オリジナルのSeptember 14, 2022時点におけるアーカイブ。. https://web.archive.org/web/20220914030758/https://www.basicbooks.com/titles/william-macaskill/what-we-owe-the-future/9781541618633/ September 12, 2022閲覧。
1 2  Wu, Jeff; Ouyang, Long; Ziegler, Daniel M.; Stiennon, Nisan; Lowe, Ryan; Leike, Jan; Christiano, Paul (27 September 2021). "Recursively Summarizing Books with Human Feedback". arXiv:2109.10862 [cs.CL]。
↑  Pearce, Hammond; Ahmad, Baleegh; Tan, Benjamin; Dolan-Gavitt, Brendan; Karri, Ramesh (2022). “Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions”. 2022 IEEE Symposium on Security and Privacy (SP) (San Francisco, CA, USA: IEEE):  754–768. arXiv:2108.09293. doi:10.1109/SP46214.2022.9833571. ISBN 978-1-6654-1316-9. https://ieeexplore.ieee.org/document/9833571.
↑  “AI Safety via Debate”. OpenAI (2018年5月3日). February 10, 2023時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
1 2  Christiano, Paul; Shlegeris, Buck; Amodei, Dario (19 October 2018). "Supervising strong learners by amplifying weak experts". arXiv:1810.08575 [cs.LG]。
↑  Banzhaf, Wolfgang; Goodman, Erik; Sheneman, Leigh et al., eds (2020). Genetic Programming Theory and Practice XVII. Genetic and Evolutionary Computation. Cham: Springer International Publishing. doi:10.1007/978-3-030-39958-0. ISBN 978-3-030-39957-3.  オリジナルのMarch 15, 2023時点におけるアーカイブ。. https://web.archive.org/web/20230315193000/https://link.springer.com/book/10.1007/978-3-030-39958-0 2022年7月23日閲覧。
↑  Wiblin, Robert (2 October 2018). "Dr Paul Christiano on how OpenAI is developing real solutions to the 'AI alignment problem', and his vision of how humanity will progressively hand over decision-making to AI systems" (Podcast). 80,000 hours. No. 44. 2022年12月14日時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
↑  Lehman, Joel; Clune, Jeff; Misevic, Dusan; Adami, Christoph; Altenberg, Lee; Beaulieu, Julie; Bentley, Peter J.; Bernard, Samuel et al. (2020). “The Surprising Creativity of Digital Evolution: A Collection of Anecdotes from the Evolutionary Computation and Artificial Life Research Communities” (英語). Artificial Life 26 (2):  274–306. doi:10.1162/artl_a_00319. ISSN 1064-5462. PMID 32271631.  オリジナルのOctober 10, 2022時点におけるアーカイブ。. https://web.archive.org/web/20221010143108/https://direct.mit.edu/artl/article/26/2/274-306/93255 September 12, 2022閲覧。.
1 2  Leike, Jan; Krueger, David; Everitt, Tom; Martic, Miljan; Maini, Vishal; Legg, Shane (2018-11-19). Scalable agent alignment via reward modeling: a research direction. arXiv:1811.07871. https://ui.adsabs.harvard.edu/abs/2018arXiv181107871L/abstract.
1 2  “Our approach to alignment research”. OpenAI (2022年8月24日). February 15, 2023時点のオリジナルよりアーカイブ。2022年9月9日閲覧。
↑  Wiggers, Kyle (2021年9月23日). “OpenAI unveils model that can summarize books of any length”. VentureBeat. July 23, 2022時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
↑ Saunders, William; Yeh, Catherine; Wu, Jeff; Bills, Steven; Ouyang, Long; Ward, Jonathan; Leike, Jan (13 June 2022). "Self-critiquing models for assisting human evaluators". arXiv:2206.05802 [cs.CL]。
Bai, Yuntao; Kadavath, Saurav; Kundu, Sandipan; Askell, Amanda; Kernion, Jackson; Jones, Andy; Chen, Anna; Goldie, Anna; Mirhoseini, Azalia; McKinnon, Cameron; Chen, Carol; Olsson, Catherine; Olah, Christopher; Hernandez, Danny; Drain, Dawn (15 December 2022). "Constitutional AI: Harmlessness from AI Feedback". arXiv:2212.08073 [cs.CL]。
↑  Moltzau, Alex (2019年8月24日). “Debating the AI Safety Debate”. Towards Data Science. October 13, 2022時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
1 2 3  Wiggers, Kyle (2021年9月20日). “Falsehoods more likely with large language models”. VentureBeat. August 4, 2022時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
↑ The Guardian (2020年9月8日). “A robot wrote this entire article. Are you scared yet, human?”. The Guardian . ISSN 0261-3077. オリジナルのSeptember 8, 2020時点におけるアーカイブ。. https://web.archive.org/web/20200908090812/https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3 2022年7月23日 閲覧。
Heaven, Will Douglas (2020年7月20日). “OpenAI's new language generator GPT-3 is shockingly good—and completely mindless”. MIT Technology Review. July 25, 2020時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
1 2  Evans, Owain; Cotton-Barratt, Owen; Finnveden, Lukas; Bales, Adam; Balwit, Avital; Wills, Peter; Righetti, Luca; Saunders, William (13 October 2021). "Truthful AI: Developing and governing AI that does not lie". arXiv:2110.06674 [cs.CY]。
↑ Alford, Anthony (2021年7月13日). “EleutherAI Open-Sources Six Billion Parameter GPT-3 Clone GPT-J”. InfoQ . February 10, 2023時点のオリジナルよりアーカイブ。2022年7月23日 閲覧。
Rae, Jack W.; Borgeaud, Sebastian; Cai, Trevor; Millican, Katie; Hoffmann, Jordan; Song, Francis; Aslanides, John; Henderson, Sarah et al. (2022-01-21). Scaling Language Models: Methods, Analysis & Insights from Training Gopher. arXiv:2112.11446. https://ui.adsabs.harvard.edu/abs/2021arXiv211211446R/abstract.
↑ Nakano, Reiichiro; Hilton, Jacob; Balaji, Suchir; Wu, Jeff; Ouyang, Long; Kim, Christina; Hesse, Christopher; Jain, Shantanu; Kosaraju, Vineet; Saunders, William; Jiang, Xu; Cobbe, Karl; Eloundou, Tyna; Krueger, Gretchen; Button, Kevin (1 June 2022). "WebGPT: Browser-assisted question-answering with human feedback". arXiv:2112.09332 [cs.CL]。
Kumar, Nitish (2021年12月23日). “OpenAI Researchers Find Ways To More Accurately Answer Open-Ended Questions Using A Text-Based Web Browser”. MarkTechPost. February 10, 2023時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
Menick, Jacob; Trebacz, Maja; Mikulik, Vladimir; Aslanides, John; Song, Francis; Chadwick, Martin; Glaese, Mia; Young, Susannah et al. (2022-03-21). “Teaching language models to support answers with verified quotes”. DeepMind. arXiv:2203.11147.  オリジナルのFebruary 10, 2023時点におけるアーカイブ。. https://web.archive.org/web/20230210114137/https://www.deepmind.com/publications/gophercite-teaching-language-models-to-support-answers-with-verified-quotes September 12, 2022閲覧。.
↑  Askell, Amanda; Bai, Yuntao; Chen, Anna; Drain, Dawn; Ganguli, Deep; Henighan, Tom; Jones, Andy; Joseph, Nicholas; Mann, Ben; DasSarma, Nova; Elhage, Nelson; Hatfield-Dodds, Zac; Hernandez, Danny; Kernion, Jackson; Ndousse, Kamal (9 December 2021). "A General Language Assistant as a Laboratory for Alignment". arXiv:2112.00861 [cs.CL]。
↑  Cox, Joseph (2023年3月15日). “GPT-4 Hired Unwitting TaskRabbit Worker By Pretending to Be 'Vision-Impaired' Human”. Vice. 2023年4月10日閲覧。
↑  “Alignment of Language Agents”. DeepMind Safety Research – Medium (2021年3月30日). February 10, 2023時点のオリジナルよりアーカイブ。2022年7月23日閲覧。
↑  McCarthy, John; Minsky, Marvin L.; Rochester, Nathaniel; Shannon, Claude E. (2006-12-15). “A Proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955” (英語). AI Magazine 27 (4):  12. doi:10.1609/aimag.v27i4.1904. ISSN 2371-9621. https://ojs.aaai.org/aimagazine/index.php/aimagazine/article/view/1904.
↑ “'The Godfather of A.I.' warns of 'nightmare scenario' where artificial intelligence begins to seek power” (英語). Fortune . 2023年5月4日 閲覧。
“Yes, We Are Worried About the Existential Risk of Artificial Intelligence” (英語). MIT Technology Review. 2023年5月4日閲覧。
↑  Ornes, Stephen (2019年11月18日). “Playing Hide-and-Seek, Machines Invent New Tools”. Quanta Magazine. February 10, 2023時点のオリジナルよりアーカイブ。2022年8月26日閲覧。
↑  “Emergent Tool Use from Multi-Agent Interaction”. OpenAI (2019年9月17日). September 25, 2022時点のオリジナルよりアーカイブ。2022年8月26日閲覧。
↑  Shermer, Michael (2017年3月1日). “Artificial Intelligence Is Not a Threat—Yet”. Scientific American. December 1, 2017時点のオリジナルよりアーカイブ。2022年8月26日閲覧。
↑ Brown, Tom B.; Mann, Benjamin; Ryder, Nick; Subbiah, Melanie; Kaplan, Jared; Dhariwal, Prafulla; Neelakantan, Arvind; Shyam, Pranav; Sastry, Girish; Askell, Amanda; Agarwal, Sandhini; Herbert-Voss, Ariel; Krueger, Gretchen; Henighan, Tom; Child, Rewon (22 July 2020). "Language Models are Few-Shot Learners". arXiv:2005.14165 [cs.CL]。
Laskin, Michael; Wang, Luyu; Oh, Junhyuk; Parisotto, Emilio; Spencer, Stephen; Steigerwald, Richie; Strouse, D. J.; Hansen, Steven; Filos, Angelos; Brooks, Ethan; Gazeau, Maxime; Sahni, Himanshu; Singh, Satinder; Mnih, Volodymyr (25 October 2022). "In-context Reinforcement Learning with Algorithm Distillation". arXiv:2210.14215 [cs.LG]。
1 2 3  Shah, Rohin; Varma, Vikrant; Kumar, Ramana; Phuong, Mary; Krakovna, Victoria; Uesato, Jonathan; Kenton, Zac (2022-11-02). “Goal Misgeneralization: Why Correct Specifications Aren't Enough For Correct Goals”. Medium. https://deepmindsafetyresearch.medium.com/goal-misgeneralisation-why-correct-specifications-arent-enough-for-correct-goals-cf96ebc60924 2023年4月2日閲覧。.
1 2  Hubinger, Evan; van Merwijk, Chris; Mikulik, Vladimir; Skalse, Joar; Garrabrant, Scott (2021-12-01). Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv:1906.01820. https://ui.adsabs.harvard.edu/abs/2019arXiv190601820H/abstract.
↑  Zhang, Xiaoge; Chan, Felix T.S.; Yan, Chao; Bose, Indranil (2022). “Towards risk-aware artificial intelligence and machine learning systems: An overview” (英語). Decision Support Systems 159:  113800. doi:10.1016/j.dss.2022.113800. https://linkinghub.elsevier.com/retrieve/pii/S0167923622000719.
↑  Demski, Abram; Garrabrant, Scott (6 October 2020). "Embedded Agency". arXiv:1902.09469 [cs.AI]。
1 2  Everitt, Tom; Ortega, Pedro A.; Barnes, Elizabeth; Legg, Shane (6 September 2019). "Understanding Agent Incentives using Causal Influence Diagrams. Part I: Single Action Settings". arXiv:1902.09980 [cs.AI]。
↑  Cohen, Michael K.; Hutter, Marcus; Osborne, Michael A. (2022-08-29). “Advanced artificial agents intervene in the provision of reward” (英語). AI Magazine 43 (3):  282–293. doi:10.1002/aaai.12064. ISSN 0738-4602.  オリジナルのFebruary 10, 2023時点におけるアーカイブ。. https://web.archive.org/web/20230210153534/https://onlinelibrary.wiley.com/doi/10.1002/aaai.12064 September 6, 2022閲覧。.
↑  “UN Secretary-General's report on "Our Common Agenda"”. p. 63 (2021年). February 16, 2023時点のオリジナルよりアーカイブ。2023年7月17日閲覧。 “[T]he Compact could also promote regulation of artificial intelligence to ensure that this is aligned with shared global values”
↑  The National New Generation Artificial Intelligence Governance Specialist Committee (2021年10月12日). “Ethical Norms for New Generation Artificial Intelligence Released”. 2023年2月10日時点のオリジナルよりアーカイブ。2023年7月17日閲覧。
↑  Richardson, Tim (22 September 2021). “UK publishes National Artificial Intelligence Strategy”. The Register.  オリジナルのFebruary 10, 2023時点におけるアーカイブ。. https://web.archive.org/web/20230210114137/https://www.theregister.com/2021/09/22/uk_10_year_national_ai_strategy/ November 14, 2021閲覧。
1 2  “The National AI Strategy of the UK” (2021年). February 10, 2023時点のオリジナルよりアーカイブ。2023年7月17日閲覧。 “The government takes the long term risk of non-aligned Artificial General Intelligence, and the unforeseeable changes that it would mean for the UK and the world, seriously.”
↑   NSCAI Final Report. Washington, DC: The National Security Commission on Artificial Intelligence. (2021).  オリジナルのFebruary 15, 2023時点におけるアーカイブ。. https://web.archive.org/web/20230215110858/https://www.nscai.gov/wp-content/uploads/2021/03/Full-Report-Digital-1.pdf October 17, 2022閲覧。
Template:Existential risk from artificial intelligence
GPT-3のような言語モデルは、しばしば虚偽を生成する。この例では、人間の虚偽や誤解を模倣したGPT-3からの誤った回答を示す。