# Best introductory overviews of AGI safety?

I'm interested what people think are the **best overviews of AI risk for various types of people**. Below I've listed as many good overviews as I could find (excluding some drafts), splitting based on "good for a popular audience" and "good for AI researchers." I'd also like to hear if people think **some of these intros are better than others** (prioritizing between intros). I'd be interested to hear about podcasts and videos as well.

I am maintaining a list at [this Google doc](https://docs.google.com/document/d/1zx_WpcwuT3Stpx8GJJHcvJLSgv6dLje0eslVKvuk1yQ/edit#) to incorporate people's suggestions.  
  
Popular audience: 

*   [Vox](https://www.vox.com/future-perfect/2018/12/21/18126576/ai-artificial-intelligence-machine-learning-safety-alignment) (Kelsey Piper)
*   [AI alignment](https://en.wikipedia.org/wiki/AI_alignment) (Wikipedia -- perhaps the most important to get right!)
*   [Why alignment could be hard with modern DL](https://www.cold-takes.com/why-ai-alignment-could-be-hard-with-modern-deep-learning/) (Ajeya Cotra)
*   [Stampy wiki](https://ui.stampy.ai/) and [aisafety.info](https://aisafety.info/) ([Stampy team](https://stampy.ai/wiki/Get_involved))
*   [The most important century blog post series summary](https://www.cold-takes.com/most-important-century/#Summary) and [AI could defeat all of us combined](https://www.cold-takes.com/ai-could-defeat-all-of-us-combined/) and [Why would AI "aim" to defeat humanity](https://www.cold-takes.com/why-would-ai-aim-to-defeat-humanity/) (Holden Karnofsky)
*   [Current work in AI alignment](https://forum.effectivealtruism.org/posts/63stBTw3WAW6k45dY/paul-christiano-current-work-in-ai-alignment) (Paul Christiano)
*   [Future of Life Institute](https://futureoflife.org/ai/benefits-risks-of-artificial-intelligence/) (Ariel Conn)
*   [Why worry about future AI?](https://www.gleech.org/ai-risk) (Gavin Leech)
*   [80k full profile](https://80000hours.org/problem-profiles/artificial-intelligence/?gclid=CjwKCAjwkaSaBhA4EiwALBgQaO-jCNCrpUMndnt67IxLF4N_QIiCrSFIwp7Gb8RyWVAdHQePo4fofRoC2vAQAvD_BwE) (Benjamin Hilton)
*   [AGI Ruin: A list of lethalities](https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities) (Eliezer Yudkowsky)
*   [Extinction Risk from Artificial Intelligence](https://aisafety.wordpress.com/) (Michael Cohen)
*   [Set Sail For Fail? On AI risk](https://nintil.com/ai-safety) (Nintil)
*   [A shift in arguments for AI risk](https://bayes.net/prioritising-ai/) (Tom Adamczewski)
*   [Is power-seeking AI an x-risk](https://arxiv.org/pdf/2206.13353.pdf) (Joseph Carlsmith)
*   [More is different for AI](https://bounded-regret.ghost.io/more-is-different-for-ai/) (Jacob Steinhardt)
*   [The Basic AI Drives](https://selfawaresystems.files.wordpress.com/2008/01/ai_drives_final.pdf) (Steve Omohundro)
*   [AI Risk Intro 1: Advanced AI Might Be Very Bad](https://www.lesswrong.com/posts/bJgEMfiD48fEJJxjm/ai-risk-intro-1-advanced-ai-might-be-very-bad) (TheMcDouglas and LRudL)
*   [Four Background Claims](https://intelligence.org/2015/07/24/four-background-claims/) (Nate Soares)

AI researchers:

*   [CHAI bibliography](https://humancompatible.ai/bibliography)
*   [Unsolved Problems in ML Safety](https://arxiv.org/abs/2109.13916) (Dan Hendrycks)
*   [AI safety from first principles](https://www.lesswrong.com/s/mzgtmmTKKn5MuCzFJ) (Richard Ngo)
*   [The alignment problem from a DL perspective](https://arxiv.org/abs/2209.00626) (Richard Ngo)
*   [X-risk analysis for AI research](https://arxiv.org/abs/2206.05862) (Dan Hendrycks and Mantas Mazeika)
*   [Why I think more NLP researchers should engage with AIS concerns](https://wp.nyu.edu/arg/why-ai-safety/) (Sam Bowman)
*   More thoughts on outreach to researchers: [Marius Hobbhahn](https://www.lesswrong.com/posts/SqjQFhn5KTarfW8v7/lessons-learned-from-talking-to-greater-than-100-academics), [AISFB](https://forum.effectivealtruism.org/posts/ozm4SpiChfAAAGnw5/announcing-the-ai-safety-field-building-hub-a-new-effort-to) and [Resources I sent to AI researchers about AIS](https://forum.effectivealtruism.org/posts/8sAzgNcssH3mdb8ya/resources-i-send-to-ai-researchers-about-ai-safety) (Vael Gates)

Alignment landscape:

*   [My Overview of the AI Alignment Landscape: A Bird's Eye View](https://www.lesswrong.com/posts/SQ9cZtfrzDJmw9A2m/my-overview-of-the-ai-alignment-landscape-a-bird-s-eye-view) (Neel Nanda)
*   [My understanding of what everyone in technical alignment is doing and why](https://www.lesswrong.com/posts/QBAjndPuFbhEXKcCr/my-understanding-of-what-everyone-in-technical-alignment-is) (Thomas Larsen and Eli Lifland) and [Alignment org cheat sheet](https://www.lesswrong.com/posts/9TWReSDKyshfA66sz/alignment-org-cheat-sheet) (Thomas Larsen and Akash Wasil)

Podcasts and videos:

*   [Intro to AI Safety, Remastered](https://www.youtube.com/watch?v=pYXy-A4siMw) (Rob Miles)
*   [Researcher Perceptions of Current and Future AI](https://www.youtube.com/watch?v=yl2nlejBcg0) (Vael Gates)
*   [AI Fire Alarm](https://www.youtube.com/watch?v=pGjyiqJZPJo) (Connor Leahy)
*   [Eliezer Yudkowksy interview with Sam Harris](https://intelligence.org/2018/02/28/sam-harris-and-eliezer-yudkowsky/) \-\- full audio is still [only available to Sam Harris subscribers](https://www.lesswrong.com/posts/Swsy9fK7Gpf3t8Ewm/can-we-get-full-audio-for-eliezer-s-conversation-with-sam)
*   [Richard Ngo](https://axrp.net/episode/2022/03/31/episode-13-first-principles-agi-safety-richard-ngo.html) and [Paul Christiano](https://axrp.net/episode/2021/12/02/episode-12-ai-xrisk-paul-christiano.html) on [AXRP](https://axrp.net/)
*   [Ensuring smarter-than-human intelligence has a positive outcome](https://intelligence.org/2017/04/12/ensuring/) (Nate Soares)
*   [AI Alignment: Why It's Hard, and Where to Start](https://www.youtube.com/watch?v=EUjc1WuyPT8) (Eliezer Yudkowsky)
*   [Brian Christian](https://80000hours.org/podcast/episodes/brian-christian-the-alignment-problem/) and [Ben Garfinkel](https://80000hours.org/podcast/episodes/ben-garfinkel-classic-ai-risk-arguments/) on the 80,000 Hours Podcast
*   [Ajeya Cotra](https://www.youtube.com/watch?v=IKFQfYaJ0AY) and [Rohin Shah](https://www.youtube.com/watch?v=_5xkh-Rh6Ec) on the Future of Life Institute Podcast
*   Some audio recordings of the readings above (e.g. [Cold Takes audio](https://podcasts.apple.com/us/podcast/cold-takes-audio/id1580097837), [reading of 80k intro](https://podcasts.apple.com/us/podcast/preventing-an-ai-related-catastrophe-article/id1245002988?i=1000582699751), [EA Forum posts](https://forum.effectivealtruism.org/posts/K5Snxo5EhgmwJJjR2/announcing-audio-narrations-of-ea-forum-posts-1), [EA Radio](https://podcasts.apple.com/us/podcast/ea-radio/id1370275378), [Astral Codex Ten Podcast](https://sscpodcast.libsyn.com/), [Less Wrong Curated Podcast](https://www.lesswrong.com/posts/kDjKF2yFhFEWe4hgC/announcing-the-lesswrong-curated-podcast), [Nonlinear Library](https://forum.effectivealtruism.org/posts/JTZTBienqWEAjGDRv/listen-to-more-ea-content-with-the-nonlinear-library))
*   "[Recommended EA talks/videos for university groups](https://docs.google.com/document/d/1O3AtC1iiesnLdv2NwKJ5i1pXZEew9Qh32KlDviwxg6M/edit)" includes some generally good channels to look for more videos from in the future