# Memetic Judo #1: On Doomsday Prophets v.3

There is a popular tendency to dismiss people who are concerned about AI-safety as "doomsday prophets", carrying with it the suggestion that predicting an existential risk in the near future would automatically discredit them (because "you know; *they* have always been wrong in the past").

## Example Argument Structure
> Predictions of human extinction ("doomsday prophets") have never been correct in the past, therefore claims of x-risks are generally incorrect/dubious.

## Discussion/Difficulties
This argument is persistent and kind of difficult to approach/deal with, in particular because it is technically a valid (yet, I argue, weak) point. It is an argument by induction based on a naive extrapolation of a historic trend. Therefore it cannot be completely dismissed by simple falsification through the use of an inconsistency or invalidation of one of its premises. Instead it becomes necessary to produce a convincing list of weaknesses - the more, the better. A list like the one that follows.

### #1: Unreliable Heuristic
If you look at history, these kind of ad-hoc "things will stay the same" predictions are often incorrect.
An example of this that is also related to technological developments would be the horse and mule populations in the US (back to below 10 million at present).
![horses and mules in US](https://bpb-us-w2.wpmucdn.com/campuspress.yale.edu/dist/0/4222/files/2023/04/horse_and_mules_pop_chart__0-1.jpg)

### #2: Survivorship Bias
Not only are they often incorrect, there is a class of predictions for which they, by design/definition, can only be correct ONCE, and for these they are an even weaker argument, because your sample is affected by things like survivorship bias. Existential risk arguments are in this category, because you can only go extinct once.

### #3: Volatile Times
We live in an highly unstable and unpredictable age shaped by rampant technological and cultural developments. The world today from the perspective of your grandparents is barely recognizable. In such times, this kind of argument becomes even weaker.
This trend doesn't seem to slow down and there are strong arguments that even benign AI would flip the table on many of such inductive predictions.

### #4: Blast Radius Induction (thanks [Connor Leahy](https://www.lesswrong.com/users/npcollapse))
Leahy has introduced an analogy of "technological blast radius" that represents an abstract way of thinking about different technologies in terms of their potential power, including their potential for causing harm either intentionally or by human error. As we are progressing through the tech tree - while many corners of it are relatively harmless or benign, the maximum "blast radius" of technology available to us necessarily increases. You can inflict more damage with a sword than with a club, even more if you have access to gunpowder, modern weapons etc. An explosion in a TNT factory can destroy a city block, and a nuclear arsenal could be used to level many cities. Now it seems to be very sensible (by induction!) that eventually, this "blast radius" will encompass all of earth. There are strong indicators that this will be the case for strong AI, and even that it is likely to occur BY ACCIDENT, once this technology has been developed.

### #5: Supporting Evidence & Responsibility
Having established this as a technically valid, yet weak argument - a heuristic for people-who-don't-know-any-better - it is your responsibility to look at the concrete evidence and available arguments our concerns about AI existential risk are based on, in order to decide whether to confirm or dismiss your initial hypothesis (which is valid). Because the topic is obviously extremely important, I implore you to do that.

### #6: Many Leading Researchers Worried
[![Paul Graham's tweet](https://i.imgur.com/uw0DtfQ.png)](https://twitter.com/paulg/status/1642110597545295872)  
The list of AI researchers worried about existential risk from AI includes extremely big names such as Geoffrey Hinton, Yoshua Bengio and Stuart Russel.

### #7: (Scientific) Doomsday Track Records Aren't That Bad
authored by [Will Petillo](https://www.lesswrong.com/users/willpetillo)  
  
Historically, the vast majority of doomsday claims are based on religious beliefs, whereas only a small minority have been supported by a large fraction of relevant subject matter experts.  If we consider only the latter, we find:

1. *Malthusian crisis*: false...but not really a doomsday prediction per se.
2. *Hole in the ozone layer*: true, but averted because of global cooperation in response to early warnings.
3. *Climate change*: probably true if we did absolutely nothing; probably mostly averted because of moderate, distributed efforts to mitigate (i.e. high investment in alternative energy sources and modest coordination).
4. *Nuclear war*: true, but averted because of global cooperation, with several terrifying near-misses...and could still happen.

This is not an exhaustive list as I am operating entirely from memory, but I am including everything I can think of and not deliberately cherry-picking examples--in fact, part of the reason I included (A) was to err on the side of stretching to include counter-examples.  Also, the interpretations obviously contain a fair bit of subjectivity / lack of rigor.  Nonetheless, in this informal survey, we see a clear pattern where, more often than not, doomsday scenarios that are supported by many leading relevant experts depict actual threats to human existence and the reason we are still around is because of active global efforts to prevent these threats from being realized.  
  
Given all of the above counterarguments (especially #6), there is strong reason to categorize x-risk from AI alongside major environmental and nuclear threats.  We should therefore assume by default that it is real and will only be averted if there is an active global effort to prevent it from being realized.

## Final Remarks
I consider this list a work-in-progress, so feel free to tell me about missing points (or your criticism!) in the comments.  
I also intend to make this a series about anti-x-risk arguments based on my personal notes and discussions related to my activism. Suggestions of popular or important arguments are welcome!