# Focusing on Flourishing Even When Survival is Unlikely (Part I)

1\. The Case
============

You've probably heard something like this before:

1.  If we survive this century, the expected value of the future is massive.
2.  If we don't survive, the expected value is near zero.
3.  Therefore, the value of an intervention is approximately proportional to how much it increases the chance of survival.

You won't go badly wrong following the conclusion, but (3) doesn't actually follow from (1) and (2). That's because interventions might vary in how they affect the expected value of the future conditional on survival.[^vuexh940rjk]

Will MacAskill makes roughly this argument in [Better Futures](https://www.forethought.org/research/better-futures) (August 2025). See the diagram below: survival-focused interventions target the red rectangle, flourishing-focused interventions target the blue. The blue rectangle might be much larger than the red rectangle -- if x-risk is 20% then even the best survival intervention can increase EV by at most 1.2x, whereas a flourishing intervention could increase EV by 5x or 5000x.

![](https://substackcdn.com/image/fetch/$s_!tpjs!,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F9888f0fc-8913-4594-a8df-59785de1f950_1600x944.png)

But is x-risk only 20%? MacAskill thinks so,[^my66sg3h5yp] but his argument applies even if extinction is very likely — say 99% — so long as there are interventions that increase flourishing by at least 100x. That's the scenario I want to discuss here in this post.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/78b7d23010537b8d42c1b12f6ce76e0551259c8df9bfdb6c.png)

My conclusion is:

1.  When survival is unlikely, focusing on flourishing becomes qualitatively different in ways that cut against focusing on flourishing. *This is what I address in this post.*
2.  I can imagine various strategies for focusing on flourishing when survival is unlikely, but none are attractive to me. *I will sketch these in a later post.*
3.  Overall, when survival is unlikely, we shouldn't focus on flourishing. Even if you think that the best possible futures are hundreds of times better than the middling futures.

2\. Challenges
==============

Recall that flourishing is the expected value *conditional on survival*. That is, flourishing-focused interventions target the survival posterior, consisting of the green and blue rectangles. Consequentially, if survival is likely then the survival posterior consists of ordinary futures, but if survival is unlikely then the survival posterior consists of weird futures, worlds very different from what we'd expect.

What kind of weird worlds?

*   An AI-enabled coup has occurred
*   Terrorist attacks on datacentres have halted development
*   Civilisational collapse destroyed most human knowledge
*   AGI turned out to be theoretically impossible
*   Aliens/simulators sabotaged our AI development
*   US and China start a nuclear conflict
*   Brain uploads arrived far sooner than expected[^ny2iyluwqq]
*   Extremely surprising civilisational sanity
*   Various cases [survival without dignity](https://www.lesswrong.com/posts/BarHSeciXJqzRuLzw/survival-without-dignity)

For more possible futures, see Bart Bussmann's [60+ Possible Futures](https://www.lesswrong.com/posts/kkGCyGBfgd7CaAShZ/60-possible-futures); which weird survival worlds are most likely will depend on your cause for pessimism.

This poses three problems:

+++ **Problem 1: Surviving worlds are harder to reason about.**

If survival is likely, then the survival posterior consists of ordinary worlds, which you can reason about using existing assumptions/models/trends. However, if survival is unlikely, then the survival posterior consists of weird worlds where our assumptions break down. This makes it much harder to estimate the impact of our interventions, because the world is unprecedented. For example, imagine if brain uploads arrive by 2030 -- this should make us more sceptical of extrapolating various economic trends that were observed before uploads.

And when you condition on survival, you update not just your empirical beliefs but your *moral* beliefs too. Suppose you're uncertain about (A) whether animals have as much moral weight as humans, and (B) whether we can build machines as smart as humans, and that (A) and (B) are correlated, both downstream of a latent variable like (H) "humans are special." Survival is much more likely if ¬B, so conditioning on survival upweights ¬B, which upweights H, which downweights A. Using the numbers below, your credence in animal moral weight drops from 59% to 34% — nearly halved — just by conditioning on survival. Gnarly!

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/b220094535fee0dc684084a86ebfa730f7a4ffe62413f610.png)

In principle, survival could be unlikely yet conditioning on it might not make worlds weird. To illustrate: suppose you're certain that humanity's survival depends entirely on the random weight initialisation of a particular pretraining run — 1% chance of good, 99% bad. Conditioning on survival, most survival worlds are ordinary in every respect except for the lucky weight initialisation. The weirdness of the weight initialisation is highly "localised", as opposed the "global weirdness" if (e.g.) aliens have sabotaged our deployment. That said, even if there are some "locally weird" survival worlds in the posterior (e.g. lucky weight initialisation), I expect that on most worldviews, the "globally" weird worlds will dominate.




++++++ **Problem 2: Surviving worlds are more diverse.**

When survival is unlikely, the survival worlds are more different from each other; this is because all ordinary worlds are alike but each weird world is weird in its own way. And because the survival worlds vary so much, it's harder to find interventions which are robustly beneficial -- an intervention that looks good in one weird survival world is likely to look poor in another. 

For example, suppose you think that if world governments proceed with the expected level of sanity, then ASI will cause extinction. But we might survive because governments showed unexpectedly *low* sanity (e.g. initiating nuclear conflict over some mundane issue) or unexpectedly *high* sanity (e.g. updating on early warning shots). Now consider an intervention which shifts power toward existing world governments at the expense of frontier AI labs: this might decrease flourishing if we survived via low governmental sanity and increase flourishing if we survived via high governmental sanity. The intervention's value flips sign depending on which weird world we end up in.

It was a bit tricky to try to justify this intuition, but here's a toy model: imagine worlds as points in R², with a Gaussian prior centered at the origin. Scatter some "attractors" randomly — some red (extinction), some green (survival). Each point in world-space inherits the fate of its nearest attractor. When most of the Gaussian mass falls in red regions, survival requires landing in one of the scattered green islands. These islands might be far apart in world-space. The survival posterior becomes multimodal, spread across disconnected regions. The diagram below illustrates: when P(survival) is low, Var(world | survival) tends to be high.

![](https://39669.cdn.cke-cs.com/rQvD3VnunXZu34m86e5f/images/dd32bfd33ea230a5ba340cdbef771c956e23e81c033528e6.png)

This diversity creates practical problems:

*   **Computational:** There are more worlds to reason about when choosing interventions.
*   **Cluelessness:** There are more worlds, so it's more likely that you are clueless about the sign of the intervention in some of the worlds.
*   **Social:** There are more worlds, so it's more likely that there is disagreement about the relative likelihood of the worlds, in ways that cause disagreement about the overall sign of the intervention.




++++++ **Problem 3: Transitional events wash out interventions.**

Ordinary worlds have more continuity between the present and the future, whereas weird worlds often involve some transitional event that explains why we survived, and these transitional events might 'wash out' your intervention.

For example, suppose you think the current policy landscape is likely to lead to extinction. Then we should be pessimistic about flourishing-focused policy interventions because, conditional on survival, there was probably some large-scale disruption of the policy landscape.




+++

In the next post, I will discuss potential strategies for focusing on flourishing when survival is unlikely. These strategies will aim to overcome some or all of the problems above.

[^vuexh940rjk]: In maths:Assuming that E(value|not survival) ≈ 0, we can decompose E(value|intervention) into the product of E(value|survival, intervention) and P(survival | intervention).This suggests that P(survival | intervention) is a good proxy for E(value|intervention), but this is only true if E(value|survival, intervention) doesn't vary much across interventions.However, E(value|survival, intervention) might vary significantly. 

[^my66sg3h5yp]: To illustrate, suppose you think that our chances of survival this century are reasonably high (greater than 80%) but that, if we survive, we should expect a future that falls far short of how good it could be (less than 10% as good as the best feasible futures). These are close to my views; the view about Surviving seems widely-held, and Fin Moorhouse and I will argue in essays 2 and 3 for something like that view on Flourishing.Introduction Better Futures (MacAskill, 3rd Aug 2025) 

[^ny2iyluwqq]: See State of Brain Emulation Report 2025 (Zanichelli et al., 17th Oct 2025)