Abstract: We evaluate the total probability of human extinction from naturally occurring processes. Such processes include risks that are well characterized such as asteroid impacts and supervolcanic eruptions, as well as risks that remain unknown. Using only the information that Homo sapiens has existed at least 200,000 years, we conclude that the probability that humanity goes extinct from natural causes in any given year is almost guaranteed to be less than one in 14,000, and likely to be less than one in 87,000. Using the longer track record of survival for our entire genus Homo produces even tighter bounds, with an annual probability of natural extinction likely below one in 870,000. These bounds are unlikely to be affected by possible survivorship bias in the data, and are consistent with mammalian extinction rates, typical hominin species lifespans, the frequency of well-characterized risks, and the frequency of mass extinctions. no similar guarantee can be made for risks that our ancestors did not face, such as anthropogenic climate change or nuclear/biological warfare. Out of all species that have existed, over 99% are now extinct 1 . Although human activity is dramatically increasing extinction rates for many species 2 , species extinctions were regular occurrences long before humanity emerged. Many of these extinctions were caused by gradual environmental shifts, evolutionary arms races, or local interspecific competition 3,4 , while others were abrupt, being part of global mass extinctions caused by asteroid impacts, volcanism, or causes as of yet to be identified  5, 6  . Could such a catastrophe befall our own species? If so, are the risks greater from natural or anthropogenic sources? Here, we evaluate the natural 'background' extinction rate for Homo sapiens. This means considerations of anthropogenic risks such as climate change and nuclear weapons are excluded from our estimates, although these clearly pose existential threats to our own species as well as others. Indeed, it has been hypothesized that the great majority of human extinction risk comes from anthropogenic sources  7, 8  . But by limiting our analysis to natural risks that our predecessors also faced, we can draw on data spanning many thousands (or millions) of years. Obtaining bounds on natural extinction rates also enables an indirect and partial test of the hypothesis that anthropogenic risks are greater than natural ones, as sufficiently low natural extinction risk will imply higher relative risks from anthropogenic sources. Estimating such an extinction rate directly is impossible. We have no examples of Homo sapiens extinction, so the most directly relevant data are non-existent. An alternative approach would be to enumerate the different types of naturally occurring hazards (e.g. asteroids, supervolcanoes), estimate their independent probability of causing extinction, and then use these probabilities to derive an aggregate extinction rate. However, this method has its own shortcomings. Beyond the great uncertainties around the probabilities of each risk, there could also be unknown risks that fail to be included. It would be hard to say with confidence that any list of risks had captured all natural hazards to humanity. We can bypass these problems by instead considering the length of time that humanity has survived so far 9,10 . This survival time can be used to estimate an upper bound on the extinction rate from all natural sources combined, including from sources for which we remain unaware. However, this approach could be subject to a particular form of sample bias known as an observation selection bias. These observer selection biases occur when a sample is not representative of all outcomes, but rather a subset of outcomes that are compatible with the existence of the observers 11 . For example, if human existence required a 10 million year (Myr) period of evolution free from asteroid impacts, any human observers will necessarily find in their evolutionary history a period of 10 Myr that is free of asteroid impacts, regardless of the true impact rate. Inferring a rate based on those 10 Myr could therefore be misleading, and methods must to be used to correct for this bias 12 .

Andrew E Snyder-Beattie
Toby Ord
Michael B Bonsall

An upper bound for the background rate of human extinction

Bounding the extinction Rate Based on Age of Humanity Anatomically modern human fossils in Ethiopia have been dated to 195 ± 5 thousand years ago (kya)  13  . A more recent fossil discovery in Morocco of an anatomically modern human has been dated to 315 ± 34 kya  14, 15  (though the fossil may exhibit more primitive neurocranial and endocranial morphology). Given that Homo sapiens has existed for hundreds of thousands of years, what can we infer about our background rate of extinction? Assuming that we share a common extinction rate with our predecessors, we can rule out rates that are too high to be compatible with this track record of survival. As our aim is to construct an upper bound, we can set aside the possibility that modern human technology, habitat range, and population size have reduced a number of natural extinction risks. The upper bound is only violated if we have reason to believe current extinction rates are higher than those our predecessors faced. Since we exclude anthropogenic risks from our analysis, we also set aside the majority of the ways in which this could be the case, although we acknowledge there exist boundary cases between purely natural and anthropogenic risks (e.g. a naturally emerging disease could be spread further by modern technology). Ultimately the scope of the upper bound is limited to all risks that have remained constant (or have been reduced) over the past few hundred thousand years. Likelihood of extinction rates. Analysis of taxonomic survivorship curves and temporal ranges for a wide variety of taxa suggest that extinction probabilities can be approximated well by assuming a constant risk of extinction over time  [16] [17] [18]  . Under this model, extinction can be represented by the exponential distribution with constant extinction rate μ. The probability that humanity goes extinct before time t is given by the cumulative distribution function P(T ≤ t) = 1 − e −μt , where T is the random variable denoting the longevity of our species. Conversely, the probability that humanity makes it beyond time t is P(T ≥ t) = e −μt . We want to evaluate the likelihood of an extinction rate μ, given the observation that humanity has lasted up to time t (so we know that the total longevity of humanity T ≥ t). This can be evaluated as the likelihood function  μ| ≥ = μ − T t e ( ) t . We compute the likelihood of extinction rates between 10 −8 and 10 −4 given a number of different plausible starting dates for Homo sapiens outlined in Fig.  1  and Table  1 . Assuming a 200 thousand year (kyr) survival time, we can be exceptionally confident that rates do not exceed 6.9 × 10 −5 . This corresponds to an annual extinction probability below roughly 1 in 14,000. The relative likelihood for such high extinction rates are below 10 −6 (one in a million) when compared to a rate of 10 −8 . If we assume that our track record extends further, this upper bound becomes stronger. Using the fossil dated to 315 ka as a starting point for humanity gives an upper bound of μ < 4.4 × 10 −5 , corresponding to an annual extinction probability below 1 in 22,800. Using the emergence of Homo as our starting point pushes the initial bound back a full order of magnitude, resulting in an annual extinction probability below 1 in 140,000. We can also relax the one in million relative likelihood constraint and derive less conservative upper bounds. An alternative bound would be rates with relative likelihood below 10 −1 (1 in 10) when compared to the baseline rate of 10 −8 . If we assume humanity has lasted 200 kyr, we obtain a bound of μ < 1.2 × 10 −5 , corresponding to an annual extinction probability below 1 in 87,000. Using the 2 Myr origin of Homo strengthens the bound by an order of magnitude in a similar way and produces annual extinction probabilities below 1 in 870,000. It is worth noting that this model can be generalised to allow for a varying extinction rate over time μ(t), so that the probability of surviving past time t is given by P(T ≥ t) = e −Θ(t)t , where ∫ μ Θ = t t s ds ( ) (1/ ) ( ) t 0 . The upper bound on Θ(t), the average extinction rate over the interval, can then be calculated in the same way as for the constant rate model.
 Observation Selection Effects The data on humanity's survival time could be subject to survivorship bias. If early Homo sapiens requires a long period of time to develop the intellectual machinery needed to make scientific observations, then such observations could not include short evolutionary histories, regardless of the extinction rate. The amount of information we could derive from a long track record of survival would therefore be limited due to this observation selection effect. Such a track record could indicate a low extinction rate, or be the byproduct of lucky ancestors surviving high extinction rates long enough to beget progeny capable of making scientific observations. One might therefore object that the bounds on the extinction rate we have estimated are too low  12, 23  . Here, we examine and respond to this concern.
 Models to quantify potential sample bias. To model observation selection bias, let us assume that after Homo sapiens first arises another step must be reached. This could represent the origin of language, writing, science, or any relevant factor that would transition early humans into the reference class of those capable of making observations (we call this step 'observerhood'). Let this step be a random variable denoted S, with cumulative distribution function F S (t). As we are examining natural risks, we assume that S and T are independent. The probability that humanity survives long enough to reach observerhood status (via intelligence, language, writing, science, etc) can be found with the following integral: ∫ > = ∞ P T S f t F t dt ( ) ( ) ( ) (1) T S 0 www.nature.com/scientificreports www.nature.com/scientificreports/ where f T (t) = μe −μt , the probability of extinction at time t. We evaluate an adjusted likelihood function  μ| > ⁎ T t ( ) , denoting that we are taking the likelihood of an extinction rate μ given that humanity has survived to time t, and the fact that we are conditioning on the existence of observers such that T > S. This results in the adjusted likelihood function: μ μ | > = > | > ⁎ T t P T t T S ( ) ( , ) (2)  ∫ = ∞ c f s F s ds 1 ( ) ( ) (3) t T S where c = P(T > S) is a normalising constant. We evaluate a model with four variations for the observerhood step: a model in which observerhood occurs as a single event that has a constant rate over time, a model with an increasing rate over time, a model with multiple steps, and a model where observerhood simply requires a fixed amount of time. If desired, we could more crisply define this observerhood property as the ability for a species to collect reliable data on its own track record of survival (e.g. via fossil dating) and analyse it. When correcting for observation selection effects, we are simply conditioning on the fact that our species has developed the ability to conduct this analysis. The observerhood property need not invoke consciousness or be the property of a biological species-a machine estimating a parameter would need to account for observer selection bias if its ability to make such estimates were correlated with the parameter in question. Model 1: Single step, constant rate. Our first model assumes that observerhood has a constant rate of occurrence θ, so that S is exponentially distributed with cumulative distribution function: F S (t) = 1 − e −θt . This model describes a process in which the transition from early humans into observers occurs by chance as a single step. This could represent the hypothesis that hierarchical language emerged in humans as the byproduct of a chance mutation  24  . With this model, the probability that observers arrive before extinction is P(T > S) = θ(θ + μ) −1 . Our likelihood function can be analytically derived:   ∫ μ θ μ θ μ | > =      +      − μ θ ∞ − − ⁎ T t e e ds ( ) (1 ) (4) t s s θ μ θ μ θ =      +      −          μ μθ − −+ e e ( 5 ) t t ( ) Model 2: single step, increasing rate. Our second model similarly assumes that a single step is needed but that the rate of observerhood increases over time. This model could represent increasing population size or population density, which could in turn drive cultural evolution and increase the probability of such a step  25  . We represent this with a Weibull distribution with cumulative distribution function = − θ − F t e ( ) 1 S t ( ) k where k > 1 indicates increasing rate over time (when k = 1, this is the same as the exponential in Model 1). We use numerical integration to evaluate the likelihood function. Model 3: multiple steps, constant rate. Our third model assumes that there are multiple steps that need to occur in a sequence in order to get observers. This could represent more incremental development of tools, culture, or language. We assume that each step is exponentially distributed with rate θ, so that the timing of the final kth step follows an Erlang distribution with cumulative distribution function: ∑ θ = − . θ = − − F t n e t ( ) 1 1 ! ( ) (6) S n k t n 0 1 Note that when k = 1, the distribution is the same as the exponential in Model 1. We use numerical integration to evaluate the likelihood function. Model 4: fixed time requirement. Our final model assumes that it takes a fixed amount of time τ to reach observerhood. This is an extreme model that allows for no chance, but could represent a gradual and deterministic accumulation of traits. The probability that observerhood has been reached before time t is therefore F S (t) = 1 [t>τ] , the characteristic function that takes the value 1 when t > τ and 0 otherwise. The probability that humanity survives past time τ is 1 − F T (τ) = e −μτ . Our likelihood function of μ is: ∫ μ μ | > = μτ μ τ − ∞ − > ⁎ T t e e d s ( ) 1 1 (7) t s s [ ]  = . μ τ − − e (8) t ( ) This likelihood expression can also be derived using the memoryless property of the exponential. It is worth noting that the fixed time model is a limiting case for both the increasing rate model and the multiple steps model. Taking the limit of Model 2 as k → ∞ results in a fixed time model with τ = θ −1 . Similarly, Model 3 converges to a fixed time model as the number of steps increases and the expected time of each step decreases (having infinitely many steps in the limit, each of which is infinitely short).
 Results of sample bias models. We evaluate the likelihood of extinction rates between 10 −8 and 10 −2 , given a human survival time of 200 kyr and a wide range of different rates at which observers could originate (Fig.  2 ). The first thing to note about the first three models is that when the observerhood rates are sufficiently rapid, the likelihood function converges to the unbiased version in the previous section. This can be verified by taking limits: for all of the models as θ → ∞ (or τ → 0 in the case of the fixed time model ),  μ| > → μ − ⁎ T t e ( ) t . If observerhood is expected to occur quickly, then we can take a 200 kyr track record of survival at face value and estimate the extinction rate without observation selection bias. However, as the observerhood rates decrease to the point where the expected observerhood time approaches an order of magnitude close to 200 kyr, observer selection bias emerges. Rates that were previously ruled out by our track record of survival are assigned higher likelihoods, since a portion of the track record is a necessity for observers (Fig.  2 ). For example in Model 1, when θ = 2 × 10 −4 (corresponding to an expected observerhood time of 20 kyr), the relative likelihood of μ = 6.9 × 10 −5 is increased by a factor of 2.3 (from 10 −6 to 2.3 × 10 −6 ). To get a likelihood of 10 −6 (corresponding to the most conservative upper bound), the rate must be set at 7.3 × 10 −5 (see all edited bounds in Table  2 ). Interestingly though, this effect is limited. Even as observerhood rates slow to the point where expected observerhood time greatly exceeds 200 kyr (for example exceeding 20 billion years), the revised upper bounds remain within a factor of 2 of the original bounds. The stricter the bound, the weaker the potential bias: for example the 10 −6 likelihood bound is only changed by a factor of about 1.2 in the limit as θ → 0. Although there would be some sample bias, there is a hard ceiling on how much our track record of survival can be distorted by observation selection effects. The reason slow rates of observerhood have a limited impact on our estimates is because if the extinction rate were exceptionally high, the lucky humans that do successfully survive to observerhood will have achieved such a status unusually quickly, and therefore will still observe a very short track record of survival. A long track record of survival is therefore still sufficient to rule out high extinction rates paired with low observerhood rates. We can demonstrate this by examining the typical time it takes for lucky survivors to reach observerhood, assuming a high extinction rate and a low observerhood rate. For example, in the single step constant rate model when www.nature.com/scientificreports www.nature.com/scientificreports/ θ = 10 −6 (corresponding to an expected observerhood time of 1 Myr) and μ = 10 −3 (corresponding to a typical extinction time of 1000 years), the expected observerhood time conditional on these high extinction rates is 1000 years. A typical observer will thus still have a very short track record of survival. Models with increasing rates or multiple steps exhibit the same property, although the bias is larger depending on parameter k. For both model 2 and 3 with θ = 10 −6 , μ = 10 −3 , and k = 2 (parameters normally corresponding to an expected observerhood time of 830 kyr for Model 2 and 2 Myr for model 3), the high extinction rates will still result in a typical observer emerging unusually early and having only about a 2000 year track record of survival. This can be also seen in Fig.  2  where for Models 1, 2, and 3, the likelihood of high extinction rates exceeding 10 −4 are still assigned low likelihood regardless of θ. However, severe observer selection bias can occur in Models 2 and 3 as k becomes larger, shaping the observerhood distribution such that early observerhood is vanishingly unlikely and late observerhood almost guaranteed. In the most extreme case this is represented by the fixed time model, where the probability of observerhood jumps from 0 to 1 when t = τ (the fixed time model is also the limiting case when k → ∞). If that fixed amount of time is long enough (say, exceeding 190 or 195 kyr), a 200 kyr track record of survival is no longer sufficient to rule out extinction rates greater than 10 −4 . This result occurs as the fixed time model prohibits any possibility of observerhood occurring unusually quickly. Any lineage of Homo sapiens lucky enough to survive long enough to obtain observer status must necessarily have a survival time greater than τ, which means that being an observer with a survival time of τ conveys zero information about the extinction rate. For numerous reasons, we find the fixed time model to be implausible. Virtually all biological and cultural processes involve some degree of contingency, and there is no fundamental reason to think that gaining the ability to make scientific observations would be any different. To illustrate a comparison, let us consider a world in which For the first three models, the unbiased model is recovered for large θ, and results start to become biased as the expected observerhood time approaches humanity's track record of survival. However, even as θ → 0, the bias is limited, and the likelihood of rates exceeding 10 −4 remains at zero. This is only violated in the final fixed time model, or in models 2 and 3 when k is sufficiently large.  www.nature.com/scientificreports www.nature.com/scientificreports/ the extinction rate is −4 (averaging one extinction every 10,000 years), but observerhood status takes a fixed 200 kyr. Under this model, humanity successfully surviving long enough to reach observer status is an event with 1 in 200 million chance. Given observation selection bias, we cannot rule out the possibility of rare events that are required for our observations. But we could ask why a 1 in 200 million chance event could not also include the possibility that modern human observers would emerge unusually rapidly. Language, writing, and modern science are perhaps highly unlikely to develop within ten thousand years of the first modern humans, but it seems exceptionally overconfident to put the odds at fewer than 1 in 200 million. A similar line of reasoning can be applied to determine whether the increasing rate and multiple step models with high k are reasonable. We test this by asking what parameters would be needed to expect a 200 kyr track record of survival with an extinction rate at our conservative upper bound of μ = 6.9 × 10 −5 . For the increasing rate model, observerhood is expected after 203 kyr with θ = 10 −7 and k = 14 and for the multiple step model, observerhood is expected after 190 kyr with θ = 10 −7 and k = 16. Although these models do not assign strictly zero probability to early observerhood times, the probabilities are still vanishingly small. With an increasing rate and these parameters, observerhood has less than a one in a trillion chance of occurring within 10,000 years (3.4 × 10 −14 ), and about 1% chance of occurring within 100,000 years. With multiple steps and these parameters, observerhood has less than one in a trillion chance of occurring within 10,000 years (5.6 × 10 −17 ), and less than a 0.02% chance of occurring within 100,000 years. In a similar fashion to the fixed time model, we feel that these models exhibit unrealistic levels of confidence in late observerhood times. Although the plausibility of the fixed time (or nearly fixed time) models is hard to test directly, the wide variance in the emergence of modern human behavior across geography offers one source of data that can test their plausibility. The Upper Palaeolithic transition occurred about 45 kya in Europe and Western Asia, marked by the widespread emergence of modern human behaviour  25  (e.g. symbolic artwork, geometric blades, ornamentation). But strong evidence exists for the sporadic appearance of this modern human behaviour much earlier in parts of Africa  26, 27  , including evidence of artwork and advanced tools as early as 164 kya  28  . Although numerous factors could have prevented the Upper Palaeolithic transition from occurring quickly, the fact that some human communities made this transition more than 100 kyr earlier than the rest of humanity indicates that a much earlier development trajectory is not entirely out of the question. In summary, observer selection effects are unlikely to introduce major bias to our track record of survival as long as we allow for the possibility of early observers. Deceptively long track records of survival can occur if the probability of early observers is exceptionally low, but we find these models implausible. The wide variance in modern human behavior is one source of data that suggests our track record is unlikely to be severely biased. We can also turn to other sources of indirect data to test for observer selection bias.
 testing the Bound with indirect Data We cross check our upper bound against four other sources of data: mammalian extinction rates, survival times of other human species, rates of potential catastrophes, and mass extinction rates. Although these alternative data do not directly predict the background extinction rate of Homo sapiens per se, the rates of extinction are likely generated by similar processes and thus enable an indirect test of the upper bound. If our upper bound is sound (not biased by observer selection effects or otherwise flawed), we can make testable predictions that it will be (A) broadly consistent with the extinction rates for similar species, and (B) not exceeded by the rate of potential catastrophes or mass extinctions. As the extinction rate of other species and catastrophes many millions of years ago have little bearing on our ability to make scientific observations, these data are also less subject to potential observer selection bias.
 Mammalian extinction rates. We first evaluate whether the upper bound is consistent with extinction rates for a typical mammalian species. Using fossil record data, median extinction rates for mammals have been estimated as high as 1.8 extinctions per million species years (E/MSY) 2 , or equivalently μ = 1.8 × 10 −6 . Other estimates using fossil record data range from 0.165 extinctions per million genus years 17 to 0.4 E/MSY for Cenozoic mammals  18  . Alternative methods using molecular phylogeny suggest a much lower rate of 0.023 E/MSY for mammals  29  and rates of 0.219-0.359 E/MSY for primates  30  , although these methods have been criticized  31  . All of these estimated background rates are consistent with our upper bound. It is worth noting that Homo sapiens may be at lower extinction risk than a typical mammalian species due to a large habitat range, large population size, and having a generalist diet, which are all traits that militate against extinction risk (whereas long generation times and large body mass are sometimes correlated with increased extinction risk)  32, 33  . Hominin survival times. Next, we evaluate whether the upper bound is consistent with the broader hominin fossil record. There is strong evidence that Homo erectus lasted over 1.7 Myr and Homo habilis lasted 700 kyr  21  , indicating that our own species' track record of survival exceeding 200 kyr is not unique within our genus. Fossil record data indicate that the median hominin temporal range is about 620 kyr, and after accounting for sample bias in the fossil record this estimate rises to 970 kyr  22  . Although it is notable that the hominin lineage seems to have a higher extinction rate than those typical of mammals, these values are still consistent with our upper bound. It is perhaps also notable that some hominin species were likely driven to extinction by our own lineage  34  , suggesting an early form of anthropogenic extinction risk. individual sources of extinction risk. The upper bound can also be evaluated against the frequency of events that could pose extinction risks (examples provided in Table  3 ). If any particular risk (such as those from asteroid impacts) is known to have a higher rate than our bound of 6.9 × 10 −5 , this could undermine and potentially falsify our hypothesis. We evaluate the frequencies of four types of potential disasters for which credible quantitative estimates exist: asteroid impacts, supervolcanic eruptions, stellar explosions, and vacuum collapse. www.nature.com/scientificreports www.nature.com/scientificreports/ All of these risks have estimated to occur with a frequency well below our bound (Table  3 ), with the exception of smaller supervolcanic eruptions. Recent work has suggested the frequency of eruptions ejecting >10 3 km 3 of material exceeds our upper bound of 6.9 × 10 −5 with a recurrence time of 17 kyr  35  . However, it is important to note that the smaller eruptions within this category do not necessarily have a high probability of causing human extinction. The most severe eruption of the past 2 million years occurred just 74 kya, and it is unclear whether the human population at the time was at risk of extinction. Some argue that the human population suffered a major bottleneck at the same time as the eruption  43  , although this theory remains controversial  44  . Some climate records averaged over decades fail to observe a severe volcanic winter in Africa at the time  45  and archaeological evidence shows that human communities in South Africa thrived both before and after the eruption  46  (although these data are not sufficient to rule out a severe short-lived catastrophe followed by a fast recovery in population). More conclusively, most members of the Hominidae family did not suffer population bottlenecks around the time, with the possible exception of Eastern chimpanzees and Sumatran orangutans  47  . The lack of dramatic evidence suggesting other species extinctions or bottlenecks undercuts the possibility that humanity's survival was highly improbable and is observed only due to observation selection effects. However, a handful of substantially larger flood basalt events have taken place over the past 250 Myr that have been linked to mass extinctions  39, 48  . These events occur with a frequency of roughly once every 20-30 Myr, much more infrequently than smaller eruptions. If we assume that human extinction is threatened only from larger volcanic eruptions well exceeding 10 3 km 3 , then none of the risk frequencies we have catalogued come within an order of magnitude of the conservative upper bound. Similarly, impacts from smaller asteroid around 1 km in diameter may not have a high probability of causing human extinction. Although it is hard to estimate the consequences of such impacts, some researchers have argued that such impacts would fall below the threshold for a global catastrophe  49  . Impacts that disperse enough dust and sulphites to significantly disrupt photosynthesis occur much more rarely, with an estimated frequency of about 15 Myr years  49  . If we assume human extinction is only threatened by these more severe impacts exceeding 5 km, each of these catastrophe frequencies falls well below even our most optimistic bound of 1 in 870,000 chance of extinction per year.
 Mass extinction frequency. A mass extinction is marked by substantially increased extinction of multiple geographically widespread taxa over a relatively short period of time  50  . There have been five major mass extinctions in the past 541 Myr  51, 52  , with many arguing that human activity is currently causing a sixth  2  . In a similar way to our previous analysis of catastrophe rates, we should expect our upper bound to be consistent with the frequency of non-anthropogenic mass extinctions. Using only the big five extinctions produces a frequency of less than one per 100 Myr, far below our upper bound. In addition to the big five, there have been 13 other mass extinctions in the fossil record  53  . Using these numbers for 18 mass extinctions over 541 Myr still results in a frequency of about one per 30 Myr, many orders of magnitude below our upper bound.
 conclusions Using the fact that humans have survived at least 200 kyr, we can infer that the annual probability of human extinction from natural causes is less than 1 in 87,000 with modest confidence (0.1 relative likelihood) and less than 1 in 14,000 with near certainty (10 −6 relative likelihood). These are the most conservative bounds. Estimates based on older fossils such as the ones found in Morocco dated to 315 kya result in annual extinction probabilities of less than 1 in 137,000 or 1 in 23,000 (for relative likelihood of 0.1 and 10 −6 , respectively). Using the track record of survival for the entire lineage of Homo, the annual probability of extinction from natural causes falls below 1 in 870,000 (relative likelihood of 0.1). We also conclude that these data are unlikely to be biased by observer selection effects, especially given that the bounds are consistent with mammalian extinction rates, the temporal range of other hominin species, and the frequency of potential catastrophes and mass extinctions. The bounds are subject to important limitations. Most importantly, they only apply to extinction risks that have either remained constant or declined over human history. Our 200 kyr track record of survival cannot rule out much higher extinction probabilities from modern sources such as nuclear weapons or anthropogenic climate change. Some naturally occurring risks could be also be worsened by anthropogenic factors: a minor asteroid impact could be interpreted as a nuclear attack and lead to retaliation  54  , or a naturally occurring disease which previously may have only been a local extinction risk could spread much further due to modern travel  23  . In the cases where a natural risk is amplified by modern conditions, we can still derive some partial information from the upper bound by evaluating how much the risk would need to change from the purely natural baseline. For  www.nature.com/scientificreports www.nature.com/scientificreports/ example, the claim that a natural disease poses a than 1 in 1,000 chance of extinction per year would require that anthropogenic conditions have increased the risk of natural disease by a factor of more than 14 to 870 (under our most conservative and optimistic upper bounds, respectively). In general, for a naturally occurring risk to violate our upper bounds via human activity by more than a factor of two, the majority of the risk would still need to come from anthropogenic circumstances. In general, we conclude that anthropogenic extinction risks are likely greater than natural ones. We do not have a long track record of data for anthropogenic risks, so evaluating this relies far more on speculation. But despite the paucity of data, the little evidence we do have seems to be indicative of rates greatly exceeding our upper bounds. During the Cuban Missile Crisis of 1962, John F Kennedy put the odds of nuclear war at 'somewhere between one out of three and even'  55  . If 0.1% of nuclear wars result in human extinction via nuclear winter, taking Kennedy's odds that year would surpass our most conservative bound by more than a factor of four (and surpass our most optimistic bound by a factor of more than 250). Anthropogenic climate change could pose existential risks as well if warming is much worse than expected. A ballpark suggestion for the probability of 20 degrees of anthropogenic climate change was placed at 1%  56  , which would make the planet largely uninhabitable for humans due to heat stress  57  . And these are not the only risks we may face. One century ago, the existential risks posed by nuclear weapons or climate change may have seemed extremely implausible. We should therefore be cautious before dismissing the potential risks that future centuries of technological development could bring, such as those stemming from biotechnology  58  or artificial general intelligence  59  . Despite the low probability of human extinction from natural causes, it may still be prudent to reduce these risks. Existential risks jeopardize not only the lives of those currently present, but also the existence of all future generations. Depending on how much value we place on such generations, it may still be cost-effective to reduce existential risks from natural sources  60  . However, given limited resources to spend on reducing existential risks, one may be better off focusing on greater risks from our own design. Figure 1 . 1 Figure1. Likelihood of extinction rates given our track record of survival so far, with estimated ranges of Hominin extinction rates, mammalian extinction rates, and mass extinction frequency included for reference. Blue horizontal lines indicate likelihood of 10% and 1%. Rates exceeding 6.9 × 10 −5 are ruled out even with the most conservative data. Extending humanity's track record of survival to match older fossils, the divergence with Homo neanderthalensis, or the origin of Homo creates even stricter bounds.
 Figure 2 . 2 Figure 2. Models of observer selection bias. Surface plots show likelihood for combinations of μ and θ (where k = 3 for Models 2 and 3) or τ in Model 4. Upper righthand plots show how likelihood shifts when θ → 0 in Model 1, and for a variety of k values in Models 2 and 3.For the first three models, the unbiased model is recovered for large θ, and results start to become biased as the expected observerhood time approaches humanity's track record of survival. However, even as θ → 0, the bias is limited, and the likelihood of rates exceeding 10 −4 remains at zero. This is only violated in the final fixed time model, or in models 2 and 3 when k is sufficiently large.
 Table 1 . 1 Survival times and resulting upper bounds. www.nature.com/scientificreports www.nature.com/scientificreports/ 
 Table 2 . 2 Upper bounds of μ with model 1 bias.
 Table 3 . 3 Catastrophe frequency estimates.


Abstract: We argue that in extensive decision problems extensive games with a single . player with imperfect recall care must be taken in interpreting information sets and strategies. Alternative interpretations allow for different kinds of analysis. We address the following issues: 1. randomization at information sets; 2. consistent beliefs; 3. time consistency of optimal plans; 4. the multiselves approach to decision making. We illustrate our discussion through an example that we call the ''paradox of the absentminded driver.'' Journal of Economic Literature Classification Numbers: C7, D0.

Michele Piccione
Ariel Rubinstein
Bob Aumann
Paolo Battigalli
Sergiu Hart
Dana Heller
Bart Lipman
Avishai Margalit
Roger Myerson
Hugh Neary
Motty Perry
Tim Van Zandt
Ruth Weintraub

On the Interpretation of Decision Problems with Imperfect Recall*

INTRODUCTION This paper is an examination of some modelling problems regarding imperfect recall within the model of extensive games. It is argued that, if the assumption of perfect recall is violated, care must be taken in interpreting the main elements of the model. Interpretations that are inconsequential under perfect recall have important implications in the analysis of games with imperfect recall. The distinction between perfect and imperfect recall for extensive games Ž . was introduced in  Kuhn 1953  . Since then, traditional game theory has excluded games with imperfect recall from its scope. In this paper, we wish to readdress this topic. Since the interpretative issues on the agenda appear within the framework of extensive games with a single player, we confine our discussion to decision problems with imperfect recall. Extensive decision problems are a special case of extensive games in that the set of players is a singleton. Our basic understanding of an extensive decision problem includes the following assertions: 1. Informational assumptions are modeled by partitioning the situa-Ž . tions histories where the decision maker takes an action into ''information sets.'' The interpretation of the ''informational structure'' is that the decision maker knows the information set he is at but, if an information set is not a singleton, he cannot distinguish among the points in the set he has reached. The decision maker, however, can make inferences or form a belief. 2. A strategy for the decision maker assigns to each information set one distinct action which is executed whenever an information set is reached. The decision maker cannot plan to assign different actions to two histories which lie in the same information set. 3. If the decision maker assesses his strategy at an information set including more than one history, he forms beliefs about the histories which led to it. These beliefs are the basis for his considerations. A decision problem exhibits imperfect recall if, at a point of time, the decision maker holds information which is forgotten later on. Specifically, an information set includes some histories which are incompatible with previously held information. Figures 1᎐3 are examples that illustrate three main aspects of imperfect recall in decision problems. The standard motive of imperfect recall appears in Example 2. The decision maker is initially informed about the move of chance and loses  . 1 which is central in this paper is rather unconventional; the decision maker does not distinguish between the first and the second nodes. Reaching the second node, he loses the information that he had previously made a choice. The inability of a decision maker to distinguish between two histories on the same path will be referred to as absentmindedness. In the above examples, the information sets determine the ability of the decision maker to recall. In practice, a decision maker can affect what he remembers. In this paper, however, we assume that the decision maker is not allowed to employ an external device to assist him in keeping track of the information which he would otherwise lose. Thus, we sidestep the decision maker's considerations regarding the trade-off between ''more memory'' and ''memory costs.''  FIG. 3. Example 3.  The common interpretation of a decision problem with imperfect recall refers to a situation in which an individual takes several successive actions and faces memory constraints. Imperfect recall, however, is not necessarily related to the mental phenomenon of memory loss and may also reflect the imperfect ability to make the inferences necessary for distinguishing among different points in the same information set. A decision maker may not realize that he is at the 17th exit along a highway, either because he does not recall whether he has passed the 16th intersection, or because he cannot infer that he is at the 17th intersection, despite the perfect pictures of each intersection in his mind. The latter interpretation of imperfect recall brings our discussion closer to the topic of ''bounded rationality.'' Ž . An alternative interpretation is found in Isbell 1957 . The decision maker is an organization consisting of many agents who have the same interests and act at different possible instances. In this case, the decision process of the organization may exhibit imperfect recall either because it must keep the instructions given to agents acting in successive situations simple or because of communication problems between agents. Agents receive instructions on how to behave and the collection of these instructions is equivalent to the notion of a strategy. From a psychological point of view, imperfect recall is a very important phenomenon as it puts severe constraints on a decision maker's behavior. We believe, however, that the ultimate proof for its relevance in economic analysis can only come from an interesting model which clearly explains an economic phenomenon. Constructing such a model is beyond the confines of this paper. The structure of the paper is as follows. We begin by presenting an example which we call the '' paradox of the absentminded dri¨er.'' This example is used to illustrate many of the points of this paper and will be referred to repeatedly. After providing the formal definition of the model, we discuss two issues of importance for decision problems with absent-mindedness. First, in Section 4, we review the circumstances in which whether the decision maker is allowed to randomize affect the analysis. Second, in Section 5, we show that the extension of Bayesian updating to decision problems with absentmindedness is not trivial. The two main topics which will be addressed are the timing of decision and the multiself approaches. The significance of these issues is marginal in decision problems with perfect recall since possible answers are inconsequential for the analysis.
 Ž . i Timing of decision. This issue deals with the interpretation of an information set as either a point of decision or a point in which a strategy is executed. In Section 6, we will show that for decision problems with imperfect recall optimal strategies may be time inconsistent and that strategies which are time consistent may not be optimal. The timing of decisions can be an important consideration for a decision maker whereas, with perfect recall, there is no reason to make some decisions at a particular point of time.
 Ž . ii The multisel¨es approach to decision making. Standard dynamic inconsistencies are generally addressed by assuming that the decision maker acts as a collection of distinct ''selves'' who behave independently. The behavior of the decision maker is analyzed as an equilibrium. In Section 7, we extend the multiself approach to decision problems with imperfect recall and show that in this extension an optimal strategy is dynamically consistent.
 THE PARADOX OF THE ABSENTMINDED DRIVER An individual is sitting late at night in a bar planning his midnight trip home. In order to get home he has to take the highway and get off at the Ž . second exit. Turning at the first exit leads into a disastrous area payoff 0 . Ž . Turning at the second exit yields the highest reward payoff 4 . If he continues beyond the second exit, he cannot go back and at the end of the Ž . highway he will find a motel where he can spend the night payoff 1 . The driver is absentminded and is aware of this fact. At an intersection, he cannot tell whether it is the first or the second intersection and he cannot Ž remember how many he has passed one can make the situation more . realistic by referring to the 17th intersection . While sitting at the bar, all he can do is to decide whether or not to exit at an intersection. We exclude at this stage the possibility that the decision maker can include random elements in his strategy. Example 1 describes this situation. Planning his trip at the bar, the decision maker must conclude that it is impossible for him to get home and that he should not exit when reaching an intersection. Thus, his optimal plan will lead him to spend the night at the motel and yield a payoff of 1. Now, suppose that he reaches an intersection. If he had decided to exit, he would have concluded that he is at the first intersection. Having chosen the strategy to continue, he concludes that he is at the first intersection with probability 1r2. Then, reviewing his plan, he finds that it is optimal for him to leave the highway since it yields an expected payoff of 2. Despite no new information and no change in his preferences, the decision maker would like to change his initial plan once he reaches an intersection! Note that if the decision maker now infers that he would have exited the highway had he passed the first intersection, his reasoning becomes circular; he must conclude that he is at the first intersection and that it is optimal to continue. We wish to emphasize that this is not a standard example of time inconsistency. Usually, time inconsistency is obtained as a consequence of Ž . changes in either preferences tastes or information regarding the moves of nature during the execution of the optimal plan. Here, preferences over final outcome are constant and the only factor intervening between planning and execution of the optimal strategy is the occurrence of the situation which calls for execution, that is, reaching the intersection. We find this example paradoxical as it exhibits a conflict between two ways of reasoning at the intersection. The first is based on a quite minimal principle of rationality; having chosen an optimal strategy, one does not have to verify its optimality at the time of execution unless there is a change in information or in preferences. In our example, the decision maker knew he would reach the intersection with certainty and his preferences are constant. This principle leads to the conclusion that the decision maker should stick to his plan to continue. The second way is based on the principle which calls at each instance to maximize expected payoffs given the relevant beliefs. In our example, this principle leads to the conclusion of exiting. The conflict between these two potential lines of reasoning is at the root of the apparent ambiguity of our example.
 EXTENSIVE DECISION MODEL In this section, a formal definition of the extensive decision model is Ž . given. The presentation follows that of  Osborne and Rubinstein 1994 .  The reader can easily identify the model with the standard definition in the ''tree'' language. Ž . ² : A finite decision problem is a five-tuple ⌫ s H, u, C, , I , where: Ž . a H is a finite set of sequences. We assume that the empty se-Ž . quence, , is an element of H and that if a , . . . , a g H and 1 K Ž . Ž . a, . . . , a / then a , . . . , a g H. 1 K 1 Ky1 Ž . We interpret a history a , . . . , a g H as a feasible sequence of 1 K Ž . actions taken by the decision maker or by chance. The history a , . . . , a 1 K Ž . g His terminal if there is no a , . . . , a , a g H. The set of terminal 1 K histories is denoted by Z. The set of actions available to the decision Ž . Ä maker or chance after a nonterminal history h is defined by A h s a: Ž . 4 Ž . h,a g H . To avoid degenerate cases we assume that A h contains at least two elements. When presenting a decision problem diagramatically, we draw H as a tree whose nodes are the set of histories with root and Ž . Ž . whose edges combine a node a , . . . , a with a node a , . . . , a . 1 K 1 Kq1 Ž . Ž . b u: Z ª R is a utility function which assigns a number payoff to each of the terminal histories. Preferences are defined on the set of all lotteries over terminal histories and satisfy the VNM assumptions.
 Ž . c C is a subset of H. We assume that the chance player moves after histories in C.
 Ž . d is the decision maker's belief about the chance player's behav-Ž . ior. assigns to each history h g C a probability measure on A h . To Ž . avoid degeneracy, we assume that h, a is strictly positive for all h g C Ž . and a g A h . Thus, the set of histories H is partitioned into three subsets: Z, the set of terminal histories; C, the set of histories after which chance moves; D s H y Z y C, the set of histories after which the decision maker moves.
 Ž . e The set of information sets, which is denoted by I, is a partition of Ž . D. We assume that for all h, hЈ in the same cell of the partition A h s Ž . A hЈ ; i.e., the sets of actions available to the decision maker at histories in the same information set are identical. For convenience, with a slight abuse of notation, we will sometimes denote the set of actions which are Ž . available at a history in X by A X . Note that, in contrast to some authors, we do not exclude from the class Ž of decision problems those which exhibit absentmindedness see definition . below . If all information sets in I are singletons we say that ⌫ is a decision problem with perfect information. Ž . A pure strategy, f, is a function which assigns to every history h g D Ž . an element of A h with the restriction that if h and hЈ are in the same Ž . Ž . information set f h s f hЈ . Notice that this definition requires that the decision maker plans an action at histories which he will not reach if he follows the strategy. We are now ready for the main definitions of this paper. The experience Ž . of the decision maker at a history h in D, denoted by exp h , is the sequence of information sets and actions of the decision maker along the history h. We adopt the convention that the last element in the sequence Ž . exp h is the information set which contains h. A decision problem has perfect recall if for any two histories, h, hЈ g D, Ž . Ž . which lie in the same information set, exp h s exp hЈ . Thus, in a decision problem with perfect recall, the decision maker ''remembers'' the succession of the information sets he has faced and the actions he has taken. A decision problem for which the above condition is violated is referred to as a decision problem with imperfect recall. Ž . Given a history h s a , . . . , a and L -K, the history hЈ s 1 K Ž . a, . . . , a is a subhistory of h. We say that a decision problem ⌫ exhibits 1 L absentmindedness if there are two histories h and hЈ such that hЈ is a subhistory of h and both belong to the same information set. The decision problem illustrated in Example 1 exhibits absentminded-Ž . ness since history B and its subhistory are in the same information set.
 THE VALUE OF RANDOMIZATION In this section, we discuss the implications of enlarging the strategy set of a decision maker to include random strategies. Given that the decision maker behaves as an expected utility maximizer, randomization over pure strategies is redundant for problems of either perfect or imperfect recall. Define a mixed strategy to be a probability distribution over the set of pure strategies. It describes a behavior in which randomization occurs only at the outset, before the decision problem unfolds. Each pure strategy induces a lottery over Z. A mixed strategy induces a lottery over Z which is the compound lottery of the lotteries induced by each of the pure strategies in its support. Therefore, no mixed strategy can be strictly preferred to all the pure strategies. Behavior strategies perform a different method of randomization. A beha¨ioral strategy, b, is a function which assigns to every history h g D, a Ž . Ž .
 Ž . Ž . distribution b h over A h such that b h s b hЈ for any two histories h and hЈ which lie in the same information set. In decision problems without Ž . absentmindedness b h is a lottery which is realized when the information set which contains h is reached. For decision problems with absentminded-Ž . ness we take b h to be a random device which is activated independently every time the information set which includes h is reached. Consider again Example 1. In this problem there are two pure strategies, ''B'' and ''E'' which yield payoffs of 1 and 0, respectively. Although the absentminded driver cannot use a pure strategy to reach home with certainty, he can toss a coin and obtain an expected payoff of 1.  25 . Note 1 that his optimal behavioral strategy is to exit with probability and yields 3 4 the expected payoff of . 3 It turns out that absentmindedness is necessary for behavioral strategy Ž . to be strictly optimal. This was shown in Isbell 1957 and we provide the proof for completeness. PROPOSITION 1. Suppose ⌫ does not exhibit absentmindedness. Then for any beha¨ioral strategy there is a pure strategy which yields a payoff at least as high. ² : Con¨ersely, suppose ⌫ s H, u, C, , I exhibits absentmindedness. Then, ² : there exist a decision problem ⌫Ј s H, uЈ, C, , I and a beha¨ioral strategy which yields a payoff strictly higher than any payoff achie¨ed by a pure strategy. Proof. See the Appendix. The Paradox of the Absentminded Dri¨er Re¨isited. The inconsistency discussed in Section 2 is not a consequence of the restriction that the strategy set includes only pure strategies and persists when the decision maker is allowed to choose random actions. The optimal behavioral 2 strategy is to choose B with probability . Reaching the intersection, the 3 driver will form beliefs about where he is. Denote by ␣ the probability he assigns to being at the first intersection. Then, his expected payoff is w 2 Ž . x Ž .w Ž . x ␣ p q4 1 y p p q 1 y ␣ p q 4 1 y p , where p is the probability of Ä Ž . 4 not exiting. The optimal p is now max 0, 7␣ y 3 r6␣ . This is inconsistent with his original plan unless ␣ s 1. In other words, his original plan is time consistent if and only if he believes that there is no chance he has passed the first intersection. We find such a belief unreasonable. Given his strategy it seems natural to assign to the second intersection a probability 2 which is times the probability assigned to the first intersection, which 3 implies ␣ s 0.6. The issue of consistent beliefs will be discussed in the next section. This type of time inconsistency can appear also in decision problems in which the optimal strategy is pure. Consider the following example shown in Fig.  4 . The optimal behavioral strategy is the pure strategy which selects L at d and L at d . To verify it, denote by ␣ and ␤ the probabilities of
 CONSISTENT BELIEFS If information sets are to be interpreted as points of decision, Examples 1 and 4 suggest that a decision maker who acts on the basis of expected utility maximization may be unable to execute the optimal strategy. The first step in addressing this issue is to specify the decision maker's beliefs at an information set which is not a singleton. As we shall see, finding an appropriate specification for decision problems with absentmindedness is not conceptually trivial. We define a belief system as a function which assigns to any Ž < . information set X and any history h g X a nonnegative number h X Ž < . such that Ý h X s 1. The interpretation is that the decision maker, h g X Ž < . upon reaching X, assigns probability h X to the possibility that he is at Ž < . h. Let p h hЈ, b be the probability that, conditional on reaching hЈ, the history h will be realized when executing the strategy b. We denote Ž < . Ž < . p h ,b by p h b . Several alternatives are conceivable for the specification of the decision maker's beliefs. Since our objective is to examine the optimality of a strategy during its execution, we find it natural to assume that the beliefs of the decision maker are related in a systematic way with the strategy to be assessed. The condition that we require a belief system to satisfy to be consistent with a behavioral strategy b mirrors the frequency approach to belief formation. Namely, if an information set X is reached with Ž < . positive probability, h X is assumed to be equal to the long run proportion of times in which ''visiting'' the information set X involves being in h for a decision maker who plays the decision problem again and again and follows b.  DEFINITION.  A belief system is consistent with the behavioral strategy b if for every information set X which is reached with positive Ž < . Ž < . Ž < . probability and for every h g X, h X s p h b rÝ p hЈ b .
 hЈg X Our definition of consistency imposes restrictions only on beliefs at information sets which are reached with positive probability. Notice that, for decision problems without absentmindedness, consistency is equivalent to Bayes' formula. For decision problem with absentmindedness, however, the denominator can be greater than one. The similarity with Bayes' formula is only notational. To clarify this definition, consider first the absentminded driver example  Our definition of consistent beliefs can also be motivated as being derived from a probability space which includes the time at which the decision maker can be. As an illustration consider Example 5 and assume that each action takes one unit of time. The relevant space of instances Ž . Ž . Ž . Ž . Ž . consists of L, 1 , L, 2 , R, 1 , and R, 2 , where x, t is the instance in which the chance player chooses x and time is t. Assuming equal probabilities for each instance is consistent with the description of the chance player in the decision problem. Then, a decision maker who is told that he is at d updates his belief by the Bayesian formula. For example, the 1 unconditional probability of the second node after R is pr4, where p is the probability of choosing B at d , and the unconditional probability of  In this paper, we adopt the above definition of consistency. However, we find the following definition of consistency reasonable as well. DEFINITION. A belief system is Z-consistent with the behavioral strategy b if for every information set X which is reached with positive probability and for every h g X < h X Ž . < Ä 4 Ý p z b r࠻ hЈ ¬ hЈgX and is a subhistory of z Ž . Äz<h is a subhistory of z4 s . < Ý p z b Ž . Äz<z has a subhistory in X 4 The rationale behind this definition is that, given the behavioral strategy b, the probability of the event that the random elements which determine Ž < . the terminal history z are realized is p z b ; if the history z includes K histories in which the decision maker is asked to act then he assigns to Ž < . each of them the ex ante probability p z b rK. To illustrate the Z-consistency, consider again the absentminded driver 1 example and the strategy that selects B with probability Note that the paradoxical flavor of the absentminded driver example is unaffected by the type of consistency we adopt. Also, if the decision problem does not exhibit absentmindedness then consistency and Z-consistency are identical. We do not have a firm view about the ''right'' definition of consistent beliefs. The issue deserves further investigation. We wish to point out that if the decision maker adopts Z-consistency then he is exposed to a sort of ''money pump.'' Consider Example 5 and the behavioral strategy that selects B with probability 1. Given Z-consistent beliefs a risk neutral decision maker, upon reaching the information set, will always accept an Ž . agreement in which he gains $1.1 if he is at L and loses $1 otherwise. If such an agreement is offered whenever the information set is reached, the resulting undesirable lottery yields the decision maker $1.1 with probability 0.5 and y$2 with probability 0.5.
 TIME CONSISTENCY When does the decision maker make his decision? Can he decide about the point of time at which a decision is made? To what extent can the decision maker commit to decisions he makes? These questions are superfluous for decision problems with perfect recall since the ex ante optimal strategy remains optimal during its execution. In the presence of imperfect recall, the optimal strategy may cease to be optimal along its execution and answers to these questions are significant for the analysis. We say that a strategy is time consistent if there is not, at any information set which is reached as the decision problem unfolds, a different strategy for the remainder of the decision problem which yields a higher expected payoff. Notice that we require the optimality to be assessed only at the information sets and not before the initial history. The significance of the statement that a strategy is optimal at an information set depends on the beliefs that the decision maker is assumed to hold. Requiring that beliefs be consistent leads to the following definition.  DEFINITION . A behavioral strategy b is time consistent if there is a belief consistent with b such that for every information set X which is reached with positive probability under b, h p z¬h,b u z G h p z¬h,bЈ u z Ž . Ž . Ž . Ž . Ž . Ž . Ý Ý Ý Ý h gX z g Z h g X z g Z for any behavioral strategy bЈ. The notion of time consistency that we use is a very strong criterion. We must emphasize that it does not rest on a model of how decisions are made at each information set. The definition presumes that the decision maker, in principle, can commit to a future course of action. Of course, if the decision maker is aware that a revision of the plan is possible at future information sets, expectations of commitments are naive. Time consistency is an assessment as to whether, conditional upon the realization of information set, there exists an alternative rule of behavior which yields the decision maker a higher payoff. If an optimal strategy satisfies this criterion, the distinction with respect to information sets as points of decision or points of execution is inconsequential for a decision maker who is implementing it and whose beliefs are consistent. If, however, an optimal strategy fails this criterion, this distinction becomes significant and alternative specifications of the domain of choice and the timing of decision of the decision maker can give rise to different analyses. A well-known result states that for decision problems with perfect recall a strategy is optimal if and only if it is time consistent. This is not the case for problems with imperfect recall. As we have seen, the optimal behavioral strategy for the absentminded driver is not time consistent. It is easy 5 to see that the only time consistent strategy is to exit with probability . In the case of absentmindedness, the divergence between optimality and time consistency is two-sided. The only optimal strategy is not time consistent and the only time consistent strategy is not optimal. The next Ž . example Fig.  6  shows that, even without absentmindedness, a strategy may be time consistent and not optimal. Consider the strategy where the decision maker plays B at both information sets. The belief system consistent with this strategy assigns equal probabilities to each of the histories in each of the information sets. Therefore, this strategy is time consistent although the optimal strategy is to choose S at both information sets. The next proposition provides conditions under which optimal and time consistent strategies are equivalent. Two histories are said to split at h if h is their longest common subhistory. PROPOSITION 2. Let ⌫ be a decision problem without absentmindedness for which for any information set X and two histories hЈ, hЉ g X which split at h g C, the information sets which appear in exp hЈ are the same as in Ž . exp hЉ .
 Ž . Ž . ) A beha¨ioral strategy for ⌫ is optimal if and only if it is time consistent. probability one to nature's move R immediately after R and probability 2 to both of nature's moves when R is followed by his action B.
 THE MULTISELF APPROACHES
 Ž . For standard dynamic inconsistencies, Strotz 1956 provides a framework of analysis in which every information set is assumed to be a point of decision and the decision maker is unable to control his behavior at future information sets. A decision maker acts as a collection of hypothetical Ž . agents selves whose plans form an equilibrium. More precisely, for any Ž . decision problem ⌫ define G ⌫ to be the extensive game in which each information set of ⌫ is assigned a distinct player, and all players have the same payoff as the decision maker. The behavior of the decision maker in Ž . ⌫ is then analyzed as an equilibrium of G ⌫ . As is well known, any optimal play for ⌫ is the play induced by some subgame perfect equilib-Ž . Ž . rium of G ⌫ , and any subgame perfect equilibrium of G ⌫ corresponds to an optimal strategy. This property has the consequence that the backward induction algorithm is a procedure for solving a decision problem with perfect information. An analogous result is valid for decision problems with perfect recall and imperfect information. In this case we use the solution concept of sequential equilibrium which combines sequential rationality with the requirement of consistent beliefs. The set of distributions over the terminal nodes generated by the sequential equilibria of Ž . G ⌫ is identical to the set of distributions generated by the optimal Ž . strategies of ⌫ see  Hendon, Jacobsen, and Sloth, 1993 .  These results are called the ''no single improvement'' property; a strategy is optimal if, conditional upon reaching an information set, the decision maker cannot improve his expected payoff by changing only the action prescribed by the strategy for that information set. It implies that it makes no difference whether we model a decision problem piecemeal or in its entirety. The equivalence of the single-self and the multiselves approaches for decision problems with perfect recall breaks down when we analyze decision problems with imperfect recall. Consider Example 3. The Ž . optimal strategy R, r corresponds to a sequential equilibrium in the corresponding multiself game. However, the multiself approach does not Ž . rule out the inferior strategy L, l ; the decision maker cannot improve his payoff by simply changing the action prescribed at a single information set. Conditional upon being at d , the choice of L is optimal if the decision 1 maker treats his behavior at d as given and unchangeable and believes he 2 will play l. The Paradox of the Absentminded Dri¨er Re-re¨isited. Consider again the paradox of the absentminded driver. The formal notion of sequential equilibrium is not applicable to decision problems with absentmindedness. However, the requirement that the behavioral strategy at each information set is optimal, given the consistent beliefs, is meaningful. Despite the fact that there is only one information set, this requirement generates a strong dependence between behavioral strategies and beliefs. As we have noted before, the only strategy which satisfies this requirement is to exit with 5 probability , whereas the strategy that maximizes expected payoffs ex ante 9 1 is to exit with probability . Nevertheless, one can modify the multiself 3 approach to avoid the possibility that an optimal strategy does not satisfy sequential rationality with respect to consistent beliefs. Beliefs consistent with the optimal behavioral strategy in the paradox of 3 the absentminded driver assign probability to the first intersection. If the 5 decision maker anticipates that his ''twin-self,'' if asked to move again, will 1 1 exit with probability , then it is optimal for him to exit with probability 3 3 3 2 Ž w x w x as he is indifferent between E yielding the expected payoff 0 q 4 5 5 8 3 1 2 28 . Ž w Ž . Ž . x w x . s and B yielding an expected payoff of 4 q 1 q 1 s . In this modification, when considering a deviation at an information set X from the behavioral strategy assigned to X, the ''self'' assumes that his ''twin-self'' will use the equilibrium strategy if asked again to act, regardless of the choice he now makes. The requirement that the decision maker uses identical randomization at all histories in the same information set is retained only in equilibrium. In an actual play of the decision problem, if he decides to deviate, he may use different randomizations at different histories in the same information set. Also notice that, for the optimal strategy to satisfy sequential rationality in this modification of the multiself approach, the decision maker must 3 believe that he is at the first intersection with probability . In particular, if 5 one replaces our definition of consistency of beliefs with Z-consistency, sequential rationality fails in this modification as well. We now define the ''modified multiself'' approach formally and prove that every optimal Ž strategy is modified multiself consistent discussions with Bob Aumann and . Roger Myerson were very helpful in shaping this part of the paper .  DEFINITION . A behavioral strategy b is modified multiself consistent if there exists a belief consistent with b such that for every information set Ž . X which is reached with positive probability and for every action a g A X Ž .Ž . Ž . for which b h a ) 0 for h g X, there is no aЈ g A X such that h P z¬ h,aЈ ,b u z Ž . Ž . Ž . Ž . Ý Ý h gX z g Z ) h P z¬ h,a ,b u z . Ž . Ž . Ž . Ž . Ý Ý h gX z g Z Note that in contrast with the approach discussed in the first part of this section, the above definition imposes requirements only on the selves which are reached with positive probability. PROPOSITION 3. If a beha¨ioral strategy is optimal then it is modified multiself consistent. Proof. See the Appendix. Proposition 3 demonstrates that, within the framework of the ''modified'' multiself approach, the inconsistency of optimal plans for decision problems with absentmindedness evaporates. The interpretative ambiguities originating in the paradox of the absentminded driver also vanish as the 2 strategy to exit with probability is singled out as the only optimal and 3 consistent strategy. The value of this resolution, however, depends on the appropriateness of the ''modified'' multiself approach as a theory of decision making under imperfect recall. The behavioral assumption of the multiself approach is that each self maximizes his conditional expected payoff given his own beliefs. In decision problems with perfect recall, beliefs at each information set are independent of the choices of previous selves and each self can determine the optimal decisions of future selves by recursion. In decision problems with imperfect recall, however, the conditional payoff of each self depends on his beliefs about the behavior of earlier selves and the independence of beliefs and actions from previous choices is an additional assumption. For decision problems with absentmindedness, the ''modified'' multiself approach assumes that a decision maker, upon reaching an information set, takes his actions to be immutable at future occurrences of that information set, no matter which course of action he is contemplating now. At the other extreme one finds the opposite axiom for which only one self resides in the information set and expects that, were the information set to occur again, he would adopt whichever behavioral rule he adopts now. In this case, the strategy to exit 5 with probability would be the consistent rule of behavior for Example 1. 9 These two assumptions reflect two alternative ways of reasoning at the information set. We find both of them to have some intuitive appeal and neither to be universally valid.
 FINAL COMMENTS Our observations are only a limited exercise intended to suggest some of the issues which a comprehensive theory of imperfect recall must confront. We conclude our discussion with the following three comments: a. Other types of imperfect recall. We model imperfect recall by including in the same information set histories which contain different experiences. Such histories are assumed to be indistinguishable for the decision maker. A different type of imperfect recall is represented by a situation in which ''I know that at the first intersection I would be aware of this but at the second intersection I do not know whether I am at the first or the second intersection.'' One way to model such a scenario is by nonparti-Ž . Ž . tional informational structures see  Geanakopolos, 1990 . Rubinstein 1991  discussed the difficulty of modelling considerations such as ''At the first intersection I know that I am there but if I reach the second intersection I Ž think that there is a 10% chance that I am at the first intersection'' see . also Fluck, 1994 . It seems to us that new analytical frameworks are needed to address these issues as well as issues such as partial recall of strategy and imperfect ability to make inferences. b. Games with imperfect recall. This paper focuses on issues related to modeling decision problems with imperfect recall. Obviously, such issues carry over to extensive games. Isbell and Kuhn show that the nonequivalence between mixed and behavioral strategies can cause nonexistence of Nash equilibria in behavioral strategies. Time inconsistency of optimal strategies for some decision problems with imperfect recall makes it possible to construct games with imperfect recall that have no equilibria which satisfy sequential rationality with respect to consistent beliefs. Ambiguities in the interpretation of games with imperfect recall is probably the reason why only a handful of papers have dealt with the topic. Ž . Ž . The few exceptions include  Isbell 1957 and Alpern 1988  , who prove the existence of mixed behavioral strategy equilibrium in a class of games with Ž . imperfect recall, and Binmore 1992 who relates imperfect recall and nonpartitional knowledge. The literature on machines playing repeated Ž . games see, for example, Rubinstein, 1986 may also be interpreted as an analysis of a class of repeated games with imperfect recall, where it is costly for the players to distinguish between different histories of the game. Further investigation is needed to adapt solution concepts designed for games with perfect recall to games with imperfect recall. c. The paradox of the absentminded dri¨er. In all its forms the absentminded driver example exhibits a conflict between two types of reasoning. Commitment to the ex ante optimal behavioral strategy is obviously the normative rule of behavior. We do not have a firm view about the resolution of the paradox. We have investigated one resolution which requires dividing a decision maker into multiple independent selves. Another resolution would entail the rejection of expected utility maximization given consistent beliefs when the information set includes histories whose probabilities depend on the decision maker's actions at that information set. Savage's theory views a state as a description of a scenario which is independent of the act. In contrast, ''being at the second intersection'' is a state which is not independent from the action taken at the first, and, consequently, at the second intersection.
 APPENDIX Proof of Proposition 1. Consider an information set X and h g X. Let Ž .Ž . b be a behavioral strategy and b h a denote the probability of choosing Ž . a g A h at h. Since ⌫ does not exhibit absentmindedness, the expected Ž .Ž . payoff from b can be written as Ý b h a g q g, where each Suppose that ⌫ exhibits absentmindedness. Then, there exist a final Ž . Ž . history z and two distinct subhistories of z, h, a and hЈ, aЈ , such that h and hЈ belong to the same information set and a / aЈ. Consider a payoff Ž . Ž . function uЈ such that uЈ z s 1 and uЈ z s 0 for z / z. The claim follows, since z cannot be reached by any pure strategy, and is reached with positive probability by any behavioral strategy which assigns a positive probability to each action. Q.E.D.   FIG. 1 . 1 FIG. 1. Example 1.
 FIG. 2 . 2 FIG. 2. Example 2.
 FIG. 4. Example 4.
 1 and the strategy that selects B with probability equal to . A consistent 2 2 belief, conditional upon the information set d , assigns probability to 1 3 Ž . being at the first intersection. Consider next Example 5 see Fig. 5 and the strategy which selects B with probability 1.
 FIG FIG. 5. Example 5.
 R is . If p s 1, the conditional probability of each of the 4 1 three nodes in d is .
 9 The problem of time consistency can also arise without absentmindedness as Example 2 shows. The optimal strategy is to choose S at d , B at1 d , and R at d . However, upon reaching d the decision maker prefers 2 3 1 using B at d and L at d . A decision maker who can postpone his 1 3 decisions until he has reached d or d can make a better decision 1 2 regarding the choice at d than a decision maker who decides ex ante. A 3 complete model should indicate if the domain of choice of the decision maker includes timing of the decision and the extent of his ability to use his knowledge at d and d to influence his choice at d .
 g and g are independent of b h . Consider a) g A h which a Ž .Ž . achieves the highest g . Setting b h a) s 1 yields a payoff at least as a high. The claim follows by repeating the argument for every information set.
 2. Let b denote the payoff that the decision Ž . maker obtains by playing b. Given an information set X, let Z X denote Ž . the set of final histories having subhistories in X. Then, we can write b as < b s p z ¬ b u z q p z b u z .
 Since ⌫ does not exhibit absentmindedness, a terminal history z in Z X Ž < . Ž < . Ž < . has a unique subhistory h in X. Thus, p z b s p h b p z h, b and
 See the Appendix. Both histories L, B and R, B are in d , but exp L, B s ,d and exp R, B s d , B, d . ) is violated, since at d the Ž . To clarify the meaning of condition ) in Proposition 2, consider first Ž Example 2. 3 . Ž . Ž . Ž d,B 1 3 . Ž . Ž 2 3 . Ž . 3 decision maker loses information about the move of chance. In Example 6, even though at no information set the decision maker knows nature's Ž . move, ) is violated since exp L s L, d Ž . Ž 2 . Ž and exp R, B s . Ž . R ,d,B ,d . Notice that, however, if the decision maker had beliefs 1 2 consistent with the strategy assigning B to d and S to d , he would assign 1 2 1 FIG. 6. Example 6.Proof.