Video: Dystopian Futures of Astronomical Suffering | Documentary about S-risks and Longtermism

From ancient prophecy to science fictionÂ  humans have long been fascinated by the future.Â Â  In recent years however a new academic fieldÂ  known as long-termism has risen to prominenceÂ Â  and captured the imagination ofÂ  a new generation of thinkers.Â Â  The central insights of longtermism is thatÂ  while life in the early 21st century may seemÂ Â  like the apex of human civilization, we may inÂ  fact be living at the very beginning of history.Â Â  There is a real possibility that our descendantsÂ  and their creations will go on to inhabit a trulyÂ Â  astronomical future. The philosopher Toby OrdÂ  has argued that there is a roughly even chanceÂ Â  that humanity will survive our present turbulentÂ  era of existential vulnerability and go on toÂ Â  author feats of creativity and ingenuity inÂ  the universe the we can scarcely comprehend.Â Â  The future of Earth originatingÂ  life could be unimaginably fast.Â Â  For example, Earth will likely be habitable forÂ  another billion years, and the solar system forÂ Â  another four to five billion years. Long-distanceÂ  space travel will present enormous challengesÂ Â  and yet there is a reasonable likelihood thatÂ  in some form our descendants will eventuallyÂ Â  reach neighboring star systems and thereÂ  establish new bastions of civilization.Â Â  From here, our entire galaxy would come withinÂ  reach. Using ships traveling at just a fraction ofÂ Â  the speed of light our descendants could colonizeÂ  the Milky Way galaxy in around 100 million years.Â Â  This might seem like a very long time to us, butÂ  in cosmological time, 100 million years is no moreÂ Â  than the blink of an eye. Indeed, in the universe,Â  time is a resource that exists in great abundance.Â Â  We live at the very beginning of what is knownÂ  as the Stelliferous era - a great epoch inÂ Â  the life of the universe in which stars shineÂ  and the universe is composed of galaxies thatÂ Â  cluster into vast galactic filaments. If starsÂ  provide the energy of advanced civilizations,Â Â  then our descendants could endure for the entireÂ  duration of the stelliferous era - approximatelyÂ Â  100 trillion years. Though it is far fromÂ  certain it is conceivable that in the eonsÂ Â  to come our descendants will eventually inhabitÂ  billions of galaxies and that we living now willÂ Â  be the progenitors of countless trillionsÂ  of minds. Considering such grand futuresÂ Â  fundamentally recontextualizes our present momentsÂ  in history. It places new and profound emphasisÂ Â  on our responsibility to survive our currentÂ  perilous era with our vast potential intact.Â Â  Indeed, the existence of trillionsÂ  of potential beings may depend on theÂ Â  actions of a comparatively tiny number ofÂ  human beings over a few short generations. But there is another side to this story. InÂ  addition to our potential for unimaginableÂ Â  feats of prosperity and flourishing,Â  there also exist darker possibilities.Â Â  These darker possibilities are oftenÂ  overlooked because, among other reasons,Â Â  humans have various persistent biases againstÂ  thinking deeply about worst case outcomes.Â Â  Nevertheless it is a disconcerting fact thatÂ  within the vastness of our potential reachÂ Â  and scope there are possible futures that may beÂ  considered worse than our extinction - much worse.Â Â  Considering these worst case scenariosÂ  can be challenging and even disturbingÂ Â  and yet in order to truly face ourÂ  great responsibility to the futureÂ Â  we must also guard against the possibilityÂ  of extremely bad futures - failed worlds. [Music] The term dystopia is used to describe a highlyÂ  undesirable future scenarios. Today we will focusÂ Â  on a specific category of dystopian scenario knownÂ  as "s-risks" an abbreviation of "suffering risks"Â Â  The general definition of an s-risk is a scenarioÂ  that causes astronomically significant amountsÂ Â  of severe suffering. In most ethical and valueÂ  systems it is generally considered uncontroversialÂ Â  the severe suffering is bad and should beÂ  avoided. But another core value in this areaÂ Â  is impartiality - that suffering matters equallyÂ  irrespective of who experiences it includingÂ Â  non-human animals. Furthermore, suffering mattersÂ  equally regardless of when it is experienced.Â Â  There is, after all, no morally relevantÂ  distinction between different points in time.Â Â  To discriminate against future individualsÂ  is no different from discrimination basedÂ Â  on skin color, place of birth, orÂ  other purely contingent factors.Â Â  Sentient beings living in the future areÂ  no less and no more deserving of moralÂ Â  consideration than beings alive now. Therefore,Â  the fact that a moral catastrophe will occur inÂ Â  the distant future does not reduce the importanceÂ  of preventing it if we have the ability to do so. A key focus in the study of s-risks isÂ  the possibility that future technologicalÂ Â  developments, in particular advancedÂ  artificial intelligence and large-scaleÂ Â  space colonization will vastly increaseÂ  the scope of our civilization, and couldÂ Â  multiply the number of sentient beingsÂ  in existence by many orders of magnitude.Â Â  While such an astronomical scope containsÂ  the potential for extremely positive futures,Â Â  there is also the potential for truly failedÂ  worlds - when the great scope of the futureÂ Â  becomes the stage of moral catastrophesÂ  of truly astronomical proportions.Â Â  Today we will consider various potential s-risks,Â  not for the purpose of eliciting fear or deterringÂ Â  technological progress, but to raise awareness ofÂ  s-risks so that we can take steps now to preventÂ Â  them from happening. We will explore some ofÂ  these steps in the final part of the video. S-risks are scenarios that involve severeÂ  suffering on a scale that vastly exceeds allÂ Â  suffering that has existed on Earth so far. ButÂ  how could such a moral catastrophe come to pass?Â Â  Why would we or our descendants allow this toÂ  happen? As we will now explore, there is a dangerÂ Â  that future actors may simply not care about, orÂ  fail to notice, the suffering their actions cause.Â Â  This brings us to our firstÂ  category: incidental s-risks.Â Â  Incidental s-risks arise when theÂ  most efficient path to achievingÂ Â  a goal creates large amounts of severeÂ  suffering as an unintended side effect.Â Â  A modern day example might be the sufferingÂ  inflicted on animals through factory farming.Â Â  Humanity raises and kills vast numbers of animalsÂ  each year on factory farms and in slaughterhouses,Â Â  often inflicting terrible suffering on them in theÂ  process. In the U.S Alone 10 billion animals areÂ Â  slaughtered every year and 99% of them are raisedÂ  in the horrifying conditions of factory farms.Â Â  As Baumann writes, (quote) "It just soÂ  happens that the most economically efficientÂ Â  way to satisfy the growing demand for cheap animalÂ  products entails miserable conditions for billionsÂ Â  of sentient beings. It is worth noting that thisÂ  is not due to intentional malice. After all,Â Â  most people do not approve of animal suffering.Â  Rather, factory farming is the result of economicÂ Â  incentives and technological feasibility coupledÂ  with a lack of moral concern." (end quote) WeÂ Â  can imagine an incidental s-risk as a futureÂ  development akin to factory farming but on anÂ Â  even more horrendous scale. Indeed one possibilityÂ  of an s-risk is that factory farming is neverÂ Â  abolished and grows alongside the burgeoningÂ  scope of an earth-originating civilization.Â Â  As the philosopher Magnus Vinding writes,Â  (quote) "An extension of factory farming intoÂ Â  space is a concrete example of how an s-risk mightÂ  materialize, that neither history nor contemporaryÂ Â  events allow us to dismiss out of hand."Â  (end quote) Researchers in this field howeverÂ Â  consider that even more serious s-risks couldÂ  accompany the creation of artificial sentience.Â Â  I have explored the concept of artificial minds inÂ  other videos. If and how they might be possible isÂ Â  a subject of significant disagreement amongÂ  scientists and philosophers. One point ofÂ Â  contention is whether a purely computationalÂ  system could ever be truly conscious.Â Â  The central issue is that we lack a clearÂ  understanding of how brains become conscious, andÂ Â  the specific role of computation in that process.Â  While we may have doubts about the possibility ofÂ Â  digital sentient minds, we do not know enoughÂ  to rule them out. Furthermore, artificial mindsÂ Â  need not be digital. Our brains, after all, areÂ  conscious. How this occurs can presumably beÂ Â  discovered, along with the means of replicatingÂ  these conditions. At this stage we should remainÂ Â  open to the possibility that fully sentientÂ  artificial minds could be created in the future.Â Â  So let us consider this possibility. ResearchersÂ  have speculated that if they are possible,Â Â  artificial minds could be createdÂ  in enormous numbers in the future.Â Â  One reason for this is that artificial mindsÂ  may have various significant advantages overÂ Â  biological ones. Digital Minds, for example,Â  could potentially be duplicated with theÂ Â  same ease that we now make copiesÂ  of software or computer programs.Â Â  If so, it could be much easier and cheaper toÂ  create artificial minds than biological ones.Â Â  The creation of these minds would conceivablyÂ  permit unprecedented scientific breakthroughs.Â Â  Advanced AI systems, for example, might run manyÂ  simulations in order to broaden their knowledgeÂ Â  of human psychology, or an attempt to predictÂ  what other agents will do in certain situations.Â Â  Another incentive to create artificialÂ  minds is that, unlike biological minds,Â Â  they are not vulnerable to aging or death.Â  If the technology is available biologicalÂ Â  individuals might seek to upload theirÂ  minds as a way to avoid these limitations.Â Â  Digital minds may also have significantÂ  advantages for space colonization.Â Â  Enormous numbers of such minds could existÂ  anywhere that computers can exist - including theÂ Â  freezing vacuum of space. It is conceivable thatÂ  an advanced civilization will build computationalÂ Â  megastructures around stars that are capableÂ  of simulating vast numbers of artificial minds.Â Â  In the literature, these megastructuresÂ  are known as "matryoshka brains."Â Â  There is, of course, the possibility that many ifÂ  not most of these minds will have overwhelminglyÂ Â  positive and worthwhile experiences. UnfortunatelyÂ  however, there are reasons to think that just likeÂ Â  non-human beings now, artificial mindsÂ  of various potential forms, will not beÂ Â  treated with due moral consideration and theirÂ  potential suffering will pose a serious s-risk.Â Â  As Baumann writes, (quote) Many forms ofÂ  artificial beings will likely be very alienÂ Â  to us and therefore pose a unique challenge toÂ  our moral sentiments. Such disembodied mindsÂ Â  may have no face, body movements, or screams toÂ  which we can relate. That makes it difficult toÂ Â  empathize with them on a visceral level. InÂ  addition, we do not yet have a reliable wayÂ Â  to detect sentience, especially in systems thatÂ  are fundamentally different from human brains.Â Â  We might therefore, fail to recognize sentienceÂ  and suffering in such voiceless beings.Â Â  The combination of potentially vast numbers ofÂ  sentient artificial minds and the foreseeableÂ Â  lack of empathy poses a serious s-risk.Â  In fact, these conditions look strikinglyÂ Â  similar to those of factory farming, whichÂ  is also characterized by economic expediencyÂ Â  in combination with moral indifferenceÂ  towards non-human beings." (end quote) The potential suffering of artificial mindsÂ  is just one example of an incidental s-riskÂ Â  and various other possibilities have beenÂ  explored by researchers. We will now however,Â Â  move on to our second category of s-risks,Â  in which enormous magnitudes of severeÂ Â  suffering are created ... deliberately. TheseÂ  scenarios are referred to as "agenital s-risks."Â Â  But why would anyone deliberately seek toÂ  cause astronomical amounts of suffering?Â Â  Unfortunately the seeds of these worst caseÂ  scenarios lie deep within human psychology.Â Â  These traits are sometimesÂ  referred to as "The Dark Tetrad."Â Â  They include narcissism, psychopathy,Â  sadism, and machiavellianism. Unfortunately,Â Â  the negative effects of these traits are clearlyÂ  visible in our world today. All too often theÂ Â  power structures of our civilization rewardÂ  the expression of these dark traits, allowingÂ Â  malevolent actors to rise into positions of power.Â  Ancient human traits of tribalism, retributivism,Â Â  and sadism are associated with some of theÂ  worst atrocities in the history of our species.Â Â  If the influence of these traits persistsÂ  into the future, then the development ofÂ Â  advanced technology, coupled with a much greaterÂ  scope of our civilization will raise the stakesÂ Â  dramatically and present a significant danger ofÂ  agential s-risks. Perhaps the greatest risks areÂ Â  associated with war and violent conflict. OneÂ  highly perilous conflict dynamic occurs whenÂ Â  agents, as part of an escalating conflict or war,Â  threatened to deliberately bring about worst-caseÂ Â  outcomes in order to force the other side toÂ  yield. This strategy is all to apparent inÂ Â  our world today, when leaders threaten the use ofÂ  nuclear weapons to help them achieve their goals.Â Â  One possibility is that agentialÂ  s-risks will have non-human origins.Â Â  A key example is artificial intelligence. AdvancedÂ  AI is increasingly thought to pose a seriousÂ Â  existential risk to human civilization. Powerful,Â  smarter-than-human AIs might break free of allÂ Â  human control. If their values are not alignedÂ  with our own, the results could be catastrophic.Â Â  Much of the potential suffering caused by rogueÂ  AIs may be incidental, as it runs roughshod overÂ Â  our interests in pursuit of its own goals. And yetÂ  it is also conceivable that AIs could deliberatelyÂ Â  cause suffering. Among various strategies, AIsÂ  might adopt alien equivalents of dark tetradÂ Â  traits, and use our vulnerability to sufferingÂ  as a way to manipulate us. Just like malevolentÂ Â  human actors, AIs might force our cooperationÂ  with threats of worst case outcomes. AdvancedÂ Â  artificial intelligence is therefore a potentialÂ  cause of both incidental and agential s-risks.Â Â  We will return to AI-related risks later,Â  as well as what we might do to prevent,Â Â  them but we'll now move on to the finalÂ  category of s-risks - those which arise fromÂ Â  natural origins. These are referred to as naturalÂ  s-risks. Natural s-risks refer to the possibilityÂ Â  that natural suffering takes place, or will takeÂ  place in the future, on an astronomical scale.Â Â  Natural s-risks are a controversial categoryÂ  because they call for us to recognize that nature,Â Â  by itself, produces enormous amounts of severeÂ  suffering. Many of us would prefer to believeÂ Â  that nature is beautiful and idyllic, howeverÂ  it is naive to overlook the tremendous sufferingÂ Â  caused by darwinian evolution. There isÂ  a possibility that, despite appearances,Â Â  life is evolving in many places throughout theÂ  universe. If this life is sentient, and suffersÂ Â  comparably to that of life on Earth, this can beÂ  viewed as a natural s-risk. Some futurists argueÂ Â  that advanced civilizations may one day interveneÂ  in the workings of evolution, perhaps evenÂ Â  seeking to eliminate suffering altogether. TheÂ  philosopher David Pearce argues that the decisionÂ Â  of our descendants, to voyage through space,Â  could be motivated, not by pure exploration,Â Â  but in response to natural s-risks - grand rescueÂ  missions to deliver life elsewhere from suffering.Â Â  With the best intentions, our descendants mayÂ  attempt to seed other worlds with darwinian life.Â Â  They might believe like some environmentalists,Â  the darwinian life has great value just as itÂ Â  is, and that we have a moral imperativeÂ  to spread it throughout the universe.Â Â  In addition to any value, the potential sufferingÂ  that would result from such a venture amounts toÂ Â  a serious s-risk, though strictly speaking,Â  this would be an incidental s-risk ratherÂ Â  than a natural one, because it occurs as aÂ  side effect in the pursuit of other goals. These are just some of the s-risks thatÂ  researchers have considered. However,Â Â  we must also be aware of the potentialÂ  for unknown s-risks that have not yetÂ Â  been thought of, or that we cannot,Â  at this stage, even conceive of.Â Â  Just as scholars in the Middle Ages couldÂ  hardly have anticipated the atomic bomb,Â Â  future developments might raise the possibilityÂ  of s-risks that we cannot yet imagine. Preventing S-Risks The scenarios we have considered may seemÂ  far-fetched or something out of science fiction,Â Â  and yet the basic criteria for s-risks seem allÂ  too plausible. Most s-risks depend only on theÂ Â  possibility that humanity is, at present, close toÂ  the beginning of this potential reach and scope,Â Â  and that the future will contain many sentientÂ  beings. This all seems possible if not likely,Â Â  so what can we do to prevent s-risks?Â  Fortunately, there are several reasonsÂ Â  to believe that we live at a unique time, inÂ  which we could have significant impact on theÂ Â  future potential of s-risks. As Baumann writes,Â  (quote) "The observation that we are still on aÂ Â  single planet that could potentially originate aÂ  vast cosmic civilization suggests that we mightÂ Â  indeed be in a unique position. If a "long" orÂ  "big" future happens, then almost all individualsÂ Â  will live in the future - which means that weÂ  can influence them but they cannot influence us."Â Â  So what can we actually do to avert s-risks? ManyÂ  of us will feel attention between responding toÂ Â  s-risks and the urgent need to reduceÂ  severe suffering in our present time.Â Â  Can we really prioritize speculative sufferingÂ  in the distant future when there is very realÂ Â  and preventable suffering in the hereÂ  and now? Simply put, we must do both.Â Â  Fortunately, there are things we can do nowÂ  that can help reduce suffering in the present,Â Â  as well as reducing the possibility of sufferingÂ  on an astronomical scale in the future.Â Â  Earlier, we explored how the creation of sentientÂ  artificial minds could present a significantÂ Â  danger of s-risks. One step we can take to preventÂ  this form of s-risk, as well as responding toÂ Â  present day suffering, is to advocate forÂ  moral circle expansion. In particular, weÂ Â  can work to end speciesism - the discrimination ofÂ  sentient beings who belong to a different species.Â Â  The sheer suffering caused byÂ  factory farming is viewed byÂ Â  many ethical theorists as among the mostÂ  pressing moral emergencies of our time.Â Â  One of the best paths to both end thisÂ  practice and protect future non-humanÂ Â  beings is to advocate for the end of speciesism.Â  In particular, we should work to expand ourÂ Â  collective moral circle to include all beingsÂ  capable of suffering - now and in the future. One other reason why thinkers considerÂ  that our time may have a uniquely greaterÂ Â  influence on the future is the near-termÂ  potential of pivotal technologies.Â Â  The emergence of these pivotal technologies couldÂ  plausibly shape events from that point onwards.Â Â  Researchers argue that the conditions inÂ  which these technologies are conceivedÂ Â  could be the crucial difference between extremelyÂ  positive and extremely negative future outcomes.Â Â  Central among these technologiesÂ  is artificial intelligence.Â Â  Broad-spectrum artificial general intelligence hasÂ  been likened to an advanced alien civilization onÂ Â  route to our planet, for which we have onlyÂ  limited time to prepare. It may be just aÂ Â  matter of years before we share our world withÂ  alien intelligences vastly greater than our own.Â Â  The arrival of this technology couldÂ  dramatically improve life in numerous ways.Â Â  It could even serve to help prevent the occurrenceÂ  of s-risks. And yet while AI could propel us intoÂ Â  a prosperous utopian future, it could plausiblyÂ  cause catastrophic disruption to our civilizationÂ Â  and, quite possibly, our extinction. AI is aÂ  risk factor in many if not most s-risk scenarios.Â Â  It could, for example, rapidly accelerateÂ  the creation of artificial sentience, or fallÂ Â  into the hands of malevolent actors. We can andÂ  should take action to prevent AI-related s-risks Research on AI safety is a small but growingÂ  field, and yet there has been even less researchÂ Â  into how AI might cause s-risks. Perhaps, ratherÂ  than ensuring that advanced AI will share ourÂ Â  present values, we should focus on making sureÂ  that it has better values than us. For example,Â Â  by giving due moral consideration to all sentientÂ  beings. More specifically, we need to do moreÂ Â  research on how AIs could develop malevolentÂ  traits. For example, just like dark tetradÂ Â  traits evolved in human evolution, AI trainingÂ  environments, with multiple competing agents,Â Â  could incentivize alien equivalence of these darkÂ  tetrad traits in the motivations of AI systems.Â Â  One upside of the comparative lack of researchÂ  into AI-related s-risks is that we should expectÂ Â  to find considerable low-hanging fruit forÂ  researchers wanting to make a difference inÂ Â  this field. Another pivotal technology is theÂ  potential of large-scale space colonization,Â Â  which would have the effect of astronomicallyÂ  increasing the scope of our civilization.Â Â  Many potential s-risks wouldÂ  follow such a development.Â Â  A promising area for reducing such s-risksÂ  is the establishment of space governance.Â Â  We currently lack a global frameworkÂ  for space governance. As of now,Â Â  space is mostly a free-for-all, which poses a riskÂ  of race dynamics and severe conflicts. It wouldÂ Â  be extremely valuable to replace the currentÂ  state of ambiguity with coherent regulationsÂ Â  that favor positive long-term outcomes if and whenÂ  large-scale space colonization becomes possible. Earlier, we discussed how s-risks could resultÂ  from the rise of malevolent actors into positionsÂ Â  of power. There are however, political reformsÂ  that can help guard against this risk factor.Â Â  One in particular is the promotion of democracy.Â  Compared to other systems of government,Â Â  modern liberal democracies offer a much betterÂ  protection of human rights and civil libertiesÂ Â  like free speech. These fundamental rightsÂ  are a precondition of being able to raiseÂ Â  moral concerns, which is one reason why theirÂ  frequent suppression in autocratic systemsÂ Â  presents a considerable danger of future s-risks.Â  While democracies are not always successfulÂ Â  in preventing malevolent rule they appearÂ  better than any other system in this regard.Â Â  It is also the case that war between democraticÂ  countries is uniquely rare. Therefore,Â Â  promoting and defending democracy is likely toÂ  be an effective strategy for reducing s-risks.Â Â  Another proposal is the idea of extendingÂ  political representation to all sentient beings,Â Â  including non-human animals. Non-human animalsÂ  cannot represent their own interests, but we couldÂ Â  for example, hire commissioners whose sole taskÂ  is to defend the interests of non-human animals.Â Â  This has been termed "sentientistÂ  democracy" or "sentiocracy."Â Â  Another intervention would be to provide politicalÂ  representation to future individuals - anotherÂ Â  class of sentient beings that go entirelyÂ  unrepresented in current institutions.Â Â  Ultimately, one of the most importantÂ  activities for reducing s-risks isÂ Â  further research. We can either get involvedÂ  in this research ourselves, or we can donateÂ Â  to organizations which are already doingÂ  it. Two such organizations are the CenterÂ Â  for Reducing Suffering and the OrganizationÂ  for the prevention of Intense Suffering. Our potential is vast beyond imagination,Â  and the significance of our time,Â Â  in shaping the long-term future,Â  may be far greater than we realize.Â Â  Now more than ever, we must seek to close the gapÂ  between our wisdom and our accelerating power.Â Â  The vision of long-termism, with its astronomicalÂ  scope, inspires us to reflect on our potential andÂ Â  what our descendants might eventually author onÂ  the cosmic stage. And yet it also highlights ourÂ Â  great moral responsibility to the future, and theÂ  countless sentient beings that could come afterÂ Â  us. All of us hope for a positive future, but inÂ  order to take our responsibility to the futureÂ Â  seriously we must also reflect on how thingsÂ  could go tragically wrong, and what we can doÂ Â  now to prevent worst-case scenarios.Â  The stakes are too high to ignore. Hello everyone! Thank you for watching thisÂ  episode of Waking Cosmos. This video wasÂ Â  inspired by the fascinating book Avoiding theÂ  Worst by Tobias Baumann. It contains a muchÂ Â  more thorough summary of s-risks, as well as whatÂ  we can do to prevent them. This book is availableÂ Â  entirely free in the description of this video,Â  including an audiobook version which I narrated.Â Â  So if you want to listen to that, the link is inÂ  the description. Thank you again for watching.Â Â  Please don't forget to "like" and subscribe forÂ  more episodes, and I look forward to chattingÂ Â  with you all in the comments. Alright,Â  that is about it from me. Until next time.