Abstract: The material and the geographical designations in this report do not imply the expression of any opinion whatsoever on the part of Global Challenges Foundation concerning the legal status of any country, territory, or area, or concerning the delimitation of its frontiers or boundaries.

Academic Project Manager, Andrew Snyder-Beattie

Member of the National Expert Panel on Climate Change and National Foreign Policy Advisory Committee, China

The main authors of this report are Dennis Pamlin, Executive Project Manager, Global Challenges Foundation and Dr Stuart Armstrong, James Martin Research Fellow, Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford. Dr Stuart Armstrong wrote the chapter covering the twelve global challenges, under the direction of Dennis Pamlin who served as project manager and himself wrote and edited the rest of the report. Seth Baum, Executive Director of the Global Catastrophic Risk Institute and affiliate researcher at the Center for Research on Environmental Decisions, Columbia University, also played an important role as he helped develop the methodology chapter regarding the selection of the global challenges with potentially infinite impacts as well as providing helpful input throughout the process. The report is the result of a collaborative approach where many people have provided invaluable contributions. The authors would therefore like to thank a few people in particular. First and foremost László Szombatfalvy, Chairman of the Global Challenges Foundation, whose work is the basis for this report and whose guidance on all levels has been invaluable. The rest of the board of the Global Challenges Foundation have also contributed in many different ways, in particular, Johan Rockström has provided important input regarding the structure and methodology. Outside the foundation Prof Nick Bostrom, Professor & Director of the Future of Humanity Institute, Oxford Martin School & Faculty of Philosophy, University of Oxford, who initiated the possibility of working with the Future of Humanity Institute at the University of Oxford, played a particularly important role. Patrick McSharry, Head of Smith School's Catastrophe Risk Financing research area, provided invaluable input regarding complex systems and ways that the economic system can respond to infinite impacts. Alex Kirby also played a key part as he did so much more than proofread the text; the report would hardly be possible to read without his help. Various additional edits and changes were made by Peter Brietbart. Others that must be mentioned, including those who participated in the workshop on 14 January 2014, at the Future of Humanity Institute (FHI), University of Oxford and the workshop at the Munich RE office in London on 15 January 2014, and helped provide input regarding the economic and finance aspects, include (in alphabetical order): Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks This is the executive summary of a report about a limited number of global risks that pose a threat to human civilisation, or even possibly to all human life.
 Summary Executive 4 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Executive Summary
 History: the LA-602 document With such a focus it may surprise some readers to find that the report's essential aim is to inspire action and dialogue as well as an increased use of the methodologies used for risk assessment. The real focus is not on the almost unimaginable impacts of the risks the report outlines. Its fundamental purpose is to encourage global collaboration and to use this new category of risk as a driver for innovation. The idea that we face a number of global challenges threatening the very basis of our civilisation at the beginning of the 21st century is well accepted in the scientific community, and is studied at a number of leading universities. I However, there is still no coordinated approach to address this group of challenges and turn them into opportunities. It is only 70 years ago that Edward Teller, one of the greatest physicists of his time, with his back-of-the-envelope calculations, produced results that differed drastically from all that had gone before. His calculations showed that the explosion of a nuclear bomb -a creation of some of the brightest minds on the planet, including Teller himself -could result in a chain reaction so powerful that it would ignite the world's atmosphere, thereby ending human life on Earth. Robert Oppenheimer, who led the Manhattan Project to develop the nuclear bomb, halted the project to see whether Teller's calculations were correct. The resulting document, LA-602: Ignition of the Atmosphere with Nuclear Bombs, concluded that Teller was wrong. But the sheer complexity drove the assessors to end their study by writing that "further work on the subject [is] highly desirable". The LA-602 document can be seen as the first global challenge report addressing a category of risks where the worst possible impact in all practical senses is infinite.
 Global risks The report conducts its exploration within carefully defined bounds, resulting in a list of twelve risks with potentially infinite outcomes. There were many challenges which might have been included on the list because of their ability to pose severe damage to humanity. They were excluded for one or more of three reasons: 1. Limited impact -tsunamis, for example, and chemical pollution.
 No effective countermeasures - the report focuses on promoting effective interventions and so ignores challenges where nothing useful can be done to prevent or mitigate the impact, as with nearby gamma-ray bursts. 3. Included in other challengesmany challenges are already covered by others, or are very similar to them. Population growth, for one, is significant for climate change and ecosystem catastrophe, but without direct large-scale impacts of its own. It is worth noting that complex systems are often stable only within certain boundaries outside which the system can collapse and rapidly change to a new stable state. Such a collapse can trigger a process where change continues for a long time until a new stable state is found. None of the risks in this report are likely to result directly in an infinite impact, and some cannot do so physically. All the risks however are big enough to reach a threshold where the social and ecological systems become so unstable that an infinite impact could ensue. This is a report about two extremes, not one. It is about how a better understanding of the magnitude of the challenges can help the world to address the risks it faces, and can help to create a path towards more sustainable development. It is a scientific assessment about the possibility of oblivion, certainly, but more than that it is a call for action based on the assumption that humanity is able to rise to challenges and turn them into opportunities. We are confronted with possibly the greatest challenge ever and our response needs to match this through global collaboration in new and innovative ways. This report has, to the best of the authors' knowledge, created the first list of global risks with impacts that for all practical purposes can be called infinite. It is also the first structured overview of key events related to such challenges and has tried to provide initial rough quantifications for the probabilities of these impacts. In the next phase of the project, these placeholder estimates will be improved and refined by a variety of methods (expert elicitation, fault trees, simulations, etc.) appropriate to each specific risk.
 The goals of the report The first of the report's goalsacknowledging the existence of risks with potentially infinite impactseeks to help key stakeholders to acknowledge the existence of the category of risks that could result in infinite impact, and to show them that we can reduce or even eliminate most of them. The second goal is to inspire by showing the practical action that is taking place today. This report seeks to show that helping to meet these global challenges is perhaps the most important contribution anyone can make today, and highlights concrete examples to inspire a new generation of leaders. The third goal is to connect different groups at every level, so that leaders in different sectors connect with each other to encourage collaboration. This will need a specific focus on financial and security policy, where significant risks combine to demand action beyond the incremental. The fourth goal is to deliver actual strategies and initiatives that produce actual results. The report is a first step and its success will ultimately be measured only on how it contributes to concrete results. The report will have achieved its goals when key decision-makers recognise the magnitude of the possible risks and our ability to reduce or even eliminate most of them. The four main goals of this report are to acknowledge, inspire, connect and deliver.
 Report structure The second part is an overview of the twelve challenges and key events that illustrate strategic work to address them. It also lists for each challenge five important factors that influence its probability or impact. The challenges are divided into four different categories: -current challenges includes those which currently threaten humanity because of its economic and technological development; -exogenic challenges are those where the basic probability of an event is beyond human control, but where the probability and magnitude of the impact can be influenced; -emerging challenges could both help reduce the risks associated with current challenges and also result in infinite impacts; -the last of the twelve challenges are global policy challenges, threats arising from future global governance as it resorts to destructive policies in response to the categories of challenge listed above. The third part of the report discusses the relationship between the different challenges, as action to address one can increase the risk of another. Many solutions can also address multiple challenges, so there are significant benefits from understanding how they are linked. The fourth part is an overview, the first ever to the authors' knowledge, of the probabilities of global challenges with potentially infinite impacts. The fifth part presents some of the most important underlying trends that influence the challenges, which often build up slowly to a threshold where very rapid changes can ensue. The sixth part presents an overview of possible ways forward.
 The first part of the report introduces and defines the global challenges and includes the methodology for selecting them. A new category of global risk Risk Probability Impact = x For several reasons the potentially infinite impacts of the challenges in this report are not as well known as they should be. One reason is the way that extreme impacts are often masked by most of the theories and models used by governments and business today. Climate change is a good example, where almost all of the focus is on the most likely scenarios, and there are few public studies that include the low-probability high-impact scenarios. In most reports about climate impacts, those caused by warming beyond five or six degrees Celsius are omitted from tables and graphs. Other aspects that contribute to this relative invisibility include the fact that extreme impacts are difficult to translate into monetary terms, as they have a global scope and often require a time-horizon of a century or more. They cannot be understood simply by linear extrapolation of current trends, and they lack historical precedents. There is also the fact that the measures required to significantly reduce the probability of infinite impacts will be radical compared to a business-as-usual scenario. A scientific approach requires us to base our decisions on the whole probability distribution. The review of literature indicates that, under a business as usual scenario, new risks with potential infinite impact are probably inseparable from the rapid technological development in areas like synthetic biology, nanotechnology and AI. Most risks are linked to increased knowledge, economic and technical development that has brought many benefits. E.g. climate change is a result from the industrial revolution and fossil fuel based development. The increased potential for global pandemics is one consequence of an integrated global economy where goods and services move quickly internationally. Similar challenges can be expected for synthetic biology, nanotechnology and AI. There are remedies, including technological and institutional, for all risks. But they will require collaboration of a sort humanity has not achieved before, and the creation of systems which can deal with problems pre-emptively. It is important to understand that much of the knowledge and many tools that we have, and will develop, can be both a risk and a solution to risks depending on context. The idea that there may be risks where the impact can be described as infinite, defined as the end of human civilisation or even human life, is not new. However, it excites relatively little political or academic interest, and the way it is treated in popular culture makes a serious discussion more difficult.
 Infinite impacts and thresholds Normal Risks
 Threshold Traditional measures and tools applicable
 New Category Requires new measures and tools impact 0 probability Risk Probability Impact = x Using traditional economic tools is problematic and can generate disagreement over issues such as discounting, which the report examines in some detail, considering for example the role of tipping points. The report distinguishes between the concepts of infinite impact -where civilisation collapses to a state of great suffering and does not recover, or a situation where all human life ends -and infinite impact thresholdan impact that can trigger a chain of events that could result first in a civilisation collapse, and then later result in an infinite impact. Such thresholds are especially important to recognise in a complex and interconnected society where resilience is decreasing. A collapse of civilisation is defined as a drastic decrease in human population size and political/ economic/social complexity, globally plunge temperatures below freezing around the globe and possibly also destroy most of the ozone layer. The detonations would need to start firestorms in the targeted cities, which could lift the soot up into the stratosphere. The risks are severe and recent models have confirmed the earlier analysis. The disintegration of the global food supply would make mass starvation and state collapse likely. As for all risks there are uncertainties in the estimates, and warming could be much more extreme than the middle estimates suggest. Feedback loops could mean global average temperatures increase by 4°C or even 6°C over pre-industrial levels. Feedbacks could be the release of methane from permafrost or the dieback of the Amazon rainforest. The impact of global warming would be strongest in poorer countries, which could become completely uninhabitable for the highest range of warming. The likelihood of a full-scale nuclear war between the USA and Russia has probably decreased. Still, the potential for deliberate or accidental nuclear conflict has not been removed, with some estimates putting the risk in the next century or so at around 10%. A larger impact would depend on whether or not the war triggered what is often called a nuclear winter or something similarthe creation of a pall of smoke high in the stratosphere that would Mass deaths and famines, social collapse and mass migration are certainly possible in this scenario. Combined with shocks to the agriculture and biosphere-dependent industries of the more developed countries, this could lead to global conflict and possibly civilisation collapse. Further evidence of the risk comes from signs that past civilisation collapses have been driven by climate change. The uncertainties in climate sensitivity models, including the tail. The likelihood -or not -of global coordination on controlling emissions. The future uptake of low carbon economies, including energy, mobility and food systems. Whether technological innovations will improve or worsen the situation, and by how much. How relations between current and future nuclear powers develop. The probability of accidental war. Whether disarmament efforts will succeed in reducing the number of nuclear warheads. The likelihood of a nuclear winter. The long-term effects of a nuclear war on climate, infrastructure and technology. A new category of global risk. Executive Summary the damage and (unlike previous, localised collapses) the whole world is potentially at risk. It seems plausible that some human lifestyles could be sustained in a relatively ecosystem independent way, at relatively low costs. Whether this can be achieved on a large scale in practice, especially during a collapse, will be a technological challenge and whether it is something we want is an ethical question. This is where an ecosystem suffers a drastic, possibly permanent, reduction in carrying capacity for all organisms, often resulting in mass extinction. Humans are part of the global ecosystem and so fundamentally depend on it. Species extinction is now far faster than the historic rate, and attempts to quantify a safe ecological operating space place humanity well outside it.
 Many of the problems of ecological degradation interact to multiply The extent to which humans are dependent on the ecosystem. Whether there will be effective political measures taken to protect the ecosystem on a large scale. The likelihood of the emergence of sustainable economies. The positive and negative impacts on the ecosystems of both wealth and poverty. The long-term effects of an ecological collapse on ecosystems. 5 key factors: An epidemic of infectious disease that has spread through human populations across a large region or even worldwide. There are grounds for suspecting that such a highimpact epidemic is more probable than usually assumed. All the features of an extremely devastating disease already exist in nature: essentially incurable (Ebola), nearly always fatal (rabies), extremely infectious (common cold), and long incubation periods (HIV). If a pathogen were to emerge that somehow combined these features What the true probability distribution for pandemics is, especially at the tail. The capacity of international health systems to deal with an extreme pandemic. How fast medical research can proceed in an emergency. How mobility of goods and people, as well as population density, will affect pandemic transmission. Whether humans can develop novel and effective anti-pandemic solutions. (and influenza has demonstrated antigenic shift, the ability to combine features from different viruses), its death toll would be extreme. The world has changed considerably, making comparisons with the past problematic.Today it has better sanitation and medical research, as well as national and supra-national institutions dedicated to combating diseases. But modern transport and dense human population allow infections to spread much more rapidly, and slums can be breeding grounds for disease. An economic or societal collapse on the global scale. The term has been used to describe a broad range of conditions. Often economic collapse is accompanied by social chaos, civil unrest and sometimes a breakdown of law and order. Societal collapse usually refers to the fall or disintegration of human societies, often along with their life support systems. The world economic and political system is made up of many actors with many objectives and many links between them. Such intricate, interconnected systems are subject to unexpected system-wide failures caused by the structure of the network -even if each component of the network is reliable. This gives rise to systemic risk, when parts that individually may function well become vulnerable when connected as a system to a self-reinforcing joint risk that can spread from part to part, potentially affecting the entire system and possibly spilling over to related outside systems. Such effects have been observed in ecology, finance and critical infrastructure such as power grids. The possibility of collapse becomes more acute when several independent networks depend on each other. Whether global system collapse will trigger subsequent collapses or fragility in other areas. What the true trade-off is between efficiency and resilience. Whether effective regulation and resilience can be developed. Whether an external disruption will trigger a collapse. Whether an internal event will trigger a collapse. Whether detection and tracking of asteroids and other dangerous space objects is sufficiently exhaustive. How feasible it is to deflect an asteroid. Whether measures such as evacuation could reduce the damage of an impact. The short-and long-term climate consequences of a collision. Whether our current civilisation could adapt to a post-impact world. 5 key factors: Large asteroid collisions -with objects 5 km or more in sizehappen about once every twenty million years and would have an energy a hundred thousand times greater than the largest bomb ever detonated. A land impact would destroy an area the size of a nation like Holland. Larger asteroids could be extinction-level events. Asteroid impacts are probably one of the best understood of all risks in this report. There has been some discussion about possible methods for deflecting asteroids found on a collision course with the planet. Should an impact occur the main destruction will not be from the initial impact, but from the clouds of dust projected into the upper atmosphere. The damage from such an "impact winter" could affect the climate, damage the biosphere, affect food supplies, and create political instability.
 Global System Collapse Major Asteroid Impact 16 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Executive Summary The true destructive potential of synthetic biology, especially the tail risk. Whether the field will be successfully regulated, or successfully manage to regulate itself. Whether the field will usher in a new era of bio-warfare. Whether the tools of synthetic biology can be used defensively to create effective counter measures. The dangers of relying on synthetic biologists to estimate the danger of synthetic biology. This could emerge through military or commercial bio-warfare, bioterrorism (possibly using dual-use products developed by legitimate researchers, and currently unprotected by international legal regimes), or dangerous pathogens leaked from a lab. Of relevance is whether synthetic biology products become integrated into the global economy or biosphere. This could lead to additional vulnerabilities (a benign but widespread synthetic biology product could be specifically targeted as an entry point through which to cause damage). The design and construction of biological devices and systems for useful purposes, but adding human intentionality to traditional pandemic risks. Attempts at regulation or self-regulation are currently in their infancy, and may not develop as fast as research does. One of the most damaging impacts from synthetic biology would come from an engineered pathogen targeting humans or a crucial component of the ecosystem. Any volcano capable of producing an eruption with an ejecta volume greater than 1,000 km 3 . This is thousands of times larger than normal eruptions. The danger from super-volcanoes is the amount of aerosols and dust projected into the upper atmosphere. This dust would absorb the Sun's rays and cause a global volcanic winter. The Mt Pinatubo eruption of 1991 caused an average global cooling of surface temperatures by 0.5°C over three years, while the Toba eruption around 70,000 years ago is thought by some to have cooled global temperatures for over two centuries. The effect of these eruptions could be best compared with that of a nuclear war. The eruption would be more violent than the nuclear explosions, but would be less likely to ignite firestorms and other secondary effects. Whether countries will coordinate globally against super-volcano risk and damage. The predictability of supervolcanic eruptions. How directly destructive an eruption would be. The effectiveness of general mitigation efforts. How severe the long-term climate effects would be. And if these motivations do not detail the survival and value of humanity, the intelligence will be driven to construct a world without humans. This makes extremely intelligent AIs a unique risk, in that extinction is more likely than lesser impacts. On a more positive note, an intelligence of such power could easily combat most other risks in this report, making extremely intelligent AI into a tool of great potential. There is also the possibility of AI-enabled warfare and all the risks of the technologies that AIs would make possible. An interesting version of this scenario is the possible creation of "whole brain emulations": human brains scanned and physically represented in a machine. This would make the AIs into properly human minds, possibly alleviating a lot of problems. Atomically precise manufacturing, the creation of effective, highthroughput manufacturing processes that operate at the atomic or molecular level. It could create new products -such as smart or extremely resilient materials -and would allow many different groups or even individuals to manufacture a wide range of things. This could lead to the easy construction of large arsenals of conventional or more novel weapons made possible by atomically precise manufacturing. AI is the intelligence exhibited by machines or software, and the branch of computer science that develops machines and software with human-level intelligence. The field is often defined as "the study and design of intelligent agents", systems that perceive their environment and act to maximise their chances of success. Such extreme intelligences could not easily be controlled (either by the groups creating them, or by some international regulatory regime), and would probably act to boost their own intelligence and acquire maximal resources for almost all initial AI motivations. Of particular relevance is whether nanotechnology allows the construction of nuclear bombs. But many of the world's current problems may be solvable with the manufacturing possibilities that nanotechnology would offer, such as depletion of natural resources, pollution, climate change, clean water and even poverty. Some have conjectured special self-replicating nanomachines which would be engineered to consume the entire environment. The misuse of medical nanotechnology is another risk scenario. The timeline for nanotech development. Which aspects of nanotech research will progress in what order. Whether small groups can assemble a weapons arsenal quickly. Whether nanotech tools can be used defensively or for surveillance. Whether nanotech tools or weaponry are made to be outside human control. The reliability of AI predictions. Whether there will be a single dominant AI or a plethora of entities. How intelligent AIs will become. Whether extremely intelligent AIs can be controlled, and if so, how. Whether whole brain emulations (human minds in computer form) will arrive before true AIs. There are two main divisions in governance disasters: failing to solve major solvable problems, and actively causing worse outcomes. An example of the first would be failing to alleviate absolute poverty; of the second, constructing a global totalitarian state. Technology, political and social change may enable the construction of new forms of governance, which may be either much better or much worse. Two issues with governance disasters are first, the difficulty of estimating their probability, and second, the dependence of the impact of these disasters on subjective comparative evaluations: it is not impartially obvious how to rank continued poverty and global totalitarianism against billions of casualties or civilisation collapse. How the severity of nondeadly policy failures can be compared with potential casualties. Whether poor governance will result in a collapse of the world system. How mass surveillance and other technological innovations will affect governance. Whether there will be new systems of governance in the future. Whether a world dictatorship may end up being constructed. 1 2 3 4 5 generic probability of intelligent life (self-)destruction, which includes uncertain risks. Anthropic reasoning can also bound the total risk of human extinction, and hence estimate the unknown component. Non riskspecific resilience and post-disaster rebuilding efforts will also reduce the damage from uncertain risks, as would appropriate national and international regulatory regimes. Most of these methods would also help with the more conventional, known risks, which badly need more investment. These represent the unknown unknowns in the family of global catastrophic challenges. They constitute an amalgamation of all the risks that can appear extremely unlikely in isolation, but can combine to represent a not insignificant proportion of the risk exposure. One resolution to the Fermi paradoxthe apparent absence of alien life in the galaxy -is that intelligent life destroys itself before beginning to expand into the galaxy. Results that increase or decrease the probability of this explanation modify the Whether there will be extensive research into unknown risks and their probabilities. The capacity to develop methods for limiting the combined probability of all uncertain risks. The capacity for estimating "out of-model" risks. Often the situation resembles a set of dominoes: if one falls, many follow. Even small impacts can start a process where different risks interact. 2. Specific measures to address a risk: Global risks often require significant changes, which will result in situations where measures to reduce the risk in one area affect the probability and/or the impact in other areas, for better or worse.
 Two things make the understanding of the relation between the global risks particularly important. collaboration difficulty of reducing risk technical difficulty of reducing risk The technical difficulty of reducing the risk and the difficulty of collaboration Below is an example of an overview of how different global risks can be plotted depending on the technical difficulty of reducing the risk and the difficulty of collaborating to reduce it.
 In order to better understand the relations between different global risks, work could start to analyse similarities and differences. Global Challenges -Twelve risks that threaten human civilisation -
 Probability These estimates are an attempt to assemble existing estimates in order to encourage efforts to improve the numbers. They express estimates of probabilities over 100 years, except in the case of extreme climate change, where the time frame is 200 years. Global challenges need to be seen in the light of trends which help to shape the wider society. These include: Poverty -although it has fallen, it could increase again. This is especially relevant to climate change and pandemics. Population growth -the UN's estimates range from 6.8 billion people by 2100 to a high-variant projection of 16.6 bn (which would require the resources of 10 Earth-like planets to provide everyone with a modern Western lifestyle). Other trends include technological development and demographic changes.  This means that we are now forced to live with the risk of various kinds of extreme disaster with the potential of severely affecting billions of people.
 Preface Preface In this Yearbook from the Global Challenges Foundation, "risk" is defined as the potential damage that can be caused by an extreme disaster multiplied by the probability that it will occur. For the risk of exceptional damage, the probability of occurrence is usually small, or very small, compared with other risks in society, but the effects can be absolutely dire, meaning they must be taken very seriously. We do not know what the exact nature of what these risks are or how they may strike. Some are obvious, others may sound like pure science fiction, but they have led many scientists to regard them as real threats -and therefore it is best to include them in the calculations. With few exceptions, humans have created these risks. There are only a few risks where we are not the cause, for example natural disasters such as an asteroid impact. We could eliminate some of these risks (e.g. nuclear war). In other cases, all we can do is minimise the likelihood of damage, since we have already crossed the threshold that can lead to serious consequences (with climate change, for example, where we have already emitted such high levels of greenhouse gases that there are small but not insignificant likelihoods of significant damage). For other risks we cannot affect the likelihood of them occurring, only minimise damage (with supervolcanic eruptions, for instance). However, here we can build social and ecological resilience so as to reduce the damage. For decisions concerning countermeasures the first important question is: What level of probability of global catastrophes are we prepared to accept? This question has not yet appeared on the political agenda. The reason is that both scientific reports and the media choose to focus on the most likely outcome of these risks. In the absence of risk analysis both decision-makers and the public remain blissfully unaware that the probabilities of certain global catastrophes are significantly higher than we would accept in our everyday lives, where incomparably smaller values are at stake. Another, very important reason for not acting against acknowledged global risks is that they require global responses and therefore global decisions. Regrettably there is no global decision-making body capable of that, no globally functioning legal system, and so there is a lack of effective tools for dealing with these challenges. The result: the risks are increased in the absence of effective measures to counter them. This report wants, on a strictly scientific basis, to identify and describe the global risks of extreme disasters, and also to report the latest developments affecting these risks and measures to face up to them. The Global Challenges Foundation's goal in this report is to accelerate effective counter-actions against global events with the potential for large-scale unwanted effects by deepening both decision makers' and the public's insights into the risks, and also to inspire both debate and welljudged decisions on these questions: -What probabilities of extreme disasters are acceptable? -Which are the optimal countermeasures? -How can an effective global decision-making system be created -with or without a global legal system? We are also convinced that knowledge of these risks is not only a prerequisite for reducing them, but also a responsibility which we owe to our children, grandchildren and to all future generations. It is up to us to decide whether these threats can possibly be reduced or not! These efforts do not only demand sacrifices on our part. They also create opportunities for everyone to make a significant contribution to improving the future of humanity: -For world leaders this means assuming their responsibility and starting to work towards common, global decision-making. -Scientists need to focus their research on areas that will help us take effective measures against the risks. -Companies should make sustainability a business model. -And there is a special opportunity for all of us -that when choosing our politicians and suppliers (of goods and services), we should consider their ambition to eliminate or at least minimise global risks and to create an efficient decisionmaking system that can manage these risks. Finally, I would on behalf of the Global Challenges Foundation extend my sincere gratitude to both Dennis Pamlin, editor of the report, and to all the scientists and other experts who have contributed their research and / or valuable comments.
 Laszlo Szombatfalvy Founder and Chairman, With such a focus it may surprise some readers to find that the report's essential aim is to inspire action and dialogue as well as an increased use of the methodologies used for risk assessment. The real focus is not on the almost unimaginable impacts of the risks the report outlines. Its fundamental purpose is to encourage global collaboration and to use this new category of risk as a driver for innovation. The idea that we face a number of global challenges threatening the very basis of our civilisation at the beginning of the 21st century is well accepted in the scientific community, and is studied at a number of leading universities.  2  But there is still no coordinated approach to address this group of challenges and turn them into opportunities for a new generation of global cooperation and the creation of a global governance system capable of addressing the greatest challenges of our time. This report has, to the best of our knowledge, created the first sciencebased list of global risks with a potentially infinite impact and has made the first attempt to provide an initial overview of the uncertainties related to these risks as well as rough quantifications for the probabilities of these impacts.
 What is risk? Risk is the potential of losing something of value, weighed against the potential to gain something of value. Every day we make different kinds of risk assessments, in more or less rational ways, when we weigh different options against each other. The basic idea of risk is that an uncertainty exists regarding the outcome and that we must find a way to take the best possible decision based on our understanding of this uncertainty.  3  To calculate risk the probability of an outcome is often multiplied by the impact. The impact is in most cases measured in economic terms, but it can also be measured in anything we want to avoid, such as suffering. At the heart of a risk assessment is a probability distribution, often described by a probability density function 4 ; see figure X for a graphic illustration. The slightly tilted bell curve is a common probability distribution, but the shape differs and in reality is seldom as smooth as the example. The total area under the curve always represents 100 percent, i.e. all the possible outcomes fit under the curve. In this case (A) represents the most probable impact. With a much lower probability it will be a close to zero impact, illustrated by (B). In the same way as in case B there is also a low probability that the situation will be very significant, illustrated by (C). The impacts (A), (B) and (C) all belong to the same category, normal impacts: the impacts may be more or less serious, but they can be dealt with within the current system. The impacts in this report are however of a special kind. These are impacts where everything will be lost and the situation will not be reversible, i.e challenges with potentially infinite impact. In insurance and finance this kind of risk is called "risk of ruin", an impact where all capital is lost.  5  This impact is however only infinite for the company that is losing the money. From society's perspective, that is not a special category of risk. In this report the focus is on the "risk of ruin" on a global scale and on a human level, in the worst case this is when we risk the extinction of our own species. On a probability curve the impacts in this report are usually at the very far right with a relatively low probability compared with other impacts, illustrated by (D) in Figure  2 . Often they are so far out on the tail of the curve that they are not even included in studies. For each risk in this report the probability of an infinite impact is very low compared to the most likely outcome. Some studies even indicate that not all risks in this report can result in an infinite impact. But a significant number of peer-reviewed reports indicate that those impacts not only can happen, but that their probability is increasing due to unsustainable trends. The assumption for this report is that by creating a better understanding of our scientific knowledge regarding risks with a potentially infinite impact, we can inspire initiatives that can turn these risks into drivers for innovation. Not only could a better understanding of the unique magnitude of these risks help address the risks we face, it could also help to create a path towards more sustainable development. The group of global risks discussed in this report are so different from most of the challenges we face that they are hard to comprehend. But that is also why they can help us to build the collaboration we need and drive the development of further solutions that benefit both people and the planet. As noted above, none of the risks in this report is likely to result directly in an infinite impact, and some are probably even physically incapable of doing so. But all are so significant that they could reach a threshold impact able to create social and ecological instability that could trigger a process which could lead to an infinite impact. For several reasons the potentially infinite impacts of the risks in this report are not as well known as they should be. One reason is the way that extreme impacts are often masked by most of the theories and models used by governments and business today. For example, the probability of extreme impacts is often below what is included in studies and strategies. The tendency to exclude impacts below a probability of five percent is one reason for the relative "invisibility" of infinite impacts. The almost standard use of a 95% confidence interval is one reason why low-probability high-impact events are often ignored. 6
 Ethical These are impacts that threaten the very survival of humanity and life on Earth -and therefore can be seen as being infinitely negative from an ethical perspective. No positive gain can outweigh even a small probability for an infinite negative impact. Such risks require society to ensure that we eliminate these risks by reducing the impact below an infinite impact as a top priority, or at least do everything we can to reduce the probability of these risks. As some of these risks are impossible to eliminate today it is also important to discuss what probability can right now be accepted for risks with a possible infinite impact.
 Economic Infinite impacts are beyond what most traditional economic models today are able to cope with. The impacts are irreversible in the most fundamental way, so tools like cost-benefit assessment seldom make sense. To use discounting that makes infinite impacts (which could take place 100 years or more from now and affect all future generations) close to invisible in economic assessments, is another example of a challenge with current tools. So while tools like cost-benefit models and discounting can help us in some areas, they are seldom applicable in the context of infinite impacts. New tools are needed to guide the global economy in an age of potential infinite impacts.
 Infinite impact The concept infinite impact refers to two aspects in particular; the terminology is not meant to imply a literally infinite impact (with all the mathematical subtleties that would imply) but to serve as a reminder that these risks are of a different nature. Climate change is a good example, where almost all of the focus is on the most likely scenarios and there are few studies that include the lowprobability high-impact scenarios. In most reports about climate impacts, the impacts caused by warming beyond five or six degrees Celsius are even omitted from tables and graphs even though the IPCC's own research indicates that the probability of these impacts are often between one and five percent, and sometimes even higher. 7 Other aspects that contribute to this relative invisibility include the fact that extreme impacts are difficult to translate into monetary terms, they have a global scope, and they often require a time-horizon of a century or more. They cannot be understood simply by linear extrapolation of current trends, and they lack historical precedents. There is also the fact that the measures required to significantly reduce the probability of infinite impacts will be radical compared to a business-as-usual scenario with a focus on incremental changes. The exact probability of a specific impact is difficult or impossible to estimate. 8 However, the important thing is to establish the current magnitude of the probabilities and compare them with the probabilities for such impacts we cannot accept. A failure to provide any estimate for these riks often results in strategies and priorities defined as though the probability of a totally unacceptable outcome is zero. An approximate number for a best estimate also makes it easier to understand that a great uncertainty means the actual probability can be both much higher and much lower than the best estimate. It should also be stressed that uncertainty is not a weakness in science; it always exists in scientific work. It is a systematic way of understanding the limitations of the methodology, data, etc. 9 Uncertainty is not a reason to wait to take action if the impacts are serious. Increased uncertainty is something that risk experts, e.g. insurance experts and security policy experts, interpret as a signal for action. A contrasting challenge is that our cultural references to the threat of infinite impacts have been dominated throughout history by religious groups seeking to scare society without any scientific backing, often as a way to discipline people and implement unpopular measures. It should not have to be said, but this report is obviously fundamentally different as it focuses on scientific evidence from peer-reviewed sources.
 Roulette and Russian roulette When probability and normal risks are discussed the example of a casino and roulette is often used. You bet something, then spin the wheel and with a certain probability you win or lose. You can use different odds to discuss different kinds of risk taking. These kinds of thought experiment can be very useful, but when it comes to infinite risks these gaming analogies become problematic. For infinite impact a more appropriate analogy is probably Russian roulette. But instead of "normal" Russian roulette where you only bet your own life you are now also betting everyone you know and everyone you don't know. Everyone alive will die if you lose. There will be no second chance for anyone as there will be no future generations; humanity will end with your loss. What probability would you accept for different sums of money if you played this version of Russian roulette? Most people would say that it is stupid and -no matter how low the probability is and no matter how big the potential win is -this kind of game should not be played, as it is unethical. Many would also say that no person should be allowed to make such a judgment, as those who are affected do not have a say. You could add that most of those who will lose from it cannot say anything as they are not born and will never exist if you lose. The difference between ordinary roulette and "allhumanity Russian roulette" is one way of illustrating the difference in nature between a "normal" risk that is reversible, and a risk with an infinite impact. An additional challenge in acknowledging the risks outlined in this report is that many of the traditional risks including wars and violence have decreased, even though it might not always looks that way in media.  10  So a significant number of experts today spend a substantial amount of time trying to explain that much of what is discussed as dangerous trends might not be as dangerous as we think. For policy makers listening only to experts in traditional risk areas it is therefore easy to get the impression that global risks are becoming less of a problem. In the media it is still common to contrast the most probable climate impact with the probability that nothing, or almost nothing, will happen. The fact that almost nothing could happen is not wrong in most cases, but it is unscientific and dangerous if different levels of probability are presented as equal. The tendency to compare the most probable climate impact with the possibility of a low or no impact also results in a situation where low-probability high-impact outcomes are often totally ignored. An honest and scientific approach is to, whenever possible, present the whole probability distribution and pay special attention to unacceptable outcomes. The fact that we have challenges that with some probability might be infinite and therefore fundamentally irreversible is difficult to comprehend, and physiologically they are something our brains are poorly equipped to respond to, according to evolutionary psychologists. This psychological denial may be one reason why there is a tendency among some stakeholders to confuse "being optimistic" with denying what science is telling us, and ignoring parts of the probability curve.  14  Ignoring the fact that there is strong scientific evidence for serious impacts in different areas, and focusing only on selected sources which suggest that the problem may not be so serious, is not optimistic. It is both unscientific and dangerous.  15  A scientific approach requires us to base our decisions on the whole probability distribution. Whether it is possible to address the challenge or not is the area where optimism and pessimism can make people look at the same set of data and come to different conclusions. Two things are important to keep in mind: first, that there is always a probability distribution when it comes to risk; second, that there are two different kinds of impacts that are of interest for this report. The probability distribution can have different shapes but in simplified cases the shape tends to look like a slightly modified clock (remember figure  1 ). In the media it can sound as though experts argue whether an impact, for example a climate impact or a pandemic, will be dangerous or not. But what serious experts discuss is the probability of different oucomes.
 The history of infinite impacts: The LA-602 document The understanding of infinite impacts is very recent compared with most of our institutions and laws. It is only 70 years ago that Edward Teller, one of the greatest physicists of his time, with his back-of-the-envelope calculations, produced results that differed drastically from all that had gone before. His calculations indicated that the explosion of a nuclear bomb -a creation of some of the brightest minds on the planet, including Teller himself -could result in a chain reaction so powerful that it would ignite the world's atmosphere, thereby ending human life on Earth.  16  Robert Oppenheimer, who led the Manhattan Project to develop the nuclear bomb, halted the project to see whether Teller's calculations were correct.  17  The resulting document, LA-602: Ignition of the Atmosphere with Nuclear Bombs, concluded that Teller was wrong, But the sheer complexity drove them to end their assessment by writing that "further work on the subject [is] highly desirable".  18  The LA-602 document can be seen as the first scientific global risk report addressing a category of risks where the worst possible impact in all practical senses is infinite.  19  Since the atomic bomb more challenges have emerged with potentially infinite impact. Allmost all of these new challenges are linked to the increased knowledge, economic and technical development that has brought so many benefits. For example, climate change is the result of the industrial revolution and development that was, and still is, based heavily on fossil fuel. The first part of the report is an introduction where the global risks with potential infinite impact are introduced and defined. This part also includes the methodology for selecting these risks, and presents the twelve risks that meet this definition. Four goals of the report are also presented, under the headings "acknowledge", "inspire", "connect" and "deliver". The second part is an overview of the twelve global risks and key events that illustrate some of the work around the world to address them. For each challenge five important factors that influence the probability or impact are also listed. The risks are divided into four different categories depending on their characteristics. "Current challenges" is the first category and includes the risks that currently threaten humanity due to our economic and technological development -extreme climate change, for example, which depends on how much greenhouse gas we emit. "Exogenic challenges" includes risks where the basic probability of an event is beyond human control, but where the probability and magnitude of the impact can be influenced -asteroid impacts, for example, where the asteroids' paths are beyond human control but an impact can be moderated by either changing the direction of the asteroid or preparing for an impact. The fifth part presents some of the most important underlying trends that influence the global challenges, which often build up slowly until they reach a threshold and very rapid changes ensue. The sixth and final part presents an overview of possible ways forward.
 Goals But we now face the possibility that even tools created with the best of intentions can have a darker side too, a side that may threaten human civilisation, and conceivably the continuation of human life. This is what all decision-makers need to recognise. Rather than succumbing to terror, we need to acknowledge that we can let the prospect inspire and drive us forward. Establish a category of risks with potentially infinite impact. Before anything significant can happen regarding global risks with potentially infinite impacts, their existence must be acknowledged. Rapid technological development and economic growth have delivered unprecedented material welfare to billions of people in a veritable tide of utopias.  21  Show concrete action that is taking place today. This report seeks to show that it is not only possible to contribute to reducing these risks, but that it is perhaps the most important thing anyone can spend their time on. It does so by combining information about the risks with information about individuals and groups who has made a significant contribution by turning challenges into opportunities. By highlighting concrete examples the report hopes to inspire a new generation of leaders. Goal 1: Acknowledge Even with those risks where many groups are involved, such as climate change and pandemics, very few today address the possibility of infinite impact aspects. Even fewer groups address the links between the different risks. There is also a need to connect different levels of work, so that local, regional, national and international efforts can support each other when it comes to risks with potentially infinite impacts.
 Identify and implement strategies and initiatives. Reports can acknowledge, inspire and connect, but only people can deliver actual results. The main focus of the report is to show that actual initiatives need to be taken that deliver actual results. Only when the probability of an infinite impact becomes acceptably low, very close to zero, and/or when the maximum impact is significantly reduced, should we talk about real progress. In order to deliver results it is important to remember that global governance to tackle these risks is the way we organise society in order to address our greatest challenges. It is not a question of establishing a "world government", it is about the way we organise ourselves on all levels, from the local to the global. The report is a first step and should be seen as an invitation to all responsible parties that can affect the probability and impact of risks with potentially infinite impacts. But its success will ultimately be measured only on how it contributes to concrete results.
 Goal 3: Connect That leaders in different sectors connect with each other to encourage collaboration. A specific focus on financial and security policy where significant risks combine to demand action beyond the incremental is required. Goal 4: Deliver A collapse of civilisation is defined as a drastic decrease in human population size and political/economic/social complexity, globally for an extended time.  25  The above definition means the list of challenges is not static. When new challenges emerge, or current ones fade away, the list will change. An additional criterion for including risks in this report is "human influence". Only risks where humans can influence either the probability, the impact, or both, are included. For most risks both impact and probability can be affected, for example with nuclear war, where the number/size of weapons influences the impact and tensions between countries affects the probability. Other risks, such as a supervolcano, are included as it is possible to affect the impact through various mitigation methods, even if we currently cannot affect the probability. Risks that are susceptible to human influence are indirectly linked, because efforts to address one of them may increase or decrease the likelihood of another. The concept of infinity was chosen as it reflects many of the challenges, especially in economic theory, to addressing these risks as well as the need to question much of our current way of thinking. The concept of a category of risks based on their extreme impact is meant to provide a tool to distinguish one particular kind of risk from others. The benefit of this new concept should be assessed based on two things. First, does the category exist, and second, is the concept helpful in addressing these risks? The report has found ample evidence that there are risks with an impact that can end human civilisation and even all human life. The report further concludes that a new category of risk is not only meaningful but also timely. We live in a society where global risks with potentially infinite impacts increase in both number and probability according to multiple studies. Looking ahead, many emerging technologies which will certainly provide beneficial results, might also result in an increased probability of infinite impacts.  26  Over the last few years a greater understanding of low probability or unknown probability events has helped more people to understand the importance of looking beyond the most probable scenarios. Concepts like "black swans" and "perfect storms" are now part of mainstream policy and business language.  27  Greater understanding of the technology and science of complex systems has also resulted in a new understanding of potentially disruptive events. Humans now have such an impact on the planet that the term "the anthropocene" is being used, even by mainstream media like The Economist.  28  The term was introduced in the 90s by the Nobel Prize winner Paul Crutzen to describe how humans are now the dominant force changing the Earth's ecosystems.  29  The idea to establish a well defined category of risks that focus on risks with a potentially infinite impact that can be used as a practical tool by policy makers is partly inspired by Nick Bostrom's philosophical work and his introduction of a risk taxonomy that includes an academic category called "existential risks".  30  Introducing a category with risks that have a potentially infinite impact is not meant to be a mathematical definition; infinity is a thorny mathematical concept and nothing in reality can be infinite.  31  It is meant to illustrate a singularity, when humanity is threatened, when many of the tools used to approach most challenges today become problematic, meaningless, or even counterproductive. The concept of an infinite impact highlights a unique situation where humanity itself is threatened and the very idea of value and price collapses from a human perspective, as the
 Life Value The following estimates have been applied to the value of life in the US. The estimates are either for one year of additional life or for the statistical value of a single life. -$50,000 per year of quality life (international standard most private and government-run health insurance plans worldwide use to determine whether to cover a new medical procedure) price of the last humans also can be seen to be infinite. This is not to say that those traditional tools cannot still be useful, but with infinite impacts we need to add an additional set of analytical tools. Some of the risks, including nuclear war, climate change and pandemics, are often included in current risk overviews, but in many cases their possible infinite impacts are excluded. The impacts which are included are in most cases still very serious, but only the more probable parts of the probability distributions are included, and the last part of the long tail -where the infinite impact is found -is excluded.  32  Most risk reports do not differentiate between challenges with a limited impact and those with a potential for infinite impact. This is dangerous, as it can mean resources are spent in ways that increase the probability of an infinite impact.
 Ethical aspects of infinite impact The basic ethical aspect of infinite impact is this: a very small group alive today can take decisions that will fundamentally affect all future generations. "All future generations" is not a concept that is often discussed, and for good reason. All through human history we have had no tools with a measurable global impact for more than a few generations. Only in the last few decades has our potential impact reached a level where all future generations can be affected, for the simple reason that we now have the technological capacity to end human civilisation. If we count human history from the time when we began to practice settled agriculture, that gives us about 12,000 years.  33  If we make a moderate assumption that humanity will live for at least 50 million more years 34 our 12,000-year history so far represents 1/4200, or 0.024%, of our potential history. So our generation has the option of risking everything and annulling 99.976% of our potential history. Comparing 0.024% with the days of a person living to 100 years from the day of conception, this would equal less than nine days and is the first stage of human embryogenesis, the germinal stage.  35  Two additional arguments to treat potentially infinite impacts as a separate category are: 36 1. An approach to infinite impacts cannot be one of trial-and-error, because there is no opportunity to learn from errors. Infinite impacts are in a different category. Institutions and individuals may find it hard to take these risks seriously simply because they lie outside our experience. Our collective fear-response will probably be illcalibrated to the magnitude of threat.
 Economic aspects of infinite impact and discounting In today's society a monetary value is sometimes ascribed to human life. Some experts use this method to estimate risk by assigning a monetary value to human extinction.  37  We have to remember that the monetary values placed on a human life in most cases are not meant to suggest that we have actually assigned a specific value to a life. Assigning a value to a human life is a tool used in a society with a limited supply of resources or infrastructure (ambulances, perhaps) or skills. In such a society it is impossible to save every life, so some trade-off must be made.  38  The US Environmental Protection Agency explains its use like this: "The EPA does not place a dollar value on individual lives. Rather, when conducting a benefit-cost analysis of new environmental policies, the Agency uses estimates of how much people are willing to pay for small reductions in their risks of dying from adverse health conditions that may be caused by environmental pollution."  39  The Two things make infinite impacts special from a discounting perspective. First, there is no way that future generations can compensate for the impact, as they will not exist. Second, the impact is something that is beyond an individual preference, as society will no longer exist. Discounting is undertaken to allocate resources in the most productive way. In cases that do not include infinite impacts, discounting "reflects the fact that there are many high-yield investments that would improve the quality of life for future generations. The discount rate should be set so that our investable funds are devoted to the most productive uses."  44   unbalance the system and eventually push it over the threshold.  46  Note that these dramatic illustrations rest on assumptions that the thresholds are still relatively benign, not moving us beyond tipping points which result in an accelerated release of methane that could result in a temperature increase of more than 8 °C, possibly producing infinite impacts.  47  Calculating illustrative numbers By including the welfare of future generations, something that is important when their very existence is threatened, economic discounting becomes difficult. In this chapter, some illustrative numbers are provided to indicate the order of magnitude of the values that calculations provide when traditional calculations also include future generations. These illustrative calculations are only illustrative as the timespans that must be used make all traditional assumptions questionable to say the least. Still, as an indicator for why infinite impact might be a good approximation they might help. As a species that can manipulate our environment it could be argued that the time the human race will be around, if we do not kill ourselves, can be estimated to be between 1-10 million years -the typical time period for the biological evolution of a successful species  48  -and one billion years, the inhabitable time of Earth.  49  Figure  5 : Nordhaus, The Climate Casino: Climate policy with a sharp tipping point at 3.5°C. This shows that the optimal temperature increase is very close to the threshold. It is constrained on the low side by abatement costs and on the high side by the sharp increase in damages.  Likewise the value of a life, $28 million, a value that is based on an assessment of how individuals chose when it comes to flying, can be seen as much too small. This value is based on how much we value our own lives on the margin, and it is reasonable to assume that the value would be higher than only a multiplication of our own value if we also considered the risk of losing our family, everyone we know, as well as everyone else on the planet. In the same way as the cost increases when a certain product is in short supply, the cost of the last humans could be assumed to be very high, if not infinite. Obviously, the very idea to put a price on the survival of humanity can be questioned for good reasons, but if we still want to use a number, $28 million per life should at least be considered as a significant underestimation. For those that are reluctant or unable to use infinity in calculations and are in need of a number for their formulas, $86 sextillion could be a good initial start for the cost of infinite impacts. But it is important to note that this number might be orders of magnitude smaller than an estimate which actually took into account a more correct estimation of the number of people that should be included in future generations as well as the price that should be assigned to the loss of the last humans. As we address very complex systems, such as human civilisation and global ecosystems, a concept as important as infinite impact in this report is that of infinity impact threshold. This is the impact level that can trigger a chain of events that results in the end of human civilisation. The infinite impact threshold (IIT) concept represents the idea that long before an actual infinite impact is reached there is a tipping point where it (with some probability) is no longer possible to reverse events. So instead of focusing only on the ultimate impact it is important to estimate what level of impact the infinity threshold entails. The IIT is defined as an impact that can trigger a chain of events that could result first in a civilisation collapse, and then later result in an infinite impact. Such thresholds are especially important to recognise in a complex and interconnected society where resilience is decreasing.
 If we assume -50 million years for the future of humanity as our reference, -an average life expectancy of 100 years 50 , and -a global population of 6 billion people 51 -all conservative estimate -, we have half a million generations ahead of us with a total of 3 quadrillion individuals. Assuming a value of $50,000 per life, the cost of losing them would then be $1.5 ×10
 Risks with infinite impact Situation that requires new measures and tools and ALARP 2.3.4 Global F-N curves Social and ecological systems are complex, and in most complex systems there are thresholds where positive feedback loops become self-reinforcing. In a system where resilience is too low, feedback loops can result in a total system collapse. These thresholds are very difficult to estimate and in most cases it is possible only to estimate their order of magnitude. As David Orrell and Patrick McSharry wrote in A Systems Approach to Forecasting: "Complex systems have emergent properties, qualities that cannot be predicted in advance from knowledge of systems components alone". According to complexity scientist Stephen Wolfram's principle of computational irreducibility, the only way to predict the evolution of such a system is to run the system itself: "There is no simple set of equations that can look into its future."  55  Orrell and McSharry also noted that "in orthodox economics, the reductionist approach means that the economy is seen as consisting of individual, independent agents who act to maximise their own utility. It assumes that prices are driven to a state of near-equilibrium by the 'invisible hand' of the economy. Deviations from this state are assumed to be random and independent, so the price fluctuations are often modelled using the normal distribution or other distributions with thin tails and finite variance." The drawbacks of an approach using the normal distribution, or other distributions with thin tails and finite variance, become obvious when the unexpected happens as in the recent credit crunch, when existing models totally failed to capture the true risks of the economy. As an employee of Lehman Brothers put it on August 11, 2007: "Events that models predicted would happen only once in 10,000 years happened every day for three days."  56  The exact level for an infinite impact threshold should not be the focus, but rather the fact that such thresholds exists and that an order of magnitude should be estimated.  57  During the process of writing the report, experts suggested that a relatively quick death of two billion people could be used as a tentative number until more research is available.  58  With current trends undermining ecological and social resilience it should be noted that the threshold level is likely to become lower as time progress. In the context of global risks with potentially infinite impact, the possibility of establishing global F-N curves is worth exploring. One of the most common and flexible frameworks used for risk criteria divides risks into three bands: 59 The bands are expressed by F-N curves. When the frequency of events which cause at least N fatalities is plotted against the number N on log-log scales, the result is called an F-N curve.  60  If the frequency scale is replaced by annual probability, then the resultant curve is called an f-N curve.  The concept for the middle band when using F-N curves is ALARP. It is a term often used in the area of safety-critical and safety-involved systems.  62  The ALARP principle is that the residual risk should be as low as reasonably practicable. The upper band, the unacceptable/ intolerable region, is usually the area above the ALARP area (see figure  8 ) By using F-N curves it is also possible to establish absolute impact levels that are never acceptable, regardless of probability (Figure  7 . Based on an actual F-n Curve showing an absolute impact level that is defined as unacceptable). This has been done in some cases for local projects. The infinite threshold could be used to create an impact limit on global F-N curves used for global challenges in the future. Such an approach would help governments, companies and researchers when they develop new technical solutions and when investing in resilience. Instead of reducing risk, such an approach encourages the building of systems which cannot have negative impacts above a certain level. Today no established methodology exists that provides a constantly updated list of risks that threaten human civilisation, or even all human life. Given that such a category can help society to better understand and act to avoid such risks, and better understand the relation between these risks, it can be argued that a name for this category would be helpful.  65  To name something that refers to the end of humanity is in itself a challenge, as the very idea is so far from our usual references and to many the intuitive feeling will be to dismiss any such thing.
 Pros The concept used in this report is "infinity". The reson for this is that many of the challenges relate to discussed. In one way the name is not very important so long as people understand the impacts and risks associated with it. Still, a name is symbolic and can either help or make it more difficult to get support to establish the new category. The work to establish a list of risks with infinite impact evolved from "existential risk", the philosophical concept that inspired much of the work to establish a clearly defined group of risks. The reason for not using the concept "existential risk and impact" for this category, beside the fact that existential impact is also used in academic contexts to refer to a personal impact, is that the infinite category is a smaller subset of "existential risk" and this new category is meant to be used as a tool, not a scientific concept. Not only should the impacts in the category potentially result in the end of all human life, it should be possible to affect the probability and/or impact of that risk. There must also exist an agreed methodology, such as the one suggested in this report, that decides what risks belong and not belong on the list. Another concept that the category relates to is "global catastrophic risk" as it is one of the most used concepts among academics interested in infinite impacts. However it is vague enough to be used to refer to impacts from a few thousand deaths to the end of human civilisation. Already in use but not clearly defined, it includes both the academic concept existential risks and the category of risks with infinite impacts. macroeconomics and its challenges in relation to the kind of impacts that the risks in this report focus on. Further, the name clearly highlights the unique nature without any normative judgements. Still, infinity is an abstract concept and it might not be best communicate the unique group of risks that it covers to all stakeholders. In the same way as it can be hard to use singularity to describe a black hole, it can be difficult to use infinity to describe a certain risk. If people can accept that it is only from a specific perspective that the infinity concept is relevant it could be used beyond the areas of macroeconomics. Two other concepts that also have been considered during the process of writing this report are "xrisks" and "human risk of ruin". Xrisk has the advantage, and disadvantage, of not really saying anything at all about the risk. The positive aspect is that the name can be associated with the general concept of extinction and the philosophical concept of existential risk as both have the letter x in them. The disadvantage is the x often represents the unknown and can therefore relate to any risk. There is nothing in the name that directly relates to the kind of impacts that the category covers, so it is easy to interpret the term as just unknown risks. Human risk of ruin has the advantage of having a direct link to a concept, risk of ruin, that relates to a very specific state where all is lost. Risk of ruin is a concept in use in gambling, insurance, and finance that can all give very important contributions to the work with this new category of risk. The resemblance to an existing concept that is well established could be both a strength and a liability. Below is an overview of the process when different names were  There will not be a perfect concept and the question is what concept can find the best balance between being easy to understand, acceptable where policy decisions needs to be made and also acceptable for all key groups that are relevant for work in these area. During the process to find a name for this category inspiration has been found in the process when new concepts have been introduced; from irrational numbers and genocide to sustainable development and the Human Development Index. So far "infinite risk" can be seen as the least bad concept in some areas and "xrisks" and "human risk of ruin" the least bad in others. The purpose of this report is to establish a methodology to identify a very specific group of risks as well as continue to a process where these risks will be addressed in a systematic and appropriate way. The issue of naming this group of risks will be left to others. The important is that the category gets the attention it deserves.   68  But through history and today it is mainly used for a religious end of time scenario. Its strong links to unscientific doom-mongers make it probably unsuitable for a scientific concept. 5. End-of-the-world risk -belongs to the irrational doomsday narratives and so is probably unsuitable for scientific risk assessments. 6. Extreme risk -is vague enough to describe anything beyond the normal, so it is probably unsuitable for risk assessments of this magnitude. 7. Unique risk -is even vaguer, as every risk is unique in some way. Probably best avoided in risk assessments. 8. Collapse risk -is based on Jared Diamond's thinking.  69  There are many different kinds of collapse and only a few result in infinite impact.
 48 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 2.3 Global challenges and infinite impact
 Estimations of impact Only literature where there is some estimation of impact that indicates the possibility of an infinite impact is included.
 Leading organisations' priorities In order to increase the probability of covering all relevant risks an overview of leading organisations' work was conducted. This list was then compared with the initial list and subjected to the same filter regarding the possibility to affect the probability or impact.
 Possibility of addressing the risk Possibility of addressing the risk: From the risks gathered from literature and organisations, only those where the probability or impact can be affected by human actions are included.
 Expert review Qualitative assessment: Expert review in order to increase the probability of covering all relevant global risks.   73  The methodology for including global risks with a potentially infinite impact is based on a scientific review of key literature, with focus on peer-reviewed academic journals, using keyword search of both World of Knowledge 74 and Google Scholar 75 combined with existing literature overviews in the area of global challenges. This also included a snowball methodology where references in the leading studies and books were used to identify other scientific studies and books. In order to select words for a literature search to identify infinite impacts, a process was established to identify words in the scientific literature connected to global challenges with potentially infinite impacts. Some words generate a lot of misses, i.e. publications that use the term but are not the focus of this report. For example "existential risk" is used in business; "human extinction" is used in memory/cognition. Some search terms produced relatively few hits. For example "global catastrophic risk" is not used much. Other words are only used by people within a specific research community: few use "existential risk" in our sense unless they are using Nick Bostrom's work. The term "global catastrophe" was identified as a phrase that referred almost exclusively to extremely negative impacts on humans, by a diversity of researchers, not just people in one research community. A list of 178 relevant books and reports was established based on what other studies have referred to, and/or which are seen as landmark studies by groups interviewed during the process. They were selected for a closer examination regarding the challenges they include.  76  The full bibliography, even with its focus on publications of general interest, is still rather long. So it is helpful to have a shorter list focused on the highlights; the most important publications based on how often they are quoted, how wellspread the content (methodology, lists, etc.) is and how often key organisations use them. -Influential in developing the field. This includes publications that are highly cited 77 and those that have motivated significant additional research. They are not necessarily the first publications to introduce the concepts they discuss, but for whatever reason they will have proved important in advancing research. -State of the art. This includes publications developing new concepts at the forefront of global challenges research as well as those providing the best discussions of important established concepts. Reading these publications would bring a researcher up to speed with current research on global challenges. So they are important for the quality of their ideas. -Covers multiple global challenges (at least two). Publications that discuss a variety of global challenges are of particular importance because they aid in identifying and comparing the various challenges. This process is essential for research on global risks to identify boundaries and research priorities. In order to identify which global challenges are most commonly discussed, key surveys were identified and coded. First, a list of publications that survey at least three global challenges was compiled, and they were then scanned to find which challenges they discussed. The publications that survey many global challenges were identified from the full bibliography. Publications from both the academic and popular literature were considered. Emphasis was placed on publications of repute or other significance.  78  To qualify as a survey of global challenges, the publication had to provide an explicit list of challenges or to be of sufficient length and breadth for it to discuss a variety of challenges. Many of the publications are books or book-length collections of articles published in book form or as special issues of scholarly journals. Some individual articles were also included because they discussed a significant breadth of challenges. A A list of 34 global challenges was developed based on the challenges mentioned in the publications. A spreadsheet containing the challenges and the publications was created to record mentions of specific challenges in each publication to be coded. Then each publication was scanned in its entirety for mentions of global challenges. Scanning by this method was necessary because many of the publications did not contain explicit lists of global challenges, and the ones that did often mentioned additional challenges separately from their lists. So it was not required that a global challenge be mentioned in a list for it to be counted -it only had to be mentioned somewhere in the publication as a challenge. Assessing whether a particular portion of text counts as a global challenge and which category it fits in sometimes requires some interpretation. This is inevitable for most types of textual analysis, or, more generally, for the coding of qualitative data. The need for interpretation in this coding was heightened by the fact that the publications often were not written with the purpose of surveying the breadth of global challenges, and even the publications that were intended as surveys did not use consistent definitions of global challenges. The coding presented here erred on the side of greater inclusivity: if a portion of text was in the vicinity of a global challenge, then it was coded as one. For example, some publications discussed risks associated with nuclear weapons in a general sense without specifically mentioning the possibility of large-scale nuclear war. These discussions were coded as mentions of nuclear war, even though they could also refer to single usages of nuclear weapons that would not rate as a global challenge. This more inclusive approach is warranted because many of the publications were not focused exclusively on global challenges. If they were focused on them, it is likely that they would have included these risks in their global challenge form (e.g., nuclear war), given that they were already discussing something related (e.g., nuclear weapons). Below are the results from the overview of the surveys.  However, it is important to note that many more studies exist that focus on individual global risks, but often without including low-probability high-impact outcomes.  80  How much work actually exists on human extinction infinite impact is therefore difficult to assess. The list of risks found in the scientific literature was checked against a review of what challenges key organisations working on global challenges include in their material and on their webpages. This was done to ensure that no important risk was excluded from the list. The coding of key organisations paralleled the coding of key survey publications. Organisations were identified via the global catastrophic risk organisation directory published by the Global Catastrophic Risk Institute.  82  They were selected from the directory if they worked on a variety of global challenges -at least three, and ideally more. The reason for focusing on those that work on multiple challenges is to understand which challenges they consider important and why. In contrast, organisations that focus on only one or two challenges may not   2 2 2 1 1 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0 0 0 4 8 12 2 6 10 14 be able to adjust their focus according to which challenges they consider the most important. The organisation coding used the same coding scheme developed for coding survey publications. References to specific global challenges were obtained from organisations' websites. Many have web pages which list the topics they work on. Where possible, references to global challenges were pulled from these pages. Additional references to these challenges were identified by browsing other web pages, including recent publications. While it is possible that some of these organisations have worked on global challenges not mentioned on the web pages that were examined, overall the main challenges that they have worked on have probably been identified and coded. So the results should give a reasonably accurate picture of what global challenges these organisations are working on. Organisations working with global challenges were initially selected on the basis of the literature overview. A snowball sampling was conducted based on the list of organisations identified, according to whether they claimed to work on global challenges and/or their web page contained information about "existential risk", "global catastrophic risk","human extinction" or "greatest global challenges". Cross-references between organisations and input during the workshops were also used to identify organisations. An initial list of 180 organisations which work with global challenges was established. Based on the production of relevant literature, which other organisations referred to the organisation, and/or are seen as influential by groups interviewed during the process, a short-list of organisations were selected for a closer examination regarding the challenges they work with. Then those working with multiple challenges were selected, resulting in a list of 19 organisations.  83  Below is the overview of the results from the overview of key organisations working with multiple global challenges. The WEF describes its perception methodology as follows: "This approach can highlight areas that are of most concern to different stakeholders, and potentially galvanise shared efforts to address them."  85  The question which people are asked to answer is: "What occurrence causes significant negative impact for several countries and industries?"  86  The respondents are then asked to provide a number on two scales from 1-4, one for impact and another for likelihood (within 10 years).  87  It is then up to the respondent to define what 1-4 means, so the major value of the report is to track the changes in perception over the years. Such perception approaches are obviously very interesting and, as the WEF states, can influence actual probability as the readers' decisions will be influenced by how different challenges are perceived. Still, it is important to remember that the report does not provide an assessment of the actual probability (0-100%) or an assessment of the impact (and not the impact on human suffering, as many respondents likely define risk in monetary terms for their own company or country). An overview of WEF reports from the last ten years indicates that the challenges that likely could happen when applying a five year horizon, like the first signs of climate change, governmental failure and traditional pandemic, are identified. On the other hand, challenges which have very big impacts but lower probability, like extreme climate change, nanotechnology, major volcanoes, AI, and asteroids, tend to get less, or no, attention. An important question to explore is whether a focus on the smaller but still serious impacts of global challenges can result in an increased probability of infinite impacts. For example, there are reasons to believe that a focus on incremental adaptation instead of significant mitigation could be a problem for climate change as it could result in high-carbon lock-in.  88  Other research indicates that focus on commercially relevant smaller pandemics could result in actions that make a major pandemic more likely. It is argued that this could happen, for example, by encouraging increased trade of goods while investing in equipment that scans for the type of pandemics that are known. Such a system can reduce the probability for known pandemics while at the same time resulting in an increased probability for new and more serious pandemics.  89  Figure This is an initial list. Additional risks will be added as new scientific studies become available, and some will be removed if steps are taken to reduce their probability  90  and/or impact so that they no longer meet the criteria.
 Four categories of global challenges The challenges included in this report belong to four categories. The first, current challenges, includes those where decisions today can result directly in infinite impacts. They are included even if the time between action and impact might be decades, as with climate change. The second category is exogenous challenges, those where decisions do not -currently -influence probability, but can influence impact. The third category is emerging challenges, those where technology and science are not advanced enough to pose a severe threat today, but where the challenges will probably soon be able to have an infinite impact. Many risks could severely damage humanity but have not been included in this report. They were excluded for one or more of three reasons: 1. Limited impact. Many challenges can have significant local negative effects, without approaching the "2 billion negatively affected" criterion -tsunamis, for example, and chemical pollution.
 No effective countermeasures. The report focuses on promoting effective interventions and so ignores challenges where nothing useful can be done to prevent or mitigate the impact, as with nearby gamma-ray bursts.
 Included in other challenges. Many challenges are already covered by others, or have a damage profile so similar that there seemed no need to have a separate category. Population growth, for one, is an underlying driver significant for climate change and eco-system catastrophe, but without direct large-scale impacts. The challenges mentioned in the reviewed literature and organisations which are not included in this report often refer to economic damage such as "fiscal crises" or "unemployment". While such impacts could have far-reaching consequences they are obviously of another magnitude than those included here. Some of the risks that were suggested and/or which exist in books and reports about global risks were rejected according to the criteria above. They include: 91 1. Astronomical explosion/nearby gamma-ray burst or supernova.  92  These seem to be events of extremely low probability and which are unlikely to be survivable. Milder versions of them (where the source is sufficiently far away) may be considered in a subsequent report.
 ͢ Not included due to: No effective countermeasures 2. False vacuum collapse. If our universe is in a false vacuum and it collapses at any point, the collapse would expand at the speed of light destroying all organised structures in the universe.  93  This would not be survivable.
 ͢ Not included due to: No effective countermeasures 3. Chemical pollution. Increasingly, there is particular concern about three types of chemicals: those that persist in the environment and accumulate in the bodies of wildlife and people, endocrine disruptors that can interfere with hormones, and chemicals that cause cancer or damage DNA. ͢ Not included due to: Limited impact 4. Dangerous physics experiments creating black holes/strangelets including high energy physics. These risks are of low probability  94  and have been subsumed under "Uncertain Risks".
 ͢ Not included due to: Included in other challenges
 Destructive solar flares. Though solar flares or coronal mass ejections could cause great economic damage to our technological civilisation, 95 they would not lead directly to mass casualties unless the system lacks basic resilience. They have been subsumed in the Global System Collapse category.
 ͢ Not included due to: Limited impact/included in other challenges
 56 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 2.5 The rseulting list of global risks using this methodology 6. Moral collapse of humanity. Humanity may develop along a path that we would currently find morally repellent. The consequences of this are not clear-cut, and depend on value judgements that would be contentious and unshared.  96   Sometimes it can take a very long time for a system to stabilise again. Looking at all the biotic crises over the past 530 million years, a research team from Berkeley found an average of 10 million years between an extinction and a subsequent flourishing of life.  102  What makes things difficult is that once a system is unstable, a small disaster can have knock-on effectsthe death of one Austrian nobleman can result in an ultimatum which draws in neighbours until Australians end up fighting Turks and the First World War is well under way, to be followed by communism, the Second World War and the Cold War. The challenge of understanding complex systems includes the fact that many of them have multiple attractors, including what are called "strange attractors".  103  Changes are close to linear as long as the system does not change very much, but once it is pushed out of balance it will get closer to other attractors, and when those become strong enough the system will tend to move towards chaos until a new balance is achieved around the new attractor.  104  None of the risks in this report is likely to result directly in an infinite impact, and some cannot do so physically. All the risks however are big enough to reach a threshold where the social and ecological systems become so unstable that an infinite impact could ensue, as the graph below shows. This graph and its accompanying text explain, how an event that reaches a threshold level could cascade into even worse situations, via civilisation collapse 105 to human extinction. The graph also seeks to illustrate the importance of ensuring ecological and social resilience, the two major insurance policies we have against a negative spiral after a major impact that takes us beyond the infinite threshold. 1. Social and ecosystem resilience. Resilient systems are naturally resistant to collapse, though this often comes at the cost of efficiency.  106  The more resilient the system, the more likely it is to be able to adapt to even large disasters. Improving resilience ahead of time can improve outcomes, even if the nature of the disaster isn't known. 2. General pre-risk collapse countermeasures. This category consists of all those measures put into place ahead of time to prevent civilisation collapse. It could include, for instance, measures to ensure continuity of government or prevent breakup of countries (or to allow these breakups to happen with the minimum of disruption). At the same time it should be noted that these kinds of measures could also trigger the breakdown.
 General mitigation and resilience. This category consists of all measures that can reduce the impact of risks and prevent them getting out of hand (excluding social and ecosystem measures, which are important and general enough to deserve their own category). 4. Pre-risk rebuilding enablers. On top of attempting to prevent collapses, measures can also be taken to enable rebuilding after a collapse.  107  This could involve building stores of food, of technology, or crucial reconstruction tools.  108  Alternatively, it could involve training of key individuals or institutions (such as the crews of nuclear submarines) to give them useful post-collapse skills. 5. Long-term impact. Some risks (such as climate change) have strong long-term impacts after years or even decades. Others (such as pandemics) are more likely to have only a short-term impact. This category includes only direct longterm impacts. 6. Post-risk politics. The political structures of the post-risk world (governmental systems, conflicts between and within political groupings, economic and political links between groups) will be important in determining if a large impact leads ultimately to civilisation collapse or if recovery is possible. 7. Post-risk collapse countermeasures. These are the countermeasures that the postrisk political structures are likely to implement to prevent a complete civilisation collapse.
 Maintaining a technology base. Current society is complex, with part of the world's excess production diverted into maintaining a population of scientists, engineers and other experts, capable of preserving knowledge of technological innovations and developing new ones. In the simpler post-collapse societies, with possibly much lower populations, it will be a challenge to maintain current technology and prevent crucial skills from being lost.  109  9. Post-collapse politics. Just as post-risk politics are important for preventing a collapse, post-collapse politics will be important in allowing a recovery. The ultimate fate of humanity may be tied up with the preservation of such concepts as human rights, the scientific method and technological progress. 10. Post-collapse external threats and risks. Simply because a risk has triggered the collapse of human civilisation, that does not mean that other risks are no longer present. Humanity will have much less resilience to deal with further damage, so the probability of these risks is important to determine the ultimate fate of humanity. 11. Anthropic effects. We cannot observe a world incapable of supporting life, because we could not be alive to observe it. When estimating the likelihood of disasters and recovery it is very important to take this effect into consideration and to adjust probability estimates accordingly.  110  12. Long-term reconstruction probability. A post-collapse world will differ significantly from a preindustrial revolution world. Easy access to coal and oil will no longer be possible. In contrast, much usable aluminium will have been extracted and processed and will be left lying on the surface for easy use. Thus it will be important to establish how technically possible it may be to have a second industrial revolution and further reconstruction up to current capabilities without creating the problems that the first industrial revolution resulted in.
 59 Global Challenges -Twelve risks that threaten human civilisation - For the selection of events information from specialised bodies and scientific journals in the area of global risk was gathered.  111  Using keywords related to the various risks, a global selection of events was sought, along with original sourcing in academic or official sources. The list of events was then ranked based on their risk relevance, i.e. their effect on the probability and/or the impact of the challenge. To finalise the list, a group of experts was consulted by email and a draft overview of the challenges was presented at a workshop at the Future of Humanity Institute (FHI) in Oxford, where additional input was provided on selection and content. Issue experts were then consulted before the final list of events was established.  112  Four categories were used to classify the different events: 1. Policy: Global or national policy initiatives that affect probability and/or impact 2. Event:
 .2 Probability Many of the expected impacts of climate change are well known, including a warming climate, more severe storms and droughts, rising sea levels, ocean acidification, and damage to vulnerable ecosystems.  114  As for all risks there are uncertainties in the estimates, and warming could be much more extreme than the middle estimates suggest. Models tend to underestimate uncertainty  115  (especially where impact on humanity is concerned,  116  where the effect also depends on modellers' choices such as the discount rate  117  ), so there is a probability 118 that humanity could be looking at a 4°C  119  or even 6°C  120  warming in the coming decades. This could arise from positive feedback loops, such as the release of methane from permafrost  121  or the dieback of the Amazon rainforests,  122  that strengthen the warming effect. So far, efforts at curbing emissions have been only moderately successful and are still very far from what is needed.  123  The impact of global warming, whether mild or severe, would be felt most strongly in poorer countries. Adaptation that can address significant warming is often very expensive,  124  and many of the poorest countries are in the tropics and sub-tropics that would be hardest hit (they could become completely uninhabitable for the highest range of warming  125  ). Mass deaths and famines, social collapse and mass migration are certainly possible in this scenario. Combined with shocks to the agriculture and biosphere-dependent industries of the more developed countries, this could lead to global conflict and possibly civilisation collapse -to the extent that many experts see climate change as a national security risk  126  . Further evidence of the risk comes from indications that past civilisation collapses have been driven by climate change.  127  Extinction risk could develop from this if the remaining human groups were vulnerable to other shocks, such as pandemics, possibly exacerbated by the changed climate.  128  There is some evidence of 6°C climate change causing mass extinction in the past,  129  but a technological species such as ourselves might be more resilient to such a shock. A unique feature of the climate change challenge is what is called geo-engineering.  130  Though this could -if it works -reduce many impacts at a relatively low cost, it would not do so evenly. Geo-engineering would possibly reduce the impacts of climate change in some countries, benefitting them while leaving others to suffer.  131  This could lead to greater political instability. One of the most popular geo-engineering ideasstratospheric sulphate aerosols -suffers from the weakness that it must be continuous.  132  If for any reason it stopped (such as a civilisation collapse), warming would resume at a significantly higher pace, reaching the point where it would have been without geo-engineering. The speed of this rebound would put extra pressure on the ecosystem and the world's political system. So the biggest challenge is that geoengineering may backfire and simply make matters worse.  134  Five important factors in estimating the probabilities and impacts of the challenge: 29. The course of international politics is extremely hard to predict, even for political scientists.  135  65 1. Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks To constrain the rise in global average temperature to less than 2°C above pre-industrial levels, a maximum of around 565 -886 billion tonnes (Gt) of carbon dioxide could be emitted before 2050.  137  The world's proven fossil fuel reserves amount to 2,860 Gt of CO2, however, and are viewed as assets by companies and countries. Since it is likely that these assets cannot be realised, these entities are over-valued at current pricesarguably, a "carbon bubble." The report provides evidence that serious risks are growing for highcarbon assets, and aims to help investors and regulators manage these risks more effectively and prepare for a global agreement on emissions reductions. It indirectly highlights part of the challenge of emissions reductions: they will mean the loss of highly valuable assets to corporations and governments. 02-May-13: CO2 at 400 PPM for the first time in > 800,000 years  138  -Event The Mauna Loa carbon dioxide record, also known as the "Keeling Curve," is the world's longest unbroken record of atmospheric CO2 concentrations. It recently reached 400 ppm (parts per million) of CO2. Such concentrations have not been reached for at least 800,000 years,  139  placing humanity in a historically unprecedented situation. Prior to the Industrial Revolution, natural climate variations caused atmospheric CO2 to vary between about 200 ppm during ice ages and 300 ppm during the warmer inter-glacial periods. The last time concentrations were as high as they are now seems to have been during the Mid-Pliocene, about 3 million years before the present when temperatures were 2-3°C warmer, and in which geological evidence and isotopes agree that sea level was at least 15 to 25 m above today's levels with correspondingly smaller ice sheets and lower continental aridity.  -Human influence on the climate system is clear. This is evident from the increasing greenhouse gas concentrations in the atmosphere, positive radiative forcing, observed warming, and understanding of the climate system. It is extremely likely that human influence has been the dominant cause of the observed warming since the mid-20th century. -Each of the last three decades has been successively warmer at the Earth's surface than any preceding decade since 1850. - -Continued emissions of greenhouse gases will cause further warming and changes in all components of the climate system. Limiting climate change will require substantial and sustained reductions of greenhouse gas emissions. -The oceans will continue to warm during the 21st century. Heat will penetrate from the surface to the deep ocean and affect ocean circulation. Further uptake of carbon by the oceans will increase ocean acidification. Global mean sea level will continue to rise during the 21st century. -It is very likely that Arctic sea ice cover will continue to shrink and become thinner. Global glacier volume will further decrease. -Most aspects of climate change will persist for many centuries even if emissions of CO2 are stopped. The global environment can be considered a global public good (i.e. non-excludable and non-rivalrous).  150  Economic theory claims that such goods will be undersupplied by the market.  151  Hence the importance of trans-national negotiations to address climate change. Despite the importance of the subject, the main achievement of the Warsaw negotiations was to keep talks on track for more negotiations in 2015.  152  Though there was general agreement on the necessity of cutting carbon emissions, the dispute was over how to share the burden of doing so. In this instance, the debate was between more-and less-developed countries, with the latter demanding compensation from the former to help them cope with the burden of reducing emissions. That particular dispute was papered over,  153  but similar ones will be likely in future due to the range of different actors and their divergent agendas. The likelihood of a full-scale nuclear war between the USA and Russia has probably decreased in recent decades due to some improvements in relations between these two countries and reductions in the size of their arsenals. Still, the potential for deliberate or accidental 165 nuclear conflict has not been removed, with some estimates putting the risk of nuclear war in the next century or so at around 10% 166 -it may have been mostly down to luck that such a war did not happen in the last half century 167 . A nuclear war could have a range of different impacts. At the lowest end is the most obvious and immediate impact: destruction and death in major cities across the world, due to the explosions themselves and the radioactive fallout. But even if the entire populations of Europe, Russia and the USA were directly wiped out in a nuclear war -an outcome that some studies have shown to be physically impossible  168  , given population dispersal and the number of missiles in existence 169 -that would not raise the war to the first level of impact, which requires > 2 billion affected.  170  A larger impact would depend on whether or not the war triggered what is often called a nuclear winter or something similar.  171  The term refers to the creation of a pall of smoke high in the stratosphere that would plunge temperatures below freezing around the globe and possibly also destroy most of the ozone layer.  172  The detonations would need to start firestorms in the targeted cities, which could lift the soot up into the stratosphere.  173  There are some uncertainties about both the climate models and the likelihood of devastating firestorms, 174 but the risks are severe and recent models 175 have confirmed the earlier 176 analysis. Even a smaller nuclear conflict (between India and Pakistan, for instance) could trigger a smaller nuclear winter which would place billions in danger.  177  The disintegration of the global food supply would make mass starvation and state collapse likely. As the world balance of power would be dramatically shifted and previous ideological positions called into question, large-scale war would be likely. This could lead to a civilisation collapse. Extinction risk is only possible if the aftermath of the nuclear war fragments and diminishes human society to the point where recovery becomes impossible  178  before humanity succumbs  179  to other risks, such as pandemics.  180  Five important factors in estimating the probabilities and impacts of the challenge: 1. How relations between current and future nuclear powers develop. 2. 5. The security of nuclear weapons and materials affects both the probability of nuclear terrorism and the control likelihood of nuclear accidents. 6. The relations between future nuclear powers will be the major determinant of whether a nuclear war breaks out. 7. The relations between current nuclear powers will be a major determinant of the relations between future nuclear powers. 8. The relations between future major nuclear powers will be the major component of determining whether a major nuclear war breaks out. 9. Relations between the USA and Russia (the only current major nuclear powers) will be a major determinant of the relations between future major nuclear powers. and led to increased sanctions  184  against the already isolated nation.  185  North Korea is the only nation to have withdrawn from the Nuclear Non-Proliferation Treaty,  186  and is the only country to have conducted nuclear tests in the 21st century, starting in 2006,  187  as well as developing a ballistic missile capability.  188  It has also been involved in the export of weapons technology, undermining the Treaty.  189  Diplomatic attempts to deal with North Korea (especially on the part of the United States) have generally been inconsistent and unsuccessful.  190  Though the situation remains a potential flashpoint for conventional and nuclear conflict, and its collapse could have disastrous consequences   196  This is but one of the many nuclear accidents  197  and incidents that peppered the Cold War and its aftermath, and which have been revealed only subsequently. We know now that there were at least three occasions -the Cuban missile crisis in 1962,  198  the Petrov incident in 1983  199  and the Norwegian rocket incident in 1995  200  -where a full-scale nuclear war was only narrowly averted.  201  Further information on these incidents, and on how they were interpreted and misinterpreted  202  by the great powers, will be important to estimate the probability of nuclear conflict in the coming decades. On a more positive note, efforts are being made to reduce the probability of inadvertent or accidental nuclear conflicts.  203  24-Jun-13: Report: "Analysing and Reducing the Risks of Inadvertent Nuclear War Between the United States and Russia"  204  -Research Though the end of the Cold War has reduced the likelihood of deliberate nuclear war, its impact on the risk of accidental nuclear war is much smaller. The arsenals remain on "launch on warning",  205  meaning that there is a possibility for a "retaliatory" strike before an attack is confirmed. The most likely cause of such an accident is either a false warning (of which there have been many, with causes ranging from weather phenomena to a faulty computer chip, wild animal activity, and controlroom training tapes loaded at the wrong time)  206  or a misinterpreted terrorist attack.  207  The report attempted a rigorous estimate of the numerical probability of nuclear war. Such numerical rigour is rare, with the exception of Hellman's estimates.  208  This report applied risk analysis methods using fault trees and mathematical modelling to assess the relative risks of multiple inadvertent nuclear war scenarios previously identified in the literature. Then it combined the fault tree-based risk models with parameter estimates sourced from the academic literature, characterising uncertainties in the form of probability distributions, with propagation of uncertainties in the fault tree using Monte Carlo simulation methods. Finally, it also performed sensitivity analyses to identify dominant risks under various assumptions. This kind of highly disaggregated analysis is most likely to elicit the best performance and estimates from experts.  209  Their conclusion was that (under the more pessimistic assumption), there was a mean 2% risk of accidental nuclear war a year (a high risk when compounded over several decades), with the risk from false alarm being orders of magnitude higher than that from terrorist attacks. The analysis suggests that the most important inadvertent nuclear war risk factor is the short launch decision times,  210  inherent in the "launch on warning" posture. Some ways of improving this were suggested, for instance by moving each country's strategic submarines away from the other's coasts.
 75 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks   217  This further serves to emphasise the weakness of international institutions where nuclear arms control is concerned. 15-Nov-13: International Physicians for the Prevention of Nuclear War report: "Nuclear Famine: Two Billion People at Risk?"  218  -Research This report is one of a series of reports and publications in recent years about the potential impacts of nuclear conflicts.  219  It looked at the likely consequences of a "limited" nuclear war, such as between India and Pakistan. While previous papers had estimated that up to a billion people might be at risk in such a conflict,  220  this report increased the estimate to two billion. The main source of this increase is decreased agricultural production in the United States  221  and in China.  222  A key component of these estimates was the severe agricultural impact of the relatively mild temperature reduction in 1816, the "year without a summer"  223  , due mainly to the "volcanic winter" caused by the eruption of Mount Tambora. The report highlights some significant areas of uncertainty, such as whether a small nuclear conflict and its consequences would lead to further conflicts across the world, and doubts whether markets, governments and other organisations could mitigate the negative impacts. The report is a reminder that even small-scale nuclear conflict could have severe consequences. 24-Nov-13: Nuclear deal with Iran may reduce risk of proliferation  224  -Policy In November, Iran struck a deal with the so called "P5+1" (the five permanent members of the security council, plus Germany). The deal, if it holds, would allow Iran to continue some uranium enrichment, but it would have to submit to inspections to ensure it wasn't developing a nuclear weapons programme (the deal would also result in eased sanctions in return). There have been longrunning fears than Iran may have been attempting to construct a nuclear weapon  225  , resulting in sanctions being imposed on it.  226  This event illustrates the surprising success of the Non-Proliferation Treaty,  227  which came into force in 1970. At the time it was proposed there were fears of very rapid proliferation of nuclear weapons.  228  And though 40 countries or more currently have the knowhow to build nuclear weapons,  229   Species extinction is proceeding at a greatly increased rate compared with historic data  232  , and attempts to quantify a safe ecological operating space place humanity well outside it.  233  Furthermore, there may be signs of a "sudden" biosphere collapse, possibly within a few generations.  234  Many of the problems of ecological degradation interact to multiply the damage and (unlike previous, localised collapses) the whole world is potentially at risk,  235  with severe challenges to countering this risk through global policy.  236  If animals are seen to have intrinsic value,  237  or if human quality of life is dependent on a functioning ecosystem,  238  the current situation already represents a large loss. Whether such a loss will extend to human lives depends on technological and political factors -technological, because it seems plausible that some human lifestyles could be sustained in a relatively ecosystem-independent way, at relatively low costs.  239  Whether this can be implemented on a large scale in practice, especially during a collapse, will be a political challenge and whether it is something we want is an ethical question. There is currently more than enough food for everyone on the planet to ensure the nutrition needed,  240  but its distribution is extremely uneven and malnutrition persists. Thus ecological collapse need not have a strong absolute effect in order to result in strong localised, or global, effects. Even a partial collapse could lead to wars, mass migrations, and social instability. It is conceivable that such a scenario, if drawn out and exacerbated by poor decision-making, could eventually lead to mass deaths and even the collapse of civilisation. Extinction risk is possible only if the aftermath of collapse fragments and diminishes human society so far that recovery becomes impossible  241  before humanity succumbs to other risks (such as climate change or pandemics). After a post-civilisation collapse, human society could still be suffering from the effects of ecological collapse, and depending on what form it took, this could make the recovery of human civilisation more challenging than in some of the other scenarios presented here. Five important factors in estimating the probabilities and impacts of the challenge:   243  And yet this biodiversity is being lost at an alarming rate -the rate of extinctions for plants and animals is 100 to 1,000 times higher than their pre-human levels.  244  A variety of methods have been suggested to halt or slow this loss, ranging from putting an explicit value  245  on biodiversity and ecosystem services (human benefits from a multitude of resources and processes that are supplied by ecosystems),  246  to performing triage on the most valuable species.  247  This research paper suggests, however, that there is a lag of several decades between human pressure on the ecosystem and ultimate species extinction. This suggests that many extinctions will continue in decades to come, irrespective of current conservation efforts. 1.
 05-Apr-13: Ocean data added to Microsoft Eye on Earth project -Initiative In order to safeguard ecological resources, it is important to track and quantify them. This has traditionally been the role of governments or non-governmental organisations.  248     254  (thus privatising that "common"). A typical example of this behaviour is the collapse of the Grand Banks fisheries off Canada's Atlantic coast in the 1990s, where cod biomass fell by over 95% from its peak and has currently not recovered.  255  It is therefore significant that the European Union has been partly successful in its attempts to control over-fishing through legislation. For instance, despite the fact that North Sea cod remains vulnerable, there has been a recent increase in stock size and a decrease in fish mortality. This may point to the potential for further ecological improvements through well-chosen policy interventions. In 2013 the IUCN added an additional 4,807 species to its Red List of Threatened Species. This brings the total to about 21,000. Some have argued that we are entering a new geological era in Earth's history: the Anthropocene  257  , when human actions are one of the major impactors on the planet's biosphere. 82 The graph shows a fairly steady growth in the (estimated) number of threatened species. This steadiness may be illusory, as the biosphere shows signs that it may be approaching a planetary-scale tipping point, where it may shift abruptly and irreversibly from one state to another. As a result, the biological resources humans presently take for granted may be subject to rapid and unpredictable transformations within a few human generations.  258  This could be seen as a great tragedy beyond purely human concerns, if animals (and animal welfare) are seen to have intrinsic value.  259  83 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Here only worldwide events are included. A widespread endemic disease that is stable in terms of how many people become sick from it is not a pandemic. Influenza subtypes  266  Infectious diseases have been one of the greatest causes of mortality in history. Unlike many other global challenges pandemics have happened recently, as we can see where reasonably good data exist. Plotting historic epidemic fatalities on a log scale reveals that these tend to follow a power law with a small exponent: many plagues have been found to follow a power law with exponent 0.26.  261  These kinds of power laws are heavy-tailed  262  to a significant degree.  263  In consequence most of the fatalities are accounted for by the top few events.  264  If this law holds for future pandemics as well,  265  then the majority of people who will die from epidemics will likely die from the single largest pandemic. Most epidemic fatalities follow a power law, with some extreme events -such as the Black Death and Spanish Flu -being even more deadly.  267  There are other grounds for suspecting that such a highimpact epidemic will have a greater probability than usually assumed. All the features of an extremely devastating disease already exist in nature: essentially incurable (Ebola  268  ), nearly always fatal (rabies  269  ), extremely infectious (common cold  270  ), and long incubation periods (HIV  271  ). If a pathogen were to emerge that somehow combined these features (and influenza has demonstrated antigenic shift, the ability to combine features from different viruses  272  ), its death toll would be extreme. Many relevant features of the world have changed considerably, making past comparisons problematic. The modern world has better sanitation and medical research, as well as national and supra-national institutions dedicated to combating diseases. Private insurers are also interested in modelling pandemic risks.  273  Set against this is the fact that modern transport and dense human population allow infections to spread much more rapidly  274  , and there is the potential for urban slums to serve as breeding grounds for disease.  275  Unlike events such as nuclear wars, pandemics would not damage the world's infrastructure, and initial survivors would likely be resistant to the infection. And there would probably be survivors, if only in isolated locations. Hence the risk of a civilisation collapse would come from the ripple effect of the fatalities and the policy responses. These would include political and agricultural disruption as well as economic dislocation and damage to the world's trade network (including the food trade). Extinction risk is only possible if the aftermath of the epidemic fragments and diminishes human society to the extent that recovery becomes impossible  277  before humanity succumbs to other risks (such as climate change or further pandemics). Five important factors in estimating the probabilities and impacts of the challenge: 1. What the true probability distribution for pandemics is, especially at the tail.
 The capacity of modern international health systems to deal with an extreme pandemic.
 How fast medical research can proceed in an emergency. 4. How mobility of goods and people, as well as population density, will affect pandemic transmission.
 Whether humans can develop novel and effective anti-pandemic solutions.
 85 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 2. As so much is known about pandemic risks compared with other risks, there are more possibilities for specific prepandemic contingency plans. 3. The effectiveness of healthcare systems will be important, especially in less developed nations where the pandemic may overwhelm the system, and then transmit from there to other nations. 4. Global coordination in detection, analysis and treatment are vital for stopping a pandemic in its early stages, and for implementing measures such as quarantines and more advanced countermeasures. 5. Poverty will affect the quality of national healthcare systems, population density and sanitation quality, the movement of local goods and people, and the effectiveness of the political response. 6. Bioterrorists may unleash a pathogen held in storage, such as smallpox. 7. Laboratory security at the top labs is insufficient for the danger at hand, and accidental release is a nonnegligible possibility. 8. Pandemics are one of the risks where there is a possibility for a very large number of direct casualties, depending on the severity of the pathogen. 9. Mass casualties and finger-pointing could destabilise the world political and economic systems. 10. If the pathogen is transmissible to farm animals, this could affect the world food supply. 11. It is unlikely the pathogen would be a recurrent, long-term risk, but variants of it could continue to affect people and animals for many years, dependent on its transmissibility and life cycle. 12. Small pandemic scares could improve global coordination on the issue. 13. Increased population density causes increased transmissibility of the pathogen, especially in urban slums. 14. Some pathogens, such as bird flu, depend on regular contact between humans and "reservoir species" in order to evolve into periodically dangerous strains. 15. If antibiotic resistance develops, humanity could see the resurgence of bacteria-based pandemics. 16. The increased movement of people and products increases the speed and spread of pandemic transmission. 17. Sanitation or its lack will strongly affect the spread of certain pathogens in key areas. 18.   284  The main lesson the WHO drew from that epidemic was that member states generally had communication issues (between ministries of health and decision,makers, and with the public), and were prepared for a pandemic of high severity and appeared unable to adapt their national and subnational responses adequately to a more moderate event. The guidance paper indicates simultaneously the weaknesses of pandemic preparations, the improvements in these preparations, and the continued role of the WHO as global directing and coordinating authority. 24-Jul-13: Bacteria become resistant to some of the last remaining antibiotics  285  -Event Bacterial infections, such as the Black Death,  286  syphilis,  287  and tuberculosis,  288  have been responsible for millions of deaths, over the thousands of years they have co-existed with humanity. Though these diseases have not been eradicated -overall, a third of the world is currently infected with the tuberculosis bacillus  289  -they have been controlled since the introduction of antibiotics, and prognostics have improved tremendously. But recently a rising number of bacteria have developed antibiotic resistance, due mainly to antibiotic over-prescription  290  and use in livestock feed.  291  This Nature report highlights the worrying way in which Enterobacteriaceae (bacteria with a 50% mortality rate) have become resistant to carbapenems, one of the last remaining antibiotics that had been effective against them. 09-Aug-13: Epihack: Digital disease surveillance hack-a-thon  292  -Initiative Beyond the formal, top-down initiatives to deal with pandemics, there are openings for bottom-up, innovative ideas. Epihack attempted to generate just such ideas, through three days of designing and hacking in Cambodia. Descriptions of the winning projects were given: -CoPanFlu: This project included home visits to collect blood samples from 807 homes and weekly follow-up phone calls to document the occurrence of infectious respiratory symptoms. These visits and phone calls caused disturbance to the participants. The new system uses SMS for users to report symptoms. Chart and map visualisation of the data (with full case details) and a fieldwork tracking tool were developed to help the research team analyse and monitor data. -DoctorMe: In addition to all of the popular features of DoctorMe (free health information for the general public), the tool now features a weekly survey for users. The survey will ask participants to select whether they are experiencing any symptoms from a list.
 88 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks The Spanish flu outbreak was the deadliest short pandemic in history, infecting about a third of the world population (≈ 500 million people) and killing 50-100 million people.  294  There have been numerous flu pandemics in the last few centuries, with three others having around a million casualties (the <PHONE> Russian Flu,  295  the <PHONE> Asian Flu, and the <PHONE> Hong Kong Flu 296 outbreaks). The most recent pandemic was that in 2009, which killed 150,000-500,000 people.  297  Thus any move towards a universal flu vaccine would be of great importance to combating such recurring pandemics. This paper, analysing the role of T cells in combating influenza, suggests a way that such a vaccine could be feasible. 28-Nov-13: Difficulties in containing the accidental laboratory escape of potential pandemic influenza viruses  298  -Research Biosafety laboratories experiment with some of the deadliest of the world's pathogens, and occasionally create new ones.  299  Their number is increasing globally, and their safety record is far from perfect, with several pathogen leaks reported 300 and others suspected  301  (the last smallpox fatality was due to a virus that escaped a lab  302  , after eradication of the virus in the wild). The rate of pathogen escape has been estimated at 0.3% per laboratory, per year  303  -a very high probability, given the 44 BSL-4 304 labs and several thousands of BSL-3 labs. There have already been three known escapes from BSL-4 labs since 1990.  305  This report uses an agent-based model to analyse whether the accidental laboratory release of pandemic flu viruses could be contained, and concludes that controllability of escape events is not guaranteed. 3-Dec-13: Global pandemic tops poll of insurance industry risks  306  -Initiative Academics and governmental  307  / supra-governmental  308  organisations have long worried about the risks of pandemics. But such organisations attract certain types of people with specific outlooks, who can be subject to further biases because of their profession and the social milieu surrounding it.  309  Insurers come from a different background, focusing on practical profitability in the business world. It is therefore instructive that they too see pandemics as among the major threats in the world today. This also implies that combating pandemics is of use not only from a humanitarian but also from an economic standpoint.
 89 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
 Current risks
 Global System Collapse Major Asteroid Impact Synthetic Biology Unknown Consequences
 Current risks
 System Collapse 3.1.5 Global
 Global system collapse is defined here as either an economic or societal collapse on the global scale. There is no precise definition of a system collapse. The term has been used to describe a broad range of bad economic conditions, ranging from a severe, prolonged depression with high bankruptcy rates and high unemployment, to a breakdown in normal commerce caused by hyperinflation, or even an economically-caused sharp increase in the death rate and perhaps even a decline in population.
 Probability Often economic collapse is accompanied by social chaos, civil unrest and sometimes a breakdown of law and order. Societal collapse usually refers to the fall or disintegration of human societies, often along with their life support systems. It broadly includes both quite abrupt societal failures typified by collapses, and more extended gradual declines of superpowers. Here only the former is included. The world economic and political system is made up of many actors with many objectives and many links between them. Such intricate, interconnected systems are subject to unexpected system-wide failures due to the structure of the network  311  -even if each component of the network is reliable. This gives rise to systemic risk: systemic risk occurs when parts that individually may function well become vulnerable when connected as a system to a self-reinforcing joint risk that can spread from part to part (contagion), potentially affecting the entire system and possibly spilling over to related outside systems.  312  Such effects have been observed in such diverse areas as ecology,  313  finance  314  and critical infrastructure  315  (such as power grids). They are characterised by the possibility that a small internal or external disruption could cause a highly non-linear effect,  316  including a cascading failure that infects the whole system,  317  as in the 2008-2009 financial crisis. The possibility of collapse becomes more acute when several independent networks depend on each other, as is increasingly the case (water supply, transport, fuel and power stations are strongly coupled, for instance).  318  This dependence links social and technological systems as well.  319  This trend is likely to be intensified by continuing globalisation,  320  while global governance and regulatory mechanisms seem inadequate to address the issue.  321  This is possibly because the tension between resilience and efficiency 322 can even exacerbate the problem.  323  Many triggers could start such a failure cascade, such as the infrastructure damage wrought by a coronal mass ejection,  324  an ongoing cyber conflict, or a milder form of some of the risks presented in the rest of the paper. Indeed the main risk factor with global systems collapse is as something which may exacerbate some of the other risks in this paper, or as a trigger. But a simple global systems collapse still poses risks on its own. The productivity of modern societies is largely dependent on the careful matching of different types of capital  325  (social, technological, natural...) with each other. If this matching is disrupted, this could trigger a "social collapse" far out of proportion to the initial disruption.  326  States and institutions have collapsed in the past for seemingly minor systemic reasons.  327  And institutional collapses can create knock-on effects, such as the descent of formerly prosperous states to much more impoverished and destabilising entities.  328  Such processes could trigger damage on a large scale if they weaken global political and economic systems to such an extent that secondary effects (such as conflict or starvation) could cause great death and suffering. Five important factors in estimating the probabilities of various impacts: 1. Whether global system collapse will trigger subsequent collapses or fragility in other areas.
 What the true trade-off is between efficiency and resilience. 3. Whether effective regulation and resilience can be developed. 4. Whether an external disruption will trigger a collapse. 5. Whether an internal event will trigger a collapse.
 91 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 4. Building resilience -the ability of system components to survive shocks -should reduce systemic risk. 5. Fragile systems are often built because they are more efficient than robust systems, and hence more profitable. 6. General mitigation efforts should involve features that are disconnected from the standard system, and thus should remain able to continue being of use if the main system collapses 7. A system collapse could spread to other areas, infecting previously untouched systems (as the subprime mortgage crisis affected the world financial system, economy, and ultimately its political system). 8. The system collapse may lead to increased fragility in areas that it does not directly damage, making them vulnerable to subsequent shocks. 9. A collapse that spread to government institutions would undermine the possibilities of combating the collapse. 10. A natural ecosystem collapse could be a cause or consequence of a collapse in humanity's institutions. 11. Economic collapse is an obvious and visible way in which system collapse could cause a lot of damage. 12. In order to cause mass casualties, a system collapse would need to cause major disruptions to the world's political and economic system. 13. If the current world system collapses, there is a risk of casualties through loss of trade, poverty, wars and increased fragility. 14. It is not obvious that the world's institutions and systems can be put together again after a collapse; they may be stuck in a suboptimal equilibrium. 15. Power grids are often analysed as possible candidates for system collapse, and they are becoming more integrated. 16. "The Centre will undertake an economic analysis of the fundamental risks to the financial system, based on an interdisciplinary approach. It will bring together experts from finance, economics, computer science, political science, law and the natural and mathematical sciences. This will allow researchers affiliated to the Centre to investigate how risk is created through feedback loops within and between the financial, economic, legal and political systems. Political decisions, for example, can directly affect people's behaviour in the financial markets, which in turn affects political decision-making and so onwith the outcomes being unexpected and complex." Besides the research results produced by the centre, its very existence shows that systemic risk is being taken seriously in academic quarters. 14-Mar-13: Systemic sovereign credit risk has "deep roots in the flows and liquidity of financial markets."  330  -Research It is important to estimate the source of systemic risk. Different mitigation policies should be implemented if sovereign systemic risks spring from financial markets rather than macroeconomic fundamentals. This paper argues that systemic sovereign risks spring from financial markets (through capital flows, funding availability, risk premiums, and liquidity shocks  331  ) rather than from fundamentals.  332  It further estimates that systemic risks are three times larger in eurozone countries than in US states. In order to mitigate or prevent systemic risk, it needs to be monitored. In this paper, the authors set out to clarify the nature and use of the systemic risk monitoring tools that are currently available, providing guidance on how to select the best set of tools depending on the circumstances. The paper breaks down the tools into four categories, each with their strengths and weaknesses: -Single risk/soundness indicators.  334  Indicators based on balance sheet data, such as financial soundness indicators (FSIs), are widely available and cover many risk dimensions. However, they tend to be backward-looking and do not account for probabilities of default or correlation structures. Moreover, only some of these indicators can be used as early-warning tools (e.g., indicators of funding structures). Market data can be used to construct complementary indicators for higher-frequency risk monitoring. -Fundamentals-based models  335  rely on macroeconomic or balance sheet data to help assess macro-financial linkages (e.g., macro stress testing or network models). By providing vulnerability measures based on actual interconnectedness and exposures, these models may help build a realistic "story". However, they often require long-term data series, assume that parameters and relationships are stable under stressed conditions, and produce only low-frequency risk estimates. -Market-based models.  336  These models uncover information about risks from high-frequency market data and are thus suitable for tracking rapidly-changing conditions of a firm or sector. These approaches are more dynamic, but their capacity to reliably predict financial stress has yet to be firmly established. -Hybrid, structural models.  337  These models estimate the impact of shocks on key financial and real variables (e.g., default probabilities, or credit growth) by integrating balance sheet data and market prices. Examples include the CCA and distance-to-default measures, which compare the market value of an entity's assets to its debt obligations. The paper concludes, however, that the systemic risk monitoring toolkit is incomplete and that "tools exist to assess most sectors and levels of aggregation, but they provide only partial coverage of potential risks and only tentative signals on the likelihood and impact of systemic risk events. As such, they may not provide sufficient comfort to policymakers." 23-Dec-13: Citigroup analysis reports reduced systemic political and financial risks in 2013 and 2014  338  -Initiative Tracking the ebb and flow of the likelihood of various risks is important for estimating where best to direct energy and resources. Even approximate, order of magnitude estimates are sufficient if they establish that some risks are much more dangerous than others (order of magnitude estimates correspond to the "Class 5 cost estimate",  339  undertaken at the very beginning of the project, between 0% and 2% of its completion). In 2013, Citigroup analysts predicted that (with caveats) systemic risks would recede in Europe during the year, a prediction which seems to have been vindicated by events. As for the future, Tina Fordham, chief global political analyst at Citigroup Global Markets, predicted that "systemic political risks will decline in 2014, but country-level and geopolitical risks remain significant." It seems positive both that market analysts are tracking systemic risks and that they see them as decreasing (though their focus is mainly on political and financial systemic risks).
 95 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Asteroids have caused significant extinction events throughout the Earth's history. The most famous is the Chicxulub impactor, which probably helped cause the extinction of the non-avian dinosaurs and more than 75% of all species.  341  Large asteroid collisions -objects 5 km or more in size -happen approximately once every twenty million years and would have an energy a hundred thousand times greater  342  than the largest bomb ever detonated.  343  A land impact would destroy an area the size of a nation like Holland.  344  Larger asteroids could be extinction level events. Asteroid impacts are probably one of the best understood of all risks in this report. Their mechanisms and frequencies are reasonably well estimated.  345  Recent ground-and space-based  346  tracking projects have been cataloguing and tracking the largest asteroids,  347  and have discovered that the risks were lower than was previously feared.  348  The projects are now cataloguing asteroids of smaller size and damage potential. There has been some speculation about possible methods for deflecting asteroids  350  , should they be found on a collision course with the planet. Such means remain speculative, currently, but may become more feasible given technological progress and potentially more affordable access to space.  351  Should an impact occur, though, asteroid impact risks are similar to those of super-volcanoes, in that the main destruction will not be wrought by the initial impact, but by the clouds of dust projected into the upper atmosphere. The damage from such an "impact winter" could affect the climate, damage the biosphere, affect food supplies, and create political instability. Though humanity currently produces enough food to feed all humans,  352  this supply is distributed extremely unevenly, and starvation still exists. Therefore a disruption that is small in an absolute sense could still cause mass starvation in the future. Mass starvation, mass migration, political instability and wars could be triggered, possibly leading to a civilisation collapse. Unless the impact is at the extreme end of the damage scale and makes the planet unviable, human extinction is possible only as a consequence of civilisation collapse and subsequent shocks.  353  Five important factors in estimating the probabilities and impacts of the challenge: 1. Whether detection and tracking of asteroids and other dangerous space objects is sufficiently exhaustive. 2. How feasible it is to deflect an asteroid. 3. Whether measures such as evacuation could reduce the damage of an impact. 4. The short-and long-term climate consequences of a collision. 5. Whether our current civilisation could adapt to a post-impact world.  2. National space programmes have always provided the impetus for space flight projects, especially the more speculative and cutting-edge ones. 3. Protecting against asteroid impacts is already accepted as a project worth funding, but increased focus on the problem could increase the ability to predict and prevent such impacts. 4. Asteroid detection and tracking continues to progress well currently, and is key to preventing such collisions in future. 5. Better global coordination is not strongly needed to track or deflect asteroids, but would be important if a large-scale evacuation was needed. 6. General mitigation efforts may help reduce the direct and indirect negative outcomes of an impact by, for instance, equipping people to deal with the changed climate. 7. Unlike many risks, there is no upper bound on how destructive an asteroid impact could be, though the largest impacts are the rarest. 8. The aftermath of an impact could greatly disrupt the world economic and political system. 9. Climate changes would be the most destructive consequences of mediumscale meteor impacts, with the world plunged into an "impact winter". 10. The effects of an impact winter could last for a long time. 11. Easier access to space would be important for any plans to actually deflect an asteroid. 12. There are currently no asteroid deflection abilities, but there are many plans that could conceivably be implemented in due course. 13. Small asteroid impacts could motivate increased anti-asteroid precautions. 14. With enough warning, it could be possible to preemptively evacuate the impact area. 15. Post-impact politics will be important for reconstruction, adapting to the changed climate, and prevention of further harm. 16. Estimating the likelihood of asteroid impacts suffers from "anthropic shadow" effects:  355  we may be underestimating the danger because if there had been many more impacts in recent times, humans would not currently be around to observe their effects and take them into account.
 99 Global Challenges -Twelve risks that threaten human civilisation -   357  when an object hit Tunguska in Siberia.  358  The meteor seemed ideal from the risk reduction perspective: a large, visible impact that attracted great attention, and a renewed commitment to asteroid precautions,  359  but no actual fatalities. 19-Jun-13: Space Research Institute of Russian Academy of Science presents a strategy to use small asteroids to deflect hazardous objects from the trajectory of collision with Earth 360 -Research Though the analysis and tracking of asteroids has progressed rapidly,  361  methods for deflecting a dangerous asteroid, should it be detected, remain speculative.  362  The Space Research Institute of the Russian Academy of Science introduces another approach: selecting small (10-15m) near-Earth asteroids and causing them to strike a larger dangerous one, altering its trajectory. The more suggestions and ideas there are for such deflections, the more likely it is that one of them will yield an implementable approach. 17-Oct-13: The probability for "Asteroid 2013 TV135" to impact Earth in 2032 is one in 63,000  363  -Event NASA reports that a 400-metre asteroid has one chance in 63,000 of impacting the Earth. An asteroid this size would produce oceanwide tsunamis or destroy land areas the size of a small state (Delaware, Estonia).  364  For comparison, the odds of dying from lightning strike are 1 in 83,930, of a snake, bee or other venomous bite or sting is 1 in 100,000, of an earthquake 1 in 131,890, and of a dog attack 1 in 147,717.  365  So the risk of asteroid death, though low, is comparable to more common risks. 28-Oct-13: United Nations to Adopt Asteroid Defence Plan 366 -Policy The UN plans to set up an International Asteroid Warning Group for member nations to share information about potentially hazardous space rocks. If astronomers detect an asteroid that poses a threat to Earth, the UN's Committee on the Peaceful Uses of Outer Space will help coordinate a mission to launch a spacecraft to slam into the object and deflect it from its collision course. This marks the first time an international body has assigned responsibility for tracking and intercepting dangerous asteroids. 14-Nov-13: Risk of medium asteroid strike may be ten times larger than previously thought  367  -Research This paper analyses in detail the Chelyabinsk impact, estimated to have had an energy of 500 kilotonnes of TNT. It demonstrates problems with the standard methods for estimating the energy of collisions -derived from nuclear weapons results  368  -and from that deduces that the number of impactors with diameters of tens of metres may be an order of magnitude higher than estimated. It argues that this demonstrates a deviation from a simple power law, and thus that there is a nonequilibrium in the near-Earth asteroid population for objects 10 to 50 metres in diameter. This shifts more of the impact risk to asteroids of these sizes. 3-Dec-13: SpaceX launches into geostationary orbit  369  -Initiative Easy access to space is important for all asteroid deflection proposals.  370  Since America retired the Space Shuttle,  371  it has been putting its hope in private space companies.  372  The success of SpaceX opens the possibility of eventual cheaper access to space. 100 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
 Exogenic risks Figure  15 : Impact effects by size of Near Earth Object  354
 Consequences Upper atmosphere detonation of "stones" (stony asteroids) and comets; only "irons" (iron asteroids) <3% penetrate to surface. Irons make craters (Barringer Crater); Stones produce air-bursts (Tunguska). Land impacts could destroy areas the size of a city (Washington, London, Moscow). Irons and stones produce ground-bursts; comets produce air-bursts. Ocean impacts produce significant tsunamis. Land impacts destroy areas the size of a large urban area (New York, Tokyo). Impacts on land produce craters; ocean-wide tsunamis are produced by ocean impacts. Land impacts destroy areas the size of a small state (Delaware, Estonia). The eruption which formed the Siberian Traps was one of the largest in history. It was immediately followed by the most severe wave of extinction in the planet's history,  374  the Permian-Triassic extinction event,  375  where 96% of all marine species and 70% of terrestrial vertebrate species died out. Recent research has provided evidence of a causal link: that the eruption caused the mass extinction.  376  There have been many other super-volcanic eruptions throughout history.  377  The return period for the largest supervolcanoes (those with a Volcanic Explosivity Index 378 of 8 or above) has been estimated from 30,000 years  379  at the low end, to 45,000 or even 700,000 years  380  at the high end. Many aspects of super-volcanic activity are not well understood as there have been no historical precedents, and such eruptions must be reconstructed from their deposits.  381  The danger from super-volcanoes is the amount of aerosols and dust projected into the upper atmosphere. This dust would absorb the Sun's rays and cause a global volcanic winter. The Mt Pinatubo eruption of 1991 caused an average global cooling of surface temperatures by 0.5°C over three years, while the Toba eruption around 70,000 years ago is thought by some to have cooled global temperatures for over two centuries.  382  The effect of these eruptions could be best compared with that of a nuclear war. The eruption would be more violent than the nuclear explosions,  383  but would be less likely to ignite firestorms and other secondary effects. Unlike nuclear weapons, a super-volcano would not be targeted, leaving most of the world's infrastructure intact. The extent of the impact would thus depend on the severity of the eruption -which might or might not be foreseen, depending on improvements in volcanic predictions  384  -and the subsequent policy response. Another Siberian Trap-like eruption is extremely unlikely on human timescales, but the damage from even a smaller eruption could affect the climate, damage the biosphere, affect food supplies and create political instability. A report by a Geological Society of London working group notes: "Although at present there is no technical fix for averting supereruptions, improved monitoring, awareness-raising and research-based planning would reduce the suffering of many millions of people."  385  Though humanity currently produces enough food to feed everyone,  386  this supply is distributed extremely unevenly, and starvation still exists. Therefore a disruption that is small in an absolute sense could still cause mass starvation. Mass starvation, mass migration, political instability and wars could be triggered, possibly leading to a civilisation collapse. Unless the eruption is at the extreme end of the damage scale and makes the planet unviable, human extinction is possible only as a consequence of civilisation collapse and subsequent shocks.  387  Prof. Michael Rampino, New York University, has estimated that a large (1,000 cubic kilometres of magma) super-eruption would have global effects comparable to an object 1.5km in diameter striking the Earth.  388  Five important factors in estimating the probabilities and impacts of the challenge: 1. Whether countries will coordinate globally against super-volcano risk and damage.  2. Further super-volcano research will be important in any mitigation and monitoring efforts. 3. Global coordination and cooperation between nations will determine research levels, the chances of evacuations, and posteruption disruption to the world political and economic system. 4. General mitigation efforts may help reduce the direct and indirect negative impact of an eruption, by, for instance, equipping people to deal with the changed climate. 5. The direct destructive effect of a super-volcano can be extensive, especially in the area around the eruption. 6. A super-volcano's main destructive impact is through its effect on the climate, akin to a nuclear winter cooling effect. This will strongly affect all impact levels, and the disruption to the world's political and economic system. 7. The level of this disruption will determine how well countries cope with the aftermath of the eruption and subsequent climate changes, and whether subsequent conflicts or trade wars will occur, adding to the damage. 8. The long-term climate impact will determine in what state the posteruption world will find itself, relevant both for reconstruction after a collapse and for preventing such a collapse. 9. Whether eruptions are fundamentally predictable or not, and how far in advance, will be very important for many mitigation strategies. 10. Better volcano monitoring and prediction (if possible) will allow such interventions as preemptive evacuations. 11. Evacuations are likely to be the only effective response to an imminent eruption, as super-volcanoes are unlikely to be controllable or divertible. 12. Post-eruption politics will be a consequence of the number of shortterm casualties, and the disruption to the world system. 13. Medium scale volcanic eruptions may persuade leaders to make the risk more of a priority. 14. Estimating the likelihood of super-volcanic eruptions suffers from "anthropic shadow" effects:  390  we may be underestimating the danger because if there had been many more eruptions in recent times, humans would not currently be around to observe their effects and take them into account.
 .2.3 Main events 15-Mar-13: Climate impact of super-volcanoes may be less than previously thought  391  -Research The Toba eruption around 70,000 years ago was one of the world's largest super-volcanic eruptions. In contrast with some theories that claim it caused a volcanic winter that may have lasted over two centuries,  392  this paper claims that analysis of ash from the Toba super-eruption in Lake Malawi shows no evidence of volcanic winter in East Africa. This further illustrates the difficulty of establishing the exact impact of large-scale disasters when the evidence record is poor. 17-Jul-13: The Volcanological Society of Japan looks at volcano and super-volcano mitigation  393  -Policy Prevention of super-volcano eruptions is impossible with current technology, but there may be some possibility of mitigating their effects. The Volcanological Society of Japan is one of the few organisations that have looked at such potential mitigation. They put the risk of super-volcanic eruptions in the context of standard volcanic eruptions, just on a larger scale (noting that super-volcanic eruptions have affected Japan in the past). Japan has been a very seismically active country for its entire history,  394  so it might be hoped that adequate volcanic mitigation measures would have been implemented. But the report notes that "remarkably few [of Japan's local governments] have drafted volcanic disaster countermeasure[s]",  395  adding that "Local governments that have actually experienced a volcanic disaster focus attention on volcanic disaster-related discussion, but most have not drafted specific procedures for volcanic disasters and seem to think that the general disaster countermeasure volume is adequate." This provokes some pessimism about the likelihood of effective planetary super-volcano mitigation measures being implemented, especially in those areas with no direct experience of volcanic risk. This is due to the normalcy bias, "the tendency to minimise the probability of potential threats or their dangerous implications,".  396  27-Oct-13: Yellowstone supervolcano larger than previously thought  397  -Research Another continuing development in the science of super-volcanoes, this paper demonstrates that the crustal magma reservoir under Yellowstone was 50% larger than was previously thought. However, despite this increase, integrated probabilistic hazard assessment shows that the biggest Yellowstone Plateau threat is from large M7+ earthquakessignificantly damaging  398  , but very unlikely to threaten billions -not from volcanic or super-volcanic eruptions. 106 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3.2 Exogenic risks 15-Nov-13: Insurance executives rank super-volcanoes low on the list of extreme risks 399 -Initiative Academics have long worried about the probability of super-volcanic eruptions. But academia attracts certain types of people with specific outlooks, who can be subject to further biases because of their profession and the social milieu surrounding it.  400  Insurers come from a different background, focusing on practical profitability in the business world and using a relatively short time horizon. So it is instructive that they do not see super-volcanoes as a major threat in the world today: "Of interest to us is the very low ranking of the user-submitted idea of supervolcanoes in the US". 20-Dec-13: Super-volcano confirmed as responsible for one of the largest extinctions in history  401  -Research The maximal destructive potential of super-volcanoes is uncertain. There have been large supervolcanic eruptions throughout history,  402  and many extinction events, but uncertainties in the geological record mean that it was hard to establish whether they were causally linked. One eloquent example was the eruption which formed the Siberian Traps  403  (one of the largest in history), and the Permian-Triassic extinction,  404  where 96% of all marine species and 70% of terrestrial vertebrates died out. The two events were close on the geological timeline, and this paper, using recent dating techniques, confirmed that the super-volcano erupted shortly before the extinction, making it the likely culprit. Synthetic biology is the design and construction of biological devices and systems to accomplish the specific goal of the synthetic biologist,  406  adding human intentionality to traditional pandemic risks. The positive and negative potentials of synthetic biology are unclear  407  -much of the information currently comes from synthetic biologists,  408  who may not be able to provide an impartial overview (the problem is exacerbated by the decentralised nature of the field  409  ). Attempts at regulation  410  or self-regulation  411  are currently in their infancy, and may not develop as fast as research does.  412  One of the most damaging impacts from synthetic biology would come from an engineered pathogen,  413  targeting humans or a crucial component of the ecosystem (such as rice, which accounts for 20% of all calories consumed by humans).  414  This could emerge through military bio-warfare,  415  commercial bio-warfare,  416  bio-terrorism  417  (possibly using dual-use products  418  developed by legitimate researchers, and currently unprotected by international legal regimes  419  ), or dangerous pathogens leaked from a lab  420  . Of relevance is whether synthetic biology products become integrated into the global economy or biosphere. This could lead to additional vulnerabilities (a benign but widespread synthetic biology product could be specifically targeted as an entry point through which to cause damage). But such a development would lead to greater industry and academic research, which could allow the creation of reactive or pre-emptive cures.  421  The impact is very similar to that of pandemics: mass casualties and subsequent economic and political instabilities leading to possible civilisation collapse. A bio-war would contribute greatly to the resulting instability. Even for the most perfectly engineered pathogen, survivors are likely, if only in isolated or mainly isolated locations.  422  Extinction risk is unlikely,  423  but possible if the aftermath of the epidemic fragments and diminishes human society to the extent that recovery becomes impossible  424  before humanity succumbs to other risks.  425  Five important factors in estimating the probabilities and impacts of the challenge: 1.   431  showing how the flu virus could be made transmissible to ferrets (and, by extension, humans). This generated protests and calls for the papers to remain fully or partially unpublished,  432  because of the potential for misuse by bio-terrorists or bio-weapons programmes. In response, researchers in the field declared a voluntary moratorium in January 2012.  433  A year later, they decided to lift the moratorium. One cannot expect workers in a field to be unbiased about their own research,  434  so it is significant that this decision was condemned by many scientists, including other virologists.  435  This provides strong evidence that ending the moratorium was a dangerous decision.   450  a declining fraction, so from a narrow economic perspective it could be argued that the impact of nanotechnology would be relatively small. However, nanotechnology could create new products -such as smart or extremely resilient materials  451  -and would allow many different groups or even individuals to manufacture a wide range of things.
 28-Feb This could lead to the easy construction of large arsenals of weapons by small groups.  452  These might be masses of conventional weapons (such as drones or cruise missiles), or more novel weapons made possible by atomically precise manufacturing. If this is combined with a possible collapse in world trade networks  453  -since manufacturing could now be entirely local -there would be a likely increase in the number of conflicts throughout the world. Of particular relevance is whether nanotechnology allows rapid uranium extraction and isotope separation  454  and the construction of nuclear bombs, which would increase the severity of the consequent conflicts. Unlike the strategic stalemate of nuclear weapons, nanotechnology arms races could involve constantly evolving arsenals and become very unstable.  455  These conflicts could lead to mass casualties and potentially to civilisation collapse if the world's political and social systems were too damaged. Some nanotechnology pathways could mitigate these developments, however. Cheap mass surveillance,  456  for instance, could catch such re-armament efforts (though surveillance could have its own detrimental effects). Many of the world's current problems may be solvable with the manufacturing possibilities that nanotechnology would make possible, such as depletion of natural resources, pollution, climate change, clean water, and even poverty.  457  There are currently few applicable international legal regimes governing nanotechnology.  458  In the media the label "grey goo"  459  is sometimes applied to nanotechnology. This is meant to describe a hypothetical situation where special self-replicating nanomachines would be engineered to consume the entire environment. It is unclear how effective they could be, and they play no role in atomically precise manufacturing.  460  Mass selfreplication would be detectable, and vulnerable to human-directed countermeasures.  461  However, it is possible that such replicating machines could endure and thrive in particular ecological niches, where the cost of removing them is too high.  462  The misuse of medical nanotechnology  463  is another risk scenario. Extinction risk is only likely as a long-term consequence of civilisation collapse, if the survivors are unable to rebuild and succumb to other threats.  464  The possibility of nanomachines or nanoweapons remaining active after a civilisation collapse may make the rebuilding more difficult, however, while the availability of atomically precise manufacturing systems, by contrast, could aid rebuilding. Five important factors in estimating the probabilities and impacts of the challenge: 1. A functional and practical design for assembling molecules is an essential feature for successful nanotechnology. There have been many designs proposed,  467  and some constructed, but not yet a fully functional molecular assembly device.  468  This design, based on principles from biology (it uses messenger RNA as its input code, and synthesises peptides) represents another step towards that important goal. 06-May-13: First weapon made with 3D printer  469  -Event It is the ability to make weapons en masse that represents one of the dangers of nanotechnology.  470  3D printing (or additive manufacturing)  471  is not nanotechnology, but can be considered a precursor, as it similarly allows small groups to design and manufacture their desired products themselves. That one of the early designs has been a functioning weapon, and that such weapon design was justified on moral grounds,  472  indicates a very high probability that nanotechnology will be used for weapon production. 07-May-13: Publication of Eric Drexler's book "Radical Abundance: How a Revolution in Nanotechnology Will Change Civilization" 473 -Research Eric Drexler is one of the pioneers of nanotechnology, and introduced the concepts to the general public with his book "Engines of Creation".  474  Twenty seven years later, he presents a history, progress report, and updated version of his vision, the central theme of which is to "imagine a world where the gadgets and goods that run our society are produced not in far-flung supply chains of industrial facilities, but in compact, even desktop-scale, machines." The revolution in manufacturing would produce the "radical abundance" of the title, with small groups and individuals capable of producing an extraordinarily wide range of products without requiring large amounts of capital or long supply chains. The risks of social and political disruption are then examined. The disruptions that can be anticipated include "falling demand for conventional labor, resources, and capital in physical production, with the potential for cascading disruptive effects throughout the global economy", as well as disruptions in supply chains, trade, dependence, and the revaluation of assets (mineral resources and large industrial facilities, for example, will lose much of their value). This would go together with an increase in surveillance capability and a potential nanotechnology arms race. The book recommends taking pre-emptive action at the international level to prepare for these disruptions. A key sign of a developing technology is interest from investment companies. Nanostart AG is an example of such a company, with extensive investments in various nanotechnology projects. Interestingly, their interests are not limited to more conventional nanotech projects, but extend to such speculative endeavours as space elevators.  476  This serves as a reminder of the potentially large profits available in nanotechnology. Thus it seems likely that when the technology matures sufficiently to cause increased risks, there will be many commercial entities heavily investing in the technology, which will make the process of regulation more contentious, possibly leading to "regulatory capture" 477 by these entities, with their interests represented rather than those of the broader community. 16-Dec-13: Nanotechnology: A Policy Primer, CRS report of Congress 478 -Policy Governmental and supragovernmental policies will be key to dealing with the dangers and destabilising influences of nanotechnology, through regulation, treaties, redistributive efforts or simply through preparing their populations for the change. And institutions such as the US Congress are keeping an eye on nanotechnology, in this case through the Congressional Research Service. This report, however, does not delve into the major risks of nanotechnology, but restricts itself to minor subjects such as the safety of nanomaterials and US competitiveness in that field. War, trade disruption and potential development and misuse of nanoreplicators  479  are not discussed. This seems to reflect a certain lack of prioritisation and perhaps even a misplaced focus on the less important risks. Major AI researchers and textbooks define the field as "the study and design of intelligent agents", where an intelligent agent is a system that perceives its environment and takes actions that maximise its chances of success.  480  Artificial Intelligence (AI) is one of the least understood global challenges. There is considerable uncertainty on what timescales an AI could be built, if at all, with expert opinion shown to be very unreliable in this domain.  481  This uncertainty is bi-directional: AIs could be developed much sooner or much later than expected. Despite the uncertainty of when and how AI could be developed, there are reasons to suspect that an AI with human-comparable skills would be a major risk factor. AIs would immediately benefit from improvements to computer speed and any computer research. They could be trained in specific professions and copied at will, thus replacing most human capital in the world, causing potentially great economic disruption. Through their advantages in speed and performance, and through their better integration with standard computer software, they could quickly become extremely intelligent in one or more domains (research, planning, social skills...). If they became skilled at computer research, the recursive selfimprovement could generate what is sometime called a "singularity",  482  but is perhaps better described as an "intelligence explosion",  483  with the AI's intelligence increasing very rapidly.  484  Such extreme intelligences could not easily be controlled (either by the groups creating them, or by some international regulatory regime),  485  and would probably act in a way to boost their own intelligence and acquire maximal resources for almost all initial AI motivations.  486  And if these motivations do not detail  487  the survival and value of humanity in exhaustive detail, the intelligence will be driven to construct a world without humans or without meaningful features of human existence. This makes extremely intelligent AIs a unique risk, 488 in that extinction is more likely than lesser impacts. An AI would only turn on humans if it foresaw a likely chance of winning; otherwise it would remain fully integrated into society. And if an AI had been able to successfully engineer a civilisation collapse, for instance, then it could certainly drive the remaining humans to extinction. On a more positive note, an intelligence of such power could easily combat most other risks in this report, making extremely intelligent AI into a tool of great positive potential as well.  489  Whether such an intelligence is developed safely depends on how much effort is invested in AI safety ("Friendly AI") 490 as opposed to simply building an AI.  491  If the returns from increased intelligence are low, intelligence explosions and extreme intelligence may not be possible. In that case, there would probably be an ecology of AIs of different levels of intelligence, performing different tasks. In this scenario, apart from the economic dislocation already noted, there is also the possibility of AI-enabled warfare and all the risks of the technologies that AIs would make possible. An interesting version of this scenario is the possible creation of "whole brain emulations," human brains scanned and physically instantiated -physically represented -in a machine. This would make the AIs into what could be called properly human minds, possibly alleviating a lot of problems. Five important factors in estimating the probabilities and impacts of the challenge: 1. The reliability of AI predictions. 2. Whether there will be a single dominant AI or a plethora of entities. 3. How intelligent AIs will become. 4. Whether extremely intelligent AIs can be controlled, and how.
 Whether whole brain emulations (human minds in computer form) will arrive before true AIs. 11. Human redundancy may follow the creation of copyable human capital, as software replaces human jobs. 12. Once invented, AIs will be integrated into the world's economic and social system, barring massive resistance. 13. An AI arms race could result in AIs being constructed with pernicious goals or lack of safety precautions. 14. Uploads -human brains instantiated in software -are one route to AIs. These AIs would have safer goals, lower likelihood of extreme intelligence, and would be more likely to be able to suffer.  495  15. Disparate AIs may amalgamate by sharing their code or negotiating to share a common goal to pursue their objectives more effectively. 16. There may be diminishing returns to intelligence, limiting the power of any one AI, and leading to the existence of many different AIs.  496  17. Partial "friendliness" may be sufficient to control AIs in certain circumstances. 18. Containing an AI attack may be possible, if the AIs are of reduced intelligence or are forced to attack before being ready. 19. New political systems may emerge in the wake of AI creation, or after an AI attack, and will profoundly influence the shape of future society. The amount of information stored in a human brain is extremely large. Similarly, the amount of information needed to perform adequately at complex human tasks is considerable -far more than is easily programmable by hand (the Cyc project, 498 for instance, started in 1984, aiming to rapidly formally codify all human common senseand is still running). Thus the interest in the field of machine learning, and in algorithms that can teach themselves skills and knowledge from raw data. With the rise of "Big Data", 499 vast databases and increased computer power, there has been a flowering of applications of computer learning.  500  This has caught the eye of the Defense Advanced Research Projects Agency (DARPA), a research arm of the US defense department responsible for the development of new technologies. In this project, DARPA aims both to "enable new applications that are impossible to conceive of using today's technology" and to simplify machines so that non-experts can effectively use them and build applications for them. This most recent project confirms the interest of the military in artificial intelligence development. 25-Apr-13: Kurzweil plans to help Google make an AI brain  501  -Initiative The idea of creating a fully general AI, an AI that is capable of all tasks requiring intelligence, went into abeyance during the AI winter,  502  a period of reduced interest and funding in AI. The term AI itself fell into disfavour.  503  But recent AI successes such as Watson's triumph on "Jeopardy!"  504  (demonstrating a certain level of natural language recognition and processing) and Google's selfdriving car  505  (demonstrating spatial awareness and movement) have revived interest in constructing a human-like mind in digital form. Kurzweil, hired by Google at the end of 2012, reveals in this interview his interest in doing just that. A notable feature of Kurzweil is his optimism about the consequences of creating AIs,  506  which could affect the level of precautions his team would include in its design. 13-Sep-13: Publication: "Responses to Catastrophic AGI Risk: A Survey" 507 -Research Since the recognition of the potential risk with AGI (Artificial General Intelligence),  508  various proposals have been put forward to deal with the problem. After arguing that uncertainty about a timeline to AI  509  does not translate into a certainty that AIs will take a long time, the paper analyses why AIs could be an existential risk. It argues that a trend toward automatisation would give AIs increased influence in society, as such systems would be easier to control, and there could be a discontinuity in which they gained power rapidly.  510  This could pose a great risk to humanity if the AIs did not share human values (intelligence and values are argued to be independent for an AI),  511  a task which seems difficult to achieve if human values are complex and fragile, 512 and therefore problematic to specify. The authors then turned to analysing the AI safety proposals, dividing them into proposals for societal action, external constraints, and internal constraints. They found that many proposals seemed to suffer from serious problems, or to be of limited effectiveness. They concluded by reviewing the proposals they thought most worthy of further study, including AI confinement, Oracle AI, and motivational weaknesses. For the long term, they thought the most promising approaches were value learning (with human-like architecture as a less reliable but possibly easier alternative). Formal verification was valued, whenever it could be implemented.   520  The book then imagines the competition between humanity and a cunning, powerful rival, in the form of the AI -a rival, moreover, that may not be "evil" but simply harmful to humanity as a side effect of its goals, or simply through monopolising scarce resources. Along with many interviews of researchers working in the forefront of current AI development, the book further claims that without extraordinarily careful planning,  521  powerful "thinking" machines present potentially catastrophic consequences for the human race. 15-Oct-13: "Racing to the precipice: a model of artificial intelligence development" lays out the dangers of AI arms races  522  -Research AIs may be developed by different groups, each desiring to be the first to produce an artificial mind. The competitive pressure will be stronger the more powerful AIs are believed to be, thus maximising the danger in those situations. This paper considers an AI arms race,  523  where different teams have the option of reducing their safety precautions in order to perfect their device first -but running the risk of creating a dangerous and uncontrollable AI. In the absence of enforceable agreements between the teams, this dynamic pushes each to take on more risk than they would want (similarly to the "prisoner's dilemma"),  524  potentially causing an extremely damaging outcome. The situation is improved if risktaking makes little difference to speed of development, if the teams have reduced enmity between them, or if there are fewer teams involved (those last two factors also help with reaching agreements). Somewhat surprisingly, information has a negative impact: the outcome is safer if the teams are ignorant of each other's rate of progress, and even of their own. 24-Oct-13: Growing researcher awareness of the threat of artificial intelligence  525  -Research Much more effort is devoted to creating AI than to ensuring that it is developed safely.  526  Those working in developing AI could be motivated to minimise the extent their creation represented a potential danger.  527
 Probability There are many different possible risks that seem individually very unlikely and speculative. Could someone develop a super-pollutant that renders the human race sterile? Could the LHC have created a black hole that swallowed the Earth?  532  Might computer games become so addictive that large populations will die rather than ceasing to indulge in them?  533  Could experiments on animals lift them to a level of intelligence comparable with humans?  534  Might some of the people sending signals to extra-terrestrial intelligences attract deadly alien attention?  535  What are the risks out there that we can't yet conceive of? These risks sound unlikely and for many possibly ridiculous. But many of today's risks would have sounded ridiculous to people from the past. If this trend is extrapolated, there will be risks in the future that sound ridiculous today, which means that absurdity is not a useful guide to risk intensity. Expert opinion provides some information on specific speculative risks. But it will tend to give them extremely low probabilities -after all, the risks are highly speculative, which also means the expert's judgement is less reliable.  536  But in these situations, the main source of probability of the risk is not the quoted number, but the much greater probability that the experts' models and world views are wrong.  537  If marginal scientific theories predict large risks, the probability is concentrated in the likelihood that the theory might be correct.  538  Conversely, if many independent models, theories, and arguments all point in the direction of safety, then the conclusion is more reliable. There are methods to estimate uncertain risks without needing to be explicit about them. One resolution to the Fermi paradox -the apparent absence of alien life in the galaxy -is that intelligent life destroys itself before beginning to expand into the galaxy. Results that increase  539  or decrease the probability of this explanation modify the generic probability of intelligent life (self-)destruction, which includes uncertain risks. Anthropic reasoning  540  can also bound the total risk of human extinction, and hence estimate the unknown component. Non-risk-specific resilience and post-disaster rebuilding efforts  541  will also reduce the damage from uncertain risks, as would appropriate national and international regulatory regimes.  542  Most of these methods would also help with the more conventional, known risks, and badly need more investment. Five important factors in estimating the probabilities and impacts of the challenge: 1. Whether there will be extensive research into unknown risks and their probabilities. 2. 3. Global coordination would aid risk assessment and mitigation. 4. Specific research into uncertain and unknown risks would increase our understanding of the risks involved. 5. General mitigation efforts are mostly general resilience building. 6. Some institutions may deliberately pursue dangerous technologies or experiments, or may convince themselves that their research is not dangerous. 7. Unforeseen accidents could be the trigger for many uncertain risks. 8. The amount of direct casualties varies wildly depending on the risk involved. 9. The disruptions to the world's economic and political system vary wildly depending on the risk involved. 10. The uncertain risk may have other disruptive effects (such as loss of trust in certain technologies). 11. The long-term impact varies wildly depending on the risk involved. 12. The world's political structure, after an unknown risk is triggered, will determine whether humanity improves or worsens the situation. 13. Some methods (such as considering the Fermi paradox) may bound the total probability of destructive uncertain risks, but these are very speculative. 14. One approach to dealing with uncertain risks is to build general adaptation and recovery methods that would be relevant to a wide class of potential disasters. This paper notes the absence of published research in this area,  544  and seeks to begin to fill the gap. It identifies methods for increasing survivor resilience and promoting successful adaptation and recovery, even for isolated communities. It recognises that the process is highly complex, and needs further research. 28-Mar-13: Paper Evaluating Methods for Estimating Existential Risks  545  -Research It would be advantageous to have a rigorous approach for estimating severe risks, including uncertain and unknown ones. This paper reviews and assesses various methods for estimating existential risks, such as simple elicitation; whole evidence Bayesian; evidential reasoning using imprecise probabilities; Bayesian networks; influence modelling based on environmental scans; simple elicitation using extinction scenarios as anchors; and computationally intensive possible-worlds modelling.  546  These methods can be applied rigorously to uncertain risks, assessing them in the same way as more standard risks. Influence modelling based on environmental scans  547  can even suggest some new as yet unknown risks. 01-Aug-13: The Fermi paradox provides an estimate of total existential risk (including uncertain risks)  548  -Research The Fermi paradox is the seeming contradiction between the apparent ease with which intelligent life could arise in the galaxy, and the lack of evidence of any such life. Many explanations have been proposed to resolve the paradox,  549  one of which is relevant to existential risks: the "Late Great Filter" explanation.  550  This posits that intelligent life is inevitably destroyed before it can expand through the galaxy. Such an explanation gives a bound to existential risk from all sources, including uncertain risks. This paper demonstrates the relative ease with which a spacefaring civilisation could cross between galaxies. Combined with recent evidence that the majority of Earth-like planets formed before the Earth,  551  this makes the absence of visible intelligent life more inexplicable, and worsens the Fermi paradox, increasing the probability of a Late Great Filter and thus of existential risk from all sources. As there is no global government, global governance typically involves a range of actors including states, as well as regional and international organisations. However, a single organisation may nominally be given the lead role on an issue."
 Probability Often global governance is confused with global government, but they are two very different things. Global governance is just a term to describe the way global affairs are managed, or not managed. Global government is the idea that the world should be run like a country with a government. The global governance system will inevitably have pros and cons, depending on the political decisions that are made. This section looks at global governance disasters. Though all the risks in this report can be exacerbated by poorly chosen policy decisions, this classification contains those problems that arise almost exclusively from bad policy choices. There are two main divisions in governance disasters: failing to solve major solvable problems, and actively causing worse outcomes. An example of the first would be failing to alleviate absolute poverty.  556  An example of the second would be constructing a global totalitarian state.  557  In general, technology, political and social change may enable the construction of new forms of governance, which may be either much better or much worse. These examples immediately illustrate two issues with governance disasters. First, the task of estimating their probability is difficult. Longterm political predictions are of questionable validity and subject to strong biases,  558  especially where strongly-held values are concerned.  559  Second, the impact of these governance disasters depends to a large extent on subjective comparative evaluations. It is not impartially obvious how to rank continued poverty and global totalitarianism versus billions of casualties or civilisation collapse.  560  The long term impact needs also to be considered: how will poverty and global governance change? If there are many generations ahead of us, then the long term state of humanity's policy 561 becomes much more important than the short term one. Five important factors in estimating the probabilities of various impacts: 1. How the severity of non-deadly policy failures can be compared with potential casualties. 2. Whether poor governance will result in a collapse of the world system. 3. How mass surveillance and other technological innovations will affect governance. 4. Whether there will be new systems of governance in the future. 5. Whether a world dictatorship may end up being constructed. The revelations caused great controversy  567  and raised questions about the NSA's surveillance oversight.  568  The episode established that discrete mass surveillance -an important component of potential To reduce poverty in the future, it is important to maintain and extend past trends in poverty mitigation. The United Nations' Poverty-Environment Initiative (PEI), launched in 2008, has had a number of success stories from Uruguay 570 to Malawi.  571  Due to increased demand from member states, the programme has been extended for another five years, 2013-2017, and may add countries such as Myanmar, Mongolia, Indonesia, Albania, Peru and Paraguay. Such programmes demonstrate that the bureaucratic/policy side of poverty reduction is supported by an international infrastructure with a strong emphasis on assessments. The effect of such approaches on overall poverty will depend on the interplay between these policies and the other side of poverty reduction: economic growth  572     576  Conversely, a resilient governance system is better able to cope with all risks, and a collapsed global system is more vulnerable to all risks. Nuclear war,  577  asteroid impacts  578  and super-volcanoes  579  have direct impacts on the climate, and, through that, on the ecosystem.  580  The kinds of mitigation efforts capable of containing the damage from a super-volcano would most likely be effective against asteroid impact damage, because of the similar nature of the impacts. The converse is not true, since one major method of reducing asteroid impact -spacebased deflection  581  -would have no impact on super-volcano risk. Solving climate change would help reduce current ecological pressure.  582  International agreements to reduce ecological damage could be extended to combating climate change as well, by establishing structures for international collaboration and encouraging resource-efficient solutions. Climate change also creates conditions more suitable for the spread of pandemics.  583  Measures to combat global pandemics, such as strengthened outbreak coordination and statistical modelling,  584  could be used to combat synthetic pathogens as well. If a safe artificial intelligence is developed, this provides a great resource for improving outcomes and mitigating all types of risk.  585  Artificial intelligence risks worsening nanotechnology risks, by allowing nanomachines and weapons to be designed with intelligence and without centralised control, overcoming the main potential weaknesses of these machines  586  by putting planning abilities on the other side. 139 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 4. Relations between global risk / 4.1 General relations between global risks and their potential impacts Conversely, nanotechnology abilities worsen artificial intelligence risks, by giving AI extra tools which it could use for developing its power base.  587  Nanotechnology and synthetic biology could allow the efficient creation of vaccines and other tools to combat global pandemics.  588  Nanotechnology's increased industrial capacity could allow the creation of large amounts of efficient solar panels to combat climate change, or even potentially the efficient scrubbing of CO2 from the atmosphere.  589  Nanotechnology and synthetic biology are sufficiently closely related  590  (both dealing with properties on an atomic scale) for methods developed in one to be ported over to the other, potentially worsening the other risk. They are sufficiently distinct though (a mainly technological versus a mainly biological approach) for countermeasures in one domain not necessarily to be of help in the other. Uncontrolled or malicious synthetic pathogens could wreak great damage on the ecosystem; conversely, controlled and benevolent synthetic creations could act to improve and heal current ecological damage. There are many secondary effects that are not covered here. Increasing nuclear power could for instance improve the outlook for climate change,  591  while increasing the risk of proliferation  592  and thus of nuclear war. There are many such effects between various strategies for addressing different risks, but they are specific enough for there to be no simple arguments of the type which says that mitigating risk X worsens risk Y.  During the process of identifying risks that could have an infinite impact it became evident that the most common question among people interested in global challenges is this: "How probable is it that this impact will ever happen?" For those with expert knowledge in one area the first question is often: "How does the probability and magnitude of impact in this area compare with the probability and magnitude of impact in other areas?" Finally, those who have tried to estimate probabilities for global challenges ask: "What is the status of knowledge in other areas compared to mine?" These are all very important questions, and this chapter is not an attempt to answer them. But, as there is no organisation, process or report that has provided an overview of quantified assessment for global challenges with potential infinite impact, the chapter does try to present the current state of knowledge in order to inspire further work.
 ALL RISKS It is easy to argue that it is too difficult, or even impossible, to assess the probabilities that are at all meaningful for the risks in this report, and therefore to exclude them. There are many good reasons for not trying, including significant uncertainty in almost all steps of the assessment. Not only do great uncertainties exist for all the risks, but the difficulties of estimating probabilities are also very different. At one end of the spectrum the probability of a nuclear war can change dramatically from one day to another due to political decisions. Much of the uncertainty is related to psychological assumptions of how different individuals will react under stress. At the other end of the spectrum there is AI, where there is not even a generally accepted understanding of the possibility of the impacts capable of creating the risks covered in this report. There are challenges with very much data, including asteroids, and other challenges with very little relevant data, such as bad future global governance. Obviously the risks also share a number of characteristics: they all have potentially extreme outcomes and have never been experienced before. The possibility of studying series of data, exploring how the outcome will change with incremental changes in input data, and testing conclusions on similar events are just a few examples of things that in most cases cannot be done. Estimating probabilities in traditional ways is therefore very difficult.  594  However, as the current lack of interest in global risks with potentially infinite impacts may in part be due to the lack of actual numbers, the best estimates that could be found are presented below with explanations. These estimates are only an attempt to assemble existing estimates in order to encourage a process to improve these numbers. These estimates range from rigorous calculations based on large amounts of high-quality data (asteroids) to guesstimates by interested experts (AI). The result is that some have a more rigorous methodology behind them, and others should be taken with a large grain of salt, but all are still very rough estimates. As science progresses they will be updated. It is even possible that some will change by orders of magnitude. But instead of no estimate at all, we now have an initial reference that we hope will trigger a discussion and collaboration that will help improve what we have already. As many of the challenges are longterm and require early action to be avoided or mitigated, the probability is provided for the next 100 years, instead of the annual probability that is often provided. The reason for this is that a 100-year perspective helps us understand that even relatively small probabilities can become significant over a century. Say that it is a one in 100 probability (1%) for an impact to occur. Over a century there is a 63.4% probability of one or more such impacts.  595  Further, structures that need to change require us to look beyond the immediate and incremental changes that most discussions focus on today.
 143 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 5. Probabilities and uncertainties -an initial overview
 Structure of the probability estimates As the different challenges are very different and the status of probability estimates varies significantly, the initial probability numbers are provided together with estimates regarding:
 The understanding of sequence This is an estimation of how well the sequence from today to a possible infinite impact is understood. At one extreme all the different paths from today to an infinite impact are understood. At the other extreme, there is only a theoretical idea that is coherent and does not break any natural laws. In the latter case there would be no understanding of how it is possible to get from where we are today to an infinite impact. A sequence is required to calculate an estimate instead of only having educated guesses.
 Data availability This is an estimate of the amount of data available to make probability assessments on all relevant steps of the sequence. In some areas a lot of hard-to-get data is needed to make an assessment (e.g a global pandemic); in other areas the data is related to secret and/ or psychological factors (e.g. large-scale nuclear war). In others relatively little data is needed (asteroids), or a lot has been done to gather data (e.g. climate change).
 Existing probability estimates These form an estimate of the kind of uncertainty that exists. This obviously depends on an understanding of sequence and data availability, but it also depends on resources and interest in communicating with the rest of the world. The estimates below are preliminary, but a sound risk approach requires stakeholders to begin to include them in strategic assessments. One group in particular is of interest and that is actuaries, the professionals who deal with the financial impact of risk and uncertainty. One of the key guiding rules they follow is to ensure a capital adequacy at a 1-in-200 level. This rule, which is included in for example ICA  596  and Solvency II, 597 provides an opportunity to discuss risks with a possible infinite impact. One contribution could be to discuss the pros and cons with different definitions of the 1-in-200 level. For example, one definition is that "each company holds enough capital to withstand the events of the next one year with a probability of 199 out of 200." 598 This would exclude many of the risks in this report and could even result in the risks increasing, as the time perspective is so short. Investments could help reduce shortterm risks at the same time as they increase long-term risks. Another definition is that "a company should hold enough capital to be able to withstand a 'reasonably foreseeable' adverse event".  599
 Nuclear War Nuclear war is the risk that started the work with scientific assessments related to infinite impact.  604  The understanding of the sequence is relatively well known. Still, the fact is that the impact will depend significantly on how serious the nuclear winter will be as the result of a war (if there is any nuclear winter at all). The probability of a nuclear winter will depend on when during the year the war happens, and what the weather is during this time. The result is that the probability of an infinite impact has an inherent uncertainty and can be estimated only once a war has already started. With many of the spillover effects occurring in remote areas, even basic data is still very rudimentary. Scientists who collect data relevant for pandemics are often working with very small resources, and there is no systematic way of collecting data on a global scale, although interesting initiatives are under way.  607  While an early warning system would be comparatively inexpensive, there are still no resources available. Most of the probability estimates made for pandemics are for their more benign versions. For the possible pandemic that could kill two billion or more there are very few estimates. Based on available assessments 608 the best current estimate of a global pandemic in the next 100 years is: 5% for infinite threshold, 0.0001% for infinite impact The reason for the big difference between threshold and impact is mainly that a pandemic will not directly affect infrastructure or the rest of the ecosystem in the way that extreme climate change or nuclear war would. This means that resilience will be relatively better after the infinite threshold is crossed. This is one of the more complex risks as it can be seen more as a heading than a description of a specific challenge with a well-defined sequence. In other words it is not one sequence, but very many still unknown sequences. The concept of ecological collapse usually refers to a situation where some part of the ecological web becomes so weak that it collapses.
 00001% There are many studies about the stability and possible collapse of different ecosystems, but there are few that look into the possibility for a full ecological collapse that would result in at least two billion people suffering. Data availability is good in many areas, but the challenge is that without an understanding of the system dynamics, and because of its complexity, there are inherent limits to how exact the knowledge is that can be achieved.  610  Regarding probability estimates, it is only ecological collapse and global system collapse of the current manmade global challenges that have no estimates for infinite impact. Based on available assessments 611 the best current estimate of an ecological catastrophe in the next 100 years is:  0.5%
 Collapse Global System Since the financial crisis the possibility of a global collapse of the current political, economic and financial system has been discussed intensively. A rapidly evolving and increasingly interconnected system, it is subject to unexpected, system-wide failures because of the structure of the network -it faces a systemic risk. Possible sequences for a global system collapse resulting in infinite impacts are very hard to establish, for three reasons. First, it is a very complicated system, with many dynamic interactions, as there are many people who, together with machines, react to each other. The current global system shows a lot of complex dynamic phenomena, such as business cycles, financial crises, irregular growth, and bullwhip effects.  612  Many nonlinear dynamic models of economics and finance present various complex dynamic behaviours such as chaos, fractals, and bifurcation. Second, it is a recent system that has been so interconnected for only a few years, as it depends on an infrastructure that did not exist before the internet, so there is little experience of how it works. Third, the system is rapidly changing and becoming even more complex as more connections are added and its speed increases. Better understanding of complex systems with multiple attractors and bifurcation behaviour will help improve the possibility of understanding the possible sequences.  613  An additional challenge for the understanding of sequences that can result in impacts beyond the infinite threshold is that almost all research being done in the area of global system collapse focuses on its economic or geopolitical implications, not on a full system collapse and not on human suffering.
 Super-volcano The super-volcano risk has many similarities with a major asteroid risk. Both have happened a number of times through our planet's history, and both have had major consequences. The understanding of the sequence is however a lot lower than for asteroids, as the mechanisms behind volcano eruptions are not very well known. The possibility of foreseeing when a supervolcano will erupt and how big the impact will be is therefore low. Compared with a major asteroid, there will therefore be much less time to prepare. There is data available for different impacts, and knowledge of where super-volcanoes might erupt is increasing, but due to the lack of understanding when it comes to the sequence, the probability estimations are still very rudimentary. A number of estimates exist where the probability is assessed, but they are quite rudimentary, based on the historic frequency of earlier supervolcano eruptions. As these are so infrequent, the uncertainty becomes very significant. The basic sequence is relatively well-known, given that it would be a more deadly version of a current virus, but there is also the possibility that a new virus (or other organism) may be found where the sequence will be unknown and therefore also much more dangerous. One of the challenges to understanding the sequence is that the spreading of synthetic biology will come either from a wilful act (e.g. terrorism) or an accident (e.g. unintentional release from a laboratory). This also makes data hard to get. There are some numbers for accidents in labs, but they are available in only a few countries and there are probably many more than those reported.  620  With terrorist acts there are probability estimates that can be used as a basis for the use of synthetic biology as well.  621  There are some existing estimates for synthetic biology, but these are based on possible use in war, where calculations depend on some specific differences from existing pathogens that are assumed to be necessary for a pandemic with an infinite impact. Based on available assessments  622  the best current estimate of an impact from synthetic biology in the next 100 years is: 1% for infinite threshold, 0.01% for infinite impact   626  For global challenges in rapidly evolving areas where incremental development might not happen and little is known about the sequence, the only way to reduce risks with possible infinite impacts might be to ensure focus on these general factors. The only estimates of probabilities that exist so far have been made by a small group with a significant proportion of people with a passion for AI. Compared with many other challenges the possibility of an AI capable of infinite impact can almost be described as all or nothing. This is also why the estimates are the same for the infinite threshold and the infinite impact. Based on available assessments 627 the best current estimate of an impact from AI in the next 100 years is: 0-10% for infinite threshold, 0-10% for infinite impact The reason for 0-10% on both impact levels is that most experts assume that the kind of AI capable of impacts beyond the infinite threshold is likely to be one that also can result in an infinite impact. If we succeed it will move beyond control very rapidly. Due to the significant impact it would have if it worked, there is no difference between the two impact levels. The most common mistake is that the most likely development of these underlying trends is taken for granted as the only possible outcome. Most of the trends have probability distributions where low-probability/ high-impact possibilities are often ignored. In this chapter some of the most important trends, where the possible outcomes can differ significantly, are described through a global risk perspective. For each of the trends the simple rule based on a risk perspective is: "Aim for the best, but be prepared for the worst". Global poverty has fallen dramatically over the last two centuries, and the fall has intensified in recent decades, raising hopes that poverty, defined by the World Bank as an income below US $1.25 per day, may be eliminated within the next 50 years. The Economist even had a cover, in June 2013, with the title "Towards the end of poverty".  631  The World Bank has set an interim target of reducing global extreme poverty to 9% of the world's population by 2020, which, if achieved, would mark the first time the rate has fallen to single digits.  632  The milestone is based on a World Bank economic analysis of global poverty trends aimed at the goal of ending extreme poverty by 2030. Reaching 9% in 2020 would mean an estimated 690m people would still be living in extreme poverty by then, 510m fewer in poverty than a decade earlier. That would be the equivalent of half the population of Africa, or more than double the population of Indonesia.  633  There are reasons to celebrate this development as more people than ever live a life where they do not have to constantly worry about their most basic needs. But there are two things worth remembering: 1. Poverty could increase again. 2. Defining poverty is difficult.   According to the 2012 Revision of the official United Nations population estimates and projections, the world population of 7.2 billion in mid-2013 is projected to increase by almost one billion people within twelve years, reaching 8.1 bn in 2025, and to further increase to 9.6 bn in 2050 and 10.9 bn by 2100.  642  These results are based on the medium-variant projection, which assumes a decline of fertility for countries where large families are still prevalent as well as a slight increase of fertility in several countries with fewer than two children per woman on average.  643  The medium projection is still dramatic as it assumes another four bn people on the planet, more than a 50% increase in population, equal to the Earth's entire population in 1975, in just 86 years.  644  The high-variant projection depicted in the figure below assumes an extra half a child per woman (on average) compared with the medium variant, implying a world population of 10.9 bn in 2050 and 16.6 bn in 2100.  645  That is equal to a 133% population increase in just 86 years . The difference between projections for 2100, from 10.9 bn people in the medium scenario, to 16.6 bn in the high scenario, equals the world population in 1995. There is also a credible low scenario with 6.8 bn by 2100.  646  A strategic approach must be based on all possible outcomes. Planning as though the world population will be only 6.8 bn is not optimistic: it is unscientific and dangerous. Even to plan for a world with 10.9 bn is not strategic as this would ignore the significant probability that the world's population would be much larger. There should be a plan for a world with 16.6 bn people, combined with a long-term strategy to ensure a sustainable population level. It is also important to ensure that more attention is paid to early warning systems that allow us to influence population development in a sustainable direction.  While weapons have become more deadly the death toll from wars has actually decreased over time.  650  How big a part technology has played by creating greater transparency, or increasing the fear of using weapons which have become too powerful (for example nuclear bombs), is disputed. But most experts agree that technology has played an important role.  651  This is not the same as saying that this development will continue. Estimating the future development of technology is very difficult. On the one hand there is evidence that technology will continue to accelerate at the pace it has achieved so far. Researchers at MIT and the Santa Fe Institute have found that some widely used formulas for predicting how rapidly technology will advancenotably, Moore's and Wright's Laws -offer superior approximations of the pace of technological progress.  652  Experts like Ray Kurzweil, who was recently hired by Google, is one of those who think that most people do not understand the implications of exponential growth in the area of technology and the results it generates in price, capacity and overall transformation of society.  653  On the other hand there are natural limits that could begin to constrain technological development in two ways. The technology itself may hit a barrier. For example, at some stage a processor may not continue to become smaller and faster, as the speed of light and quantum mechanics will limit its development.  654  There might be other ways to overcome such boundaries, but no exponential trend can last forever. Second, nature itself may set limits. We may choose to take more care of the planet, or limits to materials like rare earths may begin to slow technology.  655  But regardless of ultimate limits, many exponential trends are likely to continue over the coming decades and will present us with new opportunities as well as risks in the 21st century, as these trends converge in a society with 20th century institutions.  Although the proportion of people who live beyond 100 is still very small, their number is growing rapidly. In 2000 there were an estimated 180,000 centenarians globally. By 2050 they are projected to number 3.2 m, an increase of about eighteen times.  659  Within the more developed regions, Japan, in particular, will experience a remarkable increase in the number of centenarians over the next half century, from fewer than 13,000 in 2000 to almost 1 m in 2050. By then Japan is expected to have by far the world's largest number and proportion of centenarians, nearly 1% of its population.  660  The stagnating and ageing population in many OECD countries and China will put pressure on current systems, which were not designed to deal with a situation of ageing and often shrinking populations in many parts of the world, while the populations in other parts of the world are rapidly growing.
 8. be assumed to increase or decrease global risks. Such a system would allow more time to develop strategies to mitigate risks and turn global challenges into opportunities for innovation and collaboration. The warning system would require significant research into infinitey thresholds. Both traditional as well as more recent methodologies based on understanding of complex systems should be encouraged. "big data".  662  These opportunities include both new ways of collecting large amounts of high-quality data, and new ways to analyse them. Early warning systems should be built to ensure that data is collected and analysed in ways that can be useful for multiple global challenges. The warnings should not only include changes in the physical world, but also indicate when decisions, investments and legal changes can The rapid technological development has many benefits, but also challenges, as risks can rapidly become very serious and reach infinite thresholds. To develop early warning systems that can gather and process data transparently is therefore of the utmost importance. Technological progress, from smart phones and sensors to significant processing power and networks, allows for totally new ways of establishing early warning systems based on so-called Institutions and universities engaged in developing new methodologies to assess global risks have a particular responsibility for developing and refining risk assessments for global challenges. methodology development could be accelerated and improved, as the possibility of learning from different areas would increase. Such a process could also encourage increased investments in methodology development based on the latest innovations, such as systemsforecasting approaches. There is currently no global coordination when it comes to risk assessments of global challenges. Different experts use different methodologies, data and ways to present their results, making it very difficult to compare the risk assessments that exist. By establishing a process that coordinates and encourages risk assessments of global challenges, Four groups are of particular importance: experts in finance, experts in security policy, lawyers with knowledge of global risks and international law, and finally a group consisting of clusters of stakeholders with solutions that can reduce the risks. Leadership networks that include participants from all four groups are of particular interest. infinite impacts, and could work on a roadmap for a future global governance system that can address existing and new global challenges. The networks should be as transparent and inclusive as possible, especially as global collaboration is needed. The use of new collaboration tools and principles, such as wikiprocesses and creative commons, should be encouraged. Tables, graphs and key conclusions in reports related to global challenges should, when possible, include the whole probability distribution.  664  Current lack of data and of scientific studies regarding low-probability high-impact outcomes in many areas should not be used as an excuse to ignore the probability distribution. This is especially important when many of the global challenges have a very long and fat "tail". Governments, major companies, NGOs, researchers and other relevant stakeholders should address the whole probability distribution, including low-probability highimpact scenarios. This would ensure that serious risks are not disregarded or obscured. In particular, leadership with a focus on multiple global challenges and the relationship between them should be highlighted, as very little is being done in this area. Governments, companies, organisations and networks working on global challenges should increase their efforts to reward leadership when they find it. Major news outlets can also report when significant positive steps are taken to reduce global risks with potential infinite impacts. New visualisation tools could help make complex systems easier to understand and also help the communication of challenges and opportunities.  663  Visualisation tools are needed both for decision makers to highlight the consequences of different strategies and for citizens to increase their basic understanding of infinite impacts. The global challenges depend on a very complex ecosystem and social system. With a global economic and technological system, which both helps and creates risks that are increasingly interconnected and difficult to understand, there is a challenge to understand the challenges. The IPCC uses specific and defined language in its reports to describe different probabilities and thus ensure clarity, but taken out of context and without supporting definitions this language can be misleading. For example, the term "very unlikely" is used by the IPCC to describe a probability of between 0-10%, 665 but out of context its use could easily be understood as a normative judgement suggesting that we do not need to engage with the risk. The language of the IPCC can be compared with that used in the Swedish National Risk Assessment (SNRA).  666  The scale of impact is not defined for the IPCC, but for the Swedish Assessment it is: Very small: no deaths or serious injuries Small: One dead and/or 1-9 seriously injured Average: 2-9 dead and/or 10-49 seriously injured Large: 10-49 dead and/or 50-100 seriously injured Very large: >50 dead and/or >100 seriously injured The use of terms that can be interpreted as having normative values to explain probability is problematic and in future all bodies, including the IPCC, should explore the possibility of using only numbers in external communications, at least in the summary for policy makers, to help everyone understand the reality of the situation. Stakeholders should explore ways to use language that better communicates how serious extreme risks are in the case of climate change, and where possible compare this with other risk areas to help illustrate the situation. Often words like "unlikely", "negligible" and "insignificant" are used to describe a risk when the probability is considered low. What is low is however relative; a low probability in one area can be extremely high in another. If I attend one of ten lectures -10% -people might say there is a low probability that I will be there. But if someone says that a new aircraft crashes once in every ten flights, most people will say that is an extremely high probability and will be likely to assume it is an early prototype that is nowhere close to commercial success. A major problem is that probabilities that ought to be seen as very high for risks with potentially infinite impact are described in a way that makes them sound less urgent than they are -by the media, business, politicians and even by scientists. One example is how probabilities are described by the Intergovernmental Panel on Climate Change. The use of methodologies and approaches from security policy and the financial sector that focus on extreme events could be used to develop strategies for rapid action beyond the incremental approaches that dominate today. Stakeholders should include the most extreme impacts in all relevant work. If the probability of infinite impacts increases instead of decreasing because of new scientific findings or lack of action, strategies should be prepared to allow more decisive action. When the impact is infinite it is not enough only to reveal the whole probability distribution. It is important also to avoid confusing uncertain risk with low risk. Infinite impacts render many of the traditional models for risk management almost meaningless. Monetary calculations are often useless, and discounting is not always advisable.  8. 9. 10. In addition, and probably equally important, is the fact that a body set up to deal with such challenges could also ensure that the links between them could be better understood. A first step could be to establish a centre for global risks and opportunities,  669  focusing initially only on knowledge-gathering and development of proposals, and with no mandate to implement any solutions. There is currently no international or global body that is coordinating work on global risks with a potentially infinite impact. A recent example is IPCC WGII Summary for policy makers. In this report http://ipcc-wg2.gov/AR5/ IPCC acknowledges the need to include low-probability highimpact scenarios: "assessment of the widest possible range of potential impacts, including low-probability outcomes with large consequences, is central to understanding the benefits and trade-offs of alternative risk management actions." Yet nothing is included in the report about impacts above 4 degrees.
 8 As an infinite impact by definition can't have happened before, models are needed. 9 Making Sense of Uncertainty: Why uncertainty is part of science http://www.lse.ac.uk/CATS/Media/SAS012-MakingSenseofUncertainty.pdf http://www-ee.stanford.edu/~hellman/Breakthrough/ book/chapters/frankenhaeuser.html "The challenge now is to help people extend their attachments, their loyalties, and their engagement, to include people outside their own narrow circle, their own country, their own imminent future. This global reorientation is a prerequisite for changing the present fatal course of development." 13 http://www-ee.stanford.edu/~hellman/Breakthrough/ book/chapters/frankenhaeuser.html 14 Two of the most famous "optimists", who tend to look at only the parts of the probability distribution that support their opinion, are the Danish writer Bjorn Lomborg and the British journalist Matt Ridley. While scientists in the areas he writes about constantly refute Lomborg, his message of optimism is well received by many policy makers and business leaders. https://www. ma.utexas.edu/users/davis/375/reading/sciam.pdf Ridley cherrypicks data and tends to avoid the probability that we will see significant warming. http://www.mattridley.co.uk/blog/the-probable-netbenefits-of-climate-change-till-2080.aspxhttp://www. rationaloptimist.com/   18  While the LA-602 document is relatively well-known, its final paragraph is often forgotten. The scientists say: "However, the complexity of the argument and the absence of satisfactory experimental foundations makes further work on the subject highly desirable." 19 Since Teller's calculations the science has developed and is now clear that the main nuclear threat that potentially could threaten human civilisation is a full nuclar war and the consequenses that this could result in from effects like a nuclear winter.  20  For an overview of positive and negative aspects of nanotechnology see for example Drexler, Eric and Pamlin, Dennis (2013): "Nano-solutions for the 21st century. Unleashing the fourth technological revolution" http://www.oxfordmartin.ox.ac.uk/downloads/ academic/201310Nano_Solutions.pdf 21 This is true for utopias ranging from Plato's Republic, via Thomas More's book Utopia, to Edward Bellamy's Looking Backward and William Morris' News from Nowhere.  22  See next chapter for the methodology.  23  A list of organisations and studies that discuss challenges that threaten human civilisation can be found here: http://en.wikipedia.org/wiki/Global_catastrophic_ risks  24  The number two billion was established during the workshop in Oxford and is not an exact number. Further research is needed to establish a better understanding of thresholds that can result in an infinite impact, depending on what challenge resulted in the two billion impact and how the estimate for an infinite impact was assumed to be between 0.01% and 10%.  25  The definition is based on the definition used by Jared Diamond: http://www.jareddiamond.org/Jared_ Diamond/Collapse.html  26    Analyzing Human Extinction Scenarios and Related Hazards" http://www.nickbostrom.com/existential/risks. html. Note that these four points were originally developed for "existential risks", those that threaten the extinction of intelligent life originating on Earth or the permanent destruction of its potential for desirable future development.  47  Timothy M. Lenton, Juan-Carlos Ciscar: Integrating tipping points into climate impact assessments. Springer Netherlands 2013-04-01 http://link.springer.com/ article/10.1007%2Fs10584-<PHONE>-8  48  Carl Sagan use 10 million years in: Sagan, Carl  (1983) . "Nuclear war and climatic catastrophe: Some policy implications". Foreign Affairs 62: 275, and The Blackwell's Concise Encyclopedia of Ecology and other sources provide an estimate of about 1 million years for mammals. http://eu.wiley.com/WileyCDA/WileyTitle/ productCd-<PHONE>.html  49  One billion years is used by Bruce E. Tonn in the paper Obligations to future generations and acceptable risks of human extinction: " Futures 41.7 (2009): 427-435.  50  In Japan in Japan the life expectancy at age zero, that is, at birth (LEB), is 83 years. In 2010 the world LEB was 67.2.  51  This is based on a low estimate for the planet's population as far out as current projections are made, 2100. http://esa.un.org/wpp/ Looking further into the far future, beyond 500 years, it is likely that humanity will have expanded into space and have a much larger population.  52     (2002) . A New Kind of Science. https://www.wolframscience.com/ 57 A possible infinite threshold could be compared with the two events that are often cited as the world's worst wars and anthropogenic disasters: the second world war, with 40-71 million dead (1.7-3.1% of the world's population) and the Mongol Conquests, <PHONE>, with 30 million dead (7.5% of the global total). See: http:// en.wikipedia.org/wiki/List_of_wars_and_anthropogenic_ disasters_by_death_toll  58  The number two billion was established during workshops arranged during the process and is not an exact number. Further research is needed to establish a better understanding of thresholds that can result in an infinite impact. The number of people dead is only one factor and different global risks are likley to have very different thresholds. 153 "The so-called "Warsaw International Mechanism for Loss and Damage" will from next year commit developed nations to provide expertise and potentially aid to countries hit by climate-related impacts. [...] However, the vague wording fell short of the kind of detailed commitments on additional funding and the commitment to compensation that many developing nations had been seeking." (source: Business Green: "COP 19: Warsaw climate deal finalised as deadlock broken").  154  See for instance Bulkeley, Harriet, and Peter Newell.: "Governing climate change." Routledge (2010).  155  See the National Research Council: "Abrupt Impacts of Climate Change: Anticipating Surprises." Washington, DC: The National Academies Press (2013).  156  See NASA's "GISS Surface Temperature Analysis."  157  See the IPCC's Fourth Assessment Report.  158  See the IPCC's Fifth Assessment Report.  159  See the IUCN's Red List of Threatened Species.  160     162  See Archer, David.: "Methane hydrate stability and anthropogenic climate change." Biogeosciences Discussions 4.2 (2007): <PHONE>.  163  For example, as climate warms, the destabilization of the West Antarctic ice sheet could raise sea level rapidly, with serious consequences for coastal communities.  169 Currently estimated at around 17,000 (source: SIPRI yearbook 2013).  170  Though it has been argued that scientists, under pressure from governments and industry, have systematically underestimated the deleterious global impact of radiation. See Perrow, Charles.: "Nuclear denial: From Hiroshima to Fukushima." Bulletin of the Atomic Scientists 69.5 (2013): 56-67.  171  The seminal American paper was Turco  173 Though some have argued for a significant climate effect of the nuclear explosions themselves, see Fujii, Yoshiaki.: "The role of atmospheric nuclear explosions on the stagnation of global warming in the mid 20th century." Journal of Atmospheric and Solar-Terrestrial Physics 73.5 (2011): 643-652.  174  There is (fortunately) very little empirical evidence on the impact of nuclear bombs on cities. Hiroshima suffered a firestorm, while Nagasaki did not -and both cities and nuclear weapons are very different now from what they were in 1945.  175   488 Dealing with most risks comes under the category of decision theory: finding the right approaches to maximise the probability of the most preferred options. But an intelligent agent can react to decisions in a way the environment cannot, meaning that interactions with AIs are better modelled by the more complicated discipline of game theory.  489  See Yudkowsky, Eliezer.: Artificial intelligence as a positive and negative factor in global risk. Global catastrophic risks 1 (2008): 303.  490  See Muehlhauser, Luke, and Nick Bostrom.: Why we need friendly AI. Think: Philosophy for Everyone (2014).  491  The balance is currently very uneven, with only three small organisations -the Future of Humanity Institute, the Machine Intelligence Research Institute and the Cambridge Centre for Existential Risk -and focused mainly on the risks of AI, along with some private individuals.  492  See Sandberg, Anders, and Nick Bostrom.: Whole brain emulation: A roadmap. Future of Humanity Institute Technical Report #2008-3, Oxford University (2008).  493  A dangerous AI would be a very intelligent that one could acquire great power in the world, and would have goals that were not compatible with full human survival and flourishing. There are a variety of ways of avoiding this. "Friendly AI" projects focus on the AI's goals directly, and attempt to make these compatible with human survival. The Oracle AI approach attempts to "box" the AI (constrain its abilities to influence the world) in order to prevent it from acquiring power. The reduced impact AI is a novel approach which attempts to construct AIs goals that are not friendly per se, but that motivate the AI to have a very limited impact on the world, thus constraining the potential damage it could do.  494  There are many different designs for AIs, and disagreements about whether they could be considered moral agents or capable of suffering. At one end of the scale is the theoretical AIXI(tl), which is essentially nothing more than an equation with vast computing power; at the other lie whole brain emulations, copies of human minds implemented inside a computer. Suffering for the first design seems unlikely; suffering for the second seems very possible. Determining whether a specific AI design can suffer will be an important part of figuring out what to do with it; it seems unlikely that we'll have hard and fast rules in advance.  495  The idea behind uploads, also called whole brain emulations, is to take a human brain, copy the position of its neurones and connections to sufficient precision, and then run the emulation forwards according to the laws of physics and chemistry. It should then behave exactly as a human brain in the real world. See http:// www.fhi.ox.ac.uk/brain-emulation-roadmap-report.pdf for more details, which also goes into the requirements and tries to estimate the likelihood off success of such an approach (it depends on certain likely but uncertain assumptions about how the brain works at the small scale; ie that most things below a certain scale can be handled statistically rather than through detailed modelling).  496  There's two ways there could be diminishing returns to intelligence: first, it might not be possible for an AI to improve its own intelligence strongly. Maybe it could find some ways of better executing its algorithms, and a few other low-hanging fruits, but it's very possible that after doing this, it will find that the remaining intelligenceincrease problems are hard, and that it hasn't improved itself sufficiently to make them easy (you could analogise this to dampened resonance -initially the AI can improve its intelligence quite easily, and this gives it more intelligence with which to work, allowing it to improve its intelligence still further; but each time, the effect is less, and it soon plateaus). The other way there could be diminishing returns to intelligence is if intelligence doesn't translate into power effectively. This is more likely with things like social intelligence: it seems plausible that no entity could convince, say, 99% of Britain to vote for their party, no matter how socially intelligent it was. Scientific and technological intelligence could conceivably be limited by creativity (unlikely), constraints on the speed of making experiments, or just the fact that humans have solved the easy problems already. The Global Catastrophic Risk Institute (GCRI) published a GCR bibliography compiled in July 2011 by Seth Baum, available at http://gcrinstitute. org/bibliography. This contains 115 entries, emphasising publications surveying the breadth of the risks or discussing other topics of general interest to the study of GCR, with less emphasis on analysis of specific global challenges. It has been updated for this Global Challenges Foundation report and now contains 178 entries. The reason for focusing on general interest publications is because the literature on specific global challenges is far too voluminous to catalogue. It would include, for example, a significant portion of the literatures on climate change, energy, nuclear weapons, infectious diseases and biodiversity, all topics that receive extensive research attention. Thus the full bibliography compiled by GCRI is only a small portion of the total global challenges literature. Publications for the full bibliography were identified in several ways. The bibliography began with publications already known to fit the selection criteria. Additional publications were identified by examining the reference lists of the initial publications. More publications were identified by searching scholarly databases (mainly Web of Science and Google Scholar) and databases of popular literature (mainly Amazon and the New York Public Library) for relevant keywords and for citations of the publications already identified. Several keywords and phrases were searched for in the databases: "existential catastrophe"; "existential risk"; "global catastrophe"; "global catastrophic risk"; "greatest global challenges"; "human extinction"; "xrisk"; and "infinite risk". The results of these searches were then screened for relevant publications. Many of the results were not relevant, because these terms are used in other ways. For example, "existential risk" is sometimes used to refer to risks to the existence of businesses, countries or other entities; "human extinction" is used in the study of memory. The publications that use these terms in the same sense as the bibliography were then further screened for publications of general global challenges interest, not for specific global challenges. The most productive search term for the database searches turned out to be "global catastrophe". This term produced a relatively large number of hits and relatively few publications on unrelated topics. Further, the term is used by researchers from a variety of different backgrounds. This makes it a particularly fruitful term for discovering new global challenges research. One hallmark of the global challenges topic is that it is studied by distinct research communities that have limited interaction with each other. As research communities often develop their own terminology, it can be difficult to discover one community by searching for another's terms. For example, "existential risk" is used heavily by researchers studying risk from artificial intelligence and other emerging technologies, but it is rarely used by researchers studying environmental risks. Discovering and connecting the disparate corners of the GCR research is an ongoing challenge for the GCR community. Bibliography searches such as these are an important way to meet this challenge.
 Bibliography Global Challenges Twelve risks that threaten human civilisation -The case for a new category of risks
 Twelve risks that threaten human civilisation -The case for a new category of risks 15 Executive Summary
 Twelve risks that threaten human civilisation -The case for a new category of risks
 Twelve risks that threaten human civilisation -The case for a new category of risks Executive Summary
 Figure 1 : 1 Figure 1:Probability density function
 Figure 2 : 2 Figure 2: Probability density function with tail highlighted
 Figure 4 : 4 Figure 4: Nordhaus, The Climate Casino: Total cost of different targets assuming limited participation and discounting of future incomes.
 Figure 6 : 6 Figure 6: Normal risks and risks with potentially infinite impact.
 Figure Figure 8: Example of F-n curve showing an absoluteimpact level that is defined as unacceptable/ infinite. i.e no level of probability is acceptable above a certain level of impact, in this case 1000 dead64
 Figure 9 : 9 Figure 9: Number of times global challenges are included in surveys of global challenges
 Figure 10 : 10 Figure 10: The global challenges included ten times or more in surveys of global challenges
 Twelve risks that threaten human civilisation -The case for a new category of risks 3.1 Current risks 3.1.1.1 Expected impact disaggregation 3.1.1
 only nine countries are currently known to possess them: the five security council members, India, Pakistan, and North Korea, plus Israel. 230 76 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3.1 Current risks 77 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3Ecological collapse refers to a situation where an ecosystem suffers a drastic, possibly permanent, reduction in carrying capacity for all organisms, often resulting in mass extinction. Usually an ecological collapse is precipitated by a disastrous event occurring on a short time scale. 231 78 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3.1 Current risks 3.1.3.1 Expected impact disaggregation 3.1.3.2 Probability Humans are part of the global ecosystem and so fundamentally depend on it for our welfare.
 Global Figure 18: Increase in the number of species assessed for the IUCN Red List of Threatened SpeciesTM (2000-2013.2). Source: http://www.iucnredlist.org/about/summary-statistics
 A pandemic (from Greek πᾶν, pan, "all", and δῆμος demos, "people") is an epidemic of infectious disease that has spread through human populations across a large region; for instance several continents, or even worldwide.
 Twelve risks that threaten human civilisation -The case for a new category of risks 3
 310 90 Global 90 Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3
 Figure 19 : 19 Figure 19: Network Diagram of connections between, banks, brokers/dealers, insurers and hedge funds. Jan 1994-Dec 1996 Source: https://app.box.com/shared/oesro8zzco0mtvuymh3f
 Figure 20 : 20 Figure 20: How the Spaceguard Survey has reduced the short-term risk of impacts from near-Earth objects 349
 2. The predictability of supervolcanic eruptions. 3. How directly destructive an eruption would be. 4. The effectiveness of general mitigation efforts. 5. How severe the long-term climate effects would be. 103 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
 Figure 21 : 21 Figure 21: Volcanic Explosivity Index 389
 118 Global 118 Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3.3 Emerging risks 01-Jun-13: Nanostart AG: Venture Capital Investments in Nanotech 475 -Initiative
 119 Global 119 Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3Artificial intelligence (AI) is the intelligence exhibited by machines or software, and the branch of computer science that develops machines and software with human-level intelligence. 120 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3
 Twelve risks that threaten human civilisation -The case for a new category of risks 3
 Figure 23 : 23 Figure 23: Number of galaxies that can reach us with speeds of 50%c, 80%c, 99%c and c, from different starting moments 552
 485 132Global 485 Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3
 Figure 24 : 24 Figure 24: Less poverty of nations -Population living below $1.25 a day at 2005 Purchasing-pover parity, % * 0=absolute equality, 100=absolute inequality Source: Economist, originally World Bank, http://www.economist.com/node/<PHONE>
 4. 1 1 Specific relations between global risks "Uncertainty is an uncomfortable position. But certainty is an absurd one." Voltaire -an initial overview 5. Probabilities and uncertainties 142 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 5. Probabilities and uncertainties -an initial overview
 Figure 27 : 27 Figure 27: Different kinds of poverty -Number of people in poverty 638
 Figure 28 : 28 Figure 28: Population of the world, <PHONE>, according to different projections and variants Source: http://esa.un.org/wpp/documentation/pdf/WPP2012_Volume-I_Comprehensive-Tables.pdf, p. xv
 Figure 29 : 29 Figure 29: Moore's forecast for PV 656 Figure 30: Global ICT development 2000-2013 657
 Figure 24 : 24 Figure 24: Population aged 80 or over 661
 Twelve risks that threaten human civilisation -The case for a new category of risks
 Figure 31 : 31 Figure 31: Comparing the probability scale in the Swedish National Risk Assessment 667 and the Likelihood Scale used by the IPCC 668
 35  http://en.wikipedia.org/wiki/Human_embryogenesis 36 These four points are slightly rewritten versions of a list of Nick Bostrom's in the text "Existential Risks:
 59 HSE 2001. Reducing Risks, Protecting People -HSE's Decision-Making Process, 2001. agreement."
 164 http://simple.wikipedia.org/wiki/Nuclear_war165  For an analysis of the possibility of accidental nuclear war between Russia and the USA, seeBarrett, Anthony M., Seth D. Baum, and Kelly Hostetler.: "Analyzing and Reducing the Risks of Inadvertent Nuclear War Between the United States and Russia." Science & Global Security 21.2 (2013): 106-133. 166 Hellman, Martin E.: "How risky is nuclear optimism?." Bulletin of the Atomic Scientists 67.2 (2011): 47-56. 167 See Lundgren, Carl.: "What are the odds? Assessing the probability of a nuclear war." The Nonproliferation Review 20.2 (2013): 361-374.
 168  A 1979 study by the United States Office of Technology Assessment, "The Effects of Nuclear War," estimated 20-160 million immediate casualties from a full-scale nuclear war.
 , Richard P., et al.: "Nuclear winter: global consequences of multiple nuclear explosions." Science <PHONE> (1983): <PHONE>. The Soviet Union was working on similar simulations; see for instance Peterson, Jeannie.: "The aftermath: The human and ecological consequences of nuclear war." New York: Pantheon (1983).
 172 See Mills, Michael J., et al.: "Massive global ozone loss predicted following regional nuclear conflict." Proceedings of the National Academy of Sciences 105.14 (2008): 5307-5312.
 See Toon, Owen B., et al.: "Atmospheric effects and societal consequences of regional scale nuclear conflicts and acts of individual nuclear terrorism." Atmospheric Chemistry and Physics 7.8 (2007): <PHONE>, and Robock, Alan, Luke Oman, and Georgiy L. Stenchikov.: "Nuclear winter revisited with a modern climate model and current nuclear arsenals: Still catastrophic consequences." Journal of Geophysical Research: Atmospheres (<PHONE>) 112.D13 (2007).

 The case for a new category of risks Executive Summary Uncertainties As the different challenges are very different and the status of probability estimates varies significantly, the initial probability numbers are provided together with estimates regarding: 1. Understanding of sequence 1. Understanding of sequence 1. Understanding of sequence 1. Understanding of sequence 2. Data availability 2. Data availability 2. Data availability 2. Data availability 3. Existing probability estimation 3. Existing probability estimation all parts all parts all parts all parts all parts all data all data all data all data all data calculations with small uncertainty calculations with small uncertainty calculations with small uncertainty none at all some parts most parts none at all some parts most parts degree of events from today's actions to infinite impact none at all degree of events from today's actions to infinite impact none at all none at all some parts some parts some parts most parts most parts most parts no data some data most data amount of data to make probability assessment on all relevant steps of the sequence no data amount of data to make probability no data some data most data of the sequence assessment on all relevant steps no data no data some data some data some data most data most data most data no estimates best guesses kind of estimation and uncertainty no estimates no estimates by experts calculations with best guesses by experts by experts best guesses large uncertainty calculations with large uncertainty calculations with large uncertainty degree of events from today's actions to infinite impact degree of events from today's actions to infinite impact degree of events from today's actions to infinite impact amount of data to make probability assessment on all relevant steps of the sequence amount of data to make probability of the sequence assessment on all relevant steps kind of estimation and uncertainty amount of data to make probability of the sequence assessment on all relevant steps kind of estimation and uncertainty 22 21 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Executive Summary
 20  , or $150 quintillion. 2.3 Global challenges and infinite impact This is a very low estimate, and Posner suggests that maybe the cost of a life should be "written up $28 million" for catastrophic risks 52 . Posner's calculations where only one future generation is included result in a cost of $336 quadrillion. If we include all future generations with the same value, $28 million, the result is a total cost of $86 sextillion, or $86 × 10 21 . This $86 sextillion is obviously a very rough number (using one billion years instead of 50 million would for example require us to multiply the results by 20), but again it is the 10 12 stars in our galaxy, and perhaps reference there are about 10 11 to magnitude that is interesting. As a Threshold something like the same number of galaxies. With this simple calculation you get 10 22 to 10 24 , or 10 to 1,000 sextillion, stars in the universe to put the cost of infinite impacts Normal risks when including future generations in Traditional measures perspective. 53 and tools applicable probability 0 impact 44 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
 140
 The extent to which humans are dependent on the ecosystem. 2. Whether there will be effective political measures taken to protect the ecosystem on a large scale. 3. The likelihood of the emergence of sustainable economies. 4. The positive and negative impacts on the eco systems of both wealth and poverty. 5. The long-term effects of an ecological collapse on ecosystems. GOVERNANCE DISASTERS 3.1 Current risks 3.1 Current risks Post-eco-collapse climate change cooperation will be important to any 1. Global coordination and attempt to control ecological damage 9. New, profitable, but environmentally Global poverty Global coordination damaging industries could put extra strain on the ecosystem. 19. Technological innovations Sustainability research may result in more sustainable economies, or in more during 2013 3.1.3.3 Main events on a large scale and prevent "races to environmentally damaging products. the bottom". 22-Jan-13: Current extinctions 10. According to some systems of value, the loss of certain animals ecological effects Long-term Global instability probably the result of past actions; Global povety Global coordination 2. Poverty is often seen as and ecosystems constitutes a moral many future extinctions to come 242 Technological through unsustainable practices, innovations exacerbating ecological damage tragedy in and of itself. -Research 20. It may be possible to Deliberate attempts to construct world dictatorship ensure human survival in semi-hydroponic food, distilled water), "closed" systems (solar power, while richer countries introduce An estimated 40% of world trade 11. Humans derive much pleasure and with minimal dependency on the Quality of life loss from ecosystem loss environmental regulations -but richer many benefits from various parts of Ecological collapse Preservation efforts New system of governance Smart sensors external ecosystem. is based on biological products Meta-uncertainty on tradeoffs nations exploit many resources (such the ecosystem, and losing this would or processes such as agriculture, between e.g. poverty, survival, as fossil fuels) in non-sustainable and result in a loss to human quality of life. 21. Over the long term, it may become forestry, fisheries and plant-derived freedom damaging ways. possible and necessary to go about pharmaceuticals, and biodiversity comprises an invaluable pool 12. Ongoing and continuous rebuilding the ecosystem and healing Threat to economies, or sustainable food supply 3. Transitioning to sustainable for innovations. Moral tragedy from ecosystem loss consequence of ecological collapse. Pollution biodiversity loss is a clear its damage. economic trajectories, could control 22. Political decisions will be the most Economic costs Failing to solve 4. Research into sustainability could Loss of biodiversity triggering famines. Improvements to global governance important problems ecological damage. 13. Ecological damage can put Making Pre-eco-collapse climate change likely factors to exacerbate or mitigate things worse the human food system in danger, an ecological disaster. allow the construction of sustainable 23. It is unclear how dependent economies or environments at costs 14. Ecological damage increases humans truly are on the that people are willing to bear. vulnerability to floods and other ecosystem, and how much Disruption to politics and economy 5. Climate change exacerbates the Rebuilding the ecosystem natural disasters. Post-eco-collapse politics damage they could inflict without threatening their own survival. Enduring poverty pressure on the ecological system Not achieving important ethical goals by changing weather patterns and Climate change 15. Disruptions to the world's political Global pollution and economic systems could trigger Lack of human flourishing Vulnerabilities to flood and other disasters increasing natural disasters in ways further conflicts or instabilities, Sustainable or non-sustainable economies ecosystems find hard to adapt to. causing more casualties and impairing New, environmentally damaging industries effective response. 6. Global pollution is a visible source of ecological damage, one that global 16. Since a lot of the world's carbon is Undesirable world system (e.g. global dictatorship) Pre-eco-collapse mitigation efforts locked up in trees, ecological collapse Technological could exacerbate climate change. innovations Collapse of agreements have had moderate success at tackling. Disruption to world politics and economy world system Post-disaster politics Long-term negative effects 7. Truly global preservation efforts 17. The ecosystem is of great may be needed for some threatened economic benefit to humanity, Meta-uncertainty on the true dependence of humanity on the ecosystem so its loss would have large Human survivability in "closed" systems natural boundaries (e.g. in the seas ecosystems that stretch beyond economic costs. and oceans). 18. Ecological damage is likely to Total short-term casualties 8. Beyond general all-purpose mitigation efforts, addressing General mitigation effort be long-term: the effects will last for Extinction many generations. Civilisation collapse this threat could include the Total short-term casualties species or genetic codes, to allow a preservation of ecosystems, Civilisation collapse Extinction subsequent rebuilding. Key Key Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts Current intervention areas Bad decisions Accidents Severe impacts Uncertain events Meta-uncertainties Risk events Direct impacts Indirect impacts Current intervention areas Bad decisions Accidents Severe impacts 80 79 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3.1 Current risks ECOLOGICAL CATASTROPHE Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 81 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
 and trade. 573  137 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 4. Relations between global risk 3.4 Global Policy risk 4. Relations between Two things make the understanding 4.1 General relations of the relation between the global between global risks risks particularly important. and their potential impacts 1. Impacts: The global risks global risks are interconnected in different Relations between global risks is an ways. Often the situation can be area where surprisingly little work is described as a set of dominoes: if being done. Most research focuses one falls, many others follow. Even on individual or closely related small impacts can start a process groups of challenges. Organisations where different challenges interact. working on global challenges are Higher temperatures due to global almost always working on individual warming can result in the spreading risks. The initial overview below is of pandemics which increase based on individual studies where tensions between countries, and different relations are analysed, but so on. no work has been identified where the relations between all twelve 2. Specific measures to address challenges have been analysed. a risk: Global risks often require significant changes in our current A risk that is natural to start with is society, from how we build cities future bad global governance, as all to how food is produced and other global challenges exacerbate provided. Such significant changes governance disasters, 575 and all other will result in situations where global challenges can potentially be measures to reduce the risk in exacerbated by governance disasters. one area affect the probability A well functioning global governance and/or the impact in other areas. system is therefore a key factor to Depending on the measure chosen address global catastrophic risks. to reduce the risk, and other complementary measures, the Conversely, avoiding governance effect can be positive or negative. disasters improves all risks, as better institutions are better able to mitigate risks. Governance disasters directly increase the problems of climate change (through a lack of coordination between countries), the risk of nuclear war (by stoking conflict between nuclear powers) and "We have some idea what might happen if, global system collapse (by weakening global responses to systemic risks). in the face of other pressing global challenges, All risks exacerbate global system collapse, by putting extra stress on an we divert our focus from making systemic interconnected system. improvements in public health and veterinary services -and that prospect is frightening." The World Bank 574 138 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
 Twelve risks that threaten human civilisation -The case for a new category of risks 4.1 General relations between global risks and their potential impacts 4.2 Specific relations between global risks Below is an example of an overview of how different global challenges can be plotted depending on the technical difficulty of reducing An international initiative should start to achieve better understanding of the relations between global challenges in order to ensure the risk and the difficulty of synergies and avoid strategies that In parallel with work to increase our collaborating to reduce it. will undermine other challenges. understanding about the general relations between global risks, work to identify more specific relations should also be initiated. This is an area where people die / suffer many pieces of research exist. But very little work has been done to combine them and assess different strategies to address specific global risks and understand how these strategies will affect other global risks. It is important to distinguish between two different support use people afraid of infections short term thinking kinds of specific relations. Extreme Climate Change Global Pandemic First, there are solution strategies for one global risk and the ways it affects other global risks. For example, using video conferences can reduce the probability of pandemics by reducing unnecessary travel. On the other hand, unsustainable use of bio-energy could increase spillover opportunities when a zoonosis (a disease transmitted more renewable energy more video meetings less meat consumption from nature accident attack from animals to humans) increases reduce risk the spread of pandemics due to an increased number of contacts between increase risk humans and infected animals in forests around the world. 593 Second, how society reacts to the very threat of different risks can affect other challenges. For example, if people are afraid of pandemics they might use more video meetings and in that way help reduce carbon emissions. solving first risk improves second risk first risk worsens second risk Attempts to develop solutions for specific global challenges should assess their impacts, positive and negative, on other challenges. In order to better understand the relations between different global challenges, work could start to analyse similarities and differences. technical difficulty of reducing risk both of the above collaboration difficulty of reducing risk 140 Global Challenges - Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 141
 15 Climate Impacts: http://www.ipcc.ch/report/ar5/wg2/ Pandemics: http://www.cmu.edu/dietrich/sds/docs/ fischhoff/AF-GPH.pdf Nuclear war: http://www.ippnw.org/nuclear-famine.html 16 http://news.stanford.edu/news/2003/september24/ tellerobit-924.html 17 http://www.fas.org/sgp/othergov/doe/lanl/ docs1/<PHONE>.pdf
 For examples see the recently established Centre for Study of Existential Risk at Cambridge. It is an interdisciplinary research centre focused on the study of human extinction-level risks that may emerge from technological advance 27 E.g. HBR blog about "black swans": http://blogs. hbr.org/2010/09/the-competing-black-swans-of-s/ and The Guardian about "perfect storms" http://www. theguardian.com/environment/earth-insight/2014/ apr/03/ipcc-un-climate-change-perfect-storm-zombieoil 28 http://www.economist.com/node/<PHONE> 29 http://e360.yale.edu/feature/living_in_the_ anthropocene_toward_a_new_global_ethos/2363/ 30 http://www.nickbostrom.com/existential/risks.html 31 https://www.youtube.com/watch?v=VjZyOTES6iQ 32 http://en.wikipedia.org/wiki/Long_tail 33 http://en.wikipedia.org/wiki/Human 34 Mammals have an average species lifespan from origin to extinction of about 1 million years, but that will not necessarily apply to humans today because of factors like local climate change or new species in the same ecological niche. The dinosaurs were around for 135 million years and if we are intelligent, there are good chances that we could live for much longer.
 Posner, Richard A. Catastrophe: Risk and Response. Oxford University Press, 2004, Loc 2378 53 http://www.esa.int/Our_Activities/Space_Science/ Herschel/How_many_stars_are_there_in_the_Universe? 54 http://www.existential-risk.org/concept.pdf 55 McSharry, Patrick E., and David Orrell (2009). A Systems Approach to Forecasting. FORESIGHT: The International Journal of Applied Forecasting, referring to Wolfram, Stephen (2002):A new kind of science, V Wolfram media. http://www.irgc. org/wp-content/uploads/2013/06/McSharry-OrrellMcSharry2009foresight.pdf 56 From A Systems Approach to Forecasting" by David Orrell and Patrick McSharry, http://www. irgc.org/wp-content/uploads/2013/06/McSharry-OrrellMcSharry2009foresight.pdf referring to Wolfram, S.
 See McManus, J. F., et al.: "Collapse and rapid resumption of Atlantic meridional circulation linked to deglacial climate changes." Nature <PHONE> (2004): 834-837. 161 See Schuur, Edward AG, et al.: "Vulnerability of permafrost carbon to climate change: Implications for the global carbon cycle." BioScience 58.8 (2008): 701-714.
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Executive SummaryExecutive SummaryGlobal Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 2. Risks with infinite impact: A new category of risks
			 .3 Global challenges and infinite impact
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 3.1 Current risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks3.3 Emerging risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks3.3 Emerging risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks3.3 Emerging risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks3.3 Emerging risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks3.3 Emerging risks
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 7. Possible ways forward
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 7. Possible ways forward
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 7. Possible ways forward
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 7. Possible ways forward
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 7. Possible ways forward
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks 7. Possible ways forward
			 Pinker, Steven. The better angels of our nature: The decline of violence in history and its causes. Penguin UK, 2011.11 Giddens, Anthony. "Risk and responsibility." The modern law review 62.1 (1999): 1-10.12 http://blogs.ei.columbia.edu/2012/01/09/evolutionarypsychology-of-climate-change/
			 Posner, Richard A. Catastrophe: Risk and Response. Oxford University Press, 2004, 38 http://en.wikipedia.org/wiki/Value_of_life 39 http://yosemite.epa.gov/EE%5Cepa%5Ceed.nsf/ webpages/MortalityRiskValuation.html 40 Posner, Richard A. Catastrophe: risk and response. Oxford University Press, 2004. Loc 2363 41 http://webarchive.nationalarchives.gov.uk/+/http:/ www.hm-treasury.gov.uk/sternreview_index.htm 42 William Nordhaus , The Stern Review on the Economics of Climate Change http://www.econ.yale. edu/~nordhaus/homepage/stern_050307.pdf 43 http://www.res.org.uk/view/art3Apr08Features.html 44 Nordhaus, William D. The Climate Casino: Risk, Uncertainty, and Economics for a Warming World. Yale University Press, 2013. Loc 2895 45 Nordhaus, William D. The Climate Casino: Risk, Uncertainty, and Economics for a Warming World. Yale University Press, 2013. Loc 2895 46 Nordhaus, William D. The Climate Casino: Risk, Uncertainty, and Economics for a Warming World. Yale University Press, 2013. Loc 3176 Endnotes 184 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Endnotes
			 Arsenals have been reduced, but there remain over 17,000 nuclear warheads in the world's arsenals (source: SIPRI yearbook 2013), down from a peak of some 68,000 in 1983, still more than enough to trigger a nuclear winter.177  See Toon, Owen B., et al.: "Atmospheric effects and societal consequences of regional scale nuclear conflicts
			 Russian Federation On Strategic Offensive Reductions.216  The Treaty between the United States of America and the Russian Federation on Measures for the Further
			 See DARPA's briefing, DARPA-SN-13-30: Probabilistic Programming for Advancing Machine Learning (PPAML) (2013).498  See for instance Curtis, Jon, Gavin Matthews, and David Baxter.: On the effective use of Cyc in a question answering system. Proc Workshop on Knowledge and Reasoning for Answering Questions. (2005).499  See The Rise of Industrial Big Data, GE Intelligent Platforms (2013).500 Such as Watson's triumph on "Jeopardy!" (source: Ferrucci, David, et al.: Building Watson: An overview of 192 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Endnotes
			 Global Challenges -Twelve risks that threaten human civilisation -The case for a new category of risks Appendix 1 -Global Challenges Bibliography