# Presentation: 720947

## Scenario Analysis Under Deep Uncertainty

**Robert Lempert**
**Senior Scientist**
**RAND Corporation**
**DOE/EPA Workshop on Modeling the Oil Transition**
**April 21, 2006**

## Uncertainty Makes Modeling theOil Transition a Significant Challenge

- Traditional analytic methods characterize uncertainties as a prelude to assessing alternative decisions
- Shaping the transition to a novel future confronts decision makers with deep uncertainty, where
- They do not know, and/or key parties to the decision do not agree on, the system model, prior probabilities, and/or “cost” function
- Decisions can go awry if decisionmakers assume future is well-characterized when it is not
- Uncertainties are underestimated
- Competing analyses can contribute to gridlock
- Misplaced concreteness can blind decision-makers to opportunities and surprise

## Novel Futures Often Include Discontinuities and Surprise

***Novel Futures Often Include Discontinuities and Surprise***

**Gross national product (billions**
**of 1958 dollars)**

**Energy use (10****15**** Btu per year)**

**2,200**

**0**

**2,000**

**1,800**

**1,600**

**1,400**

**1,200**

**1,000**

**800**

**600**

**400**

**200**

**0**

**Historical trend continued**

**1970**

**1920**

**1929**

**1940**

**1950**

**1960**

**1910**

**1973**

**1973**

**1900**

**1890**

**20**

**40**

**60**

**80**

**100**

**120**

**140**

**160**

**180**

**Data analysis and scenarios can fail to capture the potential for surprise**

**1975 Scenarios**

## Even in Fantasy, the Future Is Often Only a Mirror of the Present

**“****Missions to Mars”**
** ****at Disneyland’s Tomorrowland**
**~1955**

## Outline

**Robust decisions under deep uncertainty**
**Analytic methods for identifying key scenarios**
**Making good use of bad models**

## Scenarios Can Help Decision Makers Grapple with Novel or Unwelcome Futures

**Attractive because have potential to**
**Break barriers to thinking about the future**
**Enrich debate and widen “strategic conversation”**
**Make risks more transparent, enhance resilience**
**Encourage buy-in from participants**
**Trigger formal planning processes**

**However, scenario approaches often...**
**Present no more than a few alternatives**
**Don’t provide for systematic comparison of options**
**Fail to engage decision makers’ mental models**
**Leave a gap between insight and actionable steps**** **

## Robust Decision Making Combines Best Features of Scenario and Analytic Approaches

**Robust Decision Making (RDM) is an iterative, analytic process for**
- Identifying strategies that perform reasonably well compared to the alternatives across a very wide range of plausible futures
- Characterizing a small number of irreducible tradeoffs inherent in the choice among such robust strategies
**RDM characterizes uncertainty, and the relationship between near-term choices and long-term outcomes, in ways that**
- Decision makers find credible
- Contribute usefully to contentious debates

## A Simple Example Helps Demonstrate Value of Robust Strategies

**We used a simple system dynamics model to assess near-term actions to promote long-term economic development and environmental quality**
**Model includes a wide range of uncertainties**
**Economic, demographic, technological**
**Environmental, including unknown threshold for abrupt changes and level of resilience**
**Behavior of future decisionmakers who may or may not choose to cooperate with one another and protect the environment**

*Robert J. Lempert, Steven W. Popper, Steven C. Bankes, 2003: *_*Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis*_*, RAND MR-1626-RPC.*

## Near-Term Policy Must Confront a Wide Range of Uncertainties

## Compare “Fixed” Near-Term Strategies  Across Scenarios

**Near term**

**Choose policies**

**Assume near-term policy continues until ****changed by future generations **

**Future decision-makers recognize and correct our mistakes**

**Future**

## Exploration DemonstratesNo “Fixed” Strategy Is Robust

**No regret**
**Mild**

**A lot**
**Overwhelming**

**1.0**

**2.0**

**3.0**

**4.0**

**–****1.0**

**0**

**1.0**

**5.0**

**3.0**

**4.0**

**2.0**

**0**

**Conventional**
**world****scenario**

**1.0**

**2.0**

**3.0**

**4.0**

**–****1.0**

**0**

**1.0**

**5.0**

**3.0**

**4.0**

**2.0**

**0**

**Conventional**
**world****scenario**

**Economic growth rate (%)**

**Decoupling****rate (%)**

***Stay the Course***

***Crash Effort***

## Examine Strategy with Indexed, Differentiated Targets (North and South) and Safety Valve

## “Safety Valve” Strategy Appears Highly Robust

***Safety valve***

**Economic growth rate (%)**

**1.0**

**2.0**

**3.0**

**4.0**

**–****1.0**

**0**

**1.0**

**5.0**

**3.0**

**4.0**

**2.0**

**0**

**Economic growth rate (%)**

**1.0**

**2.0**

**3.0**

**4.0**

**–****1.0**

**0**

**1.0**

**5.0**

**3.0**

**4.0**

**2.0**

**0**

**U.S. in 19th**
**century**

**U.S. since 1950**

**U.S. in 20th**
**century**

**U.S. in 19th**
**century**

**U.S. since 1950**

**U.S. in 20th**
**century**

**Decoupling rate (%)**

## Outline

**Robust decisions under deep uncertainty**
**Analytic methods for identifying key scenarios**
**Making good use of bad models**

## Robust Decision Making Identifies Robust Strategies and Their Residual Vulnerabilities

**	**

**	****Characterize scenarios representing key vulnerabilities and opportunities**

**Identify tradeoffs among well-hedged strategies**

**Select candidate robust strategy**

## These Residual Vulnerabilities Represent Key Scenarios Important to Decision Makers

**	**

**	****Characterize scenarios representing key vulnerabilities and opportunities**

**Identify tradeoffs among well-hedged strategies**

**Select candidate robust strategy**

## Expert “Red-Teaming” Can Help Identify Surprises

**“****Safety Valve” strategy is robust against a very wide range of futures**
***many of which we might regard as surprising***
**Test strategy against discontinuous “surprises”**
**Rapid technological advance that eliminates emissions**
**Plague that decimates population for twenty years**
**Future generations whose utility is completely disconnected from the environment**

## Safety Valve Still Robust with Surprises

**Economic growth rate**

***Safety valve***

## Analytic Tools Can Help Identify Key Vulnerabilities of Strategies

- Calculate “regret” of Safety Valve strategy over experimental design of many tens of thousands of plausible  futures
- “Data-mining” method identifies low-dimensional, easy-to-interpret region of input parameter space where strategy performs poorly
- Decisionmakers should focus attention on this “Low Global Decoupling” scenario

## What are Best Strategies to Hedge Against These Vulnerabilities?

**Assessment of strategies over two computer-generated scenarios **

**Regret in SV01.005.002 “Satisficing” Futures**

**Regret in low-global- decoupling futures**

## Detailed Analysis Suggests Best Safety Valve Design

***Detailed Analysis Suggests ******Best Safety Valve Design***

- 1:100

- 1:10

- 1:1

- 10:1

- 100:1

- SV01-0.5%-0.2%

- SV02-1%-1.5%

- SV01-1%-1.5%

- SVab-x%y%
- a = N milestone
- b = S milestone
- x% = N cost threshold
- y% = S cost threshold

- SV02-1%-1.5%

- SV02-0.5%-0.2%

- SV01-1%-1.5%

- Robust
- Regions

- }

**Initial candidate strategy**

## These Analytic Methods  for Generating Narrative Scenarios May Improve Stakeholder Processes

- California Department of  Water Resources (DWR) used  stakeholder process to  develop three scenarios to inform their latest 25-year  water  plan
- Scenarios had mixed success with stakeholders
- We are working with DWR and other California water agencies to identify scenarios representing vulnerabilities of current plans
- Using formal evaluation methods to assess how they affect decision making

## Outline

**Robust decisions under deep uncertainty**
**Analytic methods for identifying key scenarios**
**Making good use of bad models**

## A “Robust” Approach Helps Find Solutions That Work Even in the Face of Potential Surprises

**Use many scenarios to imagine the future —not a single forecast **
**Seek robust strategies that do well across many scenarios assessed according to several values — not optimal strategies**
**Employ strategies that evolve over time in response to changing conditions — not "fixed" strategies**

## Modeling Ought  to Help  Identify Robust Shaping Strategies to Help Navigate the Oil Transition

**When faced with novel futures, most forecasts are likely to  be wrong**
**But models can usefully serve as “scenario generators” that help identify and test potentially robust near-term strategies**
**At their  best, scenarios can help change how  decision makers understand the world and the effects of their actions upon it**
**In practice, model-generated  scenarios often scan a small range of relatively safe futures**
**Reconceiving scenarios as vulnerabilities of candidate strategies  can make them powerful  tools in the development of  robust strategies**

## More Information

- Robert J. Lempert, David G. Groves, Steven W. Popper, Steve C. Bankes: "A General, Analytic Method for Generating Robust Strategies and Narrative Scenarios,” _Management Science_, vol 52, no 4, April 2006
- Steven W. Popper,  Robert J. Lempert, and Steven C. Bankes: "Shaping the Future," _Scientific American_, vol 292, no. 4  pp. 66-71, April 2005
- Robert J. Lempert, Steven W. Popper, Steven C. Bankes:      _Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis_, RAND MR-1626-RPC, Aug 2003
**www.rand.org/ise/projects/improvingdecisions **

## Slide 27