8 B

- i APPENDIX 8B. UNCERTAINTY AND VARI ABILIT Y

TABLE OF CO NTENTS

8B.1 INTRODUCTION

................................................................................................... 8B-1

8B.2 UNCERTAINTY

..................................................................................................... 8B-1

8B.3 VARIABILITY

........................................................................................................ 8B-1

8B.4 APPROACHES TO UNCERTAINTY AND VARIABILIT Y ............................... 8B-2

8B.5 PROBABILITY ANALYSIS AND THE USE OF CRYST AL BALL

................... 8B-2

LIST OF FIGURE S

Figure 8 -B.5.1 Normal Probability Distribution ..................................................................8B-38B

- 1 APPENDIX 8B. UNCERTAINTY AND VARI ABILITY

8B.1 INTRODUCTION

Analysis of a potential energy efficiency standard inv olves calculating effects, for example, the effe ct of a standard on consumer life- cycle cost (LCC). T o perform the calculation, the analyst must first: (1) specify the equation or model that will be used; (2) define the quantities in the equation or model; and (3) provide numer ical values for each quantity. In the simplest case, the equation is unambiguous ( it contains all relevant quantities and no others), each quantity has a single numerical value, and the calculat ion produces a single value. Unambiguousness and precision are rarely the case , however. In most cases, the model and/or the numerical values for each quantity in the model are not completely known ( i.e. , there is uncertainty) or the model and/or the numerical values for each quantity in the model depend upon other conditions ( i.e.

, there is variability).

Thorough analysis involves accounting for uncertainty and variability. Although the simplest analysis involves a single numerical value for each quantity in the cal culation, arguments can arise about the appropria te value for each quantity. Explicit analysis of uncertainty and variability provides more complete information to the decision

- making process.

8B.2 UNCERTAINTY

When making observations of past events or speculating about the future, imperfect knowledge is the rule rather than the exception. For example, the energy consumed by a particular type of appliance (such as the average residential clothes washer ) is not recorde d directly , but rather estimated based on available information. Even direct laboratory

measurements have a margin of error. When estimating numerical values expected for quantities at some future date, the exact outcome rarely is known.

8B.3 VARIABILITY

Varia bility in the calculation of a quantity means that different applications or situations produce different numerical values. Specifying an exact value for a quantity may be difficult because the value depends on something else. For example, the number of ho urs a household

operates a clothes washer depends on the specific circumstances and behaviors of the occupants ( e.g. , number of persons, personal habits ). Variability makes specifying an appropriate population value more difficult, because no one value is likely to be representa tive of the entire population. Surveys can be helpful here, and analysis of surveys can relate the variable of interest ( e.g. , hours of use) to other variables that are better known or easier to forecast ( e.g. , number of persons per household).8B

- 2 8B.4 APPROACHES TO UNCERT AINTY AND VARIABILIT Y

This section describes two approaches to uncertainty and variability:

**•** scenario analysis and

**•** probability analysis.

Scenario analysis uses a single numerical value for each quantity in a calculation, then changes one (or more) of the numerical values and repeats the calculation. Numerous

calculations are performed , which provide some indication of the exten t to which the result depends on the assumptions. For example, the LCC of an appliance could be calculated based on electricity costs of 2, 8, and 14 cents per kilowatt -hour.

The advantages of scenario analysis are that each calculation is simple; a range of estimates is used; and cross over points can be identified. (An example of a crossover point is the energy rate above which the LCC is reduced, holding all other inputs constant ; t hat is, the energy rate at which the consumer achieves savings in operating costs that more than compensate for the increased purchase price .) The disadvantage of scenario analysis is that there is no information

about the likelihood of each scenario.

Probability analysis considers the probabili ties within a range of values. For quantities characterized by variability ( e.g. , electricity rate s in different households), surveys can be used to generate a frequency distribution of numerical values ( e.g. , the number of households subject to electricity rates at particular levels) to estimate the probability of each value. For quantities characterized by uncertainty, statistical or subjective measures can be used to provide probabilities ( e.g. , manufacturing cost to improve energy efficiency to a given level may be estimated to be $10 ± $3).

The major disadvantage of the probability approach is that it requires more information, namely information about the shapes and magnitudes of the variability and uncertainty of each quantity. The advantage of the probability approach is that it provides more information about the outcome of the calculations; that is, it provides the probability that the outcome will be in a particular range.

Scenario and probability analysis provide some indication of the robustness of a policy given the identified uncertainties and variability. A policy is robust when the impacts are acceptable over a wide range of possible conditions.

8B.5 PROBABILITY ANALYSIS AND THE USE OF CRYSTAL BALL

To quantify the uncertainty and variability that exist in inputs to the engineering, LCC, and payback period analyses, DOE used Microsoft Excel spreadsheets combined with Crystal Ball, a commercially available add

- in software , to conduct probability analyses. The probability analyses used Monte Carlo simulation and probability distributions.8B

- 3 Simulation refers to any analytical method meant to duplicate a real

- life system, especially when other analyses are too mathematically complex or difficult to reproduce. Without the aid of simulation, a spreadsheet model will reveal only a single outcome, generally the most likely o r average outcome. Spreadsheet risk analysis uses both a spreadsheet model and simulation to automatically analyze the effect of varying inputs on outputs of the modeled system. One type of spreadsheet simulation is Monte Carlo simulation, which randomly

g enerates values for uncertain variables numerous time s. Monte Carlo simulation was named for Monte Carlo, Monaco, where the primary attractions are casinos containing games of chance. Games of chance such as roulette wheels, dice, and slot machines, exhibi t random behavior. The random behavior in games of chance is similar to how Monte Carlo simulation selects variable values at random to simulate a model. When you roll a die, you know that a 1, 2, 3, 4, 5, or 6 will come up, but you do not know which numbe r for any particular roll. So too with variables that have a known range of values but an uncertain value for any particular time or event ( e.g. , product lifetime, discount rate, and installation cost).

For each uncertain variable (a variable that has a range of possible values), a probability distribution is used to define possible values. The type of distribution selected is based on the conditions surrounding that variable. Types of p robabi lity distribution s include the following.

Figure 8B.5.1 Normal Probability Figure 8

- B.5.2 Triangular Figure 8 -B .5.3 Uniform Probability

Distribution Probability Distribution

Distribution

During a simulation, multiple scenarios are calculated by sampling values repeatedly from the probability distributions for the uncertain variables. Crystal Ball simulations can consist of as many trials (or scenarios) as desired —hundreds or even thousands. During a single trial, Crystal Ball randomly selects a value from the defined possibilities (the range and shape of the probab ility distribution) for each uncertain variable and then recalculates the spreadsheet. NORMAL TRIANGULAR UNIFORM