APPENDIX 8B. UNCERTAINTY AND VARIABILITY

TABLE OF CONTENTS

8B.1 INTRODUCTION ...................................................................................................8B-1

8B.2 UNCERTAINTY .....................................................................................................8B-1

8B.3 VARIABILITY........................................................................................................8B-1

8B.4 APPROACHES TO UNCERTAI NTY AND VARIABILITY ...............................8B-2

8B.5 PROBABILITY ANALY SIS AND THE USE OF CRYSTAL BALL...................8B-2

LIST OF FIGURES

Figure 8B.5.1 Normal Probability Distribution ..................................................................8B-3 Figure 8B.5.2 Triangular Probability Distribution..............................................................8B-3 Figure 8B.5.3 Uniform Probability Distribution.................................................................8B-3

8B-iAPPENDIX 8B. UNCERTAINTY AND VARIABILITY

8B.1 INTRODUCTION

Analysis of an energy-efficiency standard involves calculations of impacts, for example, the impact of a standard on consumer life-cycle cost (LCC). In order to perform the calculation, the analyst must first: 1) specify the equation or model that will be used; 2) define the quantities in the equation; and 3) provide numerical values for each quantity. In the simplest case, the equation is unambiguous (contains all relevant quantities and no others), each quantity has a single numerical value, and the ca lculation results in a single valu e. However, unambiguity and precision are rarely the case. In almost all cases , the model and/or the numerical values for each quantity in the model are not completely known (i .e., there is uncertainty) or the model and/or the numerical values for each quantity in the m odel depend upon other conditions (i.e., there is variability).

Thorough analysis involves accounting for un certainty and variability. While the simplest analysis involves a single numerical va lue for each quantity in a calculation, arguments can arise about what the appropriate value is for each quantity. Explicit analysis of uncertainty and variability is intended to provide more comple te information to the decision-making process.

8B.2 UNCERTAINTY

When making observations of past events or speculating about the future, imperfect knowledge is the rule rather than the exception. For example, the energy actually consumed by a particular appliance type (such as the average U.S. commercial ai r conditioner or heat pump) is not directly recorded, but rather estimated based upon available information. Even direct laboratory measurements have some margin of error. When estimating numerical values expected for quantities at some future date, th e exact outcome is rare ly known in advance.

8B.3 VARIABILITY

Variability means that differe nt applications or situations produce different numerical values when calculating a quantity. Specifying an exact value for a quantity may be difficult because the value depends on something else. For example, the number of hours an air conditioner is operated by a household depends upon the specific circumstances and behaviors of the occupants (e.g., number of persons, personal habits about how comfor table the person wants to be, etc.). Variability makes specifying an appropriate population valu e more difficult in as much as any one value may not be representati ve of the entire population. Surveys can be helpful here, and analysis of surveys can relate the variable of interest (e.g., hours of use) to

other variables that are better known or easie r to forecast (e.g., persons per household).

8B-18B.4 APPROACHES TO UNCERT AINTY AND VARIABILITY

This section describes two approach es to uncertainty and variability:

• scenario analysis, and

• probability analysis.

Scenario analysis uses a single numerical value for each quantity in a calculation, then changes one (or more) of the numerical values and repeats the calcula tion. A number of calculations are done, which provide some indicati on of the extent to which the result depends upon the assumptions. For example, the life-cycle co st of an appliance co uld be calculated for energy rates of 2, 8, and 14¢ per kWh.

The advantages of scenario analysis are that each calculation is simple; a range of estimates is used; and crossover points can be iden tified. (An example of a crossover point is the energy rate above which the life-c ycle cost is reduced, holding a ll other inputs constant. That is, the crossover point is the ener gy rate at which the consumer achieves savings in operating expense that more than compensate for the in creased purchase expense. ) The disadvantage of scenario analysis is that th ere is no information about th e likelihood of each scenario.

Probability analysis consider s the probabilities within a ra nge of values. For quantities with variability (e.g., electricity rates in different households), surveys can be used to generate a frequency distribution of numerical values (e.g., the number of households with electricity rates at particular levels) to estimate the probability of each value. For quantities with uncertainty, statistical or subjective measures can be used to provide probab ilities (e.g., manufact uring cost to improve energy efficiency to some level may be estimated to be $10 ± $3).

The major disadvantage of the probability appr oach is that it requires more information, namely information about the shapes and magnit udes of the variability and uncertainty of each quantity. The advantage of the probability appro ach is that it provides greater information about the outcome of the calculations, that is, it provide s the probability that the outcome will be in a particular range.

Scenario and probability analys is provide some indication of the robustness of the policy given the uncertainties and variability. A polic y is robust when the impacts are acceptable over a wide range of possible conditions.

8B.5 PROBABILITY ANALYSIS AND THE USE OF CRYSTAL BALL

To quantify the uncertainty and variability th at exist in inputs to the engineering, LCC, and payback period (PBP) analyses, the Depart ment used Microsoft Excel spreadsheets combined with Crystal Ball, a commercially av ailable add-in, to conduct probability analyses. The probability analyses used Monte Carlo simulation and probability distributions.

8B-2Simulation refers to any analytical method m eant to imitate a real-l ife system, especially when other analyses are too mathematically comp lex or too difficult to reproduce. Without the aid of simulation, a spreadsheet model will only reveal a single outcome, generally the most likely or average scenario. Spreadsheet risk analysis uses both a spreadsheet model and simulation to automatically analyze the effect of varying inputs on out puts of the modeled system. One type of spreadsheet simulati on is Monte Carlo simulation, which randomly generates values for uncertain variables again and again to s imulate a model. Monte Carlo simulation was named for Monte Carlo, Monaco, where the primary attractions are casinos containing games of chance. Games of chance such as roulette wheels, dice, and slot machines,

exhibit random behavior. The random behavior in games of chance is similar to how Monte

Carlo simulation selects variable values at random to simulate a model. When you roll a die, you know that either a 1, 2, 3, 4, 5, or 6 will come up, but you do not know which for any particular

roll. It's the same with the vari ables that have a known range of values but an uncertain value for any particular time or event (e.g., equipment lif etime, discount rate, and installation cost).

For each uncertain variable (one that has a range of possible values), possible values are defined with a probability distribution. The t ype of distribution selected is based on the conditions surrounding that va riable. Probability dist ribution types include:

NORMAL TRIANGULAR UNIFORM

Figure 8B.5.1 Normal Probability Figure 8A.5.2 Tria ngular Figure 8A.5.3 Uniform Probability

Distribution Probab ility Distribution

Distribution

During a simulation, multiple scenarios of a model are calculated by repeatedly sampling values from the probability distri butions for the uncertain variables and using those values for the cell. Crystal Ball simulations can consist of as many trials (or scenarios) as desired—hundreds or even thousands. During a singl e trial, Crystal Ball randomly selects a value from the defined possibilities (the range and shape of the probabilit y distribution) for each uncertain variable and then recalculates the spreadsheet.

8B-3