APPENDIX 8B. UNCERTAINTY AND VARIABILITY

 TABLE OF CONTENTS 

8B.1 INTRODUCTION .................................................. 8B-1


8B.2 UNCERTAINTY ................................................... 8B-1


8B.3 VARIABILITY .................................................... 8B-1


8B.4 APPROACHES TO MODELING UNCERTAINTY AND VARIABILITY ...... 8B-2


8B.5 PROBABILITY ANALYSIS AND THE USE OF CRYSTAL BALL ........... 8B-3


LIST OF FIGURES 

Figure 8B.5.1 Normal Probability Distribution .................................. 8B-3


Figure 8B.5.2 Triangular Probability Distribution ................................ 8B-3


Figure 8B.5.3 Uniform Probability Distribution .................................. 8B-3


8B-i 

 EE-RM/STD-00-550 COMMENT 196
APPENDIX 8B. UNCERTAINTY AND VARIABILITY 

8B.1 INTRODUCTION 

Analysis of an energy efficiency standard involves calculations of impacts—for example, 

the impact of a standard on life-cycle cost (LCC). To perform the calculation, the analyst must 

first: 1) specify the equation or model that will be used; 2) define the quantities in the equation; 

and 3) provide numerical values for each quantity. In the simplest case, the equation is 

unambiguous (i.e., contains all relevant quantities and no others), each quantity has a single 

numerical value, and the calculation results in a single value. However, unambiguity and precision 

are rarely the case. Even in the simplest analysis, involving a single numerical value for each 

quantity in a calculation, arguments can arise in determining the appropriate value for each 

quantity. In almost all cases, the model and/or the numerical values for each quantity in the model 

are not completely known (i.e., there is uncertainty) or the model and/or the numerical values for 

each quantity in the model depend on other conditions (i.e., there is variability). 

Thorough analysis involves accounting for uncertainty and variability. While the simplest 

analysis involves a single numerical value for each quantity in a calculation, arguments can arise 

about what the appropriate value is for each quantity. DOE intends explicit analysis of 

uncertainty and variability to provide more complete information to the decision-making process. 

8B.2 UNCERTAINTY 

When making observations of past events or speculating about the future, imperfect 

knowledge is the rule, rather than the exception. For example, the energy actually consumed by a 

particular piece of equipment (such as a distribution transformer) is not directly recorded, but 

rather estimated based on available information. Even direct laboratory measurements have some 

margin of error. When estimating numerical values expected for quantities at some future date, 

the exact outcome is rarely known in advance. 

8B.3 VARIABILITY 

Variability means that different applications or situations produce different numerical 

values when calculating a quantity. Specifying an exact value for a quantity may be difficult 

because the value depends on something else. For example, the number of hours a transformer is 

run at a particular load depends on the specific circumstances and behaviors of the building 

owners or utilities (e.g., square footage served, hours of operation). Variability makes specifying 

an appropriate population value more difficult, since any one value may not be representative of 

the entire population. In such cases, surveys can be helpful, and analysis of surveys can relate the 

variable of interest (e.g., hours of use) to other variables that are better known or easier to 

forecast (e.g., building square footage). 

8B-1 

 EE-RM/STD-00-550 COMMENT 196
8B.4 APPROACHES TO MODELING UNCERTAINTY AND VARIABILITY 

This section describes two approaches to modeling uncertainty and variability: 

C scenario analysis, and 

C probability analysis. 

Scenario analysis uses a single numerical value for each quantity in a calculation, then 

changes one (or more) of the numerical values and repeats the calculation. A number of 

calculations are performed that provide some indication of the extent to which the result depends 

on the assumptions. For example, the LCC of a piece of equipment could be calculated for 

energy rates of 2¢, 8¢, and 14¢ per kilowatt-hour (kWh). 

The advantages of scenario analysis are that each calculation is simple, a range of 

estimates is used, and crossover points can be identified. (An example of a crossover point is the 

energy rate above which the LCC is reduced, holding all other inputs constant. That is, the 

crossover point is the lowest energy rate at which an owner achieves savings in operating expense 

that more than compensate for the increased purchase expense.) The disadvantage of scenario 

analysis is that there is no information about the likelihood of each scenario. 

Probability analysis considers the probabilities within a range of values. For quantities 

with variability (e.g., electricity tariffs in different regions), surveys can be used to generate a 

frequency distribution of numerical values (e.g., the number of buildings subject to particular 

tariffs) to estimate the probability of each value. For quantities with uncertainty, statistical or 

subjective measures can provide probabilities (e.g., manufacturing cost to improve energy 

efficiency to some level may be estimated to be $10 ± $3). 

The major disadvantage of the probability approach is that it requires more information, 

namely information about the shapes and magnitudes of the variability and uncertainty of each 

quantity. The advantage of the probability approach is that it provides greater information about 

the outcome of the calculations, i.e., it provides the probability that the outcome will be in a 

particular range. 

Scenario and probability analyses provide some indication of the robustness of a policy 

given the uncertainties and variability. A policy is robust when the impacts are acceptable over a 

wide range of possible conditions. 

8B.5 PROBABILITY ANALYSIS AND THE USE OF CRYSTAL BALL 

To quantify the uncertainty and variability that exist in inputs to the engineering, LCC, and 

PBP analyses, DOE used Microsoft Excel spreadsheets combined with Crystal Ball, a 

commercially available add-in, to conduct probability analyses. The probability analyses used 

Monte Carlo simulation and probability distributions. 

8B-2 

 EE-RM/STD-00-550 COMMENT 196
Simulation refers to any analytical method meant to imitate a real-life system, especially 

when other analyses are too mathematically complex or too difficult to reproduce. Without the 

aid of simulation, a spreadsheet model will only reveal a single outcome, generally the most likely 

or average scenario. Spreadsheet risk analysis uses both a spreadsheet model and simulation to 

automatically analyze the effect of varying inputs on outputs of the modeled system. One type of 

spreadsheet simulation is Monte Carlo simulation, which randomly generates values for uncertain 

variables again and again, to simulate a model. Monte Carlo simulation was named for Monte 

Carlo, Monaco, where the primary attractions are casinos containing games of chance. Games of 

chance such as roulette wheels, dice, and slot machines exhibit random behavior. The random 

behavior in games of chance is similar to how Monte Carlo simulation selects variable values at 

random to simulate a model. When you roll a die, you know that either a 1, 2, 3, 4, 5, or 6 will 

come up, but you do not know which for any particular roll. It is the same with the variables 

(e.g., equipment lifetime, discount rate, installation cost) that have a known range of values but an 

uncertain value for any particular time or event. 

For each uncertain variable (one that has a range of possible values), possible values are 

defined with a probability distribution. The type of distribution selected is based on the conditions 

surrounding that variable. Normal, triangular, and uniform probability distribution types are 

shown in Figures 8B.5.1, 8B.5.2, and 8B.5.3, respectively. 

Figure 8B.5.1 Normal Figure 8B.5.2 Triangular Figure 8B.5.3 Uniform 

Probability Probability Probability 

Distribution Distribution Distribution 

During a simulation, multiple scenarios of a model are calculated by repeatedly sampling 

values from the probability distributions for the uncertain variables and using those values for the 

cell. Crystal Ball simulations can consist of as many trials (or scenarios) as desired—hundreds or 

even thousands. During a single trial, Crystal Ball randomly selects a value from the defined 

possibilities (the range and shape of the probability distribution) for each uncertain variable and 

then recalculates the spreadsheet. 

8B-3 

 EE-RM/STD-00-550 COMMENT 196