A-1

APPENDIX A: APPROACH FOR UNCERTAINTY AND VARIABILITY

A.1 INTRODUCTION

Analysis of an energy efficiency standard involves calculations of impacts, for example the impact

of a standard onconsumerlife-cycle cost (LCC). In order to perform the calculation, the analyst must first:

1) specify the equation or model that will be used; 2) define the quantities in the equation; and 3) provide

numerical values for eachquantity. In the simplest case, the equation is unambiguous (contains all relevant

quantities and no others), each quantity has a single numerical value, and the calculation results in a single

value. However, unambiguity and precision are rarely the case. In almost all casesthe model and/or the

numerical values for each quantity in the model are not completely known (i.e, there is uncertainty) or the

model and/or the numericalvaluesfor eachquantityinthe modeldepend upon other conditions (i.e., there

is variability).

Thorough analysis involves accounting for uncertainty and variability. While the simplest analysis

involves a single numerical value for each quantity in a calculation, arguments can arise about what the

appropriate value is for each quantity. Explicit analysis ofuncertaintyand variabilityisintended to provide

more complete information to the decision process. 

A.2 UNCERTAINTY

When making observations of past events or speculating about the future, imperfect knowledge is

the rule rather than the exception. For example, the energy actually consumed by a particular appliance

type (such as the average U.S. central air conditioner or heat pump) is not directly recorded, but rather

estimated based upon available information. Even direct laboratory measurements have some margin of

error. When estimating numerical values expected for quantities at some future date, the exact outcome

is rarely known in advance.

A.3 VARIABILITY

Variability means that different applications or situations produce different numerical values when

calculating a quantity. Specifying an exact value for a quantity maybe difficult because the value depends

on something else. For example, the number of hours an air conditioner is operated by a household

depends uponthe specific circumstances and behaviorsofthe occupants(e.g.,numberofpersons, personal

habits about how comfortable the personwantsto be, etc.). Variability makes specifying an appropriate

population value more difficult in as much as any one value may not be representative of the entire

population. Surveys can be helpful here, and analysis of surveys can relate the variable of interest (e.g.,

hours of use) to other variables that are better known or easier to forecast (e.g., persons per household).

EE-RM-STD-98-440 COMMENT 241ZZ
A-2

A.4 APPROACHES TO UNCERTAINTY AND VARIABILITY

Two approaches to consider uncertainty and variability are described here: 

C scenario analysis and 

C probability analysis.

Scenario analysis uses a single numericalvalue for eachquantityina calculation, thenchanges one

(or more) of the numerical values and repeats the calculation. A number of calculations are done, which

provide some indication of the extent to whichthe result depends uponthe assumptions. For example, the

life cycle cost of an appliance could be calculated for energy rates of 2, 8, and 14 cents per kWh. 

The advantages ofscenario analysis are that eachcalculationissimple; a range of estimatesis used;

and crossover points can be identified. (An example of a crossover point is the energy rate above which

the life cycle cost isreduced, holding allotherinputs constant. That is, the crossover point isthe energy rate

at whichthe consumer achievessavings inoperating expense that more than compensate forthe increased

purchase expense.) The disadvantage ofscenario analysisisthat there is no informationabout the likelihood

of each scenario.

Probability analysis considers the probabilities within a range of values. For quantities with

variability (e.g., electricity rates in different households), surveys can be used to generate a frequency

distributionofnumericalvalues(e.g., the number of households with electricity rates at particularlevels)to

estimate the probabilityofeachvalue. For quantities with uncertainty, statistical or subjective measures can

be used to provide probabilities (e.g., manufacturing cost to improve energy efficiency to some level may

be estimated to be $10 ± $3). 

The major disadvantage of the probability approach is that it requires more information, namely

information about the shapes and magnitudes of the variability and uncertainty of each quantity. The

advantage ofthe probabilityapproachisthat it provides the greatest information about the outcome of the

calculations, that is, the probability that the outcome will be in any particular range is provided. 

Scenario and probabilityanalysis provide some indicationofthe robustness ofthe policy given the

uncertainties and variability. A policy is robust when the impacts are acceptable over a wide range of

possible conditions.

A.5 PROBABILITY ANALYSIS AND THE USE OF CRYSTAL BALL

To quantify the uncertainty and variability that exist in inputs to the Engineering, Life-Cycle Cost

(LCC), and Payback Period (PBP) analyses, Microsoft Excel spreadsheets combined with Crystal Ball

(a commerciallyavailable add-in) wereusedto conduct probabilityanalyses. The probability analyses used

EE-RM-STD-98-440 COMMENT 241ZZ
A-3

NORMAL

Figure A.1a Normal Probability

Distribution

TRIANGULAR

Figure A.1b Triangular Probability

Distribution

UNIFORM

Figure A.1c Uniform Probability

Distribution

Monte Carlo simulation and probability distributions.

Simulationreferstoany analyticalmethod meant to imitate a real-life system, especiallywhenother

analyses are too mathematically complex or too difficult to reproduce. Without the aid of simulation, a

spreadsheet model will only reveal a single outcome, generally the most likely or average scenario.

Spreadsheet risk analysis uses botha spreadsheet modeland simulationto automatically analyze the effect

of varying inputs on outputs of the modeled system. One type of spreadsheet simulation is Monte Carlo

simulation, which randomly generates values for uncertain variables over and over to simulate a model.

MonteCarlo simulation was namedforMonteCarlo, Monaco, where the primaryattractions are casinos

containing games of chance. Games of chance such as roulette wheels, dice, and slot machines, exhibit

random behavior. The random behavior in games of chance is similar to how Monte Carlo simulation

selects variable values at random to simulate a model. Whenyouroll a die, youknowthat either a 1, 2, 3,

4, 5, or 6 will come up, but you don't know which for any particular roll. It's the same with the variables

that have a known range of values but an uncertain value for any particular time or event (e.g. equipment

lifetime, discount rate, and installation cost). 

For each uncertain variable (one that has a range of possible values), possible values are defined

with a probability distribution. The type of distributionselected is based on the conditions surrounding

that variable. Probability distribution types include: 

During a simulation, multiple scenarios of a model are calculated by repeatedly sampling values

from the probability distributions for the uncertain variables and using those values for the cell. Crystal

Ball simulations can consist of as many trials (or scenarios) as desired – hundreds or even thousands.

During a single trial, CrystalBallrandomly selectsavalue fromthe defined possibilities(the range and shape

of the probability distribution) for each uncertain variable and then recalculates the spreadsheet. 

EE-RM-STD-98-440 COMMENT 241ZZ