NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

Chapter 21:

Residential Lighting Evaluation 

Protocol

The Uniform Methods Project: Methods for 

Determining Energy Efficiency Savings for 

Specific Measures

Created as part of subcontract with period of performance 

September 2011 – December 2014

This supersedes the version originally published in April 2013.

Scott Dimetrosky, Katie Parkinson, 

and Noah Lieb

Apex Analytics, LLC

Boulder, Colorado

NREL Technical Monitor: Charles Kurnik

Subcontract Report

NREL/SR-7A40-63205

February 2015
NREL is a national laboratory of the U.S. Department of Energy

Office of Energy Efficiency & Renewable Energy

Operated by the Alliance for Sustainable Energy, LLC

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Contract No. DE-AC36-08GO28308

National Renewable Energy Laboratory

15013 Denver West Parkway

Golden, CO 80401

303-275-3000 • www.nrel.gov

Chapter 21:

Residential Lighting Evaluation 

Protocol

The Uniform Methods Project: Methods for 

Determining Energy Efficiency Savings for 

Specific Measures

Created as part of subcontract with period of performance 

September 2011 – December 2014

This supersedes the version originally published in April 2013.

Scott Dimetrosky, Katie Parkinson, 

and Noah Lieb

Apex Analytics, LLC

Boulder, Colorado

NREL Technical Monitor: Charles Kurnik

Prepared under Subcontract No. LGJ-1-11965-01

Subcontract Report

NREL/SR-7A40-63205

February 2015
NOTICE

This report was prepared as an account of work sponsored by an agency of the United States 

government. Neither the United States government nor any agency thereof, nor any of their employees, 

makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, 

completeness, or usefulness of any information, apparatus, product, or process disclosed, or represents 

that its use would not infringe privately owned rights. Reference herein to any specific commercial 

product, process, or service by trade name, trademark, manufacturer, or otherwise does not necessarily

constitute or imply its endorsement, recommendation, or favoring by the United States government or any 

agency thereof. The views and opinions of authors expressed herein do not necessarily state or reflect 

those of the United States government or any agency thereof.

This report is available at no cost from the National Renewable Energy

Laboratory (NREL) at www.nrel.gov/publications.

Available electronically at http://www.osti.gov/scitech

Available for a processing fee to U.S. Department of Energy

and its contractors, in paper, from:

U.S. Department of Energy

Office of Scientific and Technical Information

P.O. Box 62

Oak Ridge, TN 37831-0062

phone: 865.576.8401

fax: 865.576.5728

email: mailto:reports@adonis.osti.gov

Available for sale to the public, in paper, from:

U.S. Department of Commerce

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

phone: 800.553.6847

fax: 703.605.6900

email: orders@ntis.fedworld.gov

online ordering: http://www.ntis.gov/help/ordermethods.aspx

Cover Photos: (left to right) photo by Pat Corkery, NREL 16416, photo from SunEdison, NREL 17423, photo by Pat Corkery, 

NREL 16560, photo by Dennis Schroeder, NREL 17613, photo by Dean Armstrong, NREL 17436, photo by Pat Corkery, 

NREL 17721.

NREL prints on paper that contains recycled content.
iii

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Acknowledgments

The chapter authors wish to thank M. Sami Khawaja from Cadmus, Seth Craigo-Snell of 

CLEAResult, and Thomas Ledyard of DNV GL for their thoughtful contributions.
iv

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Acronyms

CFL compact fluorescent lamp

CV coefficient of variation

DEP Duke Energy Progress

EISA Energy Information and Security Act

HOU hours of use

HVAC heating, ventilation, and air conditioning

ISR in-service rate

LED light-emitting diode

TRM Technical Reference Manual

W Watt
v

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Table of Contents

1 Measure Description ............................................................................................................................ 1

2 Application Conditions of Protocol .................................................................................................... 2

3 Savings Calculations ........................................................................................................................... 3

4 Measurement and Verification Plan.................................................................................................... 4

4.1 Number of Measures Sold or Distributed ........................................................................................ 4

4.2 Delta Watts ...................................................................................................................................... 5

4.3 Approaches for Estimating Baseline Wattage ................................................................................. 5

4.4 Recommended Approach................................................................................................................. 8

4.5 Replacement of Efficient Lighting Products With Newer Efficient Lighting Products ................ 11

4.6 Uncertainty About the Baseline and the Need for Ongoing Research........................................... 12

4.7 Annual Operating Hours................................................................................................................ 12

4.8 Peak Coincidence Factor................................................................................................................ 13

4.9 Metered Data Collection Method................................................................................................... 14

4.9.1 Logger Type.................................................................................................................... 14

4.9.2 Length of Metering Period.............................................................................................. 14

4.9.3 Information Collected On Site ........................................................................................ 15

4.9.4 Data Integrity .................................................................................................................. 15

4.9.5 Metering Sample Design................................................................................................. 15

4.10 Using Secondary Data ............................................................................................................. 17

4.10.1 Similarities in Service Territories ................................................................................... 17

4.10.2 Sample Size..................................................................................................................... 17

4.10.3 Length of Metering Period.............................................................................................. 18

4.10.4 Adjustments To Reflect Hours of Use by Room Type ................................................... 18

4.11 Snapback/Rebound or Conservation Effect............................................................................. 18

4.12 In-Service Rate ........................................................................................................................ 18

4.13 Interactive Effects With Heating, Ventilating, and Air Conditioning..................................... 21

5 Other Evaluation Issues..................................................................................................................... 23

5.1 Cross-Customer Class Sales .......................................................................................................... 23

5.2 Cross-Service Area Sales (Leakage).............................................................................................. 24

5.3 Estimating Cross-Customer Class and Cross-Service Area Sales ................................................. 24

6 Program Evaluation Elements........................................................................................................... 26

References................................................................................................................................................. 27

Resources.................................................................................................................................................. 29

List of Figures

Figure 1. Example of manufacturer-rated baseline wattage................................................................... 6

Figure 2. Baseline wattage by lumens, EISA versus ENERGY STAR.................................................... 8

List of Tables

Table 1. Strengths and Limitations of Alternative Delta Watts Estimation Approaches..................... 7

Table 2. Standard Lamp Estimated Baseline Wattage for Lumen Equivalencies .............................. 10

Table 3. Specialty Lamp Estimated Baseline Wattage for Lumen Equivalencies .............................. 10

Table 4. Estimated CFL HOU From Recent Metering Studies .............................................................. 13

Table 5. Example of Calculated CV From a Recent Lighting Metering Study..................................... 16

Table 6. Estimated First-Year ISRs From Recent Evaluations of CFL Upstream Lighting 

Programs............................................................................................................................................. 19

Table 7. Estimated 3-Year ISR Calculations........................................................................................... 21
1

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

1 Measure Description

In recent years, residential lighting has represented a significant share of ratepayer-funded 

energy efficiency electricity savings. Utilities have achieved most of these savings by promoting 

the purchase and installation of compact fluorescent lamps (CFLs), both standard “twister” bulbs 

and specialty CFLs such as reflectors, A-Lamps, globes, and dimmable lights. Some energy

efficiency programs have also promoted ENERGY STAR® lighting fixtures, and recent 

programs have introduced solid-state light-emitting diode (LED) lamps. 

The 2007 Energy Independence and Security Act (EISA) requires that, from 2012 through 2014, 

the energy efficiency of most types of screw-based light bulbs improves by approximately 28%, 

as measured by the efficacy in units of lumens per Watt (lm/W). 

EISA requirements take effect in phases, beginning with 100-W equivalents in 2012, 75-W 

equivalents in 2013, and 60-W and 40-W equivalents in 2014. The legislation also has a 

“backstop” provision ensuring that the previous EISA requirements produce savings equal to or 

greater than an efficiency standard of 45 lm/W. If this goal has not been achieved by 2020,

1

legislation will prohibit the sale of any general service lamp2 that does not meet the minimum 

efficiency standard. 

Since EISA took effect in 2012, many lighting efficiency programs have continued to realize

significant savings, but evaluating these programs has become increasingly complex. One reason 

is that EISA makes it difficult to determine the baseline. In addition to yearly baseline changes 

caused by the time-phased approach, the requirements on manufacturers do not necessarily 

reflect what is available for purchase, or what is in their inventory.

Thus, this evaluation protocol was updated in 2014 to resolve some evaluation uncertainties 

affecting residential lighting incentive programs, including: 

• Sell-through periods of EISA-affected bulbs3

• Questions about EISA exemptions

• How the EISA 2020 backstop provision affects savings estimates.4

This revised version also updates other research topics—such as protocols for estimating hours 

of use (HOU) and in-service rates (ISRs)—based on recent research.

 1 In California, the backstop provision takes effect in 2019. 2 EISA defines a general service lamp as one that (1) is intended for a general service or general illumination 

application (whether incandescent or not); (2) has a medium screw base; and (3) is capable of being operated at a 

voltage at least partially within the range of 110 to 130.

3 Sell-through is the amount of time a retail outlet sells its non-EISA compliant inventory after a phase deadline 

passes.

4 The first version of the UMP Residential Lighting Evaluation Protocol was finalized in April 2013.
2

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

2 Application Conditions of Protocol

Program administrators typically deliver residential lighting measures via these four 

mechanisms:

• Upstream buy-down/markdown. The most common approach to achieve residential 

lighting savings is offering “upstream” incentives to manufacturers (buy-down) or to 

retailers (markdown) that reduce the cost of CFLs and LEDs for consumers. Because this 

delivery mechanism offers the discount at the time of purchase (that is, at the point of 

sale), the customers are not required to complete an application or any paperwork.

• Direct install. Many program administrators who offer residential audit programs 

include the direct installation of CFLs or LEDs at the time of an audit. Most programs 

offer audits at either no cost or at a highly discounted cost to the customer, and there is 

usually no additional cost for the installed bulbs.

• Giveaways. A number of program administrators have provided CFLs free of charge to 

residential customers through the mail, at customer service offices, or at events organized 

by community or religious organizations or by local government agencies. In some 

programs, the CFLs are mailed to customers upon request only. In other programs, the 

CFLs are distributed without prior customer request. The amount of customer 

information collected at the time of giveaway events varies; some program administrators 

require full name and contact information and others require no information. 

• Coupons. Some program administrators have relied on instant (point-of-sale) or mail-in 

coupons as the incentive mechanism for residential lighting products. These coupons 

typically require that customers provide their names and contact information to obtain the 

product at the discounted price or to receive the rebate.

Although this Residential Lighting Evaluation Protocol applies to all these delivery mechanisms, 

the strategies for collecting and analyzing the data necessary to calculate the savings tend to 

vary. This protocol highlights and provides details about the strategies and approaches to data 

collection and analysis.

5

Also, program administrators may need to prioritize their evaluation resources to determine

particular combinations of measures and delivery strategies, based on criteria such as the 

contribution to savings and the assessed uncertainty of those savings estimates. (For example, 

uncertainty can occur with programs that have not been evaluated for a while or that have 

shifting baselines.) 

 5 As discussed in Considering Resource Constraints in the “Introduction” chapter to this UMP report, small utilities 

(as defined under the U.S. Small Business Administration regulations) may face additional constraints in 

undertaking this protocol. Therefore, alternative methodologies should be considered for such utilities.
3

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

3 Savings Calculations6

Evaluators can calculate gross energy first-year energy savings from residential lighting 

measures through a number of algorithms; however, this protocol recommends the following 

general algorithms:

kWhsaved = NUMMEAS * (∆W/1,000) * HRS * ISR * IEe (1)

kWsaved = NUMMEAS * (∆W/1,000) * PCF * ISR * IEe (2)

where:

kWhsaved = first-year electricity energy savings measured in kilowatt-hours

kWsaved = first-year electricity peak demand savings measured in kilowatts

NUMMEAS = number of measures sold or distributed through the program

∆W = delta Watts (baseline wattage minus efficient lighting product wattage)

HRS = annual operating hours

PCF = peak coincidence factor

ISR = in-service rate

IEe = cooling and heating interactive effects

This chapter covers the recommended techniques for estimating each of these parameters, based 

on either primary or secondary data.

 6 As presented in the Introduction, the methods focus on energy savings and do not include other parameter 

assessments such as net-to-gross, peak coincidence factor (or demand savings), incremental cost, or measure life.
4

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4 Measurement and Verification Plan

Evaluators should calculate the savings from residential lighting measures through a mix of 

measured and estimated parameters. This protocol recommends this approach, which is similar to 

Option A of the International Performance Measurement and Verification Protocol, because the 

values for some parameters (such as annual HOU) can be directly measured through metering. 

However, evaluators should estimate others parameters (such as delta Watts for upstream 

lighting programs) through other techniques.

4.1 Number of Measures Sold or Distributed

The administrator (or a third-party implementation contractor) should track the number of 

measures sold or distributed through a program and compile this information in a database that 

contains as much detail as possible about the measures delivered. For example, the detailed 

information for each transaction in an upstream program should include:

• Product shipment dates from manufacturer to retailer, where applicable 

• Detailed product information such as:

o Bulb type (CFL, LED)

o Wattage

o Style and features (twister, reflector, A-Lamp, globe, dimmable)

o Manufacturer and product identifier (UPC or SKU codes)

o Rated lumens

o Date of retail sale, if available.

• Number of products incented (number of packs and number of bulbs per pack)

• Date incentive paid

• Dollar value of incentives paid 

• Location where products were sold (including retailer name, address, city, state, and ZIP 

code)

• Final retail sales price of product, if available

• Company contact information (store manager or corporate contact name and phone 

number)

• Assumptions about any parameters to savings estimates.

For programs using other delivery strategies, administrators should collect similar details. For 

example: 

• An audit program would typically require the numbers and types of products installed, 

the wattage of the replaced bulb and location (room type), the date of installation, and 

contact information. 
5

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

• A giveaway program would typically require at least the customer contact information, 

the quantity and type of product given away, and the detailed product information 

previously listed.

At a minimum, the evaluation should include a basic verification of savings, whereby the 

evaluator (1) sums the detailed transactions and (2) attempts to replicate the calculation of total 

claimed savings for the specific time period, such as a program year or cycle, during which the 

savings were claimed. 

Evaluators should treat discrepancies between the claimed and verified number of measures as 

adjustments to the number of program measures. In other words, if the number of measures 

claimed by a program administrator does not match the detailed tracking data, the evaluator 

should regard the amount recorded in the tracking data as the correct number.

4.2 Delta Watts 

The difference between the wattage of the efficient lighting measure and the wattage of the 

assumed baseline measure is the delta Watts. As noted, administrators should enter the wattage 

of the efficient measure in the program tracking database. 

Where possible―such as with direct install programs―the implementation contractor should 

record the wattage of the particular lamp that the program measure replaces.7 Typically, this is 

done at the time of the audit, when auditors replace the existing measure with the efficient 

measure. However, this is not possible for most program delivery strategies, so evaluators often 

need to estimate baseline wattage. The baseline assumptions need to incorporate the transition to 

EISA standards that began in 2012.

4.3 Approaches for Estimating Baseline Wattage

Recent studies have used these approaches for estimating baseline wattage:

• Self-report. Evaluators use customer surveys conducted after the installation to collect 

information about the wattage that consumers used before installation of the energyefficient lighting.

• In-home inspections to examine the wattage of equivalent fixtures. The 

implementation contractor examines the labeled wattage of bulbs in similar fixtures in 

each home to estimate the wattage the consumer used before the energy-efficient lighting

was installed.

• Multipliers. Evaluators assume the baseline to be a multiple—for example, three or four 

times the wattage—of the efficient measure; thus, the evaluator will use one value (a 

single multiplier) across all program bulbs. 

• Lumen equivalence. EISA standards require that lumen ranges and assumptions about

the equivalent wattage of incandescent lights be specified on all retail lamp packaging 

(see Figure 1).

 7 The baseline lamp typically has a much shorter lifetime than the retrofit lamp and the baseline may shift over the 

life of the retrofit lamp (particularly because of EISA).
6

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

• ENERGY STAR lumen equivalence (manufacturer rating). Most energy-efficient 

lighting products prominently list the replacement wattage assumptions on the box (see 

Figure 1), and ENERGY STAR guidelines require these bulbs to use specific baseline 

wattages based on lumen bins.8 The Energy Labeling Rule9 requires manufacturers to 

include detailed information about lamp output and efficacy as part of the “Lighting 

Facts” label now required. 

Figure 1. Example of manufacturer-rated baseline wattage10

Table 1 lists the strengths and limitations of each of these approaches.

 8 ENERGY STAR Lamps V1.0 requires a standard manufacturer baseline rating scale based on brightness (lumens) 

and bulb shape. Detailed specifications are available online at 

https://www.energystar.gov/products/specs/lamps_specification_version_1_0_pd. 9 Information about this rule is available online at: http://www.business.ftc.gov/documents/bus26-lighting-factsquestions-and-answers-manufacturers. 10 Information about this rule is available online at: http://www.energystar.gov/index.cfm?c=cfls.pr_cfls_lumens.

Equivalent 

wattage
7

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Table 1. Strengths and Limitations of Alternative Delta Watts Estimation Approaches

Approach for 

Estimating 

Baseline 

Wattage

Strengths Limitations Recent Studies 

Using Approach

Estimated 

Incandescent 

to CFL Ratio11

Customer selfreport

Captures customer 

intentions and bin 

shifting.12

Potentially low recall and 

social desirability bias.

Duke Energy 

Residential Lighting 

Program (2010)

4.25

Examining 

equivalent 

fixtures

Actual recording of 

baseline wattage for 

existing measures.

Difficult to identify 

equivalent fixtures and 

high cost to conduct 

statistically 

representative on-site 

study.

California 

Upstream CFL 

Program (2006–

2008)

3.6

Standard 

multipliers

Low effort and low 

cost. Accuracy 

derived from 

empirical program 

data and, perhaps, 

better funded studies.

Determining the 

appropriate multiplier for 

the program is difficult 

without basing it on 

another approach, or 

relying on other studies. 

The resulting estimate 

can be biased depending 

on the distribution of bulb 

type and wattages. 

Mid-Atlantic TRM / 

Vermont Energy 

Investment 

Corporation (2011)

3.95

Ohio TRM / 

Vermont Energy 

Investment 

Corporation (2010)

4.25

ENERGY 

STAR lumen 

equivalence 

(manufacturerrated baseline 

wattage)

Widely available, 

relatively inexpensive 

to implement. Data 

based off of wattage

rating on package, 

which is often 

prominently displayed 

on the product. 

Approach is 

consistent with 

ENERGY STAR v1.0 

specification.

May not match the EISA 

lumen bins or not 

adjusted for EISA (i.e., 

uses legacy bulb 

wattages). 

Wisconsin Focus 

on Energy 

Residential Lighting 

Program (2007)

4.0

EISA lumen 

equivalence

Widely available, 

relatively inexpensive 

to implement. In 

some cases, matches 

the marketed 

baseline wattage or 

matches up with 

EISA standards.

May provide conservative 

estimate in cases where 

marketed baseline 

wattage exceeds rated 

lumen output.

Commonwealth 

Edison PY3 

Residential Lighting 

Program (2012) 

N/A

 11 The incandescent-to-CFL wattage will vary, based on the types of bulbs promoted (for example, standard versus

specialty) and the typical CFL wattage for a particular program. Evaluators sometimes show this ratio as the ratio of 

the delta Watts to CFL. (For example, the Mid-Atlantic Technical Reference Manual (TRM) recommends a delta 

Watts-to-incandescent ratio of 2.95). 12 Bin shifting occurs when consumers do not replace bulbs with the same comparable wattage as the previous bulb

(see Section 3.4).
8

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Further complicating the assessment of baseline wattage is that the lumen equivalency bins for 

EISA legislation do not align with the ENERGY STAR lumen bins. This inconsistency results in 

EISA baselines varying from those noted on bulb packaging (see Figure 2).

Figure 2. Baseline wattage by lumens, EISA versus ENERGY STAR13

4.4 Recommended Approach

Consumers are more likely to purchase bulbs based on the rated baseline wattage rather than on 

the lumens.14 Thus, for direct-install programs, the implementation contractor should collect 

baseline wattage information when the measure is installed. Where baseline information cannot 

be collected, the Residential Lighting Evaluation Protocol recommends using an adjusted 

ENERGY STAR lumen equivalency rating (manufacturer-rated baseline wattage) and then 

adjusting these estimates for the EISA requirements. The protocol recommends this approach 

because the manufacturer-rated baseline wattage for an ENERGY STAR bulb must be based on 

ENERGY STAR lumen categories. This approach incorporates EISA requirements, which are 

based on lumen output. 

 13 EISA bins are provided in the legislation online at http://www.lightopedia.com/_files/eisa/energy-independenceand-security-act-of-2007.pdf and ENERGY STAR bins are provided in the ENERGY STAR Lamp Specifications 

online at: https://www.energystar.gov/products/specs/lamps_specification_version_1_0_pd. 14 Recent studies have shown that consumers are still largely unaware of lumens. For example, a forthcoming study 

from New York found that only 57% of respondents had even heard of the term lumens and, of those, more than

80% could not say how many lumens a 60-W bulb uses.
9

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Recent research indicates that there may be significant “sell through” of existing product during

the phase-in years (Navigant and Apex 2012; The Cadmus Group 2010).

15,16,17 The Residential 

Lighting Evaluation Protocol recommends conducting primary research to determine actual 

market conditions during the phase-in of the EISA requirements and use a staged approach to 

phasing in the requirements. If primary research is not available, as a conservative estimate, the 

Residential Lighting Evaluation Protocol recommends using the new baseline values only after 

each phase-in of the legislation has been in effect for 6 months. (See additional discussion in 

Section 3.6.) 

Alternatively, for studies that have sufficient budget to screen for a statistical sample of recent 

CFL purchasers, evaluators may use the self-report approach to estimate delta Watts (as well as 

other purchase attributes including location and price). The Residential Lighting Evaluation 

Protocol recommends, however, that the customer self-report approach apply these time limits 

(from the time the consumer purchased the bulb):

• A maximum 3-month window for standard spiral CFLs

• Up to 1 year for specialty CFLs and LEDs, as these have far lower purchase incidence 

but represent larger purchase decisions. 

When consumers do not replace bulbs with the same comparable wattage as the previous bulbs, 

this is called bin shifting. For example, a consumer may replace a 60-W bulb with a 75-W 

equivalent. Consumers can bin shift to higher- or lower-than-expected wattages. The self-report 

approach offers the advantage of capturing consumer bin shifting, although there is little 

evidence that consumers bin shift when purchasing CFLs (Navigant et al. 2012).

18

EISA legislation does not apply to all bulb types, so as the first step in determining baseline 

wattage, evaluators need to establish whether a bulb is exempt from EISA requirements. To do 

this, evaluators need to classify bulbs by shape, base type, lumens, and specialty features. 

Commonly used EISA-exempt bulbs include:

• Three-way bulbs

• Globes with ≥ 5-in. diameter or ≤ 749 lumens

• Candelabra base bulbs with ≤ 1049 lumens.

19,20

 15 Navigant and Apex (2012) found that 55 of 70 stores visited had at least one phased-out bulb 6 months after EISA 

implementation date. These noncompliant bulbs were found in all retail channels. 

16 The Cadmus Group and NMR (2010) also found significant quantities of EISA phased-out bulbs after the EISA 

implementation dates. 

17 At the time of this protocol, all the EISA lighting requirements were implemented except the 2020 backstop 

provision. These sell-through periods are applicable for evaluating 2013 and 2014 lighting programs. 18 Navigant et al. (2012) found that only 2.6% of purchased CFLs might have been a different equivalent wattage

than the incandescent bulbs they replaced. 

19 See EISA legislation for the full list of exemptions. 20 Flood and reflector lamps have separate EISA requirements that took effect in July 2012. The flood- and reflectorspecific lm/W requirements should be used as the baseline for any program equivalent lamps.
10

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

The baselines for exempt bulbs should match the manufacturer-rated wattage (Column C in 

Table 2 and Table 3). 

When synchronizing evaluated baselines to those noted on bulb packaging, it is important to be 

aware that the recommended lumen equivalencies differ for standard and specialty bulb shapes, 

in line with ENERGY STAR labeling requirements. Table 2 provides the assumed baseline 

wattage based on lumen range for standard lamps (medium screw-base bulbs that are not globe, 

bullet, candle, flood, reflector, or decorative shaped). Evaluators can use the manufacturerrecommended baseline wattage for bulbs with lumens outside the lumen values shown in the 

table. Baselines in Table 2 apply to twist/spiral and A-Lamp shaped bulbs and incorporate EISA 

phase-in periods through 2014.

Table 2. Standard Lamp Estimated Baseline Wattage for Lumen Equivalencies

Minimum Lumens (a) Maximum Lumens (b)

Incandescent Equivalent Wattage

Baseline

(Exempt Bulbs) (c)

Baseline

(Post-EISA) (d)

2,000 2,600 150 72

1,600 1,999 100 72

1,100 1,599 75 53

800 1,099 60 43

450 799 40 29

310 449 25 25

Table 3 provides the assumed baseline wattage—based on lumen range—for specialty and 

decorative shaped lamps. Evaluators can use manufacturer-recommended baseline wattage for 

bulbs with lumens outside the values shown in Table 3. Specialty lamps are medium screw-base 

bulbs that are globe, bullet, candle, or decorative shaped.21 Baselines in Table 3 incorporate 

EISA requirements. 

Table 3. Specialty Lamp Estimated Baseline Wattage for Lumen Equivalencies

Lumen Bins Incandescent Equivalent Wattage

Decorative Shape (A) Globe Shape (B) Baseline

(Exempt Bulbs) (C)

Baseline

(Post-EISA) (D)

1,100–1,300 150 72

650–1,099 100 72

575–649 75 53

500–699 500-574 60 43

300–499 350–499 40 29

150–299 250–349 25 25

90–149 15 15

70–89 10 10

Evaluators should calculate baseline wattage for each lamp in the tracking database. Therefore, 

an evaluator will calibrate the total estimated delta Watts to the actual type and number of 

measures sold or distributed through the program. 

 21 Bulb shapes that fit into this category are B, BA, C, CA, DC, F, and G lamp shapes.
11

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Bulbs expected to be in use in 2020 and beyond will be affected by the EISA backstop provision 

mentioned in Section 1. The life cycle savings of CFLs, therefore, should either terminate for any 

remaining years in the expected useful life beginning in mid-2020, or be substantially reduced 

after 2020 to account for the backstop provision.22 Similarly, the life cycle savings for LEDs 

should incorporate this upcoming baseline change.

4.5 Replacement of Efficient Lighting Products With Newer Efficient

Lighting Products

The previously discussed methodology assumes that at the time of measure failure, the consumer 

has the choice of installing an energy-efficient lighting product or a standard-efficiency lighting 

product, regardless of what was previously installed. In areas with a long history of CFL 

promotion―and as market penetration increases for CFLs and other high-efficiency lighting 

products―consumers are more likely to be using some fraction of the energy-efficient lighting 

products distributed through programs to replace installed CFLs that fail. 

Two approaches are commonly used to address this issue. To avoid underestimating program 

savings, the Residential Lighting Evaluation Protocol recommends that evaluators apply only 

one of these adjustments:

• Least-efficient baseline approach. To determine gross savings, assume the baseline is 

the least-efficient product available with equivalent lumen output (e.g., EISA-compliant 

halogen), even if the customer previously had a CFL or an LED installed in the socket. 

With this approach, customers installing efficient lamps in sockets previously occupied 

by similarly efficient lamps would be handled under investigation of program attribution. 

(Nexus Market Research, Inc. 2009).23

• Market saturation baseline approach. Revise the baseline wattage assumptions to 

reflect the share of in-kind replacement of efficient lamps. This approach requires that

data be collected on the proportion of high-efficiency lamps distributed through the 

program that are replacing existing CFLs. This approach is an alternative way to capture 

attribution, as customers with CFLs are presumably those who are also most likely to be 

free riders and to replace an efficient bulb with another efficient bulb.

As more efficiency programs promote LEDs, further research will be required to investigate the 

likelihood that energy efficiency-minded consumers are replacing CFLs with LEDs. 

 22 For example, a CFL sold and installed in 2014 that has an 8-year expected useful life could claim savings for the 

first 6 years (through 2019), then zero savings for the final 2 years of the expected useful life. Alternatively, a 

program could claim the minimal savings (because the 45 lm/W standard is close, but not exactly equal to, the 

efficacy of a CFL) for the remaining useful life.

23 This study found that 43% of respondents (24 of 56) stated that the CFLs recently purchased and not installed 

were intended for use to replace incandescent lighting. That is, 57% of the respondents intended to use the stored 

CFLs to replace existing CFLs when they failed. Evaluators used this to discount the delta Watts, if respondents who 

already intend to replace CFLs with CFLs are presumably counted as free riders; however, program attribution 

should already incorporate any necessary adjustments.
12

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4.6 Uncertainty About the Baseline and the Need for Ongoing 

Research 

The recommended Residential Lighting Evaluation Protocol acknowledges continued 

uncertainties around the residential lighting market in the next few years. These uncertainties 

deal with the types and prices of lighting products that will be available on the market. Another 

source of uncertainty is consumer reaction to the requirements and new products, which includes 

behaviors such as product hoarding, bin shifting to different incandescent wattage levels, and 

how quickly retailers sell through their existing product inventories. The uncertainty around 

EISA was further heightened in December 2011 with the passage of the fiscal year 2012 

omnibus spending bill, which included a rider that halted funding for the U.S. Department of 

Energy to enforce the new standards.24 The National Electrical Manufacturers Association, 

representing more than 95% of the U.S. lighting manufacturing industry, issued a press release 

after the bill passed, stating that it did not support the bill. The National Electrical Manufacturers

Association also pointed out that American manufacturers have invested millions of dollars in 

transitioning to energy-efficient lighting, and EISA gave state attorneys general the authority to 

enforce the standards. 

Thus, in cases where actual preprogram measure wattage is not available, the Residential 

Lighting Evaluation Protocol recommends that evaluators continue to adopt the EISA standards 

as the new baseline. However, program administrators who have adequate resources should 

conduct ongoing monitoring and research to determine whether the delta Watts assumptions 

reflect actual market conditions during the phase-in of the EISA requirements and use a lagged 

approach to phasing in the requirements. For example, after conducting shelf stocking studies for 

several Massachusetts program administrators, evaluators implemented a time-dependent, 

shifting baseline (Cadmus and NMR 2013). This approach incorporated data showing that 

consumers in the territory were: (1) purchasing incandescent bulbs subsequent to a 6-month 

phase-in period, and (2) purchasing EISA-compliant halogens before the EISA implementation 

date. Consequently, program administrators with adequate resources should attempt to measure 

the extent of these effects; in the absence of primary research, the 6-month sell-through period 

can be used as a conservative default option. 

4.7 Annual Operating Hours 

HOU represents the estimated hours per year that consumers will use the energy-efficient 

lighting product. Recent studies have shown that the estimated average HOU for CFLs ranges

from a low of 1.5 to a high of 3 hours per day (see Table 4). Myriad factors affect the expected 

number of hours per year that consumers use energy-efficient lighting products, including 

differences in demographics, housing types and vintages, CFL saturation, room type, electricity 

pricing, annual days of sunshine, and even an “urban canyon” effect. Thus, extrapolation of data 

from one region to another has not successfully accounted for these influencing factors 

(Navigant Consulting and Cadmus Group, Inc. 2011).25 If extrapolation must be done (because a 

program was recently launched or because insufficient resources are available to conduct a 

metering study) evaluators may use secondary data from other metering studies (discussed in 

 24 This rider has been extended in subsequent resolutions including the fiscal year 2013 budget appropriations. 25 This study revealed a significant difference in average daily HOU compared to extrapolating the HOU from the 

ANCOVA model developed as part of the evaluation of the 2006–2008 California Upstream Lighting Program.
13

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

greater detail in Section 4.9). Based on these disparate results, this protocol recommends that 

program administrators—either on their own or through collaborations with neighboring 

utilities—collect primary data through a metering study of residential lighting measures. 

Table 4. Estimated CFL HOU From Recent Metering Studies

Region Author

Sample 

Size 

(Homes)

# of Efficient 

Bulbs 

Metered

Estimated 

Average 

Daily HOU

California 

(PG&E, SCE, 

and SDG&E 

service areas)

KEMA, Inc. and The 

Cadmus Group, Inc. 

(2010),

DNV-GL (2014)

≈1,200 N/A

1.9 (2006–

2008 cycle)

1.7 (2010–

2012 cycle)

Georgia 

(Georgia Power 

Company)

Nexant and Apex 

Analytics LLC (2013) 125 594 2.8

Massachusetts, 

Rhode Island, 

Vermont, 

Connecticut

Nexus Market Research, 

Inc. et al. (2009) 157 657 2.8

Massachusetts, 

Rhode Island, 

Connecticut, 

New York

NMR Group and DNVGL (2014) 848 5,730*

2.9 (efficient 

bulbs)

2.7 (all bulbs)

Michigan

Opinion Dynamics and 

the Cadmus Group, Inc. 

(2012)

153 710 2.26

Illinois Navigant Consulting 

(2012) 67 527 2.7

North Carolina 

(Duke Energy 

Progress [DEP])

Navigant Consulting, 

Apex Analytics LLC

(2012)

100 413 2.9

Maryland 

(EmPOWER)

The Cadmus Group, 

Inc., and Navigant 

Consulting (2011)

61 222 3.0

North Carolina, 

South Carolina

TecMarket Works and 

Building Metrics (2011) 34 156

2.5 (North 

Carolina)

2.7 (South 

Carolina)

Ohio

Vermont Energy 

Investment Corporation 

(from Duke Energy)

N/A N/A 2.8

Pacific 

Northwest

Northwest Regional 

Technical Forum, based 

on CA, KEMA, Inc. and 

The Cadmus Group 

(2010)

N/A N/A

1.9 for 

existing 

homes, 1.5 

for new 

homes

4.8 Peak Coincidence Factor

Peak coincidence factor is typically defined as the fraction of the peak demand of a population 

that is in operation at the time of system peak. Thus, it is the ratio of the population’s demand at 

the time of the system peak to its noncoincident peak demand. For residential lighting, it would 
14

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

represent the amount of time in which lights are on divided by the total time in the peak defined 

period (i.e., the percentage of time that lights are on during the peak period). Note that while the 

protocols below focus on HOUe, the same principles all apply to the estimate of the peak 

coincidence factor. For more information on the definition of peak demand, see the UMP 

Chapter: Peak Demand and Time-Differentiated Energy Savings Cross-Cutting Protocols.

4.9 Metered Data Collection Method

Base metering on the following factors and associated guidelines:

• Logger type

• Length of metering period

• Information collected on site

• Data integrity.

4.9.1 Logger Type

This protocol recommends change-of-state loggers over periodic readings for standard bulbs 

because they can capture short intervals and switch rates (the number of times lights are turned 

on and off). For dimmable and three-way bulbs, the protocol recommends using light intensity 

loggers. Current-sensing meters (rather than light-sensing meters) are an effective approach for 

outdoor conditions in which ambient light can potentially inflate the estimated HOU.

4.9.2 Length of Metering Period 

The length of the metering period depends on the focus of and available resources for the study. 

For example:

• If the intent of the study is to measure energy usage without concern for estimating 

summer peak demand (coincidence factor), use a limited metering period. Evaluators can 

limit the metering period to several weeks before and after the equinox (spring or fall). 

The general premise supporting annualizing metering periods shorter than 1 year is that 

the annual average use occurs precisely on the equinox; in fact, the equinox represents 

the annualization equation’s intercept. A 2013 study demonstrated the precision of 

relying on a short period surrounding the equinox relative to using a complete 12 months 

of metering data (Shepherd et al. 2013).

• If the metering study in question is concerned with both energy and demand, conduct 

logging for at least 6 months and capture summer, winter, and at least one shoulder 

season—fall or spring. Ideally, evaluators should install loggers immediately preceding 

either the summer or winter solstice to capture a complete 6 months of data. In this case, 

an annualization adjustment is not required. If the metering period is shorter than 6

months and the meter placement is not coincident with the solstice, annualize the data—
15

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

using techniques such as sinusoidal modeling—to reflect a full year of usage (DVN-GL 

2014).

26

4.9.3 Information Collected On Site

Conduct in-home lighting audits at all homes participating in the metering study. To allow for an 

estimate of saturation of high-efficiency lighting equipment, the auditors should record the 

number and types of high-efficiency lighting products by fixture and room type and conduct a 

full inventory of sockets. Evaluators should collect on-site information specifically related to the 

logger placements that details room type, window orientation, fixture type, notes about possible 

ambient light issues, etc.

4.9.4 Data Integrity

Clean and thoroughly check all metered data for errant and erroneous observations. For example, 

at the moments of installation and removal, clip the downloaded data to eliminate extraneous 

readings. Also, omit data from broken loggers or loggers removed by residents. Also omit data 

from loggers suspected to have metered daylight/ambient light. Finally, examine the data for 

“flicker” (that is, very frequent on/off cycling) and clean the raw data to correct for flicker. 

Evaluators can perform computer programming via R, SAS, or other statistical software that 

allows data from flickering bulbs to effectively remain on for the duration of the flickering event, 

rather than appear to be repeated on/off events. 

4.9.5 Metering Sample Design

Ideally, evaluators should conduct metering for large samples of all major lighting types 

(including incandescent baseline lamps and fixtures); however, in practice, most evaluators do 

not have adequate resources for a scope of this size. Consequently, to optimize the allocation of 

moderate evaluation resources, it is important to target the metering to select lighting measures—

typically CFLs or LEDs—that represent the greatest savings in a residential lighting program. 

(This is especially true for retrospective program savings.)

Where savings are used prospectively, it is important to attempt to meter all lighting types, as 

recent studies have found that efficient bulbs have higher HOU than do average bulb types.27

For measures representing a small percentage of savings (such as LEDs in more recent 

programs), estimate the overall HOU by examining the CFL HOU for similar rooms and fixture 

types.

Given the difficulty of identifying program bulbs in an upstream program, field technicians may 

place loggers on energy-efficient bulbs in a random sample of homes that have installed similar 

measures, even if those measures are not definitely known to be part of a markdown or buydown program. For homes that have many energy-efficient lighting products, evaluators may 

meter a subsample of fixtures, as long as they are selected randomly within the home. For 

 26 Sinusoidal modeling assumes that HOU will vary inversely with hours of daylight over the course of a year. 

Sinusoid modeling shows that: (1) HOU change by season, reflective of changes in the number of daylight hours and 

weather; and (2) these patterns will be consistent year to year, in the pattern of a sine wave. 27 For example, the NMR metering study from New England estimated daily HOU for all bulbs at 2.7 hours/day, but 

3.0 hours/day for efficient bulbs. However, the authors do not believe this difference is due to saturation, but rather 

to a combination of selective installation (i.e., higher use sockets and fixture use) and potentially to snapback. 
16

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

example, if a home has CFLs in 10 fixtures, place meters on three to five randomly selected 

fixtures (DNV-GL 2014).

28 This will minimize the invasiveness in homes that are highly 

saturated with energy-efficient lighting products and be cost effective, enabling metering of a 

larger sample of sockets in an equivalent number of homes. 

Determine the total number of loggers installed based on the desired levels of statistical 

confidence and precision, assuming a coefficient of variation (CV) based on recent studies of 

programs with similar CFL saturation (using the maturity of program as a proxy, if necessary) 

and housing characteristics (The Cadmus Group, Inc. 2010) (Navigant Consulting and Cadmus 

Group, Inc. 2011). Historically, the CV has been assumed (and sometimes reported) as 

approximately 0.5 or 0.6. However, this CV may be considerably too low when accounting for 

the serial correlation of usage (and error) across light circuits within a home. For example, a 

recent lighting HOU study from New England, based on more than 800 homes and 5,700 

loggers, recommends that evaluators use a CV of 1.2 for all rooms combined, with CVs ranging 

from 0.89 to as high as 1.6 by room type (Table 5, NMR Group and DNV-GL 2014). 

Table 5. Example of Calculated CV From a Recent Lighting Metering Study

Room Type CV

Sample Size for 90/10 

Confidence/Precision

(# of Loggers)

Bathroom 1.38 515

Bedroom 1.15 358

Dining Room 1.10 327

Exterior 0.89 214

Kitchen 0.93 233

Living Space 1.04 293

Other 1.60 693

Household 1.20 390

This Residential Lighting Evaluation Protocol recommends that, at a minimum, room type be 

considered as a subsample, because room type is one of the most important determinants of 

HOU. Therefore, the program administrator should work with the program evaluator to establish 

well-defined targets for the total number of room types to meter. Subsampling by room types 

(rather than by home type) allows for a potentially more homogeneous population unit because 

of more consistent usage within room types rather than across homes. 

When calculating the HOU from the meter data collected, the precision estimates should take 

into account the primary sampling unit (household) and other subsample units (room type). Most 

statistical packages used for HOU estimation allow for clustering of the sampling unit 

(household) to account for correlation. 

The confidence and precision of the HOU estimate is not simply a factor of the variance across 

each hour for each logger. Using these units would lead to grossly overestimated precision, if 

based on every hour across the metering period. Furthermore, the evaluator’s calculations should 

not ignore the error inherent in the HOU from an annualization model. Rather, when estimating 

 28 A number of studies, including the evaluation of the California Upstream Lighting Program, provide publicly 

available examples of how to randomly select fixtures for metering.
17

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

the overall HOU, any evaluator’s model or calculation should estimate the annualized HOU for 

each logger across all hours, account for the error from this model, and then use these estimates 

as the starting point for the room-based and household-based averages.

Following the metering effort and the annualization of results, compare the distribution of 

loggers by room type to the actual distribution of energy-efficient lighting products per room 

type, as noted at the time of the audit. Then weight the HOU to reflect the actual distribution of 

lighting products by room type. For example, if 10% of the loggers are installed in kitchen 

fixtures, but the audit data reveal that 15% of all CFLs are installed in kitchens, weight the data 

from the loggers in kitchens up by 1.5 when calculating total HOU. 

It is also important to estimate the HOU by room type because direct-install programs often 

target higher use fixtures and sockets in higher use rooms. If administrators of these programs 

track the room types associated with the installation of efficient lighting products, evaluators can 

then base HOU on room type.

Evaluators should also compare the demographic and household characteristics of the metering 

sample with the characteristics of the total population of homeowners believed to have purchased 

energy-efficient lighting products. (Evaluators can collect this information through telephone 

surveys.) If significant differences appear and there is a large enough sample to support 

reweighting based on such characteristics, evaluators should weight the results to reflect these 

differences.

4.10 Using Secondary Data

Metering is the recommended approach; however, program administrators who are just 

launching a program—or who do not have sufficient resources to conduct a metering study—

may use secondary data from other metering studies. This protocol recommends using the 

following criteria when selecting and using secondary data to estimate HOU:

• Similarities in service territories

• Adequate sample size for reasonable confidence/precision levels

• Length of metering period

• Adjustments to reflect hours of use by room type.

4.10.1 Similarities in Service Territories 

Selecting a similar service territory based on geographic proximity or latitude and as many 

common demographic and household characteristics as possible will increase the likelihood that 

the secondary data will provide a valid, reasonable, and accurate estimate. 

4.10.2 Sample Size

The number of observations varies considerably between studies, so evaluators should compare 

the sample size, standard errors, and precision levels at equivalent confidence levels across 

studies to ensure a selected study has reasonable confidence and precision levels. 
18

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

4.10.3 Length of Metering Period

The protocol recommends conducting studies that capture both winter and summer usage to 

estimate overall annual use.

4.10.4 Adjustments To Reflect Hours of Use by Room Type 

To extrapolate HOU from one region to another, one approach is to calibrate the HOU based on 

the efficient bulb saturation by room type. If possible, weight the HOU by room type from a 

secondary data source by the room type distribution of efficient lighting for the region under 

study.

4.11 Snapback/Rebound or Conservation Effect

Snapback or rebound refers to changes in use patterns that occur after an energy-efficient 

product is installed, resulting in reducing the overall measure savings. For example, when 

residential lighting customers use a CFL for more hours per day than they used the replaced 

incandescent bulb, this constitutes snapback. This behavior change may be due to factors such as 

the cost savings per unit of time from the CFL or a concern that turning CFLs on and off 

shortens their effective useful life (although most consumers are probably unaware of this 

effect). Some customers, however, might have lower HOU after installing a CFL, perhaps

because they also want to reduce energy consumption or are dissatisfied with the quality of light. 

Residential lighting programs do not typically allow metering to be conducted both before and 

after the installation of energy-efficient lighting. However, a recent lighting study in the 

Northeast found that the HOU were higher for sockets with efficient bulbs compared to all 

sockets in the house (NMR Group and DNV-GL 2014). The difference was believed to be due 

to: (1) differential socket selection (households selecting higher use locations for their highefficiency light bulbs); (2) shifting usage (households install an efficient bulb in a socket and 

then begin to use that socket in lieu of sockets containing inefficient bulbs); and/or (3) snapback. 

However, this evaluation did not collect any data to determine which of these three theories is 

correct, or the proportion of the difference between efficient and inefficient HOU that is 

attributable to each type of behavior. Therefore, the Residential Lighting Evaluation Protocol 

recommends researching for snapback/rebound effects in future HOU estimates. 

4.12 In-Service Rate 

The ISR represents the percentage of incented residential lighting products that are ultimately 

installed by program participants. ISRs vary substantially based on the program delivery 

mechanism, but they are particularly important in giveaway or upstream programs where the 

customer is responsible for installation. 

For the upstream programs shown in Table 6, three factors have led to first-year ISRs well below 

100%: (1) the often deeply discounted price; (2) the inclusion of program multipacks; and (3) the 

common practice among consumers of waiting until a bulb burns out before replacing it. 
19

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Table 6. Estimated First-Year ISRs From Recent Evaluations

of CFL Upstream Lighting Programs

Region Author

Percentage of CFLs 

Installed the First 

Year After Purchase*

Connecticut, 

Massachusetts, 

Rhode Island, Vermont

Nexus Market Research Inc., 

et al. (2009) 76%

North and South Carolina Navigant Consulting and 

Apex Analytics (2013) 79%

Illinois Navigant Consulting (2013) 72%/82% 

(standard/specialty)

*Based on program year only, not years subsequent to the program year or several years in a multiyear program 

cycle.

The Residential Lighting Evaluation Protocol recommends that evaluators use the methods 

appropriate to the specific delivery mechanism to estimate ISRs:

• For direct-install programs, conduct verification (such as telephone survey or site visits) 

to assess installation and early removal (that is, removal prior to failure).

• For giveaway or coupon programs, conduct verification when customer contact 

information is available. Also, ask respondents whether the installation location was 

within the relevant service territory, and whether the measure was installed in a home or a 

business. If the installation was in a business, ask about the type of business. 

• If customer information is not available, rely on either secondary data (such as those from 

a similar program where customer information was collected) or on the in-home audit 

approach (described in the next bullet).

• For upstream programs, calculate ISRs through an in-home audit. Because program 

bulbs cannot be easily identified, evaluators can calculate the ISR as the number of 

installed bulbs purchased in a recent 12-month period divided by the total number of 

bulbs purchased in the same 12-month period. If the sample size of homes with bulbs 

purchased in a recent 12-month period is insufficient to provide the necessary levels of 

confidence and precision, apply a long-term ISR using all bulbs, regardless of the time of 

purchase. 

• Although the in-home audit is the recommended approach, evaluators can use a telephone 

survey when program administrators are just launching a program or have insufficient 

resources to conduct an in-home audit. To minimize recall bias, the callers should focus 

questions only on products purchased in a recent 12-month period rather than the period 

covering the long-term ISR. (Respondents are expected to have better recall about the 

percentage of bulbs purchased and installed within the past 12 months compared to the 

percentage of bulbs they have ever purchased and installed.)

Although first-year ISRs for upstream programs are less than 100%, recent studies have 

demonstrated that consumers plan to install most of the incented bulbs; however, they sometimes 
20

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

wait until an existing bulb burns out (Navigant and Itron 2010).

29 As a result, for savings that 

occur in years following the year that the incentive was paid, program administrators have used

the following approaches to account for these bulbs that are subsequently installed:30

• Stagger the timing of savings claims. In this method, all the program expenses are 

claimed during the program year, but the savings (and, therefore, the accompanying 

avoided-cost benefits) are claimed in the years during which the program measures are 

estimated to be installed. This approach more accurately captures the anticipated timing 

and quantity for the realized savings.

• Discount future savings. In this method, all the costs and benefits are claimed during the 

program year, but the savings (in terms of avoided costs, kilowatt-hours, or kilowatts) 

from the expected future installation of stored program bulbs are discounted back to the 

program year using a societal or utility discount rate.31 This method offers the simplicity 

of claiming all costs and benefits during the program year, and thus not having to track 

and claim future installations.

To calculate the installation rate trajectories, the Residential Lighting Evaluation Protocol 

recommends using the findings from Navigant Consulting and Apex Analytics (2013). In 

contrast to previous studies, the DEP study examined actual bulb installations for 1 year, with 

on-site verifications conducted three times over the course of the study and a self-reported light 

bulb storage log updated monthly. 

The DEP study found that 79% of CFLs are installed within 12 months of purchase (Year 1), 

87.7% of CFLs are installed within 24 months of purchase (Year 2), and 93.6% are installed 

within 36 months of purchase (Year 3). Stated another way, an additional 14.6% of CFLs from 

the original total (or approximately 69% of those remaining in storage) were installed within 3

years of purchase. Because the sample size of bulbs that were still in storage more than 3 years 

after purchase was small, the DEP evaluation team did not estimate ISRs for bulbs installed in 

Year 4 or beyond. 

Evaluators can follow this trajectory and calibrate to individual service territories using the 

methodology below.32 As outlined in Table 7, program administrators use their researched value 

for their Year 1 ISR (ISRYR1) and determine the percentage of stored bulbs installed in each of 

the next 2 years: 

 29 For example, the evaluation in this study found that about 90% of customers with CFLs in storage were waiting 

until a working incandescent or CFL burned out before they installed the stored CFLs (Table 3-6). 30 The selection of approach will depend upon the study purpose and regulatory requirements. 31 Energy or demand savings are not normally discounted; however, this approach provides simplicity for 

calculating program benefit/cost ratios and the actual net present value of avoided costs, which often are used for 

cost recovery. For programs that want to bid into capacity markets (for example, PJM), the staggered approach is 

recommended because it more accurately captures the actual timing and cumulatively increasing nature of the 

demand savings. 

32 Because this methodology allows for a customized first year ISR, it is also applicable to LEDs, which might have 

higher first-year ISRs because of higher incremental cost and typically fewer bulbs, on average, per package, 

compared to CFLs.
21

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

• Year 2 installation of stored bulbs is calculated by multiplying the percentage of bulbs in 

storage (1-ISRYR1; or Storage%YR1) by 41%, and adding that to the first year ISR. In 

other words, 41% of the bulbs in storage are assumed to be installed in Year 2.

• Year 3 installation of stored bulbs is calculated by multiplying the percent of bulbs in 

storage (Storage%YR1) by 69%, and adding that to the first year ISR. In other words, an 

additional 28% of bulbs in storage are installed in Year 3.

Table 7. Estimated 3-Year ISR Calculations

Year

Incremental % 

Installed (of Original 

Bulbs in Storage)

Total % Installed

(of Original 

Bulbs in Storage)

ISR Calculation Example 

ISR*

Year 1 NA NA Researched Value 75%

Year 2 41% 41% (Storage%YR1 * 

41%)+ISR YR1 85%

Year 3 28% 69% (Storage%YR1 * 

69%)+ISR YR1 92%

*This rate represents the percentage of bulbs purchased in Year 1 and installed by the end of each following year.

ISRs beyond Year 3 were not calculated as part of the installation rate study, because the sample 

remaining in storage after 3 years was too small to reasonably estimate installations. However, 

the Residential Lighting Evaluation Protocol recognizes that bulbs may continue to be installed 

after 3 years, and another recent study suggested that the ISR might be as high as 97% within

4 years of purchase (DNV-GL 2014). Therefore, the Residential Lighting Evaluation Protocol 

recommends assuming that up to 97% of bulbs in storage are installed within 4 years of 

purchase,33 and using the following ISR for Year 4:

ISRYear4 = 97% - ISRYear3

As noted in the delta Watts discussion, this methodology does not adjust for CFL-to-CFL 

replacement, which will likely be handled during assessments of program attribution.

4.13 Interactive Effects With Heating, Ventilating, and Air 

Conditioning

CFLs and LED lamps emit less waste heat than incandescent bulbs, which affects heating, 

ventilating, and air conditioning (HVAC) energy requirements. These effects vary based on 

space conditioning mode, saturation of space heating and cooling technologies and their relative 

efficiencies, and climate zones. The influence of climate zone on interactive effects depends on a 

variety of house-specific factors. 

Taking all these factors into account, the net impact on lighting energy cost savings could be 

positive, negative, or neutral (Parekh et al. 2005; Parekh 2008). In cooling-dominated climates, 

the interactive effects are positive, resulting in additional savings from decreased cooling load. 

However, in heating-dominated climates, the interactive effects are negative, with decreased 

savings from increased heating load. 

 33 If the ISR already meets or exceeds 97% after 3 years, however, additional installations should be claimed for 

Year 4.
22

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Because of the potential impacts of interactive effects, the Residential Lighting Evaluation 

Protocol recommends including these effects in evaluations of residential lighting programs.34

One common approach is to estimate these effects with simulation models, examining a mix of 

typical housing types (such as different vintages) and reflecting the estimated saturation, fuel 

shares, and size/efficiency of HVAC equipment. (That is, the percentage of homes that have air

conditioning or electric versus gas heat.) If necessary, use secondary sources—such as the 

Residential Energy Consumption Study—to estimate these inputs. Brunner et al. (2010) used 

another approach that entailed conducting a billing analysis.

Some regions have developed interactive effects calculators based on such simulations (for 

example, in California, the Database for Energy Efficiency Resources35 and the Regional 

Technical Forum in the Northwest). Such regional collaboration can minimize the cost of 

determining the interactive effects for regions that do not already have such a tool. 

If regional collaboration is not an option and the program administrator does not have the 

resources to complete the simulations, the Residential Lighting Evaluation Protocol recommends 

using a value from an existing resource. It recommends that the value used reflects key 

similarities between the program administrator’s territory and the territory from which the data 

are taken. At a minimum, these key similarities should be the climate (heating and cooling 

degree days and, ideally, the latitude), HVAC system types, HVAC fuel types, and HVAC 

system saturations.

 34 Interactive effects are relevant for bulbs installed in conditioned spaces only. Thus, exterior lights will not have 

HVAC interactive effects.) 

35 www.deeresources.com/DEER2011/download/LightingHVACInteractiveEffects_13Dec2011.xls.
23

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

5 Other Evaluation Issues

The incentive structure of upstream lighting programs does not inherently allow for assurances 

that each purchaser of a program bulb is a residential customer in the sponsoring program 

administrator’s service territory. Therefore, some program bulbs may be purchased by 

nonresidential customers or customers served by other utilities. This section discusses these 

parameters.

5.1 Cross-Customer Class Sales

Nonresidential customers typically use lighting products for more hours per day than do 

residential customers. Typically, nonresidential customers also have higher peak coincidence 

factors. Therefore, the lighting products that are incentivized through a residential lighting 

program but installed in nonresidential sockets may lead to higher savings than those assumed 

through the previously discussed methods. 

Evaluators estimate this parameter via customer intercept surveys, where, at the time of sale, 

customers who purchase lighting products participate in a short survey about intended 

installation location and facility type. Evaluators also estimate this parameter through surveys 

with store managers (asking them to estimate the percentage of bulbs sold to nonresidential 

customers), residential customers (asking them if they purchased discounted lighting products 

and installed them in businesses), or with the owners of small businesses (asking them where 

they typically purchase lighting products). 

The Residential Lighting Evaluation Protocol recognizes these key limitations in estimating this 

parameter: 

• Customer intercepts may not represent all program sales. Conducting customer 

intercept surveys can be expensive, and they are typically conducted only in high-volume 

stores (such as Home Depot, Lowe’s, and Walmart). In some cases, these surveys are 

conducted only during high-volume promotions. Also, because some retailers refuse to 

allow the surveys on their premises, the surveys conducted for a program may not be 

representative of total sales. 

• Accuracy from intercepts is further challenged because business owners and contractors: 

(1) may be a minority of purchasers; (2) may purchase more units per visit than 

residential purchasers; and (3) may not purchase during the same times as the average 

residential purchaser.

• Surveys lack high reliability. Store managers usually do not have detailed information 

about program bulb purchasers, so their estimates of sales to nonresidential customers 

may be unreliable. There are also challenges when surveying small business customers, 

such as nonresponse bias (that is, calling a small business and not getting cooperation 

from the business decision maker to take a survey). And recall bias among survey 

participants may make quantifying the number and type of bulbs purchased by this 

channel difficult. 
24

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

5.2 Cross-Service Area Sales (Leakage)

Recent studies have also attempted to estimate the number of program bulbs sold to customers 

outside the program administrator’s service territory. This is commonly referred to as leakage or 

spillage. 

The most common approaches to determining leakage have been clearly delineated in 

Arkansas,

36 cited here in order of preference.

• Customer intercept surveys. This is the preferred method of primary data collection for 

actual participants, although it can be very difficult to receive permission from 

participating retailers. The sampling strategy used should attempt a random mix of 

entities (geographic, retailer, day of week, and avoiding promotional events only).

• Geo-mapping with general population surveys. This method involves modeling 

leakage scores based on the geographic proximity of participating retailers to sponsoring 

utility customers relative to other utility customers (nonsponsoring). Refine the model by 

using general population telephone surveys to confirm purchasing behavior for 

sponsoring and nonsponsoring utilities in the region.

• Opt-in surveys. This involves including a label or note with each incented product 

among all participating retailers with instructions about how to participate in survey.

(Ideally, the survey should be multimodal: reply card, online, and phone number.) Low 

response rates and nonresponse bias are drawbacks.

This protocol recognizes these key limitations in estimating leakage:

• Cross-region sales. Many neighboring service territories are now targeted by residential 

lighting programs; thus, there is a lesser incentive to shop outside one’s own service 

territory to purchase less expensive lighting products. In some cases, program bulbs cross

over in both directions across service boundaries, which may offset the effect in either or 

both territories. 

• Many programs now limit the number of participating retailers, so leakage is 

minimized. Many program administrators now require retailers participating in upstream 

programs to be located far enough within the service territory or to be surrounded by a 

certain percentage of program customers to minimize potential leakage.

5.3 Estimating Cross-Customer Class and Cross-Service Area Sales 

In addition to the limitations presented above, these parameters may also at least partially offset 

each other. (That is, the increased savings of sales to nonresidential customers may be at least 

partially offset by leakage.)37 Given this, it is reasonable to exclude these parameter estimates 

 36 Arkansas 2013 TRM, Protocol K: Leakage. 37 These protocols do not imply that these effects will be exactly offsetting, only that they work in opposite 

directions: sales to nonresidential customers will typically lead to greater savings, and cross-service area sales will 

lead to lower savings in the sponsor’s service territory. Note also that the longer HOU for commercial installations 

may, in fact, more than offset reduced savings from leakage. For example, if nonresidential HOU were shown to be 

four times the residential HOU in a given jurisdiction, a rate of 5% nonresidential installations would have an 

amplified effect of generating close to 20% of the overall energy savings for the program.
25

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

from impact evaluations of upstream residential lighting programs. In addition, given the 

opposing directions of these parameters, either both—or none—of these parameters should be 

incorporated. Thus, do not claim increased savings from sales to business customers without also 

adjusting for leakage, and do not decrement program savings from leakage without also 

incorporating sales to business customers.38

 38 Exceptions can be made in cases where program administrators are surrounded by other service territories offering 

similar programs. In these cases, sales to business customers can be claimed without reducing sales from leakage. 

An example of this is in Pennsylvania where the Phase II Evaluation Framework recommends that evaluation 

contractors assume that leakages into and out of each utility territory effectively offset each other because they offer 

the same or similar upstream lighting programs.
26

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

6 Program Evaluation Elements

Residential lighting programs offer a range of measures through multiple delivery strategies; the 

upstream CFL program is currently the most ubiquitous. Program administrators who offer a 

variety of measures and rely on multiple delivery strategies may need to prioritize their 

evaluation resources based on criteria such as contribution to savings and assessed uncertainty. 

Evaluators should assess savings through a mix of primary and secondary data, using 

International Performance Measurement and Verification Protocol Option A (Retrofit Isolation: 

Key Parameter Estimates). 

A key area that needs additional research involves the assumptions about baseline wattage as 

EISA standards take effect and as LEDs become a larger source of program savings. For 

example, in absence of the program: What percentage of customers would have installed a CFL, 

rather than a program-incented LED?

The recent lighting HOU study from the Northeast found that HOU were higher for sockets with 

efficient bulbs compared to all sockets in the house (NMR Group 2014). More research is needed 

to determine if this difference is due to differential socket selection, shifting usage, snapback, or 

some combination of these three factors. 
27

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

References

Brunner, E.J.; Ford, P.S.; McNulty, M.A.; Thayer, M.A. (2010).”Compact Fluorescent Lighting 

and Residential Natural Gas Consumption: Testing for Interactive Effects.” Energy Policy. 

(38:3); pp. 1288–1296. http://www.sciencedirect.com/science/article/pii/S0301421509008313.

Cadmus Group, Inc. (December 2010). Lighting and Appliance Evaluation—PY 2. Prepared for 

Ameren Illinois. 

Cadmus Group, Inc.; NMR Group, Inc. (June 8, 2013). Residential Lighting Shelf Survey and 

Pricing Analysis, Final Report. Prepared for Cape Light Compact, National Grid, NSTAR 

Electric, Unitil, Western Massachusetts Electric Company, and Energy Efficiency Advisory 

Council Consultants.

DNV-GL. (August 4, 2014). California Upstream and Residential Lighting Impact Evaluation: 

Work Order 28 (WO28) Final Report. Prepared for California Public Utilities Commission, 

Energy Division.

KEMA, Inc.; Cadmus Group, Inc. (February 8, 2010). Final Evaluation Report: Upstream 

Lighting Program. Prepared for California Public Utilities Commission, Energy 

Division. http://www.calmac.org/publications/FinalUpstreamLightingEvaluationReport_Vol2_C

ALMAC.pdf. 

KEMA, Inc. (February 25, 2005). CFL Metering Study. Prepared for California’s InvestorOwned Utilities (Pacific Gas & Electric Company [PG&E], Southern California Edison [SCE], 

and San Diego Gas and Electric [SDG&E]).

Navigant Consulting and Apex Analytics, LLC (November 7, 2013). Storage Log Study of CFL 

Installation Rate Trajectory. Prepared for Duke Energy Progress. 

Navigant Consulting; Apex Analytics, LLC (September 5, 2012). Preliminary Shelf-Stocking 

Study Results for the Residential ENERGY STAR Lighting Program. Prepared for Progress 

Energy Carolinas.

Navigant Consulting; Cadmus Group, Inc. (January 15, 2011). EmPOWER Maryland 2010 

Interim Evaluation Report. Prepared for Baltimore Gas and Electric, Potomac Electric Power 

Company, Delmarva Power and Light (DPL), Southern Maryland Electric Cooperative

(SMECO), and Allegheny Power (AP). 

Navigant Consulting; Itron (December 21, 2010). Plan Year 2 Evaluation Report: Residential 

ENERGY STAR Lighting. Prepared for Commonwealth Edison Company. Navigant Consulting, 

Cadmus Group, Inc., and Apex Analytics LLC. (June 27, 2012). EM&V Report for the 2010-

2011 Residential Energy Star Lighting Program. Prepared for Duke/Progress Energy Carolinas

Nexant, Inc.; Apex Analytics LLC (October 4, 2013). Georgia Power Company Residential 

Lighting Saturation and Metering Study. Prepared for Georgia Power Company.
28

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Navigant Consulting; Itron; Opinion Dynamics; Michaels Energy (October 23, 2013). 

Residential Energy Star Lighting PY5 Evaluation Report. Presented to Commonwealth Edison 

Company. http://www.ilsag.info/comed_eval_reports.html. 

Nexus Market Research, Inc.; RLW Analytics, Inc. GDS Associates (January 20, 2009). 

Residential Lighting Markdown Impact Evaluation. Prepared for Markdown and Buydown

Program Sponsors in Connecticut, Massachusetts, Rhode Island, and 

Vermont. http://www.env.state.ma.us/dpu/docs/electric/09-64/12409nstrd2ae.pdf.

NMR Group; DNV GL (May 5, 2014). Northeast Residential Lighting Hours-of-Use Study. 

Prepared for Connecticut Energy Efficiency Board, Cape Light Compact, Massachusetts Energy 

Efficiency Advisory Council, National Grid Massachusetts, National Grid Rhode Island, New 

York State Energy Research and Development Authority.

Opinion Dynamics; The Cadmus Group, Inc. (August 7, 2012). Michigan CFL Hours of Use 

Study.

Parekh, A.; Swinton, M.C.; Szadkowski, F.; Manning, M. (2005). Benchmarking of Energy 

Savings Associated with Energy Efficient Lighting in Houses. National Research Council 

Canada. NRCC-50874. http://nparc.cisti-icist.nrccnrc.gc.ca/npsi/ctrl?action=shwart&index=an&req=20377557.

Parekh, A. (2008).”Do CFLs Save Whole-House Energy?” Home Energy Magazine, 

November/December 2008. pp. 20–22.

Shepherd, B.; Rambo, E.; Busker, M. (2013). Annualization of Results of Residential Lighting 

Meter Data, presented at 2013 International Energy Program Evaluation Conference, August 

2013, Chicago, IL.

TecMarket Works; Building Metrics (February 15, 2011). Duke Energy Residential Smart $aver 

CFL Program in North Carolina and South Carolina: Results of a Process and Impact 

Evaluation.

Vermont Energy Investment Corporation (August 6, 2010). State of Ohio Energy Efficiency 

Technical Reference Manual. Prepared for the Public Utilities Commission of 

Ohio. http://amppartners.org/pdf/TRM_Appendix_E_2011.pdf. 

Vermont Energy Investment Corporation (July 2011). Mid-Atlantic Technical Reference Manual, 

Version 2.0. Facilitated and Managed by Northeast Energy Efficiency 

Partnerships. http://neep.org/uploads/EMV%20Forum/EMV%20Products/A5_Mid_Atlantic_TR

M_V2_FINAL.pdf. 
29

This report is available at no cost from the National Renewable Energy Laboratory (NREL) at www.nrel.gov/publications.

Resources

American Society of Heating, Refrigerating and Air-Conditioning Engineers (ASHRAE) (2000). 

Research Project 1093 Compilation of Diversity Factors and Schedules for Energy and Cooling 

Load Calculations. ASHRAE Research Report.

ASHRAE (2002). Guideline 14-2002: Measurement of Energy and Demand Savings.

ASHRAE (2010). Performance Measurement Protocols for Commercial Buildings. 

ASHRAE. (TBD). Guideline 14-2002R (Revision of Guideline 14, currently in process, 

publication date TBD).

Glacier Consulting (March 6, 2008). Analysis of Delta Watts Values for CFLs Rewarded 

Through the Residential Lighting Program During FY07. Prepared for the State of Wisconsin 

Public Service Commission. http://www.cee1.org/eval/db_pdf/1215.pdf. 

Evaluation Framework for Pennsylvania Act 129 Phase II Energy Efficiency and Conservation 

Programs (May 31, 2013). Prepared by the Statewide Evaluation Team: GDS Associates, 

Nexant, Research Into Action, and Apex Analytics.

PA Consulting Group and NMR Group (April 22, 2010). Focus on Energy Evaluation 2009 CFL 

Savings Analysis.

RLW Analytics, Inc.; Nexus Market Research (April 22, 2005). Extended Residential Logging 

Results. Prepared for National Grid.