# Cape Light Compact

Annual Report on Energy Efficiency Activities in 2007

Submitted to the Massachusetts Department of Public Utilities and the Massachusetts Division of Energy Resources

August 27, 2009

### This page intentionally blank.

# Table of Contents

I. Executive Summary ..................................................................................................... 1
A. Introduction ........................................................................................................ 1 B. Report Organization ........................................................................................... 2 C. Summary of Results ........................................................................................... 4 D. Summary of Results by Sector........................................................................... 4

II. Overview of Evaluation Methodology...................................................................... 12

III. Impacts by BCR Activity......................................................................................... 16
A. Residential........................................................................................................ 16
B. Low-Income ..................................................................................................... 20
C. Commercial & Industrial.................................................................................. 23

Appendices..................................................................................................................... .. 27 Appendix 1. Glossary of Terms and Abbreviations.............................................. 28 Appendix 2. 2007 Evaluation Impact Parameters................................................. 32 Appendix 3. Post Program Savings A ttributed to Selected 2007 Market Transformation Initiatives...................................................................................... 35 Appendix 4. Calculation of Shareholder Incentive............................................... 36 Appendix 5. Summary of 2007 Energy E fficiency Evaluation Reports ............... 37

### 1 

## I. Executive Summary

A. Introduction
Since July 2001 the Cape Light Compact has de livered energy efficiency programs to all member towns on Cape Cod and Martha’s Vineyard. This Annual Report provides detailed information on the Compact’s energy efficiency activities and savings during the course of calendar year 2007.
The Compact’s 2007 EEP was approved when the 2007-2012 Energy Efficiency Plan was approved on December 24, 2007. Using the 2007 funding as approved by the Department, the Compact implemented the following set of efficiency programs in 2007:

• 

### The Residential ENERGY STAR ® 

### New Construction Program , which provides home buyers, home builders, and construction trade allies with technical assistance and financial incentives to increase the efficiency of homes that are newly built or undergo major renovations. Results of this program are shown in the Residential Lost Opportunity row of Table 2 and in Section III. Results of the Low-Income New Construction Program, which provi des low-income housing development agencies, weatherization assistance progr am (“WAP”) providers, and residential construction trade allies with incentives to increase the home energy rating of new low-income housing, are also included.

• 

### The Residential Conservation Services RCS/MassSAVE Program , which provides all interested residential customers with energy savings educa tion, the opportunity for a home energy audit and financial in centives for numerous electric and nonelectric efficiency measures, including financial support to switch electric space heating systems to more efficient systems that use alternative fuels. Results of this program are shown in the Residential Retrof it 1-4 row of Table 2 and in Section III.

• 

### The Residential ENERGY STAR Products and Services Program , which seeks to increase the availability and use of ENERGY STAR qualified lighting and appliances, including: clothes washers, room air conditioners, dehumidifiers and refrigerators. This program is used to implement the Northeast Energy Efficiency Partnership (“NEEP”) initiativ es and other regional market transformation efforts. Results of this program are divided appr opriately between the Residential Lighting and Residential Appliances rows of Table 2 and in Section III.

• 

### The Low-Income Single Family Program , which provides low-income customers in single-family dwellings with assistance in purchasing and in stalling efficient lighting, appliances, and weat herization measures. These services are similar to, but more extensive in ability to leve rage program benefits and offer higher incentives to eligible customers, than in the RCS/MassSAVE program. Results of this program are shown in the Low Inco me Retrofit 1-4 row of Table 3 and in Section III.

• 

### The Low-Income Multi-Family Program , which provides owners and managers of low-income multi-family dwellings with assistance in purchasing and installing

### 2 efficient lighting, appliances and space heat ing measures. Results of this program are shown in the Low Income Retrofit Multifamily row of Table 3 and in Section III.

• 

### The Commercial and Industri al New Construction Program , which provides technical assistance and financial incentiv es to increase the efficiency in the construction, renovation, and/or remode ling of all commerc ial, industrial, government and multi-family housing facili ties. Results of this program are included in the C&I Lost Opportunity row of Table 4 and in Section III.

• 

### The Medium and Large Commercial and Industrial Retrofit Program , which provides technical and financ ial assistance to medium and large commercial and industrial (“C&I”) customers seeking to do discretionary replacements of existing operating equipment and processes in th eir facilities with high-efficiency alternatives. Results of th is program are included in the C&I Large Retrofit row of Table 4 and in Section III.

• 

### The Small Commercial and Industrial Retrofit Program , which provides technical assistance, financial incentiv es and direct installation to small C&I customers to replace existing operating equipment and syst ems with high-efficiency equipment. Results of this program are included in the C&I Small Retrofit row of Table 4 and in Section III.

• 

### The Government Agencies Program , which provides technical assistance and financial incentives 1 

### to all government facilities, including municipal, state and federal facilities. For the purposes of repor ting the results of this program in this Annual Report, in Table 4 and in Section III, the results of efficiency activities with small government customers are included in the C&I Small Retrofit row, while the results of efficiency activities with large government customers are included in the C&I Large Retrofit row. The results of government new construction activities are included in the C&I Lost Opportunity row.

• 

### The Commercial and Industrial Products and Services Program , which seeks to increase the availability and use of more efficient motors, lighting designs, and HVAC systems. This program is used to implement NEEP and other regional market transformation initiatives. The resu lts of this program are included in the C&I Lost Opportunity row of Table 4 and in Section III.



## B. Report Organization
This Executive Summary provides an overvie w of the Compact’s energy efficiency programs’ (referred to as BCR Activities) bene fits and costs. For each sector there are

1 Unlike the Compact’s other C&I Programs, where a customer co-pay is required, the Government program covers the entire cost of energy efficiency se rvices resulting from an aud it up to a cap of $75,000 per project.

### 3 tables summarizing the lifetime energy sa vings, lifetime capacity savings, the nonelectric benefits (NEBs), and the dollar values of the total benefits 2 

### and the total costs.
The savings data are presented in terms of both “preliminary” and “evaluated” data.

• The preliminary data refers to savings estimates that are based on the evaluation

impact factors that were used in th e 2006 Annual Report and the Approved Energy Efficiency Plan: 2007 – 2012 (referred to as the 2007 EEP). 3 

### Using this data allows for the most direct comparison with the estimated savings from the 2007 EEP.

• The evaluated data typically isolates for cha nges to savings results based on

evaluation impact factors from all of th e program evaluations that have been prepared since the EEP was filed. Ho wever, in 2007, the Compact undertook a major initiative to adopt assumptions that were consistent with the other Massachusetts Program Administrators. Ma ny measure lives, free ridership rates, spillover rates, in-service rates, energy savings, loadshapes, demand savings, coincidence factors and non-electric be nefits were updated. As a result, the comparison of the preliminary and evaluate d data does not isolate for changes to savings results based on evaluation impact f actors in this year. The impact factors are a subset of the assumptions used in planning and reporting. The evaluated data do present our best estimate of the effi ciency savings, based on the evaluation information available at this time. Appendi x 2 presents the impact factors that were used to prepare the evaluated results.

Section II of this Annual Report provides a di scussion of the methodology that is used for program monitoring and evalua tion. It presents a brief summary of the types of evaluations that are used, and a descripti on of the methodology for estimated net energy savings. It also includes a list of the evaluation studies that were used to prepare the 2007 evaluated efficiency savings results. These evaluation studies are also used to inform program design and delivery.
Section III of this Annual Report provides more detailed results of the program activities. The tables in this section include info rmation regarding the number of program participants, the annual efficiency savings and non-electric benefits, the benefit-cost ratio of the program, and the savings impacts by type of end-use (lighting, HVAC, motors, refrigeration, hot water, and end-user behavior). This se ction also summarizes recent evaluation report findings where relevant. Finally, the appendices provide more detail regarding the monitoring and evaluation result s and the program savings. Of particular interest in this Annual Repor t, Appendix 3 provides greater detail of program budgets (by category) and savings (by type).

2 The Compact is submitting, consistent with other Program Administrators practice and statewide guidance from the Department of Energy Resources, its benefit-cost ratios for its 2007 energy efficiency programs with additional capacity benefits in th e form of a demand reduction induced price effect (“DRIPE”).

3 D.P.U. 07-47, Petition of Cape Light Compact Seek ing Certification of Energy Efficiency Plan: 2007- 2012 (Stamp-approved December 24, 2007).

### 4 

## C. Summary of Results
Table 1 provides a summary of the program e xpenses and savings. It also presents the percent change between the final evaluated results and (a) the preliminary evaluated results, and (b) the estimates of expenses a nd savings targets in the Compact’s EEP. The values in the “Amount” column are the 2007 resu lts, based on all eval uations available at this time.

(Eval-Pre)/Eval (Eval-Plan)/Eval Prorgram Implementation Expenses $4.909 $ - Millions 0% 2% Total Expenses $5.048 $ - Millions 0% 0% Annual Energy Savings 17.149 GWh -25% 33% Annual Summer Demand Savings 2.409 MW -34% 32% Annual Winter Demand Savings 4.142 MW -29% 40% Lifetime Energy Savings 158.599 GWh -54% 33% Lifetime Demand Savings 29.510 MW-Years -40% 36% Total Resource Cost Test 4.428 Benefit / Cost -40% 35% Performance Incentive - After Taxes - $ - Millions - 0% SAVINGS AND EXPENSES FOR 2007 TABLE 1

Measurement Amount Percent Change Comparison Units

Program implementation expenses include all of the co sts incurred by the Compac t, except for monitoring and verification costs. Total expenses include pr ogram implementation costs, plus monitoring and verification costs, plus customer contributions.



### The Compact’s 2007 program implementation e xpenses were roughly 2% higher than the 2007 budgets in the EEP. However, the total expenses were the same as the 2007 budgets in the EEP, indicating that higher impl ementation costs were offset by slightly lower evaluation expenditures. This was due to economies of scale that were achieved by participation in joint studies with other Program Administrators.
The annual energy savings achieved in 2007 were 33% higher than those estimated in the 2007 EEP. The demand savings achieved in 2007 were also signi ficantly higher than those estimated in the 2007 EEP. In gene ral, the Compact experienced greater participation in the Residential Lighting pr ogram and greater uptake on lighting measures in the Low Income Retrofit Multifamily and C&I Lost Opportunity programs than planned. The benefit-cost ra tio of the 2007 programs in to tal was 4.43. This indicates that the Compact’s programs in total are highly cost-effective, where every $1.00 spent reduces the net cost of electricity by $4.43.



## D. Summary of Results by Sector

1. Residential Programs
Table 2 presents the lifetime energy savings , lifetime capacity savings, and lifetime nonelectric benefits for each of th e residential programs. It also presents the total cumulative benefits and costs, in 2007 present value dolla rs. These total benefits and costs are used to determine whether each program is cost-e ffective, based on th e total resource cost (TRC) test.

### 5 Benefit-Cost Ratio Activit y Preliminary Report Preliminary Report Preliminary Report $-Benefits $-Costs A02a Residential Lost Opportunity 1,255 1,140 761 756 $375,239 $474,428 $704,150 $306,845 A02b Residential HVAC - - - - $0 $0 $0 $0 A03a Residential Retrofit 1-4 54,568 10,560 605 4,522 $753,119 $1,991,9 37 $3,653,397 $1,200,152 A03b Residential Retrofit Multifamily NA NA NA NA NA NA NA NA A03c Residential Load Response NA NA NA NA NA NA NA NA A04a Residential Lighting 39,217 64,573 2,350 4,189 $321,378 $526,704 $6,612,875 $466,145 A04b Residential Appliances 2,233 3,548 640 1,429 $818,544 $1,067,473 $1,620,579 $327,929 Total 97,273 79,821 4,356 10,896 $2,268,280 $4,060,542 $12,591,000 $2,301,072 TABLE 2 SUMMARY OF RESIDENTIAL BCR ACTIVITIES Lifetime MWH Lifetime kW TRC Values Lifetime $ NEB



### Figures 1 through 4 present the same information as Table 2. They indicate that most of the residential energy and capacity savings are obtained from the Residential Retrofit 1-4 and Residential Lighting programs; that most of the non-electric bene fits come from the Residential Retrofit 1-4 and Re sidential Appliances programs; and that all residential programs are cost effective.

FIGURE 1 RESIDENTIAL LIFETIME MWH

- 20,000 40,000 60,000 80,000 100,000 120,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME MWH A04b Residential Appliances

A04a Residential Lighting

A03c Residential Load Response

A03b Residential Retrofit Multifamily
A03a Residential Retrofit 1-4

A02b Residential HVAC

A02a Residential Lost Opportunity

### 6 FIGURE 2 RESIDENTIAL LIFETIME kW

- 2,000 4,000 6,000 8,000 10,000 12,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME kW A04b Residential Appliances

A04a Residential Lighting

A03c Residential Load Response

A03b Residential Retrofit Multifamily
A03a Residential Retrofit 1-4

A02b Residential HVAC

A02a Residential Lost Opportunity

FIGURE 3 RESIDENTIAL LIFETIME $ NEB

$0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 $3,500,000 $4,000,000 $4,500,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME $ NEB A04b Residential Appliances A04a Residential Lighting A03c Residential Load Response A03b Residential Retrofit Multifamily
A03a Residential Retrofit 1-4
A02b Residential HVAC A02a Residential Lost Opportunity

### 7 FIGURE 4 RESIDENTIAL TRC VALUES

$0 $2,000,000 $4,000,000 $6,000,000 $8,000,000 $10,000,000 $12,000,000 $14,000,000

$-Benefits $-Costs

BENEFITS VERSUS COSTS$ TRC VALUE A04b Residential Appliances A04a Residential Lighting A03c Residential Load Response A03b Residential Retrofit Multifamily
A03a Residential Retrofit 1-4
A02b Residential HVAC A02a Residential Lost Opportunity



### 2. Low-Income Programs
Table 3 presents the lifetime energy savings , lifetime capacity savings, and lifetime nonelectric benefits for each of the low-income programs. It also presents the total cumulative benefits and costs, in 2007 presen t value dollars. These total benefits and costs are used to determine whether each pr ogram is cost-effective, based on the total
resource cost test.

Benefit-Cost Ratio Activit y Preliminary Report Preliminary Report Preliminary Report $-Benefits $-Costs B02a Low-Income Lost Opportunity - - - - $0 $0 $0 $0 B03a Low-Income Retrofit 1-4 4,953 4,220 250 282 $779,124 $2,091,049 $2,451,669 $525,065 B03b Low-Income Retrofit Multifamily 3,102 3,486 66 101 $882,299 $1,284,389 $1,561,098 $102,129 Total 8,054 7,705 316 383 $1,661,423 $3,375,439 $4,012,768 $627,195 TABLE 3 SUMMARY OF LOW-INCOME BCR ACTIVITIES Lifetime MWH Lifetime kW TRC Values Lifetime $ NEB



### Figures 5 through 8 present the same informati on graphically as liste d in Table 3. They indicate that Low Income Retrofit 1-4 is contributing a greater amount of energy and capacity savings and non-electric benefits as compared to the Low Income Retrofit Multifamily program and that all programs are cost effective.

### 8 FIGURE 5 LOW-INCOME LIFETIME MWH

- 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME MWH B03b Low-Income Retrofit Multifamily
B03a Low-Income Retrofit 1-4

B02a Low-Income Lost Opportunity

FIGURE 6 LOW-INCOME LIFETIME kW

- 50 100 150 200 250 300 350 400 450

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME kW B03b Low-Income Retrofit Multifamily
B03a Low-Income Retrofit 1-4

B02a Low-Income Lost Opportunity

### 9 FIGURE 7 LOW-INCOME LIFETIME $ NEB

$0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 $3,500,000 $4,000,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME $ NEB B03b Low-Income Retrofit Multifamily
B03a Low-Income Retrofit 1-4

B02a Low-Income Lost Opportunity

FIGURE 8 LOW-INCOME TRC VALUES

$0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 $3,500,000 $4,000,000 $4,500,000

$-Benefits $-Costs

BENEFITS VERSUS COSTS$ TRC VALUE B03b Low-Income Retrofit Multifamily
B03a Low-Income Retrofit 1-4 B02a Low-Income Lost Opportunity



### 3. Commercial & Industrial Programs
Table 4 presents the lifetime energy savings , lifetime capacity savings, and lifetime nonelectric benefits for each of the Commercial & Industrial programs. It also presents the total cumulative benefits and costs, in 2007 pr esent value dollars. These total benefits and costs are used to determine whether each program is cost-effective, based on the total resource cost (TRC) test.

### 10 Benefit-Cost Ratio Activity Preliminary Report Preliminary Report Preliminary Report $-Benefits $-Costs C02a C&I Lost Opportunity 35,916 18,085 8,683 4,769 $8 $18,462 $2,356,411 $455,662 C03a Large C&I Retrofit 47,828 12,999 16,815 3,713 $406 $10,755 $1,751,863 $685,189 C03b Small C&I Retrofit 55,700 39,989 11,183 9,748 $12,861 $134,322 $5,218,426 $1,787,534 Total 139,444 71,073 36,681 18,231 $13,275 $163,539 $9,326,699 $2,928,385 TABLE 4 SUMMARY OF C&I BCR ACTIVITIES Lifetime MWH Lifetime kW TRC Values Lifetime $ NEB



### Figures 9 through 12 present the same information as Table 4. They indicate that most of the Compact’s C&I energy and capacity savings and non-electric benefits are obtained from the Small C&I Retrofit program and th at all programs are cost effective.

FIGURE 9 C&I LIFETIME MWH

- 20,000 40,000 60,000 80,000 100,000 120,000 140,000 160,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME MWH C03b Small C&I Retrofit C03a Large C&I Retrofit C02a C&I Lost Opportunity

### 11 FIGURE 10 C&I LIFETIME kW

- 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME kW C03b Small C&I Retrofit C03a Large C&I Retrofit C02a C&I Lost Opportunity

FIGURE 11 C&I LIFETIME $ NEB

$0 $20,000 $40,000 $60,000 $80,000 $100,000 $120,000 $140,000 $160,000 $180,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME $ NEB

C03b Small C&I Retrofit C03a Large C&I Retrofit C02a C&I Lost Opportunity

### 12 FIGURE 12 C&I TRC VALUES

$0 $1,000,000 $2,000,000 $3,000,000 $4,000,000 $5,000,000 $6,000,000 $7,000,000 $8,000,000 $9,000,000 $10,000,000

$-Benefits $-Costs

BENFITS VERSUS COSTS$ TRC VALUE C03b Small C&I Retrofit C03a Large C&I Retrofit C02a C&I Lost Opportunity



## II. Overview of Evaluation Methodology
Preliminary versus Evaluated Results
As noted above, the savings data in this report are presented in terms of both “preliminary” and “evaluated” data.

• 

### The preliminary data refers to savings estimates that are based on the evaluation impact factors 4 

### that were used in the 2006 A nnual Report and the Approved Energy Efficiency Plan: 2007 – 2012 (referred to as the 2007 EEP). 5 

### Using this data allows for the most direct comparison with the estimated savings from the 2007 EEP.

• 

### The evaluated data typically isolates for cha nges to savings results based on evaluation impact factors from all of th e program evaluations that have been prepared since the EEP was filed. Ho wever, in 2007, the Compact undertook a major initiative to adopt assumptions that were consistent with the other Massachusetts Program Administrators. Ma ny measure lives, free ridership rates, spillover rates, in-service rates, energy savings, loadshapes, demand savings, coincidence factors and non-electric be nefits were updated. As a result, the comparison of the preliminary and evaluate d data does not isolate for changes to savings results based on evaluation impact f actors in this year. The evaluated data

4 Evaluation impact factors include measure lives, free- ridership rates, spillover rates, in-service rates, and realization rates.

5 D.P.U. 07-47, Petition of Cape Light Compact Seek ing Certification of Energy Efficiency Plan: 2007- 2012 (Stamp-approved December 24, 2007).

### 13 does present our best estimate of the effi ciency savings, based on all the evaluation information available at this time. A ppendix 2 presents the impact factors that were used to prepare the evaluated results.

Evaluation Studies Used in Pr eparing 2007 Evaluated Results
Since its inception in July 2001, the Compact has participated in many state-wide and regional monitoring and evaluation studies, al ong with other energy efficiency Program Administrators. The Compact has also conduc ted several evaluation studies specific to its own programs. It is common for energy efficiency program evaluators to update parameters on a multi-year cycle, unless significant program changes warrant more frequent study.
The evaluation studies completed in 2007 that were used to u pdate impact factors or to inform the process of program delivery are li sted below. In 2007 the studies included a mix of process and impact evaluation and ot her research. The executive summaries of these reports are included in Appendix 5.

• Evaluation of the Massachusetts New Homes with ENERGY STAR, Findings and Analysis, April 24 ,2008 by Nexus Market Research, Inc. and Dorothy Conant, Consultant.

• The Massachusetts New Homes with ENERGY STAR PROGRAM, 2007 Progress Report, Final Report, May 30, 2008

• MassSAVE Final Summary QA/QC and Im pact Study Report, April 8, 2008 by RLW Analytics, Inc.

• Residential Lighting Measure Life Study, June 4, 2008, by Nexus Market Research, Inc. and RLW Analytics, Inc.

• Market Progress and Evaluation Repor t (MPER) for the 2007 Massachusetts ENERGY STAR Lighting Program, June 16, 2008, by Nexus Market Research, Inc. and RLW Analytics, Inc.

• Measure Life Report, Residential and Commercial/Industria l Lighting and HVAC Measures, prepared for the New Engla nd State Program Working Group (SPWG), June 2007, by GDS Associates, Inc. 6



### • Coincidence Factor Study, Residential Room Air Conditioners, Prepared for Northeast Energy Efficiency Partners hip New England Evaluation and State Program Working Group, June 23, 2008

• Coincidence Factor Study, Residential and Commercial Industrial Lighting Measures, Prepared for New Engla nd State Program Working Group, Spring 2007, Prepared by RLW Analytics

6 This study was completed and filed in 2007, but results were used to update some measures for the 2007 Annual Report.

### 14 • Multiple Small Business Services Programs Impact Evaluation 2007, Final Report Update, by Summit Blue Consulting, September 2, 2008.

• Freeridership and Spillover Study 2007 by PA Consulting: Cape Light Compact Results, June 23, 2008

• Freeridership and Spillover Study 2007 by PA Consulting: National Grid Results, June 23, 2008

Types of Evaluations
The evaluation of 2007 energy efficiency progr am impacts reflects the Compact’s efforts to apply appropriate methodologies and adjust them for individual program characteristics. The diverse nature of the programs, includi ng the magnitude of preliminary kW and kWh impacts, the numbe r of customers served, and the end uses affected, calls for the adoption of different evaluation approaches. Evaluations of some programs use several methodologies to deve lop overall impact results and provide meaningful feedback on program delivery a nd direction. Some of these methodologies are briefly described below.
Survey-Based Impact Parameter Studies . Survey-based impact parameter studies focus on the analysis of information collected th rough customer surveys. They are generally used to measure free-ridership and spillover. These studies provide timely feedback to program managers as well as input to the impact evaluations.

• In 2007, the Cape Light Compact commissi oned a survey-based study to assess free-ridership and spillover impacts from its Small Commercial and Industrial Retrofit, Medium and large Commercial and Industrial Retrofit and Commercial and Industrial New Construction Program s. This study was conducted in coordination with other utilities and co mmon survey instruments and research methods were used. The results of the study are included in Appendix 5.
Billing Analyses . Billing analyses involve the analys is of billing data, combined in some cases with survey data, to determine imp acts for programs where a large number of participants install similar measures. Since billing data are available for all customers, billing analysis techniques may include repr esentative samples of both participants and non-participants in an evaluation.

• In 2007, the Cape Light Comp act commissioned a billing analysis to assess the realization rate for energy savings from lighting measures in its Small Commercial and Industrial Retrofit program. This study was conducted in coordination with other u tilities and common research methods were used.
Site Specific Measurement Analysis . Impact evaluations for many of the end uses and programs covered in this report rely on engi neering estimates that are based on sitespecific metering and on-site telephone a ssessments of measure performance and persistence.

Process and Market Pr ogress Evaluation Studies . Process evaluations review energy efficiency program design and implemen tation, and recommend modifications to

### 15 program delivery. The scope of these evalua tions includes all aspects of the program including administrative efficiency, the quali ty of service provided, and the databases used for program tracking and reporting. Pro cess evaluations assess the early stages of energy efficiency programs. They specifica lly provide an assessment of (a) whether actual operations resemble the intended program design and operation plan, and (b) whether real-world experience shows that the original program design and implementation plan are appropriate given the existing field conditions.

• In 2007, the Cape Light Compact jointly funded a market progress study of Residential Lighting. This study updated various market tran sformation tracking factors.
Economic Modeling and Analysis Studies. The benefits and cost -effectiveness of energy efficiency programs are based on modeling and an alysis that values energy efficiency in relation to the avoided costs of energy supply projected ove r the life of the programs and measures installed. Avoided costs are typica lly projected based on forecasting models.
The cost-effectiveness results presented in th is report – both prelim inary and evaluated – are all based on the avoided cost estimates that were used in preparing the 2007 EEP. This approach allows for a more direct co mparison of the economi c results between the 2007 EEP and the 2007 Annual Report. The avoi ded cost estimates used for both of these studies are taken from the followi ng report: Synapse Energy Economics, Inc., Avoided Energy Supply Costs in New England, 2007 Final Report, prepared for the Avoided Energy Supply Component (AESC) Study Group, Revised - January 3, 2008.

Generic Impact Equations
The general form of the impact equation fo r most of the measures installed is:
Net Impacts = Gross Impacts * Realizati on Rate*(1-Free-Ridership + Spillover) * Persistence Factor.

Realization Rates are study-specific parameters, whic h typically compare the energy or demand performance of installed equipment to in itial estimates of performance. They are typically based on engineeri ng or billing analysis.
Free-ridership includes both partial and pure free-ri dership, where such information is available, as required by D.T.E 98-100.
In energy efficiency programs, spillover may occur among both participants and nonparticipants. Both participant and nonpa rticipant spillover were used in the calculation of savings for commer cial and industrial programs, consistent with D.T.E. 98- 100. The nonparticipant spillover impact used in this report is based on the combined results of National Grid and Compact surveys.

Persistence indicates the continued presence of sa vings over time as indicated by followup surveys that confirm the measure remain s installed, and verify it is operating as intended. As defined by the 2005 Measure Life Study, “Savings persistence is the percent change in expected savings due to changed operating hours, changed process

### 16 operation, and/or degradation in equipment efficiency relative to the baseline efficiency option”.

Measure lives are applied to net annual kW and kW h to calculate lifetime kW and kWh. As defined by the 2005 Measure Life Study 7 

### , measure life is “The median number of years that a measure is installed and operati onal. This definition implicitly includes equipment life and measure persistence, but not savings persiste nce....In addition, this definition conforms in letter or in spirit with the definition of measure life used by most national utilities.”

Performance Metrics
As a not-for-profit inter- governmental organization, the Compact does not require shareholder performance incentives, and thus does not need to monito r or track any form of performance metrics.



## III. Impacts by BCR Activity

A. Residential

1. By BCR Activity
Table 5 presents a summary of the number of customers served, the annual savings, the lifetime savings, and the costs incurred for the residential programs. It also presents the benefit-cost ratio, based on the total resource cost test. The costs and benefits used to derive this ratio are the same as those presented in Table 2.

The Residential Retrofit 1-4 and Residential Lighting Programs provide the greatest annual energy and capacity savings. The Reside ntial Lighting program is particularly cost-effective.

Benefit-Cost Ratio BenefitCost

Activity kWh kWh per Cust kW $-NEB MWH kW $-NEB Activity per Cust TRC

A02a Residential Lost Opportunity 146 90,499 620 33.32 $21,383 1,140 756 $474,428 $306,845 $2,102 2.29 A02b Residential HVAC - - NA - $0 - - $0 $0 NA NA A03a Residential Retrofit 1-4 1,633 1,087,932 666 246.07 $98,381 10,560 4,522 $1,991,937 $1,200,152 $735 3.04 A03b Residential Retrofit Multifamily NA NA NA NA NA NA NA NA NA NA NA A03c Residential Load Response NA NA NA NA NA NA NA NA NA NA NA A04a Residential Lighting 4,551 9,972,772 2,191 645.57 $83,477 64,573 4,189 $526,704 $466,145 $102 14.19 A04b Residential Appliances 2,115 273,731 129 131.25 $76,257 3,548 1,429 $1,067,473 $327,929 $155 4.94 Total 8,445 11,424,934 1,353 1,056.21 $279,497 79,821 10,896 $4,060,542 $2,301,072 $272 5.47 TABLE 5 IMPACT BY RESIDENTIAL BCR ACTIVITIES

Annual Lifetime CostParticipant

7 Measure Life Study Report prepared for the Massachusetts Join t Utilities by Energy Resource Solutions (ERS), October 10, 2005.

### 17 2. By End Use
Table 6 presents a summary of the lifetime energy savings, capacity savings, and nonelectric benefits, by the different end-uses addressed in the residential programs. Lighting and HVAC provide the majority of energy savings from the residential programs. Many of the residential non-electric benefits are from hot water savings, as a result of the saved water from ENERGY STAR clotheswashers.

End Use Preliminary Report Preliminary Report Preliminary Report Lighting 46,383 69,355 2,779 4,502 $361,920 $562,732 HVAC 49,555 7,949 1,008 5,026 $1,652,177 $3,540,605 Refrigeration 537 403 72 35 $0 $0 Hot Water 798 2,114 496 1,333 $552,574 $701,868 Motors 0 0 0 0 $0 $0 End User Behavior NA NA NA NA NA NA Total 97,273 79,821 4,356 10,896 $2,566,671 $4,805,206 TABLE 6 IMPACT BY RESIDENTIAL END-USES Lifetime MWH Lifetime kW Lifetime $ NEB



### Figures 13 through 15 present the same information as Table 6.

FIGURE 13 RESIDENTIAL LIFETIME MWH - END-USE

0 20,000 40,000 60,000 80,000 100,000 120,000

Preliminary Report

Lifetime MWH

PRELIMINARY VERSUS EVALUATEDLIFETIME MWH End User Behavior

Motors

Hot Water Refrigeration

HVAC Lighting

### 18 FIGURE 14 RESIDENTIAL LIFETIME kW - END-USE

0 2,000 4,000 6,000 8,000 10,000 12,000

Preliminary Report

Lifetime kW

PRELIMINARY VERSUS EVALUATEDLIFETIME kW End User Behavior

Motors

Hot Water Refrigeration

HVAC Lighting

FIGURE 15 RESIDENTIAL LIFETIME $ NEB - END- USE

$0 $1,000,000 $2,000,000 $3,000,000 $4,000,000 $5,000,000 $6,000,000

Preliminary Report
Lifetime $ NEB

BENEFITS VERSUS COSTSLIFETIME $ NEB End User Behavior

Motors

Hot Water Refrigeration
HVAC Lighting

### 19 3. Program Evaluation

The Residential ENERGY STAR ® New Construction Program 

### In 2008, the Joint Management Committee (JMC) completed an evaluation report 8 

### on evaluation work conducted in 2007 . The evaluation included:

• A process evaluation focusing on the transition to a new implementation contractor and program cha nges introduced in 2007;

• Interviews with implementation contractor staff, HERS raters, Joint Management Committee members, ENERGY STAR bu ilders, and program managers; and,

• An estimation of the incremental costs required to reach ENERGY STAR certification.

The Residential Lighting Program 

### In 2008, a process and impact evaluation was conducted on the ENERGY STAR Lighting program. 9 

### This study updated numerous market transformation tracking factors such as consumer awareness, satisfaction, use and purchases of compact fluorescent lights and energy-saving fixtur es and CFL sales per household in Massachusetts and the United States. In addition, the study:

• Surveyed marketing and stocking practices as well as salesperson training practices

• Assessed manufacturers’ marketing, product development and sales

• Documented net-to-gross analysis, ma rket penetration of ENERGY STAR lighting products, and

• Reviewed the data collec tion process used to track activity in cooperative promotions.
Also in 2008, a measure life study 10 

### provided updated estimates of measure lives for compact fluorescent bulbs and exterior fixtures.

The Residential ENERGY STAR Products and Services Program



### Coincidence factor studies for room air conditioners 11 

### and lighting 12 

### updated coincidence factors to the 2008 ISO-NE definition.

8 Evaluation of the Massachusetts New Homes with ENERGY STAR, Findings and Analysis, April 24, 2008 by Nexus Market Research, Inc. and Dorothy Conant, Consultant.

9 Market Progress and Evaluation Report (MPER) for the 2007 Massachusetts ENERGY STAR Lighting Program, June 16, 2008, by Nexus Market Research, Inc. and RLW Analytics, Inc.

10 Residential Lighting Measure Life Study, June 4, 2008, by Nexus Market Research, Inc. and RLW Analytics, Inc.

### 20 Residential Conservation Services/MassSAVE 

### In 2008, the Compact co-sponsored a study 13 

### in which a series of evaluation activities for the MassSAVE Program were conducted, incl uding evaluation of th e performance of onsite QA/QC work, measure level impact analys is and a natural gas billing analysis.
Findings and recommendations from the study include:

• Sponsors should consider using a single tool consistently among all vendors to calculate savings;

• Sponsors should consider developing a singl e database accessible to all vendors to track activity consistently across the st ate and remove error between DOER and vendor-related program tracking;

• An ongoing QA process should be initiated to ensure that the quality of installation is sustained; and,

• Vendors should install additional weatheri zation measures, CFLs and/or domestic hot water measures to take advant age of all opportunities in a home.



## B. Low-Income

1. By BCR Activity
Table 7 presents a summary of the number of customers served, the annual savings, the lifetime savings, and the costs incurred for the low-income programs. It also presents the benefit-cost ratio, based on the total resource cost test. The costs and benefits used to derive this ratio are the same as those presented in Table 3.
The Low Income Retrofit 1-4 Program contri butes greater annual energy and capacity savings due to the fact that th ere are a greater number of pa rticipants in this program.

Benefit-Cost Ratio BenefitCost

Activity kWh kWh per Cust kW $-NEB MWH kW $-NEB Activity per Cust TRC

B02a Low-Income Lost Opportunity - - NA - $0 - - $0 $0 NA NA B03a Low-Income Retrofit 1-4 442 336,520 761 31.13 $139,547 4,220 282 $2,091,049 $525,065 $1,188 4.67 B03b Low-Income Retrofit Multifamily 209 236,515 1,132 6.59 $103,462 3,486 101 $1,358,119 $102,129 $489 16.01 TOTAL 651 573,035 880 37.72 $243,009 7,705 383 $3,449,168 $627,195 $963 6.52 TABLE 7 IMPACT BY LOW-INCOME BCR ACTIVITIES

Annual Lifetime CostParticipant

11 Coincidence Factor Study, Resi dential Room Air Conditioners, Prepared for: Northeast Energy Efficiency Partnership New England Evaluation and State Program Working Group, June 23, 2008.

12 Coincidence Factor Study, Residential and Commerc ial Industrial Lighting Measures, Prepared for New England State Program Working Group, Spri ng 2007, Prepared by RLW Analytics.

13 MassSAVE Final Summary QA/QC and Impact Study Report, April 8, 2008 by RLW Analytics, Inc.

### 21 2. By End Use
Table 8 presents a summary of the lifetime energy savings, capacity savings, and nonelectric benefits, by the different end-uses a ddressed in the low-income programs. As for residential, most of the energy and demand savings are from the Lighting and HVAC end uses. Most of the low-income non-electric benefits come from the HVAC measures. This is because the home energy audits result in benefits associat ed with (a) improved property values, (b) reduced fire, illness and moving costs, and (c) fossil-fuel savings. All of the low-income programs also have non-electric benefits as a result of reduced usage of the low-income discount rate. Th e low income programs also have non-electric benefits that are experienced by non-low-income residential customers, such as lighting O&M savings and reduced water usage.

Preliminary Report Preliminary Report Preliminary Report Lighting 4,894 4,493 276 292 $112,560 $92,567 HVAC 2,962 3,013 12 54 $1,526,253 $3,311,739 Refrigeration 195 195 27 33 $18,863 $22,472 Hot Water 4 5 1 3 $3,747 $22,390 Motors 0 0 0 0 $0 $0 End User Behavior NA NA NA NA NA NA Total 8,054 7,705 316 383 $1,661,423 $3,449,168 TABLE 8 IMPACT BY LOW-INCOME END-USES Lifetime MWH Lifetime kW Lifetime $ NEB End Use



### Figures 16 through 18 present the same information as Table 8.

FIGURE 16 LOW-INCOME LIFETIME MWH - END-USE

0 1,000 2,000 3,000 4,000 5,000 6,000 7,000 8,000 9,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME MWH End User Behavior

Motors

Hot Water Refrigeration

HVAC Lighting

### 22 FIGURE 17 LOW-INCOME LIFETIME kW - END-USE

0 50 100 150 200 250 300 350 400 450

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME kW End User Behavior

Motors

Hot Water Refrigeration

HVAC Lighting

FIGURE 18 LOW-INCOME LIFETIME $ NEB - END-USE

$0 $500,000 $1,000,000 $1,500,000 $2,000,000 $2,500,000 $3,000,000 $3,500,000 $4,000,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME $ NEB End User Behavior

Motors

Hot Water Refrigeration
HVAC Lighting

### 23 3. Program Evaluation
The Compact conducted no new evaluation activ ities since the 2004 pr ocess evaluation of the low income program and the addition, in 2006, of low-income non-electric benefits (NEBs) to the estimates of low-in come multifamily program impacts.



## C. Commercial & Industrial

1. By BCR Activity
Table 9 presents a summary of the number of customers served, the annual savings, the lifetime savings, and the costs in curred for the commercial & i ndustrial programs. It also presents the benefit-cost ratio, based on th e total resource cost test. The costs and benefits used to derive this ratio are the same as those presented in Table 4.
The Small C&I Retrofit Program contribute s the most annual and lifetime energy and capacity savings and non-electric benefits due to high participati on in this program.

Benefit-Cost Ratio Benefit-Cost

Activity kWh kWh per Customer kW $-NEB MWH kW $-NEB Activity per Customer TRC

C02a C&I Lost Opportunity 29 1,205,495 41,569 317.90 $1,231 18,085 4,769 $18,462 $455,662 $15,712 5.17 C03a Large C&I Retrofit 15 890,949 59,397 244.99 $827 12,999 3,713 $10,755 $685,189 $45,679 2.56 C03b Small C&I Retrofit 290 3,054,330 10,532 752.23 $10,427 39,989 9,748 $134,322 $1,787,534 $6,164 2.92 TOTAL 334 5,150,775 15,421 1,315.12 $12,485 71,073 18,231 $163,539 $2,928,385 $8,768 3.18 TABLE 9 IMPACT BY C&I BCR ACTIVITIES Annual Lifetime CostParticipant



### 2. By End Use
Table 10 presents a summary of the lifetim e energy savings, capacity savings, and nonelectric benefits, by the different end-uses addressed in the co mmercial & industrial
programs. Most of the energy and capacity savings ar e obtained primarily fr om lighting measures. The non-electric benefits in the C&I sector are primarily from reduced O&M costs as a result of efficient light bulbs with longer operating lives.

Preliminary Report Preliminary Report Preliminary Report Lighting 82,624 42,916 20,439 11,124 $8,160 $160,325 HVAC 10,778 3,401 5,233 1,927 $954 $3,215 Motors / Drives 25,496 13,046 8,826 3,576 $501 $0 Refrigeration 20,547 11,710 2,183 1,603 $3,660 $0 Hot Water 0 0 0 0 $0 $0 Compressed Air NA NA NA NA NA NA Process NA NA NA NA NA NA End User Behavior NA NA NA NA NA NA Total 139,444 71,073 36,681 18,231 $13,275 $163,539 TABLE 10 IMPACT BY C&I END-USES Lifetime MWH Lifetime kW Lifetime $ NEB End Use



### Figures 19 through 21 present the same information as Table 10.

### 24 FIGURE 19 C&I LIFETIME MWH - END-USE

0 20,000 40,000 60,000 80,000 100,000 120,000 140,000 160,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME MWH End User Behavior

Process Compressed Air

Hot Water Refrigeration

Motors / Drives

HVAC Lighting

FIGURE 20 C&I LIFETIME kW - END USE

0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME kW End User Behavior

Process Compressed Air

Hot Water Refrigeration

Motors / Drives

HVAC Lighting

### 25 FIGURE 21 C&I LIFETIME $ NEB - END-USE

$0 $20,000 $40,000 $60,000 $80,000 $100,000 $120,000 $140,000 $160,000 $180,000

Preliminary Report

PRELIMINARY VERSUS EVALUATEDLIFETIME $ NEB End User Behavior

Process Compressed Air
Hot Water Refrigeration
Motors / Drives
HVAC Lighting



### 3. Program Evaluation
In 2007 and 2008, the Cape Light Compact co-s ponsored several evaluations pertaining to commercial and industrial programs.

1. The Cape Light Compact comm issioned a billing analysis 14 

### to assess the realization rate for energy savings from lighting measures in its Small Commercial and Industrial Re trofit program. While this study was conducted in coordination with other u tilities and common research methods were used, territory-specific real ization rates were estimated. The realization rate of 1.04 indicates that the engineering estimates of savings from this program provided a good, slightly conservative, estimate of changed consumption.

2. The Cape Light Compact commissioned an update of free-ridership and spillover estimates 15,16 

### for key commercial and industria l retrofit and new construction measures. This study was conducted in coordination with ot her utilities and common research methods were used. Terri tory-specific results were developed. Where appropriate, for some measures and for nonparticipant spillover, where sample sizes from the Cape were extremely small and it was not possible to obtain

14 Multiple Small Business Services Programs Impact Evaluation 2007, Final Report Update, by Summit Blue Consulting, September 2, 2008.

15 Freeridership and Spillover Study 2007 by PA Consulting: Cape Light Compact Results, June 23, 2008.

16 Freeridership and Spillover Study 2007 by PA Co nsulting: National Grid Results, June 23, 2008.

### 26 responses to produce meaningful results, the Cape Light Compact applied results from the National Grid territory.
The key findings were as follows:

• Free-ridership levels for major measur es (e.g. lighting) decreased in large commercial and industrial retrofit and held steady at under 10% in small commercial and industrial retrofit program;

• “Like” participant spillover remained small, decreasing very slightly in these programs; and,

• Non-participant spillover similarly, remained small, decreasing slightly since 2005.

3. In addition, the Cape Light Compact co-sponsored regional research 17 

### to update coincidence factors for commercial and industrial lighting measures to the ISONE definitions of peak coincidence.

17 Coincidence Factor Study, Reside ntial and Commercial Industrial Ligh ting Measures, Prepared for New England State Program Working Group, Sp ring 2007, Prepared by RLW Analytics.

### 27 

## Appendices

### 28 

## Appendix 1. Glossary of Terms and Abbreviations 18



### Annual kWh Reduction Expected net annual energy savi ngs after all impact factors have been taken into consideration.
AMP Appliance Management Program
BBRS Board of Building Regu lations and Standards
CAP Community Action Program
CEE Consortium for Energy Efficiency
CFL Compact Fluorescent Lamps
Coincident Peak Demand Demand for electricity at the time of the Company’s peak demand.
Delta Watts The difference in the wattage between pre-existing or baseline lighting equipment and energy efficient lighting equipment.
Demand The amount of electric energy used by a customer or a piece of equipment at a specific tim e, expressed in kilowatts. Demand Adjustment Factor This factor is a combination of one or more evaluation impact parameters applied to gross demand savings in the calculation of net demand savings. Diversity That characteristic of a variety of electric loads whereby individual maximum demands usually occur at different times. Diversity Factor Percent of savings available atthe time of the Company’s peak demand.
DOE Department of Energy
DOER Massachusetts Division of Energy Resources
D&R D&R International, the contra ctor to DOE and EPA that monitors sales of ENERGY STAR® appliances.
DSM Demand Side Management
DTE Massachusetts Department of Telecommunications and Energy
EFLH Equivalent Full Load Hours

18 Much of this glossary was taken from Massachusetts Electric and Nantucket Electric, 2003 Energy Efficiency Annual Report , submitted to the Massachusetts Department of Telecommunications and Energy, September 2004. In addition to this glossary, a glossa ry completed in March 2009 for the Regional EM&V Forum with additional terms and acronyms is now available at: http://www.neep.org/EMVinfo.html

### 29 Energy Adjustment Factor A factor made up of one or more evaluation impact parameters applied to gross kW h savings in the calculation of net kWh savings.
EPA Environmental Protection Agency
EPACT Energy Policy Act
ENERGY STAR® Brand name for the voluntary energy efficiency labeling initiative sponsored by the U.S. Environmental Protection Agency and Department of Energy.
Free Riders Customers who participate in an energy efficiency program but would have installed the sa me measure(s) on their own if the program had not been available. Free-Ridership Rate The percent of savings attr ibutable to Free Riders.
Gross kW Expected demand reduction based on a comparison of standard or replaced equipment, and equipment installed through an energy efficiency program.
Gross kWh Expected kWh reduction based on a comparison of standard or replaced equipment, and e quipment installed through an energy efficiency program.
GWh Gigawatt-hour – a measure of electricity usage over time equal to 1,000 megawatt-hour s or 1,000,000 kilowatt-hours.
Hours of Use The estimated number of hours per year that a measure operates.
Hours of Use Realization Rate Ratio of actual metered hours of use data to estimated hours of use data.
HP Horsepower
HVAC Heating Ventilation and Air Conditioning Impact Factor Generic term for persistence, realization rates, in-service rates, non-coincid ent connected demand factors, etc., developed during the evaluation of energy efficiency programs and used to calculate net savings.
JMC The Joint Management Committee of utility and non-utility parties that manages the ENERGY STAR® Homes Program.
kWh Kilowatt-hour – The basic unit of electric energy usage over time. One kWh is equal to one kW of power supplied to a circuit for a period of one hour.
kW Kilowatt – A measure of electric demand – 1000 watts
kW – Years See: Lifetime kW
Lifetime The expected length of time, in years, that an installed measure will be in service and producing savings.
Lifetime kW The expected demand savings over the lifetime of an

### 30 installed measure, calculated by multiplying the annual peak kW reduction associated with a measure by the expected lifetime of that measure. It is expressed in units of kWyears. Lifetime MWh The expected energy savings over the lifetime of an installed measure, calculated by multiplying the annual MWh reduction associated with a measure by the expected lifetime of that measure.
LIHEAP Low Income Heating Assistance Program
Maximum Annual kW Savings Peak annual demand savings of a measure. At the program level, this equals the sum of the annual peak demand savings across all measures.
Measure Specific technology or practice that produces energy and/or demand savings for which the company provides financial incentives.
MPER Multi-Year Program Evaluation and Market Progress Reporting, or Market Progre ss and Evaluation Report, developed for various residential programs.
MW Megawatt – a measure of electric demand equal to 1,000 kilowatts.
MWh Megawatt-hour – a measure of energy use over time equal to 1,000 kilowatt-hours.
NATE North American Technician Excellence Program
NEEP Northeast Energy Efficiency Partnerships
O&M Operation and Maintenance Off-Peak energy kWh The kWh reduction that occurs during the Company’s offpeak hours for energy. (Monday-Friday 9 p.m. to 8 a.m. and all day of weekends and holidays) On-Peak Energy kWh The kWh reduction that occurs during the Company’s onpeak hours for energy. (Monday-Friday 8 a.m. to 9 p.m., except holidays)
Persistence Rate Percentage of first year ener gy or demand savings expected to persist over the life of the installed energy efficiency equipment; developed by conducting surveys of installed equipment several years after installation to determine presence and operational capab ility of the equipment.
RCS Residential Conservation Services. Formerly Energy Conservation Services or ECS Seasonal (Winter/Summer) kW The net demand reduction during either the Winter or Summer seasons. Spillover Additional energy efficient equipment installed by customers

### 31 that were influenced by the Company’s sponsored program, but without direct fi nancial or technical assistance from the program. Spillover is separated into Participantand Nonparticipant factors. Non-participating customers may be influenced by product availabi lity, publicity, education, and other factors that are affected by the program. Spillover Rate Estimate of energy savings attributable to spillover effects expressed as a percent of savi ngs installed by participants through an energy efficiency program.
VSD Variable Speed Drive
WAP Weatherization Assistance Program
Watt The basic electrical unit of power.

### 32 

## Appendix 2. 2007 Evaluation Impact Parameters
The table below presents the impact factors that were used to calculate the evaluated savings for the commercial and industrial pr ograms in 2007. Impact parameters for the C&I New Construction, C&I Large Retrofit, C&I Small Retrofit and C&I Products & Services programs were updated based on the evaluation studies conducted by the Compact in 2007 and 2008. Impact factors for Government programs were not evaluated in 2007 or 2008. Impact factors shown below represent the common assumptions developed by Massachusetts pr ogram administrators, based on a review of best available information on measures in statewide programs. Impact factors are a subset of all of the assumptions used in planning and reporti ng. The impact factors in bold were updated based on evaluation studies.
The impact factors in all of the tables in Appendix 2 are not the only assumptions that were updated in 2007, due to the fact that in 2007, the Compact undertook a major initiative to adopt assumptions that were consistent with the other Massachusetts Program Administrators. Many measure lives , free ridership rates, spillover rates, in-service rates, energy savings, loadshapes, demand savings , coincidence factors and non-electric benefits were updated. These tables do not provide a comprehensive overview of all of the assumptions that changed in 2007. Rather , they provide an overview of the impact factors that changed in 2 007 due to evaluation studies.

Table A2.1 Commercial & Industrial Program Evaluation Impact Factors

BCR Activity Program End Use Measure Life Free Ridership

Rate Spillover [Participant] Rate Spillover[NonParticipant] Rate InService

Rate kWh Realization

Rate C02a C&I Lost Opportunity C02a C&I New Construction ALght 15 13.00% 0.00% 2.60% 100% 100% C02a C&I Lost Opportunity C02a C&I New Construction CMoDr 20 100.00% 0.00% 2.60% 100% 100% C02a C&I Lost Opportunity C02b C&I Govt New Construction ALght 15 0.00% 0.00% 2.60% 100% 100% C03a Large C&I Retrofit C03a C&I Large Retrofit ALght 13 35.20% 0.70% 2.60% 100% 100% C03a Large C&I Retrofit C03a C&I Large Retrofit CMoDr 15 19.30% 0.00% 2.60% 100% 100% C03a Large C&I Retrofit C03a C&I Large Retrofit DRefr 13 7.80% 0.40% 2.60% 100% 100% C03a Large C&I Retrofit C03a C&I Large Retrofit BHVAC 23 12.50% 5.20% 2.60% 100% 100% C03a Large C&I Retrofit C03c C&I Govt Large ALght 13 0.60% 3.40% 2.60% 89% 100% C03a Large C&I Retrofit C03c C&I Govt Large CMoDr 15 0.00% 0.00% 2.60% 100% 100% C03a Large C&I Retrofit C03c C&I Govt Large BHVAC 10 0.00% 0.00% 2.60% 100% 100% C03b Small C&I Retrofit C03b C&I Small Retrofit ALght 13 7.70% 0.30% 2.60% 86% 104% C03b Small C&I Retrofit C03b C&I Small Retrofit BHVAC 9 2.10% 25.40% 2.60% 100% 100% C03b Small C&I Retrofit C03b C&I Small Retrofit DRefr 13 7.80% 0.40% 2.60% 100% 100% C03b Small C&I Retrofit C03d C&I Govt Small ALght 13 0.60% 3.40% 2.60% 89% 100% C03b Small C&I Retrofit C03d C&I Govt Small BHVAC 10 0.00% 0.00% 2.60% 100% 100% C03b Small C&I Retrofit C03d C&I Govt Small CMoDr 15 0.00% 0.00% 2.60% 100% 100% C03b Small C&I Retrofit C03d C&I Govt Small DRefr 11 0.00% 0.00% 2.60% 100% 100% C02a C&I Lost Opportunity C04c C&I Products & Services ALght 15 27.90% 13.40% 2.60% 100% 100% C02a C&I Lost Opportunity C04c C&I Products & Services CMoDr 15 28.80% 9.20% 2.60% 100% 100% C02a C&I Lost Opportunity C04c C&I Products & Services BHVAC 15 14.80% 5.90% 2.60% 100% 100% Note: Shaded cells indicate impact factors that are neither 100% for the In-Service Rate or kWh Realization Rate nor 0% for the Free Ridership, Par ticipant Spillover, or Non-Participant Spillover Rates.

### 33 The table below presents the impact factors that were used to calculate the evaluated savings for residential programs offered by the Cape Light Compact in 2007. Impact factors shown below represent common assumptions developed by Massachusetts program administrators, based on a review of best available information on measures in statewide programs. The impact factors in bold were updated based on evaluation studies.

Table A2.2 Residential Program Evaluation Impact Factors

BCR Activity Measure Measure Life Free Ridership

Rate Spillover [Participant] Rate Spillover[NonParticipant] Rate InService

Rate A02a Residential Lost Opportunity CFL 7 2.00% 0.00% 0.00% 90% A02a Residential Lost Opportunity FIXTUREIN 20 8.00% 4.00% 0.00% 95% A02a Residential Lost Opportunity DISHWASHER 12 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity REFRIG 13 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HEATSYSTEM 18 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity AIRSEAL - electric 15 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERS - ES 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERSC - ES 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERSD - ES 15 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERSS - ES 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERS - CP 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERSC - CP 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERSD - CP 15 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity HERSS - CP 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity LIHERS - ES 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity LIHERSC - ES 25 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity LIHERSD - ES 15 0.00% 0.00% 0.00% 100% A02a Residential Lost Opportunity LIHERSS - ES 25 0.00% 0.00% 0.00% 100%

BCR Activity Measure Measure Life Free Ridership

Rate Spillover [Participant] Rate Spillover[NonParticipant] Rate InService

Rate

A03a Residential Retrofit 1-4 BOILRWATER 20 0.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 CFL 7 2.00% 0.00% 0.00% 90% A03a Residential Retrofit 1-4 HOTWATER 7 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 HVAC 17 0.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 INDIRECTDH 20 0.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 SWITCH 20 0.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 TORCHIERE 8 0.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 T-STAT 10 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 AIRSEAL - electric 15 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 AIRSEAL - oil 15 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 AIRSEAL - gas 15 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 AIRSEAL - other 15 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 INSULATION - electric 25 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 INSULATION - other 25 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 INSULATION - gas 25 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 INSULATION - oil 25 2.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 FURNACE 20 0.00% 0.00% 0.00% 100% A03a Residential Retrofit 1-4 REFRIGESV 13 35.00% 36.00% 0.00% 100% A03a Residential Retrofit 1-4 REFRIGRV 1 35.00% 36.00% 0.00% 100%

BCR Activity Measure Measure Life Free Ridership

Rate Spillover [Participant] Rate Spillover[NonParticipant] Rate InService

Rate A04a Residential Lighting CFL 5 0.00% 0.00% 0.00% 165% A04a Residential Lighting CFLNCP 7 0.00% 0.00% 0.00% 165% A04a Residential Lighting FIXTUREIN 20 8.00% 4.00% 0.00% 95% A04a Residential Lighting FIXTUREOUT 6 12.00% 7.00% 0.00% 87% A04a Residential Lighting T ORCHIERE 8 6.00% 3.00% 0.00% 83% A04b Residential Appliances CLOTHESWAS 11 10.00% 0.00% 0.00% 27% A04b Residential Appliances ECMHEAT 20 0.00% 0.00% 0.00% 100% A04b Residential Appliances DEHUMESV 12 0.00% 0.00% 0.00% 100% A04b Residential Appliances DEHUMRV 4 0.00% 0.00% 0.00% 100% A04b Residential Appliances ROOMACESV 11 0.00% 0.00% 0.00% 100% A04b Residential Appliances ROOMACRV 4 0.00% 0.00% 0.00% 100% Note: Shaded cells indicate impact factors that are neither 100% for the In-Service Rate nor 0% for the Free Ridership, Participant Spillover, or Non-Participant Spillover Rates.

### 34 The table below presents the impact factors that were used to calculate the evaluated savings for low income programs offered by the Cape Light Compact in 2007. Impact factors shown below represent the common assumptions developed by Massachusetts program administrators, based on a review of best available information on measures in statewide programs. The Compact’s low in come program impact factors were not updated in 2007 due to evaluation studies.
Table A2.3 Low Income Program Evaluation Impact Factors

BCR Activity Measure Measure Life Free Ridership

Rate Spillover [Participant] Rate Spillover[NonParticipant] Rate InService

Rate

B03a Low-Income Retrofit 1-4 CFL 16 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 HEATSYSTEM 15 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 HOTWATER 7 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 AIRSEAL - electric 15 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 AIRSEAL - oil 15 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 AIRSEAL - gas 15 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 INSULATION - electric 25 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 INSULATION - gas 25 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 INSULATION - oil 25 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 REFRIGESV 13 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 REFRIGRV 1 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 DEHUMESV 12 0.00% 0.00% 0.00% 100% B03a Low-Income Retrofit 1-4 DEHUMRV 4 0.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily CFL 16 0.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily CLOTHESWAS 11 10.00% 0.00% 0.00% 27% B03b Low-Income Retrofit Multifamily HEATSYSTEM 20 0.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily HVAC 15 0.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily T-STAT 5 2.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily AIRSEAL - electric 15 0.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily AIRSEAL - gas 15 0.00% 0.00% 0.00% 100% B03b Low-Income Retrofit Multifamily INSULATION - gas 21 0.00% 0.00% 0.00% 100% Note: Shaded cells indicate impact factors that are neither 100% for the In-Service Rate nor 0% for the Free Ridership, Participant Spillover, or Non-Participant Spillover Rates.

### 35 

## Appendix 3. Post Program Saving s Attributed to Selected 2007 Market Transformation Initiatives
The Compact has not developed estimates of post program savings associated with market transformation initiatives. It is our understanding that this issue has not been considered a high priority for DOER or other Program Administrators.

### 36 

## Appendix 4. Calculation of Shareholder Incentive
The Cape Light Compact does not require shareholder incentives to implement its energy efficiency programs. Therefore, this section is not relevant to the Compact.

### 37 

## Appendix 5. Summary of 2007 Energy Efficiency Evaluation Reports
The following studies were used in prepari ng the evaluated results presented in this Annual Report. The executive summaries of th ese reports are attached below. The full copies of these reports are available from the Compact upon request.

• Evaluation of the Massachusetts New Homes with ENERGY STAR, Findings and Analysis, April 24 ,2008 by Nexus Market Research, Inc. and Dorothy Conant, Consultant.

• The Massachusetts New Homes with ENERGY STAR PROGRAM, 2007 Progress Report, Final Report, May 30, 2008.

• MassSave Final Summary QA/QC and Impact Study Report, April 8, 2008 by RLW Analytics, Inc.

• Market Progress and Evaluation Repor t (MPER) for the 2007 Massachusetts ENERGY STAR Lighting Program, June 16, 2008, by Nexus Market Research, Inc. and RLW Analytics, Inc.

• Residential Lighting Measure Life Study, June 4, 2008, by Nexus Market Research, Inc. and RLW Analytics, Inc.

• Coincidence Factor Study, Residential Room Air Conditioners, Prepared for: Northeast Energy Efficiency Partners hip New England Evaluation and State Program Working Group, June 23, 2008.

• Coincidence Factor Study, Residential and Commercial Industrial Lighting Measures, Prepared for New Engla nd State Program Working Group, Spring 2007, Prepared by RLW Analytics.

• Multiple Small Business Services Programs Impact Evaluation 2007, Final Report Update, by Summit Blue Consulting, September 2, 2008.

• Freeridership and Spillover Study 2007 by PA Consulting: Cape Light Compact Results, June 23, 2008.

• Freeridership and Spillover Study 2007 by PA Consulting: National Grid Results, June 23, 2008.22 Haskell Street, Cambridge, MA 02140 Phone: (617) 497-7544 Fax: (617) 497-7543 www.nexusmarketresearch.com 

# Evaluation of the Massachusetts New Homes with ENERGY STAR Æ



## Findings and Analysis

Executive Summary

April 24, 2008

Submitted to: Joint Management Committee

Submitted by: Nexus Market Research, Inc. Dorothy ConantNexus Market Research 

## Table of Contents Executive Summary .............................................................................................................. ......1 Program Overview ............................................................................................................... ...............1
Conclusions .................................................................................................................... ....................2
Recommendations................................................................................................................ ...............3Evaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 1

Nexus Market Research 

## Executive Summary

This document is a market progress evaluation report (MPER) on the Massachusetts New Homes with ENERGY STAR Æ 

### Program run by the Joint Management Committee (JMC) in Massachusetts. This report contains the findings and analysis of the evaluation work conducted in 2007, with the individual evaluation reports on which the findings and analysis are based as appendices. The evaluation work conducted for the 2007 program includes:

• A process evaluation focusing on issues related to the transition to a new implementation contractor and program changes introduced in 2007. Interviews were conducted with JMC members, non-utility parties (NUPs), implementation contractor personnel and HERS raters. (Process Evaluation, Appendix A)

• In-depth interviews with forty ENERGY STAR builders (Builder Interviews, Appendix B)

• Interviews with the managers of six ENERGY STAR Homes programs using independent HERS raters to certify homes and/or recruit builders with builders paying for HERS raterís services, and four companies providing HERS rating services in Massachusetts (Review of Market-Based HERS Model, Appendix C)

• Estimation of the incremental costs required to reach ENERGY STAR certification and more stringent HERS levels for single and multifamily homes with different heating systems (Incremental Cost Analysis, Appendix D)

In-depth builder interviews and home buyer surveys were previously completed for the 2002, 2003, 2004, and 2006 programs while an incremental cost analysis was previously conducted in 2002. Where appropriate, the findings from the previous year studies are compared.



## Program Overview The Massachusetts New Homes with ENERGY STAR Program certified more than 1,200 housing units in 2007, bringing the total number certified since program inception to more than 13,300. The estimated percentage of new homes completed in Massachusetts that are ENERGY STAR certified climbed from three percent in 1999 to 16% in 2006. In 2007 the penetration of ENERGY STAR homes fell to nine percent. The penetration of single family ENERGY STAR homes remained consistent at seven percent, but the penetration of multifamily ENERGY STAR units dropped sharply because multifamily units in high-rise buildings not built under residential building code are no longer eligible for ENERGY STAR certification.

The total number of housing units certified in 2007 is slightly lower than one-half the number certified in 2006. In addition to the exclusion of units in multifamily buildings not built using residential building code, total building permits issued in Massachusetts in 2007 were 24% lower than in 2006. Excluding multifamily units in high-rise buildings not built using residential building code, more than 1,700 housing units were certified in 2006. The number of comparable housing units certified in 2007 is 26% lower, which is consistent with the 24% percentage drop in permits issued.Evaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 2

Nexus Market Research 

### The year 2007 was a transition period which presented several challenges for the program. First, it had to deal with changing to a new implementation contractor, ICF. The program also began moving toward a more market-driven model. Meanwhile, stricter EPA requirements for ENERGY STAR certification, involving Thermal Bypass Checklist (TBC) requirements and duct leakage standards, had to be met by all homes, regardless of when they signed up. Thus, all builders needed to be informed of the change in contractors, be assigned new account managers and HERS raters, and be supported in dealing with stricter requirements for certification.



## Conclusions The four reports summarized in this MPER and attached as appendices support the following key conclusions:

• Overall success. The program has overall successfully dealt with the transitions it faced in 2007. It is now transitioned to ICF, and the sponsors and NUPs are satisfied with ICFís performance implementing the program in 2007. Moreover, the program sustained a high level of builder satisfaction despite the many changes builders had to deal with in 2007.

• Overall marketing. Program marketing is likely to be one of the most challenging issues the program will have to deal with in 2008. Given budget constraints, the programís marketing director is working to identify and introduce marketing approaches that will produce the maximum bang for the buck.

• Marketing support. The 2008 agreement form lists the marketing support available through the program and asks builders to indicate what support they are interested in getting. This is an easy and cost effective way of letting all participating builders know what is available and identifying builders who want marketing support. The program is addressing the need to update and expand the amount of information available to builders and consumers on their web site.

• Training. The program is addressing a critical need to provide more training to builders and HVAC and insulation contractors to ensure projects pass the TBC and meet program duct and air sealing standards.

• Incremental costs. Incremental costs for reaching the minimum ENERGY STAR level from baseline building practices are slightly lower than five years ago; however, incremental costs for reaching higher efficiency, lower HERS levels, rise sharply, especially going from HERS 85 to HERS 70. Since over half of the housing units certified in 2007 achieved HERS indices of 70 or lower, the incremental cost of getting to a HERS index of 65 or lower may be manageable for many builders. Tiered incentives, such as those being offered in 2008, encourage builders to strive for lower HERS indices.Evaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 3

Nexus Market Research 

### • Paying for HERS rater services. Most builders do not know the cost of the HERS rater services they receive from the program and all interviewed HERS raters say they plan to charge builders for extra hours of support in 2008. A big questions going forward is whether or not builders will be willing to pay for additional hours of support and, if they are not willing to pay, what impact this will have on the number of homes failing to pass the final inspection.

• Moving beyond ENERGY STAR. A majority of sponsors, NUPs, ICF staff, HERS raters and builders support raising the Massachusetts Energy Code to ENERGY STAR standards. With the average HERS index dropping each year, some interviewees strongly support moving the program to very high performance levels over time (e.g., zero net-energy homes). Also, the sponsors and NUPs agree that the program should look into quantifying and claiming non-energy benefits.

• Moving to a market-driven model. Multiple program designs are compatible with a successful market-based model using independent HERS raters. Moreover, existing companies offering HERS rating services in Massachusetts have the resources available to meet the Massachusetts programís needs for rating services and are now providing those services to builders.



## Recommendations Based on these conclusions and findings presented in this report, which are addressed in more detail in the appendices, the NMR evaluation team offers several sets of recommendations to ensure the program continues to successfully deal with changes in the marketplace and manage its transition to a market-driven model. Before listing detailed recommendations, it is important to note the need for an organized system of tracking the actions taken throughout the year to address the large number of recommendations that are generated by an annual MPER. For example, the JMC may wish to include quarterly discussions on the specific actions taken and progress made in addressing the MPERís recommendations, including a discussion of the reasons some recommendations cannot or should not be addressed at this time.

The first set of recommendations addresses the programís need to find as many cost effective ways as possible to market itself, support builders marketing their ENERGY STAR homes, and educate consumers on the benefits of buying and living in an ENERGY STAR home. Builders say the public does not value the ENERGY STAR label on homesóthis needs to change for the program to grow the market share of ENERGY STAR homes and move to a market-driven model. Having said this, the NMR team recognizes that some sponsors reach their participation goals and have all available funds committed before the end of the year and, therefore, are not necessarily looking to increase marketing in their service territories.

• Maximize free press opportunities. Alert local newspapers to ENERGY STAR model home openings, builders who are installing renewable technologies or building ENERGY STAR-certified green homes and any other events that could merit
coverage.Evaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 4

Nexus Market Research 

### • Track marketing activity and, to the extent feasible, its effect on interest and participation in the program. Sponsors want to see that marketing efforts are producing tangible results before increasing marketing budgets.

• Formulate a positive, productive response for any oversubscription that may

occur. Explain in marketing materials that there are a limited number of incentives available. Explain that if all available incentives are accounted for, builders already participating in the program can hire their HERS rater to certify additional homes, and that the program will continue to provide information on HERS rater certification and technical support resources available to new builders and homeowners interested in building an ENERGY STAR home. Explore being able to claim savings from these homes as spillover in regulatory filings.

As already noted, the program has taken an important step in encouraging and supporting builders in the marketing of their ENERGY STAR homes by listing the marketing support available in the 2008 agreement and asking builders to indicate what support they are interested in getting. Additional recommendations in this area include:

• Offer co-op advertising to builders. Three-fourths of builders interviewed in 2007 say they would be ìsomewhatî or ìveryî likely to take advantage of co-op advertising.

• Consider compensating builders willing to have an ENERGY STAR model home open to the public for several months. The model home would have informational materials available for visitors and displays explaining what goes into building an ENERGY STAR home and potential energy savings. Over one-third of interviewed builders say they would be very interested in doing this.

• Consider promoting participating builders as an approach that can: 1) address builders' concerns that the program does not partner with them enough, 2) leverage builders to do more of what the program wants them to do, such as market the ENERGY STAR status of their homes more aggressively and participate in more trainings, 3) reward leading builders for their efforts while sending business their way, and 4) set leading builders up as ambassadors for the program. Consider setting up a tiered promotion schedule: the more builders do (the more homes they have certified, the lower the HERS indices they achieve, the more trainings they or their subcontractors attend, etc.), the more they will be promoted in web site listings, named in advertising, featured in newspaper articles, and awards, etc.Evaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 5

Nexus Market Research 

### In the area of consumer marketing, the first recommendation bears repeating from previous yearsí MPERs.

• In consumer marketing, stress that the ONLY way to be sure a home is energy efficient is to have it verified by a third party using blower door and duct blaster tests, and this is what the program does. Ninety percent of builders say it is important to market the program to consumers, and 78% of builders say they would be more likely to use ENERGY STAR in their marketing, or to increase the emphasis on ENERGY STAR in their marketing, if the program marketed directly to

consumers.

• Incorporate ENERGY STAR Homes in sponsor marketing for ENERGY STAR lighting and appliances. The message could be a one liner saying, ìAnd, if you are buying a new home, look for an ENERGY STAR-labeled home.î Builders say home buyers know about ENERGY STAR appliances, but not ENERGY STAR homes.

• Target marketing to market players working with new-home buyers. For example, the program could develop, and distribute ENERGY STAR home brochures to mortgage lenders and ask them to hand a brochure to every customer applying for a mortgage for a new home in Massachusetts.

• Consider marketing messages/approaches that leverage other efforts in the state that encourage energy-efficient building , such as the net zero energy buildings task force recently announced by Governor Patrick. 1



### • Seek out and take advantage of opport unities to partner with green building programs to reach consumers interested in energy efficiency. Homebuyers are more likely to have heard of green building than ENERGY STAR-certified homesó builders say ìgreenî is the buzz word in the new construction market.

• Find a way to get green building programs to include ENERGY STAR in their marketing messages. The message should be that homes need to be ENERGY STAR certified to meet their programís energy-efficiency requirements, and that the New Homes with ENERGY STAR Program offers incentives and technical support to builders of ENERGY STAR homes.

The next set of recommendations deals with the programís long-term transition to a marketdriven model. Given recent program changes in certification requirements and the implementation contractor, the NMR team agrees with interviewees who believe it would be good for the program to have a year or two of continuity and stability before introducing any major changes affecting builders. Beyond that, the NMR team offers the following recommendations for moving on the road to a market-driven model, many of which are also suggested by interviewees:

1 Massachusetts Governor Deval Patrick announced the establishment of a task force on net zero energy buildings on March 12, 2008 at the Northeast Sustainable Energy Association's (NESEA) Building Energy Conference and Trade Show at the Seaport World Trade Center in Boston.Evaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 6

Nexus Market Research 

### • Delay having builders pay for HERS rater services at least until 2010, but begin preparing them for having to pay for at least a portion of those services. Many builders are totally unaware of the cost of hiring a HERS rater and many will not be happy about having to pay for a HERS raterónothing that adds to their costs, especially in a slow market, is likely to be well received. It will be important to soften the blow by making it clear that until now the Massachusetts program has been paying for all HERS rater services and, assuming the Massachusetts program increases incentives to help defray the cost of hiring a HERS rater, builders should be told how much the incentives were increased to help them cover the cost of hiring a HERS rater.

• All reviewed market-driven model programs offer some level of builder and/or HERS rater training. Training is important to making these models work. Assess buildersí, subcontractorsí and HERS ratersí needs fo r additional training and offer free or low-cost training opportunities. Builder and subcontractor training are especially important as the program looks to move toward higher performance tiers.

• Increase communication and coordination with the CoolSmart Program to better address quality central air conditioning installations.

• Continue to explore options for including high-rise multifamily projects in the
program, given their importance in the new home market in Massachusetts.

• If the Massachusetts program continues to offer on-site technical support to builders who are having trouble meeting program requirements it will be important to have clear protocols in place for deciding when the HERS rater should go to the program to provide support to a builder, and when it would be appropriate for the HERS rater to offer additional support for an additional cost.

• Tied to the previous recommendation, the program should tell all HERS raters marketing their services to builders to clearly distinguish between services that address meeting program requirements for certification and value-added services available at an additional cost.

The final set of recommendations deals with maintaining the operational effectiveness of the program, especially in view of changes in 2008.

• Keep everyone informed. It is important that builders get consistent information and direction to avoid confusion and complaints. At the program level, annual group meetings with builders and HERS raters to review program changes and address known issues are a start; perhaps biannual meetings with HERS raters in the first two years of the market-based delivery model would ensure even better communication. At the builder/project level, the HERS rater is likely to be the builderís main contact and it is important to take steps to ensure the technical support and advice being givenEvaluation of the Massachusetts New Homes with ENERGY STAR Program Page 

### 7

Nexus Market Research 

### to builders to ensure projects meet the programís technical standards are consistent and appropriate.

• Schedule at least one JMC meeting a year devoted to talking with account managers and HERS raters about what is happening in the field. The process evaluation interviews revealed that sponsors were unaware of how many hours of technical support many builders need and that these builders were not being charged for extra support.

• Address the irregularity that some HERS raters consistently failed homes in 2007 if there was no manual J load calculation and others did not. This issue has already been addressed in meetings with HERS raters serving the program in 2008. Any remaining irregularities, if they exist will likely be identified through the quality control inspections planned for 2008.

• Closely track bulb installations in 2008 to assess the impact on the number of bulbs installed of having buildersí electricians, rather than HERS raters, order and install bulbs. Be prepared to step in, if necessary, to facilitate the process and make sure bulbs are available at the desired time and installed in all appropriate locations.

• Implement a simpler process for reviewing and approving external communication pieces. One option is to have one JMC member take responsibility for consolidating all JMC member comments and edits and providing one final set of comments/edits to ICF. JMC members could take turns doing this so it does not become a burden for any one sponsor.

• Enlist the help of builders and other interested parties in efforts to encourage towns and communities to raise their Energy Code to ENERGY STAR standards. Many builders support a move to ENERGY STAR standards and they can provide first hand testimony based on real experience that achieving ENERGY STAR standards is neither onerous nor cost prohibitive. Also, explore options for claiming savings if the program directly facilitates code improvements to the higher, ENERGY STAR standards.

# The Massachusetts New Homes with ENERGY STAR ® 

# Program

2007 PROGRESS REPORT



# Final Report

June 13 , 200 8

Submitted to:

The Massachusetts New Homes with ENERGY STAR Program

Joint Management Committee

Submitted by:

Dorothy Conant, Co nsultant

## Table of Contents

1. INTRODUCTION

................................ ................................ ................................ .. 1

2. METRICS

................................ ................................ ................................ ............. 1

3. OVER THE YEARS

................................ ................................ .............................. 1

3.1 Massachusetts Housing Permits

................................ ................................ ......................... 2

3.2 Annual Signed Housing Units

................................ ................................ ............................. 4 3.2.1 Recruited Housing Units Compared to Statewide Permits Issued

......................... 4 3.2.2 Status of Signed Housing Units

................................ ................................ .............. 5

3.3 Certified Housin g Units as Percentage of Statewide Completed Housing Units

................ 6

3.4 HERS Ratings

................................ ................................ ................................ ..................... 8

3.5 Spending per Participating Housing Unit

................................ ................................ .......... 11



### 4. 2007 PROGRAM ACTIVIT Y

................................ ................................ ............... 14

4.1 Projects and Housing Units Recruited

................................ ................................ .............. 14

4.2 2007 Recruited Projects by Size

— Number of Units

................................ ......................... 17

4.3 Builder Mix

— New and Repeat Participants

................................ ................................ ...... 18

4 .4 2007 Completions and Signings

................................ ................................ ...................... 20 4.4.1 Distribution of Completions across Sponsor Territories

................................ ....... 23 4.4.2 Distribution of Signings across Sponsor Territories

................................ ............. 26



### 5. P ENETRATION OF ENERGY STAR MEASURES

................................ ............ 29

6. DUCT LEAKAGE

................................ ................................ ............................... 31

## List of Figures

Figure 3.1 Massachusetts Housing Permits Issued 1980

– 2007

................................ ............................... 2 Figur e 3.2 Year

- to

- Date Total Permits Issued 2002

– 2008

................................ ................................ ........ 3 Figure 3.3 Year

- to

- Date Single family Permits Issued 2002

– 2008

................................ ............................ 3 Figure 3.4 Year

- to

- Date Multifamily Permits Issued 2002

– 2008

................................ ............................... 3 Figure 3.5 Annual Signed Housing Units

................................ ................................ ................................ ..... 4 Figure 3.6 Year

- End 2006 Status of Signed Housing Units

................................ ................................ ......... 5 Figure 3.7 ENERGY STAR Completions as Percent of State

- Wide Completions

................................ ....... 7 Figure 3.8 ENERGY STAR, EMU and Code Plus Completions as Percent of State Completions

............. 7 Figure 3.9 1999

– 2006 HERS Ratings

................................ ................................ ................................ ........ 8 Figure 3.10 2007 HERS Indices

— All Homes

................................ ................................ ............................ 10 Figure 3.11 2007 HERS Indices

— Single Family Homes

................................ ................................ ........... 10 Figure 3.12 2007 HERS Indices

— Multifamily Units

................................ ................................ .................. 10 Figure 3.13 Annual Program Spending

................................ ................................ ................................ ...... 12 Figure 3.14 Annual Spending per Housing Unit

................................ ................................ ........................ 13 Figure 4.1 Historical Signed Projects by Housing Type

................................ ................................ ............. 14 Figure 4.2 Historical Signed Housing Units by Housing Type

................................ ................................ ... 15 Figure 4.3 Percent of 2003

- 2007 Market Rate versus Low Income Projects Signed

.............................. 16 Figure 4.4 Percent of 2003

- 2007 Market Rate versus Low Income Housing Units Signed

.................... 16 Figure 4.5 2003

— 2007 Signed Projects by Number of Housing Units per Project

................................ ... 17 Figure 4.6 2007 Signed Projects by Number of Housing Units per Project

................................ ............... 18 Figure 4.7 Historical Percentages of Sig ned Projects from New Builders

................................ ................. 19 Figure 4.8 All 2007 Completions by Housing Type

................................ ................................ .................... 20 Figure 4.9 2007 Market Rate Completions by Housing Type

................................ ................................ .... 21 Figure 4.10 2007 Lo w Income Completions by Housing Type

................................ ................................ .. 22 Figure 4.11 Electric Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

.... 23 Figure 4.12 Gas Sponsor 2007 Completed ENERGY STAR, EMU and C ode Plus Housing Units

.......... 24 Figure 4.13 Electric Sponsor Signed ENERGY STAR and Code Plus Projects and Housing Units

......... 26 Figure 4.14 Electric Sponsor Signed ENERGY STAR and Code Plus Market Rate an d Low Income Housing Units

................................ ................................ ................................ ................................ .............. 26 Figure 4.15 Gas Sponsor Signed ENERGY STAR and Code Plus Projects and Housing Units

.............. 27 Figure 4.16 Gas Sponsor Signed ENERGY STAR and Code Plus Market Rate and Low Income Housing Units

................................ ................................ ................................ ................................ ............................ 27 Figure 5.1 Percent of Certified Housing Units Installing ENERGY STAR Lighting and Appliance Measures 2002

– 2007

................................ ................................ ................................ ............................... 29 Figure 5.2 Percent of ENERGY STAR

- Certified Housing Un its Installing ENERGY STAR Heating Systems, Windows and Central Air Conditioning

................................ ................................ ....................... 30 Figure 5.3 Comparison of 2007 ENERGY STAR versus Code Plus Housing Units Installing ENERGY STAR Heating Systems, Windows and Air Conditioning

................................ ................................ ............ 31



## List of Tables

Table 3.1 Annual Program Spending, Signings and Completions

................................ ............................. 11 Table 4.1 Electric Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

....... 25 Table 4.2 Gas Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

............. 25 Table 4.3 Electric Sponsor Signed Total ENERGY STAR and Code Plus Projects and Housing Units

... 28 Table 4.4 Gas Sponsor Signed Total ENERGY STAR and Code Plus Projects and Housing Units

........ 28

## 1. Introduction

The annual Massachusetts New Homes with ENERGY STAR ®



## 2. Metrics Program 200 7 Progress Report is a summary of 20 0 7 program activity. Program performance information includes historical as well as current information to show the growth of the p rogram over time.

There were no metrics in 2007

.



## 3. Over the Years

The f igures on the following pages show hi storical data on housing permits issued

, homes recruited and certified, and the p rogram’s achievements since 1999. They show the number of housing units recruited each year, the average HERS ratings of homes completed in each year, the average cost per si gned housing unit and per completed housing unit each year, and completed housing units each year as a percent of estimated total annual housing units completed in Massachusetts. All 2007 data on s ignings and completions include homes certified by ICF, Co nservation Services Group ( CSG ) and Unitil , and homes recruited by ICF

. As these figures will show, the number of housing permits issued, the number of housing units signed and the number of ENERGY STAR housing units completed in 2007 are all lower than i n 2006. However, single family ENERGY STAR homes maintained their share of new single family homes completed in Massachusetts. As of the end of 200 7 , the program has ENERGY STAR certified over 13 ,000 housing units.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 2



## 3.1 **Massachusetts Housing Permits**

The n umbers of both single family and multifamily permits issu ed in Massachusetts fell for the second year in a row . Sing le family permits issued in 2007 , at 8,928 permits, are at their lowest level during the 1980 to 2007 period . ( Fi gure 3

. 1 ) Annual multifamily permits issued, which
grew consistently from 2002 through 2005, also dropped in 2006 and 2007 , but remain above 19 90 through 2002 levels. In the last two years, the number of total permits issued has fallen by 36%, the numbe r of single family permits issued by 37% and the number of multifamily permits issued by 35%.

Figure 3

. 1 Massachusetts Housing Permits Issued 1980

– 200 7

24,171

15,358 19,580 45,215

8,928 28,110

6,430 17,105

0 5,000 10,000 15,000 20,000 25,000 30,000 35,000 40,000 45,000 50,000

1980 1982 1984 1986 1988 1990 1992 1994 1996 1998 2000 2002 2004 2006 2008Massachusetts Housing Permits Issued



### Total Single Family Multifamily 

### Multifamily Single Family Total Massachusetts Permits

Figure 3

. 2

, Figure 3

. 3 , and Figure 3

. 4 on the following page show year

- to

- date total, single family and multifamily permits issued f rom 2002 t hrough April 200 8. 1

1 Total permits for each year are the final revised ann ual totals which may be higher or lower than the published December year

- to

- date totals. 

### Total permits issued in 200 7 are 22 % lower

, single family permit s 18 % lower and multi family permits 29 % lower than in 200 6 . Total permits issued January through April 200 8 are 38 % lower, single family permits 39 % lower and multi family permits 38 % lower than in the first four months of 200 7

.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 3



### Figure 3

. 2 Year

- to

- Date Total Permits Issued 2002

– 200 8

17,465 20,257 22,477 24,171

19,580

14,874

3,128

0 5,000 10,000 15,000 20,000 25,000

Jan Feb Mar April May June July Aug Sept Oct Nov Dec


### 2002 2003 2004 2005 2006 2007 2008 

### Total Permits Year-to-Date

Figure 3

. 3 Year

- to

- Date Single family Permits Issued 2002

– 200 8

13,636 13,037 14,842 14,223

10,916

8,703

1,559

0 2,000 4,000 6,000 8,000 10,000 12,000 14,000 16,000

Jan Feb Mar April May June July Aug Sept Oct Nov Dec 

### 2002 2003 2004 2005 2006 2007 2008 

### Single Family Permits Year-to-Date

Figure 3

. 4 Year

- to

- Date Multifamily Permits Issued 2002

– 200 8

3,829 7,220 7,635

9,948
8,664

6,171

1,569 0 2,000 4,000 6,000 8,000 10,000 12,000

Jan Feb Mar April May June July Aug Sept Oct Nov Dec 

### 2002 2003 2004 2005 2006 2007 2008 

### Multifamily Permits Year-to-Date2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 4



## 3.2 **Annual Signed Housing Units**

3.2.1 Recruited Housing Units Compared to Statewide Permits Issued Figure 3

. 5 shows new housing units recruited to participate in the Massachusett s p rogram as a percentage of all housing permits issued in the state fell only a fraction of a percent in 2007

— from 13.2% in 2006 to 13.0% in 2007. The number of signed housing units and the number of permits issued each fell by virtually the same percent age from 2006 to 2007

— the number of signings fell 23% and the number of permits fell 22%

— resulting in almost no change in signings as a percentage of permits issued.

Prior to 2006, signings include only housing units signed and committed to being built to ENERGY STAR standards. 2006 signings include both ENERGY STAR and Energy Measure Upgrade (EMU) housing units and 2007 signings include both ENERGY STAR and Code Plus housing units

. Also, from 2003 through 2006 signings were tracked by housing type. T he 2007 signing information is based on signed agreements ICF received in 2007 and the number of housing units in those agreements allocated to 2007 and 2008 ; no breakdown of signings by housing type is available.

Figure 3

. 5 Annual Signed Housing Units ( Percent of MA Statew ide Permits)

5% 12% 16%
14% 15% 20%

8% 12% 13.0% 13.2%
10% 28% 30%

16.3% 1,409 14%

8% 10.7% 1,171

0% 5% 10% 15% 20% 25% 30% 35%

1999 2000 2001 2002 2003 2004 2005 2006 2007 

### All Signings Multifamily Single Family 

### Multifamily

Single Family ENERGY STAR & Code Plus Signings Percent of Permits

Total Signings2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 5



### 3.2.2 Status of Signed Housing Units Since its inception, the Massachusetts p rogram has certified over 13,000 ENERGY STAR housing units. Previous progress reports showed th e status at the end of the current reporting year of all homes recruited in each year from 1999 on. With the change to a new implem entation contractor in 2007, using a different data tracking approach, detailed information on projects by the year they ori ginally signed up to participate in the program is no longer available. This does not mean one tracking approach is better than another , it simply means information is tracked differently, with some information being tracked in more detail and some in les s detail

. Consistent with the purpose of the annual progress reports to provide historical data on program activity, Figure 3

. 6 shows the reported status at the end of 2006 of all 20,118 housing units signed up to participate in the program from 1999 through 2006.

Figure 3

. 6 Year

- End 2006 Status of Signed Housing Units ( 1999

- 2006 Signings)



### 18%

59% 23%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Year-End 2006 Status of 20,118 Housing Units Signed 1999-2006 Dropped Out

Active

Completed2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 6



## 3.3 **Certified Housing Units as Percentage of Statewide Completed Housing Units** 

Figure 3

. 7 on the following page shows annual ENERGY STAR housing units certified e ach year through the p rogram as a percentage of estimated total annual completed housing units in Massachusetts. The housing units certified in any year includ e housing units recruited in previous years. From 1999 through 2006 t he number of housing units certified through the
p rogram increased each year, both in number and in percent of t otal completed housing units

, peaking in 2006 at 2, 610 housing units and 1 5.5 % of the market

. In 2007, both the number of total housing units certified and their share of total housing units completed in Massachusetts dropped sharply.

Figure 3

. 8 on the following page is the same as Figure 3

. 7 except that the 2006 and 2007 numbers include housi ng units completed through the p rogram under the EMU and Code Plus participation paths. Including housing units participating though non

- ENERGY STAR paths increases the number of housi n g units completed through the program from 2,610 to 3,318 in 2006 and from 1,286 to 1,616 in 2007, and increases the percentag e of completed housing units participating in the program from 15.5% to 19.7% in 2006 and from 9.0% to 11.2% in 2007

.
In 2003, the p rogram began tracking recruited and completed homes under the Census Bureau single family and multifamily housing category definitions, which is how h ousing permit data are reported

. Under the Census Bureau definitions, single family includes fully d etached housing units , semi detached (semi attached, side

- by

- side) housing units , row houses, and townhouses. In the case of attached units, each must be separated from the adjacent unit by a ground

- to

- roof wall and must not share heating/air

- conditioning systems or inter

- structural public utilities such as water supply, power supply, or sewage disposal lines. Because housing units certified as ENERGY STAR since 2003 are tracked using the Census Bureau definitions, it is possible to separately calculate t he percentage s of multi family and single family housing units completed in the state that are ENERGY STAR certified. Including EMU and Code Plus housing units in 2006 and 2007 raises the percentage of completed multifamily housing units that participated in the program from 35.2% to 47.6% in 2006 and from 14.1% to 19.6% in 2007 , and the percentage of completed single family homes that participated in the program from 6.8% to 7.3% in 2006 and from 7.0% to 8.1% in 2007

.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 7



### Figure 3

. 7 ENERGY STAR Completions as Percent of State

- Wide Completions

3% 4% 6% 9% 10% 12% 19%

9.0% 1,286 9% 15.5% 2,610 14.1% 533 35.2% 1,820

18% 25%

7.0% 733

6% 6.8% 790 7% 6%

0% 5% 10% 15% 20% 25% 30% 35% 40%

1999 2000 2001 2002 2003 2004 2005 2006 2007 All ENERGY STAR Completions Multifamily ENERGY STAR Single Family ENERGY STAR 

### Multifamily



### Single Family Program ENERGY STAR Completions as Percent of Statewide Completions

Total Completions



### Figure 3

. 8 ENERGY STAR, EMU and Code Plus Completions as Percent of State Completions

3% 4% 6% 9% 9% 10% 12% 19.7% 18% 19%

6% 6% 7% 11.2% 1,616 25% 47.6% 2,462

19.6% 767

7.3% 8.1% 849

0% 10% 20% 30% 40% 50% 60%

1999 2000 2001 2002 2003 2004 2005 2006 2007 

### All Completions All Multifamily All Single Family Multifamily

Single Family Program ENERGY STAR, EMU & Code Plus Completions as Percent of Statewide Completions

856 3,318 

### Total Completions2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 8



## 3.4 **HERS Ratings**

T he HERS i nd ex approach to rating homes was introduced in 2006

. Most homes certified in 2006 (92%) were rated using the traditional HERS score . Homes completed in 2007 were rated using the HERS index approach.

Figure 3

. 9 shows 1999 throu gh 2006 average and maximum HERS score s for p articipating certified homes. As shown, the average HERS score for all certified housing units increased each year : average single family HERS scores varied little, while average multifamily HERS

scores consis tently increased . T he average HERS score of housing units certifi ed in 1999 w as 86.7 and by 2006 climbed to 89.3

. This 2. 6 point increase in the average HERS score equates to an increase of 13 % in energy savings

. 2 

### Figure 3

. 9 al so shows that each year some certified homes achieved HERS scores exceeding the 86.0 score required for ENERGY STAR certification by more than five points , scoring 91.0 or better

. In three years some housing units achieved HERS scores of 94.0 or higher, w hich represent s energy savings of 40% over a home with a minimum ENERGY STAR qualifying HERS score of 86.0.

Figure 3

. 9 1999

– 200 6 HERS Ratings (Completed Housing Units Each Year Rated Using the Tradition al HERS Score )88.287.994.192.188.188.288.294.092.788.189.589.093.693.088.489.689.392.694.188.1

82.0 84.0 86.0 88.0 90.0 92.0 94.0 96.0

Average Single Family Average Multifamily Average All Homes Max Single Family Max MultifamilyTraditional HERS Score



### 2003 2004 2005 2006

2 Based on five percent increase in savings per point increase in HERS score.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 9



### The HERS Index rating system introduced in 2006 is described as follows on the energystar.gov website: 3



### Figure 3

. 10 “The HERS Index is a scoring system established by the Residential Energy Services Network (RESNET) in which a home built to the specifications of the HERS Reference Home (based on the 2006 International Energy Conservation Code) scores a HERS Index of 100, while a net zero energy home scores a HERS Index of 0. The lower a home’s HERS Index, the more energy efficient it is in
compa rison to the HERS Reference Home. Each 1

- point decrease in the HERS Index corresponds to a 1% reduction in energy consumption compared to the HERS Reference Home. Thus a home with a HERS Index of 85 is 15% more energy efficient than the HERS Reference Ho me and a home with a HERS Index of 80 is 20% more energy efficient.”

Homes in Massachusetts m ust achieve a HERS Index of 85 or lower to be ENERGY STAR certified. Assuming a one percent increase in energy savings per point decrease in the HERS Index, a HERS Index of 50 reflects 50 % energy savings compared to the HERS reference home (2006 International Energy Conservation Code) and energy savings 35 percentage points higher than needed to be EN ERGY STAR certified (85.0 HERS I ndex).

, Figure 3

. 11 and Figure 3

. 12 on the next page show the individual HERS indices ach ieved by homes certified in 2007 for all certified homes, certified single family homes and certified mu ltifamily units, respectively. As shown, the average and median HERS indices are the same for all home s , single family homes and multifamily units

— average HERS index is 68 and median HERS index is 69. The average 68 HERS I ndex reflects 32% energy savings compared to the 2006 Interna tional Energy Conservation Code reference home, and energy savings 17 percentage points higher than required for ENERGY STAR certification in Massachusetts. T he HERS indices achieved by single family homes cover a much larger range : HERS indices achieved by mu ltifamily units range from 85 to 53, while HERS indices achiev ed by single family homes range from 85 to 14. Thirty

- one, or four percent

, of the single family certified homes achieved HERS indices of 50 or less, which i s considered an indication that a home would qualify for the $2,000 federal tax credit.

T he 2008 program encourages builders to build to higher efficiency levels by paying a higher incentive for homes that achieve a HERS index of 65 or lower. Over one

- third (35%) of the housing units certified in 2007 achieved HERS indices of 65 or lower

— 33% of certified single family homes and 39% of certified multifamily units

.

3 http://www.energystar.gov/index.cfm?c=bldrs_lenders_raters.nh_HERS2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 10



### Figure 3

. 10 2007 HERS Indices

— All Home s

10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0

1,227 ENERGY STAR-Certified HomesHERS Index

Minimum (best) 14 Median

69

Average 68 2007 HERS Indices Tax Credit 50 2008 Tier II Incentive

Level

65



### Figure 3

. 11 2007 HERS Indices

— Single Family Homes

10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0

700 Single Family ENERGY STAR-Certified HomesHERS Index

Minimum (best) 14 Median

69

Average 68

2007 Single Family HERS Indices Tax Credit 50 2008 Tier II Incentive

Level

65



### Figure 3

. 12 2007 HERS Indices

— Multifamily Units

10.0 20.0 30.0 40.0 50.0 60.0 70.0 80.0 90.0

527 Multifamily ENERGY STAR-Certified UnitsHERS Index

Minimum (best) 53 Median

69

Average 68

2007 Multifamily HERS Indices 2008 Tier II Incentive

Level

652 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 11



## 3.5 **Spending per**  **Participating Housing Unit** 

Table 3.1 shows that the number of housing units signed in one year peaked in 2005 at 4,761. In 2006 and 2007 , homes were signed under non

- ENERGY STAR as well as ENERGY STAR participation paths

— E NERGY STAR and EMU pat hs in 2006 and ENERGY STAR and Code Plus paths in 2007 . The total number of housing units signed dropped sharply in 2006 and again in 2007 reflecting the impacts of no longer recruiting multifamily units in buildings over three stories and the slow down i n the new construction market. T he annual number of housing units completed through the program rose steadily through 2006, then plunged in 2007. The drop in 2007 completions again reflects the impacts of the depressed market for new housing and the
prog ram no longer being able to certify multifamily units in buildings over three stories.

Table 3

. 1 Annual Program Spending, Signings and Completions

Year Spending $Thousands Housing Units Signed Housing Un its Completed

2000 $3,160 2,085 565
2001 $3,434 2,715 965
2002 $4,078 2,423 1,435
2003 $4,160 2,063 1,630
2004 $ 5,193 3,320 1,854
2005 $5,284 4,761 2,358
2006 * $5,390 2,580 3,318
2007 * $3,610* * 1,994 1,316 

### * 2006 and 2007 include ENERGY STAR, EM U and Code Plus housing units

. 

### * * Preliminary Estimate2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 12



### Figure 3

. 13 sho ws annual spending by electric program s ponsors increased each year from 2000 to 2006, then declined sharply in 2007. Preliminary estimates of 2007 spending p rovided by electric program sponsors show the drop in 2007 spending is predominantly due to a decrease in low income spending: low income spending fell by $1.2 million and non

- low income spending by $600 thousand

. 4



### Figure 3

. 13 Annual Program Spending

$0.8 $0.6 $0.7 $1.5 $2.7 $3.5 $3.4 $3.7

$0.6 $1.4 $0.7 $1.8 $3.0 $3.9

$2.5 $3.6

$0.0 $1.0 $2.0 $3.0 $4.0 $5.0 $6.0

2000 2001 2002 2003 2004 2005 2006 2007Millions



### Low Income Non Low Income $3.4 $4.1 $4.2

$3.2 $5.2 $5.3 

### Total Spending $5.4

$3.6

4 The cost data are from annual reports filed with the Division of Energy Resources (DOER) by the electric utilities and the Cape Light Compact. The cost data include customer incentives plus in

- house and contracted out expenses for planning and administration, marketing, an d implementation. The cost data do not include evaluation expenses, market research expenses, performance incentives, other costs or participant costs.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 13



### Figure 3

. 14 shows the annual spending per signed housing unit and per completed housing unit. The dramatic decrease in spending per completed housing unit from the early years of the
p rogram is largely a reflection of the lag between the time housing units are signed up and the time they are certified. 2006 and 2007 include housing units signed and completed under both ENERGY STAR and non

- ENERGY STAR paths. The decrease in 2007 spending per signed housing unit is the result of the number of units signed decreasing by a lower percentage than spending decreased

— spending dropped 33% and the number of housing units signed dropped 23%. Conversely, the increase in 2007 spending per co mpleted housing unit is the result of the number of completed housing units decreasing by a higher percentage than spending decreased

— spending dropped 33% and the number of completed housing units dropped 60%.

Figure 3

. 14 Annual Spending per Housing Unit

$2,801 $2,241

$1,515 $1,265 $1,683 $2,016 $1,564 $1,110 $2,552 $2,842 $5,592

$3,558

$1,624 $2,743
$2,089

$1,810

$0 $1,000 $2,000 $3,000 $4,000 $5,000 $6,000

2000 2001 2002 2003 2004 2005 2006 2007 

### Year Housing Unit Signed or Completed ( 2006 and 2007 Include ENERGY STAR, EMU and Code Plus Housing Units 

### ) Spending Per Completed Housing Unit

Spending Per Signed Housing U it2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 14



## 4. 2007 Program Activity

The p rogram strives to recruit a mix of project and housing types and to bring new builders into the p rogram , as well as maintain the involvement of currently participating buil ders

, to sustain the p rogram ’s momentum.



## 4.1 **Projects and Housing Units Recruited** 

Historically, t he p rogram has strive d to recruit a mix of market rate and low income housing types including single family detached, single family attached and multifamily ho using. Information on 2007 signings includes a breakdown by market rate versus low income, but does not include a breakdown by housing type. Figure 4

. 1 shows historical data on the percentage of 2003 through 2006 projects recrui ted by major housing type : single family detached, single family attached and multifamily. Figure 4

. 2 shows historical data on the percentage of 2003 through 2006 housing units recruited by major housing category: single family detached, single family attached and multifamily. As shown, single family detached projects have consistently accounted for a sizable majority of the projects recruited each year , but for less than one

- third of the housing units recruited because most si ngle family detached projects involve only one or two homes

. Figure 4

. 1 Historical Signed Project s by Housing Type (Census Bureau Definitions)



### 80% 70% 59% 72% 69% 8%

7% 15% 19% 12% 12% 24% 26% 9% 19%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### 2003 2004 2005 2006 2003-2006Percent of Projects Signed



### Single Family Detached Single Family Attached Multifamily2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 15



### Figure 4

. 2 Historical Signed Housing Units by Housing Type ( Census Bureau Definitions)



### 32% 29% 16% 25% 24% 19%

7% 21% 35%
20% 49% 64% 64% 41% 57%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### 2003 2004 2005 2006 2003-2006Percent of Units Signed



### Single Family Detached Single Family Attached Multifamily



### Going forward, given that the program can no t currently recruit multifamily units in buildings over three stories except under specific conditions, the percentage of multif amily units recruited each year is likely to be lower than in previous years unless the program finds a way to bring high

- rise multifamily buildings back into the program.

Figure 4.3 and Figure 4

. 4 on the following page show, respectively, the annual percentages of market rate and low income projects and housing units recruited from 2003 through 2007. Low income projects are defined as any project that includes at least one low income u nit. As shown, the percentage of projects recruited in 2007 that include low income units is higher than in previous years, but low income units recruited in 2007, as a percentage of all recruited units, is consistent with previous years.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 16



### Figure 4

. 3 Percent of 2003

- 2007 Market Rate versus Low Income Projects Signed



### 94% 91% 89% 96% 81% 90% 6% 9% 11% 19% 10% 4%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### 2003 2004 2005 2006 2007 2003- 2007Percent of Projects Signed



### Market Rate Low Income

Figure 4

. 4 Percent of 2003

- 2007 Market Rate versus Low Income Housing Units Signed



### 79% 83% 79% 94% 84% 83% 21% 17% 21% 6% 16% 17%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### 2003 2004 2005 2006 2007 2003- 2007Percent of Units Signed



### Market Rate Low Income2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 17



## 4.2 **200 7**  **Recruited Projects by Size** 

**— Number of Units** 

Figure 4

. 5 shows the annual percentage s of projects and housing units signed from 2003 through 2007 falling into various size categories based on the number of housing units in th e project. Not surprisingly

, single

- home s account for more than half of all projects signed in each year but 2007 (48% in 2007)

, but only a small percent age of the housing units , ranging between four percent of housing units in 2005 and 2007 and seven per cent in 2003. The majority of housing units recruited in every year are in proj ects with over 25 housing units , ranging between 6 5% of housing units recruited in 2004 and 80% in 2003.

Figure 4

. 5 2003

— 20 07 Signed Projects by Number of Housing Units per Project

70% 55% 56% 64% 48%

6% 8%

14% 7% 12%

18% 8%

22% 25% 69% 67%

6% 7% 25% 

### 20% 11% 23% 19% 16% 22% 65% 75% 80% 11% 15% 8% 12% 13%

0% 20% 40% 60% 80% 100%



### 2003 2004 2005 2006 2007 2003 2004 2005 2006 2007

Single Homes Small 2-5 Unit Projects
Mediium 6-25 Unit Projects Large Over 25 Unit Projects

Percent of Projects Percent of Housing Units



### Figure 4.6 is another way of showing the very high percentage of projects that have very few housing units. Going forward it will be interesting to see i f the percentage of one and two unit projects decreases as the p rogram moves further toward a market driven model using independent HERS raters to recruit projects. HERS raters will likely target multiple unit projects because they will be more profitable , requiring less time per housing unit to service than single

home projects. On the other hand, if the number of one and two unit projects is predominantly driven by small builders who want to participate in the program and homeowners who want their
custo m home built to ENERGY STAR standards the percentage of one and two unit projects may continue to be high.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 18



### Figure 4

. 6 200 7 Signed Projects by Number of Housing Units per Project

118 135 204 276

0 50 100 150 200 250 300

167 Agreements Received in 2007Number of Housing Units

One or Two Units 3 to 10 Units 11 to 25 Units 26 to 100 Units >100 Units 60% of Agreements Received in 2007 Have Only One or Two Housing Units 14% of Agreements Have 3 to 10 Units 15% of Agreements Have 11 to 25 Units 8% of Agreements have 26 to 100 Units 2% of Agreements (4 projects) Have

Over 100 Units



## 4.3 **Builder Mix**

**— New and R**  **epeat Participants**

Historically, program sponsors have been interested in the number of projects and units coming from builders new to the program. 2007 signing data by new versus repeat builder are not available. Figure 4

. 7 sh ows the percentages of annual project and housing unit signings associated with builders new to the program. As shown, the percentage of projects coming from builders new to the program steadily decreased from 76% in 2002 to 56% in 2006. This decline is reasonable because each year the population of builders who have participated in the program grows. Over the 2002 through 2006 period, 62% of all projects signed were with builders who had not previously participated in the program. The percentage of sig ned housing units coming from new builders has varied from year to year, ranging from a low of 47% in 2002 to a high of 68% in 2006. Over the 2002 through 2006 period, 58% of all housing units signed were with builders who had not previously participated in the program. The program has clearly been successful in bringing new builders into the program. With ICF’s ongoing efforts to attract new builders, along with the many independent HERS raters now working in the field to recruit builders, it seems reas onable to assume that a large percentage of the builders signing agreements each year will continue to be new to the program.2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 19



### Figure 4

. 7 Historical Percentages of Signed Projects from New Builders



### 76%

47% 68% 66% 60% 59% 57% 53% 56% 68% 62% 58%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### Percent of Projects Percent of UnitsPercent of Signed Projects & Units From New Builders



### 2002 2003 2004 2005 2006 2002-2006 New Builders2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 20



## 4.4 **200 7**  **Completions and Signings** 

Figure 4

. 8 shows the percentages of 2007 completions by housing category for combined ENERGY STAR, EMU and Code Plus completions ; only ENERGY STAR completions ; and only EMU and Code Plus completions.

Figure 4

. 8 All 2007 Completions by Housing Type



## 2007 Total Completions ENERGY STAR, EMU & Code Plus (n=1,616) Single Family Attached 29% 

### Mutifamily 47% Single Family Detached 24%



## 2007 Completions

ENERGY STAR (n=1,286) Single Family Attached 32% 

### Mutifamily 43% Single Family Detached 25%



## 2007 Completions

EMU & Code Plus (n=330) Single Family

Attached 18%

Mutifamily 65% Single Family

Detached 17%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 21



### Figure 4

. 9 shows the percentages of 2007 market rate completions by housing category for combined ENERGY STAR , EMU an d Code Plus completions; only ENERGY STAR completions ; and only EMU and Code Plus completions.

Figure 4

. 9 2007 Market Rate Completions by Housing Type



## 2007 Market Rate Completions ENERGY STAR, EMU & Code Plus (n=1,250) Single Family Attached 30% 

### Mutifamily 43% Single Family Detached 27%



### 2007 Market Rate Completions

ENERGY STAR (n=997) Single Family Attached 33% 

### Mutifamily 40% Single Family Detached 28%



### 2007 Market Rate Completions ENERGY STAR, EMU & Code Plus (n=1,250) Single Family Attached 30% 

### Mutifamily 43% Single Family Detached 27%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 22



### F igure 4.10 shows the percentages of 2007 low income completions by housing category for combined ENERGY STAR , EMU and Code Plus completions; only ENERGY STAR completions ; and only EMU and Code Plus completions.

Figure 4

. 10 2007 Low Income Completions by Housing Type



## 2007 Low Income Completions ENERGY STAR, EMU & Code Plus (n=366) Single Family Attached 23%



### Mutifamily 63% Single Family

Detached 14%



## 2007 Low Income Completions

ENERGY STAR (n=289) Single Family Attached 28%



### Mutifamily 55% Single Family

Detached 17%



## 2007 Low Income Completions

EMU & Code Plus (n=77)

Single Family

Attached 6%

Single Family

Detached 1%



### Mutifamily 92%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 23



### 4.4.1 Distribution of Completions across Sponsor Territories

Electric Sponsor territories Figure 4.11 shows the percentage of total ENER GY STAR, EMU and Code Plus housing units completed in 2007 in each of the electric sponsors’ service areas and in municipal electric service areas ; the percentage of total ENERGY STAR

- certified housing units completed in 2007 in each of the electric sponso rs’ service areas and in municipal electric service areas; and the percentage of total completed EMU and Code Plus housing units completed in 2007 in each of the electric sponsors’ service areas and in municipal electric service areas

. Figure 4

. 11 Electric Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

All 2007 Units Completed ENERGY STAR, EMU & Code Plus (n=1,616) CLC 8%

Muni 1% UNITIL 2%

WMECO 8%


## National Grid 39% 

## NSTAR 42%

2007 ENERGY STAR Units Completed (n=1,286) CLC 10%

Muni 1% UNITIL 3% WMECO, 9%



## National Grid 41%



## NSTAR 36%

2007 EMU & Code Plus Units Completed (n=330) CLC 0.3%

Muni 0.6% WMECO

5% 

### National Grid 29% 

# NSTAR 65%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 24



### Gas Sponsor Territories Figure 4.12 shows the percentage of total ENERGY STAR , EMU and Code Plus housing units completed in 2007 in each of the gas sponsors’ service areas; the percentage of total ENERGY STAR

- certified housing units completed in 2007 in each of the gas sponsors’ service; and the percentage of total completed EMU an d Code Plus housing units completed in 2007 in each of the gas sponsors’ service areas. Table 4.1 and Table 4.2 on the following page show the data used to generate Figure 4

. 11 and Figure 4

. 12

.

Figure 4

. 12 Gas Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

All 2007 Units Completed ENERGY STAR, EMU & Code Plus (n=1,616) BRKSH 4%


### Bay State 21% NE 2%

No Gas Sponsor 16% 

### KeySpan 39%

NSTAR 18%

2007 ENERGY STAR Units Completed (n=1,286) 

### NSTAR 21% 

## KeySpan 33%

No Gas Sponsor

15% NE 2%



### Bay State 25% BRKSH 5%

2007 EMU & Code Plus Units Completed (n=330) BRKSH 3% Bay State 5% NE 6%



### No Gas Sponsor 22% 

### KeySpan 61%

NSTAR 4%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 25



### Table 4

. 1 Electric Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

Electric Sponsors EMU & Code Plus Completions ENERGY STAR Completions Total Completions: ENERGY STAR EMU Code Plus Total Units Market Rate U nits Low Income

Units Total Units Market Rate Units Low Income

Units

NGRID 97 84 13 533 419 114 630
NSTAR 215 153 62 460 327 133 675
Western Mass Electric ( WMECo ) 15 13 2 115 87 28 130
Cape Light Compact (CLC) 1 1 0 127 114 13 128
Unitil 0 0 0 36 3 5 1 36 Municipals (Muni) 2 2 0 15 15 0 17
Totals: 330 253 77 1,286 997 289 1,616



### Table 4

. 2 Gas Sponsor 2007 Completed ENERGY STAR, EMU and Code Plus Housing Units

Gas Sponsors EMU & Code Plus Compl etions ENERGY STAR Completions Total Completions: ENERGY STAR EMU Code Plus Total Units Market Rate Units Low Income

Units Total Units Market Rate Units Low Income

Units
Bay State Gas 16 8 8 319 259 60 335 KeySpan 200 196 4 427 294 133 627

Berkshire Gas (BRKSH) 10 10 0 61 49 12 71

NSTAR 14 11 3 271 192 79 285 New England 19 19 0 21 21 40 No Gas Sponsor 71 9 62 187 182 5 258
Totals: 330 253 77 1,286 997 289 1,6162 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 26



### 4.4.2 Distribution of Signings across Sponsor Territories

Electric Sponsor Territori es Figure 4.13 show s the percentage of total ENERGY STAR and Code Plus projects and housing units signed in 200 7 in each of the e lectric s ponsors’ service areas and in municipal electric service areas

. Figure 4

. 14 shows the percentage of total market rate and total low income housing units signed in 2007 in each of the electric sponsors’ service areas and in municipal electric service areas.

Figure 4

. 13 Electric Sponsor Signed ENERGY STAR and Code Plus Projects and Housing Units



### Total 2007 Signed Projects
ENERGY STAR & Code Plus (n=167) 

## NSTAR

24% UNITIL 1%

WMECO 22%



### National Grid 42% CLC 8% Muni 3%



### Total 2007 Signed Units
ENERGY STAR & Code Plus (n=1,994) Muni 3% CLC 5% National Grid 37% WMECO 8% UNITIL

1% 

## NSTAR

46%

Figure 4

. 14 Electric Sponsor Signed ENERGY STAR and Code Plus Market Rate and Low Income Housing Units

Total 2007 Signed Market Rate Units ENERGY STAR & Code Plus (n=1,681) Muni 3% CLC 6% 

### National Grid 37% WMECO 8% UNITIL 1% 

## NSTAR

46%

Total 2007 Signed Low Income Units ENERGY STAR & Code Plus (n=313) WMECO 10% CLC 1%



### National Grid 42% 

## NSTAR

47%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 27



### Gas Sponsor Territories Figure 4

. 15 show s the number and percentage of total ENERGY STAR and Code Plus projects and housing units signed in 2007 in each of the g as s ponsors’ service areas and in areas where

n atural gas is not available or ga s providers do not sponsor the p rogram

. Figure 4

. 16 shows the percentage of total market rate and low income housing units signed in 2007 in each of the g as

s ponsors’ service areas and in areas wh ere natural gas is not available or gas provider s do not sponsor the p rogram

. Table 4

. 3 and Table 4

. 4 on the following page show the data used to generate Figure 4

. 13 though Figure 4

. 16

. Figure 4

. 15 Gas Sponsor Signed ENERGY STAR and Code Plus Projects and Housing Units



### Total 2007 Signed Projects
ENERGY STAR & Code Plus (n=167) NSTAR 12% KeySpan 30%

No Gas Sponsor 25%

NE 2%



### Bay State 22% BRKSH 8%



### Total 2007 Signed Units
ENERGY STAR & Code Plus (n=1,994) 

### NSTAR 15% 

### KeySpan 42% No Gas Sponsor 4% NE 3%



### Bay State 33% BRKSH 4%



### Figure 4

. 16 Gas Sponsor Signed ENERGY STAR and Code Plus Market Rate and Low Income Housing Units

Total 2007 Signed Market Rate Units ENERGY STAR & Code Plus (n=1,681) 

### NSTAR 14% 

### KeySpan 42% No Gas Sponsor 5% NE 3%



### Bay State 32% BRKSH 4%

Total 2007 Signed Low Income Units ENERGY STAR & Code Plus (n=313) 

### NSTAR 19% 

### KeySpan 38% No Gas Sponsor 4% 

### Bay State 33% BRKSH 7%2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 28



### Table 4

. 3 Electric Sponsor Signed Total ENERGY STAR and Code Plus Projects and Housing Units

All 2007 ENERGY STAR and Code Plus Signed Projects and Units

Electric Sponsors And Municipals Total Projects Market Rate Projects Low Income Projects Total Units Market Rate Units Low Income

Units
NGRID 70 54 16 746 615 131
NSTAR Electric 40 31 9 921 774 147 Western Mass Electric (WMECo) 37 31 6 158 127 31 Cape Light Compact (CLC) 13 12 1 108 104 4
Unitil 2 2 0 11 11 0 Municipals (Muni ) 5 5 0 50 50 0

Totals: 167 135 32 1,994 1,681 313



### Table 4

. 4 Gas Spon sor Signed Total ENERGY STAR and Code Plus Projects and Housing Units

All 2007 ENERGY STAR and Code Plus Signed Projects and Units



### Gas Sponsors Total Projects Market Rate Projects Low Income Projects Total Units Market Rate Units Low Income

Units

Bay St ate Gas 37 29 8 649 545 104
KeySpan 50 35 15 829 711 118

B erkshire Gas (BRKSH) 14 11 3 86 65 21

NSTAR 20 18 2 291 233 58
New England Gas (NE) 4 4 0 51 51 0

No Gas Sponsor 42 38 4 88 76 12

Totals: 167 135 32 1,994 1,681 3132 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 29



## 5. Penetration of ENERGY STAR Measures

Figure 5

. 1 shows the percentage of housing units certified in each year from 2002 through 2006 that installed ENERGY STAR lighting measures through the program and the percentage of housing units certified in each y ear from 2002 through 2007 that installed ENERGY STAR dishwashers and refrigerators. ICF was not able to provide the number of homes installing lighting measures in 2007, but says that virtually all the homes they certified installed at least some compact fluorescent bulbs (CFLs) because HERS raters were incentivized to have builders install CFLs. The total number of CFL bulbs installed in ENERGY STAR, EMU and Code Plus housing units in 2007 is 31

, 819 : this is 14,407 bulbs, or 31%, lower than in 2006. A ll ENERGY STAR lighting fixtures installed through the program in 2007 were installed either in homes certified by CSG early in 2007 or in housing units certified by Unitil. The total number of CFL fixtures installed in 2007 is 1,422 : this is 3 ,109 fixtu res , or 69 %, lower than in 2006.

P ercentage s of ENERGY STAR housing uni ts installing ENERGY STAR dishwashers and refrigerators dropped considerably in 2007

. The number of ENERGY STAR housing units installing ENERGY STAR clothes washers is also tracked, but t he information is not shown in Figure 5

. 1 because the percentages are so low , ranging between one and five percent : the percentage of ENERGY STAR homes installing ENERGY STAR clothes washers in 2007 is two percent

.

Figure 5

. 1 Percent of Certified Housing Units Installing ENERGY STAR Lighting and Appliance Measures 2002

– 200 760%57%59%59%58%47%70%68%50%49%72%62%58%34%80%76%65%43%47%6%70%45%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### Fixtures Bulbs Dishwashers RefrigeratorsPercent of ENERGY STAR-Certified Homes



### 2002 2003 2004 2005 2006 20072 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 30



### Figure 5

. 2 shows the percent ages of ENERGY STAR

- certified housing unit s that installed ENERGY STAR heating systems

, windows and central air conditioning

. As shown, the percentage of homes installing ENERGY STAR heating systems increased steadily from 2003 through 2006 to 87% and then fell to 72 % in 200 7. The percentage of homes installing ENERGY STAR windows has varied over the last few years . In 2005, almost three

- fourths (73%) of all certified homes installed ENERGY STAR windows; in 2006 the percentage of homes installing ENERGY STAR windows fell sharply to 56% and in 20 0 7 rebounded to a high of 83% of certified homes. T he percentage of housing units installing ENERGY STAR central air conditioning remains low: the 2006 to 2007 decline in the percentage of homes installing ENERGY STAR central air conditioning is like ly a reflection of the change in the ENERGY STAR criteria for central air conditioning from SEER 13 to SEER 14. Also, the penetration of ENERGY STAR central a ir conditioning will , reasonably, always be lower tha n the penetration of ENERGY STAR windows and hea ting systems because all homes have heating systems and windows, but only some homes have central air conditioning. Putting the 7% penetration of ENERGY STAR central air conditioning into better perspective , j ust over half (55%) of the housing units certi fied in 2007 have central air conditioning and 12% of the se homes installed ENERGY STAR equipment.

Figure 5

. 2 Percent of ENERGY STAR

- Certified Housing Units Installing ENERGY STAR Heating Systems , Windows and Central Air Conditioning 2002

– 200 766%69%53%77%73%87%56%15%72%83%37%77%52%3%7%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### Heating Systems Windows Central A/CPercent of ENERGY STAR-Certified Homes



### 2002 2003 2004 2005 2006 20072 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 31



### ICF was able to provide data on the number of 2007 EMU and Code Plus housing units that installed ENERGY STAR heating systems, windows and central air conditioning. Figure 5

. 3 sho ws that virtually the same percentages of ENERGY STAR and EMU/ Code Plus homes installed ENERGY STAR heating systems, but that EMU/ Code Plus housing units were much less likely than ENERGY STAR housing units to have ENERGY STAR windows or central air condit ioning.

Figure 5

. 3 Comparison of 2007 ENERGY STAR versus Code Plus Housing Units Installing ENERGY STAR Heating Systems , Windows and Air Conditioning



### 7% 

## 83% 72% 73%

1% 

## 52%

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



### ENERGY STAR Heating Systems ENERGY STAR Windows ENERGY STAR Central A/CPercent of Housing Units



### ENERGY STAR Housing Units Code Plus Housing Units



## 6. Duct Leakage

Revised EPA s tandards for ENERGY STA R certification require ducts be sealed and tested to have leakage at or below 6 cfm25 per 100 square feet of conditioned floor area . However, duct leakage testing is not required if all ducts and air handling equipment are in conditioned space and envelo pe leakage is 3 ACH50 or less. The majority of homes participating in the Massachusetts program have ducts.

In 2006, the program certified 938 housing units that had ducts and met the new d uct leakage requirement s : the average duct leakage in these ho mes was 2.3 cfm25 per 100 square feet of conditioned floor area.

In 2007, almost all certified housing units (96%) had ducts. The average duct leakage in 2007 for ENERGY STAR

- certified housing units with ducts in unconditioned space was 4.3 cfm25 per 1 00 square feet of conditioned floor area : 5.1 cfm25 per 100 square feet of conditioned floor area in multifamily units and 3.9 cfm25 per 100 square feet of conditioned floor area in single2 007 Massachusetts New Homes with ENERGY STAR ® Program Progress Report Page 32



### family homes. Of the certified housing units with ducts, 42% had all ducts installed in conditioned space and met the envelope leakage requirement to waive duct testing : 51% of multifamily units with ducts and 36% of single family homes with ducts

. The program encourages builders to install ducts in conditioned space and, based on findings from the 2005 Baseline study, it has been successful . In the 2005 Baseline Study, only 11% of the single family homes with ducts had all ducts installed in conditioned space

.

## MassSAVE Final Summary QA/QC and Impact Study Report

April 8, 2008

Prepared for:

Bay State Gas Berkshire Gas Cape Light Compact KeySpan Energy Delivery National Grid NSTAR Electric and Gas Corporation Unitil/Fitchburg Gas & Electric

New England Gas Company Northeast Utilities/Western Massachusetts Electric

Prepared by: RLW Analytics

179 Main Street Middletown, CT 06457 (860) 346-5001

## T ABLE OF 

## C ONTENTS



## 1 I NTRODUCTION 

## .......................................................................... 1

1.1 Objectives ....................................................................................... 3


## 2 M ETHODOLOGY 

## .......................................................................... 3

2.1 Summary of QA/QC and Measur e Level Impact Methodology ............... 4

2.2 Summary of Billing Anal ysis Method ology........................................... 6


## 3 R ESULTS 

## .................................................................................. 8

3.1 Objective 1: Perform QA/QC visi ts on installed measures. ................... 8

3.2 Objective 2: Provide feedback and impacts for targeted measures. .... 10

3.3 Objective 3: Provide a check of gas pr ogram impacts at a high level. . 13


## 4 C ONCLUSIONS AND 

## R ECOMMENDATIONS 

## .......................................... 14

T ABLE OF 

## T ABLES



### Table 1: Summary of Vend or Provided Data (PY2006) .............................................. 2 Table 2: Summary of Vendor Data by Fuel with per Home Savings (PY2006) ...... 3 Table 3: Final On-Site Sample Design ........................................................................... 5 Table 4: Measure Ratings and Status Summary .......................................................... 9 Table 5: Comparison of 2004 and 2007 Satisfaction Levels .................................... 10 Table 6: Summary of Savings (the rms) for Fossil Fuel by Measure ....................... 11 Table 7: Summary of Savings (kWh) fo r Electric Measures in the Program ......... 11 Table 8: Final Results (Customer Level) ..................................................................... 13 Table 9: Final Results (Program Level) ....................................................................... 13 Table 10: Summary of Savings Impacts ..................................................................... 14 Table 11: Recommendations for Savings Estimate Improvements ........................ 15



## T ABLE OF 

## F IGURES



### Figure 1: Overview of 2007 Mass SAVE Evaluation Work Approach ......................... 4 Figure 2: Sampled Lighting Logger Profiles for the Average Weekday and Weekend .......................................................................................................................... 12Massachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 1



### ________________________________________________________________________ 

## 1 Introduction
This report is intended to provide a summa ry of results as gathered from a series of evaluation activities performe d for the MassSAVE Program in 2007 on participants from the 2006 program year. These activities comprised of the performance of on-site QA/QC work, meas ure level impact work and a natural gas billing analysis.

The MassSAVE program provides toll free phone energy saving tips and screening, a comprehensive home energy assessment and follow-up services to all Massachusetts residential electric an d gas customers. The State Legislature of Massachusetts mandates the MassSAVE program for all regulated electric and gas distribution companies, and the MA Division of Energy Resources (“MA DOER”) regulates the program. MassSAVE is the new name for the Massachusetts Residential Conservation Services Program, which has been available since 1980. The overa rching goal of the progra m is to deliver non-lowincome residential customers with services that are intended to simplify customer participation and provide a “one-stop sh opping” home energy efficiency and renewable energy service. The Progra m refers low-income customers to the appropriate provider of low-income progra ms. There were five primary program vendors that provided program services on behalf of the spon soring entities in 2006.

The MassSAVE Program has been designed to provide several benefits to participants including redu ced energy costs, improv ed home performance, a healthier home environment, and improved home comfort all year long. Services provided include the following:

 Educational materials/services to help consumers save energy,

 In-home services available at no cost for customers who plan to invest in energy saving improvements,

 Step-by-step guidance to lead cust omers through the installation of energy-saving measures an d receiving incentives,

 Incentives toward the installati on of energy-s aving measures,

 Quality installations performed by fu lly licensed and insured contractors who warrant all workmanship and materials for one year, and

 Quality inspections available to ensu re that the job was done right.

To illustrate the diverse offering of me asures and the multi fuel nature of the MassSAVE Program, Table 1 presents the 2006 program year participant application and annual savings data as ga thered from program vendors by fuel type in kBtus. It should be noted that these values are slightly different than DOER activity for 2006; which may be the re sult of the timing of the DOER dataMassachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 2



### ________________________________________________________________________ extract from the vendors or changes in the crediting of savings among programs after the data is sent along to DOER. HDMC also have savings in this table with completion dates in their data that ap peared to fall outsid e of 2006, although they may have been credit ed to the 2006 program for fi scal reasons. The gas and oil savings are generally associated wi th insulation and air sealing measures while electric savings are generally associat ed with lighting and refrigeration. The specific measures provided by th e vendors are dependent on their contracting utility. As such, not all vendors provide all measures.

Annual Non-Electric Fuel Savings (in kBtu) Measure/Vendor CET CSG HDMC Rise Unitil Total Air Sealing/Weatherization 91,633 3,282,198 25,326,356 857,535 67,699 29,625,421 Attic Insulation 209,786 9,496,745 19, 890,865 2,127,130 74,497 31,799,023 Basement Insulation 19,990 852,943 719,194 184,914 6,110 1,783,151 DHW 0 77,189 4,730,428 41,752 61,222 4,910,591 Duct Insulation 56,640 23,108 1,645,777 4,946 31,108 1,761,578 Duct Sealing 0 24,493 0 21,940 0 46,433 Heating System 0 5,812,480 0 56,240 0 5,868,720 Indirect DHW 0 2,664,000 0 64,154 0 2,728,154 Lighting 0 0 0 0 0 0 Pipe Insulation 267,044 0 1,495,694 4,639 0 1,767,377 Refrigerator 0 0 0 0 0 0 Rim Joist Insulation 0 279,737 0 0 69,159 348,896 Thermostat 156,668 1,827,432 3,121,640 65,987 116,254 5,287,981 Wall Insulation 563,245 12,895,430 18,272,786 503,378 39,579 32,274,418 Windows 0 649,440 0 527,647 0 1,177,087 Non-Electric Total 1, 365,006 37,885,195 75,202,740 4,460,260 465,629 119,378,830 No. of Applications 48 1,790 5,293 366 20 7,517 Annual Electric Savings (in kBtu) Air Sealing/Weatherization 0 88,645 14,335 118,312 5,130 226,421 Attic Insulation 0 250,783 0 234,389 15,170 500,341 Basement Insulation 0 42,533 0 31,329 0 73,862 DHW 0 28,034 31,182 37,435 1,553 98,205 Duct Insulation 0 0 0 0 0 0 Duct Sealing 0 16,552 0 0 0 16,552 Heating System 0 0 0 0 0 0 Indirect DHW 0 40,000 0 0 0 40,000 Lighting 0 26,732,130 5,717,303 1,787,736 46,048 34,283,217 Pipe Insulation 0 0 0 1,744 0 1,744 Refrigerator 0 2,311,293 893,523 708,103 0 3,912,919 Rim Joist Insulation 0 3,060 0 0 0 3,060 Thermostat 0 393,637 0 70,526 9,350 473,513 Wall Insulation 0 24,389 0 6,435 0 30,824 Windows 0 113,421 0 70,014 0 183,435 Electric Total 0 30,044,477 6,656,343 3,066,023 77,251 39,844,094 Total Savings 1,365,006 67,929,672 81,859,083 7,526,283 542,880 159,222,924 No. of Applications 0 6,741 2,713 1,291 58 10,804 

### Table 1: Summary of Vendor Provided Data (PY2006)Massachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 3



### ________________________________________________________________________ The following table provides a breakdown of vendor provided program data by fuel type, in therms 1 

### ; which is the unit of measurem ent used to report impacts later in this report. Other fuel savings is not provided in this table but is an additional 3,128,692 kBTU; or 31,287 therms in 106 homes. It is interesting to note the increased average savings per participating home for oil (180 therms per home) as opposed to ga s (142 therms per home).

Oil Gas Electric Tracking Savings (kBTU) 49 ,077,055 67,173,083 39,844,094 # Homes Treated 2,727 4,739 10,804 Avg. Savings per Home 179.9 therms 141.7 therms 1,081 kWh 

### Table 2: Summary of Vendor Data by Fuel with per Home Savings (PY2006)

1.1 Objectives The overall evaluation work performed in 2007 was designed to accomplish three primary objectives. Each of these is provided below.

Objective 1: Perform QA/QC visits and provide feedback on energy efficiency measures as installed by vendors. (Appendix A) Objective 2: Provide specific feedback and im pacts for targeted oil and gas measures provided through the program as well as all electric measures. (Appendix B) Objective 3: Provide a reality check of gas progra m impacts at a high level. This was done through a billing analysis of natural gas customers. (Appendix C)

The first two study activities have been reported out previous to this summary report (the QAQC results in October, 2007 and the measure level impacts in November, 2007). This report is intended to present and synthesize the results of all three activities in a single document along with pertinent final conclusions and recommendations.



## 2 Methodology Figure 1 presents a brief overview of th e 2007 MassSAVE evaluation activities. As discussed above, three ac tivities were performed in this study designed to achieve the three objectives. These acti vities are summarized below and include the performance of QA/QC work, measure level impact work and a natural gas billing analysis. A detailed discussion of the various methodologies employed can be found in the appendices to this report.

1 To convert the values from KBTU to therms, we di vided KBTU by 100. A therm is equal to 100,000 BTUs.Massachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 4



### ________________________________________________________________________Work performed April through December, 2007 Overview of 2007 MassSAVE Evaluation Work

Natural Gas Billing Analysis QA/QC and Measure Impact Work

Data acquisition of 2007 measure level activity by participant from each program vendor

Sampling of 90 homes to represent vendors and largest

measures contributing to energy savings Acquisition of gas billing data for participants and

non-participants from

natural gas utility

Visits to 90 homes with appropriate blower door testing, infrared imaging and audit data collection Performance of timeseries comparison/cross sectional design billing analysis

Performance of DOE-2 simulation modeling and other engineering analyses to estimate measure level oil,

gas and electric energy savings Performance of timeseries comparison/cross sectional design billing analysis

Documentation and delivery of

QA/QC and measure level results to study sponsors. Documentation and delivery of billing analysis results to study sponsors.

Synthesis of all evaluation efforts into a single summary report.



### Figure 1: Overview of 2007 MassSA VE Evaluation Work Approach

2.1 Summary of QA/QC and Measure Level Impact Methodology To achieve the QA/QC measure level impact objectives, a total of 90 homes were selected for visits. These homes were se lected from vendor data based upon the presence of measures installed that had be en designated as a significant energy savings measure 2 from 2006 DOER data. Generally, we sought to perform

2 Air Sealing, Attic Insulation, Wall Insulation, Thermostats, and Heating Systems. Refrigerators were also later targeted for purposes of determining electric sa vings. Lighting measures were not targeted due to their ubiquity but were analyzed wherever encountered.Massachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 5



### ________________________________________________________________________ allocation of the 90 available site visits proportional the distribution of the program savings among the vendors. Ho wever, we had a minimum threshold of 6 homes randomly selected for each vend or to ensure a reasonable assessment of quality of measure installation practices . The final sample design is provided in Table 3 below where the ‘pri’ column s show the primary measure for which a home was selected for visit and the to tal columns show the total number of measures observed.

Major Measure/Vendor CET CSG HDMC RISE AES/Unitil Total Pri. Total Pri. Total Pri. Total Pri. Total Pri. Total Pri. Total Attic Insulation 2 3 5 16 7 12 4 9 2 2 20 42 Air Sealing/Weatherization 2 4 5 15 7 16 4 8 2 3 20 46 Heating System 0 0 5 9 0 0 0 0 0 0 5 9 Thermostats 3 4 5 6 7 7 1 1 2 2 18 20
Wall Insulation 3 4 5 9 7 9 2 3 0 0 17 25 Supplemental Refrigerators 0 0 0 0 6 7 4 6 0 0 10 13 Total 10 15 25 55 34 51 15 27 6 7 90 155


### Table 3: Final On-S ite Sample Design
At each home, all measures installed were observed fo r QA/QC purposes but only primary gas or oil measures and all el ectric measures were targeted for the measure level impact analysis. At each site, data was collected and diagnostic testing was performed as needed to inform simulation or re-engineering analysis in the interest of providing the energy im pacts of the targeted installed measures and to provide feedback on the quality of measure installation. This included blower door tests, lighting loggers, infra- red scans of wall insulation applications (when possible) and full home audits. On ce all data was collected, savings work was performed for each electric measure in each home and the major gas and oil measures along with QA/QC assessments fo r all measures encountered. As part of the on-site, the audito r also asked the homeowner questions regarding their perception and experien ce with the program.

Measure level impacts for the targeted oil and gas measures and majority of electric measures were calculated usin g DOE-2 models. DOE-2 is a detailed hourly building energy analysis tool that we used to predict the energy use of each measure in each home before an d after the measure was applied. RLW used six typical home models in estimati ng the savings based upon the condition observed on-site (including blower door results) and baseline (pre-retrofit) conditions. These six models represen ted the various home types encountered in the field with respect to enco untered heating and cooling system configurations. The models were calibrate d with actual gas and electric billing data. More detailed information on the savings approach for all measures and fuels is contained in Append ix B of this document.Massachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 6



### ________________________________________________________________________ 2.2 Summary of Billing Analysis Methodology The billing analysis uses a participant group and the control group and employs a “time-series comparison/cross sectiona l experimental design”. The time series/cross sectional design helps to re duce concerns about self-selection bias and free-ridership and helps the evaluation achieve internal and external validity. Internal validity means the evaluation is conducted in a manner that allows the results to isolate the impact of the activity being stud ied. When other factors are not recognized, the changes attributed to the program may be the result of other phenomena. For example, if the expe riment does not recognize the dynamic nature of a participant’s operational or end-use characteristics, their change in usage could be explained by changes in other participant characteristics.

In addition, the research design can help achieve external va lidity by ensuring that the results are representative of a la rger population of interest, allowing for the findings to be generalized. For ex ample, for the MassSAVE analysis, the information determined by a sample of participants, and the corresponding control group, permits the evaluation to represent the total program impacts.

The MassSAVE natural gas energy im pacts were determined through a multivariate regression (MVR) analysis . A more detailed discussion of the methodology is provided in Appendix C. The MVR uses the temperature normalized annual consumpt ion (NAC) for the particip ants and representative control group and tracking system data. The regression protocol used was a comprehensive and systematic approach that has been applied with great success to the analysis of market base d programs. The regression protocol consists of six steps that result in the selection of an optimal model that accurately quantifies the program impa ct. The six steps of the regression protocol follow.

Step 1: The Simple Model: During this step an initial regression model was developed using ordinary least squares (" OLS"). This simple model determined the effect of one important variable (i.e ., the participation indicator variable status, or the participant’s engineering estimate of savings) on savings while controlling for all ot her variables.

Step 2: Regression Diagnostics As a resu lt of the residual standard deviation being related to the size of the customer 's gas usage or demand, one regression assumption most often violated is that th e standard deviation of the error terms, (or "residuals") is not constant across th e range of predicted values. When the standard deviation residuals are related to the predicted values, the model is said to be "heteroscedastic." Heteroscedasti city can often be detected in crosssectional models used to analyze DS M program impact. During this step, verification that the regression assu mptions are valid was performed and anyMassachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 7



### ________________________________________________________________________ added multivariate regression analyses to address this issue were performed under a weighted least squares ("WLS") approach. Step 3: Weighted Least Squares: As discussed above, one of the fundamental regression assumptions is that the stan dard deviation of the error terms (or residuals) has a constant va riance across the range of predicted values. When the residuals are related to the predicte d values, the model is said to be heteroscedastic. When heteroscedasticity is present, an ordinary least squares (OLS) approach to establishing the re lationship between the dependent and independent variables may be inappropriate. Weighted least sq uares (WLS) is one approach to correct for heteroscedasticity in regression analysis.

According to econometric theory, WLS pr ovides the most reliable estimate of savings and an accurate measure of the re sulting reliability. The theory of WLS depends on a correct specification of the heteroscedasticity. The theory assumes that a positive-valued variable can be sp ecified; say z, such that the residual standard deviation is proportional to z. Usually, z is taken to be some measure of size (for example, the pre-retrofit NAC consumption). These results of the MassSAVE models suggested that the e rror terms may be heteroscedastic. Accordingly, weighted least squares (“WL S”) regression was used to develop the final models.

Step 4: The Unabridged Mode l: During this step an initial regression analysis (using OLS, or if more appropriate, a WLS approach) is performed. A multivariate regression full analysis mode l, the unabridged model, is developed. This model consists of any available va riable that may be significant in the determination of the program impacts. After the development of the unabridged model, a residual analysis is performed. This analysis is used to diagnose, analyze, and correct if necessary, any outlie rs. Under WLS, this step is used to determine the best gamma for use in creating the optimal weights.

Step 5: The Refined Model: The fifth step develops the refine d model, based on the unabridged model, and if using WLS, the optimal value of gamma. A step wise regression approach is used to el iminate any insignificant variables of the unabridged model. After th is step, the refined mode l will feature only those variables that have mathematical signific ance in the determination of the energy or demand savings. Step 6: Calculation of Energy Savings: The final step in th e analysis estimates the energy savings by using the resultant models. In this step the savings are calculated using both the unabridged an d the refined models to examine the impact on savings of removing the st atistically insignificant terms.Massachusetts Final Summary QA/QC and Impact Study Report

___________________ Page 8



### ________________________________________________________________________ 

## 3 Results

The following sections pres ent results for each aforem entioned study objective.

3.1 Objective 1: Perform QA/QC visits on installed measures. As discussed earlier, a total of 90 qualit y control visits were performed at homes of 2006 participants in the MassSAVE Program. These inspections were conducted during the peri od of June through Augu st, 2007. The table below summarizes the QA/QC visits. It shows the number of measures assessed, the status of the measure as observed and determined by RLW au ditors, the typical installations found onsite, and a su mmary of issues encountered.

In general terms, we observed the majority of measures to be operating as intended and well installed. There were two instances where major measures were reported in the tracking system but were not found onsite during the audit. In one of these instances, wall insulation was reported to have been installed, but the customer informed the auditor that attic insulation had been installed instead of wall insulation. In another in stance the tracking system reported that basement insulation was installed at a si te, but upon inspectio n of the basement, none was found.

Programmable thermostats were found still installed in all cases with the exception of one home in which the homeowner had removed it. However, there were several instances in which th e thermostat was observed to be in manual mode and several more instances where the customer had set their programmable thermostat to a schedule that mimicked the schedule they had used with their old manual thermostat.

Other measures such as lighting and ai r sealing are reported below as not installed when the homeowner didn’t re member the measure being installed and the auditor could not find evidence of the work. It should be noted that the customers often had trouble remembering what bulbs or air sealing measures had been installed through the program. Evidence of air sealin g can be difficult to find if it is installed in an attic, or sealing small cracks in the basement. In addition, the installed lighting measures could have been removed. As a result of these possibilities, the vendor may have installed these measures even though they were not found to be inst alled during the QA/QC audit.Massachusetts Final Summary QA/QC and Impact Study Report ___________________________________ Page 9



### ______________________________________________________________________________________ Program Measure Qty Assessed

Onsite Rating/Status

Good Fair Poor Not Installed Removed Assessment and Issues Encountered

Air Sealing/ WXN 46 29 7 3 6 1 Majority of installations done very well. Multiple instances where customer was unable to recall weatherization measure installation. Multiple instances where the home showed potential for additional tightening (9 homes). Two instances where the weather-stripping fell off.

Attic Insulation 40 36 4 0 0 0 Nearly all installations of high quality. Isolated instances where application was not uniform. Two instances where the stairs and/or doors leading to the treated attic area were not insulated.

Basement/ Floor Insulation 10 7 2 0 1 0 Vast majority of installations done well. In one home the insulation was not installed. In another home a small section had fallen out of place, and in another home some compression was observed.

Domestic Hot Water 20 19 0 0 1 3 Majority of installations installed and operating as intended. Three instances where customer removed the measure (2 aerators and one low flow showerhead) and one instance where customer was unable to recall the installation. Duct Insulation 4 4 0 0 0 0 All installed. Heating System 9 9 0 0 0 0 All verified to be ENERGY STAR compliant and fully operational.

Lighting 47 45 0 0 8 9 Many still installed and operating. At eight homes the customers were unable to recall the installation of seventeen bulbs and they were not found by the auditor. At nine homes customers removed a total of thirteen bulbs. Refrigerator 13 13 0 0 0 0 All verified to be ENERGY STAR compliant and fully operational.

Thermostat 20 19 0 0 0 1 Vast majority well installed. One instance where customer removed the measure and three homes in which the thermostat was on a manual setting.

Wall Insulation 25 18 5 1 1 0 Generally installed well based upon evidence available. A few instances where gaps in treatment were found. Two instances where drill holes not well covered or patched and one instance where exterior tiles had been broken and not repaired. The one instance where insulation was not installed appeared to be an error in the tracking data where attic insulation was installed in place of wall insulation.
Windows 3 3 0 0 0 0 All well installed.
Total 237 202 18 4 17 14 

### - Table 4: Measure Ratings and Status SummaryMassachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 10



### ________________________________________________________________________ Table 5 presents the satisfaction rating s of the various measure types installed by homeowners visited in the QA/QC sample . The rating scale ranges from 1 to 6 with 1 representing “Very Dissatisfied” and 6 representing “Very Satisfied”. Also provided are the satisfaction resu lts from the 2004 QA/QC work, in which the same satisfaction question and rating scale was asked of 2003 program participants. There is only one measur e type that experienced a substantial reduction in satisfaction rating fr om the 2004 work, which was basement insulation. This was due pr imarily to a respondent th at was upset that another inaccessible portion of the basement co uld not be treated. One other measure had a slight reduction in average satisfac tion ratings while the remainder of the measures had improvements in ratings; including im provements of ten percent or more for three measures (attic insulati on, air sealing, and lighting). In 2004, it was suggested that satisfaction ratings be trended over time as a metric that is regularly tracked as a measur ement of program success.

Satisfaction Rating 1 (very dissatisfied) to 6 (very satisfied) Measure n 2004 Average n 2007 Average Refrigerator 14 5.9 13 5.8 Attic Insulation 26 5.0 40 5.6 Air Sealing 26 4.8 35 5.4 Basement Insulation 13 5.2 9 4.2 Wall Insulation 22 4.8 25 5.1 Lighting 37 4.8 47 5.5 Domestic Hot Water 16 4.9 23 5.1 Heating Systems - - 9 5.6 Thermostats 6 5.5 20 5.6 Table 5: Comparison of 2004 and 2007 Satisfaction Levels

3.2 Objective 2: Provide feedback and impacts for targeted measures. As part of the 90 on-sites performe d and discussed above, RLW estimated savings for targeted oil and gas measur es and all electric measures. Table 6 provides a summary of all program fossil fu el savings by primary measure, with all other program measures not analyzed included as a bundle. Appendix B provides additional breakdowns of resu lts, including oil versus gas and by vendor. The last row above the total shows the RLW calculated interactive heating penalty in therms for all lighting and refrig erator applications also analyzed as part of the on-sites. Base d upon those measures analyzed in this study, we estimate the total program fo ssil fuel impact of the program to be 905,781 therms at +/- 18% at 90% conf idence. This would reflect a 76% realization rate for the programs gas an d oil savings. Combining these overall results with the total number of fossil fu el homes in the population provides a per home savings estimate of nearly 160 therms according the vendor savings estimates and 120 therms according to the RLW savings work . Detail on theMassachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 11



### ________________________________________________________________________ reasons behind the various measure level results can be found in Appendix B of this report.

Measure Vendor RLW Realization Rate Relative Precision Air Sealing/Weatherization 296,254 106,277 36% +/- 30% Attic Insulation 317,990 234,852 74% +/- 25% Wall Insulation 322,744 383,109 119% +/- 24% Heating System 58,687 122,531 209% +/- 24% Thermostats 52,880 22,996 43% +/- 33% All Other Fossil Fuel Measures 145,233 145,233* 100%* N/A Interactive Penalty from lighting and refrigerators 0 -109,217 N/A N/A Totals 1,193,788 905,781 76% +/- 18% *Savings assumed to have a 100% realization rate as they were not analyzed as part of this study. 

### Table 6: Summary of Savings (the rms) for Fossil Fuel by Measure Total program electric energy (kWh) for a ll measures, including those installed in fossil fuel heated homes, are shown in Table 7, below. Most vendors did not count electric savings for measures primaril y installed to produce savings in fossil fuel heated homes, which pushed the realization rate for ‘All Other Measures’ above 100% as RLW did include electric sa vings due to central air conditioning systems found in many of the homes. These results produce a per home electric savings estimate of nearly 1,080 kWh a ccording the vendor savings estimates and 502 kWh according to the RLW savings work.

Summary of Electric Impacts (kWh) Vendor RLW Realization Rate Relative Precision Lighting 10,047,836 3,814,420 38% +/- 33% Refrigerators 1,146,811 1,071,685 93% +/- 13% All Other Measures 482,989 540,117 112% +/- 29% Totals 11,677,636 5,426,222 47% +/- 20% 

### Table 7: Summary of Savings (kWh) fo r Electric Measures in the Program Lighting represents the majority of the electric savings in th e program, and thus we calculated demand and estimated coin cident demand reductions. The total connected demand reduction for lighting in the sample is 15.8 kW, based on 324 CFLs observed and monitored by RLW at 49 sites. The inst allation of bulbs averaged 6.6 CFLs per home based on RLW data, where the vendor data indicated 7.2 CFLs per home.

The figure below shows the weighted (by connected lighting Watts reduced) averages of monitored on-t imes for all 291 loggers that were utilized for this project. It is important to recognize th at this project was conducted during the Summer and early Fall, so that there were no Winter measurements of lighting usage. Also, it is important to note that these load shap es do not represent typical lighting load shapes for a typical home with all lights considered. Instead, they are true load profiles for the sample of lights observed and metered on-site.Massachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 12



### ________________________________________________________________________ Average Lighting Usage Profiles

0.00 0.02 0.04 0.06 0.08 0.10 0.12 0.14 0.16 0.18 0.20

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Hour EndingLighting Fraction On Weekdays

Weekends



### Figure 2: Sampled Lighting Logger Profiles for the Average Weekday and Weekend Coincident peak summer demand redu ction for the samp le, based on the average of the four hours from 1 to 5 PM during a typical day in July was found by RLW to be 0.949 kW. This is base d upon a percent on -time during this window of 6.0%. Total program impacts may be inferred by multiplying these estimates by the ratio of total program po pulation (homes, or applications) to the sample population. This ratio is 327. 4 (16,042/49 from Appendix B), so the unadjusted program-wide peak demand reductions are about 311 kW for the summer ISO window. Note that this procedure for calculating program-wide savings fails to adjust the estimates for any sampling bias that may be present in the RLW sample. Without ve ndor estimates of these same demand reductions, it is impossible to estimate the sampling bias.

Since the logger data from this study were gathered during the summer months, adjustments must be made to estimate winter peak reductions. From a study performed for NECPUC in 2007 3 

### , the weighted coin cident peak summer coincident factor was estimated to be 8. 2% and the winter was estimated to be 29.8%. Using these factors, we can calcul ate a summer to winter factor ratio of 3.63. Applying this to our calculated su mmer coincident factor of 6.0% provides us with an estimated winter coincide nt factor of 21.8% and a calculated Coincident peak winter demand reduct ion of 3.4 kW for the sample (21.8% *15.8 kW).

3 “Coincidence Factor Study Residential and Co mmercial & Industrial Lighting Measures”, Prepared for the New England State Program Working Group (SPWG), RLW Analytics Inc. Spring 2007.Massachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 13



### ________________________________________________________________________ 3.3 Objective 3: Provide a check of gas program impacts at a high level. The check of gas program impacts wa s performed through a multivariate regression (MVR) analysis of natural ga s participant and non-participant billing data. This table shows the customer level results by strata. As this table shows, the stratification allows the results to be weighted to accurately represent the population.

The average net gas savings from the billing analysis is estimated to be 62 Therms, or 5.3% of the participant’s pr e-program NAC. The confidence interval around this estimate is from 53.9 Therms to 70.0 Therms. This is a ±12.0% relative confidence interval. An actu al savings of 62 Therms is 54% of the tracking system savings estimate (i.e., th e realization rate). It should be noted that this customer level result as well as the program level result below, is a net savings estimate.

Interestingly, the lowest strata (partici pants with tracking less than 175 Therms) had a realization rate of 84%. The savings estimates increase with the stratum 4



### . However, the realization rate decreases. This indicates that the accuracy of the tracking estimates of savings decrease as the estimates increase. The largest strata tended to have more attic and wall insulation measures installed than the lower strata

Pop.

Strata N Low Limit High Limit Pop. Analysis Data Set Parts Control Group Estimated Savings (ccf) Realization Rate 1 4,145 - 175 60 65 1,137 1,080 50 84% 2 472 175 640 350 351 1,219 1,158 138 39% 3 120 640 10,000 1,102 921 1,742 1,812 165 15% Total 4,737 115 115 1,161 1,107 62 54% Stratum Limits (Therm Savings) Average Savings (Therm) PreNAC (Therm)



### Table 8: Final Results (Customer Level)

Table 9 presents the program level result s from the billing analysis. The net program gas savings is estimated to be 293,416 Therms. The relative confidence interval around this esti mate is 255,374 Thermsto 331,457 Therms; which calculates to a ±13% interval.

Strata N Total Savings (Therm) Conifidence Interval (+/- therm) RCI 1 4,145 208,594 21,238 ±10.2% 2 472 65,013 8,177 ±12.6% 3 120 19,809 8,626 ±43.5% Total 4,737 293,416 38,042 ±13.0%


### Table 9: Final Result s (Program Level)

4 Note that the high limit for stratum 3 is a stratum limit and not an actual savings value from the data.Massachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 14



### ________________________________________________________________________ 

## 4 Conclusions and Recommendations
Based upon the performance of the on -sites, QA/QC results, measure level performance analysis and billing anal ysis of gas savings we believe the MassSAVE Program can be characterized as providing a high quality value added service to customers that is producing su bstantial levels of electric, gas and oil energy savings. The followi ng table provides the saving s estimates as calculated from the measure level on-site work and the billing analysis and compares them to the overall program and per home estima tes calculated from the vendor data. Vendor reporting to the DOER for all of the sponsors sh ould be revised to reflect the findings in this study. Since a ll sponsors do not use vendor data for institutional reporting, they need to exercise care when systematically incorporating results from these studie s into savings calculations used for institutional reporting.

The measure level on-site work estimate d gross savings for all fossil fuels and provides a program level savings estima te of 905,781 therms and a per home estimate of 119.6 therms with a precision of ±18%. The measure level on-site work also provides an esti mated gross savings for elect ricity of 5,426,222 kWh at the program level and a per home estima te of 502 kWh with a precision of ±20%. The billing analysis measured net savings for natural gas only and provides a program level savings estima te of 293,416 therms and a per home estimate of 62 therms with a precision of ±13%.
Estimate Source Program Level Per Home Precision All Fossil Fuels (therms)
Vendor 1,193,788 157.7 N/A
RLW On-Site Measure 905,781 119.6 ±18% Natural Gas (therms)
Vendor 545,296 115 N/A Billing Analysis 293,416 62 ±13% Electricity (kWh)
Vendor 11,677,636 1,080 N/A
RLW On-Site Measure 5,426,222 502 ±20%


### Table 10: Summary of Savings Impacts During the process of determining progra m impacts from the on-site work, there were opportunities noted in which vendor savings estimates coul d be adjusted or refined to reflect more accurate saving s in their tracking data. These are presented in detail on a case by ca se basis in Appendix B and further summarized below. We do believe that th e sponsors should co nsider the idea of utilizing a single tool consistently among all vendors to calculat e savings. Such a tool should include all fuel savings an d would ensure a regular mechanism isMassachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 15



### ________________________________________________________________________ used across vendors. To the extent that savings are going to continue to be aggregated by DOER and the program co st effectiveness depends on program wide savings assumptions, using a single savings calc ulation methodology across vendors makes a great deal of sense. Should the development of a single tool among vendors be d eemed too resource intensive and logistically prohibitive gi ven the structure and contracts in place among the many sponsors and vendors, an alternative is to coalesce assumptions to the extent possible for use among all ve ndors and savings work. We provide vendor specific recommendat ions below as well as measure level unitized savings work in Appendix B report to facilitate change s in savings work at whatever level of adjustment the pr ogram sponsors are able to prioritize pending the determination of available resources and time.
Measure Vendor Suggestion

Air Sealing HDMC Reconsider use of assumed savings per home in lieu of calculating savings uniquely for each home treated*.

CET Consider reviewing the calculatio n of savings as the impact per square foot appears to be low compared to other vendors.

Attic Insulation RISE Consider reviewing the calculatio n of oil savings, as the per application (home) savings estimates appear to be low.

All Consider the performance of a targeted review of how vendors are estimating attic insulation savings, especially when existing insulation is in place*. Heating system CSG Reconsider use of assumed sa vings in lieu of customized estimates of savings.

Thermostats All Consider implementing a selection process where programmable thermostats are in stalled in homes where they are desired and consider exclud ing applications where setback is already being practiced with the existing thermostat*.

Lighting All To the extent that sockets in areas of higher use are available at the time of CFL installation, those areas should be prioritized for treatment. Also consider tracking the interactive effects for lighting*.

* Quality control activities that that co uld be expected to reduce some of the specific savings estimation problems. 

### Table 11: Recommendations for Savings Estimate Improvements Based upon both the on-site measure leve l impact work and the QA/QC work, we provide additional conclusions below, wi th suggestions made as appropriate.

 Conclusion 1: Early in the evaluation process, RLW gathered 2006 program data from both DOER and the individual vendors. We anticipated the savings activity from both sources to align closely with one another. However, the vendor provided activity was noted to be 2.9% lower than the DOER activity with va rying vendor level differences; including two thatMassachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 16



### ________________________________________________________________________ were more than 10% off. While th ere are several reasons why this may occur, such as changes in the credit ing of savings among programs after the data is sent along to DOER, it is desirable that DOER and vendor records of program activity be aligned and consistent.

o Suggestion 1a: If this is the result of program savings being credited differently after data is submitte d to DOER, DOER might consider providing more time for vendors to settle their savings allocation process before providing their records of program activity.

o Suggestion 1b: The sponsors might consider the development of a single database that is accessible to all vendors and other interested parties that tracks all program activi ty in a consistent manner. Such a system would allow DOER and other parties to query the data for activity and would remove any e rror between the DOER gathered savings and program activity and the ac tual vendor data in the system.

 Conclusion 2: An integral part of our on-s ite activities was in verifying vendor provided measure level data. In working with the vendor data for the MassSAVE Program, several instances occurred in which tracking data appeared to be incorrect due to input errors, mis-assigned fuel types or mis-categorized measure type.

o Suggestion 2: To the extent that savings measure type categorization and fuel savings is im portant to properly characterizing savings at the vendor level and at the aggregate le vel (DOER), we recommend a program improvement goal of establishing checks in the vendor tracking systems to ensure th e proper association of savings to measure types and fuels.

 Conclusion 3: In the analysis of measure savings from the on-sites, it was noted that several vendors do no t track electric savings for measures installed primarily to generate sa vings in fossil fueled homes.

o Suggestion 3: Electric savings should be calculated and tracked for those major measures that are anti cipated to provide electric cooling kWh savings in addition to the primary fossil fuel savings.

 Conclusion 4: In general, the installations observed on-site were determined to be good, with occasional and isolated observations of major measure installations that either co uld not be verified or could be improved and some instances of tracki ng system errors. These findings suggest that major measure installations are being installed within reasonable quality tolerance bounds, providing a solid foundation for continuing measure installation improvements.

o Suggestion 4a: At a high level, we reco mmend that an ongoing QA Process be initiated as an integral part of the MassSAVE Program. The purpose of this activity would not be to “police” the program installations, but rather to ensure th at the current quality of installation is sustained, that ongoing improvem ent of major measure installationsMassachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 17



### ________________________________________________________________________ are sought and to make certain th at some of the smaller measures such as lighting are getting installed and are generating energy savings for the program. This process should be incorporated in a way that seeks continual process impr ovements through its outcomes. A specific structure for such a QA pro cess is provided in Appendix A of this report.

o Suggestion 4b: While a high level QA proce ss would likely benefit the savings being generated in the program, some specific measure level QA work available for consideration ca n also be expected to result in higher energy savings. Specifica lly, the sponsors might consider implementing a selection proce ss where programmable thermostats are installed in homes where they are desired and have the capability to generate the most savings thro ugh setbacks that are not already behaviorally performed with the curre nt thermostat. In addition, when thermostats are installed, the co ntractor should make sure the homeowner knows how to use the ther mostat properly. Further, to the extent that sockets in areas of higher use are available at the time of CFL installation those areas sh ould be prioritized for treatment.

 Conclusion 5: Participants are reporting sati sfaction with many elements of the program and its delivery.

o Suggestion 5: As the program continues to evolve and the QA/QC process matures, continue to track me asure satisfaction and consider it an important metric that should be monitored regularly.

 Conclusion 6 : Some vendors track installations by job as opposed to number of units installed.

o Suggestion 6 : To facilitate any subsequent QA/QC work, ex-post savings work or measure level a ggregation work we recommend the tracking of specific quantities of units installed among these measure types (lighting, weatherization, DHW) and locations installed.

 Conclusion 7: Several homes visited had si ngle doors treated with weather-stripping, small quantities of CFLs installed and/or single installations of DHW measures such as faucet aerators or low flow showerheads. It was noted at these homes that other opportunities for these measures existed but were not treated.

o Suggestion 7: Consider or encourage vend ors to install additional sets of weatherization measures (weather-stripping), CFLs and/or DHW measures to take advantage of all opportunities at a home.

 Conclusion 8: Nearly all attic insulation was well installed at the sites visited and few instances of wall in sulation gaps were evidenced. However, in a few isolated instances , attic insulation was observed to have uneven depth and at a couple of homes wall insulation was notedMassachusetts Final Summary QA/QC an d Impact Study Report _________________ Page 18



### ________________________________________________________________________ either to have not been done by the homeowner or to have some gaps in coverage.

o Suggestion 8a: We believe the isolated instances of gaps in wall insulation and uneven attic insula tion do not warrant a substantial recommendation. A brief reminder for vendors and contractors may be appropriate at this time regarding the need for uniform depth of attic insulation to achieve proper material density (and R-Value) in treated homes.

o Suggestion 8b: The timing of on-sites in the summer made interpretation of infra-red imaging for the wall insulation difficult due to low temperature differentials between the indoor and outdoor temperatures at many homes. We suggest that during the colder months a targeted series of home s with wall insulation treatment undergo infra-red imaging in the inte rest of providing more conclusive evidence of their proper installation.22 Haskell Street, Cambridge, MA 02140 Phone: (617) 497

- 7544 Fax: (617) 497

- 7543 www.nexusmarketresearch.com 

# Market Progress and Evaluation Report (MPER) For the 2007 Massachusetts ENERGY STAR ® 

# Lighting Program



# Final

July 1 , 2008

Volume 1 Findings and Analysis

Submitted to: 

## Cape Light Compact Massachusetts Electric Company Nantucket Electric Company NSTA R Electric Western Massachusetts Electric Company Unitil

Submitted by: 

## Nexus Market Research, Inc. RLW Analytics, Inc. Dorothy ConantMassachusetts ENERGY STAR Lighting MPER 2007

Nexus M arket Research 

### Contents

Executive Summary

................................ ................................ ................................ ........................ 2 Findings

................................ ................................ ................................ ................................ ....... 2 Program achievements

................................ ................................ ................................ ............ 2 The current state of the market

................................ ................................ ............................... 3 Torchieres

................................ ................................ ................................ ............................... 7 Involvement of manufacturing and retail partners

................................ ................................ .. 7 NCP Program Procedures

................................ ................................ ................................ ....... 8 Technical Considerations

................................ ................................ ................................ ........ 9 Conclusions

................................ ................................ ................................ ............................... 11 Recommendations

................................ ................................ ................................ ..................... 12

Appendices (In Separate Volume) Appendix A Product Quality Assessment and Technological Developments Appendix B: Consumer Survey Appendix B

- 1: Consumer Survey Report Appendix B

- 2: Phone Survey Que stionnaire Appendix C: Retailer Survey Appendix C

- 1: Retailer Survey Memo Appendix C

- 2: Retailer Survey Data Collection Tools Appendix D: Process Documentation Appendix D

- 1: Process Documentation Memo Appendix D

- 2: Process Discussion Guides Appendix E: Da tabase Analysis Appendix F: Lighting Price Analysis Memo Appendix G: 2006 Net

- to

- Gross Memo Appendix H: Socket Count Appendix H

- 1: Socket Count Memo Appendix H

- 2: Socket Count Data Collection ToolMassachusetts ENERGY STAR Lighting MPER 2007 Page 2

Nexus M arket Research 

## Executive Summary This report summarizes the results and implications of evaluation activities completed as part of the Market Progress and Evaluation Report (MPER) for the 2007 Massachusetts ENERGY STAR  

### Lighting Program by Nexus Market Research, Inc. (NMR), RLW Analytics, Inc. (RLW), and Dorothy Conant. It reviews key findings, provides an analysis of the degree to which the compact fluorescent lamp ( CFL ) , fixture, and torchiere markets have been transformed, and offers several recommendations for consideration by the Sponsors: Cape Light Compact, Unitil, National Grid, NSTAR, and Western Massachusetts Electric. A full report of findings and analysis for the 2007 MPER is provide d to the Sponsors under separate cover.



## Findings A wide variety of research activities were conducted in support of the program evaluation. The results can be categorized as documenting the achievements and costs of the program, describing the current st ate of markets for energy

- efficient lighting products, outlining the involvement of manufacturing and retail partners in the program, and identifying technical considerations. Key points in each of these categories are listed below.
Program achievements

• S ince 2002, the Sponsors’ Lighting Program has included three basic components:

− The ENERGY STAR  

### Lights catalog (and website)

− Instant rebate coupons

− Negotiated Cooperative Promotions (NCP) In 2007, 89% of the products distributed through the Massachusetts ENERGY STAR Lighting program came from the NCP component.

• Program

- related shipments of CFLs have increased dramatically since the Sponsors first began offering a joint efficient lighting program

— from 158,000 in 1998 to 2.6 million in 2007. 1



### • The multiple program components

— the catalog, retail coupons, and the Negotiated Cooperative Promotions

— appear to meet somewhat different needs of the market and provide complementary opportunities for consumers and for retailers.

• Program

- related shipments of ENERGY S TAR

- qualified fixtures and torchieres have decreased, after peaking in 2004.

− Relevant shipments of ENERGY STAR

- qualified fixtures grew from 46,000 in 1998 to 119,000 in 2004, falling to 39,000 in 2007.

− Relevant shipments of ENERGY STAR

- qualified torchieres increased from 774 in 1998 to 38,000 in 2004, falling to 3,300 in 2007.

• The program appears to be highly cost effective.

− Despite the fact that total program spending decreased to $7.3 million in 2007 after peaking at $12.8 million in 2004, program expens e per product remains stable and low at $2.82. It is worth noting that these figures are based only on products directly

1 As documented through supporting sales data for coupon, catalog, and NCP markdown programs and through documen ted shipping confirmation for the NCP buydown program.Massachusetts ENERGY STAR Lighting MPER 2007 Page 3

Nexus Market Research 

### incentivized through the program, and there is strong evidence that the program has yielded significant spillover savings. The majori ty of the budget is directed to customer incentives, including NCP buydown and markdown costs. Collectively, the percentage of Sponsors’ budgets allocated per the regulatory reporting categories to planning and administration, marketing, and technical supp ort dropped from 61% in 2003 to 39% in 2007, with a commensurate percentage increase going to customer incentives during the same time period.

− Lifetime savings resulting from the 2007 program year are estimated to be 1,665 ,000 MWh, as adjusted for hour s o f use, in

- service rates, free ridership, and spillover. The cost per average MWh saved is estimated to be about $ 5 . CFLs, which account for 98% of program sales in 2007 account for about 95% of these savings.

The current state of the market Here, we summ arize findings with regard to customer awareness, product quality, environmental considerations, customer satisfaction, and pricing, as well as trial and usage and sales volumes.

• Customer awareness for CFLs is nearly universal, but awareness of energy

- savi ng fixtures has remained stable since 2005.

− Ninety

- four percent (94%) 2



### • Product quality appears to be relatively high. of respondents were familiar with CFLs in 2007

— compared with 77% in 2005, and 28% were familiar with energy

- saving fixtures in 2007

— compared with 29% in 2005.

− Almost half of retailers report no product returns through the program; participating retailers with returns estimate an average rate of 2.5%, mostly because products were broken or defective at the time of purchase or because the customer did not like the look of the product after it was installed. EFI 3



### − While the product quality of standard CFLs has greatly improved, program administrators and manufacturers agree that the quality of some types of specialty CFLs, such as three

- way, dimmab le, and recessed CFLs could be improved. Two factors to note in this regard include: reports that in the 2007 catalog program, the return rate is about 3 to 4%. These return rates include products returned because they are damaged (as through shipping), defective , or because customers have changed their minds about their choices.

 All reflector bulbs were required to undergo re

- qualification for the ENERGY STAR program in 2005. This requirement appears to have weeded out many of the poorly perform ing products.

 The dimming capability of CFLs continues to be problematic at the low end of the dimming range, when color shifts become apparent and shut

- off

2 Except where otherwise noted, the results presented for 2005 through 2007 are weighted by number of households in Massachusetts or the number of products. The demographic characteristics of telephon e survey respondents do not always mirror exactly those of the general population. The weighting system helps correct the differences so that product counts better reflect the purchasing habits of the Massachusetts population. Results from 2002 through 200 4 are not weighted, which somewhat limits comparability of results.
3 The Energy Federation Inc. (EFI) is a contractor hired by the Sponsors to provide program fulfillment services for the catalog, process rebate coupons for retailer reimburse, process inv oices for the NCPs for manufacturer reimbursement, and maintain program databases.Massachusetts ENERGY STAR Lighting MPER 2007 Page 4

Nexus Market Research 

### points are not consistent or as low as incandescents. Some manufacturers we interviewed are addre ssing these issues.

− Results from a separate measure life study conducted by NMR for the Sponsors and other residential lighting programs in New England suggest that CFL product quality improved over the study period from 2002 through 2005, but dropped in 2006. More specifically, except for 2006, each successive generation of CFLs studied survived longer than the last. The drop in 2006 could be a random event or may represent a shift in quality, but small sample sizes and a limited number of failures in recent years curtailed our ability to conduct meaningful statistical analyses to more fully explore survival rates over time

. 4



### − Independent testing of products has been supported by the Sponsors through the Program for the Evaluation and Assessment of Resid ential Lighting (PEARL) Board. 5 

### Our analysis in the 2005 MPER and 2006 MPER found that only a small proportion of CFLs sold in Massachusetts were delisted as ENERGY STAR

- qualified products as a result of PEARL testing cycles. 6



### − There is concern among some PEARL Board members that historical issues with the ENERGY STAR program’s timeliness and/or transparency of the de

- listing process might continue when it tak es over the testing process. Both the PEARL Board and the Consortium for Energy Efficiency (CEE) have recommended that the Department of Energy (DOE) allow comprehensive audits of the test results in order to ensure that DOE follows the appropriate proto cols regarding product nomination and procurement, testing, data reporting, marginal failures, disqualification appeals, and delisting. The PEARL Board has decided to conduct another round (Cycle 9) of testing in 2008, which will most likely be the final product testing cycle for the program. The ENERGY STAR program will assume oversight of a third

- party CFL testing and verification program in the new CFL
program r equirements, Version 4.0, which becomes effective in December 2008.

• Environmental considerations , based on the presence of mercury in CFLs, have become more important to CFL stakeholders , as evidenced by more media attention, consumer concerns, regulatory mandates, product offerings from manufacturers, manufacturer marketing strategies, disposal options, and ENERGY STAR specifications.

− In Massachusetts, the Mercury Management Act (sign ed July 28, 2006) phases out the use of mercury in products (not including CFLs) and regulates the disposal of products containing mercury (including CFLs). If recycling efforts do not meet targeted levels, the law requires lamp manufacturers to provide up to $1 million per

4 NMR, Inc. and RLW Analytics, Inc. Residential Lighting Measure Life Study, Final June 4, 2008. Submitted to New England Residential Lighting Program Sponsors.
5 The PEA RL program is chaired and coordinated by the Natural Resources Defense Council (NRDC), with a testing lab at the Lighting Research Center (LRC) at Rensselaer Polytechnic Institute in Troy, NY. PEARL was established to replicate the tests that manufacturer s are required to perform before submitting products for ENERGY STAR qualification in order to verify compliance with ENERGY STAR specifications. The sponsors include the Massachusetts energy efficiency program administrators through the Northeast Energy Efficiency Partnerships, other regional program administrators, as well as the EPA and DOE.
6 Sponsors monitor testing and screen incentivized products throughout the year to ensure that they are ENERGY STAR

- qualified. Sponsors also included a provision in the 2006 RFP to partners that “The Sponsors reserve the right to exclude specific ENERGY STAR qualified products from this promotion for any reason including results of independent, third

- party testing.”Massachusetts ENERGY STAR Lighting MPER 2007 Page 5

Nexus Market Research 

### year to the Massachusetts Department of Environmental Protection (MA DEP) for grants to municipalities and/or regional authorities that are collecting and recycling
mercury

- containing lamps. 7



### − The Version 4.0 specification for ENERGY STA R

- qualified CFLs, effective December 2008, sets mercury dosage and labeling requirements for qualified products. For lamps under 25 watts the maximum mercury dosage is 5 mg per lamp, and for 25 to 40 watt lamps the maximum dosage is 6 mg per lamp. Manufa cturers must also label product packaging with the ‘Hg’ symbol within a circle, include text that the ‘Lamp Contains Mercury,’ and list the website ‘www.epa.gov/bulbrecycling’ or ‘www.lamprecycle.org’ on the packaging. 8



### − The ENERGY STAR program has also p roduced a consumer information fact sheet on mercury in CFLs and a disposal protocol for spent bulbs. 9



### − Many CFL manufacturers are introducing products with dosing as low as 2 to 3 mg of mercury and best practice offerings from manufacturers currently inclu de products in the 1 to 2 mg range. These low

- mercury products are not universally available across all CFL product lines and they are more expensive to produce. There are concerns about product quality with decreases in mercury levels.

− Manufacturers ar e also responding to international interest in mercury. The European Union’s Restriction on Hazardous Waste Substances Directive (RoHS) adopted a mercury limit of 5 mg per bulb. 10



### − An industry

- wide solution to financing CFL recycling programs remains unresolved. A few energy

- efficiency programs or states, such as Wisconsin Focus on Energy and Efficiency Maine, h ave established coordinated regional recycling programs, and some manufacturers offer their partners the option of including plastic bags in product packaging for containment of used CFLs or offer collection bins or buckets that can be placed in retail loc ations for spent CFL collection and recycling. RoHS also includes restrictions on lead used in glass and solder. Japan, S outh Korea, and China also are developing similar mercury restrictions.

− The Sponsors see their role in dealing with mercury and recycling as primarily educational; they have been approached by recycling vendors but are hesitant to take responsibility for recycling efforts. Most b elieve the manufacturers and retailers (and the government) should be responsible for coming up with recycling programs and providing disposal sites.

− Significantly more consumer respondents in 2007 compared to previous years are aware of some type of haz ard associated with CFLs and disposal requirements; however, the vast majority of all current and previous users

— 77%

— are unable to identify any hazards associated with CFLs. The few respondents who were aware of

7 Fact Sheet, Summary: Massachusetts Mercury M anagement Act. Massachusetts Department of Environmental Protection. http://www.mass.gov/dep/toxics/laws/hglawfax.pdf (Accessed 4

- 18

- 07).
8 www.l amprecycle.org is sponsored by the National Electrical Manufacturers Association (NEMA), an organization representing many lighting manufacturers. The website serves as a clearinghouse for information on recycling fluorescent lamps.
9 http://www.energystar.gov/ia/partners/promotions/change_light/downloads/Fact_Sheet_Mercury.pdf (Accessed 4

- 29

- 08).
10 Directive 2002/95/EC of the European Parliament and of the Council, January 27, 2003 as reported in the Official Journal of the European Union, 13.2.2003. http://eur

- lex.europa.eu/LexUriServ/LexUriServ. do?uri=OJ:L:2003:037:0019:0023:EN:PDF (Accessed 4

- 29

- 08).Massachusetts ENERGY STAR Lighting MPER 2007 Page 6

Nexus Market Research 

### hazards note that the products require spec ial disposal (8%) or that the bulbs contain mercury (8%). Forty

- three percent of those who have used CFLs have also had to dispose of at least one bulb. Seventy

- two percent of these respondents simply threw the products away in the trash and 26% recycled the CFL or dropped it off at a recycling center. We calculate that about six million CFLs have been removed from service in Massachusetts since 1998, which could mean that 72% of those, or more than four million, have been thrown in the trash.

• Customer satisfaction with energy

- efficient lighting products is high. A large majority of consumers is satisfied with their energy

- saving products. Seventy

- six percent of those who currently use CFLs rate their satisfaction as a 4 or a 5 on a five

- point scale, a s do 78% of those who currently use energy

- saving fixtures. These results are similar to satisfaction levels in previous years.

• Trial and usage of CFLs has increased compared to 2005, but trial and usage of energy

saving fixtures has remained stable.

− Sev enty

- nine percent of all households in Massachusetts have tried CFLs and 76% currently use CFLs. In 2005, 64% of all households in Massachusetts had tried CFLs and 56% of households were using CFLs at the time of the survey

.

− In 2007, 14% of all Massachuse tts households have tried energy

- saving fixtures and 12% currently have at least one installed. In 2005, 13% of all Massachusetts households had tried energy

- saving fixtures and 11% had at least one energy

- saving fixture currently installed.

− About one

- q uarter of households in Massachusetts are still not using CFLs. Most of those households (16% of all households) are aware of the technology, but have not tried CFLs; 6% of households are not aware of CFLs; and 3% are previous users. A lack of familiarit y (73% of all households) is the primary reason why respondents have not tried energy

- saving fixtures

− As of early 2008, about 21.4% of all bulbs in use in Massachusetts homes were CFLs

— amounting to about 22.6 million CFLs. As of early 2006, about 11.3% of all bulbs in use in Massachusetts homes were CFLs

— amounting to about 12 million CFLs.

− There are about 5.2 million additional CFLs in storage in Massachusetts homes, or about 2.7 bulbs per household (past and current CFL users). In early 2006, there were about 3.8 million CFLs in storage locations. The average number of stored bulbs per household is similar to that found in 2005 and 2006, but the number of households storing bulbs has increased. The fact that CFLs are held in storage suggests that
many p eople plan to install CFLs to replace incandescents or other CFLs as they burn out, and signals that CFLs are achieving the status of standard options to meet lighting needs. The volume of CFLs in storage may be the result of the promotion of multi

- packs t hrough the NCP program and also suggests the potential value of an effort to educate consumers about replacing incandescents with CFLs right away rather than waiting for the incandescents to burn out.

• Sales volumes continue to increase, but at a lower rate than in the past few years.

− Market

- level sales of medium screw

- based CFLs in Massachusetts have increased from 430,000 units in 2000 to about 13.3 million in 2007, and now account forMassachusetts ENERGY STAR Lighting MPER 2007 Page 7

Nexus Market Research 

### about 19% of sales of medium

- screw based bulbs in the Commonwealth. 11 

### This estimate is validated by an analysis we conducted comparing CFL sales and usage patterns from 1998 through 2007 using various sources and assumptions

, including estimates from the Measure Life Study the sponsors are p articipating in. 12



### − The increase in market

- level CFL sales from 2005 to 2006 was about 4.1 million units, or about 65%, while the increase from 2006 to 2007 was only about 2.1 million units, or about 21%.

− Meanwhile, U.S. sales of CFLs increased from 82 million units in 2004 to 89 million in 2005

— up by 9%, to 163 million in 2006

— up by 81%, and in 2007 they increased to 349 million units

— or about 114% over 2006. More CFLs per household are still sold in Massachusetts than in the U.S. as a whole, indicating earlier market adoption, but the r est of the U.S. appears to be catching up. This issue will be explored more fully in the Net

- to

- Gross memo, after more sales data from individual states become available , likely in the fall of 2008

.
Torchieres The following are some key findings related t o torchieres:

• Program

- supported sales of torchieres increased from 774 in 1998 to 38,499 in 2004, but has dropped dramatically to 3,320 in 2007. This drop appears to be related to reduced Sponsor efforts in response to perceived changes in market demand.

• Energy

- efficient torchiere installations account for only 0.1 % of the estimated 1,665 ,000 MWh in lifetime savings resulting from the 2007 program year (adjusted for hour s of use, in

- service rates, free ridership, and spillover).

• Sponsor incentive amounts for the instant coupons were $15 to $20 in 2007, the same as in 2006. Incentive amounts for torchieres were $17 under buydown agreements and $20 for markdowns, an increase of $5 per unit compared to 2006.

• Seven percent (7%) of consumers surveyed report purchasing a torchiere

- style lamp in 2007. Of those purchasing torchieres, 29% say the torchieres they purchased use CFLs. The responses from 2007 show a decrease in the use of halogen bulbs and an increase in the use of CFLs compared to previous years, but these shifts are not statistically significant.
Involvement of manufacturing and retail partners

• The residential lighting market is changing fast, with rapidly increasing sales of CFLs all over the US. Even so, the Massachusetts program, along with other energy

- efficient lighting programs offered elsewhere, continue to be important to manufacturers and retailers and help to stimulate their production and marketing of energy

- efficient lighting products.

− Fifty

- eight percent (58%) of participating re tailers say they would continue to offer CFLs and energy

- saving fixtures if the program were not in place.

11 Not all CFLs sold in the state are currently being used in residential homes; some CFLs are used in non

- residential applications and some are in storage. Identifying the portion of annual sales that are not currently being used in residential applications is beyond the scope of this study.
12 NMR, Inc. and RLW Analytics, Inc. Residential Lighting Measure Life Study, Final June 4, 2008. Submitted to New England Residential Lighting Program Sponsor s.Massachusetts ENERGY STAR Lighting MPER 2007 Page 8

Nexus Market Research 

### • The program continues to attract participating manufacturers and retailers and thus increases the availability of energy

- efficient lighting product s in Massachusetts. The number of retail locations selling ENERGY STAR

- qualified lighting products through the program has increased from 248 in 2002 to 833 in 2007, fewer than the 954 stores in 2006. Furthermore, the Massachusetts program supports sales of about 734 CFL models, 13



### • The range of store types participating in the Massachusetts p rogram includes bargain, hardware, home improvement, department, price club, grocery, drug/convenience, specialty/electrical, and home furnishing stores. In 2007, home improvement stores and hardware stores were the sources of the greatest number of NCP s ales, while two years ago, in 2005, bargain stores dominated sales. Program sales at grocery stores more than doubled in 2007 compared to 2006 and slightly exceeding sales in 2005. Sales at drug and convenience stores dropped dramatically in 2007 compare d to the previous two
years

. including about 126 models in the NCP program. Seventy

- four percent (74%) of the products sold through the NCP program were standard CFLs in the 13 to 23 watt
range.

NCP Program Procedures

• The NCP program, as an industry

- sponsored initiative, 14



### • The structure of the 2007 program follows that of the 2006 program. To encourage industry partners to provide the program with “document ed, credible, store

- level sales reports,” is a dynamic program. Sponsor budget allocations and manufacturer/retailer product mix and counts are established at the beginning of the program year, but allocations and spending shift according to feedback from manufacturers and retailers and Sponsor budgets.

15



### • While there were more buydown agreements com pared to markdown agreements Sponsors offered higher incentives for products sold through the markdown model (which bases all of the incentive payments on sales data) compared to those sold through buydowns (which pays the majority of the inc entive to the industry partner upon receipt of confirmed shipment reports and the remaining amount based on confirmed sales data). Both markdowns and buydowns provide reduced product pricing for the

consumer. 16

13 This figure is a simple sum of unique models from each of the program delivery modes and may include duplicate model types across programs. 14 The Sponsors invite lighting manufacturers to partner with retailers for promotions of qualified lighting pro ducts. The manufacturer

- retailer partner teams are encouraged to submit proposals for a variety of promotion types, including product buydowns and markdowns that reduce the price consumers pay for the products, coupons, and advertising.
15 Request for Pr oposals. Northeast ENERGY STAR Lighting Initiative, Negotiated Cooperative Promotion, 2007 ENERGY STAR Lighting Campaign, April 1, 2007

– December 31, 2007. January 16, 2007.
16 The Sponsors negotiate NCP agreements, also known as memorandum of understand ing agreements (MOUs) with these industry partners that detail the promotion terms. 

### in 2007, the invoiced value of all 2007 markdown agreements was higher, at about $2.7 million

— more than twice the value of buydown agreements ($1.2 million). Additionally, some of the manufacturers/retailers that traditionall y have participated in the program through buydown agreements shifted to the markdown model in 2007.Massachusetts ENERGY STAR Lighting MPER 2007 Page 9

Nexus Market Research 

### • The NCP program continues to rely on the industry expertise of its contractors, EFI and Lockheed Martin to work with manufacturers and retailers. These c ontractors ensure that product shipments and sales occur and that supporting documentation is maintained.

Technical Considerations The lighting market is dynamic with respect to technological progress. It is thus important to track this as well as sales o f CFLs. This section addresses two new technologies: the GU24 standard for fixtures, and solid

- state lighting (SSL), represented largely by light

- emitting diodes (LEDs), the most commonly mentioned emerging SSL technology. The adoption of the Federal Ene rgy Independence and Security Act of 2007 (EISA 2007) , which sets efficiency standards for medium, screw

- base lamps beginning in 2012, also may impact the development of more efficient lighting technologies, including improvements to incandescents, halogen s, CFLs, and SSL.

GU

- 24 Standard for Fixtures

• GU

- 24 fixture technology establishes a new line voltage socket and pin base for replaceable ballasts that is standardized for interchangeability across manufacturers. The idea of the standard is to simplify m atching of fixtures, ballasts, and pin

- based CFLs for manufacturers, retailers, and consumers. GU

- 24 technology can be retrofitted to existing incandescent fixtures. It also allows consumers greater choice of features (light output, light color, dimmabil ity) from a single fixture depending on the GU

- 24 ballast/lamp combination used, and theoretically extends fixture life because the ballasts are replaceable. GU

- 24 lamps are available in one

- and two

- piece lamp and ballast combinations, but aside from the flexibility that two

- piece units offer for replacing just one part of the lamp/ballast configuration, the integrated lamp/ballast combination may be more consumer

- friendly.

• The portion of a finalized Version 4.2 of the ENERGY STAR specification for residen tial light fixtures that addresses GU

- 24 fixtures will take effect on August 1, 2008. The revised version of the specification focuses on the performance requirements for products and fixtures that use GU

- 24 bases. Specifications for fixtures that use ot her lamp bases (not GU

- 24 or LED ) are not aff ected by the revised Version 4.2 specification. All fixture models using GU

- 24 bases that had previously been qualified under the Version 4.0 specification must b e re

- qualified under Version 4.2

.

• Fixture manufa cturers continue to indicate strong support for GU24 technology; many are transitioning to the GU24 standard and phasing out CFL fixtures with other lamp bases.

• This development may have a downside, however. The GU24 standard introduces a new set of pin

- based fixtures and CFLs to the market, overlaying the somewhat complicated array of pin

- based fixtures and CFLs that currently exists. Insofar as the GU24 standard is successful, it may undercut the sales potential of ENERGY STAR

- qualified fixtures that a re currently on the market and make it more difficult for consumers to find replacement bulbs for the fixtures they already have.

• Considering both the benefits and the issues involved, program administrators from Massachusetts are mixed on the benefits of relying on GU24 technology, fearing that it will not inherently promote the use of ENERGY STAR

- qualified products. Moreover,Massachusetts ENERGY STAR Lighting MPER 2007 Page 10

Nexus Market Research 

### there will need to be more pin

- based bulbs available for this technological development to succeed.

Solid

- State Lighting (SSL)

• Effective immediately, the EPA issued a technical amendment to the Residential Lighting Fixture (RLF) criteria, Version 4.2 on June 2, 2008. This amendment incorporates test procedures and metrics to allow LED

- based fixture performance to be evaluated aga inst the requirements of the existing RLF specification.

• Solid

- state lighting (SSL), in the form of white light

- emitting diodes (LEDs), has a number of attributes that make it an attractive option for the future, including the fact that they are mercury

- fr ee and contain no other known toxins. The new ENERGY STAR specification for solid

- state luminaires (SSL), Version 1.0 has been finalized and the criteria will become effective September 30, 2008. The criteria use a two

- tiered approach to qualifying produ cts:

o Category A

— Focus on near

- term applications, mostly niche lighting (task, outdoor, etc.)

o Category B

— Likely 3

- 5 years out, will replace Category A with efficacy targets for a wider range of products, possibly including general lighting applications.

• Th e two

- tiered specification approach recognizes that the technology is rapidly changing and that there are some applications of the technology that are currently commercially viable as energy

- efficient lighting alternatives. Flexibility with the developmen t of the Category B applications allows for the technology to continue its rapid improvement.

• LEDs currently are not a near

- term or even medium

- term replacement for CFLs , although there are some promising reflector lamp applications that might be competi tive in the medium

- term . Experts disagree as to whether screw

- in LEDs will ever be developed for widespread use. Recently EFI started to offer a small selection of recessed or downlight LED fixtures that are available through the Sponsors’ Internet catal og.

• There is a consensus among manufacturers and DOE that not rushing LEDs to market will allow the industry and DOE to learn about the technology, including its benefits and limitations, and will help to avert inaccurate product claims or inferior produc ts that do not meet consumer expectations.

Energy Independence and Security Act of 2007

• EISA 2007 is a federal act, signed into law on December 19, 2007, that sets maximum wattage levels for medium, screw

- base lamps by lumen output. The standards are to become effective under a phased approach beginning in 2012 when general service bulbs will be required to use about 20

- 30% less energy than current incandescent bulbs. Products that meet EISA 2007 standards may include more efficient incandescents and ha logens, which will not necessarily offer the energy savings that are possible with CFLs and will likely be possible through SSL products by that time.Massachusetts ENERGY STAR Lighting MPER 2007 Page 11

Nexus Market Research 

## Conclusions As was the case when observed in last year’s (2006) MPER, the market for CFLs continues t o evolve rapidly, at international, national and state levels. In 2005 the rate of change in Massachusetts was faster than in the U.S. as a whole

— and had been, as far as records indicate, at least since 1998. Starting in 2006, however, that trend reversed , a reversal that accelerated in 2007; the rate of change in the national market appears to have been faster than the rate of change in Massachusetts. Having started earlier and more intensively, the Massachusetts CFL market remains ahead of the national market, and is building off a larger base on a per

- household basis

— but it shows signs of leveling off even as the national market is accelerating. This suggests that the incremental market effects in Massachusetts this year compared to last year are decre asing and may indicate that Massachusetts market is maturing

. Moreover, pending national legislation could provide a new floor or baseline for lighting efficiency by 2012, consolidating gains and ensuring sustainability. Given these trends, the Sponsors sh ould consider ways to modify the nature of their program support during the current transition period.

With 13.3 million CFLs sold in 2007, the number sold per household in Massachusetts was 5. 45 CFLs, significantly higher than the 3.12 per household aver age sold in the U.S. as whole

— an average that includes Massachusetts and other states with active programs. While the U.S. household average likely overstates baseline sales and so should be used only qualitatively, not to derive market based estima tes or net

- to

- gross estimates. Hence, CFL sales per capita in Massachusetts appear to be 65% greater than the national average. However, as recently as 2006 they were about 193% greater, when sales per household were 4.26 CFLs in Massachusetts, compared to 1.46 nationally. As tracked by the U.S. Department of Commerce and adjusted to reflect residential sales, there were about 82 million CFLs sold in 2004, increasing to 90 million in 2005, 163 million in 2006, and 349 million CFLs in 2007.

Behind this floodti de of sales, there appears to be a confluence of trends, including the following:

• Higher energy prices

— including not only electricity prices but, more visibly, gasoline prices at the pump

• Lower CFL prices

— the average price difference between a 25

- watt CF L and a 100

- watt incandescent in Massachusetts in 2007 was $2.80, compared to a difference of $4.39 in 2005

• Greater industry commitment

— for example, Wal

- Mart committed to selling 100 million CFLs in 2007, and met the target by September 17



### • Continuing suppo rt from energy

- efficiency programs; Massachusetts sponsors are among the early champions, especially with the markdown/buydown model, which seems to drive higher sales than alternative program models such as coupons

• Increased environmental awareness (the “ Al Gore Effect”)

— for example, 61% of Americans surveyed in March of 2008 said that the effects of global warming have already begun to happen, compared to 48% who said the same thing in 1997 18

17 Wal

- Mart. 2007. “Wal

- Mart Surpasses Goal to Sell 100 Million Compact Fluorescent Light Bulbs Three Months Early.” http://walmartstores.com/FactsNews/NewsRoom/6756.aspx . Bentonville, Ark.: Wal

- Mart Stores, Inc.
18 Gallup Poll, accessed in http://www.pollingreport.com/enviro.htmMassachusetts ENERGY STAR Lighting MPER 2007 Page 12

Nexus Market Research 

### In keeping with the high number of CFLs sold per household, 94 % of Massachusetts consumers are aware of CFLs, 79% of all households have tried them, and 76% currently use them. As of early 2008, there are about 22.6 million CFLs installed in the Commonwealth

— or 21.4% of all installed medium screw

- based lamps

— and the re are another 5.2 million in storage, nearly all of which consumers intend to use. The fact that consumers are keeping CFLs for later use indicates that they have become a standard option

— something they can reach for in the closet rather than making a sp ecial trip to the store or placing a catalog or online order.

The complement of the 21.4% of sockets being occupied by CFLs is the 62.5 % that are not, indicating significant remaining potential in the Massachusetts residential market. However, with the m arket evolving so rapidly at the national level, the question is how much more additional savings could be captured with the program than without it. The challenge for the Sponsors is to target their efforts where market momentum is lacking.

Despite the uncertainty, it is also important to keep in mind the change in lighting efficiency standards that will become effective in 2012 through EISA 2007

. EISA 2007 will use a phased approach to introduce more efficient lighting standards, beginning in 2012 Som e standards and codes take a long time to work through the population of measures. Clothes washers, for example, last 15 years or so, and half of the less efficient models remain in place for that many years after a change in standards. Incandescent bulbs , in contrast, last only for about 1000 hours on average, and bulbs placed in frequently used applications may need to be replaced several times a year. When the standard becomes effective, one could expect most inefficient bulbs to be changed out in a fa irly short time.

In contrast, the market for CFL fixtures is not transformed; it appears to remain dependent on the Sponsors’ support for survival. Furthermore, Sponsors face the challenge of determining whether to include or change the focus of their ENERGY STAR

- qualified fixture efforts to the GU24 standard.

Meanwhile, technological developments with LEDs also are occurring rapidly. LEDs are expected to have efficie ncy levels similar to CFLs, but most LEDs still are not competitive in terms of perf ormance or cost. Currently, LEDs are able to replace only limited numbers of fixtures and bulbs.



## Recommendations The findings and conclusions drawn from the research summarized above and through inferences that are made throughout the evaluation process lead to several recommendations for consideration by the Sponsors.

Program Structure

• Continue support for each of the current program’s components in the short term

— that is, the next year or so.

− NCPs are an extremely cost

- effective way to increase sale s of energy

- efficient lighting products that are ready for the mass market.

− The two other program components

— coupons and the lighting catalog

— also fill important functions. Coupons provide an important option to allow programMassachusetts ENERGY STAR Lighting MPER 2007 Page 13

Nexus Market Research 

### participation by retailers wh o do not have electronic sales tracking systems. The catalog provides a venue for the introduction of new products to consumers and for the sale of products (such as pin

- based replacement bulbs and LEDs) that are difficult for them to find elsewhere.

− To expand the range of product offerings

— especially fixtures

— that retailers may be hesitant to carry because of low sales volumes, it may be worthwhile to expand the NCP process to include lighting products sold by third parties over the Internet or through catalogs by parties other than the Sponsors.

• Continue to improve tracking of NCP program activity.

− Additional documentation on sales and customers may be needed to support evaluation needs and documentation in support of the Forward Capacity Market. Spons ors need to consider how this may be done within the structure of the NCP
program.

• Annual program planning may be too slow to respond to the rapid changes occurring in the residential lighting market; the Sponsors may, therefore, consider requiring industr y partners to provide more frequent and comprehensive data exchanges to better assess the market and the role of the program in this market.

Product Incentives

• The incentive structure serves at least two needs: to continue to fill the remaining potential for CFL use among all Massachusetts consumers and to encourage the development, production, sales, and use of better and more specialized CFLs. In the near future, the incentive structure may also be used to encourage the development, production, sales, a nd use of other energy

- efficient lighting options, such as SSL.

• We recommend comparing CFL prices for the same packages at the same chains in Massachusetts and one or more non

- program states, to help determine how much of Sponsors’ incentives are being p assed on to consumers.

• Given the increasing levels of sales outside the program, the Sponsors should consider further reductions that would lead toward eliminating the incentives for “plain vanilla” CFLs in the 13 to 25 watt range.

• Continuation of highe r incentives for specialty CFLs that have passed quality tests is warranted because these products are more likely to satisfy consumer expectations. Also increased support for newer specialty CFL applications on the market such as floods or candelabra bas ed bulbs (once ENERGY STAR labeled products become available) to further support their adoption into socket types with lower saturations of CFLs.

• Consider reducing the incentives paid for CFLs sold at “big box” stores relative to the incentive levels offer ed at other types of stores.

• Continue offering lower incentives paid for CFLs in multi

- packs relative to incentives paid for products sold individually.

• Consider higher incentives for CFLs with color rendition properties that better meet customer requireme nts. Regardless of whether or not such differentiated incentives are adopted, the program should seek to educate consumers about choosing CFLs with color properties appropriate to needs or expectations.

• Consider higher incentives for CFLs with lower mercu ry content. (See “Mercury in CFLs” below.)Massachusetts ENERGY STAR Lighting MPER 2007 Page 14

Nexus Market Research 

### Changing Markets

• Stepped

- up advertising may be warranted to persuade those with just a few CFLs to purchase more and install them into other sockets. There remains a sizable segment of the population that has st ill not tried a CFL (21% in 2007), but reaching the group of non

users may prove to be a considerable challenge , given the high level of awareness of CFLs among the general population and the rate at which CFLs have diffused to the rest of the population

, leaving a hard

- to

- reach group of “laggards” or “late adopters” as the remaining non

- users. The Sponsors may also want to consider a boost in advertising to leverage the consumer interest in environmental concerns by tying CFLs to the climate change issue or to leverage the dollar savings that will be achieved by switching.

− It may be possible to get to those who have not tried CFLs through mainstream media outlets, such as television, radio, and print advertising

— as expensive as that is

— or other means such as the Internet, more point

- of

- purchase materials at retailers, or direct mail. The Sponsors’ increased support for advertising could be leveraged through cooperative advertising agreements with retailers and manufacturers, perhaps tied to NCP agreements. The increased advertising could also be partially supported with the funds resulting from reduced incentives.

− Strategies for reaching consumers who have not tried CFLs or are not familiar with CFLs could include continued additional targeted marketing a t grocery stores. The telephone survey revealed that these consumers are more likely to shop for incandescent light bulbs while grocery shopping than are CFL users; continued greater exposure of CFLs in the venues where current non

- users are most likely t o shop for light bulbs should help to raise awareness of the technology.

• Given the rapid development of LED technology, there are likely to be more LED applications, and the Sponsors should plan on increasing the promotion of those applications as they b ecome available.

− The Sponsors should begin promoting LEDs after the ENERGY STAR specification comes out, and invite proposals from manufacturers.

• Consider how EISA 2007 will impact the lighting market, as more efficient incandescent and halogen products are introduced. These products may meet EISA 2007 efficiency standards and will be mercury free, but the energy savings from CFLs and LEDs (Tier 2 products will likely be available by then) may be superior. To avoid lost savings in the regulatory switch, Sponsors may want to think now about positioning the type of products that will give them the most energy savings.

− The incandescent wattage scale for defining lighting needs will need to change under EISA 2007. One way to encourage consumer choices of t he most efficient products is to encourage an efficacy standard or lumens

- per

- watt labeling system. A categorical lumen

- per

- watt scale can help ensure that efficiency standards are maintained.

ENERGY STAR

- Qualified Fixtures

• Given findings from this and previous MPERs, we recommend that Sponsors continue to provide only that support which is necessary to ensure that manufacturers have a market for the current generation of ENERGY STAR

- qualified fixtures until the next generation becomes more widely avail able. The ENERGY STAR

- qualified fixture market appearsMassachusetts ENERGY STAR Lighting MPER 2007 Page 15

Nexus Market Research 

### to remain dependent on Sponsors’ support. A more significant commitment to the current generation of ENERGY STAR

- qualified fixtures might be read as an implied promise to consumers that would be diff icult to fulfill: that replacement lamps and ballasts for any fixture supported by the program in the past will continue to be available for years, if not decades. (Meeting this implicit commitment may be possible through the ENERGY STAR Lights catalog, s o long as the number of models involved is limited. That alone may justify the continued use of the catalog.)

Mercury in CFLs

• There is a need to establish a long

- term solution for CFL disposal in Massachusetts. Currently, 43% of CFL users have had to d ispose of at least one bulb and most of those (72%) have simply thrown the CFL in the trash. An industry

- wide solution to financing CFL recycling programs remains unresolved. Under the Massachusetts Mercury Management Act, 19

• Sponsors have encouraged manufact urers and retailers to help build a mercury recycling infrastructure for CFLs via retail and other avenues for the 2008 program. In 2007, Sponsors provided consumers with some information on recycling at presentations and events and two partnered with the Cambridge post office on a pilot recycling effort. Recycling centers exist in various parts of the state, including each town in Barnstable County, but access to such centers is not available or convenient for all consumers across the state. The Sponsors should continue to encourage mercury disposal solutions with industry partners a law enacted July 28, 2006, program sponsors, retailers, and manufacturers are not required to collect spent bulbs. As of May 1, 2008, the Massachusetts law prohibits the disposal of products containing mercury in trash and wastewater. The Act requires that by January 1, 2007, manu facturers of lamps that contain mercury must implement a plan for educating users about recycling “end of life” lamps. The law establishes recycling targets for mercury

- containing lamps, phased in over the next few years.

• The Sponsors see their role in dealing with mercury and recycling as primarily educational; they have been approached by recycling vendors but are hesitant to take responsibili ty for recycling efforts. Most believe the manufacturers and retailers (and the government) should be responsible for coming up with recycling programs and providing disposal sites. Given the potential downsides of not addressing mercury

— environmental da mage and missed energy

- savings if consumers are reluctant to use CFLs because of the mercury content

— Sponsors should consider additional education efforts and a continued effort to work with industry, regional, and national partners to establish permanent CFL recycling solutions.

19 Fact Sheet, Summary: Massachusetts Mercury Management Act. Massachuset ts Department of Environmental Protection. http://www.mass.gov/dep/toxics/laws/hglawfax.pdf (Accessed 4

- 18

- 07).Massachusetts ENERGY STAR Lighting MPER 2007 Page 16

Nexus Market Research 

### Product Quality

• Sponsors have been strong supporters of product quality assurance efforts for CFLs, and these efforts have been valuable in improving manufacturer accountability for product quality in the ENERGY STAR program. Th e Sponsors may wish to actively engage with the ENERGY STAR program, which will be taking over product quality testing and verification functions from PEARL, to voice their support and to provide relevant technical information.

• As SSLs, currently in the fo rm of LEDs, become available, Sponsors may also wish to actively engage in the on

- going planning and discussions for establishing quality standards. Strong product quality will help to assure consumer acceptance of the technology and the associated energy

- savings that will come from using these products to displace inefficient lighting technologies.

Considerations for the Forward Capacity Market

• The Sponsors should consider tracking some products sold through the NCP retail channel and determining their d isposition, hours of use, wattage displaced, etc. This could contribute to a refinement of Sponsors’ assumptions about in

- service rates and sales and bolster the case for including CFL savings in the FCM. 20



### • In their bid into the FCM, the Sponsors may also w ish to consider additional properties of the effect of CFLs on demand for electricity, such as heat factor to account for cooling savings (in instances in which the retrofit occurs in a n electrically cooled space) and increased heating fuel needed because of reduced heat (in cases of electric heating fuels

. These factors

— in addition to wattage reduction, hours of use, measure life, and in

service rate

— are included in Efficiency Vermont’s Technical Resource Manual.

• At the end of the transition period for FCM , the Sponsors will need to be able to provide program sales data by load zone. For states with only a single load zone this is not a significant concern, however, since Massachusetts has three load zones, a plan for allocating lighting program sales shou ld be fully explored, vetted and put into place prior to the onset of the full FCM. We recommend that as part of the coming evaluation cycle, resources be set aside to ensure that the processes for tracking and confirming retail sales and a system to allo cate them by load zone is assessed and functioning in a manner consistent with the ISO FCM requirements for submitting this demand reduction

resource.
It is important to point out that these are recommendations for maintaining and improving a program that is an unqualified success. The Sponsors should be commended for their design, implementation, and continuing support of a program that is having a profound effect on the market and that is saving so much energy cost effectively.

20 We offered some suggestions for establishing a system to track CFLs in Appe ndix H of the 2006 Lighting MPER report.22 Haskell Street, Cambridge, MA 02140 Phone: (617) 497-7544 Fax: (617) 497-7543 www.nexusmarketresearch.com 

# RESIDENTIAL LIGHTING

MEASURE LIFE STUDY

FINAL
June 4 2008

Submitted to: New England Residential Lighting Program Sponsors

Submitted by: Nexus Market Research, Inc. RLW Analytics, Inc.Nexus Market Research 

## Table of Contents

1 Executive Summary ............................................................................................................. ... 1 2 Introduction ............................................................................ Error! Bookmark not defined. 2.1 Structure of this Report ................................................... Error! Bookmark not defined. 2.2 List of Terms Used in this Report ................................... Error! Bookmark not defined. 2.3 Current Lighting Programs in New England ................... Error! Bookmark not defined. 3 Sample Design, Methodology, and Measure Life ................. Error! Bookmark not defined. 3.1 Sample Design ................................................................. Error! Bookmark not defined. 3.2 On-Site Methodology ...................................................... Error! Bookmark not defined. 3.3 Potential Bias................................................................... Error! Bookmark not defined. 3.4 Decision Table for Including Products in Analysis ........ Error! Bookmark not defined. 3.5 Examining Survival and Failure Rates of Included Products ....... Error! Bookmark not defined. 3.6 Measure Life Analyses .................................................... Error! Bookmark not defined. 3.7 Parametric Regression Analysis ...................................... Error! Bookmark not defined. 4 Installation Rates for M easure Life Products ......................... Error! Bookmark not defined. 5 Current Disposition and Use of Measure Life Products ........ Error! Bookmark not defined. 5.1 Location of Use ............................................................... Error! Bookmark not defined. 5.2 Products That Have Not Been In stalled and Are Not Traceable ... Error! Bookmark not defined. 5.3 Product Disposition ......................................................... Error! Bookmark not defined. 5.4 Replacement of Removed Measure Life Products .......... Error! Bookmark not defined. 6 Counting Products in the Home ............................................. Error! Bookmark not defined. 6.1 Socket Count of All Lighting Products ........................... Error! Bookmark not defined. 6.2 All Lighting Products in Storage ..................................... Error! Bookmark not defined. 7 Markdown Products and Program Spillover .......................... Error! Bookmark not defined. 7.1 Possible Markdown Products in Respondent Homes ...... Error! Bookmark not defined. 7.2 Coupon and Direct Install Spillover ................................ Error! Bookmark not defined. 8 Recommendations .................................................................. Error! Bookmark not defined. Appendix A: Sample Design and Bias........................................... Error! Bookmark not defined. A.1 Sample Development ...................................................... Error! Bookmark not defined. A.2 Sample Design ................................................................. Error! Bookmark not defined. A.3 On-Site Methodology ...................................................... Error! Bookmark not defined. A.4 Bias Resulting from Sample Design and Methodology .. Error! Bookmark not defined. A.4.1 Potential Sources of Bias from Sampling Procedures ........... Error! Bookmark not defined. A.4.2 Potential Sources of Bias from Respondent Recall . Error! Bookmark not defined. A.5 Decision Table for Including Products in Analysis ........ Error! Bookmark not defined. Appendix B: Preliminary Examination of Measure Life Data ...... Error! Bookmark not defined. B.1 Examining Survival and Failure Rates of Included Products ....... Error! Bookmark not defined. B.2 Product Quality over Time .............................................. Error! Bookmark not defined.Nexus Market Research 

### Appendix C: Measure Life Analyses ............................................. Error! Bookmark not defined. C.1 Life Tables ....................................................................... Error! Bookmark not defined. C.2 Logit Regression Models ................................................ Error! Bookmark not defined. C.3 Parametric Regression Analysis ...................................... Error! Bookmark not defined. Appendix D: Not Found/Not Recalled Products ........................... Error! Bookmark not defined. D.1 Background Information ................................................. Error! Bookmark not defined. D.2 Ability to Recall over Time ............................................. Error! Bookmark not defined. D.3 Location of Original Installation ..................................... Error! Bookmark not defined. D.4 Relationship to Participants who Purchase the Most Products ..... Error! Bookmark not defined. D.5 Various Analysis of Model Numbers .............................. Error! Bookmark not defined. D.5.1 Miscategorization as Spillover ................................. Error! Bookmark not defined. D.5.3 Rare Products or Incorrect Model Numbers ............ Error! Bookmark not defined. D.5.3 Overlap with Most Commonly Obtained Products .. Error! Bookmark not defined.Nexus Market Research 

## List of Tables

Table 1–1: Recommended Estimates of Measure Life – Decimals ................................................ 1 Table 1–2: Recommended Estimates of Measure Life – Integers .................................................. 1 Table 1–3: Status of Measure Life Products for Use in Analysis ................................................... 3 Table 2–1: Annual Product Distri bution in Massachusetts by Type and Program Component a



### ................................................................................................ Error! Bookmark not defined. Table 3–1: Products Available for Study Sample .......................... Error! Bookmark not defined. Table 3–2: Distribution of Sample d CFLs, External Fixtures, and Internal Fixtures by State and Program .................................................................................. Error! Bookmark not defined. Table 3–3: Categorization of M easure Life Products Based on Rules of Decision Table .... Error! Bookmark not defined. Table 3–4: Characterization of Products by Auditor’s Ability to Locate Them . Error! Bookmark not defined. Table 3–5: Coupon CFL Failure and Surv ival by Year of Purchase ........... Error! Bookmark not defined. Table 3–6: Direct Install CFL Failure and Survival by Year of Purchase ... Error! Bookmark not defined. Table 3–7: Exterior Fixtur e Failure and Survival by Year of Purchase ...... Error! Bookmark not defined. Table 3–8: Interior Fixtures Failure and Surviv al by Year of Purchase ...... Error! Bookmark not defined. Table 3–9: Recommended Estimates of Measure Life – Decimals Reported .... Error! Bookmark not defined. Table 3–10: Recommended Estimates of Measure Life – Integers Reported .... Error! Bookmark not defined. Table 4–1: Product Installation Rates by Year .............................. Error! Bookmark not defined. Table 5–1: Location of Inst alled Lighting Products ...................... Error! Bookmark not defined. Table 5–2: Percentage of Products Responde nts Did Not Recall by Quality of Product Information ............................................................................ Error! Bookmark not defined. Table 5–3: Locating Interior Fi xtures by Type of Fixture ............. Error! Bookmark not defined. Table 5–4: Disposal Method of lighti ng products that broke or burned out Error! Bookmark not defined. Table 5–5: Disposal method of lighting products removed while still working . Error! Bookmark not defined. Table 5–6: Replacement of Removed Lighting Products .............. Error! Bookmark not defined. Table 6–1: Distribution of Lighting Products by State .................. Error! Bookmark not defined. Table 6–2: Lighting Products by Location in Household .............. Error! Bookmark not defined. Table 6–3: Lighting Produc ts by Fixture Type .............................. Error! Bookmark not defined. Table 6–4: Lighting Products by Control ...................................... Error! Bookmark not defined. Table 6–5: Wattage of All In stalled Lighting Products ................. Error! Bookmark not defined. Table 6–6: Light Bu lbs in Storage ................................................. Error! Bookmark not defined. Table 6–7: Why CFL Was Not Installed ....................................... Error! Bookmark not defined. Table 6–8: What Bulb Will CFL Replace...................................... Error! Bookmark not defined. Table 7–1: Overlap of Measure Life Product Models with Markdown Models. Error! Bookmark not defined.Nexus Market Research 

### Table 7–2: Source of CFLs Observed in Respondents’ Homes .... Error! Bookmark not defined. Table 7–3: Calculating Possibl e CFL Spillover Purchases ............ Error! Bookmark not defined. Table 7–4: Motivation for Buying CFL Products .......................... Error! Bookmark not defined. Table 8–1: Recommended Estim ates for Measure Life ................. Error! Bookmark not defined. Table A–1: Products Listed in Data bases Received from Sponsors ............ Error! Bookmark not defined. Table A–2: Products Available for Study Sample ......................... Error! Bookmark not defined. Table A–3: Desired and Actual Comple tions by Year and Type of Program .... Error! Bookmark not defined. Table A–4: Distribution of Sample d CFLs, External Fixtures, and Internal Fixtures by State and Program .................................................................................. Error! Bookmark not defined. Table A–5: Decision Tabl e for Useful Life ................................... Error! Bookmark not defined. Table A–6: Categorization of Measure Life Products Based on Rules of Decision Table .... Error! Bookmark not defined. Table A–7: Characterizati on of Products by Auditor’s Ability to Locate Them Error! Bookmark not defined. Table B–1: Coupon CFL Failure and Su rvival by Year of Purchase .......... Error! Bookmark not defined. Table B–2: Direct Install CFL Failure and Survival by Year of Purchase .. Error! Bookmark not defined. Table B–3: Exterior Fixt ure Failure and Survival by Year of Purchase ...... Error! Bookmark not defined. Table B–4: Interior Fixt ures Failure and Survival by Year of Purchase ..... Error! Bookmark not defined. Table C–1: Estimated Measure Li fe, Logit Regression Analysis .. Error! Bookmark not defined. Table C–2: Estimated Measure Life, Parametric Regression Analyses ...... Error! Bookmark not defined. Table C–3: Estimated Measure Life of Products al so Offered in Markdown Programs, Parametric Regression Analyses .............................................................. Error! Bookmark not defined. Table C–4: Recommended Estimates of Measure Life – Decimals Reported .... Error! Bookmark not defined. Table C–5: Recommended Estimates of Measure Life – Integers Reported Error! Bookmark not defined. Table D–1: Number and Percentage of Not Foun d/Not Recalled Products Error! Bookmark not defined. Table D–2: Percentage of Products Res pondents Did Not Recall by Quality of Product Information ............................................................................ Error! Bookmark not defined. Table D–3: Status of Lighti ng Products with Model Number s Sometimes Found/Recalled Error! Bookmark not defined. Table D–4: Status of All Lighting Products, Except those Not Found/Not Recalled............ Error! Bookmark not defined.Nexus Market Research 

## List of Figures

Figure 3-1: Survival an d Failure Rates of Coupon CFLs (Raw Data) ......... Error! Bookmark not defined. Figure 3-2: Survival and Failure Rates of Direct Install CFLs (Raw Data) Error! Bookmark not defined. Figure 3-3: Survival and Failure Rates of Exterior Fixtures (Raw Data) .... Error! Bookmark not defined. Figure 3-4: Survival and Failure Rates of Interior Fixtures (Raw Data) ..... Error! Bookmark not defined. Figure A-1: Percentage of Products Found by Auditor or Recalled by Respondent ............. Error! Bookmark not defined. Figure B-1: Survival an d Failure Rates of Coupon CFLs (Raw Data) ........ Error! Bookmark not defined. Figure B-2: Survival and Failure Rates of Direct Install CFLs (Raw Data) Error! Bookmark not defined. Figure B-3: Survival and Failure Rates of Exterior Fixtures (Raw Data) ... Error! Bookmark not defined. Figure B-4: Survival and Failure Rates of Interior Fixtures (Raw Data) .... Error! Bookmark not defined. Figure B-5: Survival Rates of CFLs by Year Obtained (Raw Data) ........... Error! Bookmark not defined. Figure B-6: Survival Rates of Exterior Fixtures by Year Obtained (Raw Data) Error! Bookmark not defined. Figure B-7: Survival Rates of Interior Fixtures by Year Obtained (Raw Data) . Error! Bookmark not defined. Figure C-1: Cumulative Survival Rates from Life Tables ............. Error! Bookmark not defined. Figure C-2: Estimated Survival Rates from Logit Regression ...... Error! Bookmark not defined. Figure C-3: Estimated Measure Life of CFLs, Parametric Regression Analyses .................. Error! Bookmark not defined. Figure C-4: Measure Life of Exterior Fi xtures, Parametric Regression Analyses ................ Error! Bookmark not defined. Figure C-5: Measure Life of Interior Fixtures, Parame tric Regression Analyses ................. Error! Bookmark not defined. Figure D-1: Percentage of Products Found by Auditor or Recalled by Respondent ............. Error! Bookmark not defined.FINAL Report on Lighting Measure Life Study June 4, 2008 Page 1

Nexus Market Research 

## 1 Executive Summary The purpose of this study is to estimate meas ure life for lighting produc ts distributed through energy efficiency programs in New England. As e xplained in more detail in the full study (see Section Error! Reference source not found. and Appendix C), we recommend three different program-specific estimates of measure life fo r CFLs (coupon, direct install, and markdown 1 

### ) and two for exterior fixtures (markdown and all other programs). These estimates and their respective confidence intervals are shown in Table 1–1 (to two decimal places) and in Table 1–2 (as integers). We do not suggest an estimate of measure life for interior fixtures as we believe the data were collected too early in their lif e cycle to provide a reliable estimate.

Table 1–1: Recommended Estimates of Measure Life – Decimals

Product Measure Life 80% Confidence Interval Low High Coupon CFLs 5.48 5.06 5.91 Direct Install CFLs 6.67 5.97 7.36 Markdown CFLs (all states) 6.82 6.15 7.44 Coupon and Direct Install Exterior Fixtures 5.47 5.00 5.93 Markdown Exterior Fixtures 5.88 5.24 6.52 All Interior Fixtures Continue using current estimates of measure life



### Table 1–2: Recommended Estimates of Measure Life – Integers

Product Measure Life 80% Confidence Interval Low High Coupon CFLs 5 5 6 Direct Install CFLs 7 6 7 Markdown CFLs (all states) 7 6 7 Coupon and Direct Install Exterior Fixtures 5 5 6 Markdown Exterior Fixtures 6 5 7 All Interior Fixtures Continue using current estimates of measure life



### Our definition of “measure life” is consistent with that used in the Measure Life Report prepared by GDS Associates for the New Engla nd State Program Working Group (SPWG). 2 

### “For programs delivered by program administrators in New England, Measure Life includes equipment life and measure persiste nce (not savings persistence).

 Equipment Life means the number of years th at a measure is installed and will operate until failure, and

 Measure Persistence takes into account busines s turnover, early retirement of installed equipment, and other reasons measures might be removed or discontinued.”

1 Due to the diversity of program types throughout the region, we use the term “markdown” to refer to both markdown programs (offered in all the states) and buydown programs (offered in some of the states). In Massachusetts, the Negotiated Cooperative Promotions (NCPs) include both markdown and buydown programs.
2 GDS Associates, Inc. (2007) Measure Life Report: Residential and Commercial/Industrial Lighting and HVAC Measures . Prepared for The New England State Program Wo rking Group for use as an Energy Efficiency Measures/Programs Reference Document for the ISO Forward Capacity Market (FCM).FINAL Report on Lighting Measure Life Study June 4, 2008 Page 2

Nexus Market Research 

### Specifically, our measure life estimates do not di stinguish between equipment life and measure persistence; our estimates—one for each meas ure category—include both those products that were installed and operated until failure (i.e., equi pment life) as well as those that were retired early and permanently removed fr om service for any reason, be it early failure, breakage, or the respondent not liking the product (i .e., measure persistence). The remainder of this executive summary provides background information about the study and highlights some of the key results and recommendations.

Sample Development and Design : The sample design for this st udy is based on the number of energy efficient lighting products distributed through energy-effi ciency programs conducted in Connecticut, Maine, Massachusetts, New Hampsh ire, Rhode Island, and Vermont (See Section Error! Reference source not found. and Appendix A). For a program to be included, we needed to be able to determine the following information for a product or the person obtaining that product: 1. Knowledge that the respondent had obtained at least one ener gy efficient lighting product through a Sponsor’s program from 2002 to 2006 2. Detailed information on the model number, ma nufacturer and wattage for the product in order to identify it in the home; for direct install programs, we also looked for the location of installation 3. Customer contact information

After reviewing the databases of households that had participated in various retail and direct install programs, NMR and RLW determined that only the coupon, single-family ENERGY STAR Homes and MassSAVE files contained sufficient product and resident contact information to use for the study. We drew a random sample of participants based on the type and number of products they had obtaine d through the programs. We collectively refer to these sample products as the “measure life products.” Auditors visited a total of 285 ho mes to conduct an inventory of lighting products and a respondent survey designed to learn more about the measure life products as well as other lighting products found in the home.

Bias Resulting from Sample Design and Methodology : The sample design and methodology used in this study introdu ce several potential sources of bias (See Appendix A): 1. The lack of adequate product and customer contact information limited the sample to the coupon and a few direct-install programs wh ile excluding products from all other Sponsor-administered lighting programs. 2. In order to complete the study in a timely and cost effective manner, the later on-site surveys targeted homes with large numbers of fixtures. This decision resulted in the unintended inclusion of a disp roportionate number of electricians, contractors, and landlords, as they had purchased large numbers of fixtures to install in locations other than their own homes. Because they installe d these products at diffe rent addresses, we were unable to verify the disposition of many of these products. Furthermore, respondents with numerous products were less lik ely to recall the dis position of at least some of them (See Appendix D).FINAL Report on Lighting Measure Life Study June 4, 2008 Page 3

Nexus Market Research 

### 3. Given the amount of time that passed between the household obtaining the lighting products and being contacted for this study, in accurate customer recall of products that the auditor did not personally observe acc ounts for the majority of products excluded from the analysis and presents a major source of potential bias 4. Because we contacted respondents at the phone number given at the time of participation, the resulting sample included only those who had not moved or changed their phone number in at least one and up to six year s. This likely means that low-income households, renters, and younger adults ar e not well represented in the sample.

While we recognize that the poten tial for bias exists, we cannot say whether such bias would produce higher or lower estimates than the ones we present here. Moreover, we find no evidence of bias across states or Sponsors.

Characterizing Products as Survived, Failed, or Excluded : In order to estimate measure life, we had to classify individual pr oducts as having “survived” or “f ailed” for a specific period of time. In cooperation with the Sponsors, we developed a “Decision Table” to guide the classification of products into one of three categor ies: 1) survived, 2) failed or 3) excluded (See Section Error! Reference source not found. and Appendix A). For a product to be classified as “survived” the auditor typically had to confirm its continued installation a nd operation visually. An exception to this was the inclusion of products reported installed in re ntals, second homes, and businesses if the respondent was in the positio n of knowing the current status of the product. “Failed” products are those that burned out, broke , or were permanently removed from service, including those that broke or faile d and were returned to the stor e. We excluded products: that could not be found (accounting for the majority of excluded products, see Appendix D); that were reported installed but th e respondent was not in the position to know if the product remained in place (e.g., by a contract or); that were installed outside of New England; that were being stored for future use; and that had been returned to the store before the product failed (e.g., a CFL may not have fit a fixture or the customer decided they di d not like a fixt ure) or given away. Table 1–3 summarizes these classifications, but see Section Error! Reference source not found. and Appendix A for more detail.

Table 1–3: Status of Measure Life Products for Use in Analysis

Product Status CFLs Fixtures Coupon Direct Install Exterior Interior Survived 48% 56% 37% 55% Failed 20% 14% 17% 6% Excluded from Analysis 32% 31% 46% 39% Total Number of Products 695 441 215 397



### Measure Life Analysis : We relied on three types of “survi val analyses” to estimate the measure life of the products distribut ed through the coupon and di rect install programs under consideration (See Section Error! Reference source not found. , Section Error! Reference source not found. and Appendix C): Method 1: Life Tables Method 2: Logit RegressionFINAL Report on Lighting Measure Life Study June 4, 2008 Page 4

Nexus Market Research 

### Method 3: Parametric Regression Models of Survival Analysis 3



### We chose estimates resulting from parametric re gression analysis. According to our results, the measure life of CFLs (coupon, direct install, a nd markdown) falls between five and one-half and seven years, while that for exterior fixtures (coupon, direct install, and markdown) is between five and one-half years and six y ears (Table 1–1). However, we do not believe that the data or results are adequate for predicting the measure life of interior fixtures because this study was conducted too early in their lifecycl e. The measure life data al so provide some indication of increased survival rates over time for CFLs, perhaps as a result of improved product quality, although the small sample size and limited number of failures in recent years curtail our ability to conduct meaningful statistical anal yses to verify improved quality.

The reader will note that we provide an esti mate of measure life for markdown CFLs and exterior fixtures. We did not includ e the lighting markdown and buydown programs (collectively referred to as “m arkdown” programs in this document since not all Sponsors used the buydown approach) in the sample of measure life products due to a lack of participant contact information. Even so, markdown progr ams account for the vast majority of lighting products distributed through the Sponsors’ program s. For this reason, we conducted analyses on the subset of products with model numbers obtained through the coupon or direct install programs that were also distri buted through markdown programs in order to provide an estimate of measure life for the markdown products (Section Error! Reference source not found. ). We supply these estimates with thr ee important caveats: 1) the po pulation who purchases markdown products may differ from those who take part in coupon or direct insta ll programs, 2) not all markdown model numbers were represented in the sample of measure life products, and 3) the distribution and usage of products actually purc hased in the markdown programs may vary from what we observed from these products obtained th rough coupon and direct in stall products. We believe it would be wise to conduct a follow-up study of the measure life of markdown products in the near future.

Measure Life Product Use and Disposition : Most of the measure lif e CFLs and many interior fixtures were found installed in the living room (22% to 23%), bedroom (16% to 24%), kitchen and dining room (11% to 19%), and the baseme nt (9% to 11%) of re spondents’ homes (See Section Error! Reference source not found. ). Interior fixtures were most commonly installed in foyers and hallways (32%). Only four percent of coupon CFLs and one percent of all other products were found in storage, lik ely reflecting the fact that many of these products had been in the respondents’ home between 18 months and si x years by the time we visited (See Section Error! Reference source not found. ). Most of the products ha d likely been installed—or misplaced—by the time we conducted the on-sites. It is also the case that direct install products are typically installed by the auditor during the vis it to the customer’s home. Alternatively, it is possible that some of the stored CFLs are meas ure life products on which we had incomplete or perhaps incorrect information from the databa se, or that respondents confused them with products they had obtained outside the coupon or direct install programs. Respondents usually replace burned out or broken CFLs with new CFLs (59%), but broken ener gy-efficient fixtures are more commonly replaced with regular fixtures and incandescent bulbs (59%) (See Section Error! Reference source not found. ). Once CFLs or fixtures bur n out, most participants throw

3 We examined five types of Parametric Regression models in this analysis.FINAL Report on Lighting Measure Life Study June 4, 2008 Page 5

Nexus Market Research 

### them away in the trash (84%). Few respondents report recycling the CFLs (14%). Previous research we have conducted in dicates that few people are aware of the mercury in CFLs, although recent media attention and Sponsor ed ucation campaigns have raised awareness. 4 

### As a result, most consumers throw the CFLs away as th ey would other bulbs. However, it is also the case that to recycle CFLs in many of the states participating in this study, users must save broken or burned out bulbs and take them to hazardous wa ste drop-off sites (often associated with towns or municipalities), usua lly on specific dates.

Identification of Markdo wn and Spillover Products : A secondary objective of this study is to estimate the number of products currently in respondent s’ homes that may ha ve potentially been purchased through lighting markdown programs run by the Sponsors. We matched a list of model numbers from Energy Federation Incorporat ed (EFI) of all markdown products offered in New England (with a separate list for Vermont ) with all the measure life products currently installed in the 285 households (See Section Error! Reference source not found. ). We also asked respondents how much they paid for the CFLs and fixtures and where they were purchased. Only two fixtures appear to be mar kdown products, but a total of 21% of all CFLs in respondents’ homes (942 in total, or 3.3 per home) are likely markdown purchases. Note that in 2006 markdown products accounted for about 85% of CFLs distributed through Massachusetts programs, but because the participants in the measure life programs have obtained products through coupon or direct insta ll programs, they may have had less need to buy markdown products than households that are not obtaining CFLs for the first time.

Finally, we estimated spillover for the coupon and direct install programs (See Section Error! Reference source not found. ). We limited the estimates to non-markdown CFLs that were purchased after the respondents’ participation in the coupon or direct install programs. Respondents had to be aware of the program and to state that their partic ipation in the coupon or direct install program strongly in fluenced their decision to purc hase the non-program products. In total, there are 892 likely sp illover CFLs found in the homes of coupon participants (4.9 per coupon household) compared to 695 coupon CFLs—amounting to spillover of 128%—and 355 likely spillover CFLs found in the homes of direct install participants (3 .4 per direct install household) compared to 441 direct install CFLs—amounting to spillover of 81%.

However, it should be noted that our estimate of spillover does not take into account program influences of which the respondent is not aware, such as the fact that the success of such programs has increased the availability and lowere d the price of CFLs. Furthermore, it does not include any program-induced purchases of CFLs th at are no longer in their homes (e.g., they may have burned out, been given away, etc.). Taking these other factors into account would tend to increase the spillover rate. In c ontrast, it is also likely that th e markdown program is responsible for some of the spillover that our methodology attributes to direct install or coupon programs.

4 NMR (2008) Telephone Survey Results for Market Progress and Evaluation Report (MPER) 2007 Massachusetts ENERGY STAR Lighting Program. Submitted to National Grid, Cape Light Compact, NSTAR Electric Company, Western Massachusetts Electric Compan y, and Unitil. Draft April 4, 2008.FINAL Report on Lighting Measure Life Study June 4, 2008 Page 6

Nexus Market Research 

### Recommendations : The findings from this analysis lead to the following recommendations for the Sponsors:

 Adopt the measure life estimat es presented in Table 1–1

 To the extent possible, collect the followi ng information in direct install programs: product type, manufacturer, model number, fi xture type, wattage, room/location of installation, date of in stallation, and any other product as well as customer information including name, address, and phone number.

 Conduct a measure life study of in terior fixtures in the futu re, as our study occurred too early in their life cycle to provide reliable estimates

 Conduct a process evaluation to examine probl ems with tracking databases; consider a study that tracks new coupon purchases over time in order to ascertain what happens to products after they leave the stores.

 Conduct a long-term measure life study relying on a panel- based approach and using a sample drawn in part from the current Markdown Impact Study being conducted for Sponsors in Connecticut, Massachusetts, R hode Island, and Vermont and marking the appropriate products with a sticker or perm anent marker for future identification.

 Continue current support for qua lity assurance efforts.

Additional Topics : This report also addre sses the following topics:

 Installation rates of measure life CFL pr oducts included in this study (See Section Error! Reference source not found. )

 Analysis of all lighting products currently installed in the participants’ homes (See Section Error! Reference source not found. )

 Analysis of all lighting products curr ently placed in storage (See Section Error! Reference source not found. )

## Final Report Coincidence Factor Study Residential Room Air Conditioners

June 27 , 200 7

June 23 , 200 8

Prepared by: RLW Analytics 179 Main Street Middletown, CT 06457 (860) 346

- 5001 

## Prepared for; Northeast Energy Efficiency Partnerships’ New England Evaluation and State Pr ogram Working Group

Sponsored by: Cape Light Compact Efficiency Vermont National Grid USA

NSTAR Electric Western Massachusetts Electric Maine PUC Fitchburg Electric New Hampshire Electric Cooperative

Public Service of New Hampshire Unit i l

### Acknowl edgement:

The authors wish to thank all of the people at the State Program Working Group (SPWG), Northeast Energy Efficiency Partnership (NEEP) who took the time to support and help with this study. Regrettably, we cannot thank everyone individually, but we do want to acknowledge the contributions made by Gail Azulay, Tom Belair, Lisa Glover, Cheryl Jenkins, Angela Li , Ralph Prahl and Elizabeth Titus. The data, insight, and support provided by these individuals helped to establish the foundation for this report. RLW assumes sole responsibility for any errors or omissions in this report.

### Exe cutive Summary

The New England State Program Working Group (SPWG) 1 contracted with RLW to calculate on

- peak and seasonal peak coincidence factors for residential ro om Air Conditioner ( R AC) measures that could be consistently applied to energy efficiency programs that may bid into the ISO

- NE Forward Capacity Market (FCM) in any of the New England states. The study cover ed f our of the six New England states

includin g Massachusetts, New Hampshire, Rhode Island and Vermont . Maine also sponsored the study although there were no participating units in the state and no on

- site metering or survey activity was

conducted in the state. Connecticut did not participate in the s tudy because they no longer offer incentives

for room AC units. 2

The study utilized interval metered power data from 93 on

- site visits that were nested within a sample of approximately 6 1 0 phone surveys. The sample was design ed to allocate on

- site visits and phone surveys equally by the six ISO

- NE load zones with participating room A C units from program years 2005 and

2006. Figure i

- 1 shows the actual distribution of data collection activities by load zones , the on

- site numbers reflect sites with both a phone survey and site visit

.NEMANHRISEMAVTWCMATotal On-site Survey 0 100 200 300 400 500 600 Distribution of On-sites and Phone Surveys Completed

On-site 15 16 19 10 15 18 93 Survey 89 83 81 93 89 82 517 NEMA NH RI SEMA VT WCMA Total

Figure i

- 1 : Distribution of On

- site and Phone Surveys

1 Represented by the state regulatory age ncies (CT DPUC, Maine PUC, MA DOER, NH PUC, RI PUC, and VT PSB) and associated energy efficiency program administrators (Cape Light Compact, Maine PUC , Efficiency Vermont, National Grid (MA, NH & RI), Northeast Utilities (CT&MA), NSTAR, PSNH, United Illumi nating, and Unitil (MA&NH)).
2 There are no participant sites in Maine, however results will be provided by adjusting the study results to Maine weather.The analysis of the primary data utilized a two step approach the first step was to create a regressio n model of the operation of the Room AC units using the real year weather data and actual metered data

. The second step was to use the resulting model to predict the operation of the room AC units across the

ISO

- NE FCM performance hours for 2007 and typic al year after adjusting for any bias in the on

- site sample. The nested on

- site sample technique was used to control for potential bias in the on

- site sample, specifically selection bias d ue to the increased probability that people who are generally home d uring the day would b e over represented in the on

- site sample

. 3

• Type of Area Served (i.e. Bedroom vs. Non

- bedroom), A multi

- variant regression model was constructed using the metered interval power data ( for 114 room AC units ) and the survey response data a long with hourly weather data from the appropriate weather station.

There were six survey variables that were found to have statistically significant impact on the regression

model as follows:

• Home During Day,

• Cooling Capacity per Area Served (BTU/f t 2 ),

• Outside Temperature when Cooling Begins,

• Schedule or Continuous Operation, and

• Cooling Setting. Each of these variables were tested to determine if there was a statistically different distribution of the variables within a load zone when compared with the mean values for the whole dataset using a T

- test methodology. The largest single change from on

- site data occurred in the occupancy variable, which had

on

- site customers reporting that 73% were generally home during the day as opposed to 53% in the l arger survey sample. Both the space type and occupancy variables did not show significant variation between the overall survey results and the load zone level survey results. Table i

- 1 provides a summary of the

chan ges to the four remaining variables, which show that unique results w ere calcula ted for the NEMA,
RI, SEMA and VT load zones. The results for the NH and WCMA load zones were identical to those provided by the model inputs using the average survey response data for all zones.

3 Phone survey results from surveys conducted during evening and weekend hours were used to establish occupancy rates for the population.Load Zone BTU/sqft Outside Temp Cont_Sched Cooling Setpoint All Zones 32.9 82.4 0.28 70.5

NEMA 35.2 83.8 0.28 70.5

NH 32.9 82.4 0.28 70.5 RI 32.9 83.9 0.35 71.5 SEMA 32.9 81.0 0.28 70.5

VT 29.7 82.4 0.28 70.5 WCMA 32.9 82.4 0.28 70.5

Table i

- 1 : Summary of Zonal Changes to Survey Variables 4

The Coincidence Factors (CFs) and Full Load Equivalent Hours (FL E Hs) were developed for the 2007 summer season using the operating profiles th at ha d been adjusted for all applicable bias using the phone survey response data as described above. The calculation of the On

- Peak CF w as relatively straightforward since the performance hours are time dependent and can be calculated without having

extr eme ambient weather conditions. The calculation of the FLEHs was also straightforward and w as calculated from the bias adjusted operating profiles directly. The weather normalized CFs and FLEHs

were comp ut ed by using the bias adjust ed regression model an d using Typical Meteorological Year
(TMY 2) weather data to calculate the results. Since the results are driven by differences in load zone variables and weather file data the results are reported out at the weather file level using survey inputs for

the applicable load zones

. The results were calculated by holding the survey variables static and then running the nine different weather files so that the hourly weather variable s could be used to provide hourly results. Table i

- 2 provides a summary of the results for all weather files using the average survey inputs for all Load zones for the On

- Peak performance hours 1:00 PM to 5:00 PM June through August using 2007 and TMY2 weather data.

Weather Files On-Peak CF Seasonal CF On-Peak CF Seasonal CF 2007 FLEH TMY2 FLEH Albany, NY 0.154 0.276 0.142 NA 224 184 Boston, MA 0.134 0.304 0.125 NA 228 175 Burlington, VT 0.139 0.276 0.119 NA 166 141 Caribou, ME 0.080 0.131 0.080 NA 60 42 Concord NH 0.143 0.290 0.134 NA 171 149 Hartford, CT 0.170 0.303 0.171 NA 272 253 Portland, ME 0.111 0.270 0.111 NA 119 102 Providence , RI 0.159 0.296 0.144 NA 245 204 Worcester, MA 0.131 0.261 0.113 NA 172 134 Average for All Load Zones 2007 Weather TMY2 Weather Average for All Load Zones Average for All Load Zones

Table i

- 2 : Summary of CF and FLEH by Weather File using Average Load Zone Data

4 The zone specific responses that are different from the average for all zones are shown in bold font.Although there were slight differences in CF and FLEHs due to zonal differences in the model inputs the

difference in the final results were not much more than ± 0.001 for CF and ± 3 hours for FLEH. Therefore although the zonal differences in survey responses for some of the model variables were statistically significant when these different model input were run the results did not provide numerically significan t differences in the results. 5

Table i

- 2 As a result we recommend that the calculation of DRV for each load zone use the CFs provided in

.

The project results are reported out by weather file because the CF a nd FLEHs were calculated using the regression model and hourly weather data. The optimum method for determining RAC savings for a sponsor that operates in multiple load zones and/or has customers that should be modeled using multiple

weather files would b e to assign load zones and weather file designations to each rebate based upon the location of the customer by town and or zip code. Once this has been accomplished then capacity or demand reduction weighted allocations can be developed for each load zone where multiple weather files are applicable. If all of the demand reduction within a load zone is associated with one weather file then
the sponsor can simply select the appropriate CF for the weather file as given in Table 

### i

- 2

.

The Seasonal Peak performance hours were calculated by determining the hours when the real

- time system load meets or exceeds 90% of the 50/50 CELT forecast for the summer 2007 period of 27,360

MW

. 6 There were a total of 24 ho u rs during t he summer of 2007 when the real

– time system load was 24,624 MW or greater, eight hours during June and 16 hours during August, and the 2007 Seasonal Peak CFs were calculated during those hours. It was not possible to calculate the TMY 2 Seasonal peak CF

v alues because of the method used to create TMY 2 weather data, which uses “typical” months to create

an annual file. 7

5 This was due to a combination of factors primarily the relatively small differences in the variables, and chang es in multiple variables canceling each other out.
6 Data taken from ISO

- NE 2007 Capacity, Energy, Load and Transmission (CELT) report dated April 20, 2007.
7 For example the June data for Albany could be from 1976, while the Boston data could be from 19 80 and Hartford from 1978. In order to develop an accurate typical regional weather model it will be necessary to select typical months from the same year for all of the regional files. The ISO

- NE report entitled “Summer 2007 Weather Normal Peak Load” noted that the weather normalized peak load for 2007 was 27,460 MW, 0. 4% (100 MW) higher than the April 2007
forecast of 27,360 MW for the summer of 2007. According to the report “ The summer of 2007 can be

characterized as normal with respect to overall temperature and humidity

. ” Therefore we would defer to

ISO

- NE charact erization of the summer of 2007 as normal with respect to temperature and humidity and

recommend that both the 2007 On

- Peak CFs and 2007 Seasonal Peak CFs be used for future year DRV calculation by the project sponsors

.Th e relative precision of the estimate d impacts provided fr o m the bias adjusted model could not be

calculated dire ctly because the model used the average inputs from the survey data and thus provided only one set of numbers depending upon the load zone and w eather file selected. A first order approximation of the relative precision is provided by the following equations;

Y = f(x) + E => E = Y

- f(x)

Y adj = f(x o ) + E

Y adj = Y + [ f(x o )

– f(x)] Where,

Y = the actual CF for the hour from the metered
f(x) = the predicted value from unadjusted model
f(x o ) = the predicted value after adjusting the model for bias

E = expected error in the adjusted model

Y adj = the predicted output from the adjusted model

Table i

- 3 provi des the estimated relative precision of the monthly and summer On

- Peak CF values using the methodology explained above. The relative precision ranged from ±14.4% for June to ±10.4% for the

summer season. Note that the mean value for June was 0.218, which was higher than expected because most of the June metered data was collected during a heat wave at the end of the month. 8

Standard Relative Month sample (n) Mean Deviation Cv Precision June 82 0.218 0.222 1.02 ±14.4% July 108 0.156 0.155 0.99 ±12.2% August 108 0.174 0.164 0.94 ±11.7% Summer 114 0.175 0.152 0.87 ±10.4%

Table i

- 3 : Estimated Relative Precision of On

- Peak CF

8 The adjusted model results reflect the mean coincident value during the entire month of June and are therefore significantly lower.



### Based on ISO

- NE characterization of the summer of 2007 as normal with respect to temperature and humidity, we recommend that both the 2007 On

- Peak CFs and 2007 Seasonal Peak CFs be used for future year estimates of Demand Reduction Values.

## Coincidence Factor Study Residential and Commercial & Industrial Lighting Measures

Spring 2007

Prepared for: 

## New England State Program Working Group (SPWG)

Prepared by:

179 Main Street Middletown, CT 06457 (860) 346

- 5001 

## Prepared for; New England State Program Working Group (SPWG)

F or use as an Energy Efficiency Measures/Programs Reference Document for the ISO Forward Capacity Market (FCM)Acknowledgement:

The authors wish to thank all of the people at the State Program Working Group (SPWG), N ortheast Energy Efficiency Partnership (NEEP) who took the time to support and help with this study. Regrettably, we cannot thank everyone individually, but we do want to acknowledge the contributions made by Julie Michals, Jeff Schleg e l Tom Belair

, Carol White, Chris Neme , Paul Horowitz, Ralph Prahl and Elizabeth Titus . The data, insight, and support provided by these individuals helped to establish the foundation for this report. RLW assumes sole responsibility for any errors or omissions in this report.

### Table of Contents

E XECUTIVE S UMMARY

................................ ................................ ................................ .. 1 Residential Lighting Coincidence Factor Results

................................ ................................ ....... 2 Commercial & Industrial Lighting Coincidence Factor Results

................................ ................. 5 Commercial & Industrial Occupancy Sensor Coincidence Factor Results

................................ . 9NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 1

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT Executive Summary The New England State Program Work ing Group (SPWG) 1

• Summer On

- Peak: average weekday from 1

- 5 PM throughout June, July an d August. contracted with RLW to calculate

coincidence factors for residential and commercial & industrial (C&I) lighting measures that could be consistently applied to energy efficiency programs that may bid into the ISO

- NE Forward Capacity Market (FCM) in any of the New England states. As directed by the SPWG, the focus of this effort was on lighting measures.

Resulting coincidence factors presented in this report were developed to work as common values accepted by all New England states for the FCM that can be applied or used as appropriate ; they

are based on measures installed by energy efficiency programs in the New England states that have supported this research effort.

T his section of the report describe s the analytical results and conclus ions for the calculation of the
Coincidence Factors (CFs) for the R esidential and Commercial & Industrial L ighting measures. Energy Efficiency demand reductions can be classified in the Forward Capacity Market (FCM) as

one of three different types of asset s, based upon the performance hours that will be used for evaluation. The most straightforward type of asset is On

- Peak, because the performance hours are

fixed and defined as follows:

• Winter On

- Peak: average weekday from 5

- 7 PM throughout December and January

.

ISO

- NE hourly load data and forecast data were obtained for the past several years from recent energy efficiency program evaluations throughout New England, as describ ed in Appendix A. They w ere analyzed to determine S easonal P eak performance hours and Critical Peak performance hours

, which are defined as follows:

• Seasonal Peak Hours occur when Real Time load is equal to or greater than 90% of the 50/50 seasonal peak l oad forecast during Summer (June

– August) or Winter (December and January) months.

1 Represented by the state regulatory agencies (CT DPUC, Maine PUC, MA DOER, NH PUC, RI PUC, and VT PSB) and associated energy efficiency program administrato rs (Cape Light Compact, Efficiency Maine, Efficiency Vermont, National Grid (MA, NH & RI), Northeast Utilities (CT&MA), NSTAR, PSNH, United Illuminating, and Unitil (MA&NH)).NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 2

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT • Critical Peak Performance Hours occur when the Day Ahead Load forecast is equal to or greater than 95% of the 50/50 seasonal peak load forecast during Summer (June

– Augus t) or Winter (December and January) months and also includes shortage hours.
 Shortage hours occur during Operating Procedure 2

Coinc idence F actors (CFs) a re defined in this study as the fractions of the connected (or rated) load (based on actual lighting Watts, motor nameplate horsepower and efficiency, AC rated capacity and efficiency, etc.) reductions that actually occur during each of the seasonal demand

windows. They are the ratio of the actual demand reduction s during the coincident window s to

the maximum connec ted load reduction s

. Under this definition other issues such as d iversity and

load factor are automatically accounted for, and only the coinci dence factor will be necessary to

determine coincident demand reductions from readily observable equipment nameplate ( rated)
information. In other words, coincident demand reduction will simply be the product of the

coincidence factor and the connected eq uipment load kW reduction

. 4 (OP4) level 6 or higher

events , at level 6 the 30

- minute operating reserve begins to be depleted

.



### Residenti al Lighting Coincidence Factor Results

Table i

- 1 and Table i

- 2 provide the un weighted and weighted, S ummer O n

- P eak and W inter

O n

- P eak CFs as well as the associated relativ e precisions for all residential lighting

. The CFs were developed using only metered data that w ere acquired during the winter (December and January) or summer (June, July and August) peak months ; the number of loggers used in the analysis is provided in the tables. The weighted CFs were developed by weighting the logger files based upon the connected load that the logger represents . I n most cases the weighted resu l t s are slightly higher than the unweighted results. The CFs for the summer range from a low of

0.0 6 for June to a high of 0. 0 94 for August , with the average summer CF between 0.076 unweighted and 0.082 weighted

. If the average is carried to only two decimal places then the result is a summer average CF of 0.08 for both methodologies. The re lative precision for the

average summer on

- peak period is ±6.1% at the 80% confidence interval.

2 Operating Procedures are from the ISO

- NE to address potential capacity shortage s.NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 3

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT Sample Size Unweighted Weighted Unweighted Data Period n CF CF Rel Precision June 210 0.060 0.069 ±11.6% July 102 0.081 0.086 ±12.5% August 189 0.094 0.092 ±8.7% Average Summer 501 0.076 0.082 ±6.1% Summer On-Peak Hours 1PM - 5PM

Table i

- 1 : Summer On

- Peak CFs and Relative Precisions Residential Lighting

Sample Size Unweighted Weighted Unweighted Data Period n CF CF Rel Precision

December 282 0.263 0.281 ±6.5%

January 264 0.301 0.320 ±6.5% Average Winter 546 0.286 0.298 ±4.5% Winter On-Peak Hours 5PM - 7PM

Table i

- 2 : Winter On

- Peak CFs and Relative Precisions Residential Lighting The winter CFs as expected are higher than the summer CFs

, ranging from 0.263 for December to 0.320 for January

, with the average winter CF for all lighting at 0.286 unweighted and 0.298 weig hted. The relative precision is better during the winter peak periods primarily because the CFs are higher and there is less variation in the data, i.e. the Coefficient of Variation (Cv) is lower. The relative precision of the average winter unweighted C F is ±4.5% at the 80% confidence interval and the December and January relative precisions are both better than ±10%

at the 80% confidence interval.

The Seasonal Summer and Winter Peak performance hours were calculated using historical load

data and the 50/50 Seasonal Peak Forecasts from the most recent C apacity E nergy L oads and

T ransmission (CELT) report . The seasonal peak performance hours were weighted based upon the frequency distribution of the hours observed w h ere the load met or exceeded 90% of t he 50/50 seasonal peak forecast

, and these values were then used to calculate a weighted CF for each of the measure types. Table i

- 3 and Table i

- 4 provide the S ummer S easonal P eak and W inter

S easonal P eak CFs for a ll residential lighting. The CFs during the summer months range from a

low of about 0.0 8 for June to a high of 0.1 0 for August , with an Average Summer CF of about

0. 09 . The relative precision during each of the summer months is within the range of ± 10% at the

80% confidence interval . The W inter Seasonal Peak CFs

, as expected

, are higher than the

S ummer Seasonal Peak CFs

, ranging from 0

. 25 in December to 0. 28 in January

, with an Average

Winter Seasonal Peak CF for all lighting at 0. 26

.NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 4

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT Sample Size Unweighted Calculated Calculated Data Period n CF CV Rel Precision June 210 0.075 2.275 ±6.3% July 102 0.091 1.884 ±5.3% August 189 0.104 1.747 ±5.2% Average Summer 501 0.088 1.967 ±3.6% Summer Seasonal Peak Hours (90% of 50/50 Peak)

Table i

- 3 : Summer Seasonal Peak CFs and Relative Precisions Residential Lighting

Sample Size Unweighted Calculated Calculated Data Period n CF CV Rel Precision

December 282 0.249 1.23 ±4.5%

January 264 0.279 1.19 ±4.5% Average Winter 546 0.264 1.21 ±3.2% Winter Seasonal Peak Hours (90% of 50/50 peak)

Table i

- 4 : Winter Seasonal Peak CFs and Relative Precisions Residential Lighting

Table i

- 5 and Table i

- 6 presents a comparison of the CFs calculated for the On

- Peak

Performance hours and the Seasonal Peak Performance hours for both the summer and winter periods. The results show that the S ummer S easonal P eak CF increases over t he Summer On

Peak for each month during the summer period and the A verage S ummer CF increases by 16%

fr om 0. 076 to 0. 088

. The increase is due to a wider range of hours being included in the weighted average calculation , among them more evening hours , when the CFs are higher . The reverse is

true for the Winter S eason al Peak CFs, which is lower than the Winter On

- Peak CFs with the

A verage W inter CF decreasing by 8% from about 0.2 9 to 0.26. The de crease is due to a wider range of hours being included in the weighted average calculation , among them more morning
and afternoon hours, when the CFs are lower.

On-Peak Seasonal % Change Unweighted Unweighted Seasonal/ Data Period CF CF On-Peak June 0.060 0.075 126% July 0.081 0.091 112% August 0.094 0.104 111% Average Summer 0.076 0.088 116%

Table i

- 5 : Comparison of Summer On

- Peak and Seasonal Peak CFs Residential LightingNECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 5

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT On-Peak Seasonal % Change Unweighted Unweighted Seasonal/ Data Period CF CF On-Peak

December 0.263 0.249 95%

January 0.301 0.279 93% Average Winter 0.286 0.264 92%

Table i

- 6 : Comparison of Winter On

- Peak and Seasonal Peak CFs Residential Lighting



### Commercial & Industrial Lighting Coincidence Factor Results

A similar Coincidence Factor analysis was also conducted for Commercial and Industrial Lighting and O ccupancy S en sor measures. The logger data were analyzed by sector so that results could be applied to multiple programs with different participation rate s among the

different sectors. Table i

- 7 and Table i

- 8 provide th e On

- Peak CFs for the ten C&I sectors along with the associated relative precisions and total estimated CFs based on a logger weighted strategy and weighting each sector equally. The Summer On

- Peak CFs indicates that the Grocery sector has the highest C F of about 0.95, while the Other sector has the lowest CF of about 0.54. All of the sectors have relative precisions that are within ± 5% at the 80% confidence interval. The Grocery sector also had the highest Winter On

- Peak CF of about 0.78, while the Sch ool sector had the lowest CF of about 0.34. Once again the relative precisions were all quite good with each sector exceeding ± 10% at the 80% confidence interval. As expected the Winter On

Peak CFs were lower than the Summer On

- Peak CFs for all of the C &I lighting sectors, because the performance hours occur later in the day as C&I facilities are shutting down and lighting is being switched off.NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 6

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT Sample Size Calculated Logger Calculated Calculated Sector Type n CF Weight CV Rel Precision Grocery 37 0.948 0.026 0.179 ±1.9% Manufacturing 169 0.729 0.119 0.488 ±2.4% Medical (Hospital) 58 0.769 0.041 0.425 ±3.6% Office 259 0.750 0.183 0.438 ±1.7% Other 192 0.543 0.136 0.675 ±3.1% Restaurant 43 0.811 0.030 0.347 ±3.4% Retail 166 0.824 0.117 0.342 ±1.7% University/College 70 0.680 0.049 0.483 ±3.7% Warehouse 59 0.781 0.042 0.359 ±3.0% School 362 0.633 0.256 0.503 ±1.7% 0.704 1.000 0.747 Summer On-Peak Hours 1PM - 5PM

Total Weighted by Logger Total Equal Weight by Sector

Table i

- 7 : Summer On

- Peak CFs and Relative Precision C&I Lighting

Sample Size Calculated Logger Calculated Calculated Sector Type n CF Weight CV Rel Precision Grocery 37 0.776 0.026 0.474 ±7.1% Manufacturing 169 0.399 0.119 0.983 ±6.9% Medical (Hospital) 58 0.603 0.041 0.593 ±7.1% Office 259 0.537 0.183 0.725 ±4.1% Other 192 0.426 0.136 0.804 ±5.3% Restaurant 43 0.663 0.030 0.557 ±7.7% Retail 166 0.655 0.117 0.592 ±4.2% University/College 70 0.523 0.049 0.679 ±7.4% Warehouse 59 0.496 0.042 0.787 ±9.3% School 362 0.343 0.256 1.010 ±4.8% 0.480 1.000 0.542 Winter On-Peak Hours 5PM - 7PM

Total Weighted by Logger Total Equal Weight by Sector

Ta ble i

- 8 : Winter On

- Peak CFs and Relative Precision C&I Lighting

Table i

- 9 and Table i

- 10 provide the Summer and Winter Seasonal

- Peak CFs for the ten C&I sectors along with the as sociated relative precisions and total estimated CFs based on a logger weighted strategy and weighting each sector equally (which is the simple average of the CFs across

all sectors

. The Seasonal Peak Performance Hours were determined by analysis of histo ric ISO

- NE Load Data and Forecast Data to determine the frequency distribution for each hour where the demand was greater than or equal to 90% of the seasonal forecast. A simple probabilistic weighting scheme was applied based upon the number of observatio n s during each hour as described inNECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 7

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT section 3 of this report. The Summer Seaso n al

- Peak CFs indicates that the Grocery sector has the highest CF of about 0.9 0 , while the Other sector has the lowest CF of about 0.4 8 . All of the sectors have relative precisi ons that are within ± 5% at the 80% confidence interval during the Summer

Seasonal Peak hours . The Grocery sector also had the highest Winter On

- Peak CF of about 0.78, while the School sector had the lowest CF of about 0.34. Once again the relative preci sions were all quite good

, with each sector exceeding ± 10% at the 80% confidence interval. As expected

, the

Winter On

- Peak CFs were lower than the Summer On

- Peak CFs for all of the C&I lighting sectors, because the performance hours occur later in the da y as C&I facilities are shutting down and lighting is being switched off.

Sample Size Calculated Logger Calculated Calculated Sector Type n CF Weight CV Rel Precision Grocery 37 0.904 0.026 0.23 ±1.5% Manufacturing 169 0.671 0.119 0.52 ±1.7% Medical (Hospital) 58 0.740 0.041 0.45 ±2.5% Office 259 0.702 0.183 0.48 ±1.2% Other 192 0.476 0.136 0.75 ±3.0% Restaurant 43 0.775 0.030 0.40 ±2.5% Retail 166 0.795 0.117 0.38 ±1.2% University/College 70 0.650 0.049 0.51 ±2.5% Warehouse 59 0.727 0.042 0.41 ±2.2% School 362 0.599 0.256 0.48 ±1.1% 0.660 1.000 0.704 Total Equal Weight by Sector Summer Seasonal Peak Hours (90% of 50/50 Peak)

Total Weighted by Logger

Table i

- 9 : Summer Seasonal Peak CFs and Relative Precision C&I LightingNECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 8

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT Sample Size Calculated Logger Calculated Calculated Sector Type n CF Weight CV Rel Precision Grocery 37 0.770 0.026 0.44 ±4.6% Manufacturing 169 0.432 0.119 0.91 ±4.2% Medical (Hospital) 58 0.618 0.041 0.58 ±4.5% Office 259 0.539 0.183 0.71 ±2.6% Other 192 0.428 0.136 0.80 ±4.4% Restaurant 43 0.644 0.030 0.59 ±5.3% Retail 166 0.647 0.117 0.59 ±2.7% University/College 70 0.528 0.049 0.60 ±4.2% Warehouse 59 0.535 0.042 0.70 ±5.6% School 362 0.388 0.256 0.85 ±2.7% 0.497 1.000 0.553 Winter Seasonal Peak Hours (90% of 50/50 Peak)

Total Weighted by Logger Total Equal Weight by Sector

Table i

- 10 : Winter Seasonal Peak CFs and Relative Precision C&I Lighting

Table i

- 11 provides a comparison of the Summer On

- Peak and Seasonal Peak CFs for each of the C&I sectors, which shows that for every sector the Summer Seasonal CFs are lower than the

Summer On

- Pe ak CFs . This means that if the C&I lighting were classified as Summer Seasonal

Peak assets the demand reductions would be lower. 

### % Change On-Peak Seasonal Seasonal / Sector Type CF CF On-Peak Grocery 0.948 0.904 95% Manufacturing 0.729 0.671 92% Medical (Hospital) 0.769 0.740 96% Office 0.750 0.702 94% Other 0.543 0.476 88% Restaurant 0.811 0.775 96% Retail 0.824 0.795 96% University/College 0.680 0.650 96% Warehouse 0.781 0.727 93% School 0.633 0.599 95% Total Weighted by Logger 0.704 0.660 94% Total Equal Weight by Sector 0.747 0.704 94% Summer

Table i

- 11 : Comparison of Summer On

- Peak and Seasonal Peak CFs C&I Lighting

Table i

- 12 provides a similar comparison of the Winter On

- Peak and Seasonal Peak CFs for each of the C&I Lighting sectors. In this case the results are mixed, with 7 of the 10 sectors show ing anNECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ _________ Page 9

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT increase in the Winter Seasonal Pea k CFs compared to the Winter On

- Peak CF. This seems to indicate that in general for the winter, C&I lighting would have more demand reduction if classified

as a Seasonal Peak asset.



### % Change On-Peak Seasonal Seasonal / Sector Type CF CF On-Peak Grocery 0.776 0.770 99% Manufacturing 0.399 0.432 108% Medical (Hospital) 0.603 0.618 103% Office 0.537 0.539 101% Other 0.426 0.428 100% Restaurant 0.663 0.644 97% Retail 0.655 0.647 99% University/College 0.523 0.528 101% Warehouse 0.496 0.535 108% School 0.343 0.388 113% Total Weighted by Logger 0.480 0.497 104% Total Equal Weight by Sector 0.542 0.553 102% 

### Winter

Table i

- 12 : Comparison of Wint er On

- P eak and Seasonal Peak CFs C&I Lighting



### Commercial & Industrial Occupancy Sensor Coincidence Factor Results

Table i

- 13 and Table i

- 14 present the Summer On

- Peak and Winter On

- Peak CFs for o ccupancy

s ensors fo r seven of the ten C&I sectors as well as the total CFs for all seven sectors on a logger weighted basis and by weighting each sector equally. During the Summer On

- Peak Period the

o ccupancy s ensor s installed in the University/College sector had the highes t CF of about 0.30, while

the Other s ector had the lowest CF of about 0.02. The S ummer On

- Peak CF for the remaining sectors ranged from about 0.21 for Manufacturing to 0.27 for the Office Sector. During the Winter

On

- Peak the Office sector had the highes t CF of about 0.31 and the Other sector had the lowest CF of 0.09. The CFs for the remaining sectors ranged from a low of about 0.17 for the Warehouse sector to a high of about 0.23 for the University/College sector. The relative precision for all of the CFs were estimated by calculating the relative precision of the occupancy sensors profiles, since only aggregate savings profiles were developed for the analysis. In this case we would recommend using the logger weighted Total CF s since the relative prec ision for individual sector results are not that good particularly during the Winter period.NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ ________ Page 10

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT Sample Size Calculated Logger Estimated Estimated Data Period n CF Weight CV Rel Precision Manufacturing 12 0.210 0.035 0.688 ±12.7% Medical 59 0.234 0.170 0.602 ±5.0%

Office 69 0.270 0.199 0.559 ±4.3% Other 56 0.017 0.161 0.793 ±6.8% University/College 16 0.304 0.046 0.678 ±10.9% Warehouse 77 0.266 0.222 0.646 ±4.7% School 58 0.239 0.167 0.828 ±7.0% 0.217 1.000 0.154 Total Weighted by Logger Total Equal Weight by Sector Summer On-Peak Hours 1PM - 5PM

Table i

- 13 : Summer On

- Peak CFs and Relative Precision C&I Occupancy Sensors

Sample Size Calculated Logger Estimated Estimated Sector Type n CF Weight CV Rel Precision Manufacturing 12 0.190 0.035 1.301 ±34.1% Medical 59 0.213 0.170 0.840 ±9.9%

Office 69 0.309 0.199 1.087 ±11.9% Other 56 0.089 0.161 1.053 ±12.8% University/College 16 0.233 0.046 0.827 ±18.8% Warehouse 77 0.175 0.222 1.082 ±11.2% School 58 0.173 0.167 1.527 ±18.2% 0.197 1.000 0.138 Total Weighted by Logger Total Equal Weight by Sector Winter On-Peak Hours 5PM - 7PM

Table i

- 14 : Winter On

- Peak CFs and Relative Precision C&I Occupancy Sensors

Table i

- 15 and Table i

- 16 provide the Summer Seasonal Peak and Winter Seasonal Peak CFs

fo r the occupancy sensors for seven of the ten C&I sectors. Once again

, during the S ummer
Seasonal Peak hours the University/College sector occupancy sensors had the highest CF of about

0.28 and the Other sector had the lowest CF of about 0.02. The CFs for the remaining sectors

rang ed from about 0.20 to 0.27. The Winter Seasonal Peak CFs were similar to the Winter On

Peak results with the Office sector having the highest CF of about 0.30 and the Other sector having the lowest CF of about 0.07. Once again the relative precision of t he CFs was estimated by using the occupancy sensor profiles and the results from the Seasonal periods are better than

for the On

- Peak periods because the results were taken across more hours. The Summer Seasonal Peak estimated relative precisions for the sectors are all within ±10% at the 8 0% confidenceNECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ ________ Page 11

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT interval, and Winter estimated relative precisions are also within that range for most of the

sectors.

Sample Size Calculated Logger Estimated Estimated Data Period n CF Weight CV Rel Precision Manufacturing 12 0.198 0.035 0.712 ±8.9% Medical 59 0.239 0.170 0.649 ±3.6%

Office 69 0.274 0.199 0.606 ±3.2% Other 56 0.024 0.161 0.808 ±4.6% University/College 16 0.283 0.046 0.720 ±7.6% Warehouse 77 0.246 0.222 0.700 ±3.3% School 58 0.209 0.167 0.739 ±4.2% 0.208 1.000 0.147 Total Weighted by Logger Total Equal Weight by Sector Summer Seasonal Peak Hours (90% of 50/50 Peak)

Table i

- 15 : Summer Seasonal

- Peak CFs and Relative Precision C&I Occup ancy Sensors

Sample Size Calculated Logger Estimated Estimated Data Period n CF Weight CV Rel Precision Manufacturing 12 0.172 0.035 1.063 ±17.3% Medical 59 0.221 0.170 0.827 ±6.3%

Office 69 0.296 0.199 0.966 ±6.9% Other 56 0.066 0.161 0.990 ±7.7% University/College 16 0.231 0.046 0.819 ±11.9% Warehouse 77 0.183 0.222 0.986 ±6.6% School 58 0.159 0.167 1.140 ±8.7% 0.191 1.000 0.133 Total Weighted by Logger Total Equal Weight by Sector Winter Seasonal Peak Hours (90% of 50/50 Peak)

Table i

- 16 : Winter Seasonal

- Peak CFs and Relative Precision C&I Occupancy Sensors

Table i

- 17 and Table i

- 18 provide a comparison of the Summer and Winter On

- Peak and Seasonal Peak CFs for occupancy sensors for seven C&I sectors as well as the totals for all seven sectors calculated on a logger weighted and sector weighted basis. The results for the Summer period show that the Summer Seasonal CFs are lower than the On

- Peak CFs for four of the seven sectors and for the total CF using both calculation methods

. The results for the Winter period are similar, with five of the sectors having lower Seasonal Peak CFs and lower Total CFs using b oth

calcul ation methods. Classifying the occupancy sensors as Seasonal Peak assets would result in a slight reduction in demand savings during both periods.NECPUC New England State Working Group 2007 Coincidence Factor Study

________________________________ ________ Page 12

________________________________________________________________ _____________

March 25 , 2 00 7 Final REPORT 

### % Change On-Peak Seasonal Seasonal / Sector Type CF CF On-Peak Manufacturing 0.210 0.198 94% Medical 0.234 0.239 102%

Office 0.270 0.274 101% Other 0.017 0.024 144% University/College 0.304 0.283 93% Warehouse 0.266 0.246 92% School 0.239 0.209 87% Total Weighted by Logger 0.217 0.208 96% Total Equal Weight by Sector 0.154 0.147 96% Summer

Table i

- 17 : Comparison of Summer On

- Peak and Seasonal Peak CFs Occupancy S ensors



### % Change On-Peak Seasonal Seasonal / Sector Type CF CF On-Peak Manufacturing 0.190 0.172 90% Medical 0.213 0.221 104%

Office 0.309 0.296 96% Other 0.089 0.066 75% University/College 0.233 0.231 99% Warehouse 0.175 0.183 105% School 0.173 0.159 92% Total Weighted by Logger 0.197 0.191 97% Total Equal Weight by Sector 0.138 0.133 96% Winter

Table i

- 18 : Comparison of Winter On

- Peak and Seasonal Peak CFs Occupancy Sensors

# FINAL REPORT - UPDATE 

# M ULTIPLE 

# S MALL 

# B USINESS 

# S ERVICES 

# P ROGRAMS



# I MPACT 

# E VALUATION


# 2007

Submitted To:
Cape Light Compact

National Grid

NSTAR

Unitil
Western Massachusetts Electric CompanyNote: This September 2, 2008 update to the fi nal report includes complete data for WMECO.

Submitted by:

Summit Blue Consulting, LLC 1722 14th Street, Ste. 230 Boulder, CO 80302 720.564.1130

Prepared by:

Michael Ozog 970.407.1048 mozog@summitblue.com

Mary Klos 608.807.0083 mklos@summitblue.comSummit Blue Consulting, LLC 1 

# T ABLE OF 

# C ONTENTS 

# 1 Executive Summary............................................................................................ 2 

# 2 Introduction ....................................................................................................... 4 

# 3 Study Methodology............................................................................................. 5

3.1 Program Tracki ng Data.................................................................................... 5

3.2 Merging with B illing Da ta ................................................................................. 6

3.3 Savings Ra tios................................................................................................. 8

3.4 Billing Anal ysis ................................................................................................ 9

3.5 Applying Thes e Results................................................................................... 10 

# 4 Study Results..................................................................................................... 11

4.1 Cape Light Compact ....................................................................................... 13

4.2 National Grid.................................................................................................. 16

4.3 NSTAR........................................................................................................... 19

4.4 Unitil ............................................................................................................. 22

4.5 Western Massachusetts Electric Company ........................................................ 25 

# 5 Reliability of Estimates...................................................................................... 28

5.1 Precisio n........................................................................................................ 28

5.2 Bias I ssues..................................................................................................... 29

Appendix A – Data Cleaning Steps................................................................................ 1

Appendix B—Complete Re gression Results .................................................................. 4Summit Blue Consulting, LLC 2 

# 1 E XECUTIVE 

# S UMMARY

The five sponsoring organizations of this study, the Cape Light Compact, National Grid, NSTAR, Unitil (Fitchburg Gas and Electric Light Company d/b/a Unitil), and Western Massachusetts Electric Company (WMECO), have offered Small Business Services (SBS) programs throughout Massachusetts to small business energy consumers for several years. A larg e share of the electric energy savings from the SBS program comes from prescriptive lighting measures.

Impact evaluations have been conducted in previous y ears to obtain estimates of realization rates for the gross energy savings resulting from the prescriptive ligh ting measures that are part of the SBS program. These realization rates reflect a comparison of estimated savings from the program tracking systems to actual customer billing data to verify the gross energy savings that were achieved. The purpose of this study is to estimate similar realization rates for 2007 program participants for each individual participating sponsor.

The statistical model that was developed to estim ate the savings from the lighting measures installed through SBS was framed within the Statistically Ad justed Engineering (SAE) approach. Under this approach, the engineering estimate of savings is in cluded as an explanatory variable in a regression equation with the billed electricity consumption as th e dependent variable. The estimated coefficient on the engineering estimate of savings may be interprete d as the realization rate. That is, the coefficient indicates the percentage of the engineering estimat e of energy savings that is realized on average according to the analysis of billing records.

Table 1-1 presents the results of this sta tistical modeling effort for each sponsor.

Table 1-1. Summary of Lighting Savings Realization Rates by Sponsor

Realization Rate T-value 1 Statistically Significant at the 90% Confidence Level? Cape Light Compact 1.04 8.58 Yes
National Grid 1.00 22.38 Yes

NSTAR 0.89 23.37 Yes
Unitil 1.02 12.39 Yes
Western Massachusetts Electric 1.03 5.7 Yes

The realization rates varied from a low of 0.89 to a high of 1.04 with most of them near the value of one. This indicates there is a good correspondence betwee n initial estimates of gross energy savings and

1 The T-value is equal to the estimated realization rate divided by its standard error. It can be used directly to test the hypothesis that the realization rate is equal to zero. A T-value of 1.645 indicates there is 90% confidence that the realization rate is statistically significant and is not zero. Higher T-values indicate higher confidence levels.Summit Blue Consulting, LLC 3 verified gross energy savings for prescriptive light ing measures in the SBS program. All of the realization rates are statistically significant at the 90% confidence level.

Table 1-2 shows the precision rates that were achieved for each sponsor. The estimates for National Grid and NSTAR have a precision rate of plus or minus 7% . Sponsors with smaller numbers of customers had precision levels in the 10% to 30% range at the 90% confidence level. This wider precision range reflects the lower certainty of the coefficient estimates given greater variability and fewer observations.

Table 1-2. Confidence Intervals and Precision Levels for Realization Rates 2

Sponsor Expected Value of Realization Rate Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level Cape Light Compact 1.04 ± 19% 0.84 1.25

National Grid 1.00 ± 7% 0.92 1.07

NSTAR 0.89 ± 7% 0.82 0.95

Unitil 1.02 ± 13% 0.88 1.16

WMECO 1.03 ± 29% 0.73 1.33

The realization rates from this study should be applie d to gross energy savings estimates from engineering calculations or deemed savings to create verified gr oss energy savings estimates. Additional estimates of free-ridership and/or spillover effects would need to come from other studies and be applied to the verified gross energy savings to estimate net energy savings. The development of net energy savings estimates is beyond the scope of this study.

2 All results shown in this table are cal culated using realization rates and T-values with six decimal points. After the calculations, the results are rounded to two decimal points for reporting purposes. This rounding method was used for all similar tables in this report.Summit Blue Consulting, LLC 4 

# 2 I NTRODUCTION

The five sponsoring organizations of this study, the Cape Light Compact, National Grid, NSTAR, Unitil (Fitchburg Gas and Electric Light Company d/b/a Unitil), and Western Massachusetts Electric Company (WMECO), have offered Small Business Services (SBS) programs throughout Massachusetts to small business energy consumers for several years. A larg e share of the electric energy savings from the SBS program comes from prescriptive lighting measures.

Impact evaluations have been conducted in previous y ears to obtain estimates of realization rates for the gross energy savings resulting from the prescriptive ligh ting measures that are part of the SBS program. These realization rates reflect a comparison of estimated savings from the program tracking systems to actual customer billing data to verify the gross energy savings that were achieved. The purpose of this study is to estimate similar realization rates for 2007 program participants for each individual participating sponsor.Summit Blue Consulting, LLC 5 

# 3 S TUDY 

# M ETHODOLOGY

This chapter presents the approach u sed in the billing data analysis to estimate realization rates for the SBS programs. It describes the data that were collected , steps taken to prepare the data and the analytical methods that were used to perform the billing analysis.


# 3.1 Program Tracking Data

Program tracking data were used to identify program participants who installed lighting measures through the small business energy efficiency programs offered by the sponsors. These tracking data provided sitespecific engineering estimates for the lighti ng measure savings for each participant.

Program tracking data covered different years for eac h sponsor. Table 3-1 shows the initial installation year for lighting measures for each participant that had tracking data and matching billing data evaluated in this study.

Table 3-1. Years Covered in Program Tracking Data for Evaluation
2006 Participants Evaluated 2007 Participants Evaluated Total Cape Light Compact 100 159 259

National Grid 97 1,200 1,297

NSTAR 106 680 786

Unitil 0 22 22

Western Massachusetts Electric 68 30 98

TOTAL 371 2,091 2,462

The program tracking data that were used in the billing data analysis included the following:

• Account or Location ID

• Customer name and town

• Description of measures installed (both lighting and other measures)

• KWh savings from installed measures

• Installation date

In constructing the particip ation variable used in the billing analys is, a zero was used for all months prior to the installation date. After that date, the particip ation variable was set to the engineering estimate of kWh savings. Since single participants may have installed multiple measures that each had different installation dates, the engineering (tracking system ) estimates of savings for these additional measures were added to the participation variable after the subsequent installation dates.Summit Blue Consulting, LLC 6 

# 3.2 Merging with Billing Data

The next step in this process was to merge the progr am tracking data with the billing systems customer information and billing databases from the sponsors. Th e critical step in this process was identifying the correct billing accounts to be matched to the tracking syst em customer data. In some cases, the customer may have more than one billing account that reflects the savings identified in the tracking system. These multiple accounts, or meters, for the same customer must be identified and added together so the total savings from the measures will be reflected in the consumption data.

A location key was created to accomplish this purpose. Each unique combination of sponsor – customer name – address was assigned a unique location key in the billing data. In this way, if a customer had two accounts at the same address those accounts would be combined. Tracking data was then matched by account number to the correct location key in the billing data. 3

Table 3-2 summarizes the disposition of all data receiv ed from the five sponsors for this study. Billing records were received for 3,186 accounts. When service addresses were examined, these accounts could be combined into 3,090 unique customer locations.

Table 3-2. Summary of Data Disposition for All Sponsors

Locations Accounts Billing Records Received 3,090 3,186 Savings Records Received 2,976 Billing Records with no Savings Data 485 493 Savings Records with no Billing Data 283 First Round Match of Billing and Savings Data 2,633 2,693 No Lighting Measures 117 Mismatched Periods for Billing and Savings Data 28 No Billing Data for Period before start of Savings 26

TOTAL MATCHED BILLING AND SAVINGS DATA 2,462

Savings records were received for 2,976 accounts. When these savings records were matched to the billing data, matches were found for 2,693 of the accounts. These accounts covered 2,633 unique customer locations. This matching process left 493 accounts in the billing data that had no matching savings records, and 283 accounts in the savings data that had no matching billing records.

This level of unmatched data is to be expected in this type of longitudinal study. A customer with a particular account number may have participated in th e savings program at some point over the period of study, and then they may have moved to a new locati on or closed their business. If they moved, the new

3 The LocID (Location ID), which identifies the loca tion/premise, was used for matching NATIONAL GRID tracking system information to their billing data. The account number was used for this purpose for all other
sponsors.Summit Blue Consulting, LLC 7 occupant would have a new account number that would no longer match the account number in the tracking system. The new occupant may have complete ly different energy usage patterns, so it is not possible to use the billing data for the new occupant to estimate savings. It is often very difficult to pickup old billing data for accounts that are no longer active. This explains why some savings records do not have matching billing data.

After the two sets of data were initially matched, additional data checking was done to make sure the matched data contained the informa tion required for the billing analysis. All matched data was analyzed by location. Since this study is focusing on savin gs for lighting measures, 117 locations that did not receive any lighting measures were excluded. An ad ditional 28 locations were excluded because all of the available billing data was for the period before th e savings measures were installed. Essentially, there was no post-participation data. And, finally, 26 lo cations were dropped because they did not have any pre-participation billing data. This left a total of 2,462 locations with matc hed billing and savings data.

This matched billing and savings data were the core da ta used for the billing anal ysis for each sponsor. In order not to bias the analysis, from this point on customers were not automatically eliminated from the analysis. Instead, manual review on a case-by-case basis was done within each sponsor’s model before any decisions were made to exclude additional data.

At this point it was possible to look at some of the attributes of the matched dataset. Table 3-3 summarizes the completeness of the monthly billing data and customer’s installa tion of other measures in addition to lighting measures.

Table 3-3. Attributes of Matched Billing and Savings Data

Locations Percent

TOTAL MATCHED BILLING AND SAVINGS DATA 2,462 100%

Incomplete Billing Data for Jan 2006 through Dec 2007 894 36% Complete Billing Data for Jan 2006 through Dec 2007 1,568 64%

Only Lighting Measure Savings 2,318 94% Lighting and Other Measure Savings 144 6%

A majority of the customer locations, 64%, had a co mplete set of monthly billing data for the study period. That means they had twenty-four monthly observations with matched billing and savings data. As explained in section 3.4, the methodology for th is study utilizes monthly observations rather than annual observations. This preserves the ability to include and make use of customer locations that do not have a complete set of twenty-four months of data, which reduces the potential bias that may occur if they all had to be deleted. Some incomplete sets may be missing only one month of data, and others may be missing over twenty months. During the modeling st ep for each sponsor, each case is reviewed manually and only those that are serious outliers are removed from the analysis.

While the emphasis of this billing analysis is on th e verification of gross energy savings from lighting measures, some customer locations may install both li ghting and other types of energy saving measures. Since the billing data reflects all of their energy use and subsequent energy savi ngs, in order to get a good fit on the modeling of energy savings all measures inst alled by the customer location must be identified and included in the model. The large majority of customer locations, 94%, installed only lightingSummit Blue Consulting, LLC 8 measures so the impact of the correct specification of other measure savings is not expected to have any significant impact on the estimate of the realization rate for lighting measures.

Technical detail on the specific steps that were take n in the data cleaning process, along with a description of the format of the final merged dataset, can be found in Appendix A.


# 3.3 Savings Ratios

Another attribute of the matched tracking and billing da ta that was analyzed was the savings ratio. The savings ratio is the annual savings estimate from the tracking system as a fraction of the total kWh use from the billing records for that customer location. If a customer has multiple accounts/meters at the same location, they have be en combined for this ratio.

Table 3-4 presents the summary of the savings ratios calculated for each customer location. Over 80% of the customer locations show savings ratios less th an 40%, with the remainder spread out over higher savings ratios. This distribution is typical for this type of data.

Table 3-4. Summary of Savings Ratios Savings Ratio Locations Percent

0% to 9% 902 37%

10% to 19% 583 24%

20% to 29% 324 13%

30% to 39% 220 9%

40% to 49% 127 5%

50% to 59% 75 3%

60% to 69% 60 2%

70% to 79% 36 1%

80% to 89% 19 1%

90% to 99% 25 1%

100% or more 91 4%

TOTAL 2,462 100%

As appealing as it may be, it is not possible to creat e an exact cut-off between savings ratios that are ‘reasonable’ and those that are not since so many fact ors can influence the savings ratios for a particular customer location. Those factors include the number and type of lighting measures that were installed, the percentage of the total lighting load that receive d measures, the percentage of the overall energy use that is used for lighting, the presence of other savi ngs measures besides lighting, and potential changes in the metering arrangement or customer occupancy an d use patterns during the entire period of study.

Sponsors with larger numbers of customer loca tions showing savings ratios greater than 100% investigated some of the individual cases and found the typical explanations of customer changes, misassigned account numbers, etc. An investigation into each case is very time-consuming since each case usually has its own unique story. Since the to tal number of cases with a savings ratio greater than 100% is very small, it was determined that they wo uld not have a significant impact on the estimation ofSummit Blue Consulting, LLC 9 the overall realization rates. In keeping with the philosophy of minimizing bias by not using arbitrary cutoffs to throw out potentially usab le cases, all cases were kept in the initial modeling efforts for each sponsor. Manual inspection of indivi dual cases was then used to identif y outliers that should be excluded from each model.


# 3.4 Billing Analysis

The statistical model that was developed to estim ate the savings from the lighting measures installed through SBS was framed within the Statistically Ad justed Engineering (SAE) approach. Under this approach, the engineering (the program tracking sy stem’s) estimate of savings is included as an explanatory variable in a regression equation with the billed electricity consumption as the dependent variable. The estimated coefficient on the engineering estimate of savings is interpreted as the realization rate. That is, the coefficient indicates the percentage of the engineering estimate of energy savings that is realized on average according to the analysis of billing records.

One assumption made in using billing analysis is that the data in the program tracking systems, which is the source of the engineering data used in the st udy, accurately reflects the individual small business project installations. Any errors in the tracking system data are inherently included in this analysis and are embedded in the realization rates.

For this analysis, data are available both across cust omers (i.e., cross-sectional) and over time (i.e., timeseries). With this type of data, known as “panel” data, it becomes possible to control at once for differences across facilities as well as differences acros s periods in time through the use of a “fixedeffects” panel model. The fixed-effect refers to the assumption that differences across customers can be explained in large part by customer-specifi c intercept terms, as discussed below.

Because the consumption data in the panel model in cludes months before and after the installation of measures through the program, the period of program pa rticipation (or the participation window) may be defined specifically for each customer. This feature of the panel model allows for the pre-installation months of consumption to act as controls for post-pa rticipation months. In addition, this model, unlike annual pre/post-participation models such as annual ch ange models, does not require a full year of postparticipation data.

The fixed effects model can be viewed as a type of differencing model in which all characteristics of the customer, which (1) are independent of time and (2) determine the level of energy consumption, are captured within the customer-specific constant term s. In other words, differences in customer characteristics that cause variation in the level of ener gy consumption, such as building size and structure, are captured by constant terms representing each unique customer facility.

Algebraically, the fixed-effect pa nel data model is described as:

y it = α i + β x it +RR·E it + ε it

where:

y it = Energy consumption for site i during month t

α i = Constant term for site i

β = Vector of coefficientsSummit Blue Consulting, LLC 10 x = Vector of variables that represent factors causing changes in monthly consumption (i.e., the time-effects variables such as weather)

RR = The estimated coefficient that represents the realization rate

E it = The engineering estimate of savings for site i during month t

ε = Error term

In practice, rather than estimating a unique intercept term for each customer, an equivalent approach is employed which expresses both the dependent and indepe ndent variables in terms of deviations from the time-series means for each customer. The resulting est imated coefficients from this "deviation from the mean" approach are equal to the coefficients f ound by having facility-specific intercept terms.

That is, it can be shown that:

ii i 

### y x α β = −

This implies that the customer-specific intercept te rm captures the difference be tween the average energy use for that customer and the predicted average en ergy use (from the model) during the time period used in the model. Therefore, the fixed-effects model e xplains the month to month deviation in energy use rather than the level of energy use.

It is possible to apply the fixed effects model to the pooled data from all of the sponsors to create a single estimate of the realization rate, or it can be applie d to data for each individual sponsor to estimate a unique realization rate for each sponsor’s program. Both applications of the model were tested to determine the best fit to the data.


# 3.5 Applying These Results

The methodology of this study was designed to devel op realization rates for gross energy savings. The gross energy savings reported in the various tracki ng systems are compared to billing usage to estimate the extent to which the predicted level of gross ener gy savings actually occurred. The customer’s electric bill will reflect the reduction in energy usage that o ccurred, but it does not provide any insight into the level of free-ridership associated with that reduction in energy use and cannot reflect what the customer would have done in the absence of the program. Th e realization rate only reflects what actually happened and can be measured.

The realization rates from this study should be applie d to gross energy savings estimates from engineering calculations or deemed savings to create verified gr oss energy savings estimates. Additional estimates of free-ridership and/or spillover effects would need to come from other studies and be applied to the verified gross energy savings to estimate net energy savings. The development of net energy savings estimates is beyond the scope of this study.Summit Blue Consulting, LLC 11 

# 4 S TUDY 

# R ESULTS

This section of the report presents the results from th e study. First, there is a discussion of modeling specifications that worked well for all sponsors, and a presentation of the overall realization rate results. After this discussion there are sections for each individua l sponsor to show the results of the data cleaning effort and the SAE model developed specifically for them.

The general model specification focused on the re lationship between total energy consumption and expected savings, with a few additional explanatory variables. The explanatory variables focused on modeling the effects of weather, seasonal variation a nd economic growth on total energy consumption. Different variable specifications were tried to find the best representation of these effects.

Several weather variables were collected and tested. These included temperature, humidity, heating degree days and cooling degree days. Temperature proved to work the best in explaining energy use variations due to weather.

Both monthly and seasonal dummy variables were create d to represent seasonal variation. In general, the seasonal indicators performed better than the monthly indicators, probably because the study period time series only covered two years of data. The seasonal i ndicators were particularly helpful for modeling the fall and winter seasons, although this varied by sponsor.

Economic growth was best represented by an annual dummy variable that allowed for an increase in energy use from one year to the next. Again, the si mple specification of this variable probably worked best due to the relatively short time period covered in the data for this study.

The first modeling attempt specified a single fixed effects model for the entire group of sponsors. This pooled sponsor model did not perform well. Many of the resultant coefficients were not statistically significant. This result is not surprising since there is no reason to expect all realization rates to be identical for the independent programs.

There are many factors which could create differences in the realization rates for each utility. For example, each sponsor independently tracks their own SBS program and deve lops their own energy savings estimates. This could create differences. They may have different implementation vendors. There may be differences in the firmographics of the customers participating in each program. Weather response for each area may vary.

Given that the pooled data model did not perform we ll and there was no overriding reason to expect the same realization rate for everyone, it was decided that it would be best to develop individual models for each sponsor. Those individual models are discussed in the remainder of this section.Summit Blue Consulting, LLC 12 Table 4-1 summarizes the realization rates that we re estimated from each of the individual models.Summit Blue Consulting, LLC 13 Table 4-1. Summary of Lighting Savings Realization Rates by Sponsor

Realization Rate T-value Statistically Significant at the 90% Confidence Level? Cape Light Compact 1.04 8.58 Yes
National Grid 1.00 22.38 Yes

NSTAR 0.89 23.37 Yes
Unitil 1.02 12.39 Yes
Western Massachusetts Electric 1.03 5.7 YesSummit Blue Consulting, LLC 14 

# 4.1 Cape Light Compact

Table 4-2 summarizes the disposition of all data rece ived from Cape Light Compact for this study. Billing records were received for 390 accounts. When service addresses were examined, these accounts could be combined into 343 unique customer locations.

Table 4-2. Summary of Data Disposition for Cape Light Compact

Locations Accounts Billing Records Received 343 390 Savings Records Received 338 Billing Records with no Savings Data 71 71 Savings Records with no Billing Data 19 First Round Match of Billing and Savings Data 272 319 No Lighting Measures 5 Mismatched Periods for Billing and Savings Data 8 No Billing Data for Period before start of Savings 0

TOTAL MATCHED BILLING AND SAVINGS DATA 259

Savings records were received for 338 accounts. Wh en these savings records were matched to the billing data, matches were found for 319 of the account s. These accounts covered 272 unique customer locations. This matching process left 71 accounts in the billing data that had no matching savings records, and 19 accounts in the savings da ta that had no matching billing records.

After the two sets of data were initially matched, additional data checking was done to make sure the matched data contained the informa tion required for the billing analysis. All matched data was analyzed by location. Since this study is focusing on savings for lighting measures, 5 locations that did not receive any lighting measures were excluded. An additi onal 8 locations were excluded because all of the available billing data was for the period before the savings measures were installed. Essentially, there was no post-participation data. This left a total of 259 locations with matche d billing and savings data.

Table 4-3 summarizes the completeness of the monthly billing data and customer ’s installation of other measures in addition to lighting measures.

Table 4-3. Attributes of Matched Billing and Savings Data for CLC

Locations Percent

TOTAL MATCHED BILLING AND SAVINGS DATA 259 100%

Incomplete Billing Data for Jan 2006 through Dec 2007 33 13% Complete Billing Data for Jan 2006 through Dec 2007 226 87%Summit Blue Consulting, LLC 15 Only Lighting Measure Savings 229 88% Lighting and Other Measure Savings 30 12%

A majority of the customer locations, 87%, had a co mplete set of monthly billing data for the study period. That means they had twenty-four monthly obs ervations with matched b illing and savings data.

A similar majority of customer locations, 88%, inst alled only lighting measures so the impact of the correct specification of other measure savings should not have a significant impact on the estimate of the realization rate for lighting measures for CLC.

Another attribute of the matched tracking and billing da ta that was analyzed was the savings ratio. The savings ratio is the annual savings estimate from the tracking system as a fraction of the total kWh use from the billing records for that customer location. If a customer has multiple accounts/meters at the same location, they have been combined for this ratio.

Table 4-4 presents the summary of the savings ra tios calculated for each customer location. Eighty percent of the customer locations show savings ratio s less than 40%, with the remainder spread out over higher savings ratios. This distribution is typical for this type of data.

Table 4-4. Summary of Savings Ratios for Cape Light Compact Savings Ratio Locations Percent

0% to 9% 88 34%

10% to 19% 50 19%

20% to 29% 39 15%

30% to 39% 31 12%

40% to 49% 19 7%

50% to 59% 9 4%

60% to 69% 6 2%

70% to 79% 2 1%

80% to 89% 0 0%

90% to 99% 5 2%

100% or more 10 4%

TOTAL 259 100%

The initial modeling process began using all of this data. During the modeling process, a few customers were identified for exclusion because their c onsumption was very low or very spotty.

A problem with autocorrelation of the data was also identified. Autocorrelation means that there is a correlation in the error terms. The error terms are the difference between the actual observed energy use and the energy use predictions from the model for each month. In the perfect model, these error terms are completely random and have no correlation to each othe r. In the real world, however, autocorrelation often occurs when evaluating time-series data becau se external conditions a nd/or customer behaviorSummit Blue Consulting, LLC 16 during one time period frequently has some carryover fro m the preceding time period. Things that affect usage this month may be similar to what affected usag e last month. It is also possible that one month’s usage may be directly related to the prior month’s usage. For example, over production in one month may be followed by reduced production in the foll owing month. There are many possible causes of autocorrelation in time-series data. 4

Autocorrelation is more likely to cause model misspecifica tion in small samples than in large samples. It was determined that the model for Cape Light Compact would be improved by correcting for autocorrelation.

After making the correction, the final model result was a realization rate of 1.04 with a t-value of 8.58. This result is statistically significant at the 90% c onfidence level. These results are summarized in Table 4-5.

Table 4-5. Lighting Savings Realization Rate for Cape Light Compact

Realization Rate T-value Statistically Significant at the 90% Confidence Level? Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level

1.04 8.58 Yes ± 19% 0.84 1.25

The exact specification and results of the final fixe d effects model for Cape Light Compact can be found in Appendix B.

4 For a discussion of possible causes of autocorrelation, see pp. 139-140 in A Guide to Econometrics, Fifth Edition

, Peter Kennedy, The MIT Press, 2003.Summit Blue Consulting, LLC 17 

# 4.2 National Grid

Table 4-6 summarizes the disposition of all data rece ived from National Grid for this study. Billing records were received for 1,846 accounts. When service addresses were examined, these accounts could be combined into 1,799 unique customer locations.

Table 4-6. Summary of Data Disposition for National Grid

Locations Accounts Billing Records Received 1799 1846 Savings Records Received 1438 Billing Records with no Savings Data 403 411 Savings Records with no Billing Data 3 First Round Match of Billing and Savings Data 1424 1435 No Lighting Measures 110 Mismatched Periods for Billing and Savings Data 17 No Billing Data for Period before start of Savings 1

TOTAL MATCHED BILLING AND SAVINGS DATA 1297

Savings records were received for 1,438 accounts. When these savings records were matched to the billing data, matches were found for 1,435 of the accounts. These accounts covered 1,424 unique customer locations. This matching process left 411 accounts in the billing data that had no matching savings records, and 3 accounts in the savings data that had no matching billing records.

After the two sets of data were initially matched, additional data checking was done to make sure the matched data contained the informa tion required for the billing analysis. All matched data was analyzed by location. Since this study is focusing on savin gs for lighting measures, 110 locations that did not receive any lighting measures were excluded. An ad ditional 17 locations were excluded because all of the available billing data was for the period before th e savings measures were installed. Essentially, there was no post-participation data. And, finally, 1 lo cation was dropped because they did not have any preparticipation billing data. This left a total of 1, 297 locations with matche d billing and savings data.

At this point it was possible to look at some of th e attributes of the matched dataset for National Grid. Table 4-7 summarizes the completeness of the monthly billing data and customer ’s installation of other measures in addition to lighting measures.

Table 4-7. Attributes of Matched Billing and Savings Data for National Grid

Locations Percent

TOTAL MATCHED BILLING AND SAVINGS DATA 1,297 100%

Incomplete Billing Data for Jan 2006 through Dec 2007 418 32% Complete Billing Data for Jan 2006 through Dec 2007 879 68%Summit Blue Consulting, LLC 18 Only Lighting Measure Savings 1,252 96% Lighting and Other Measure Savings 45 4%

A majority of the customer locations, 68%, had a co mplete set of monthly billing data for the study period. That means they had twenty-four monthly obs ervations with matched b illing and savings data.

A large majority of customer locations, 96%, installe d only lighting measures so the impact of the correct specification of other measure savings should not have a significant impact on the estimate of the realization rate for lighting measures for National Grid.

Another attribute of the matched tracking and billing da ta that was analyzed was the savings ratio. The savings ratio is the annual savings estimate from the tracking system as a fraction of the total kWh use from the billing records for that customer location. If a customer has multiple accounts/meters at the same location, they have been combined for this ratio.

Table 4-8 presents the summary of the savings ratios calculated for each customer location. Eighty-five percent of the customer locations show savings ratio s less than 40%, with the remainder spread out over higher savings ratios. This distribution is typical for this type of data.

Table 4-8. Summary of Savings Ratios for National Grid Savings Ratio Locations Percent

0% to 9% 514 40%

10% to 19% 325 25%

20% to 29% 162 12%

30% to 39% 103 8%

40% to 49% 60 5%

50% to 59% 34 3%

60% to 69% 30 2%

70% to 79% 16 1%

80% to 89% 11 1%

90% to 99% 9 1%

100% or more 33 2%

TOTAL 1,297 100%

The initial modeling process began using all of this data. During the modeling process, two customers were identified for exclusion because they did not fit the model well. 5 After excluding these customers,

5 These two customers were identified as outliers because they dramatically changed th e results when they were included in the model, suggesting that they were both outlie rs (different from other people), and had undue influenceSummit Blue Consulting, LLC 19 the final model result was a realization rate of 1.00 wi th a t-value of 22.3. This result is statistically significant at the 90% confidence level. Th ese results are summarized in Table 4-9.

Table 4-9. Lighting Savings Realization Rate for National Grid

Realization Rate T-value Statistically Significant at the 90% Confidence Level? Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level

1.00 22.38 Yes ± 7% 0.92 1.07

The exact specification and results of the final fixe d effects model for National Grid can be found in Appendix B.

on the model. Every individual customer was tested for their influence on the model, and no other customers displayed this type of impact on the model results.Summit Blue Consulting, LLC 20 

# 4.3 NSTAR

Table 4-10 summarizes the disposition of all data rece ived from NSTAR for this study. Billing records were received for 801 accounts. When service addresses were examined, these accounts could be combined into 800 unique customer locations.

Table 4-10. Summary of Data Disposition for NSTAR

Locations Accounts Billing Records Received 800 801 Savings Records Received 1050 Billing Records with no Savings Data 0 0 Savings Records with no Billing Data 249 First Round Match of Billing and Savings Data 800 801 No Lighting Measures 0 Mismatched Periods for Billing and Savings Data 4 No Billing Data for Period before start of Savings 10

TOTAL MATCHED BILLING AND SAVINGS DATA 786

Savings records were received for 1,050 accounts. When these savings records were matched to the billing data, matches were found for 801 of the accounts. These accounts covered 800 unique customer locations. All billing data that was received was matched to a savings record, but 249 accounts in the savings data had no matching billing records.

After the two sets of data were initially matched, additional data checking was done to make sure the matched data contained the informa tion required for the billing analysis. All matched data was analyzed by location. All of the NSTAR locations had receive d lighting measures. Four locations were excluded because all of the available billing data was for the pe riod before the savings measures were installed. Essentially, there was no post-participation data. Te n locations were dropped because they did not have any pre-participation billing data. This left a total of 786 locations with matche d billing and savings data.

At this point it was possible to look at so me of the attributes of the matched dataset for NSTAR.

Table 4-11 summarizes the completeness of the monthly billing data and customer ’s installation of other measures in addition to lighting measures.

Table 4-11. Attributes of Matched Billing and Savings Data for NSTAR

Locations Percent

TOTAL MATCHED BILLING AND SAVINGS DATA 786 100%Summit Blue Consulting, LLC 21 Incomplete Billing Data for Jan 2006 through Dec 2007 436 55% Complete Billing Data for Jan 2006 through Dec 2007 350 45%

Only Lighting Measure Savings 752 96% Lighting and Other Measure Savings 34 4%

Forty-five percent of customers had a complete set of monthly billing data for the study period. That means they had twenty-four monthly observations with matched billing and savings data.

The large majority of customer locations, 96%, inst alled only lighting measures so the impact of the correct specification of other measure savings should not have a significant impact on the estimate of the realization rate for lighting measures for NSTAR.

Another attribute of the matched tracking and billing da ta that was analyzed was the savings ratio. The savings ratio is the annual savings estimate from the tracking system as a fraction of the total kWh use from the billing records for that customer location. If a customer has multiple accounts/meters at the same location, they have been combined for this ratio.

Table 4-12 presents the summary of the savings ratios calculated for each customer location. Eighty-one percent of the customer locations show savings ratio s less than 40%, with the remainder spread out over higher savings ratios. This distribution is typical for this type of data.

Table 4-12. Summary of Savings Ratios for NSTAR Savings Ratio Locations Percent

0% to 9% 275 35%

10% to 19% 183 23%

20% to 29% 107 14%

30% to 39% 71 9%

40% to 49% 40 5%

50% to 59% 27 4%

60% to 69% 16 2%

70% to 79% 11 1%

80% to 89% 7 1%

90% to 99% 9 1%

100% or more 40 5%

TOTAL 786 100%

The initial modeling process began using all of this data. During the modeling process, approximately 400 monthly observations were eliminated as outliers. This represents about 2% of the monthly data. Results from the final model are summarized in Table 4-13.

Table 4-13. Lighting Savings Realization Rate for NSTARSummit Blue Consulting, LLC 22 Realization Rate T-value Statistically Significant at the 90% Confidence Level? Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level

0.89 23.37 Yes ± 7% 0.82 0.95

The exact specification and results of the fixed eff ects model for NSTAR can be found in Appendix B.Summit Blue Consulting, LLC 23 

# 4.4 Unitil

Table 4-14 summarizes the disposition of all data r eceived from Unitil for this study. Billing records were received for 23 accounts. When service address es were examined, there were no indications of multiple accounts at the same customer location.

Table 4-14. Summary of Data Disposition for Cape Light Compact

Locations Accounts Billing Records Received 23 23 Savings Records Received 28 Billing Records with no Savings Data 1 1 Savings Records with no Billing Data 6 First Round Match of Billing and Savings Data 22 22 No Lighting Measures 0 Mismatched Periods for Billing and Savings Data 0 No Billing Data for Period before start of Savings 0

TOTAL MATCHED BILLING AND SAVINGS DATA 22

Savings records were received for 28 accounts. When these savings records were matched to the billing data, matches were found for 22 of the accounts. Th is matching process left 1 accounts in the billing data that had no matching savings reco rds, and 6 accounts in the savings data that had no matching billing records.

After the two sets of data were initially matched, additional data checking was done to make sure the matched data contained the info rmation required for the billing analysis. All 22 cases for Unitil contained the required information.

At this point it was possible to look at some of th e attributes of the matched dataset for Unitil. Table 4-15 summarizes the completeness of the monthly billing data and customer’s installa tion of other measures in addition to lighting measures.

Table 4-15. Attributes of Matched Billing and Savings Data for Unitil

Locations Percent

TOTAL MATCHED BILLING AND SAVINGS DATA 22 100%

Incomplete Billing Data for Jan 2006 through Dec 2007 0 0% Complete Billing Data for Jan 2006 through Dec 2007 22 100%

Only Lighting Measure Savings 22 100% Lighting and Other Measure Savings 0 0%Summit Blue Consulting, LLC 24 All customer locations had a complete set of monthly billing data for the study period. That means they had twenty-four monthly observations with matched billing and savings data.

In addition, all of the customer locations installe d only lighting measures so the impact of the correct specification of other measure savings will not have an y impact on the estimate of the realization rate for lighting measures for Unitil.

Another attribute of the matched tracking and billing da ta that was analyzed was the savings ratio. The savings ratio is the annual savings estimate from the tracking system as a fraction of the total kWh use from the billing records for that customer location.

Table 4-16 presents the summary of the savings ratio s calculated for each customer location. Seventyseven percent of the customer locations show savings ratios less than 40%, with the remainder spread out over higher savings ratios. This distribu tion is typical for this type of data.

Table 4-16. Summary of Savings Ratios for Unitil Savings Ratio Locations Percent

0% to 9% 5 23%

10% to 19% 4 18%

20% to 29% 4 18%

30% to 39% 4 18%

40% to 49% 2 9%

50% to 59% 0 0%

60% to 69% 2 9%

70% to 79% 1 5%

80% to 89% 0 0%

90% to 99% 0 0%

100% or more 0 0%

TOTAL 22 100%

The initial modeling process began using all of this data. During the modeling process, two customers were identified as outliers. However, due to the sm all total number of customers in Unitil dataset, extra work was done to keep as much data as possible fro m those two customers in the model. Individual monthly observations for those two customers were evaluated and only twenty-two monthly observations were identified as outliers and excluded from the mode l. This reduced the total number of monthly observations from 504 to 482.

A problem with autocorrelation of the data was also identified. Autocorrelation means that there is a correlation in the error terms. The error terms are the difference between the actual observed energy use and the energy use predictions from the model for each month. In the perfect model, these error terms are completely random and have no correlation to each othe r. In the real world, however, autocorrelation often occurs when evaluating time-series data becau se external conditions a nd/or customer behavior during one time period frequently has some carryover fro m the preceding time period. Things that affect usage this month may be similar to what affected usag e last month. It is also possible that one month’sSummit Blue Consulting, LLC 25 usage may be directly related to the prior month’s usage. For example, over production in one month may be followed by reduced production in the foll owing month. There are many possible causes of autocorrelation in time-series data. 6

Autocorrelation is more likely to cause model misspecifica tion in small samples than in large samples. It was determined that the model for Unitil would be improved by correcting for autocorrelation.

After making the correction, the final model result was a realization rate of 1.02 with a t-value of 12.39. This result is statistically significant at the 90% c onfidence level. These results are summarized in Table 4-17.

Table 4-17. Lighting Savings Realization Rate for Unitil

Realization Rate T-value Statistically Significant at the 90% Confidence Level? Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level

1.02 12.39 Yes ± 13% 0.88 1.16

The exact specification and results of the final fixed effects model for Unitil can be found in Appendix B.

6 For a discussion of possible causes of autocorrelation, see pp. 139-140 in A Guide to Econometrics, Fifth Edition

, Peter Kennedy, The MIT Press, 2003.Summit Blue Consulting, LLC 26 

# 4.5 Western Massachusetts Electric Company

Table 4-18 summarizes the disposition of all data received from Western Massachusetts Electric Company for this study. Billing records were recei ved for 126 accounts. When service addresses were examined, these accounts could be combined into 125 unique customer locations.

Table 4-18. Summary of Data Disposition for Western Massachusetts Electric Co.

Locations Accounts Billing Records Received 125 126 Savings Records Received 122 Billing Records with no Savings Data 10 10 Savings Records with no Billing Data 6 First Round Match of Billing and Savings Data 115 116 No Lighting Measures 2 Mismatched Periods for Billing and Savings Data 0 No Billing Data for Period before start of Savings 15

TOTAL MATCHED BILLING AND SAVINGS DATA 98

Savings records were received for 122 accounts. Wh en these savings records were matched to the billing data, matches were found for 116 of the account s. These accounts covered 115 unique customer locations. This matching process left 10 accounts in the billing data that had no matching savings records, and 6 accounts in the savings data that had no matching billing records.

After the two sets of data were initially matched, additional data checking was done to make sure the matched data contained the informa tion required for the billing analysis. All matched data was analyzed by location. Since this study is focusing on savings for lighting measures, 2 locations that did not receive any lighting measures were excluded. All locations had billing data available for the period after the savings measures were installed. However, there we re 15 locations that were excluded because there was no billing data for the period before the installation of the savings measures. This left a total of 98 locations with usable matc hed billing and savings data.

Table 4-19 presents some attributes of the matched da ta. The table first summarizes the completeness of the monthly billing data, and then presents informatio n on customers’ installation of other measures in addition to lighting measures.Summit Blue Consulting, LLC 27 Table 4-19. Attributes of Matched Billing and Savings Data for WMECO

Locations Percent

TOTAL MATCHED BILLING AND SAVINGS DATA 98 100%

Incomplete Billing Data for Jan 2006 through Dec 2007 7 7% Complete Billing Data for Jan 2006 through Dec 2007 91 93%

Only Lighting Measure Savings 63 64% Lighting and Other Measure Savings 35 36%

A majority of the customer locations, 93%, had a co mplete set of monthly billing data for the study period. That means they had twenty-four monthly obs ervations with matched b illing and savings data.

A majority of customer locations, 64%, installed only lighting measures while 36% of the locations installed lighting and other measures. This is a large share of locations with savings from other measures. It is important to include savings from these other m easures in the WMECO model to properly isolate the realization rate for lighting measures.

Another attribute of the matched tracking and billing da ta that was analyzed was the savings ratio. The savings ratio is the annual savings estimate from the tracking system as a fraction of the total kWh use from the billing records for that customer location. If a customer has multiple accounts/meters at the same location, they have been combined for this ratio.

Table 4-20 presents the summary of the savings ratio s calculated for each customer location. Twentynine percent of the customer locations show savi ngs ratios greater than 50% for WMECO. This is considerably higher than the typical ten percent of cas es in this range for the other sponsors. It is an indication of possible mis-match between savings and billing data. Since it is unlikely that savings are really greater than 50% of total use in this ma ny cases, the twenty-eight cases in this group were considered bad data and excl uded from the modeling process.Summit Blue Consulting, LLC 28 Table 4-20. Summary of Savings Ratios for Western Massachusetts Electric Co. Savings Ratio Locations Percent

0% to 9% 20 21%

10% to 19% 21 22%

20% to 29% 12 12%

30% to 39% 11 11%

40% to 49% 6 6%

50% to 59% 5 5%

60% to 69% 6 6%

70% to 79% 6 6%

80% to 89% 1 1%

90% to 99% 2 2%

100% or more 8 8%

TOTAL 98 100%

The initial modeling process began using the 70 cases with good data. During the modeling process, monthly observations were eliminated for four cust omers identified as outliers. Results from the final model are summarized in Table 4-21.

Table 4-21. Lighting Savings Realization Rate for Western Massachusetts Elec. Co.

Realization Rate T-value Statistically Significant at the 90% Confidence Level? Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level

1.03 5.7 Yes ± 29% 0.73 1.33

The exact specification and results of the final fixed effects model for Western Massachusetts Electric Company can be found in Appendix B.Summit Blue Consulting, LLC 29 

# 5 R ELIABILITY OF 

# E STIMATES

This section will start with a comparison of the statis tical reliability of the estimates of realization rates for each sponsor. This will be followed by an iden tification and discussion of methods used to control relevant types of bias associated with the data collection and analysis efforts that were part of this study.


# 5.1 Precision

The statistical reliability of the estimates of the rea lization rates come directly from the fixed effects regression models. Since the realization rates are estim ated as coefficients in the models, the standard errors that are calculated for each of the coefficients provide a measure of the reliability of the results. The standard errors can be evaluated at any specified confidence level to determine the range of precision for each estimate at that confidence level.

It was hoped that this study would provide estimates of realization rates with plus or minus 10% precision at the 90% confidence level. Given that this study is based on the population of participants, and not a sample, the resulting precision rates reflect the best in formation available. Table 5-1 shows the precision rates that were achieved for each sponsor. The est imates for National Grid and NSTAR have a precision rate of plus or minus 7% which is better than the 10% goal. Sponsors with smaller numbers of customers had precision levels in the 10% to 30% range at the 90% confidence level. This wider precision range reflects the lower certainty of the coefficient estim ates given greater variability and fewer observations.

Table 5-1. Confidence Intervals and Precision Levels for Realization Rates

Sponsor Expected Value of Realization Rate Precision at the 90% Confidence Level Lower Bound of Realization Rate at 90% Confidence Level Upper Bound of Realization Rate at 90% Confidence Level Cape Light Compact 1.04 ± 19% 0.84 1.25

National Grid 1.00 ± 7% 0.92 1.07

NSTAR 0.89 ± 7% 0.82 0.95

Unitil 1.02 ± 13% 0.88 1.16

WMECO 1.03 ± 29% 0.73 1.33

It is not possible to improve the precision range for Cape Light Compact, Unitil and Western Massachusetts Electric Company given that each mode l was developed using all available data for the population. There is no way to add more observations . The precision range simply is what it is.Summit Blue Consulting, LLC 30 Figure 5-1 compares the precision range for each realization rate estimate at the 90% confidence level. There is a 90% chance that the real realization rate is somewhere within the illustrated bar, and the expected value of the realization rate is the midpoint , or diamond, on the bar. This figure graphically illustrates the similarity of realization rates for a ll of the sponsors. The one exception is NSTAR which has a lower probable realization rate.Summit Blue Consulting, LLC 31 Figure 5-1. 90% Confidence Intervals for Realization Rates

0.70 0.80 0.90 1.00 1.10 1.20 1.30 1.40

Cape Light Compact National Grid NSTAR Unitil WMECORealization Rate



# 5.2 Bias Issues

The potential for bias is always a concern in any statis tical evaluation. Every effort is made to minimize potential bias in the study design and analysis methodology. Even so, bias is difficult to eliminate entirely. Potential sources of bias in this study will be identified and discussed here.

Selection bias. Selection bias refers to problems that may arise because the participants selected for evaluation may be different than the general populatio n of all participants. The methodology for this study was specifically designed to include all program participants rather than just a sample of participants. This eliminates the possibilit y of sampling error or non-response error.

A different type of selection bias does occur, though, in that the necessary data needed for analysis is not always available for all customers who participated in the program. This may happen because customers move or go out of business and they do not have comple te billing data for the study period. This type of selection bias was reduced by using a fixed effect s regression model based on monthly observations. By combining all customers into a single model, it was not necessary to exclude cust omers just because they didn’t have a complete set of data. Whatever data that was available could be used, as long as there was data available for before and after the program pa rticipation date. The use of monthly observations instead of annual data also allowed for the inclusion of more customers. While previous sections of this report thoroughly reported the customer exclusions that had to be made because of insufficient data, everySummit Blue Consulting, LLC 32 effort was made to retain information for as many customers as possible. This was done to minimize selection bias.

Measurement bias. Measurement bias occurs when measuremen t techniques do not accurately represent what has actually occurred. The me asurements used in this study were primarily billing data for energy use, tracking systems for program part icipation data, and weather data.

Billing data is generally very accurate at the meter le vel since it is the basis for customer payments and distribution company revenues. Measurement bias can occur when meter-level da ta has to be combined to match the facility that installed the energy saving measures. The potential for this type of measurement bias was minimized by paying careful attention to the facility location information and creating algorithms to combine meter information at the facility location level.

The measurement bias from the tracking systems is unkn own. Since tracking systems are unique to each sponsor, the measurement bias could easily vary by s ponsor. An important aspect of the design of this study to minimize measurement bias was the inclusion of savings for other measures besides lighting if the customer installed several types of measures. In this way, chang es to the bill related to savings from non-lighting measures could be quantified separately a nd kept from influencing the realization rate for lighting savings. Likewise, if a customer installe d multiple lighting measures at different times, measurement bias was reduced by adding each additiona l lighting increment in the appropriate month. Careful use of all the information available in th e tracking systems helped reduce measurement bias.

The weather data came from the National Oceanic and Atmospheric Administration (NOAA) database. This is a reliable source of weather measurements with a high internal standard for minimizing measurement bias. The most likely source of bias re lated to weather data could come from the selection of weather-monitoring sites to use for the study. If the chosen site does not have weather that matches the locations of the program participants, there could be some measurement bias. By agreement of the sponsors, data from the Worcester, Massachusetts monitoring site was used for this study. It was felt that this would be representative weat her data for all participants.

Even when care is taken to reduce measurement bias, it is still a real issue that can’t be completely addressed in a billing analysis. Problems like 1) mis -recording of how many lights have been changed out on a project; 2) mis-stating the wattage deltas; 3) mis-estimating hours of use; 4) errors in copying information from paper to the tracking systems, etc. create measurement bias which is difficult to correct.Summit Blue Consulting, LLC A-1 

# A PPENDIX 

# A

– D ATA 

# C LEANING 

# S TEPSSummit Blue Consulting, LLC A-1 The data received from the five participating sponso rs was first formatted to make it ready for the SAE model. Four main steps were carried out to format the data:

Step 1: Bring in tracking data

• Import the tracking data for all the sponsors. Run a program to clean up the data and move it into standard field names of the same length and type so that the files can be merged together.

• Add a sponsor name column to the records so the lines can be separated by sponsor.

• Create a measure description field which contains as much information as possible about the measures, to help with categorizing them.

• Use the Location ID instead of the Account Number for National Grid.

• If the Install Start Date is unknown, make it the same as the Install End Date.

• Create a list of all measures by sponsor. Manually assign a measure category to each. The categories are INT-Interior Lighting, EXT-Exte rior Lighting, CNT-Lighting Controls, LED-LED Exit Lights, and OTH-Other non-lighting measures. Note: If the measure description for HID lights did not identify whether it was an interior or exterior installation, all large HID lights were coded as EXT.

• Assign the Measure Categories to the measures. Sum savings for each Account Number and each Measure Category.

• If there is an InstallStartDate and an InstallS topDate, assume the savings start is halfway in between.

• For each customer measure summary, create a record for every day from the measure start to the current day. Each day is annual savings divided by 365.25.

• Summarize by year and month, creating a file whic h contains the savings for that customer and category for each calendar month. Savings can vary by month because a customer does not necessarily install all measures at the same time.

• Merge in the number of calendar days for each month.

Step 2: Bring in weather data

• Get daily weather data from NOAA for January 2006 to the present for Worcester, Massachusetts.

• Convert date and time values to a standard format.

• Get rid of duplicate hourly data.

• Summarize by day, giving maximum, minimum and average temperature and humidity values.

• Calculate Heating Degree Days and Cooling Degree Days. These are defined as: A "degree day" is a unit of measure for recording how hot or how cold it has been over a 24-hour period. The number of degree days applied to any particular day of the week is determined by calculating the mean temperature for the day and then compari ng the mean temperature to a base value of 65 degrees F. (The "mean" temperature is calculated by adding together the high for the day and the low for the day, and then dividing the result by 2. )If the mean temperature for the day is, say, 5 degrees higher than 65, then there have been 5 c ooling degree days. On the other hand, if theSummit Blue Consulting, LLC A-2 weather has been cool, and the mean temperature is, say, 55 degrees, then there have 10 heating degree days (65 minus 55 equals 10. 7

• Summarize data by calendar month.

Step 3: Bring in the billing data.

• Bring in data for each sponsor.

• Filter out records with zero billing kWh or days billed = 0

• Create a total address field for each record. It is a combination of fields that will equate to a street address and city. This will be used to determine multiple accounts/meters at one location.

• For Cape Light Compact data, eliminate true dupli cates. (If a record was for the same account and the same bill read date and it had the same kWh r eading, it was considered a duplicate record and excluded. If it matched on account and bill read da te but had a different kWh reading, it was considered an additional meter reading for that customer and added in.) Add up all kWh for records with the same account and date.

• If the billing start date is missing, look at the prev ious record’s end date and use that to estimate the start date.

• Create a unique LocationKey for each location b ased on sponsor, customer name and total address.

• Add LocationKey to all billing records.

• Split the billing cycles into days and average use per day, and re-combine to create average use per day for calendar months for each customer.

• Summarize by LocationKey and date because there ca n be multiple accounts/meters at a location. Create a field to keep number of meters per location.

Step 4: Merge all data together

The final format of the data is as shown in Table A-1.

7 http://www.srh.noaa.gov/ffc/html/degdays.shtmlSummit Blue Consulting, LLC A-3 Table A-1. Final Field Names and Descriptions Sponsor CLC (Cape Light Compact) NGRIDEI (National Grid EI Program) NGRIDSBS (National Grid SBS Program)
NSTAR
UNITIL
WMECO Location Key This identifies a unique customer location. The Location Key is created by SBC to identify all billin gs accounts/meters that should be combined for the same customer lo cation. They are unique across
sponsors. Year 4 digit year Month 2 digit month aveDailykWh Average daily kWh for all accounts at the given location during the calendar month. Multiply times the number of days in the calendar month to get monthly kWh use.

AveMeters The number of accounts/meters that were combined at this location. CalendarDays The number of days in this calendar month. INTAveSavingsbyDay Average daily kWh savings for all INTERIOR LIGHTING measures at this location. It is zero before installation and pro-rated during the month of installation. After installation, it is the annual savings/365. Multiply times the number of days in the calendar month to get monthly kWh savings. EXTAveSavingsbyDay Same as above, but for EXTERIOR LIGHTING measures CNTAveSavingsbyDay Same as above, but for LIGHTING CONTROLS measures LEDAveSavingsbyDay Same as above, but for LED EXIT LIGHT measures OTHAveSavingsbyDay Same as above, but for OTHER NON-LIGHTING measures AveMonthTemp Average temperature over the calendar month AveMonthHumid Average humidity over the calendar month AveMonthTHI Average Temperature-Humidity Index over the calendar mo AveMonthCDD Average daily Cooling Degree Days over the calendar month. Multiply times the number of days in the calendar month to get monthly CDD. AveMonthHDD Average daily Heating Degree Days over the calendar month. Multiply times the number of days in the calendar month to get monthly HDD.Summit Blue Consulting, LLC A-4 

# A PPENDIX 

# B—C OMPLETE 

# R EGRESSION 

# R ESULTSSummit Blue Consulting, LLC A-5 Cape Light Compact

Fixed Effects
Valid cases: 6626 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 1121894812.046 Degrees of freedom: 6621 R-squared: 0.008 Rbar-squared: 0.007 Residual SS: 1113205978.299 Std error of est: 410.040 F(5,6621): 10.336 Probability of F: 0.000

Standard Prob Standardized Cor with Variable Estimate Error t-value >|t| Estimate Dep Var

------------------------------------------------------------------------------- ENGEST -0.556819 0.102258 -5.445243 0.000 -0.071749 -0.050515 Y2006 -43.977805 11.024341 -3.989155 0.000 -0.052690 -0.027371 TEMP -0.222113 0.498526 -0.445540 0.656 -0.008498 0.021098 WINTER -11.459092 16.223236 -0.706338 0.480 -0.013121 -0.030007 FALL 48.945383 13.281218 3.685308 0.000 0.051131 0.048186
Full R-Squared 0.99142136

Correcting for Auto - Fixed Effects
Valid cases: 6320 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 944872919.361 Degrees of freedom: 6315 R-squared: 0.014 Rbar-squared: 0.014 Residual SS: 931411635.767 Std error of est: 384.047 F(5,6315): 18.254 Probability of F: 0.000

Standard Prob Standardized Cor with Variable Estimate Error t-value >|t| Estimate Dep Var

------------------------------------------------------------------------------- ENGEST -1.044631 0.121806 -8.576193 0.000 -0.116372 -0.083389 Y2006 -83.943844 14.135899 -5.938345 0.000 -0.081191 -0.037345 TEMP -0.266063 0.538866 -0.493746 0.622 -0.008305 -0.003017 WINTER 5.189615 16.503706 0.314451 0.753 0.005494 -0.011542 FALL 45.261941 14.112699 3.207178 0.001 0.045031 0.034562

Rho 0.39420908
Full R-Squared 0.99282231

Where:
ENGEST is the engineering estimate of savings for lighting installations Y2006 is a 1-0 indicator variable for the year (2006 = 1, 2007 = 0) TEMP is the average outdoor temperature WINTER is a 1-0 indicator for the winter season (Nov, Dec, Jan, Feb) FALL is a 1-0 indicator for the fall season (Aug, Sep, Oct)Summit Blue Consulting, LLC A-6 National Grid

Fixed Effects
Valid cases: 30687 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 1173571906.175 Degrees of freedom: 30681 R-squared: 0.024 Rbar-squared: 0.024 Residual SS: 1145032063.390 Std error of est: 193.185 F(6,30681): 127.454 Probability of F: 0.000

Standard Prob Standardized Cor with Variable Estimate Error t-value >|t| Estimate Dep Var

------------------------------------------------------------------------------- ENGEST -0.996846 0.044549 -22.376604 0.000 -0.137559 -0.121435 Y2006 -10.319893 2.492057 -4.141115 0.000 -0.025828 0.037706 TEMP 1.672938 0.109735 15.245208 0.000 0.133899 0.078244 WINTER 26.236214 3.570675 7.347691 0.000 0.062854 -0.035702 FALL 0.976119 2.872861 0.339772 0.734 0.002171 0.024890 OTH -0.405922 0.120313 -3.373897 0.001 -0.019333 -0.015588
Full R-Squared 0.97682783

Where:
ENGEST is the engineering estimate of savings for lighting installations Y2006 is a 1-0 indicator variable for the year (2006 = 1, 2007 = 0) TEMP is the average outdoor temperature WINTER is a 1-0 indicator for the winter season (Nov, Dec, Jan, Feb) FALL is a 1-0 indicator for the fall season (Aug, Sep, Oct) OTH is the engineering estimate of savings for other non-lighting installationsSummit Blue Consulting, LLC A-7 NSTAR

Fixed Effects
Valid cases: 17242 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 229178552.244 Degrees of freedom: 17237 R-squared: 0.047 Rbar-squared: 0.046 Residual SS: 218493996.234 Std error of est: 112.587 F(5,17237): 168.581 Probability of F: 0.000

Standard Prob Standardized Cor with Variable Estimate Error t-value >|t| Estimate Dep Var

------------------------------------------------------------------------------- ENGEST -0.885723 0.037906 -23.366015 0.000 -0.186509 -0.165201 Y2006 -10.676367 1.888320 -5.653896 0.000 -0.045519 0.029048 TEMP 0.531244 0.113074 4.698214 0.000 0.072038 0.104894 SPRING -26.088793 2.966681 -8.793934 0.000 -0.097974 -0.081442 WINTER -7.718591 4.090038 -1.887168 0.059 -0.031022 -0.052191
Full R-Squared 0.94147675

Where:
ENGEST is the engineering estimate of savings for lighting installations Y2006 is a 1-0 indicator variable for the year (2006 = 1, 2007 = 0) TEMP is the average outdoor temperature SPRING is a 1-0 indicator for the spring season (Mar, Apr, May) WINTER is a 1-0 indicator for the winter season (Nov, Dec, Jan, Feb)Summit Blue Consulting, LLC A-8 Unitil

Fixed Effects
Valid cases: 504 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 1730032.034 Degrees of freedom: 499 R-squared: 0.196 Rbar-squared: 0.189 Residual SS: 1391714.651 Std error of est: 52.811 F(5,499): 24.261 Probability of F: 0.000

Standard Prob Standardized Cor with Variable Estimate Error t-value >|t| Estimate Dep Var

------------------------------------------------------------------------------- ENGEST -0.930846 0.097731 -9.524597 0.000 -0.439871 -0.320634 Y2006 -23.587213 5.462856 -4.317744 0.000 -0.198191 0.032768 TEMP 1.587119 0.234567 6.766155 0.000 0.425325 0.142999 FALL -7.618144 6.145638 -1.239602 0.216 -0.056276 -0.008843 WINTER 30.358766 7.550044 4.021005 0.000 0.243257 -0.001259
Full R-Squared 0.90739409

Correcting for Auto - Fixed Effects
Valid cases: 482 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 1018574.737 Degrees of freedom: 477 R-squared: 0.289 Rbar-squared: 0.283 Residual SS: 724134.666 Std error of est: 38.963 F(5,477): 38.791 Probability of F: 0.000

Standard Prob Standardized Cor with Variable Estimate Error t-value >|t| Estimate Dep Var

------------------------------------------------------------------------------- ENGEST -1.019792 0.082285 -12.393360 0.000 -0.543793 -0.477561 Y2006 -17.849496 6.390393 -2.793177 0.005 -0.123638 0.149736 TEMP 1.223129 0.206217 5.931270 0.000 0.294394 0.077451 FALL -1.677211 5.138308 -0.326413 0.744 -0.014126 -0.053817 WINTER 26.607001 6.020478 4.419416 0.000 0.227542 0.106901

Rho 0.49804523
Full R-Squared 0.97783136

Where:
ENGEST is the engineering estimate of savings for lighting installations Y2006 is a 1-0 indicator variable for the year (2006 = 1, 2007 = 0) TEMP is the average outdoor temperature WINTER is a 1-0 indicator for the winter season (Nov, Dec, Jan, Feb) FALL is a 1-0 indicator for the fall season (Aug, Sep, Oct)Summit Blue Consulting, LLC A-9 WMECO

Fixed Effects
Valid cases: 66 Dependent variable: KWH Missing cases: 0 Deletion method: None Total SS: 59866765.910 Degrees of freedom: 1483 Mean SS: 40368.69 Std error of est: 200.919 F(66,1483): 206.86 Probability of F: 0.000

Standard Prob Variable Estimate Error t-value >|t|

--------------------------------------------------------- ENGEST -1.03228 0.1811 -5.7 0.000 Y2006 -35.4394 11.0316 -3.21 0.001 TEMP 0.548403 0.3304 1.66 0.097 OTHER 0.565913 0.1256 4.5 0.000

Full R-Squared 0.9523

Where:
ENGEST is the engineering estimate of savings for lighting installations Y2006 is a 1-0 indicator variable for the year (2006 = 1, 2007 = 0) TEMP is the average outdoor temperature OTHER is the engineering estimate of savings for other non-lighting installations

# Cape Light Compact

2007 Commercial and Industrial Programs Free-ridership and Spillover Study Final Executive Summary June 23, 2008Cape Light Compact 6/23/08



# Cape Light Compact

2007 Commercial and Industrial Programs Free-ridership and Spillover Study Final Executive Summary June 23, 2008

© PA Knowledge Limited 2008

PA Consulting Group 6410 Enterprise Lane

Suite 300 Madison, WI 53719 Tel: +1 608 443 2700 Fax: +1 608 661 5181 www.paconsulting.com Prepared by: Laura Schauer

Pam Rathbun Jeremy Kraft Eric Rambo

Version: 1.0i

Cape Light Compact.6/23/08 

### TABLE OF CONTENTS

1. Executive Summary: Cape Light Compact 1-1 1.1 Study Objectives 1-1 1.2 Study Methodology 1-1 1.3 Total Participant Free-ridership Estimates 1-2 1.4 Participant “Like” Spillover Estimates 1-5 1.5 Nonparticipant Spillover Estimates 1-71-1

Cape Light Compact.6/23/08 1. EXECUTIVE SUMMARY: CAPE LIGHT COMPACT

This Executive Summary summarizes the findings of t he program year 2007 Commercial and Industrial Free-ridership and Spillover Study for C ape Light Compact. The emphasis of this study was to assess program free-ridership, partici pant spillover, and nonparticipant spillover for the following programs offered by Cape Light Co mpact:

• New Construction

• Medium and Large Commercial and Industrial Retrofi t

• Small Commercial and Industrial Retrofit

The 2007 Free-ridership and Spillover Studies ran c oncurrently for National Grid, Cape Light Compact, United Illuminating, and Unitil.

1.1 STUDY OBJECTIVES The primary objective of the year 2007 Commercial a nd Industrial Free-ridership and Spillover Study was to assist Cape Light Compact in quantifying the net impacts of their commercial and industrial energy efficiency program s by estimating the extent of:

• Program free-ridership

• Participant “like” spillover

This executive summary provides the free-ridership and participant spillover estimates for 2007. The study also includes a nonparticipant spil lover analysis. Cape Light Compact vendors were not included in this particular portio n of the study due to lack of information; however, the results, determined in aggregate acros s all participating sponsors, are presented at the end of this section. First, a summ ary of the methodology is provided.

1.2 STUDY METHODOLOGY

The methodology used for this year’s study follows the standardized methods developed in 2003 for a group of Massachusetts energy efficiency program administrators 1

. To accomplish the above objectives, telephone surve ys were conducted with samples of 2007 program participants in each of the programs, with design professionals and equipment vendors involved in these 2007 installations. The p rogram participant sample consisted of unique utility accounts , not unique customers names. The same customer nam e, or business entity, can have multiple accounts in multiple loca tions, but program technical support and incentives are provided on behalf of an individual account. Thus, for the purposes of this study, a customer or participant is defined as a un ique account.

The majority of these telephone interviews were com pleted with program participants

1 Pamela Rathbun, Carol Sabo, and Bryan Zent, Standardization Methods for Free-ridership and Spillover Evaluation—Task 5 Final Report (Revised), prepared for National Grid, NSTAR Electric, Northeast Utilities, Unitil, and Cape Light Compact , June 16, 2003.1. Executive Summary: Cape Light Compact...

1-2

Cape Light Compact.6/23/08 between March 14 and May 2, 2008. All sampled parti cipating customers were mailed a letter on Cape Light Compact letterhead in advance of the telephone call. This letter explained the purpose of the call, informed customers that someon e would be calling them in the next couple of weeks to ask them some questions about th eir experience with the programs, and thanked them for their cooperation in advance. This advance letter and repeated call attempts resulted in a high response rate of 81 percent, whi ch increases the level of confidence in the survey results. The duration of interviews with pro gram participants averaged nine minutes.

In addition to the customer surveys, surveys were c onducted with design professionals and vendors identified by customers as being the most k nowledgeable about the decisions to install the equipment through Cape Light Compact’s programs. These surveys were used for estimating free-ridership for those installations w here the design professional/equipment vendor was more influential in the decision than th e customer.
The number of survey completions for some measure c ategories is low because the number of installations within these measure categories fo r program year 2007 was small. Thus, although a high percentage of the 2007 program part icipants completed surveys, some caution should be used when interpreting the result s.

1.3 TOTAL PARTICIPANT FREE-RIDERSHIP ESTIMATES

A program’s free-ridership rate is the percentage of program participants deemed t o be free riders. A free rider

refers to a customer who received an incentive thr ough an energy efficiency program who would have installed the sam e or a smaller quantity of the same high efficiency measure on their own within one year if the program had not been offered. For free riders, the program is assumed to have had no influ ence or only a slight influence on their equipment purchase decision. Consequently, none or only some of the energy savings of equipment purchased by this group of customers shou ld be credited to the energy efficiency program. Free riders account for costs, but not ben efits, to the program, driving benefit-cost ratios down.

For programs that offer monetary incentives for mul tiple measure categories (e.g., motors, lighting, HVAC), it is important to estimate free-r idership by specific measure category. Category-specific estimates produce feedback on the program at the level at which it actually operates and allow for cost-effectiveness testing b y measure category.

In addition, for commercial and industrial incentiv e programs, free-ridership has often been found to be highly variable among measure categorie s, making it essential to produce measure category-specific estimates. The ability to provide reliable estimates by measure category is dependent on the number of installation s within that measure category—the fewer installations, the less reliable the estimation.

It is also important to measure the extent of free-ridership for each customer. Pure free rid ers (100%) would have installed exactly the same quanti ty and type of equipment within one year in the absence of the program. Partial free riders (1–99%) are those customers who would have installed some equipment within one year on th eir own, but a smaller quantity and/or a lesser efficiency. Thus, the program had some impac t on their decision. Non-free riders (0%) are those who would not have installed any high eff iciency qualifying equipment within one year in the absence of the program services. The to tal free-ridership estimates in this report include pure, partial, and non-free riders.1. Executive Summary: Cape Light Compact...

1-3

Cape Light Compact.6/23/08 This year’s approach to estimating free riders foll ows the approach outlined in the Standardization Methods... report, which consisted of a sequential question t echnique to identify free riders. This sequential approach asks program participants about the actions they would have taken if the program had not been offere d. This approach is considered an accurate method of estimating the actual level of f ree-ridership among program participants because it addresses the program’s impact upon proj ect timing, measure quantity, and efficiency levels while explicitly recognizing that the cost of energy-efficient equipment can be a barrier to installation in the absence of utility -sponsored energy efficiency programs. This method is also recommended because it walks survey respondents through their decision process with the objective of helping them recall t he program’s impact upon all aspects of project decision-making.
One issue with the method is how to handle response s of “don’t know.” The “don’t know” responses to the initial free-ridership question ar e assigned a free-ridership value of zero percent. For these cases, we then check their respo nses to the consistency questions and their response to open-ended question and adjust th e free-ridership rate as appropriate. Note that program total free-ridership (pure and partial ) rates illustrated in the following tables are weighted by measure category kWh savings as well as the disproportionate probability of being sampled.
New Construction Participant Free-ridership Rates. Table 1-1 summarizes the total freeridership results overall and by measure category f or 2007 New Construction installations. The total free-ridership for the 2007 program year was 14.6 percent.

Table 1-1 Cape Light Compact New Construction Program Total P articipant Free-ridership Rates All 2007 Installations

Measure Description Total Participant Free-ridership Rate

# Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004

Variable Speed Drives 1/1 100.0% ±0.0% — — Lighting 3/5 13.0% ±20.2% 72.6% —

Overall New Construction 3/5 14.6% ±21.2% 61.4%

—

Overall survey and population participant counts do not equal the sum of measure category survey and population participant counts; the same participant may be represented in multiple measure categories.

No surveys were conducted with New Construction par ticipants in 2004. No surveys were done with VSD measures in 2005.1. Executive Summary: Cape Light Compact...

1-4

Cape Light Compact.6/23/08 Medium and Large C&I Retrofit Program Participant F ree-ridership Rates. Table 1-2 summarizes Products and Services free-ridership res ults for 2007 HVAC installations. Refrigeration and HVAC installations could not be a ssessed; only one account was identified for each measure, and those accounts were not reach able for this study. The total freeridership for the 2007 program year was 23.1 percen t.

Table 1-2 Cape Light Compact Medium and Large C&I Retrofit Pr ogram Total Participant Free-ridership Rates All 2007 Installations

Measure Description Total Participant Free-ridership Rate

# Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004

Variable speed Drives 2/2 19.3% ±0.0% 81.1% 22.4%
HVAC 0/1 — — — — Refrigeration 0/1 — — — — Lighting 2/2 35.2% ±0.0% — —

Overall Products & Services 4/6 23.1% ±20.0% 81.1% 19.5%

Overall survey and population participant counts do not equal the sum of measure category survey and population participant counts; the same participant may be represented in multiple measure categories. *Variable Speed Drives and Lighting measures were n ot included in the 2005 or 2004 studies under the Products and Services program.

Small C&I Retrofit Participant Free-ridership Rates. Table 1-3 summarizes the total freeridership results overall and by measure category f or 2007 Small C&I Retrofit Program installations. The total free-ridership for the 200 7 program year was 7.4 percent, slightly lower than the 2005 rate for the Small C&I Retrofit progr am.1. Executive Summary: Cape Light Compact...

1-5

Cape Light Compact.6/23/08 Table 1-3 Cape Light Compact Small C&I Retrofit Program Total Participant Free-ridership Rates All 2007 Installations

Measure Description Total Participant Free-ridership Rate
# Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004*

HVAC 22/35 2.1% ±3.1% 55.7% 8.1% Refrigeration 45/58 7.8% ±3.1% 9.0% — Lighting 65/166 7.7% ±4.2% 7.1% 5.8%

Overall Retrofit 108/215 7.4% ±2.9% 8.0% 6.1%

Overall survey and population participant counts do not equal the sum of measure category survey and population participant counts; the same participant may be represented in multiple measure categories. *Rates were pulled from the 2004 Small C&I Retrofit category. Refrigeration was not part of the Retrof it sample in 2004.

1.4 PARTICIPANT “LIKE” SPILLOVER ESTIMATES

Spillover

refers to additional energy-efficient equipment in stalled by a customer due to program influences but without any financial or tec hnical assistance from the program. Participant “like” spillover refers to the situation where a customer installed equipment through the program in the past year and then insta lled additional equipment of the same type due to program influences. In contrast to free-ride rship, spillover adds benefits to the program at no additional cost, increasing the program benef its and benefit-cost ratio.
Survey free-ridership questions were followed by qu estions designed to measure "like" spillover. These questions asked about recent purch ases (since program participation in 2007) of any additional energy-efficient equipment of the same type as installed through the program that were made without any technical or financial assistance from the uti lity. A “like” spillover estimate was computed based on how much m ore of the same energy-efficient equipment the participant installed outside the pro gram and did so because of their positive experience with the program.
One of the issues with attempting to quantify spill over savings is how to value the savings of measures installed outside the program since we are relying on customer self-reports of the quantity and efficiency of any measures installed. We used a conservative approach and reported only those measures installed outside the program that were of exactly the same type and efficiency as the ones installed through t he program. Our conservative approach allowed customers to be more certain about whether the equipment they installed outside the program was the same type as the program equipment. This, in turn, made it possible for us to use the estimated program savings for that measu re to calculate the customer’s “like” spillover savings.

When reviewing the measure category “’like’ spillov er,” it is important to consider the number of survey completions that the estimate is based up on. The number of survey completions for1. Executive Summary: Cape Light Compact...

1-6

Cape Light Compact.6/23/08 some measure categories is low because very few cus tomers in the sample installed the

measure.
New Construction Participant “Like” Spillover Rates . Table 1-4 presents the like spillover rate for year 2007 New Construction participants ov erall and by measure. The New Construction program has no spillover saving attrib utions for the purchase of like equipment outside of the program.

Table 1-4

Cape Light Compact New Construction Program Partici pant

“Like” Spillover Rates All Year 2007 Installations

Total Participant “Like” Spillover Rate Measure Description # Accounts

(Survey/Pop) 2007 2007 90% Error Margin 2005 2004

Variable Speed Drives 1/1 0.0% — 0.0% —
Lighting 3/5 0.0% — 0.0% —

Overall 3/5 0.0% — 0.0% —

Overall survey and population participant counts do not equal the sum of measure category survey and population participant counts; the same participant may be represented in multiple measure categories.

No surveys were conducted with New Construction par ticipants in 2004. No surveys were done with VSD measures in 2005.
Medium and Large C&I Retrofit Program Participant “ Like” Spillover Rates. Table 1-5 presents the like spillover rate for year 2007 Prod ucts and Services installations. The rate is lower than in 2005; however, the population and sam ple size were extremely small.

Table 1-5 Cape Light Compact Medium and Large C&I Retrofit Pr ogram Participant “Like” Spillover Rates All Year 2007 Installations

Measure Description Total Participant “Like” Spillover Rate

# Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004

Variable speed Drives 2/2 0.0% ±0.0% — —
HVAC 0/1 — — 6.3% 0.0% Refrigeration 0/1 — — — — Lighting 2/2 0.7% ±0.0% — —

Overall Products & Services 4/6 0.2% ±2.0% 6.3% 0.0%1. Executive Summary: Cape Light Compact...

1-7

Cape Light Compact.6/23/08 Overall survey and population participant counts do not equal the sum of measure category survey and population participant counts; the same participant may be represented in multiple measure categories. *Variable Speed Drives and Lighting measures were n ot included in the 2005 or 2004 studies under the Products and Services program.

Retrofit Participant “Like” Spillover Rates. Table 1-6 summarizes the like spillover rate for year 2007 Retrofit installations overall and by mea sure. The spillover rate in 2007 is slightly higher than 2005.

Table 1-6

Cape Light Compact Small C&I Retrofit Program Participant “Like” Spillover Rates All Year 2007 Installations

Measure Description Total Participant “Like” Spillover Rate
# Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 +

HVAC 22/35 25.4% ±9.3% 0.0% 0.4% Refrigeration 45/58 0.4% ±0.7% 9.0% — Lighting 65/166 0.3% ±0.9% 0.6% 3.5%

Overall Retrofit 108/215 1.7% ±1.4% 0.5% 2.7%

Overall survey and population participant counts do not equal the sum of measure category survey and population participant counts; the same participant may be represented in multiple measure categories. *Rates were pulled from the 2004 Small C&I Retrofit category. Refrigeration was not part of the Retrof it sample in 2004.

+Rates were pulled from the 2004 Small C&I Retrofit category. Refrigeration was not part of the Retrof it sample in 2004.

1.5 NONPARTICIPANT SPILLOVER ESTIMATES

Nonparticipant spillover refers to energy efficient measures installed by program nonparticipants due to the program's influence. The program can have an influence on design professionals and vendors as well as an influence o n product availability, product acceptance, customer expectations, and other market effects, al l of which may induce nonparticipants to buy high efficiency products. Total nonparticipant spillover would also include responses from nonparticipating designers and vendors.

The methodology for the 2007 study estimated only a portion of nonparticipant like-measure spillover based on responses from design profession als and vendors participating in National Grid, United Illuminating, and Unitil’s Medium and Large Commercial programs 2 . Cape Light Compact vendors were not included in this study due to insufficient data; however, two of the three vendors that were indicated within Cape Light Compacts’ vendor data overlapped with National Grid’s vendor sample and were surveyed.

2 Nonparticipant spillover for small business program s was not estimated because of the small number of vendors involved in delivering the program.1. Executive Summary: Cape Light Compact...

1-8

Cape Light Compact.6/23/08 The data for the analysis could have been collected from nonparticipants directly or from the design professionals and vendors who recommended, s old, and/or installed qualifying high efficiency equipment. We chose to survey the design professionals and vendors primarily because they could typically provide much more accu rate information about the efficiency level of installed equipment than could the nonpart icipants. Experience has shown that customers cannot provide enough data about the new equipment they have installed to allow for accurate estimates of the energy savings achiev ed from the equipment. While they usually can report what type of equipment was installed, th ey typically cannot provide sufficient information about the quantity, size, efficiency, a nd/or operation of that equipment to allow us to determine whether the equipment is "program-elig ible." On the other hand, design professionals and equipment vendors who have worked with the program are typically more knowledgeable about equipment and are familiar with what is and is not "program-eligible."

Another argument in favor of using design professio nals and equipment vendors to estimate nonparticipant spillover was that we could use data in the program tracking system database to attach kWh savings estimates to nonparticipant s pillover. In the program tracking system database, measure-specific program kWh savings are associated with each design professional and vendor who participated in the pro gram in 2007.

To determine nonparticipant spillover, design profe ssionals and equipment vendors were asked (by measure category they installed in the pr ogram) what percent of their sales were program-eligible and what percent of these sales di d not receive an incentive through the programs. They were then asked about the program’s impact on their decision to recommend/install this efficient equipment outside the program. Using the survey responses and measure savings data from the program tracking system, the participating vendor nonparticipant like spillover savings could be esti mated for each design professional/vendor and the results extrapolated to the total program s avings.

This method of estimating nonparticipant spillover is a conservative estimate for two reasons. First, not all design professionals and equipment v endors who are familiar with the programs specified and/or installed equipment through the pr ogram in 2007. Thus, we miss any nonparticipant spillover that was associated with t hese other design professionals/vendors (although it is less likely these design profession als/vendors had nonparticipant spillover if they were not involved with the program in 2007).
Second, this method only allows us to extrapolate n onparticipant spillover for those same measure categories that a particular design profess ional/vendor was associated with for the 2007 programs. Thus, if a vendor installed program- eligible equipment in other measure categories in the year 2007 outside the program, bu t none through the program, we did not capture nonparticipant spillover savings with that particular type of equipment. In essence, we measured only "like" nonparticipant spillover; that is, spillover for measures like those installed through the program in 2007.

The nonparticipant spillover results for the Medium and Large Commercial and Industrial programs are based on surveys with 106 design profe ssionals and vendors out of a population of 237 National Grid, United Illuminatin g, and Unitil vendors. Because of the significant overlap in sponsors’ territories, as we ll as vendors across sponsors, we report the results in aggregate rather than by sponsor. The an alysis indicates that the combined nonparticipant spillover from the medium and large commercial and industrial programs amounted to 2,603,307 kWh in the 2007 program year, which is approximately 2.6 percent of the total savings produced by these programs combin ed (Table 1-7).1. Executive Summary: Cape Light Compact. . .1-9Cape Light Compact.6/23/08Table 1-7Nonparticipant “Like” Spillover Results for ProgramYear 2007National Grid, Unitil, and United Illuminating VendorsA B C D E F G H ISurveyCategoriesVendorPopulationkWhSavings3Number of FirmsSurveyed withkWh Savings/Number of Firmsin Program withkWh SavingsSurveyedkWhSavings4SurveyedSavingsCoverageRate(D/B)NonparticipantSpillover fromSurveyedFirms (kWh)5EstimatedSpilloverPercent(F/D)90% CINonparticipantSpilloverExtrapolatedto Population(kWh)(B*G)Motors 102,873 6/16 38,077 37.0% 0 0.0% 0.0% 0HVAC 10,877,314 27/602,175,56520.0%79,1493.6%0.7% to 3.5%395,726VSD62,393,842 11/28802,20233.5%115,56914.4%4.8% to 25.3%344,868Lighting 56,560,136 60/15120,074,39135.3%603,5723.0%1.0% to 4.6%1,700,580Compressed Air 4,671,464 10/221,743,11237.3%60,4983.5% 1.3% to 6.3% 162,132Refrigeration 4,758,046 2/61,197,31225.2% 0 0.0% 0.0% 0Other719,474,88410/334,998,94025.7% 0 0.0% 0.0% 0Total98,838,559106/25431,029,59931.4% 858,788 2.6% 1.0% to 3.7%2,603,3073The vendor population kWh savings represents the total savings for all measures for Medium and Large C&I programs for actual vendors. Spillover is measured for each vendorassociated with the program.4The total surveyed kWh savings represents the totalsavings for all surveyed design professionals andsurveyed vendors in the program tracking system database whose namessuggested they were actual vendors, not participants.5Net of “like” spillover for the customers associated with the surveyed design professionals/vendors, as identified from the participating customer survey.6One VSD response suggested spillover but could notrespond to the percentage question (VNP3). We imputed the percentage with the values from other VSD vendors that couldrespond to this question. Only one case was considered in the imputation, with a value of 50 percent.7“Other” is a residual category consisting of measures remaining from “Custom” after equipment was reassigned to existing categories such as “Motors,” “HVAC,” or “Lighting,” as wellas process equipment, process cooling equipment, and comprehensive chillers.

# National Grid

2007 Commercial and Industrial Programs Free-ridership and Spillover Study Final Executive Summary June 23, 2008

# National Grid

2007 Commercial and Industrial Programs Free-ridership and Spillover Study Final Executive Summary June 23, 2008

© PA Knowledge Limited 2008

PA Consulting Group 6410 Enterprise Lane

Suite 300 Madison Wisconsin 53719 Tel: +1 608 443 2700 Fax: +1 608 661 5181 www.paconsulting.com Prepared by: Laura Schauer

Pam Rathbun Jeremy Kraft Eric Rambo

Version: 1.0i

National Grid 6/23/08 

### TABLE OF CONTENTS

1. Executive Summary: National Grid 1-1 1.1 Study Objectives 1-1 1.2 Study Methodology 1-1 1.3 Total Participant Free-ridership Estimates 1-2 1.4 Participant “Like” Spillover Estimates 1-7 1.5 Nonparticipant Spillover Estimates 1-111-1

National Grid 6/23/08 1. EXECUTIVE SUMMARY: NATIONAL GRID

This Executive Summary summarizes the findings of t he program year 2007 Commercial and Industrial Programs Free-ridership and Spillover St udy for National Grid customers. The purpose of this study was to assess program free-ri dership, participant spillover, and nonparticipant spillover for the following programs offered by National Grid:

• Energy Initiative

• Design 2000 plus

• Small Business Services

The 2007 Free-ridership and Spillover Studies ran c oncurrently for National Grid, Cape Light Compact, United Illuminating, and Unitil.

1.1 STUDY OBJECTIVES The primary objective of the program year 2007 Comm ercial and Industrial Programs Freeridership and Spillover Study was to assist Nationa l Grid in quantifying the net impacts of their commercial and industrial energy efficiency program s by estimating the extent of:

• Program free-ridership

• Participant “like” spillover

• Nonparticipant “like” spillover

This executive summary provides the free-ridership, participant spillover and nonparticipant spillover estimates for 2007. First, a summary of t he study methodology is provided.

1.2 STUDY METHODOLOGY

The methodology used for this year’s study follows the standardized methods developed in 2003 for a group of Massachusetts energy efficiency program administrators 1

. To accomplish the above objective, telephone survey s were conducted with samples of 2007 program participants in each of the programs and wi th design professionals and equipment vendors involved in these 2007 installations. The p rogram participant sample consisted of unique electric utility accounts , not unique customer names. The same customer name , or business identity, can have multiple accounts in mu ltiple locations, but program technical support and incentives are provided on behalf of an individual account. Thus, for the purposes of this study, a customer or participant i s defined as a unique account.

The majority of these telephone interviews were com pleted with program participants between March 5 and May 9, 2008. All sampled partic ipating customers were mailed a letter on National Grid letterhead in advance of the telep hone call. This letter explained the purpose

1 Pamela Rathbun, Carol Sabo, and Bryan Zent, Standardization Methods for Free-ridership and Spillover Evaluation—Task 5 Final Report (Revised), prepared for National Grid, NSTAR Electric, Northeast Utilities, Unitil, and Cape Light Compact , June 16, 2003.1. Executive Summary: National Grid...

1-2

National Grid 6/23/08 of the call, informed customers that someone would be calling them in the next couple of weeks to ask them some questions about their experi ence with the programs, and thanked them for their cooperation in advance. This advance letter and repeated call attempts resulted in a high response rate of 76 percent, which increa ses the level of confidence in the survey results. The duration of interviews with program pa rticipants averaged ten minutes.

In addition to the customer surveys, surveys were c onducted with:

• Design professionals and vendors identified by cus tomers as being the most knowledgeable about the decisions to install the eq uipment through National Grid’s Design 2000 plus

or Energy Initiative programs. These surveys were used for estimating free-ridership for those installations w here the design professional/equipment vendor was more influential in the decision than the customer.

• Design professionals and equipment vendors who had recommended, sold and/or installed equipment through National Grid’s Design 2000 plus or Energy Initiative programs, as well as Unitil and United Illuminating ’s Medium and Large Commercial design professionals and vendors. These surveys wer e used for estimating the extent of nonparticipant “like” spillover for the Sponsor’ s programs.
The number of survey completions for some measure c ategories is low because the number of installations within these measure categories fo r program year 2007 was small. Thus, although a high percentage of the 2007 program part icipants completed surveys, some caution should be used when interpreting the result s.

1.3 TOTAL PARTICIPANT FREE-RIDERSHIP ESTIMATES

A program’s free-ridership rate is the percentage of program participants deemed t o be free riders. A free rider

refers to a customer who received an incentive thr ough an energy efficiency program who would have installed the sam e or a smaller quantity of the same high efficiency measure on their own within one year if the program had not been offered. For free riders, the program is assumed to have had no influ ence or only a slight influence on their equipment purchase decision. Consequently, none or only some of the energy savings of equipment purchased by this group of customers shou ld be credited to the energy efficiency program. Free riders account for costs, but not ben efits, to the program, driving benefit-cost ratios down.

For programs that offer monetary incentives for mul tiple measure categories (e.g., motors, lighting, HVAC), it is important to estimate free-r idership by specific measure category. Category-specific estimates produce feedback on the program at the level at which it actually operates and allow for cost-effectiveness testing b y measure category.

In addition, for commercial and industrial incentiv e programs, free-ridership has often been found to be highly variable among measure categorie s, making it essential to produce measure category-specific estimates. The ability to provide reliable estimates by measure category is dependent on the number of installation s within that measure category—the fewer installations, the less reliable the estimation.

It is also important to measure the extent of free-ridership for each customer. Pure free rid ers (100%) would have installed exactly the same quanti ty and type of equipment within one year in the absence of the program. Partial free riders (1-99%) are those customers who would1. Executive Summary: National Grid...

1-3

National Grid 6/23/08 have installed some equipment within one year on th eir own, but a smaller quantity and/or a lesser efficiency. Thus, the program had some impac t on their decision. Non-free riders (0%) are those who would not have installed any high eff iciency qualifying equipment within one year in the absence of the program services. The to tal free-ridership estimates in this report include pure, partial, and non-free riders.

This year’s approach to estimating free riders foll ows the approach outlined in the Standardization Methods... report, which consists of a sequential question te chnique to identify free riders. This sequential approach asks program participants about the actions they would have taken if the program had not been offere d. This approach is considered an accurate method of estimating the actual level of f ree-ridership among program participants because it addresses the program’s impact upon proj ect timing, measure quantity, and efficiency levels while explicitly recognizing that the cost of energy-efficient equipment can be a barrier to installation in the absence of utility -sponsored energy efficiency programs. This method is also recommended because it walks survey respondents through their decision process with the objective of helping them recall t he program’s impact upon all aspects of project decision-making.
One issue with the method is how to handle response s of “don’t know.” The “don’t know” responses to the initial free-ridership question ar e assigned a free-ridership value of zero percent. For these cases, we then check their respo nses to the consistency questions and their response to open-ended question and adjust th e free-ridership rate as appropriate. Note that program total free-ridership (pure and partial ) rates illustrated in the following tables are weighted by measure category kWh savings as well as the disproportionate probability of being sampled. When reviewing the measure category free-ridership rates it is important to consider the number of survey completions that the estimate is based upon.1. Executive Summary: National Grid...

1-4

National Grid 6/23/08 Energy Initiative Participant Free-ridership Rates. Table 1-1 summarizes the total freeridership results overall and by measure category f or 2007 Energy Initiative installations. The overall Energy Initiative program free-ridership fo r the 2007 program year was 10.5 percent, which is higher than the 8.9 percent rate found for 2005 installations, and higher than the 6.7 percent rate found in 2004. The HVAC installations’ free-ridership rates dropped significantly in 2007, which reflects changes in program requirem ents related to HVAC efficiency levels.
The rate was highest for VSD measures (33.1 percent ), which changed significantly from previous years. A close review of the data shows th at the relatively small population, and high savers with high free-ridership rates, drove the ra te up for this measure category.

Table 1-1 National Grid Energy Initiative Program Total Parti cipant Free-ridership Rates All 2007 Installations

Total Participant Free-ridership Rate Measure Description # Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 2002

Custom: Process, HVAC, Drivepower, Lighting, O&M 50/118 7.4% ±4.6% 8.4% 5.5% 10.9%

Motors 6/6 21.0% ±0.0% 32.4% 15.2% 9.1% HVAC 23/29 12.5% ±5.5% 40.9% 0.3% 43.8% VSD 12/18 33.1% ±12.9% 2.0% 0.1% 0.5% Lighting: T8, Other Fluorescent, CFL, Controls, HID, LED Exit Signs 93/397 10.2% ±4.5% 5.9% 7.5% 14.5%

Compressed Air 15/19 5.7% ±4.5% 36.8% 26.4% 17.1% Overall Energy Initiative Program 178/558 10.5% ±3.1% 8.9% 6.7% 15.3%

Overall survey and population participant counts do not equal the sum of measure category survey and p opulation participant counts; the same participant may be rep resented in multiple measure categories.1. Executive Summary: National Grid...

1-5

National Grid 6/23/08 Design 2000 plus

Participant Free-ridership Rates. Table 1-2 summarizes Design 2000 plus

free-ridership results overall and by measure cate gory for 2007 installations. The overall Design 2000 plus program free-ridership rate for 2007 was 19.0 perc ent, which is lower than the 21.7 percent rate found for 2005 installat ions and slightly higher than the 18.1 percent rate found for 2004 installations. As with the Energy Initiative program, HVAC measures’ free-ridership rates dropped significantl y between 2005 and 2007. VSD and compressed air measures had the highest free-riders hip rates.

Table 1-2 National Grid Design 2000 plus Program Total Participant Free-ridership Rates All 2007 Installations

Total Participant Free-ridership Rate

Measure Description # Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 2002

Custom: Process, HVAC, Drivepower, Lighting 42/84 14.6% ±6.3% 10.1% 2.4% 16.6%

Motor: New 37/88 28.8% ±9.3% 15.1% 15.5% 40.1% Motor: Failed/Stock 22/32 11.2% ±6.2% 23.5% 21.8% 2 3.2% HVAC (Unitary): Packaged A/C and Water Source Heat Pump 55/124 14.8% ±5.9% 56.4% 5.3% 40.0%

HVAC (Non-unitary): Programmable Thermostat, Energy Management System, Chiller, Control 36/69 8.3% ±5.2% 49.1% 64.0% 39.5%

VSD 10/16 58.7% ±15.7% 8.6% 43.5% 19.3% Lighting: T8, Other Fluorescent, CFL, Controls, HID, LED Exit Signs 51/123 27.9% ±7.9% 28.2% 50.6% 36.3%

Compressed Air 55/113 33.8% ±7.5% 32.7% 17.6% 20.6% Overall Design 2000 plus Program 241/516 19.0% ±3.0% 21.7% 18.1% 27.2%

Overall survey and population participant counts do not equal the sum of measure category survey and p opulation participant counts; the same participant may be rep resented in multiple measure categories.

Custom measures include six Comprehensive Program p articipants. The Comprehensive Program free-ridersh ip rate was zero percent for 2007 installations.1. Executive Summary: National Grid...

1-6

National Grid 6/23/08 Small Business Services Participant Free-ridership Rates. Table 1-3 summarizes the results overall and by measure category for 2007 Sm all Business Services installations. The total free-ridership rate for 2007 was 5.5 percent, which is slightly higher than past years.

Table 1-3 National Grid Small Business Services Program Total Participant Free-ridership Rates All 2007 Installations

Total Participant Free-ridership Rate Measure Description # Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 2002

Lighting: Fluorescent with ELIG/3’4’8’ Lamp & EEMAG, Standard Ballast, Exit Sign, Compact Fluorescent, HID 243/1,329 5.8% ±2.2% 2.3% 1.0% 1.0%

Non-lighting: Water Heater Wrap, Programmable Thermostat, Economizer 64/155 3.6% ±2.9% 2.0% 1.3% 1.0%

Overall Small Business Services Program 284/1,441 5.5% ± 2.0% 2.2% 1.0% 1.0%

Overall survey and population participant counts do not equal the sum of measure category survey and p opulation participant counts; the same participant may be rep resented in multiple measure categories.1. Executive Summary: National Grid...

1-7

National Grid 6/23/08 State-level Participant Free-ridership Rates. Table 1-4 shows Massachusetts has a total 2007 free-ridership rate of 12.7 percent, New Hamps hire 21.1 percent, and Rhode Island 10.0 percent.

Table 1-4

National Grid State-level Total Participant Free-ri dership Rates

All 2007 Installations

Total Participant Free-ridership Rates

Massachusetts New Hampshire Rhode Island Program #

Accounts Surveyed 2007

TotalFreeridership 90%

Error Margin #

Accounts Surveyed 2007

TotalFreeridership 90%

Error Margin #

Accounts Surveyed 2007

TotalFreeridership 90%

Error Margin

Energy Initiative 128 11.3% ±3.8% 8 24.2% ±11.1% 42 7.9% ±5.9%

Design 2000plus 154 18.6% ±3.8% 8 51.1% ±15.2% 79 18.1% ±5.1%

Small Business Services 127 6.9% ±3.4% 45 0.5% ±0.8% 112 4.3% ±2.8%

ALL 392 12.7% ±2.4% 60 21.1% ±3.4% 228 10.0% ±2.8%

Overall survey and population participant counts do not equal the sum of program survey and population participant counts; the same participant may be rep resented in multiple programs.

1.4 PARTICIPANT “LIKE” SPILLOVER ESTIMATES

Spillover

refers to additional energy-efficient equipment in stalled by a customer due to program influences but without any financial or tec hnical assistance from the program. Participant “like” spillover refers to the situation where a customer installed equipment through the program in the past year and then insta lled additional equipment of the same type due to program influences. In contrast to free-ride rship, spillover adds benefits to the program at no additional cost, increasing the program benef its and benefit-cost ratio.
Survey free-ridership questions were followed by qu estions designed to measure "like" spillover. These questions asked about recent purch ases (since program participation in 2007) of any additional energy-efficient equipment of the same type as installed through the program that were made without any technical or financial assistance from the uti lity. A “like” spillover estimate was computed based on how much m ore of the same energy-efficient equipment the participant installed outside the pro gram and did so because of their positive experience with the program.
One of the issues with attempting to quantify spill over savings is how to value the savings of measures installed outside the program since we are relying on customer self-reports of the quantity and efficiency of any measures installed. We used a conservative approach and reported only those measures installed outside the program that were of exactly the same type and efficiency as the ones installed through t he program. Our conservative approach allowed customers to be more certain about whether the equipment they installed outside the1. Executive Summary: National Grid...

1-8

National Grid 6/23/08 program was the same type as the program equipment. This, in turn, makes it possible for us to use the estimated program savings for that measu re to calculate the customer’s “like” spillover savings.

Note that the “like” spillover rates illustrated in the following tables are weighted by measure category kWh savings and the disproportionate proba bility of being sampled. When reviewing the measure category “’like’ spillover,” it is impo rtant to consider the number of survey completions that the estimate is based upon. The nu mber of survey completions for some measure categories is low because very few customer s in the sample installed the measure. Thus, although a high percentage of the 2007 progra m participants completed surveys, some caution should be used when interpreting the result s.
Energy Initiative Participant “Like” Spillover Rate s . Table 1-5 presents the “like” spillover rates for year 2007 Energy Initiative participants, overall and by measure category. The estimate of “like” spillover savings attributable t o the overall Energy Initiative program for the purchase of like equipment outside of the program i s 3.3%, which is slightly higher than previous years’ spillover rates.

Table 1-5

National Grid Energy Initiative Program Participant “Like” Spillover Rates All Year 2007 Installations

Total Participant “Like” Spillover Rate

Measure Description # Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 2002

Custom: Process, HVAC, Drivepower, Lighting, O&M 50/118 8.5% ±4.9% 1.9% 0.7% 1.4%

Motors 6/6 13.9% ±0.0% 1.7% 0.0% 2.5%
HVAC 23/29 5.2% ±3.5% 0.3% 27.0% 7.9%
VSD 12/18 0.0% ±0.0% 0.2% 16.2% 12.0% Lighting: T8, Other Fluorescent, CFL, Controls, HID, LED Exit Signs 93/397 1.8% ±2.0% 1.4% 0.4% 2.1%

Compressed Air 15/19 0.0% ±0.0% 0.0% 0.0% 0.0% Overall Energy Initiative Program 178/558 3.3% ±1.8% 1.4% 2.7% 2.7%

Overall survey and population participant counts do not equal the sum of measure category survey and p opulation participant counts; the same participant may be rep resented in multiple measure categories.1. Executive Summary: National Grid...

1-9

National Grid 6/23/08 Design 2000 plus

Participant “Like” Spillover Rates. Table 1-6 presents the “like” spillover rates for year 2007 Design 2000 plus installations overall and by measure category. The overall Design 2000 plus program spillover rate for the 2007 program year w as 5.9 percent, which is lower than 8.8 percent rate found for 2005 installations but higher than 2004 and 2002 rates.

Table 1-6 National Grid Design 2000 plus Program Participant “Like” Spillover Rates All Year 2007 Installations

Total Participant “Like” Spillover Rate

Measure Description # Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 2002

Custom: Process, HVAC, Drivepower, Lighting 42/84 3.7% ±3.4% 11.3% 1.8% 1.0%

Motor: New 37/88 9.2% ±5.9% 5.3% 2.4% 8.4% Motor: Failed/Stock 22/32 4.5% ±4.0% 10.4% 3.4% 21. 4% HVAC (Unitary): Packaged A/C and Water Source Heat Pump 55/124 5.9% ±3.9% 6.6% 2.5% 2.3%

HVAC (Non-unitary): Programmable Thermostat, Energy Management System, Chiller, Control 36/69 15.2% ±6.8% 0.2% 1.4% 4.1%

VSD 10/16 0.0% ±0.0% 0.0% NA 3.9% Lighting: T8, Other Fluorescent, CFL, Controls, HID, LED Exit Signs 51/123 13.4% ±6.0% 8.4% 0.3% 1.1%

Compressed Air 55/113 0.0% ±0.0% 0.2% 1.0% 0.2% Overall Design 2000 plus Program 241/516 5.9% ±1.8% 8.8% 1.4% 2.0%

Overall survey and population participant counts do not equal the sum of measure category survey and p opulation participant counts; the same participant may be rep resented in multiple measure categories.

Custom measures include six Comprehensive Program p articipants. The Comprehensive Program spillover ra te was zero percent for 2007 installations.1. Executive Summary: National Grid...

1-10

National Grid 6/23/08 Small Business Services Participant “Like” Spillover Rates. Table 1-7 summarizes the “like” spillover rates for year 2007 Small Business installations overall and by measure category. The overall Small Business Services progr am spillover rate was 2.0 percent, which is comparable with previous years’ rates.

Table 1-7 National Grid Small Business Services Program Parti cipant “Like” Spillover Rates All Year 2007 Installations

Total Participant “Like” Spillover Rate

Measure Description # Accounts (Survey/Pop) 2007 2007 90% Error Margin 2005 2004 2002

Lighting: Fluorescent with ELIG/3’4’8’ Lamp & EEMAG, Standard Ballast, Exit Sign, Compact Fluorescent, HID 243/1,329 2.2% ±1.4% 2.0% 0.5% 2.0%

Non-lighting: Water Heater Wrap, Programmable Thermostat, Economizer 64/155 0.7% ±1.3% 1.0% 0.3% 0.6%

Overall Small Business Services Program 284/1,441 2.0% ± 1.2% 1.9% 0.4% 1.9%

Overall survey and population participant counts do not equal the sum of measure category survey and p opulation participant counts; the same participant may be rep resented in multiple measure categories.1. Executive Summary: National Grid...

1-11

National Grid 6/23/08 State-level Participant “Like” Spillover Rates. Table 1-8 shows the “like” spillover rate for Massachusetts is 3.5 percent, New Hampshire is 2.1 percent, and Rhode Island is 4.6 percent. The surveyed number represents the number of accounts surveyed within each state.

Table 1-8 National Grid State-level Program Participant “Like ” Spillover Rates by Account All Year 2007 Installations

Total Participant “Like” Spillover Rates
Massachusetts New Hampshire Rhode Island Program #

Accounts Surveyed 2007

Spillover 90%

Error Margin #

Accounts Surveyed 2007

Spillover 90%

Error Margin #

Accounts Surveyed 2007

Spillover 90%

Error Margin Energy Initiative 128 4.3% ±2.4% 8 0.0% — 42 0.4% ±1.4%
Design 2000plus 154 2.2% ±1.5% 8 0.0% — 78 14.2% ±4.6%

Small Business Services 127 2.5% ±2.1% 45 3.9% ±2.1% 112 1.1% ±1.5%

ALL 392 3.5% ±1.3% 60 2.1% ±1.4% 228 4.6% ±2.0%

Overall survey and population participant counts do not equal the sum of program survey and population participant counts; the same participant may be rep resented in multiple programs.

1.5 NONPARTICIPANT SPILLOVER ESTIMATES

Nonparticipant spillover refers to energy efficient measures installed by program nonparticipants due to the program's influence. The program can have an influence on design professionals and vendors as well as an influence o n product availability, product acceptance, customer expectations, and other market effects, al l of which may induce nonparticipants to buy high efficiency products. Total nonparticipant spillover would also include responses from nonparticipating designers and vendors.

The methodology for the 2007 study estimated only a portion of nonparticipant like-measure spillover based on responses from design profession als and vendors participating in National Grid, United Illuminating, and Unitil’s Medium and Large Commercial programs 2 . Cape Light Compact vendors were not included in this study due to insufficient data; however, two of the three vendors that were indicated within Cape Light Compacts’ vendor data overlapped with National Grid’s vendor sample and were surveyed.

The data for the analysis could have been collected from nonparticipants directly or from the design professionals and vendors who recommended, s old, and/or installed qualifying high

2 Nonparticipant spillover for small business program s was not estimated because of the small number of vendors involved in delivering the program.1. Executive Summary: National Grid...

1-12

National Grid 6/23/08 efficiency equipment. We chose to survey the design professionals and vendors primarily because they could typically provide much more accu rate information about the efficiency level of installed equipment than could the nonpart icipants. Experience has shown that customers cannot provide enough data about the new equipment they have installed to allow for accurate estimates of the energy savings achiev ed from the equipment. While they usually can report what type of equipment was installed, th ey typically cannot provide sufficient information about the quantity, size, efficiency, a nd/or operation of that equipment to allow us to determine whether the equipment is "program-elig ible." On the other hand, design professionals and equipment vendors who have worked with the program are typically more knowledgeable about equipment and are familiar with what is and is not "program-eligible."

Another argument in favor of using design professio nals and equipment vendors to estimate nonparticipant spillover was that we could use data in the program tracking system database to attach kWh savings estimates to nonparticipant s pillover. In the program tracking system database, measure-specific program kWh savings are associated with each design professional and vendor who participated in the pro gram in 2007.

To determine nonparticipant spillover, design profe ssionals and equipment vendors were asked (by measure category they installed in the pr ogram) what percent of their sales were program-eligible and what percent of these sales di d not receive an incentive through the programs. They were then asked about the program’s impact on their decision to recommend/install this efficient equipment outside the program. Using the survey responses and measure savings data from the program tracking system, the participating vendor nonparticipant like spillover savings could be esti mated for each design professional/vendor and the results extrapolated to the total program s avings.

This method of estimating nonparticipant spillover is a conservative estimate for two reasons. First, not all design professionals and equipment v endors who are familiar with the programs specified and/or installed equipment through the pr ogram in 2007. Thus, we miss any nonparticipant spillover that was associated with t hese other design professionals/vendors (although it is less likely these design profession als/vendors had nonparticipant spillover if they were not involved with the program in 2007).
Second, this method only allows us to extrapolate n onparticipant spillover for those same measure categories that a particular design profess ional/vendor was associated with for the 2007 programs. Thus, if a vendor installed program- eligible equipment in other measure categories in the year 2007 outside the program, bu t none through the program, we did not capture nonparticipant spillover savings with that particular type of equipment. In essence, we measured only "like" nonparticipant spillover; that is, spillover for measures like those installed through the program in 2007.

The nonparticipant spillover results for the Medium and Large Commercial and Industrial programs are based on surveys with 106 design profe ssionals and vendors out of a population of 237 National Grid, United Illuminatin g, and Unitil vendors. Because of the significant overlap in sponsors’ territories, as we ll as vendors across sponsors, we report the results in aggregate rather than by sponsor. The an alysis indicates that the combined nonparticipant spillover from the medium and large commercial and industrial programs amounted to 2,603,307 kWh in the 2007 program year, which is approximately 2.6 percent% of the total savings produced by these programs com bined (Table 1-9).1. Executive Summary: National Grid...1-13National Grid 6/23/08Table 1-9Nonparticipant “Like” Spillover Results for ProgramYear 2007National Grid, Unitil, and United Illuminating VendorsA B C D E F G H ISurveyCategoriesVendorPopulationkWhSavings3Number of FirmsSurveyed withkWh Savings/Number of Firmsin Program withkWh SavingsSurveyedkWhSavings4SurveyedSavingsCoverageRate(D/B)NonparticipantSpillover fromSurveyedFirms (kWh)5EstimatedSpilloverPercent(F/D)90% CINonparticipantSpilloverExtrapolatedto Population(kWh)(B*G)Motors 102,873 6/16 38,077 37.0% 0 0.0% 0.0% 0HVAC 10,877,314 27/602,175,56520.0%79,1493.6%0.7% to 3.5%395,726VSD62,393,842 11/28802,20233.5%115,56914.4%4.8% to 25.3%344,868Lighting 56,560,136 60/15120,074,39135.3%603,5723.0%1.0% to 4.6%1,700,580Compressed Air 4,671,464 10/221,743,11237.3%60,4983.5% 1.3% to 6.3% 162,132Refrigeration 4,758,046 2/61,197,31225.2% 0 0.0% 0.0% 0Other719,474,88410/334,998,94025.7% 0 0.0% 0.0% 0Total98,838,559106/25431,029,59931.4% 858,788 2.6% 1.0% to 3.7%2,603,3073The vendor population kWh savings represents the total savings for all measures for Medium and Large C&I programs for actual vendors. Spillover is measured for each vendorassociated with the program.4The total surveyed kWh savings represents the totalsavings for all surveyed design professionals andsurveyed vendors in the program tracking system database whose namessuggested they were actual vendors, not participants.5Net of “like” spillover for the customers associated with the surveyed design professionals/vendors, as identified from the participating customer survey.6One VSD response suggested spillover but could notrespond to the percentage question (VNP3). We imputed the percentage with the values from other VSD vendors that couldrespond to this question. Only one case was considered in the imputation, with a value of 50 percent.7“Other” is a residual category consisting of measures remaining from “Custom” after equipment was reassigned to existing categories such as “Motors,” “HVAC,” or “Lighting,” as wellas process equipment, process cooling equipment, and comprehensive chillers.