Evaluation of Retrofit 

Delivery Packages

M. Berman, P. Smith, and E. Porse

Alliance for Residential Building Innovation (ARBI)

July 2013
NOTICE

This report was prepared as an account of work sponsored by an agency of the United States 

government. Neither the United States government nor any agency thereof, nor any of their 

employees, subcontractors, or affiliated partners makes any warranty, express or implied, or 

assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any 

information, apparatus, product, or process disclosed, or represents that its use would not infringe 

privately owned rights. Reference herein to any specific commercial product, process, or service 

by trade name, trademark, manufacturer, or otherwise does not necessarily constitute or imply its 

endorsement, recommendation, or favoring by the United States government or any agency 

thereof. The views and opinions of authors expressed herein do not necessarily state or reflect 

those of the United States government or any agency thereof.

Available electronically at http://www.osti.gov/bridge

Available for a processing fee to U.S. Department of Energy

and its contractors, in paper, from:

U.S. Department of Energy

Office of Scientific and Technical Information

P.O. Box 62

Oak Ridge, TN 37831-0062

phone: 865.576.8401

fax: 865.576.5728

email: mailto:reports@adonis.osti.gov

Available for sale to the public, in paper, from:

U.S. Department of Commerce

National Technical Information Service

5285 Port Royal Road

Springfield, VA 22161

phone: 800.553.6847

fax: 703.605.6900

email: orders@ntis.fedworld.gov

online ordering: http://www.ntis.gov/ordering.htm

Printed on paper containing at least 50% wastepaper, including 20% postconsumer waste
iii

Evaluation of Retrofit Delivery Packages

Prepared for: 

The National Renewable Energy Laboratory 

On behalf of the U.S. Department of Energy’s Building America Program 

Office of Energy Efficiency and Renewable Energy 

15013 Denver West Parkway

Golden, CO 80401 

NREL Contract No. DE-AC36-08GO28308 

 

Prepared by: 

M. Berman, P. Smith, and E. Porse

Alliance for Residential Building Innovation (ARBI)

Davis Energy Group, Team Lead 

123 C Street 

Davis, California 95616 

NREL Technical Monitor: Michael Gestwick

Prepared under Subcontract No. KNDJ-0-40340-00

July 2013
iv

[This page left blank]
v 

Contents

List of Figures ........................................................................................................................................... vii

List of Tables ............................................................................................................................................ viii

Definitions ................................................................................................................................................... ix

Executive Summary.................................................................................................................................... 1

Section 1: Introduction ....................................................................................................................... 3

1.1 Background ..........................................................................................................................5

1.2 Problem Statements .............................................................................................................5

1.3 Residential Retrofit Pilot Programs: Goals and Elements ...................................................6

1.3.1 Pilot Program Structures ..........................................................................................6

1.3.2 Pilot Program Goals .................................................................................................7

1.3.3 Pilot Program Elements ...........................................................................................8

1.4 Residential Retrofit Program Evolution...............................................................................8

1.4.1 Geographic Focus and Participants ..........................................................................9

1.4.2 Timeline of Events .................................................................................................10

Section 2: Cost Analysis .................................................................................................................. 10

2.1 Generating Cost Savings Through Economies of Scale ....................................................11

2.1.1 Administrative Program Design ............................................................................11

2.1.2 Bulk Purchasing .....................................................................................................11

2.1.3 Home Energy Assessments ....................................................................................12

2.1.4 Standardized Packages ...........................................................................................12

2.1.5 Marketing and Outreach ........................................................................................13

2.2 Risk Factors .......................................................................................................................13

Section 3: Mathematical and Modeling Methods ........................................................................... 14

3.1 Description of Pilot Delivery Models ................................................................................14

3.1.1 Performance Guarantees ........................................................................................18

3.1.2 Modeling Assumptions ..........................................................................................20

3.2 Modeling Methodology .....................................................................................................20

3.2.1 Incentive Equity .....................................................................................................20

3.2.2 Package Design Tool .............................................................................................21

3.2.3 Predicting Energy Use ...........................................................................................21

Section 4: Experimental Methods.................................................................................................... 21

4.1 Research Questions ............................................................................................................21

4.2 Technical Approach ...........................................................................................................23

4.2.1 Performance Guarantee Pilot .................................................................................23

4.3 Measurement Strategies .....................................................................................................24

4.4 Evaluation Methods ...........................................................................................................25

Section 5: Results ............................................................................................................................. 25

5.1 Overview ............................................................................................................................25

5.2 Data Analysis .....................................................................................................................26

5.2.1 Energy Savings ......................................................................................................26

5.2.2 Economies of Scale ................................................................................................26

5.2.3 Cost Savings...........................................................................................................27

5.2.4 Lead Source Results ...............................................................................................27

5.3 Differences in Results from Various Business Models .....................................................31

5.4 Homeowner Motivations ...................................................................................................32

5.5 Administrative Program Design ........................................................................................33
vi 

5.6 Draft Guide to Residential Retrofit Program Design .........................................................33

5.6.1 Identify Early Adopters, Not Neighborhoods ........................................................34

5.6.2 Use Consistent Messaging to Promote Consumer Awareness ..............................34

5.6.3 Limit Participating Contractors as Appropriate .....................................................34

5.6.4 Utilize Professional Sales Personnel ......................................................................34

5.6.5 Streamline Assessment and Quality Assurance Procedures ..................................34

Section 6: Conclusions ..................................................................................................................... 35

References................................................................................................................................................. 37

Appendix A ................................................................................................................................................ 38

Appendix B ................................................................................................................................................ 39
vii 

List of Figures

Figure 1. Timeline of pilot programs....................................................................................................... 10

Figure 2. Program structure for the Energy Challenge pilot (San Joaquin County, East Bay). ....... 15

Figure 3. Program structure for the Los Angeles County pilots.......................................................... 16

Figure 4. Program structure for Sonoma County pilot. ........................................................................ 17

Figure 5. San Joaquin Energy Challenge totals for leads, assessments, and upgrades.................. 28

Figure 6. Los Angeles marketing pilot totals for leads, assessments, and upgrades. ..................... 30

Figure 7. Los Angeles County contractor pilot totals for leads, assessments, and upgrades. ....... 30

Figure 8. Sonoma County pilot totals for leads, assessments, and upgrades................................... 31

Unless otherwise noted, all figures were created by the ARBI team.
viii

List of Tables

Table 1. Summary of Key Dates for LSRP Pilots. .................................................................................... 4

Table 2: Program Characteristics for Each Retrofit Pilot. ...................................................................... 8

Table 3: Sources of Leads........................................................................................................................ 13

Table 4: Research Questions and Extent of Confidence in Answers to Date. ................................... 22

Table 5: Measurement and Assessment Procedures for Developing Performance Guarantees ..... 24

Table 6: Lead Source for HEAs. .............................................................................................................. 28

Table 7: Lead Sources for HEUs. ............................................................................................................ 29

Table 8: Pilot Statistics as of September 1, 2012 .................................................................................. 31

Table 9: Consumer Motivations for Completing Home Energy Upgrades.......................................... 33

Table 10: Identified Risk Factors for Residential Retrofit Programs................................................... 38

Unless otherwise noted, all tables were created by the ARBI team.
ix

Definitions

ABAG Association of Bay Area Governments

ARBI Alliance for Residential Building Innovation (program team)

BA Building America

BPI Building Performance Institute

BBP Better Buildings Program

CEC California Energy Commission

CPUC California Public Utilities Commission

CRM Customer Relations Management

DEG Davis Energy Group (ARBI team lead)

DOE U.S. Department of Energy

EEM Energy Efficient Mortgage

EGIA Electric & Gas Industries Association

EIM Energy Improvement Mortgage

EUC Energy Upgrade California, a statewide program

FHFA Federal Housing Financing Agency

GHS Green Home Solutions

HEA Home Energy Assessment

HEU Home Energy Upgrade

HUD U.S. Department of Housing and Urban Development

IOU Investor Owned Utility

JTS Job Tracking System

LRF Loan Loss Reserve Fund

LSRP Large-Scale Residential Retrofit Pilot

MLS Multiple Listing Services

OBF On-Bill Financing

PACE Property Assessed Clean Energy Program

QC Quality Control

RECS Residential Energy Consumption Survey

RFQ Request for Qualifications

RLF Revolving Loan Fund

RMI Rocky Mountain Institute (an ARBI member)

STC Standing Technical Committee

WNA Whole Neighborhood Approach
1 

Executive Summary

Residential energy retrofit activities are a critical component of efforts to increase energy 

efficiency in the U.S. building stock. To date, however, consumer adoption of retrofits has been 

limited. Nationwide, retrofits account for a small percentage of aggregate energy savings at 

relatively high per unit costs. For example, the Better Buildings Program (BBP) Energy Upgrade 

California (EUC) program found that typical whole-house home energy upgrade (HEU) costs are 

$7,000 to $14,000 per house with projected energy savings of 18% to 30%. A few programs 

have implemented large numbers of retrofits through "giveaway" programs that target lowincome areas. While significant market penetration of deep residential retrofits is necessary to 

reduce nationwide energy use, programs that achieve this uptake remain elusive. 

A number of barriers must be overcome to achieve high retrofit volumes in the current market. 

These include first cost and financing challenges, the high cost of home energy assessments

(HEAs), a lack of experienced whole-house performance sales personnel, logistical challenges 

associated with bulk equipment purchasing, and a limited understanding of market dynamics and 

homeowner motivations. 

The U.S. Department of Energy Building America research team, Alliance for Residential 

Building Innovation (ARBI), is evaluating opportunities to improve and streamline delivery of 

residential retrofit measures. Through several large-scale residential retrofit program (LSRP) 

pilots, ARBI is: 

• Identifying relevant and replicable economies of scale for the HEU process

• Identifying less costly ways to perform HEAs that determine appropriate retrofit 

measures

• Developing standardized packages of energy efficiency measures that provide costeffective and reliable energy efficiency upgrades to consumers 

• Identifying information required by key stakeholders to make retrofit project 

implementation decisions 

• Evaluating the value of energy savings performance guarantees for homeowners 

• Researching and disseminating information related to business development such as sales 

and marketing 

• Evaluating retrofit policies, audit (HEA) procedures, tools, and protocols required for 

broad implementation of retrofit measures in existing homes. 

ARBI is testing the efficacy of each of these components by coordinating efforts on large-scale 

retrofit pilot programs throughout California in portions of Sonoma, Los Angeles, and San 

Joaquin Counties, CA. As of September 2012, three of the four programs are either closing or 

closed (Walnut/Diamond Bar and Palmdale in Los Angeles County, and Sonoma County), while 
2 

one program is ongoing (the Energy Challenge in San Joaquin County, the Tri-Valley and the 

East Bay). 

Early results in 2011 from the pilot in San Joaquin County indicated slow uptake of deep retrofits

in the two months following the May 2011 kickoff. In response, ARBI coordinated with its 

partners in the region to implement a series of changes in an effort to improve marketing efforts 

and increase uptake. These included expansion of the targeted area and a reduction in the number 

of steps homeowners were required to take to speak directly with the designated contractor. 

Similarly, the pilots in Sonoma and Los Angeles Counties, which formally began in August 

2011, had slow uptake in the first few months. ARBI worked with its partners in Sonoma and 

Los Angeles to increase uptake by applying and adapting lessons learned from the San Joaquin 

County pilot. The results to date from these changes emphasize how program structures must

respond to emerging market trends in order to promote successful programs. Program structures 

with fewer administrative levels proved more adept in adapting to market trends. 

To date, penetration rates for HEUs in the targeted communities are low but show some 

potential. The LSRP pilots highlighted the importance of several key factors in designing HEU

programs: 

1. Economies of scale in retrofit markets are possible. ARBI identified efficiencies related 

to marketing and outreach efforts, risk mitigation techniques, standardized package 

development, centralized bulk purchasing, and performance guarantees. 

2. Administrative infrastructure has significant influence on the ability of local contractors 

and administrators to adapt to emerging market trends. In general, wider involvement of 

organizations can divert limited resources away from sales-focused activities. 

3. Professional sales personnel are extremely important to a successful program design to 

increase homeowner motivation and uptake. 

4. Contractor expertise, as well as the way that contractors are presented to potential 

customers, can significantly affect program success. Limiting consumer choice to verified 

contractors can help consumers select a contractor. 

5. While it may be tempting to replicate a successful practice from one region in other 

regions immediately, care must be taken to examine the unique cultural, bureaucratic, 

environmental, and financial factors at play in each region. Thus, best practices for 

scalability and applicability will be critical in determining how to move successful 

residential retrofit tactics to other regions. 

6. Performance guarantees for upgrades combined with the ability to track energy usage 

hold promise for significantly increasing homeowner uptake of HEUs. 

7. Further technology development can play a key role in combating consumer perceptions. 

Of note, technologies that allow consumers to better understand their energy usage and 

expected savings can assist them in altering behavior and making wiser decisions 

regarding HEUs. 
3 

Davis Energy Group (DEG) has identified seven key components relevant to the design of future 

retrofit programs: 

1. Target broad communities rather than neighborhoods

2. Identify early adopters in a community 

3. Use contractors that are already embedded and respected in the community

4. Employ professional sales staff, who are involved from the beginning 

5. Utilize consistent, constant marketing and outreach throughout the program 

6. Simplify necessary paperwork and procedures for both the contractor and the homeowner 

7. Do not expect a short-term program to be a self-starter in a new industry. Retrofit 

programs are more effective as a catalyst for efforts that are already underway.

Section 1: Introduction 

Residential retrofit programs seek to increase market uptake of HEUs through a variety of 

incentive and education initiatives. While successful retrofit programs can result in energy and 

cost savings for consumers, the vast majority of residential energy retrofit programs suffer from 

low uptake. This low uptake can be attributed to a number of challenges, including: lack of 

homeowner knowledge about energy retrofit advantages and benefits; high upfront costs; and 

lack of marketplace value for energy efficiency measures in real estate prices. Sustainable and 

profitable business models for energy efficiency retrofits in the residential sector do not exist on 

a wide scale. Furthermore, challenges associated with obtaining energy use data, limited 

financing options, and a lack of experienced, professional retrofit sales personnel also impede 

uptake. Several organizations, including Lawrence Berkeley National Laboratory, have 

highlighted the challenges in getting homeowners to participate in energy efficiency retrofit 

programs (Fuller et al. 2010; Zimring et al. 2011). 

Building America (BA) Standing Technical Committee (STC) members identified a series of 

gaps and barriers to focus the research efforts of BA teams. For the past year, several needs 

related to energy efficiency retrofits were identified, including: 

• Better understanding of key motivations, drivers, and needs of various audiences in the 

energy efficiency value chain (Implementation STC) 

• Cost-effective and accurate “Drive-by-Audit” methods for existing homes to prioritize 

home attributes and identify when testing is required, without overly sacrificing energy 

model accuracy (Analysis Methods & Tools STC)

• Field data collection procedures and house simulation protocols that optimize cost and 

accuracy tradeoffs to increase the credibility and profitability of analysis efforts in the 

field (Analysis Methods & Tools STC)

Since 2011, ARBI has managed a series of LSRP pilots in California that seek to increase uptake 

of residential energy retrofits in California and address barriers identified by the BA STC 
4 

members. These pilots focus on targeted areas of Sonoma, Los Angeles, and San Joaquin 

counties in California.. The three pilots contain similar core elements, while employing tailored 

marketing and outreach strategies based on the specifics of the target regions. This has allowed

ARBI to compare and contrast various strategies. As of September 2012, the pilot in Sonoma 

County has ended, the pilots in Los Angeles County are nearing completion, and the Energy 

Challenge in San Joaquin County is ongoing and is expanding to the East Bay. Table 1

summarizes key dates for inception, revision, and closing for each of the pilots, as applicable. 

Table 1. Summary of Key Dates for LSRP Pilots. 

Program 

(Location) 

Program 

Inception 

Date

Program 

Revision 

Date(s)

Program 

End Date

Details

Energy Challenge

 (San Joaquin County; 

 Pleasanton, Dublin, 

 and Livermore; 

 Alameda and Santa 

 Clara County) May 2011

July 2011 &

May 2012,

Oct. 2012

Sept. 2013 

(expected)

First expansion occurred 

to encompass entire city 

of Stockton then to all of 

San Joaquin County. 

Next expansion to 

Pleasanton, Livermore 

and Dublin. (Tri-Valley)

Final expansion planned 

for Alameda and Santa 

Clara counties

Sonoma County

 (Rohnert Park, Santa

 Rosa, Coffey Park, 

 Piner)

Aug. 2011 Feb. 2012 March 2012

Program began with the 

Rohnert Park and Santa 

Rosa neighborhoods, 

expanded to Coffey Park 

and Piner neighborhoods.

Palmdale

 (LA County) July 2011 Nov. 2011 Sept. 2012 Expanded area includedall of West Palmdale

Walnut/Diamond Bar

 (Los Angeles County) Aug. 2011 n/a April 2012

Terminated with no 

program expansion 

attempts

The objective of the LSRP pilots is to increase energy efficiency in the residential sector through 

improved uptake of whole-house HEUs. More specifically, ARBI seeks to identify economies of 

scale in program design and implementation related to: 

1. Program administration costs

2. Marketing/outreach and sales

3. Risk mitigation techniques

4. Standardized packages of energy efficiency measures that are broadly applicable across 

neighborhoods, regions, and programs 

5. Centralized bulk purchasing available to participating contractors

6. Performance guarantees and energy modeling procedures. 

Identifying these economies of scale can improve the cost effectiveness of residential retrofits for 

both consumers and contractors. 
5 

1.1 Background

Increasing residential retrofit uptake is a complex problem that involves energy efficiency design 

and technology, markets, and consumer behavior. Successful strategies must incorporate 

participants across sectors that become linked through the market chain. Such participants

include consumers, industry contractors, energy efficiency professionals, utilities, researchers, 

and program administrators. Our experience and evaluations lead us to the conclusion that 

successful retrofit programs can take hold only through an integrated approach that draws on best 

practices from business, engineering, and marketing. 

Current residential retrofit programs have extremely low penetration rates, in part because 

homeowners are not well educated about the benefits of energy efficiency retrofit measures and 

retrofit costs are often high. Behavioral research indicates that people are not accustomed to 

making conscious decisions about energy use and are unaware of behaviors that may waste 

energy (Lutzenhiser 1993; Fuller et al. 2010). 

While middle-income homeowners represent the bulk of potential retrofit participants, they face 

a range of barriers, including: 1) first costs of retrofit; 2) cost recovery timeframe; 3) competing 

time and monetary constraints, and 4) limited financing options. Typical upgrade costs to 

improve energy consumption by 20% in a home range from $7,000-$14,000. The nationwide 

investment required to upgrade one-third of all middle class homes with energy efficiency 

retrofits ranges from $30-100 billion. In contrast, program funding for deep energy retrofits is 

estimated to be about $7.7 billion over the next decade (Zimring et al. 2011). 

Previous research and other retrofit programs have mirrored the difficulties in promoting deep 

retrofits in the residential market. BetterBuildings for Michigan has instituted a “neighborhood 

sweeps” program that challenges neighborhood organizers to sign up homeowners at several 

different levels of retrofit packages. From 2010-2011, the program showed notable results in 

signing up 22% of targeted homes for a base package (basic home weatherization), though only 

2% had signed up for upgrades beyond that package (DOE 2011d). In many cases, significant 

levels of incentives ranging from $1,500-$9,000, depending upon level of retrofit savings 

achieved and utility/jurisdiction offerings, are required to incentivize homeowners to undertake 

energy upgrades (DOE 2012a; DOE 2011a). 

1.2 Problem Statements

ARBI has identified several problems related to enacting residential energy efficiency retrofit 

programs, including: 

1. Limited knowledge exists regarding effective and efficient marketing and outreach 

techniques 

2. Lack of successful bulk purchasing programs that provide consumers with cost savings 

3. The optimal level and structure of quality control (QC) activities is not clear and 

potentially varies by program and region 

4. Limited industry experience in deciding how and what kinds of data to collect to facilitate 

evaluation of retrofit efforts, program administration, and consumer satisfaction
6 

5. Limited understanding of the role of program design in promoting uptake (i.e. number of 

contractors to include in a program). 

Through the LSRP pilots, ARBI has been researching these questions to provide empirical 

evidence that can improve program design and management. 

1.3 Residential Retrofit Pilot Programs: Goals and Elements

The LSRP pilots were designed to examine the research questions across varying factors of 

climate, program structure, scale, and contractor involvement. At the core of each pilot is a 

structure that brings together ARBI participants with state and local agencies, as well as regional 

contractors and sub-contractors. In all pilots, ARBI implemented regional- or neighborhood-level 

marketing and program design approaches in order to achieve economies of scale and maximize

homeowner participation. 

While the LSRP pilots were designed to address key questions and knowledge gaps, ARBI is 

keenly aware of the need for program evaluation activities that inform management and 

approaches. The team has been continuously refining program design and marketing and 

outreach strategies, including: 

• Bulk purchasing logistics and product sourcing 

• Program monitoring and data tracking systems

• Marketing and outreach strategies based upon lessons learned 

• Inclusion of professional sales people to those pilots that started with only contractor 

sales staff

• Addition of a performance guarantee pilot to the Energy Challenge. 

Figure 1 and Section 1.4 provide details regarding the evolution of each program from inception 

to present.

1.3.1 Pilot Program Structures

Two types of retrofit program pilots have been studied: Contractor Pilots and Marketing Pilots. 

Contractor Pilots target a neighborhood of similar homes for a limited time period. One or two 

contractors are selected and utilized to conduct energy assessments and complete energy upgrade 

installations. The contractor(s) are selected through an RFQ process and selected contractors are 

the primary sales force. The contractors receive support for marketing, bulk purchasing, and 

logistics. Contractor Pilot program structures were established for the Energy Challenge in San 

Joaquin County, the Palmdale pilot in Los Angeles County, and the pilot in Sonoma County. 

Marketing Pilots target one subdivision of similar homes for a limited time period including 

participation by multiple authorized installation contractors that have agreed to participate. 

Contractors act as their own sales force and handle all logistics. Participation in bulk purchasing 

is optional. The Marketing Pilot structure was established in Walnut and Diamond Bar in Los 

Angeles County. 
7 

1.3.2 Pilot Program Goals

Each of the pilots had initial quantitative and qualitative goals based upon budgets, length of 

time in the field, and the objectives of participants in each implementation structure. The overall 

goals are summarized below.

Quantitative Goals

Each LSRP had initial goals for HEUs based on early conceptions of achievability. These initial 

goals proved to be unrealistically high, as was also the case for the overall EUC Program. At the 

outset, the nascent states of the industry and consumer understanding regarding energy efficiency 

upgrades was not understood, as recently described in The San Francisco Chronicle (Baker 

2012); see Appendix B. In that article, the EUC pilots have spent over $97 million to date to 

induce homeowners to upgrade 5,130 homes, as opposed to a goal in excess of 10 times that 

number of homes (costs of typical upgrades range from $7,000 to $14,000). The early goals of 

the LSRP pilots were re-aligned to represent market realities in the process of program 

evaluation. 

In addition to these quantitative goals for HEUs, another goal for all of the pilots was to achieve 

cost savings through economies of scale. This goal examines cost savings for consumers through 

standardized upgrade packages and improved delivery mechanisms. 

Qualitative Goals

The pilots sought to characterize consumer motivations for undertaking retrofit activities. More 

specifically, the pilots sought to identify and document the range of viable motivations that 

homeowners communicated as key for choosing to pursue an upgrade. 

The pilots also sought to determine the extent of applicability for standardized measure packages 

and options among a variety of typical housing types. For standardized packages to be successful 

in the marketplace, they must be reasonably replicable across neighborhoods and regions. 

Moreover, assessment costs must be streamlined. In many cases, the costs of completing HEAs 

and modeling constitute 10% of the overall project costs. Identifying economies of scale in the 

determination of appropriate packages has the potential to streamline the assessment process and 

reduce time and costs. 

One unique goal of the LSRP pilots was to test the effects of professional sales personnel on 

market uptake of HEUs. Some results from retrofit programs indicate that offering sales training 

to contractors increased conversions from leads to upgrades (DOE 2011b). In the LSRP pilots, 

some pilots employed professional sales people as part of the organizational structure, while 

others utilized installers as the sales personnel. Moreover, in some pilots, sales personnel were 

added during the pilot term. For instance, in the Contractor Pilots, the contractor in the Energy 

Challenge and one of the two contractors in the Palmdale pilot had professional sales people on 

staff. The other contractor in the Palmdale pilot and the contractor in the Sonoma County pilot 

used the installation contractor as the sales staff. While all of the pilots had slow starts as 

homeowners started getting educated about the upgrade program, the contractors that had 

professional sales people on their staff started making headway and getting contracts signed 

whereas the contractors without professional sales people did not capture sales. The Sonoma 

County pilot program design was adjusted by adding a professional sales person to the 
8 

contractor’s staff. Within six weeks, the first sale was made, but the pilot was terminated a year 

early after being out in the field for only eight months. 

The contractors for the Marketing Pilot in Walnut and Diamond Bar (Los Angeles County) did 

not employ dedicated sales personnel. As the program structure was open where any approved 

contractor could sell upgrades, Los Angeles County chose to maintain this program structure 

throughout the course of the nine-month pilot. This provided an opportunity to test how an open 

market pilot performed against pilots with one or two pre-vetted contractors for a given area. 

Another qualitative goal for each of the pilots was to determine what types of marketing and 

outreach strategies were most effective in engaging homeowners for initial consultations through 

available communication methods. The pilots allowed homeowners to inquire about retrofit 

options through call center numbers, websites, or direct contact with participating contractors. 

Data tracking and consumer surveys sought to understand the extent to which different 

marketing strategies attracted customers through different contact mediums. In addition, data 

tracked the retention from initial contacts (leads) to assessments, and finally to upgrades. 

1.3.3 Pilot Program Elements

All of the pilots began by utilizing targeted marketing approaches that identified potential highuptake residential neighborhoods based on infrastructure, climate, and strength of community. 

Next, neighborhoods were analyzed to determine those that could maximize economies of scale 

through continuity of construction and composition. The objective was to capture economies of 

scale by streamlining the HEA process and facilitating efficient HEU modeling, package design 

and bulk purchasing. Neighborhood infrastructure data was combined with GIS analysis 

techniques and in-person data collection to identify potentially high-impact neighborhoods 

within the pilot program communities. Subsequently, marketing techniques were tailored to the 

needs and characteristics of the communities, relying on judgments from sales and contracting 

personnel to determine the most effective tactics. Standard upgrade packages and options were 

developed based upon energy models built by ARBI in coordination with input from drawings, 

field assessments, and feedback from the qualified selected contractors. Table 2 outlines the 

main elements of each program.

Table 2. Program Characteristics for Each Retrofit Pilot. 

Program 

Characteristics

BBP Alameda 

County

San Joaquin 

County

Sonoma

County

Palmdale Walnut/

Diamond Bar

Preferred Contractors No 1 1 1 2 No

Bulk Purchasing No Yes Yes Yes Yes Optional

Standardized Packages No Yes Yes Yes Yes Yes

Professional Sales 

Personnel

No Planned Yes No Partial No

1.4 Residential Retrofit Program Evolution

All of the pilots have undergone changes to their program design including marketing and 

outreach approaches, participating contractors, geographic focus, and more. All pilots began by 

identifying specific neighborhoods that met key characteristics in order to focus marketing 

efforts. The composition of program structure, contractor participation, and inclusion of sales 

personnel varied. Additionally, the marketing and outreach strategies employed by each pilot 

varied widely. 
9 

1.4.1 Geographic Focus and Participants

The geographic focus for each of the pilots began with confined neighborhoods of similar houses 

ranging from 500 to 1,250 units. In almost every program, the initial geographic focus was 

expanded to include more neighborhoods. The Los Angeles County pilot in Palmdale, which 

initially began in a small neighborhood area, was expanded to include more of Palmdale. The 

Sonoma County pilots in Rohnert Park and Santa Rosa were expanded to include the Piner and 

Coffey Park neighborhoods, though this expansion occurred very late in the pilot evolution. Even 

more, the Energy Challenge in San Joaquin County initially focused on a neighborhood in 

Stockton (Lincoln Village West), but was expanded several times to include all of Stockton, then 

all of San Joaquin County, and then the Tri-Valley cities of Dublin, Pleasanton, and Livermore. 

These expansions were in response to low uptake at first, but then sought to promote potentially 

effective strategies through continued expansion. As of September 2012, the pilot was soliciting 

proposals to expand into the East Bay area, including Alameda and potentially Santa Clara

Counties. 

Several pilots also changed the composition of contractors throughout the LSRP evolution. For 

instance, the Energy Challenge in San Joaquin County started with one contractor, Green Home 

Solutions (GHS), which is local to the Stockton area. After two months of little uptake, the pilot 

was expanded to add another neighborhood across town but still with the same contractor. After 

the August 2011 expansion to the Tri-Valley area, however, it was determined that other

contractors local to that area were necessary in order to spur growth. Program managers released 

an RFQ in late summer of 2012 for a local contractor in the East Bay area to work with the 

Energy Challenge to help provide that local trust and feel that the homeowners are requesting. 

Thus, the evolution of contractors in the Energy Challenge has responded to market realities. 

In the Sonoma County pilots in Rohnert Park and Santa Rosa, uptake was also very slow several 

months after initiation. In February, 2012 the pilot was slightly expanded to two additional 

neighborhoods, Coffey Park and Piner. The contractor, Beanstalk Energy, was not local to the 

neighborhoods, but a professional sales person who knew the neighborhoods was added to the 

team to work with Beanstalk and generate leads. Though the pilot began generating leads and 

signed one upgrade contract, it was terminated in March 2012. 

The Los Angeles County pilot in Palmdale went through an expansion into all of West Palmdale

in early 2012, maintaining the same contractors in the expanded neighborhoods. One of the two 

contractors, Alpine Green Services, opened an office in Palmdale and started attending local 

events. This resulted in community recognition as a local contractor, which served to build trust 

and generate retrofit activity. In contrast, the other contractor, REAS, did not open an office in 

Palmdale and did not attend local events. Thus, although no programmatic changes were made to 

the pilot, the composition of contractors evolved. 

Finally, the Marketing Pilot in Walnut and Diamond Bar, which began in August 2011, did not 

go through an expansion or make any adjustments during its nine- month run. It was terminated 

by Los Angeles County in March 2011. 
10 

1.4.2 Timeline of Events

Figure 1: Timeline of pilot programs. 

Section 2: Cost Analysis

Costs for HEUs are a critical component of consumer and contractor engagement. Program 

elements and structures, along with the cost of materials and labor, contribute to the consumer 

costs for completing an upgrade. At the same time, risk factors can drive up the cost of the 

assessment and upgrade process. Economies of scale drive the reduction of costs for various 

programmatic elements, making them a key component for program goals. Meanwhile, program 

administrators, contractors, and consumers learn how to identify and manage various risk factors 

that can drive up costs. Effective program design incorporates existing knowledge to both reduce 

costs and manage risk.

LSRP Launch 

(May 2011)

Expanded LSRP 

to all Stockton 

(July 2011)

Sonoma & Los 

Angeles Pilots 

Launch 

(August 2011)

Expanded Palmdale 

Contractor Pilot to West 

Palmdale

(November 2011)

Expanded 

Sonoma with 

additional 

neighborhoods 

(February 2012)

Expanded LSRP 

to San Joaquin 

County, Sonoma 

Pilot Ended 

(March 2012)

Palmdale Pilot Ended 

(September 2012)

Expanded LSRP to East Bay 

(Alameda, Contra Costa and 

Santa Clara) (October 2012)
11 

2.1 Generating Cost Savings Through Economies of Scale

A primary goal of the LSRP pilots was to identify economies of scale and cost savings for the 

residential retrofit market. The program began with several strategies related to maximizing 

efficiency, including targeted neighborhoods, bulk purchasing, and others. As the pilots evolved, 

economies of scale were identified in some pre-selected areas such as bulk purchasing, but others 

became apparent. Such discoveries, including the need to streamline the HEA process, 

development of standardized retrofit packages, simplified incentives, minimal administration, 

consistent messaging, and centralized logistics have informed LSRP management and serve as 

the basis for continued research. 

2.1.1 Administrative Program Design

Each LSRP pilot was established with a unique administrative program design that met the needs 

of local, state, and federal participants. These designs include both streamlined and complex 

constructs, with associated pros and cons. Thus far, a significant factor in successful pilots has 

been program adaptability in responding to market trends. For instance, the programmatic 

changes in the Energy Challenge were possible due to a small and flexible management structure 

that involved the contractor, ARBI participants, and the California Energy Commission. 

In contrast, the Energy Upgrade California (EUC) pilots in Los Angeles and Sonoma County 

have larger administrative infrastructures. This increases resources needed to coordinate 

activities and decreases responsiveness to market trends. According to an article in the San 

Francisco Chronicle dated September 27, 2012 (See Appendix B), the EUC pilots have spent 

over $97 million to induce homeowners to upgrade 5,130 homes. Nevertheless, larger 

administrative structures can be responsive. For instance, the EUC LSRP pilot in Palmdale, 

where more programmatic changes have been possible, experienced strong growth compared to 

Sonoma County after it was able to expand to all of West Palmdale. ARBI attributes this to both 

an active project team and highly motivated contractor and city personnel. The program 

expansion was not as broad as originally requested, but it did provide more room for the 

contractors to operate. In Sonoma County, two months of the total eight month-long pilot was 

spent seeking approval to expand the program. An expansion covering a significant portion of 

Rohnert Park was proposed. In contrast, the addition of two relatively small targeted 

neighborhoods was approved. This contributed to the significant challenges faced by the 

program and was not an effective decision-making structure to promote success in a pilot with a 

restricted timeline. 

The involvement of local municipal support is complex, providing both opportunities and 

challenges. For governments with an empowered single point of contact for local involvement, 

coordination of marketing activities is simplified. In the case of the Los Angeles Contractor Pilot 

in Palmdale, the single layer of municipal involvement by the City of Palmdale sped the 

decision-making process, allowing the contractor to respond to market trends more efficiently. In 

other cases, however, a larger web of local involvement can slow down decision-making 

processes. In the Sonoma County pilot, this larger web led to slow decision-making processes for 

many decisions.

2.1.2 Bulk Purchasing

Bulk purchasing programs that centralize the purchase and distribution of retrofit materials can 

save consumers money and contractors time. The Contractor and Marketing Pilots each had a 
12 

different delivery model approach to bulk purchasing. The Contractor Pilots required their 

contractors to purchase the materials and equipment needed for the upgrades through the bulk 

purchasing program and encouraged contractors to pass on savings to homeowners. The 

Marketing Pilot delivery model, however, offered the bulk purchasing program to the 

participating contractors, but did not require them to buy their materials and equipment from the 

purchasing program. If they chose to participate in the purchasing program, they were required to 

pass the savings on to the homeowners in the pilot. For contractors who participated in the bulk 

purchasing programs in both types of pilots, the centralized purchasing and logistics services 

saved time and resources, freeing contractor time to perform other activities and reducing overall 

overhead expenses for jobs. 

2.1.3 Home Energy Assessments

The HEAs, including an initial walkthrough, blower door and combustion safety testing, 

modeling, and homeowner consultation stages, comprise a significant percentage (as much as 

10%) of total costs for a given upgrade project. Initially, it was believed that standardized 

packages were appropriate for given neighborhoods with similar age and house structures. As the 

program evolved, analysis revealed that often, a similar set of energy efficiency measures were 

recommended for houses spanning different climates zones and styles. Thus, significant time and 

resources were being spent in the assessment process even when the outcome was often similar. 

It may be possible to further streamline this process by categorizing houses by style, vintage, 

location and a limited number of other parameters, and developing standardized packages by 

category. This could cut the cost of the HEA by more than half for most houses, potentially 

saving billions of dollars when applied to the U.S. housing stock. 

2.1.4 Standardized Packages 

Each of the pilots utilized a standard upgrade package approach in conjunction with bulk 

purchasing of the materials and equipment items in the standard package. As the pilots have 

evolved, it has become apparent that standard packages are more widely applicable than 

originally believed. While early pilot procedures sought to identify a specific package that was 

tailored to the home assessment, broader analysis across regions revealed that similar home 

styles across neighborhoods, climatic zones and age groups result in similar sets of prescribed 

packages. This insight has potential to reduce HEU costs significantly. 

ARBI has begun characterizing packages in several levels. Requisite measures are applicable to 

all houses (unless previously treated) in a category. As an example, requisites for a single-story, 

slab-on-grade house with an attic include duct sealing, air sealing, attic insulation, and lighting 

upgrades. Next, options are considered based on assessment of a quick walk-through, including 

measures such as installation of a water heater blanket. Finally, larger, more expensive 

equipment is discussed with the homeowner, including water heater and HVAC replacement. As 

data continue to accrue, ARBI is refining these assessments and hopes to continue the important 

research of characterizing widely applicable standardized packages in order to reduce assessment 

costs. 

While bulk purchasing has the potential to lead to savings, analysis of the upgrade contracts 

signed by the contractors in the Energy Challenge and Palmdale pilots was inconclusive, 

indicating modest savings of 5% to 7% due to bulk purchasing. This was primarily due to the 

lack of uptake by homeowners. Analysis of the retail costs of the upgrades, the wholesale costs 
13 

of the equipment and materials, the labor rates, and the profit margin percentage, indicates that if 

volume is achieved, economies of scale would be realized and cost savings could be passed onto 

the homeowners. 

2.1.5 Marketing and Outreach

Consistent and constant marketing are necessary in order to identify effective messaging, 

collateral designs, and outreach strategies. In the LSRP pilots, a variety of marketing and 

outreach approaches are being tested, allowing ARBI to compare and contrast program successes 

and draw inferences. ARBI tracked the results from various outreach techniques, noting potential 

influencing factors through quantitative and qualitative analysis. Various outreach techniques, 

including print, broadcasting, and electronic media advertising, neighborhood-based events, and 

canvassing, are being systematically tested to understand how each contributes to generation of 

sales leads. The results being tracked include the number of leads generated, dates of leads, 

method of engagement, and more. 

Table 3 below shows the number of leads, assessments and upgrades associated with various 

outreach and marketing methods. Lead sources in the Energy Challenge focused on the 

professional sales people and their referral networks through banks and real estate agents 

whereas, in the other pilots, the majority of the leads centered on a canvassing effort that took 

place in those pilots. The Contractor Pilot in Palmdale also had a high number of leads come 

from events the local contractor attended on a regular basis.

Table 3. Sources of Leads.

Lead Source Energy 

Challenge

Sonoma 

Contractor Pilot

Palmdale 

Contractor Pilot

Walnut/Diamond 

Bar Marketing Pilot

Bank/Mortgage 103

Call Center 7 1 1

Canvassing 25 22 26 23

Contractor 27 1 11 3

Direct Mail 27 8 19 3

Event May 1 (EC only) 9

Existing Customer 2

Lead Purchase 5

Other 4 2 7 1

Print Media 66 1 8 2

Real Estate Agent 177

Radio 12 3

Store Front 14

Website 30 1 12 2

Word of Mouth 74 1 2

Workshop/Event 3 52

Totals 585 37 141 34

2.2 Risk Factors

For each of the three pilot programs, ARBI monitored the effects of a variety of programmatic, 

technical, and perceptual risk factors and documented how they were addressed. Some risk 

factors identified prior to pilot launches never materialized, while other unexpected risks had 
14 

significant impacts on pilot outcomes. The specific risks that arose were analyzed for effects on 

pilot outcomes and are listed below. The Results (Section 5) discuss the findings of the pilots 

with respect to many of these risk factors. Appendix A lists the risk factors originally identified 

in ARBI’s Measure Guideline document entitled, Mitigation of Technical Risk Factors 

document, including: 

Programmatic Risks

• Low uptake, where economies of scale are not achieved

• Financing options are either not available, or are not attractive to homeowners

• Logistics for procuring, delivering, and storing bulk equipment purchases are challenging 

• Business model is not attractive to contractors. 

Technical Risks

• Installations underperform homeowners' expectations of energy savings 

• Installations underperform relative to predicted energy savings. 

Perceptual Risks

• Consumer lack of awareness of energy efficiency issues is a significant barrier to market 

uptake. The need for PSA/consumer education is currently unfulfilled by the various 

programs. This need should be fulfilled by federal or state government agencies. 

• Consumer perceptions of expected savings negatively affect their willingness to pay for 

HEUs. 

An examination of the risk factors reveals the interrelatedness of factors, even those in different 

categories. For example, perceptual risk related to consumer awareness of energy efficiency 

influences low uptake and failure to achieve economies of scale. If consumers lack an

understanding of energy efficiency issues and perceive that improvements in comfort, livability, 

and expected savings do not justify the upfront cost of energy efficiency measures, then they are 

unlikely to pursue retrofits. Without sufficient demand, the business is difficult to sustain. Thus, 

the analysis of risk factors is an important step to identifying the correlated technical and marketbased aspects of residential retrofit programs. 

Section 3: Mathematical and Modeling Methods

The LSRP pilots employ a number of unique programmatic and business models in order to test 

various opportunities for market efficiencies and best practices. 

3.1 Description of Pilot Delivery Models

The pilots all maintain a similar base structure that includes a client, program administration, and 

sub-contractors. Where the pilots differ, however, involves the number of intermediate layers of 

program administration, which affects the efficiency of decision-making processes in an

evolving marketplace. Specifically, additional layers mean high-level decision makers are further 

separated from ground-level market realities. This creates additional documentation and 

procedural requirements to effectively prove that new pilot approaches are necessary. 
15 

In the Energy Challenge pilot (San Joaquin County, East Bay); the delivery model is streamlined, 

as shown in Figure 2. The Energy Challenge pilot is funded with state funds from the Public 

Interest Energy Research (PIER) Program of the California Energy Commission. The Energy 

Challenge’s administrative delivery design allows for quick decision making on program 

adjustments, design, marketing materials, etc. This structure has proven flexible and nimble, 

giving the program the opportunity to make changes to its design and marketing approaches as 

dictated by feedback from the field within two weeks or less. 

The Los Angeles County pilot utilized a slightly more complex administrative structure, as 

shown in Figure 3. This pilot was funded with stimulus funds provided by the DOE through the 

CEC. Additional administrative layers were present in decision-making processes. All decisions 

went through each layer before making and implementing final decisions. The average time for a 

decision in this structure was six weeks to two months. While some mid-program changes were 

able to be made, it took significantly longer to effectively communicate ground-level data to 

upper levels of management. 

Figure 2. Program structure for the Energy Challenge pilot (San Joaquin County, East 

Bay).

Davis Energy Group

California Energy Commission

Polaris Contractor
16 

Figure 3. Program structure for the Los Angeles County pilots. 

The Sonoma County pilot administrative design had even more layers than the pilots in Los 

Angeles County creating a log jam for decisions on program design changes. The average 

decision time for this pilot was usually around two months. 

U.S. Dept. of Energy

California Energy Commission

Los Angeles County

Bevilaqua Knight, Inc.

Davis Energy 

Group

Contractors

Energy 

Coalition
17 

Figure 4. Program structure for Sonoma County pilot. 

The program administrative structures in Los Angeles and Sonoma Counties inhibited nimble, 

quick decision making that is necessary, especially in time-limited pilot programs for a nascent 

industry. 

Delivery methods for marketing and sales also differed among the pilots. Contractors in the 

Contractor Pilots in Sonoma County and Palmdale were selected based on a Request for 

Qualifications (RFQ) process. Contractors had to meet minimum criteria, including BPI 

certification and evidence of programmatic capabilities. During the selection processes in both 

counties, multiple contractors responded to the RFQs. In Los Angeles County, four contractors 

responded to the RFQ for the Contractor Pilot in Palmdale. Two contractors were ultimately 

chosen, one with strong subcontractors and another with strong marketing and administration 

skills. For the Marketing Pilot in Los Angeles County (Walnut and Diamond Bar), all EUCapproved contractors were invited to participate, resulting in six responses from willing 

contractors. The ARBI team ultimately identified four qualified contractors to participate in the 

Marketing Pilot. Unlike the Contractor Pilots, the Marketing Pilot remained open to additional 

qualified contractors. Finally, in Sonoma County, three contractors responded to the distributed 

RFQ, and one was chosen to participate in the pilot. 

U.S. Dept. of Energy

California Energy Commission

Los Angeles County

Association of Bay Area Governments

Sonoma County Transportation Authority/

Regional Climate Protection Agency

Bevilaqua Knight, Inc.

Community Solutions Group Davis Energy Group

Contractor
18 

In the Energy Challenge Contractor Pilot, GHS was selected as the initial partner in the pilot.

GHS was pre-vetted prior to preparation of the program proposal, offered considerable match 

funding (in excess of $250K), and was included as a subcontractor in the contract between DEG 

and the CEC. The contract also allowed for the addition of other contractors through the 

implementation of a RFQ selection process. As the pilot expanded to other counties it became 

apparent that local contractors were important in spurring leads and upgrades. Thus, the Energy 

Challenge effort in East Bay is currently going through the RFQ process to find one or two local 

qualified contractors that will perform marketing, assessments, and upgrades in the East Bay. 

Of all the contractors involved in the various pilots, only two have maintained professional sales 

people on staff. Notably, these two contractors have also signed up the most HEUs in their 

respective pilots: the Energy Challenge and the Palmdale Contractor Pilot. Professional 

salespeople understand how to work with homeowners, find key motivations, and allay financing 

and logistics worries. Professional sales personnel have proven quite beneficial in reducing 

associated risk factors and increasing the cost effectiveness of LSRP pilot expenditures. 

Contractors have differing approaches concerning marketing and outreach. The two contractors 

with professional sales people have handled many marketing and outreach activities internally, in 

addition to marketing efforts from the EUC or Energy Challenge. Both contractors promoted 

themselves rather than the programs, thereby building trust and community ties with 

homeowners. In contrast, other contractors in Sonoma County and the Marketing Pilot in Los 

Angeles County chose to do no additional marketing, instead relying on the program for all 

marketing and outreach efforts. 

From the perspective of the homeowner, the varying approaches to contractor selection and 

participation have a key distinction. In the Contractor Pilots, customers have been provided with 

one, or possibly two, pre-vetted contractors to perform the work, while in the Marketing Pilot, 

the homeowners were provided with a longer list of program-qualified contractors to choose 

from. This complicated the homeowner decision-making process, and may partly explain the 

total lack of uptake in the Marketing Pilot.

3.1.1 Performance Guarantees

Performance guarantees are typically agreements between contractors and customers that 

guarantee a minimum level of performance for a purchased item or service. For energy 

efficiency retrofits, this means that homeowners who purchase an HEU would be assured of a 

minimum level of heating and cooling energy savings. This may be an important component in 

addressing risk factors associated with consumer awareness and acceptance of the retrofit 

industry. Performance guarantees provide security for consumers and potentially a competitive 

advantage for contractors. While other related industries such as photovoltaic power installers 

have successfully implemented performance guarantees, the residential energy efficiency sector 

has not widely explored this approach. The guarantee as part of a well-priced and well-marketed 

package promises to be a useful tool to combat lack of education and consumer complacency in 

the energy efficiency marketplace.

As of this writing, the LSRP pilot is initiating a Performance Guarantee pilot to test this 

approach. The delivery model for performance guarantees will be slightly different from that of 

other industries. ARBI reviewed performance guarantees and quality control procedures from 
19 

various industries to determine cost effectiveness, performance, and ease of implementation. For 

example, in the residential solar industry, performance is guaranteed in terms of kWh produced 

per year. Possibilities for the whole-house performance industry include guaranteed energy 

savings as compared to pre-retrofit energy usage, or a guaranteed energy budget; i.e. energy 

usage will not exceed a specified amount of therms and kWh per year. In addition, the guarantee 

can be applied to all energy use, or limited to cooling and heating energy use only. 

These guarantees require QC procedures to help ensure proper installation of materials and 

equipment. Each of the models utilizes a somewhat punitive approach, whereby payments are 

only made if goals are not met. In contrast, ARBI assessed that an incentive-based approach was 

more appropriate in the residential energy upgrade market. The program is structured to provide 

homeowners with an additional monetary incentive of $400 to meet a total energy budget, 

including therms and kWh. If the house uses the budgeted therms and kWh or less for a one-year 

period post-retrofit, the incentive is paid. Thus, the incentive seeks to induce desired behavior. In 

many energy retrofits, consumers respond to the energy savings by changing habits such as 

thermostat set points to increase their personal comfort. While a valid outcome, this action fails 

to produce predicted energy savings. This issue is of significant concern for the retrofit industry 

and must be addressed if performance guarantees are to become viable. The ARBI Performance 

Guarantee pilot aims to reduce the likelihood of changes in homeowner behavior by 

incentivizing energy savings after the retrofit is completed. This performance guarantee model 

includes all kWh and therms used in the house, including miscellaneous loads. It incentivizes

energy savings behaviors. 

In the case that the homeowner does not meet the first year post-retrofit energy target, ARBI will 

first check to see if any new load-intensive appliances were bought by the homeowner after the 

earlier energy modeling. If the model contains everything in the house, then the set points in the 

thermostat will be checked. If these are within the prescribed range, then the HVAC system will 

be checked along with other systems in the house to determine the possible source of the increase 

in usage. If all of these prerequisites are in order, the homeowner will still receive the $400 

incentive even though the house did not perform as advertised. When a contractor provides such

assurances, risk transfers from the consumer to the service provider. Subsequently, sufficient 

benefits in the form of increased sales must exist for the contractor to offer performance 

guarantees and assume this risk. 

Finally, for the residential energy efficiency industry to utilize performance guarantees, several 

other requirements must be met. Energy models must accurately predict energy savings, enabling 

consumers to understand potential benefits of upgrades. This is both a technical and 

programmatic challenge. Market-ready technologies such as “smart” thermostats are capable of 

monitoring and storing set point data, which is needed to monitor homes’ compliance with 

performance guarantee pilot rules. More and better data from a growing set of residential retrofit 

programs will feed more accurate modeling efforts. At the same time, experienced energy 

efficiency professionals can accurately provide performance guarantees based on existing models 

(Chitwood 2011). Through the application of consistent HEA procedures, enhanced energy 

modeling calibration procedures, as well as proper installations and quality control, the 

uncertainty in energy model results may be low enough so that simulated energy savings for a 

majority of HEUs may be sufficiently accurate to provide performance guarantees. In addition, 

programs must be developed that provide incentives for contractors to do quality work, 
20 

essentially giving them “buy-in” to the upgrade process. Only through a combination of these 

advances can performance guarantees be realized. 

3.1.2 Modeling Assumptions

For the Performance Guarantee pilot, ARBI has developed criteria for homes needed for the 

initial pilot, including 

1) Houses built between 1970-1995 

a. Preferably production homes so this process can be repeatable

b. Preferably a higher energy user

2) Data acquisition requirements 

a. Access to 12+ months pre-retrofit data

b. Access to 12+ months post-retrofit data matching the pre month for month 

3) Owner participation

a. Willingness to install a Wi-Fi thermostat that can be monitored

b. Monitors and maintains set points 

4) Minimum upgrade measures

a. Air sealing

b. Duct sealing

c. Attic Insulation

5) Pre- and post-retrofit analysis. 

The pre-retrofit energy usage is analyzed to confirm typical, seasonal energy use. Houses with 

unusual energy use (e.g. high shoulder use; high winter electricity use) do not qualify for the 

Performance Guarantee pilot.

3.2 Modeling Methodology

Building energy models can be used for a number of different purposes and models are best 

utilized based on their strengths. ARBI has drawn on existing industry knowledge and research 

to characterize the main uses of each of these models, as described below. In some instances, 

industry participants may utilize models inappropriately. Contractors and program managers 

should determine the purpose of a building energy model for a given instance and use this to 

guide the type of model and amount of resources spent developing it. 

3.2.1 Incentive Equity

Utility rebate programs seek assurances through reasonable and documented verification that the 

rebates they provide in exchange for home energy upgrades meet specified criteria. Performing a 

building energy model before an upgrade can provide utilities with this information. While not 

always indicative of actual home performance, such modeling approaches allow utilities to 

characterize relative rankings of various home energy upgrades, which is useful to differentiate 

between rebate levels. Thus, while pre-upgrade building models used for this purpose do not 

necessarily provide an accurate assessment of post-assessment performance, they do provide 

utilities with a verifiable methodology to assess the extent of upgrade features for a given 

project.
21 

3.2.2 Package Design Tool

Building models can be used to design an upgrade package for a given house. For this use, 

models must specify in greater detail the characteristics of the house and potential upgrade 

options. Contractors will use this model as a guide to present various options to homeowners, 

including estimates for the best package options. 

3.2.3 Predicting Energy Use

Building models can also be used to predict energy use following an upgrade that includes 

specific measures. In this procedure, homeowners are given an expected amount of energy 

savings so long as there are no significant changes in occupancy behavior. Calibrating a model 

for this use involves significantly more time and resources. Accuracy and consistency are key 

targets when using building energy models to predict savings. 

Section 4: Experimental Methods

4.1 Research Questions 

While a wide array of relevant research questions can be explored regarding market-based 

residential retrofit programs, the LSRP pilots were designed to test a limited set of questions 

related to applicability and scalability of business models for residential energy efficiency 

retrofits. The initial questions below were considered important to these pilots. As the pilots 

evolved, some research questions were answered, while new questions arose. ARBI has sought 

to promote flexibility in the program pilots in order to allow programs to address relevant new 

questions. 

Initial questions were focused on how to generate cost savings through economies of scale, the 

role of communities and early adopters in promoting retrofits, and effective marketing and 

outreach techniques. As the pilots developed, questions related to program design and 

implementation for large-scale retrofit programs arose, leading to their inclusion in the test plan 

for the pilots. The refined questions still included marketing and outreach, but also considered 

how professional sales people and bulk purchasing approaches can increase uptake and generate 

economies of scale. In addition, the role of performance guarantees in attracting customers was 

considered and incorporated into pilot research efforts. Finally, technical considerations 

regarding acquisition and analysis of energy use data were considered as part of analyzing 

performance. Research questions from the test plan and the extent to which they have been 

answered are detailed in Table 4. 
22 

Table 4. Research Questions and Extent of Confidence in Answers to Date.

Research Question(s) Answered? 

Marketing, Outreach and Sales

• What techniques, including newspaper inserts, canvassing, direct 

mail, radio and television advertisements, and word-of-mouth, 

are most cost effective? 

yes

• What information do consumers need in order to motivate them 

to undertake energy efficiency upgrades, and what methods are 

effective in getting consumers this information? 

preliminary

• Do consumers respond better to marketing focused on programs 

or contractors?

preliminary

• What is the role of consumer champions, i.e. consumers who 

have completed an HEU and are willing to provide testimonials 

and promote the approach to neighbors, in attracting customers? 

preliminary

• How do differences in business models, such as sales-oriented 

vs. installation-oriented contractors, influence program uptake 

results? Do different models show variance in the average 

penetration rate for various pilots? 

preliminary

Economies of Scale and Bulk Purchasing

• Can large-scale residential retrofit programs be developed that 

take advantage of economies of scale in order to reduce costs of 

marketing, modeling, package design, equipment purchasing, 

and installation?

preliminary

• Do savings generated through bulk purchasing get passed along 

to consumers? 

preliminary

• What program designs are cost-effective and efficient in product 

delivery?

yes

• How can the needs of contractors regarding purchasing and 

delivery of equipment be met more efficiently?

preliminary

Gathering and Evaluating Energy Use Data

• What are the most effective platforms for collecting and 

analyzing energy usage data? 

preliminary

Performance Guarantees and Energy Modeling

• Is the uncertainty of predicting energy savings with available 

modeling software within an acceptable range?

preliminary

• Do performance guarantees help attract and retain consumers? TBD
23 

4.2 Technical Approach

In order to address the research questions, ARBI collects and analyzes data, including 

quantitative and qualitative surveys of participants, energy use data, and statistics for marketing 

and outreach approaches. This information assists in evaluating the effectiveness of various 

strategies for program design. ARBI is assessing participation rates across pilots and 

neighborhoods to examine which marketing and outreach strategies are most effective in the 

context of other programmatic factors. 

As part of the technical assessment process for program pilot activities, ARBI has: 

• Reviewed significant differences in sales infrastructure between pilots and compared 

results (in terms of program uptake)

• Assessed whether advertising and marketing campaigns effectively generated traffic for

program websites and call centers

• Determined the percentage of sales leads that completed HEAs

• Determined the percentage of HEAs that led to HEUs (i.e. conversion rate)

• Identified techniques for increasing conversion rates

• Evaluated overall energy savings of retrofit efforts

• Evaluated efficiencies in logistics in bulk purchasing

• Assessed quality control techniques

• Evaluated various techniques for collecting energy usage data

• Evaluated how standardized packages are best developed

• Evaluated how widely individual standardized packages can be applied.

To obtain energy use data, ARBI is now working with both homeowners and utilities to get 

energy utility data for post-retrofit evaluation. Although this is a more laborious and timeconsuming process, experience has shown it to be more reliable than relying on third-party 

software platforms. ARBI currently has data for approximately 20 houses. There is a significant 

lag time between the completion of an upgrade and when sufficient energy use data exists to 

evaluate performance. Notably, this lag is about 10 months. 

4.2.1 Performance Guarantee Pilot

Measuring performance guarantee data related to energy use is a programmatic and technological 

challenge. These challenges include working with consumers and utilities to capture and assess 

energy use data that can help to establish guidelines for contractors seeking to offer performance 

guarantees. Table 5 outlines the general procedure for developing a performance guarantee. 
24 

Table 5. Measurement and Assessment Procedures for Developing Performance Guarantees. 

1. Collect pre-retrofit building data as part of the HEA (following procedures established 

by the Building Performance Institute) 

o Determine building properties including the envelope and equipment

o Identify occupant schedules, which includes characterizing appliance use and times 

of occupancy

o Survey homeowner about seasonal heating and comfort concerns

o Obtain utility data for energy consumption

2. Disaggregate utility data

3. Normalize disaggregated utility data to TMY3 weather data 

(TMY3 weather file)

4. Develop Baseline model using HEA data and TMY3 weather data

5. Calibrate Baseline model using utility data and HEA data

o Limit adjustment to BPI Standard 2400 Standardized Qualification of Whole-House 

Energy Savings Estimates

6. Develop Post-Retrofit model using TMY3 weather and proposed Energy Conservation 

Measures

o Calculate predicted annual energy savings

7. Implement and install Energy Conservation Measures

o Perform post-retrofit test-out

8. Tune model with test-out data

o Re-calculate predicted annual energy savings and uncertainty

9. After one year, determine if predicted savings were achieved

o Gather post-retrofit utility data and use steps 2 and 3 above to normalize utility data 

an calculate savings

As part of the LSRP pilots, ARBI is instituting a performance guarantee pilot effort that builds 

on the above procedures to assure proper assessment of energy savings potential. ARBI has 

surveyed and adapted performance guarantee models from other related industries such as solar 

power that approach leasing arrangements as unsecured loans sold as monthly payments. Energy 

efficiency retrofits may achieve the same effect by selling home performance as a periodic 

service with a performance guarantee. The methodology on how to structure this service may 

vary, however a service arrangement represents less uncertainty to the homeowner and a more 

effective sales mechanism to the contractor. The new pilot allows contractors to provide 

consumers with specific assurances of post retrofit energy savings and will be starting in the field 

in October 2012. 

4.3 Measurement Strategies

To capture relevant data, ARBI is utilizing a residential project tracking system developed for 

the EUC) Incentive program for Southern California Edison Co. ARBI led efforts to assess the 
25 

system, implement customization, beta test, and coordinate with participating contractors for 

usage. The system enables ARBI to keep track of the number of HEAs completed, the number of 

HEUs completed, materials and measures installed, costs associated with installation, elapsed 

time for completion of HEAs and HEUs, and other critical program data. 

Another key component of the measurement plan is capturing actual energy usage data for 

participating houses. Collecting billing data from utilities and homeowners is challenging. To 

overcome this significant obstacle, ARBI began working with MyEnergy to run its proprietary 

web portal to automatically download pre and post retrofit utility bill data. MyEnergy had 

agreements with a large number of participating utilities across the country, including all of the 

California electric and gas utilities servicing the areas of the LSRP pilots. Unfortunately, using 

the MyEnergy portal turned out to be labor intensive and results were mixed due to the 

complexity of linking utility accounts to MyEnergy, as well as utility website upgrades that 

resulted in inconsistencies and bugs with the MyEnergy data collection software. Other emerging 

platforms, such as the Green Button from Pacific Gas & Electric, provide consumers with similar 

capabilities to visualize and assess energy use. However, the data cannot be accessed by a third 

party. Instead, the homeowner must download it and transmit it to the program administrator. 

Few homeowners are both capable and willing to do this. Consequently, ARBI is now focusing 

its data gathering efforts on access from the utilities. Although arduous and time consuming, this 

route is proving to be the most reliable. ARBI continues to research these and other approaches 

and platforms are being tested to determine the best array of tools that provide homeowners an 

efficient and effective way to participate in energy monitoring. 

4.4 Evaluation Methods

ARBI utilized rigorous empirical assessment methodologies, including data collection and 

surveys, in order to evaluate the effectiveness of implemented programs. More specifically, 

ARBI evaluated program implementation timeframes (time between inquiries, assessments, and 

upgrades), relationships between marketing approaches and sales outcomes, conversion rates, 

energy savings, consumer motivations, and more. ARBI has also begun to evaluate pre- and 

post-retrofit energy data in order to determine performance. It has also assessed homeowner 

motivations through surveys to gauge market interest. Quantitative and qualitative empirical 

results are presented for these topics in the Results section. 

Section 5: Results 

5.1 Overview

The pilot programs are providing data on the effectiveness and shortcomings of current program 

designs, which ARBI is using to assess and, as necessary, refocus program activities in order to 

achieve a successful large-scale retrofit program. Results to date provide encouragement 

regarding the viability of energy efficiency retrofits for residential housing, while also 

highlighting early trends for efficient program design. As of September 2012, the Energy 

Challenge has been the most successful to date, with approximately 21% of generated leads 

resulting in HEUs. As all pilots ended early with the exception of the Energy Challenge, the 

majority of the data for the finished pilots will be complete while the Energy Challenge data will 

still need to be viewed dynamically. 
26 

5.2 Data Analysis

ARBI analyzed early energy usage data for pre- and post-retrofit consumption using several 

different sources, including: 1) utility data gathered from the My Energy and similar web portals; 

2) utility data sent by homeowners; and 3) billing data gathered by contractors. ARBI also 

gathered and analyzed marketing and installation costs by tracking marketing techniques and

leads generated, tracking of conversion rates and reviewing energy upgrade job contracts 

specifying retrofit measures, rebates and installed costs. Data was examined to identify trends in: 

lag time between assessments and signed upgrade contracts; conversion rates for leads to 

assessments and assessments to contracts; and average cost per job. 

Early analysis of pre- and post-retrofit energy usage for participating homeowners indicate that 

some homes performed better than the predicted energy savings, while others underperformed 

relative to predictive savings. ARBI is using further energy use data and consumer surveys to 

identify the reasons for this variance in performance. At the same time, it has informed 

programmatic decisions for development of a performance guarantee pilot by highlighting the 

heterogeneity in both consumer motivations and post-retrofit behavior, which is an important 

aspect of the market for program managers to recognize. 

5.2.1 Energy Savings

This analysis is in the early stages for all of the pilots as retrofits are just being completed and 

energy data is starting to come from the utilities and the homeowners. The intent is to gather data 

for fifty (50) retrofits, but data acquisition has proven challenging. 

ARBI has analyzed three houses to date that have enough pre- and post-retrofit data. The results 

indicate that two of the three houses are actually using more energy than prior to the retrofit. At 

first, this was an issue of concern regarding the validity of the predictive models and quality of 

work. But, as research continued, ARBI found that the motivations for these homeowners to 

undertake retrofits included comfort and indoor air quality issues rather than energy savings. 

Thus, the results become more understandable with post-retrofit evaluation activities. The third 

house is actually saving 26% of the energy it used before the upgrade. In this instance, the 

homeowner’s motivation was to save money on their energy bills. 

As more retrofits are completed and more energy data becomes available, ARBI will continue to 

analyze the data to evaluate performance and identify the relationships between homeowner 

motivations, predictive modeling and actual savings. 

5.2.2 Economies of Scale

The ARBI team began with the intention of generating economies of scale through 

neighborhood-scale targeting efforts and bulk purchasing. To date, however, ARBI has learned 

that economies of scale for residential retrofits do exist, but not always in the predicted ways. 

After several months in the field, the project teams for all the pilots learned that finding cohesive 

neighborhoods of similar house styles was not nearly as important as identifying early adopters. 

Such persons may have a wide variety of motivations, including ideological motivations, desire 

for cost savings, or a risk-averse personality that enjoys new market opportunities. Early 

adopters are critical in maximizing social networks and word-of-mouth marketing opportunities. 

Once early adopters are identified, marketing efforts can determine if those neighborhoods meet 

the Whole Neighborhood Approach requirements to achieve the economies of scale goals. All of 
27 

the pilots revised their marketing messaging to try to find those early adopters who would help 

proselytize their good experiences with the program and the contractor. 

In addition to identification of neighborhoods with early adopters, the scope of program efforts 

necessary to achieve economies of scale is larger than first considered. Rather than targeting 

specific neighborhoods, programs must target wider areas where qualified, local contractors are 

engaged with the community and willing to undertake significant grassroots marketing 

themselves. The cost and knowledge barriers to residential energy retrofits are significant enough 

that neighborhood-scale marketing efforts are too localized to generate sufficient leads, 

assessments, and upgrades. When program managers expand the target marketing area to the city 

or county, however, uptake increases dramatically. Thus, economies of scale for marketing of 

retrofit programs do exist, just at a wider scale for this nascent market. 

Standardized packages offer a significant opportunity to capitalize on economies of scale in 

supplying the residential retrofit market. Initially, ARBI believed that standard packages would 

be applicable at a neighborhood scale, where houses were of similar age, size, builder, and 

layout. Through analysis, ARBI determined that packages actually have wider applicability 

across distinct geographic and climatic zones. For example, ARBI team members discovered that 

house-specific modeling for several “L-style” ranch homes in different climate zones resulted in 

the same set of package options. Moreover, even when the retrofit programs used intensive 

marketing, the percentage of homeowners in a given neighborhood with interest and resources in 

pursuing retrofits is low. Thus, both technical and marketing results of program design indicate 

that a broader scale of retrofit program design is appropriate. 

5.2.3 Cost Savings

Analysis of contracts and other documentation from each upgrade in the Energy Challenge and 

Palmdale Contractor Pilots were inconclusive in determining if price savings in economies of 

scale were passed on to homeowners. In essence, discounts associated with bulk purchasing have 

not materialized due to limited demand. Suppliers are not sufficiently motivated to maintain 

discounts to ensure consistent savings that contractors can subsequently use to base prices. The 

contractors utilizing the bulk purchasing program in these pilots have committed to pass on any 

savings to the homeowners as long as their profit margin stays intact, but proven results are 

critical in convincing industry collaborators that such savings are consistent. However, 

participating contractors have attested to significant time savings by being able to purchase all 

(or nearly all) of the equipment and materials needed for a HEU from one source. In addition, 

having the bulk purchasing program arrange logistics, including product delivery when and 

where (contractor warehouse or job site) needed saves contractors additional time, creates 

efficiencies, and reduces contractor’s administrative costs.

5.2.4 Lead Source Results

Homeowners respond to various marketing messages depending upon the relevance of the 

message to their personal motivations. ARBI found that in all of the pilots, consistent program 

messaging served to educate homeowners about the existence of the program and associated 

incentives. The constant and clear messaging helped move early inquiries to the subsequent steps 

of assessments and upgrades. 
28 

Table 6 provides a breakdown of the source of leads for various programs. The highest 

conversion was in the Energy Challenge through bank referrals (73%). For the other pilots, the 

highest conversion rate was either through individual contractors (50%) or direct mail pieces 

(38%). 

Table 6. Lead Source for HEAs. 

Lead Source Energy 

Challenge 

Sonoma 

Contractor Pilot 

Palmdale 

Contractor Pilot 

Walnut/Diamond 

Bar Marketing Pilot 

Bank/Mortgage 73 

Call Center 

Canvassing 1 3 1 

Contractor 14 7 2 

Lead Source 

Energy 

Challenge 

Sonoma 

Contractor Pilot 

Palmdale 

Contractor Pilot 

Walnut/Diamond 

Bar Marketing Pilot 

Direct Mail 10 3 3 1 

Event May 1 (EC only) 3 

Existing Customer 

Lead Purchase 2 

Other 1 1 4 

Print Media 30 1 1 

Real Estate Agent 95 

Radio 3 

Store Front 4 

Website 7 1 1 1 

Word of Mouth 28 1 2 

Workshop/Event 1 6 

Totals 272 10 25 4 

Figure 5. San Joaquin Energy Challenge totals for leads, assessments, and upgrades. 

0

100

200

300

400

500

600

May-11 Jul-11 Oct-11 Jan-12 Apr-12 Jul-12 Oct-12

San Joaquin County Energy Challenge

Leads

HEA

HEU
29 

Conversion rates for leads to upgrades are significantly lower. This signifies that many 

homeowners do not continue with the HEU process once receiving an assessment. The highest 

conversion rate for the Energy Challenge was again in bank referrals (31%). The Palmdale pilot 

was tied with a 25% conversion rate from leads to upgrades for contractor referrals and “other” 

outreach strategies. The Sonoma County and Walnut/Diamond Bar pilots had very few upgrades, 

so these conversion rates provide limited value. 

Table 7. Lead Sources for HEUs. 

Lead Source Energy 

Challenge

Sonoma 

Contractor Pilot

Palmdale 

Contractor Pilot

Walnut/Diamond 

Bar Marketing Pilot

Bank/Mortgage 39

Call Center

Canvassing 1 1

Contractor 4 4

Direct Mail 6

Event May 1 (EC only) 1

Existing Customer

Lead Purchase 2

Other 1 2

Print Media 18

Real Estate Agent 33

Radio

Store Front 2

Website 3

Word of Mouth 12 1

Workshop/Event 1 2

Totals 123 1 9 0
30 

Figure 6. Los Angeles marketing pilot totals for leads, assessments, and upgrades. 

Figure 7. Los Angeles County contractor pilot totals for leads, assessments, and upgrades. 

0

5

10

15

20

25

30

35

40

Jun-11 Aug-11 Oct-11 Jan-12 Apr-12

Los Angeles County Marketing Pilot

Leads

HEA

HEU

0

10

20

30

40

50

60

70

80

90

Los Angeles County Contractor Pilot

Leads

HEA

HEU
31 

Figure 8. Sonoma County pilot totals for leads, assessments, and upgrades. 

To put these conversion rates into perspective, we looked at the whole EUC Los Angeles County 

program’s numbers. 

Table 8. Pilot Statistics as of September 1, 2012. 

Program

Total 

Leads HEAs HEUs % Conversion

# of 

Households

Penetration 

Rate

EUC LA 

County N/A N/A 1978 N/A 4,477,300 .04%

Palmdale 134 22 8 4% 8,000 .1%

Energy 

Challenge 581 292 91 16% 117,257 .07%

As shown in Table 8, the Energy Challenge leads in conversion rates. This is likely attributable 

to its streamlined program administration, local contractor and professional sales people. The 

Palmdale Contractor Pilot also has a higher penetration rate than the overall EUC Los Angeles 

County program. This pilot does have a local contractor with a professional sales person, but is 

still dealing with the complex program administration structure including a two-month long 

review process for cooperative advertising efforts. 

5.3 Differences in Results from Various Business Models

Business models play an important role in the success of residential retrofit programs. The LSRP 

pilots indicated that professional sales people are critical in promoting retrofit programs and 

increasing uptake. While all of the pilots had slow starts as homeowners became educated about 

upgrade programs, contractors that had professional sales people on their staff made headway at 

a faster rate and signed contracts for upgrades. In contrast, contractors without professional sales 

people did not capture sales. As a telling example, after the pilot team for the Rohnert Park and 

0

5

10

15

20

25

30

35

40

Jun-11 Aug-11 Oct-11 Jan-12 Apr-12 30-Jun-12

Sonoma County Pilot 

Leads

HEA

HEU
32 

Santa Rosa pilots refined the program design by adding a professional sales person, the first 

upgrade contract was signed within six weeks. 

Business models must also be receptive to cultural and economic factors. For instance, the Los 

Angeles County pilot in Walnut and Diamond Bar experienced programmatic issues related to 

cultural and economic barriers. Even more than typical neighborhoods, the Walnut and Diamond 

Bar areas were particularly sensitive to economic considerations. For most homeowners, English 

was a second language. Contractors that were not local and established in the community were 

not very successful. Thus, while efficiency upgrades are about technical competence, marketing 

is fundamentally about people. 

More broadly, local contractors are very important to program success. The building industry 

remains a local industry where consumers rely on word-of-mouth recommendations and trusted 

referrals. For the retrofit industry, ARBI found this to be true in all the pilots. Homeowner 

feedback surveys in Sonoma and Los Angeles counties indicated that local contractors were very 

important in motivating the consumer to pursue an upgrade, since they sought a reputable 

professional. Additionally, outside contractors that make an effort to become locally engaged 

also found success in penetrating markets. ARBI determined that contractors who made a 

commitment to a pilot area by establishing a regional office and attending local events closed 

nearly twice as many contracts as those who did not. 

Programs that are open to all potential contractors can be successful (DOE 2012b; DOE 2011c). 

While programs often utilize a number of contractors to provide consumers with choices, results 

from the LSRP pilots indicate that some segment of the consumer market would rather be given 

the name of one or two endorsed contractors with whom to follow up. Many consumers are busy 

with work, family, school, and other commitments, so interviewing multiple contractors is not 

feasible. It is likely that market opportunity exists for pilots that promote a select set of 

contractors, as well as open ones with many contractors.

5.4 Homeowner Motivations

Consumer motivations are a critical component to developing any industry, including residential 

retrofits. ARBI has focused on understanding the variety of homeowner motivations for 

inquiring and eventually undertaking HEUs. Through the pilots, ARBI identified key issues that 

persuaded homeowners to sign up for a retrofit. These issues included comfort, energy savings, 

and cost savings. Most importantly, program staff and sales personnel must quickly identify the 

key motivator for a consumer before attempting to convince them to undertake an energy 

upgrade. Energy upgrades compete with other possible expenditures in the marketplace and must 

be framed to be both cost effective and responsive to personal motivations. 

Early on, analysis from ARBI noted the importance of finding early adopters who can spur 

neighborhood-level growth through word-of-mouth and localized advertising techniques. 

According to survey results for the Energy Challenge pilot conducted by Polaris, the 

homeowners who undertook a HEU did so for a variety of reasons ranging from comfort issues 

to energy cost savings to age of equipment. First-time homebuyers were also convinced that the 

upgrade was a good thing to do while in the mortgage process. A high percentage of the 

homeowners, 40% of the respondents, cited multiple reasons for performing the upgrade on their 

home, as shown in Table 9. 
33 

Table 9. Consumer Motivations for Completing Home Energy Upgrades. 

Consumer Motivation Number of Respondents

Comfort 5

Cost of Utility Bills 8

Home Purchase 4

Environmental Consciousness 3

Program Cost 3

Need 6

Multiple Reasons 8

Totals 37

Survey results yielded several quotes that provided insights into these motivations: 

• “We were freezing to death in our house...we actually wanted the whole house to be the 

same temperature.”

• “Cost savings on PG&E. We just bought the house and when we had the assessment done 

we found out that certain ducts weren’t even connected. We were just heating underneath 

the house.” 

The complete survey results are not available yet from Cadmus who conducted all homeowner 

surveys for the Los Angeles County pilots. These results are anticipated for the final technical 

report.

5.5 Administrative Program Design

As discussed in Section 2.1.1, reduced program infrastructure, especially the number of levels in 

the governance structure, has shown to be an important factor in program management. The 

administration of retrofit programs must reflect the geographic and market realities that drive 

upgrades. Many of the initial program structures involved a collection of contractors, local 

governments, state government entities, and ARBI participants. The targeted neighborhood 

programs meant that involvement of local governments and community organizations was 

important. As the successful retrofit pilots expanded their geographic focus, however, the local 

government involvement became more complex. Expanding a retrofit program with additional 

government jurisdictions involved takes significantly more time and resources spent on 

coordination. In the instance of the Energy Challenge, the program began in a neighborhood, but 

has expanded to included counties in the Central Valley and East Bay. 

Thus, it is important that the program structure reflect consumer needs rather than jurisdictional 

constraints. Pilots with greater success were able to respond to market trends by expanding target 

areas. For a given program, the original collection of involved participants may not continue to 

be the optimal mix of participants as market factors change. The LSRP pilots have revealed the 

critical need for flexible program structures that can adapt to market realities. There are likely a 

number of potential program structures that can operate in a flexible manner, but in general, 

fewer participants can simplify coordination. 

5.6 Draft Guide to Residential Retrofit Program Design

The LSRP pilots provide a rich landscape for studying various strategies and best practices in 

retrofit programs. Through the evolution of the pilots, however, similar lessons were repeatedly 

revealed. Five lessons were particularly apparent as each pilot progressed. Each lesson is 

discussed here individually. 
34 

5.6.1 Identify Early Adopters, Not Neighborhoods

Finding early adopters is a key strategy for any retrofit program. Only after these “champions” 

are found can a program start looking at possible targeted neighborhoods to test whether there is 

any possibility of conducting a whole neighborhood program and gaining economies of scale. It 

makes more sense to find the people first as opposed to the buildings, since “people buy 

upgrades, not buildings.” Utilizing these early adopters as proselytizers to promote the upgrade 

experience at events, in media, and through word of mouth campaigns creates a more 

comfortable environment for others to participate. 

5.6.2 Use Consistent Messaging to Promote Consumer Awareness

A big lesson learned in all of the pilots was that once the initial marketing/advertising push was 

completed, the number of calls, website visits, and other inquiries diminished immediately. Once 

the pilots began a consistent messaging effort that included simple and constant messages in a 

variety of placements, homeowners gained familiarity with the program. According to feedback 

from the contractors, homeowners had more knowledge of the program after an extended 

messaging campaign. 

5.6.3 Limit Participating Contractors as Appropriate

Limiting the participating contractors to two pre-vetted program contractors for a given area 

allows the homeowner to still have a choice of contractors, but also gives them the assurance that 

these contractors will perform following the program guidelines. While a variety of open and 

selective contractor programs may be appropriate, a definite market exists for programs that 

utilize one or two verified contractors, which are promoted to all inquiring homeowners. At the 

same time, the variety of consumers in the marketplace means that programs with many 

contractors are also viable in some instances. For a nascent retrofit market, however, consumers 

can become quickly overwhelmed with a large number of contractors to choose from, as was the 

case on the EUC website. Many consumers did not continue in the program due to this 

confusion. ARBI found that it is advantageous to provide consumers with a trusted local 

contractor who can guide consumers through the multi-stage process of an energy efficiency 

upgrade. 

5.6.4 Utilize Professional Sales Personnel

Contractors with professional sales people outperform contractors without professional sales 

people by 100% for signed contracts. The Sonoma pilot showed this point clearly. Once a 

professional sales person was hired, a contract was signed within six weeks of starting, with 

others in the pipeline. This was after many months of stagnant program activity. Unfortunately, 

the pilot was canceled the day the first contract was signed due to lack of numbers. The other 

pilots that employed professional sales people (in Palmdale and San Joaquin County) also 

performed relatively well.

5.6.5 Streamline Assessment and Quality Assurance Procedures

According to the homeowner survey results, many homeowners were concerned about the 

amount of time they spent at home for the assessment and quality assurance visits by the utility. 

This is evidenced by the quotes below from the Energy Challenge survey: 

“The only thing that we find that we don’t understand is maybe in the beginning they 

should tell you who all is going to want to be coming out to inspect your house after 
35 

the fact. Well, we were going to get it inspected by PG&E you know, but now there 

seems to be some kind of third party contractor. I don’t know who the heck he is.” 

“Well, as I say the whole paperwork approval thing and having to get approval from the 

state and I don't know how many agencies were involved in this besides when I 

contacted so that was time consuming.” 

When designing a retrofit program, simplicity is critical. Homeowners are busy and desire an 

easy process for achieving their goal. In addition, the process should be streamlined for 

contractors, who are just as busy. At the beginning of these pilots, all of which were associated 

with a utility program, the contractors were excited about participating. As the pilots continued 

on, contractors became increasingly frustrated with the utilities requirements for modeling, 

paperwork and quality assurance. Some contractors offered to pay the homeowners the utility 

incentives themselves rather than take them through the program because pursuing the incentive 

program would often cost the contractor more than the incentives. Simplicity is critical. 

Another lesson learned is that building a bulk purchasing program is time consuming. Sourcing 

new products, building relationships with manufacturers and distributors, negotiating pricing, 

and handling the delivery logistics for each project requires constant attention. Thus, companies 

that wish to establish such programs must assign dedicated, experienced personnel to run such 

programs. 

Section 6: Conclusions

The success of LSRP is dependent upon the talent and energy of the participants (marketers, 

auditors, salespeople, contractors and administrators), as well as the ability of the infrastructure 

to respond to market requirements. To that end, several conclusions are critical for success. First, 

economies of scale in retrofit markets are possible. ARBI identified efficiencies related to 

marketing and outreach efforts, risk mitigation techniques, completion of HEAs, standardized 

package development, centralized bulk purchasing, and performance guarantees. Second, 

administrative infrastructure has significant influence on the ability of local contractors and 

administrators to adapt to emerging market trends. In general, wider involvement of 

organizations can divert limited resources away from sales-focused activities. Third, contractor 

expertise, as well as the way that contractors are presented to potential customers, can 

significantly affect program success. Limiting consumer choice to verified contractors can 

actually assist consumer decision-making. Fourth, while it may be tempting to replicate a 

successful practice from one region in other regions immediately, care must be taken to examine 

the unique cultural, bureaucratic, environmental, and financial factors at play in each region. 

Thus, best practices for scalability and applicability will be critical in determining how to move 

successful residential retrofit tactics to other regions. Finally, further technology development 

can play a key role in combating consumer perceptions. Of note, technologies that allow 

consumers to better understand their energy usage and expected savings can assist them in 

altering behavior and making wiser decisions regarding HEUs. 

Further research is recommended in a number of areas. This high cost of completing HEAs and 

modeling individual houses, combined with the consistent HEA results attained for broad 
36 

categories of houses, leads to the conclusion that many houses can be categorized and energy 

efficiency measures pre-selected. Parameters for categorizing houses would include age, 

location, number of stories, foundation type and a small number of other easily determined datapoints. Recommended measures, based on building science and modeling of other houses in the 

same category, could likely lead to a pre-selected set of required measures (such as air sealing), 

plus options based upon the age of existing equipment. The result would be a tool that radically 

reduces, or eliminates, the time needed to complete an HEA.
37 

References 

Baker, David R. (2012). “Energy Upgrade Falls Far Short of Goal - SFGate.” SFGate.com, 

September 22. http://www.sfgate.com/news/article/Energy-Upgrade-falls-far-short-of-goal3886924.php#page-2. 

Chitwood, Rick. (2011). “Performance Guarantees... or Getting Paid to Do Good Work & 

Delivering Real Performance” presented at the Energy Savings You Can Bank On: Residential 

Energy Efficiency Financing Mechanism, October 12, Oakland, CA. 

DOE.( 2011a). Spotlight on Austin, Texas: Best Offer Ever Produces Upgrades in Record Time. 

Better Buildings Program. Washington, D.C.: U.S. Department of Energy. 

DOE. (2011b). Spotlight on Maine: Contractor Sales Training Boots Energy Upgrade 

Conversions. Better Buildings Program. Washington, D.C.: U.S. Department of Energy. 

DOE. (2011c). Spotlight on Austin, Texas: Let Your Contractor Be Your Guide for Big Rewards. 

Better Buildings Program. Washington, D.C.: U.S. Department of Energy. 

DOE. (2011d). Spotlight on Michigan: Sweeping the State for Ultimate Success. Better Buildings 

Program. Washington, D.C.: U.S. Department of Energy. 

DOE. (2012a). Spotlight on Maine: Transition to a Sustainable Level of Incentives. Better 

Buildings Program. Washington, D.C.: U.S. Department of Energy. 

DOE. (2012b). Spotlight on Portland, Oregon: Making the Program Work for Contractors. 

Better Buildings Program. Washington, D.C.: U.S. Department of Energy. 

Fuller, Merrian C.; Kunkel, Cathy; Zimring, Mark; Hoffman, Ian; Lindgren Soroye, Katie; and 

Goldman, Charles. (2010). Driving Demand for Home Energy Improvement: Motivating 

Residential Customer to Invest in Comprehensive Upgrades That Eliminate Energy Waste, Avoid 

High Bills, and Spur the Economy. Lawrence Berkeley National Laboratories.

Lutzenhiser, L. (1993). “Social and Behavioral Aspects of Energy Use.” Annual Review of 

Energy and the Environment 18 (1) (November): 247–289. 

doi:10.1146/annurev.eg.18.110193.001335. 

Zimring, Mark; Borgeson, M.G; Hoffman, Ian; Goldman, Charles; Stuart, E.; Todd, A.; and 

Billingsley, M. (2011). Delivering Energy Efficiency to Middle Income Single Family 

Households. Berkeley, CA: Lawrence Berkeley National Laboratories.
38 

Appendix A

ARBI has focused on promoting and achieving energy savings for residential retrofits as part of 

its strategies for successful technical and programmatic approaches for the residential energy 

efficiency sector. Prior to program implementation, ARBI identified a series of risk factors 

related to technical and economic components of the programs, which were seen as potential 

impediments to program success if not addressed through proper mitigation strategies. These 

identified risk factors are listed in Table 10 below. 

Table 10. Identified Risk Factors for Residential Retrofit Programs.

Programmatic Risks

a. Low uptake, resulting in failure to achieve economies of scale;

b. High uptake, resulting in long delays and lost business;

c. Resistance from homeowners to allow access to key areas of the home;

d. Unattractive business model for contractors;

e. Inability of contractors to keep up with workload requirements;

f. Failure by contractors to adhere to program guidelines. 

Technical Risks

a. Low-quality installations and lack of QC; 

b. High number of callbacks for service issues;

c. Post-installation technical problems such as moisture, mold, or combustion, resulting 

from tightening of a thermal envelope during a Home Energy Upgrade; 

d. Failure of upgraded homes to perform to expectations of homeowners;

e. Failure of upgraded homes to produce energy savings predicted by modeling. 

Controlling for these risk factors requires careful planning, coordination, and data collection 

activities on the part of the ARBI team and its partners working to implement retrofit programs. 

Using established databases and performance metrics, ARBI collaborates with selected 

contractors to gather data and assess progress towards mitigating these identified risk factors.
39 

Appendix B

Energy Upgrade falls far short of goal

David R. Baker

SFGate.com, Saturday, September 22, 2012 

Backed by $146 million from President Obama's stimulus, California last year launched an effort 

to help as many as 100,000 homeowners save energy by providing rebates for new insulation, 

windows and furnaces.

The stimulus money has been spent, but as of July, just 5,130 homes received upgrades or 

qualified for rebates, according to the California Energy Commission. The money also funded 

3,728 energy-efficiency projects at businesses through May - mostly improvements to ventilation 

systems and lighting controls. 

Why have so few benefited? In part, it's because the state and its partners spent more money 

launching and running the program than they did on rebates. 

According to figures from the Energy Commission, $40.9 million went directly into rebates for 

homeowners and businesses, while $56.5 million was spent implementing the program, 

marketing it to the public, training contractors and making sure projects saved as much energy 

as intended. 

In addition, homeowners just haven't been clamoring for the rebates. 

The state's tough economic climate undercut the program, called Energy Upgrade California. 

Cash-strapped Californians balked at investing thousands of their own dollars in home 

improvements, even if the rebates would later cover some of the cost. Banks were hesitant to 

lend money for the supplemental work. As a result, $48.5 million that could have paid for more 

rebates was used to provide loans for homeowners. 

Is it worth it?

Critics question whether it's worth the expense.

"Here's one thing that all people, of both parties, tend to agree on - they want good value for their 

tax dollar," said Kris Vosburgh, executive director of the Howard Jarvis Taxpayers Association. 

"And this is a glaring example of folks not getting good value for their dollar."

Although the stimulus money is gone, Energy Upgrade continues to offer rebates funded from 

other sources, including Californians' electric bills. 

The program is a wide-ranging collaboration of the Energy Commission, the state's utility 

companies and many city and county governments. The Energy Commission took the lead 
40 

coordinating Energy Upgrade during its first year. That role will now fall to the utility companies 

and local governments, with oversight from the California Public Utilities Commission. 

State officials say the program's initial, federally backed phase laid the groundwork for the 

future, even if it didn't issue nearly as many rebates as hoped. The Energy Commission made the 

100,000-home prediction when it announced the program's launch in March 2011. The 

prediction didn't include a specific time frame, and the commission now considers it an openended goal. 

"We're at the front end of this, and we're making an investment to get the ball rolling forward," 

said Andrew McAllister, a member of the Energy Commission. "I feel like we're not doing 

that badly." 

The program, he said, will help create a market for businesses that provide home energy retrofits;

much like the way California's rebates for solar power systems spurred the growth of the state's 

solar-installation industry. But he acknowledged Energy Upgrade's slow start.

Utilities chip in millions

"It turns out that it's a pretty difficult thing to do," McAllister said. "There's no getting 

around that." 

California's large utility companies, which have contributed $116 million of their ratepayers' 

money to Energy Upgrade, have asked the California Public Utilities Commission to approve 

funding for the program for the next two years. The Utilities Commission has signaled that it 

wants the program to continue, subject to periodic cost-effectiveness reviews. 

Local governments, meanwhile, have kicked in an additional $50 million, although much of that 

money came from the U.S. Department of Energy. The program's total funding to date has been 

roughly $312 million. But the stimulus portion of the funding, tracked by the Energy 

Commission, has been fully allocated. 

Energy efficiency has been an obsession of California government for decades. 

Starting in the 1970s, the state began imposing tough energy-efficiency standards for new 

buildings and home appliances. As a result, the amount of electricity Californians use per person 

has stayed relatively flat, despite the advent of home computers and big-screen TVs. From 1973 

through 2005, per capita electricity use rose just 14 percent in the state, while it jumped 60 

percent in the nation as a whole. 

Energy Upgrade California was designed to squeeze more energy savings from existing homes. 

The state offers rebates ranging from $1,500 to $4,000, depending on how much electricity or 

natural gas each home improvement project saves. To qualify, home retrofits must cut at least 10 

percent of the building's energy use, with the biggest rebates reserved for projects that cut 40 

percent or more. 
41 

City, county rebates

Many cities and counties offer additional rebates or incentives. For a time, San Francisco 

residents could get as much as $3,000 from the city, in addition to $4,000 from the state. (San 

Francisco now offers $1,000 per retrofit, shrinking the payout to stretch the funding.) 

Energy Upgrade treats each building as an entire system. Participants must bring in contractors 

to study how their homes use energy and recommend the most cost-effective means to trim that 

use. The approach maximizes energy savings, but it also makes the program more complex. 

Homeowners can't simply add insulation on their own and then ask the state for a check. 

"You look at your whole home together, not measure by measure," said Friday Apaliski, 

outreach coordinator for the San Francisco Department of the Environment, which has helped 

fund 222 upgrades so far. "It's new to people. This is a longer decision-making process, and it 

does require quite a bit of interaction with your contractor."

At the same time, many Californians aren't in a position to spend thousands of dollars retrofitting 

their homes, even if they can recoup some of the money later. In the aftermath of the housing 

bust, many homeowners remain "underwater," owing more for their property than it's 

currently worth.

"The level of investment is not to be ignored," said Tory Weber, manager of business programs 

in Southern California Edison's energy-efficiency division. The average Edison customer 

participating in Energy Upgrade spent $12,000 on home improvements before receiving rebates.

"Granted, we're providing significant funds," Weber said. "But especially in this economy, that's 

something to remember."

Homeowners who try to take out loans for the work often face resistance from bankers who 

aren't familiar with whole-house energy retrofits, McAllister said. As a result, some of the 

Energy Upgrade funding has been used to provide loans for homeowners to participate in the 

program. McAllister expects that banks will become more comfortable loaning money for 

retrofits as energy upgrades become more common. 

$22.8 million on marketing

The program's unfamiliarity to homeowners and contractors also proved to be a drain on its 

budget. The state and its partners spent more than $22.8 million just on marketing Energy 

Upgrade, paying for ads, a website and workshops for homeowners and lenders. An additional 

$5.1 million went to teaching contractors how to perform the upgrades and meet state standards 

for the work. 

The commission considers many of those expenses to be short-term costs, necessary for building 

the program's infrastructure and kick-starting the market for home energy upgrades. California 

officials want to see roughly 8 million retrofits by 2020. Otherwise, the state will need to build 
42 

more power plants than are currently planned. And California will have a harder time meeting its 

goals of cutting carbon dioxide emissions and fighting global warming. 

"Obviously, more retrofits is better," McAllister said, referring to the current numbers. "If we 

can't get up into the hundreds of thousands and millions of retrofits by 2020, we're not going to 

meet our efficiency goals." 

That will require greater interest from homeowners, however. And it leaves open the question of 

whether Energy Upgrade California has been worth the cost.

"Building standards and appliance standards make a big difference - that we can tell," said Mark 

Toney, executive director of The Utility Reform Network consumer group. "But we're skeptical 

about the current direction on the rebates. The whole home retrofit thing - the numbers just 

haven't borne out yet that this is a good use of ratepayer funds." 
DOE/GO-102013-3905 ▪ July 2013

Printed with a renewable-source ink on paper containing at 

least 50% wastepaper, including 10% post-consumer waste.