Measure by Measure: the Real Reasons for Gaps in Claimed and Evaluated Savings

Energy efficiency programs can deliver on large energy efficiency savings goals, but when they don’t, the reasons for the gap needs to be examined. Many projects of various types were evaluated through California’s Investor Owned Utility (IOU) sponsored energy efficiency programs for program years 2006 through 2008. Analysis of these individual projects, mainly involving lighting and Heating, Ventilation and Air Conditioning (HVAC) systems, revealed that the technical or gross savings gap at a specific site was due to a combination of factors at each site, including improper assumptions about operating hours, operating load, or the applicable baseline. This paper highlights the primary factor in the savings gap for different types of measures, concentrating on the gross savings claims (as opposed to the net-to-gross or net-offree–ridership claims). The primary factor affecting energy savings for projects in various programs is identified, and the results are aggregated by end use and measure type. Identifying the importance of these factors can lead evaluators and implementers in better focusing on and documenting the parameters surrounding those factors for a particular measure. This topic is especially relevant as utility energy efficiency goals change and as utilities nationwide are being rewarded for their efficiency efforts – and penalized when they fail. Importance of Energy Evaluation in the Buildings Sector The buildings sector accounts for a large portion of total annual national energy use (EIA 2010). Energy use reductions in this sector have been the target of energy savings efforts and program offerings by governments and utilities. The evaluation of any program is important in ascertaining the achieved results as compared to the claimed results, and the evaluation of energy programs targeted at the buildings sector is no exception. The diversity of programs across different subsectors and technologies, along with multiple project drivers and market actors, make these evaluation efforts particularly important and instructive in determining the important factors to concentrate on when claiming – and verifying – energy savings. This attention is particularly useful in ensuring the ex-ante claimed and ex-post verified energy savings are accurate and defensible. A reliable estimate of the energy savings from an equipment-retrofit project is important for utility companies, performance contractors, and building owners (Yalcintas 2008). This paper focuses on the commercial and institutional portion of the buildings sector. The sector is diverse and comprises a large percentage of total building energy use. Commercial and institutional buildings are generally larger than buildings comprising the residential sector (with the exception of some multi-family housing complexes) and entail greater energy use per building. Energy impact evaluation reports authorized by the California Public Utility Commission (CPUC) Energy Division (ED) for the 2006-2008 evaluation cycle were reviewed for this study. We analyzed the impact evaluation reports for four contract groups; these are publicly available 4-237 ©2012 ACEEE Summer Study on Energy Efficiency in Buildings at www.calmac. org. Further, we reviewed a large number of site specific energy project impact evaluations that formed the basis for these reports. This review enabled a deeper assessment and quantification of the frequency for the possible reasons for the savings gap that exists between claimed and evaluated savings. The results and findings of the energy evaluations of energy efficiency programs in the program years 2006 – 2008 conducted for the CPUC (ADM et al. 2010; Itron et al. 2010; SBW Consulting et al. 2010; Summit Blue et al. 2010) are examined to assess the critical areas of importance for improvement in energy savings estimation and verification. When we consider the program induced energy savings, it is important to realize that there are both gross impact and net impact components. These components lead to gross realization rates, net to gross ratios, and net realization rates combining those factors. This paper concentrates on the gross energy savings and gross realization rate. The gross impact component of energy savings refers to the savings technically possible through the implementation of the measure. These are the savings that an energy end user or program participant most commonly associates with energy savings. The gross realization rate is calculated by dividing the ex-post gross savings by the claimed ex-ante savings. Gross realization rates were calculated and are shown in this paper for different measure types based on site specific energy project realization rates. The measure type realization rate was calculated as a simple average from the site specific project gross realization rates collected from the site specific project impact evaluations that form the basis of the impact evaluation reports reviewed for this paper (ADM et al. 2010; Itron et al. 2010; SBW Consulting et al. 2010; Summit Blue et al. 2010) Research Performed to Determine Factors for the Energy Savings Gap Gross impacts are typically determined for larger commercial energy projects through application review, project documentation review, site visits, measurement and verification (M&V) activities, and on-site interviews. Metering/monitoring may be accompanied by data collection of pre-retrofit and post-retrofit operating conditions. The evaluators determine an expost savings figure, reflecting the gross energy savings found based on actual ‘as found’ operating conditions. The evaluators attempt to ascertain the most accurate baseline condition to determine actual grid-level impacts over the life of the installed measure, reflecting the preretrofit operating conditions which would have prevailed in the absence of the program. In some cases, a different or more accurate methodology for energy savings may be used, perhaps involving the knowledge of post installation energy use or other parameters not available during project implementation and ex-ante savings estimation. Adjustments are included for occupancy, baseline, weather and other parameters that have changed from the pre-retrofit conditions to the post-retrofit conditions. Information may be collected on the remaining useful life (RUL) of the equipment and the normal replacement schedule for equipment, in order to differentiate retrofits where the program induced early equipment replacement verses a normal replacement at the end of equipment life. In the case of early equipment replacement, dual baselines are appropriate. The baseline until the end of the useful life of the equipment is the existing in-situ equipment; the baseline after the end of the remaining life is the code required or standard practice replacement equipment. More details and a description of dual baseline are provided in a recent decision for the CPUC addressing this and other topics (ALJ 2011). 4-238 ©2012 ACEEE Summer Study on Energy Efficiency in Buildings Site specific project reports for four contract groups in the CPUC funded energy evaluations for the 2006-2008 cycle were analyzed, across contract groups, in order to compile a relatively large number of projects in each measure category; this was done to enable a deeper assessment of the possible reasons for the savings gap that existed between claimed and evaluated savings. The evaluated contract groups included 129 projects. Program descriptions from abstracts on the CALMAC website and the number of projects per program are summarized as follows:  Commercial Facilities [67 projects] (ADM et al. 2010) The Commercial Facilities Contract Group evaluated two market-sector focused incentive programs. This report presents the energy impact results for the Pacific Gas and Electric (PG&E) High Tech and Large Commercial customer segments. The programs claimed savings through equipment retrofit, retrocommissioning and new construction activities with particular emphasis on providing design assistance for installing custom energy efficiency measures.  Major Commercial [43 projects] (SBW Consulting et al. 2010) The Major Commercial contract group includes four measure categories (Custom Lighting, Custom HVAC, Custom Other and Audit) within commercial, industrial, and agricultural programs that were implemented by Southern California Edison (SCE), Southern California Gas (SCG) and San Diego Gas and Electric (SDGE).  Government Partnerships [13 projects] (Summit Blue et al. 2010) This report presents the evaluation results for the energy efficiency projects and programs in the governmental sector. This report summarizes the impact evaluation activities related to following subset of partnerships: the University of California, California State University (UC/CSU) Partnership Program, the California Community Colleges (CCC) Partnership Program, and the Palm Desert Partnership Program. The measures evaluated in this program include: early retirement of residential air conditioners, refrigerant charge and airflow (RCA) adjustment for both residential and commercial refrigerators.  PG&E Fabrication, Process and Manufacturing [6 projects] (Itron et al. 2010) This report presents the evaluation results for the energy efficiency projects and programs within the scope of the California Public Utilities Commission’s (CPUC) Pacific Gas and Electric Fabrication, Process, and Manufacturing (PG&E Fab) evaluation contract group. PG&E carried out a very large and extensive implementation effort in the industrial sector. There are a number of specific findings that help to explain why the evaluated expost savings estimates are significantly below the ex-ante estimates. These include the following: errors in baseline determination; inadequate basis for claims of how some measures save energy; inadequate enforcement of program and policy rules; inadequate consideration of total system energy analysis; ef