reports). The treatment of households that close their accounts – including any changes in tenants at the same site – may also vary between studies.

### Controlling for Double Counting

For programs in which efficiency measures can be tracked to a specific household (e.g., installation of insulation by a contractor), double-counted savings can be directly determined as the incremental participation in the non-behavior program. While most program evaluations to date have assigned all the incremental double counted savings to the other (non-behavioral) program, there is increasing recognition and acceptance that the behavior programs helped cause these additional savings, and thus could be credited with some percentage of these savings. Differences in the handling of double counted savings could lead to significantly different savings estimates for the same program.

### Expected Useful Life/Persistence

There is substantial debate and ongoing research regarding the Expected Useful Life (EUL) and persistence of energy savings from behavior-based programs. There are two key aspects in this discussion. The first aspect is how savings will vary from when participants first receive reports to when they receive reports after many months or even over a year or two. The second aspect is how long, if at all, savings might continue after participants stop receiving home energy reports.

## Summary of Findings

This paper examined both variation in savings and variations in methods for behavior-based programs. The examination of savings found that, while there is a wide range of savings (from 1.2% to 3.3%) the average saving was 2.1%, with moderate precision levels (± 15%). Even small differences in the percent saving, however, can translate to significant savings when multiplied by tens of thousands of customers. In terms of variation in methods, there are a number of methodological decisions that will lead to significantly different savings assumptions. These decisions range from the study design (e.g., experimental vs. quasi-experimental design and length of study period), to data analysis decisions (e.g., comparison of energy usage vs. change in energy usage, handling of opt out households) to policy decisions (e.g., controlling for double counting).

## Conclusion: Should Behavior Program Savings be Included in TRMs?

As noted above, one of the primary goals of a TRM is to provide transparency and consistency in the savings estimates. This paper demonstrates that there is a fairly wide range of savings estimates for behavior-based programs, plus there are many methodological decisions that can drive the ultimate outcome for measure savings. In fact, it is quite possible that one reason for the range of savings estimates is that the methodology adopted between many of these studies differed. Unfortunately, a number of reports did not contain sufficient detail to determine the exact details on the methodology to determine if this assumption of causation is valid, but the findings are suggestive that the differences in methodology at least contributed to the differences in savings estimates. The authors of this paper highly recommend that more research be conducted to understand exactly how the differences in methodology lead to differences in expected savings. Based on these findings, it appears that, at a minimum, methods protocols for behavior-based programs should be included in TRMs. This will accomplish a number of objectives, including:

- • that recommended methodologies are utilized, and that sub-optimal techniques • that recommended methodologies are utilized, and that sub-optimal techniques and shortcuts (e.g., shorter analytical periods) are avoided;

2013 International Energy Program Evaluation Conference, Chicago