- • for more consistent approaches between program evaluators, which will allow • for more consistent approaches between program evaluators, which will allow for improved validity in comparing findings from one study to another;
- • for more transparency and understanding of how the results were derived, • for more transparency and understanding of how the results were derived, particularly if reporting requirements are also included as part of the protocol development.

The authors also believe that it is likely that, in the near future, the savings values may be stable enough that they too can be deemed in TRMs. The first step in this direction might be deeming savings values based on specific program design and implementation, basing the savings off of prior research findings.4 This approach also accounts for potential regional differences (e.g., weather impacts) or implementation differences (e.g., targeting high vs. moderate energy user). As methodologies become more consistent and savings across regions become more comparable it then may be possible disaggregate legitimate regional/implementation differences from methodological differences, and to deem prospective savings estimates for similar program deployment (e.g., identical home energy reports and targeted customer types) in different regions.

### Will Deeming Methodology Stifle Research Innovation?

One concern with including behavioral-based programs in a TRM is that it will stifle research innovation. This same argument, in fact, is used against TRMs in general: once savings/methods are deemed, will it lead to less research (or less innovative research) to figure out what these measures/programs are actually savings? This argument is especially poignant with behavior-based programs, however, as they are still perceived as new programs with many areas of uncertainty (e.g., the breakdown of savings from behaviors vs. measure installations). Including methodological protocols in TRMs, however will reduce these areas of uncertainty by providing improved transparency and consistency: true differences in savings between regions, implementation contractors, and/or reporting strategies can be determined through vastly improved cross-study comparison. This is not only one of the primary purposes of a TRM, but also the goal of another recent Department of Energy (DOE) initiative called the Uniform Methods Project.5 In addition, TRMs need to be written in a way that they allow for improved methods/approaches, with the understanding that these changes may need to get applied prospectively, or even retrospectively, depending on the policy of the specific TRM. This is no different from a TRM algorithm that might be used for an air-conditioner or CFLs: as improved information becomes available (e.g., metered data on effective full load hours or run times) these data get incorporated into the next iteration of the TRM. TRMs are living documents, and can incorporate improved models/methodologies as research demonstrates they are improvements over existing approaches, just like they incorporate changes to other parameter inputs. Behavior-based programs, therefore, are ready for TRM prime time in the form of methodological protocols. Once this step is accomplished, it could pave the way for behavior-based programs to be included in TRMs with deemed savings values.

2013 International Energy Program Evaluation Conference, Chicago