From: Frey, Nathan J. < Nathan_J._Frey@omb.eop.gov > 

Sent: Friday, July 10, 2015 5:54 PM 

To: Culligan, Kevin; Harvey, Reid; Vasu, Amy 

Cc: Szabo, Aaron 

Subject: Summary of Interagency Comments Under EO 12866 and 13563 -- Draft EM&V 

guidance 

Attached please find a summary of interagency comments on the EPA draft EM&V guidance 

associated with the Clean Power Plan draft final rule. 

 

Thanks and please let me know if you have any questions. 

Nathan 

<< SummaryofinteragencycommentsunderEO12866_13563_EO 12866_EM&V Guidance.docx >> 
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

1

Major Comments  

1. Instead of providing options or “optional” resources, Reviewers recommend providing specific

direction about what is required.

2. Reviewers recommend explicit discussion of what is “presumptively approvable.” What will be

presumptively approvable?  “Best practices” is used on page 9, and “Recommendations” is used

throughout section 3.  Reviewers recommend that the Uniform Methods Project protocols are

presumptively approvable.

a. Though Appendix C provides useful examples, we are concerned that it may be of

limited utility without more specificity about what is required and the approaches

(which could be cited here) that would be presumptively approvable.  Reviewers

recommend expanding appendix C into a full list of pre‐approved measures and

associated protocols.  

b. Suggest using the terms: “likely to be approved” and “may be approved pending review”

– This structure would give entities some level of regulatory certainty, while still

allowing maximum flexibility. Reviewers believe entities will choose more regulatory

certain methods, which will likely reduce the administrative burden to EPA.  

3. Section 4 (Supplemental Guidance for Specific Categories of EE) is missing CHP and T&D savings.

Reviewers recommend adding a discussion of CHP and a discussion of T&D savings.  

4. This guidance should explicitly be linked to the federal plan, indicating that energy efficiency

ERCs and allowances could be used under the federal plan if they are created in accordance with

the guidance.  

5. Reviewers recommend including (or referencing the location in the preamble or the TSD) the

steps required for efficiency credit approval under both plan types. It would also be useful to

include examples of roles and responsibilities.

6. Reviewers recommend issuing a standard savings reporting form that includes a description of

evaluation methods. The Northeast Energy Efficiency Partnership standard reporting form could

serve as an example. Reviewers offer to provide technical assistance in this effort.

Technical Comments

1. Recommend clarifying throughout whether state regulator approval means state air office

(responsible for CAA) or state PUC/similar body (often responsible for EE funding) or both.

2. Reviewers recommend separating guidance and protocol documents, for example on page 12

and page 30 (Table 3‐1).  For example, SEE Action provides guidelines, but UMP provides

protocols.

3. Any mention of IPMVP should also include instructions about which IMPVP method and what

additional EM&V guidance would be paired with IPMVP. IPMVP alone, without including

additional detail, should not be an acceptable protocol.  

4. Appendix C is missing references to relevant NEEP protocols. Reviewers suggest adding them for

each measure.
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

2

5. Reviewers suggest eliminating references to outdated materials, e.g. page C‐6 Fels, M.F., ed.

(February/May 1986). Energy and Buildings: Special Issue Devoted to Measuring Energy Savings:

The Scorekeeping Approach. (9:1&2).  

6. Introduction, page 2. Reviewers suggest the following edit. “There is also limited U.S. experience

with...”

7. Page 10: Reviewers suggest editing or removing this sentence because it is incorrect: “Although

randomized control trials are a very reliable approach for determining EE savings, for practical

reasons they cannot always be applied to many types of programs. RCTs are recommended

when appropriate and possible.”

8. Page 3: Guidance does not seem to address the “cross‐jurisdiction inconsistencies” that EPA

acknowledges. Lack of EM&V consistency threatens the ability to issue tradable emission

reduction credits.  

9. Page 3: Reviewers support EPA’s effort in “striking a reasonable balance between EMV rigor and

credibility on the one hand, and the costs associated” but believes that EPA should do more to

address cross‐jurisdiction consistency.  

10. Page 4: Regarding the statement “this guidance is consistent with “best practice” levels of

EM&V,” Reviewers believe that what EPA has included can be less rigorous than best practice”

and that the options given should be limited to more accurately reflect best practice.

11. Page 19: Regarding, “For deemed savings to be used, they must be documented in a publicly

available document or database (frequently referred to as a technical reference manual or TRM)

that is reviewed and updated at least every 3 years to keep current with new data, changing

baselines, and technology innovation.”  This requirement is too vague to result in the quality

TRM that EPA envisions. EPA should specify that all values, not just some values, should be

updated every 3 years.  

12. Regarding page 7 and B‐5, accredited independent verifier has yet to be defined and much of

the quality of the EM&V EPA asks for is dependent on high quality accredited independent

verifiers being in place.  Reviewers suggest keeping this concept but clarifying what is

envisioned.

13. In several places, such as page 22 and 42, the guidance discusses 80/10 (confidence/precision)

but it is unclear if this is the minimum standard and how this applies. Suggest clarification.

Reviewers think 80/10 makes sense as a minimum standard, allowing for some justified

exceptions.  

14. Page 3 blue box: Reviewers suggest adding a footnote to the last sentence in the blue box. One

of the items not included is how to calculate avoided emissions from an EE measure. This

document should include explain why this is not needed for crediting as an ERC, for example by

referencing the MWh in the denominator explanation from the preamble. Reviewers can also

provide a reference to a forthcoming NREL paper, “Energy Efficiency under Alternative Carbon

Policies: Incentives, Measurement, and Interregional Effects,” that includes a technical

explanation.
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

3

15. Page 39. Clarify how frequently a measure needs to be measured to ensure accurate

persistence/lifetime calculation.  Does every measure in a project need to be checked at least

once every 2 years? Or 3 years? This section generally could use clarification; it allows too much

user flexibility.

16. Page 42. The use of expert opinions, e.g. Delphi, should only be permitted when paired with

market data.  

17. Page 46: Clarify that evaluation cycle timeline on page 46 “applies to once every 4  years or

whenever new federal or state standards are put in place for the same product?

18. B‐5 Suggest including Delphi method in glossary of terms, as it is mentioned multiple times.  

19. Page A‐3. Part 3, Step 6, should also reference a UMP option.