From: Frey, Nathan J. <Nathan_J._Frey@omb.eop.gov> 

Sent: Thursday, July 23, 2015 11:05 PM 

To: Culligan, Kevin; Harvey, Reid; Vasu, Amy 

Cc: Szabo, Aaron; Grossman, Andrea 

Subject: Summary of Interagency Comments Under EO 12866 and 13563 -- CPP Draft EM&V 

guidance 

Attached please find a summary of additional interagency comments under EO 12866 and 13563 

on the draft EPA EM & V guidance associated with the Clean Power Plan. This should be the final 

submission of interagency comments on this document. 

 

Thanks and please let me know if you have any questions. 

Nathan 

<< SummaryofinteragencycommentsunderEO12866_13563_EO 12866_EM&V 

Guidance_7_23_2015.docx >> 
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

    

Additional Principal Comments

1. * EPA should take comment on whether EPA should release a specific list of possible efficiency

measures and recommended EM&V protocols for each measure.

2. * EPA should take comment on whether EPA should categorize the list of possible measures and

EM&V protocols for each measure as “likely to be approved” and “may be approved pending review.”   

Additional Technical Comments

1. * Page 1 states that "Further, this document does not apply to states submitting mass‐based

plans. In these cases, compliance is determined solely based on measured CO2 emissions at the

stack." Reviewer notes that the Preamble does allow energy efficiency measures that occur in

mass states to generate ERCs for compliance with a rate‐based target, provided that there is an

adequate demonstration that the savings offsets generation in a state with a rate‐based target.

Reviewer suggests including this clarification to maintain consistency between the scope of this

document and the range of ERCs allowed to be generated by EE in the preamble.  

2. * Page 9, “When applying measurement and verification methods”

Reviewer recommends that for all M&V methods, a requirement should be in place that to

ensure the evaluator did indeed adhere sufficiently to the protocol. EPA could take comment on

how.

3. * Page 9: EPA’s guidance needs to provide more parameters about how IPMVP is implemented

before it can be considered a best practice.

4. * Page 12, “When using stipulated effective useful life (EUL) values (see subsection on Effective

Useful Life and Persistence of Savings) for reporting savings, report “annualized” savings as if

they started accruing on January 1 of the reporting year and continued in the program for a full

year  (irrespective of which day the EE activity began saving energy).”

Reviewer suggests requiring tracking systems to record measure date of installation where

possible.

5. * Page 14, “Are developed by independent, third parties”

Reviewer recommends the guidance further define independent, third parties. Reviewer

recommends a provision that these entities are “not subject to undue influence” and attest to

this this statement when conducting evaluation activities.

6. * Page 14, “Update deemed savings values  on a going‐forward basis (retroactive updating is not

required); see subsection on Reporting Incremental and Cumulative Savings.”

Reviewer recommends requiring that all deemed values and estimates must be no more than 5

years old unless further justification is provided.

7. * Page 16, “These level of accuracy and reliability of quantified and verified MWh savings values

is determined by the rigor of the EM&V methods applied”

Along with the description of the methods, EPA should take comment on how EPA could

determine the rigor of the methods applied. Once the rigor is determined, the evaluator could

classify the rigor as high, medium or low along with a justification.
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

    

8. * Page 20, “Persistence  studies conducted by independent entities at least once every 5 years

to determine values.”

Reviewer recommends setting a threshold for which persistence studies are required. For

example, programs/measures that produce large savings should be required to determine EUL

on the basis of a persistence study.

9. * Page 26, “In this way, EPA recognized that the application of EM&V guidelines and protocols

requires professional judgment and assessment of the EE activities to determine the appropriate

EM&V method, assumptions, other factors to apply, and applicability within the context of EPA

emissions guidelines, calculations and other elements”

Reviewer recommends that for common, simple measures, more prescriptive methods for

measuring savings should be required. An evaluator may use alternate methods but those

alternatives should be subjected to further review by EPA or the state.

Reiterated Comments  

1. Instead of providing options or “optional” resources, reviewers recommend providing specific

direction about what is required.

2. Reviewers recommend explicit discussion of what is “presumptively approvable.” What will be

presumptively approvable?  “Best practices” is used on page 9, and “Recommendations” is used

throughout section 3.  Reviewers recommend that the Uniform Methods Project protocols are

presumptively approvable.

a. Though Appendix C provides useful examples, we are concerned that it may be of

limited utility without more specificity about what is required and the approaches

(which could be cited here) that would be presumptively approvable.  Reviewers

recommend expanding appendix C into a full list of pre‐approved measures and

associated protocols.  

b. Suggest using the terms: “likely to be approved” and “may be approved pending review”

– This structure would give entities some level of regulatory certainty, while still

allowing maximum flexibility. Reviewers believe entities will choose more regulatory

certain methods, which will likely reduce the administrative burden to EPA.  

3. Section 4 (Supplemental Guidance for Specific Categories of EE) is missing CHP and T&D savings.

Reviewers recommend adding a discussion of CHP and a discussion of T&D savings.  

4. This guidance should explicitly be linked to the federal plan, indicating that energy efficiency

ERCs and allowances could be used under the federal plan if they are created in accordance with

the guidance.  

5. Reviewers recommend including (or referencing the location in the preamble or the TSD) the

steps required for efficiency credit approval under both plan types. It would also be useful to

include examples of roles and responsibilities.
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

    

6. Reviewers recommend issuing a standard savings reporting form that includes a description of

evaluation methods. The Northeast Energy Efficiency Partnership standard reporting form could

serve as an example. Reviewers offer to provide technical assistance in this effort.

Technical Comments * (page numbers have been updated to new version and comments addressed

have been deleted)

1. Recommend clarifying throughout whether state regulator approval means state air office

(responsible for CAA) or state PUC/similar body (often responsible for EE funding) or both.

2. Appendix C is missing references to relevant NEEP protocols. Reviewer suggests adding them for

each measure.

3. Reviewer suggests eliminating references to outdated materials, e.g. page C‐6 Fels, M.F., ed.

(February/May 1986). Energy and Buildings: Special Issue Devoted to Measuring Energy Savings:

The Scorekeeping Approach. (9:1&2).  

4. Introduction, page 2. Reviewers suggest the following edit. “There is also limited U.S. experience

with...”

5. Page 10: Reviewers suggest editing or removing this sentence because it is incorrect: “Although

randomized control trials are a very reliable approach for determining EE savings, for practical

reasons they cannot always be applied to many types of programs. RCTs are recommended

when appropriate and possible.”

6. Page 3: Guidance does not seem to address the “cross‐jurisdiction inconsistencies” that EPA

acknowledges. Lack of EM&V consistency threatens the ability to issue tradable emission

reduction credits.  

7. Page 3: Reviewers supports EPA’s effort in “striking a reasonable balance between EMV rigor

and credibility on the one hand, and the costs associated” but believes that EPA should do more

to address cross‐jurisdiction consistency.  

8. Page 4: Regarding the statement “this guidance is consistent with “best practice” levels of

EM&V,” reviewers believe that what EPA has included can be less rigorous than best practice”

and that the options given should be limited to more accurately reflect best practice.

9. In several places, such as page 22 and 42, the guidance discusses 80/10 (confidence/precision)

but it is unclear if this is the minimum standard and how this applies. Suggest clarification.

Reviewers think 80/10 makes sense as a minimum standard, allowing for some justified

exceptions.  

10. Page 4 blue box: Reviewers suggest adding a footnote to the last sentence in the blue box. One

of the items not included is how to calculate avoided emissions from an EE measure. This

document should include explain why this is not needed for crediting as an ERC, for example by

referencing the MWh in the denominator explanation from the preamble. Reviewers can also

provide a reference to a forthcoming NREL paper, “Energy Efficiency under Alternative Carbon

Policies: Incentives, Measurement, and Interregional Effects,” that includes a technical

explanation.
Summary	of	Interagency	Working	Comments	on	Draft	Language	under

EO12866/13563	Interagency	Review.		Subject	to	Further	Policy	Review.

    

11. Page 39. Clarify how frequently a measure needs to be measured to ensure accurate

persistence/lifetime calculation.  For example, does every measure in a project need to be

checked at least once every 2 years? Or 3 years? This section generally could use clarification; it

allows too much user flexibility.

12. Page 35. The use of expert opinions, e.g. Delphi, should only be permitted when paired with

market data.  

13. Page 40: Clarify that evaluation cycle timeline on page 46 “applies to once every 4 years or

whenever new federal or state standards are put in place for the same product?

14. B‐3 Suggest including Delphi method in glossary of terms, as it is mentioned multiple times.  

15. Page A‐1. Part 3, Step 6, should also reference a UMP option.