# Presentation: 761384

## Part 4:  M&V Plan Review

- _Key Documents_
- Risk / Responsibility Matrix
- M&V Plan Template
- Detailed Guidelines for FEMP M&V Option A	www.eere.energy.gov/femp/financing/superespcs_mvresources.cfm

## Phase 2: Project Development

- _Contractor Responsibilities_
- Develop Risk & Responsibility Matrix
- Develop M&V Overview
- Explain and justify approach.
- _Owner Responsibilities_
- Review Risk & Responsibility Matrix and M&V Overview and provide feedback

## Phase 3: Negotiation and Award

- _Contractor Responsibilities_
- Perform Detailed Energy Survey (DES) and document baseline information
- Modify M&V plan to satisfy owner needs and desires
- _owner Responsibilities_
- Review and approve M&V Plan
- Witness and observe DES

## Risk & Responsibility Allocation

- How to allocate risks and responsibilities? Typically:
- Performance: Contractor
- Usage: owner
- Financial: Shared
- M&V approach should focus on:
- Verifying performance
- Characterizing usage
- Minimizing uncertainty cost-effectively

## Cost Effectiveness

- Need to balance M&V rigor with project risk
- Measure things that need measuring
- Consider required precision
- Estimate (stipulate) where risks are acceptable
- Law of Diminishing Returns applies
- Typically, initial M&V costs will be 3 – 15% of the capital cost (see DO-2); annual M&V costs will be 3 – 15% of the savings (see DO-3)

## M&V Costs

**M&V Rigor**

**M&V Cost**

**Incremental Value of information**

**$**

## M&V Costs and Project Savings

- Source: Initial Applications of the FEMP Measurement & Verification Guidelines in Super ESPC Delivery Orders: Observations & Recommendations. Schiller Associates, 1999

## Reviewing Measurement & Verification Plans for Federal ESPC Projects

**CAUTION: TECHNICAL EXPERTISE IS REQUIRED!**
- M&V Plan review requires qualitative assessment of engineering strategies
**Intimate understanding of project helpful:**
- Owner goals
- Site specific issues
- Agreed-upon risk allocation
- Objectives and constraints for each ECM

## Overview of 6 review steps

**STEP 1: Gather project documentation**
**STEP 2: Rank ECMs based on savings**
**STEP 3: Initiate review**
**STEP 4: Evaluate M&V approaches**
**STEP 5: Develop written report**
**STEP 6: Ensure key issues are resolved**

## Overview of 8 review steps (cont)

**STEP 5: Evaluate M&V approach**
**STEP 6: Develop report using review template**
**STEP 7: Provide written review to Owner & DOE**
**STEP 8: Ensure that action is taken on key issues**

## Step 1: Gather project documentation

- Final proposal, including technical appendices
- Note available documentation

## Step 2: Rank ECMs based on 			savings

- Identify all sources of savings for each ECM Rank ECMs on:
- % total cost savings (energy & water, O&M)
- % total commodity cost savings (energy & water)
- Note % savings for ECMs
- Focus on largest $ saving measures first
- O&M generally verified separately from energy & water savings

## Step 3: Initiate Review 

- Review Final Proposal
- Sections of proposal other than the M&V plan may be required
- Identify obvious deficiencies
- Use graded approach in review efforts

## Step 4: Evaluate M&V approaches

- Are M&V approaches appropriate?
- Note source of savings for each ECM
- Is M&V adequate for all sources?
- Review Risk & Responsibility Matrix
- Does M&V plan match proposed risk allocation?

## Step 4: Evaluate M&V approaches 		(cont.)

**Evaluate quality of baseline developed**
- Were key variables measured?
- Are assumptions reasonable and well documented?
- Were performance characteristics recorded?
- Are the energy and savings calculations correct? Reasonable?
- Were utility or weather based models validated?

## Step 4: Evaluate M&V approaches (cont.)

**Evaluate performance period activities:**
- What is the likelihood of success for this measure?
- Higher risk = more rigorous M&V
- Will key variables affecting ECM be measured? How often?
- Will meaningful performance period data be used to calculate savings?
- Sufficient to support guarantee?
- If only post-install measurements are used, is this data likely to change?
- Is the savings risk level appropriate? Aligned w/ R&R matrix?
- Have all objectives and constraints for the project been considered? For each ECM?
- Have O&M reporting responsibilities been adequately defined?

## Step 4: Evaluate M&V approaches 		(cont.)

**Evaluate overall project assumptions**
- Are contracted energy rates based on actual rates?
- Are marginal (not blended) rates used?
- Are escalation factors based on latest NIST data or other credible source?
**Evaluate proposed M&V Costs**
- Do M&V costs align with planned activities?

## Step 5: Develop written report

- Transfer key notes & findings to written report
- Summarize recommendations for each ECM in Summary of Findings

## Step 6: Ensure issues are resolved

- Send review to Owner staff
- Note dates and names of recipients
- Confirm actions are implemented to address recommendations
- If not implemented, provide written explanation
**Documentation is critical in contracts that will most likely outlive staff**

## M&V Plan Reviews - Lessons Learned

- Use a Savings/M&V Options Table
- Prioritize the review
- Improve proposed M&V approaches
- Document recommendations

## 1)  Use a Savings/M&V Options Table

- Gives a one-page picture that IDs all savings streams and associated M&V strategies
- Helps spot missing M&V options
- Helps you prioritize your M&V Plan review
- Is easy to create,
- Documents an important step in starting a review
- Is a valuable reference any time M&V of the project is revisited
- Helps focus future performance period reviews when details have been forgotten and the project begins providing measured results

**Notes:**
A Savings/M&V Options table will get you going and help guide you on your path through the review.  In addition, a table of this type documents important steps in the review and you will find it valuable in the future; thus it should be a permanent record in the project file.

## 2) Prioritize Your Review

- Use Savings/M&V Options Table to focus your review
- ID where most of the project savings lie
- Assess error potential with each savings stream to ID highest priority streams
- Prioritize ECMs/savings streams in accordance with savings potential and error risk

**Notes:**
A completed Savings/M&V Options Table will ID the biggest savers for you.  In order of importance, the reviewer can then begin to assess the potential for error associated with each savings stream to find out where the major M&V concerns lie. 

## Savings/M&V Options Table (in order of decreasing ECM savings) 

| ECM  | Description of ECM |   Electric Savings kWh/yr  |   Electric Savings $/yr  |  M&V Optn  | Savings Natl Gas MMBtu/ yr | Savings Natl Gas $/yr |  M&V Optn  |  Total Energy Cost Savings, ($/yr)  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 40 | Steam Traps/Regs | 0 | 0 | NA | 37,077 | 177,600 | None | 177,600  |
| 42 | Airside VFDs | 1,082,991 | 54,615 | B/A | 20,629 | 98,811 | B/A | 153,426  |
| 36 | Lighting | 1,227,500 | 61,903 | A | -481 | -2,305 | A | 59,598  |
| 43 | Water VFDs | 824,595 | 41,584 | A | 0 | 0 | NA | 41,584  |
| 30 | Balancing | 255,637 | 12,892 | B/A | 1,331 | 6,373 | B/A | 19,265  |
| 7 | B-3 Schedule | 199,151 | 10,043 | A | 1,607 | 7,698 | A | 17,741  |
| 6 | B-2 Schedule | 140,446 | 7,083 | A | 1,607 | 7,698 | A | 14,781  |
| 8 | B-4 Schedule | 63,240 | 3,189 | A | 1,205 | 5,773 | A | 8,962  |
| 4 | Deck T Reset | 51,109 | 2,577 | A | 1,298 | 6,217 | A | 8,794  |
| 5 | B-1 Schedule | 58,666 | 2,959 | A | 1,045 | 5,003 | A | 7,962  |
| 3 | Schedule DX | 9,865 | 497 | A | 52 | 251 | A | 748  |
|   | TOTAL | 3,913,200  |  197,343  |   |    65,369  |  313,118  |   |   510,460  |

**Notes:**
To begin, the Savings/M&V Options Table is sorted by ECM in order of descending savings. This is the first cut at establishing a priority for M&V efforts for the project.  From this sort, the highest savers where M&V concerns are the highest are easily seen at the top of the list in the far right column.  In contrast, the lower-priority savers are also easily seen at the bottom of the ranking.  Note that the two ECMs with the largest savings account for 65% of the total savings associated with the project.   If the top four are considered, a full 85% of the project’s savings are accounted for.  Thus, sound M&V for the few and highest priority ECMs will cover M&V for most of the savings of a project.  The second cut at establishing a priority for M&V efforts is to look at the potential uncertainty in estimating the savings for the higher priority ECMs.  If two ECMs produce a similar amount in cost savings (say $50,000/yr each), the one which contains the largest uncertainty in the estimate of savings should be the higher priority for M&V.  It is this ECM out of the two that poses the greatest shortfall risk to the customer.

## 3)  Improve proposed M&V approaches when needed 

- Change to more rigorous M&V options
- Add measurements to minimize uncertainties (particularly for large savings streams where errors could be most significant)
- Reduce M&V costs (switch M&V option, optimize sampling, etc.)
- Optimize sampling:
- minimizing number of measurements by focusing on the largest contributors
- rotating sampling during the performance period
- assuring random sampling where appropriate

**Notes:**
Most M&V Plans offer the potential for improvement.  There are many strategies for improving M&V.  Here are a few.  A review should produce improvements to M&V Plans and offer the same or lower risk to the customer. 

## 4)  Document recommendations from M&V Plan reviews

- Use to verify that ESCO/customer have addressed recommendations

**Notes:**
Documenting the results of M&V Plan reviews should be done, can be used for communicating improvement needs, can be used to verify implementation of recommendations, and documents that an important PF activity has been completed.

## Summary

- GET IT RIGHT!   M&V Plan reviews are critical:
- Great protocols/guidance do not equal great M&V Plans
- M&V Plans define project success
- Weak M&V Plans = weak projects
- Combine guidance with good engineering judgment to produce great M&V Plans

**Notes:**
If you are going to get any review right, make it the M&V Review.  For the M&V Plan creates the foundation for the future of the project.  The Post-Installation Report and all future Annual Reports will have their strengths defined in the M&V Plan.

## Summary (continued)

- Think of reviews as a process such that your experience will lead to quality improvements
- Use/develop review tools, they will:
- Focus your efforts on a path through the myriad details involved
- Help you minimize oversights
- Provide useful reference material for reviews and a basis for QA improvements

## More M&V resources:

- M&V Guidelines, M&V Planning Tool, etc:
- _[http://www.eere.energy.gov/femp/financing/superespcs_mvresources.cfm](http://www.eere.energy.gov/femp/financing/superespcs_mvresources.cfm)_
- Draft: Reviewing Measurement & Verification Plans for Federal ESPC Projects* (**and other M&V documents)*:
- _[http://ateam.lbl.gov/mv/](http://ateam.lbl.gov/mv/)_

## Review & Discussion

- Performance must be verified to guarantee savings
- owner often assumes usage risk
- Uncertainty is inherent in M&V
- M&V costs need to be balanced against project risks

## Review Questions

- How do we measure savings?
- When might an owner accept performance risk?

- Click hereto go to M&V Part 5

**Notes:**
Answers

1) Performance: ESCO. Usage: owner. 

2) No. A building addition external to the project would not require a baseline adjustment.

3) Yes. A building addition within the boundary would require adjusting the baseline to account for the increased load.