**Seeing Savings From an ESPC Project in Fort Polk's Utility Bills**

**DRAFT**

**September 23, 2004**

**John A. Shonder**

**Patrick J. Hughes**

**Oak Ridge National Laboratory**

**Seeing Savings From an ESPC Project in Fort Polk's Utility Bills**

[Executive Summary 1](#executive-summary)

[1. Introduction 5](#introduction)

[2. When Savings Should or Should Not be Apparent in Utility Bills
5](#when-savings-should-or-should-not-be-apparent-in-utility-bills)

[3. Case Study of Utility Bill Analysis: Fort Polk ESPC Project
6](#case-study-of-utility-bill-analysis-fort-polk-espc-project)

[3.1 Fort Polk ESPC Project 7](#fort-polk-espc-project)

[3.2 ORNL's 1998 Evaluation of the Fort Polk Project
7](#ornls-1998-evaluation-of-the-fort-polk-project)

[3.3 Data Available for This Case Study
8](#data-available-for-this-case-study)

[3.4 Resolution of Electricity Use Savings From Utility Bill Analysis
9](#resolution-of-electricity-use-savings-from-utility-bill-analysis)

[3.5 Resolution of Reduced Peak Demand From Utility Bill Analysis
11](#resolution-of-reduced-peak-demand-from-utility-bill-analysis)

[3.6 Persistence of Savings 12](#persistence-of-savings)

[3.7 Resolution of ESPC Cost Savings From Utility Bill Analysis
15](#resolution-of-espc-cost-savings-from-utility-bill-analysis)

[4. M&V Lessons Learned 19](#mv-lessons-learned)

[4.1 Risk and Responsibility Matrix 19](#risk-and-responsibility-matrix)

[4.2 ECM Performance 20](#ecm-performance)

[4.3 Weather 20](#weather)

[4.4 Energy Loads 22](#energy-loads)

[4.5 Energy Price Escalation 23](#energy-price-escalation)

[4.6 Summary of M&V Best Practices 28](#summary-of-mv-best-practices)

[5. Implications for Agencies Deciding to Reconcile ESPC Savings and\
Utility Bills
29](#implications-for-agencies-deciding-to-reconcile-espc-savings-and-utility-bills)

[6. Conclusions 30](#conclusions)

[7. References / Bibliography 32](#references-bibliography)

Appendix A: Utility Bill Analysis to Determine Electrical Energy Savings

Appendix B: Utility Bill Analysis to Determine Electricity Demand
Savings

> Appendix C: Analysis to Determine Persistence of Savings
>
> Appendix D: Utility Bill Analysis to Determine Electrical Cost Savings

**Seeing Savings From an ESPC Project in Fort Polk's Utility Bills**

## Executive Summary

While it is generally accepted that the energy projects implemented by
federal agencies save energy and costs, the savings are usually not
obvious in the utility bills. This is true for many valid technical
reasons, even when savings are verified in other ways to the highest
degree of certainty. However, any perceived deficiency in the evidence
for savings is problematic when auditors or other observers evaluate
energy projects and energy management programs.

Only in rare cases can savings from energy projects be seen in the
facility's utility bills, simply as a matter of scale. The typical
energy project affecting 25% of a facility's load and reducing energy
use by 20% would yield a change in the utility bill of only 5%, which is
in the same range as variations due to weather. FEMP M&V guidelines
(FEMP 2002) recommend against using simple utility bill analysis for any
federal energy savings performance contract (ESPC) projects. The
guidelines specify that regression modeling is appropriate when
predicted savings are greater than about 10 to 20% of the site's energy
use at the meter on a monthly basis (p. 164). Also required are at least
12 and preferably 24 months of pre-installation data to calculate a
baseline model, and at least 9 and preferably 12 months of
post-installation data to calculate first-year savings.

Utility bill analysis under these circumstances can resolve savings when
comparing the periods immediately before and after the retrofits.
However, factors that affect energy use change constantly, and those
changes compound over the years. After several years it would be
impossible through any kind of utility bill analysis to distinguish
between the performance of the energy conservation measures installed in
the energy project and the effects of factors such as operating
schedules, occupancy levels, or new or disconnected loads.

In the special case of a large geothermal heat pump (GHP) retrofit at
the Army's Fort Polk in 1995 -- 1996, the authors' analysis of utility
bills does unequivocally confirm and quantify savings. (See the
appendices for discussion of analysis methods.) Using utility bills from
12 months immediately before and after the retrofits, we show that the
electricity savings are 24.3 ± 4.0 million kWh (a 95% confidence
interval), and that the utility bills predict peak summer demand savings
of 7.27 ± 3.1 MW. These results agree with our 1998 evaluation of the
project (Hughes and Shonder 1998), which was based on data from
15-minute interval submetering of the electric distribution feeders
serving only the family housing areas, where the project was
implemented, not on data from the base-wide utility meters.

Analysis of the utility bill data did indicate that Fort Polk's
base-wide annual electricity use increased by about 13 million kWh, or
about half of the ESPC project's savings, between the first
post-retrofit year (1996-97) and 2002-03. Analysis of data from four
feeders serving about 12% of family housing --- a large sample for such
an analysis --- shows that between the first post-retrofit year and
2003-04 electricity use in family housing increased about 2.2%, or 0.31%
per year. This family housing increase, which is far less than we
predicted in 1998 due to plug load growth, accounts for about 1 million
of the total 13 million kWh rise in electricity use during the 1996-97
to 2002-03 period. The savings due to the GHP retrofits have apparently
persisted. Under the ESPC, the energy service company (ESCO) is not
responsible for plug load growth by tenants in housing, but in this
cooling dominated climate, an added benefit of the highly efficient GHPs
is that they minimize the net impact of growing plug loads on housing
electricity use.

Agencies that decide to correlate the impact of individual ESPC projects
directly to changes in their utility bills, either at the site or agency
level, will probably need to consider implementing a system for tracking
what their utility bills would have been had the energy efficiency
projects never been implemented. Since most agencies have tracking
systems for their actual energy use and costs across all agency sites,
perhaps these same systems could be enhanced for this purpose. However,
agencies will have to weigh the value of the extra effort required to
calculate actual savings against its cost.

The current state of the art in federal M&V practices enables the
government to cost-effectively verify savings to an acceptable degree of
certainty and without allocating unmanageable risks to the ESCO that
would inevitably burden projects with premium pricing and financing
costs as compensation for bearing those risks. Generally the ESCO's
risks are limited to guaranteeing the performance of the ECMs.
Performance is translated into contracted cost savings assuming typical
weather, pre-retrofit baseline load levels for the non-project-related
loads, and stipulated energy cost escalation rates.

While contracted savings as calculated may differ from actual cost
savings in a given year, over the contract term the two values tend to
converge because over time the actual weather (for example) will tend to
conform to the same average used for the experience-based stipulation.
An additional margin of safety (and cost savings) is afforded by the
fact that ESCOs universally guarantee less than 100% of estimated
savings to increase their certainty of meeting the guarantee.

Our further analysis of Fort Polk's utility bills illustrates the
effects of basing calculations of contracted savings on stipulated
typical weather and an energy cost escalation rate of 0.5% per year. The
differences between actual savings and contracted savings (see
Conclusions and figures 12 and 13) vary over the 6 years analyzed, but
cumulatively over the period the differences decline.

While it is true that contracted ECM cost savings may differ from actual
ECM cost savings in a given year (because, for example, the weather was
not typical or energy rates were not as forecasted), the fact remains
that the government has remedies against the ESCO if the verified
contracted cost savings do not match or exceed the guaranteed cost
savings each year. When appropriate assumptions and choices are made,
annual contracted and actual savings will be reasonably similar, and
over the contract term contracted and actual savings tend to converge.

The alternative to using simplifying assumptions for the purpose of
calculating savings --- having the ESCO take the risk that factors such
as the weather, future energy rates, and the government's own operating
hours and non-project-related loads will affect savings --- would be a
poor and expensive choice for the government.

**Seeing Savings From an ESPC Project in Fort Polk's Utility Bills**

## 1. Introduction

Federal agencies have implemented many energy efficiency projects over
the years with direct funding or alternative financing vehicles such as
energy savings performance contracts (ESPCs). While it is generally
accepted that these projects save energy and costs, the savings are
usually not obvious in the utility bills. This is true for many valid
technical reasons, even when savings are verified in other ways to the
highest degree of certainty. However, any perceived deficiency in the
evidence for savings is problematic when auditors or other observers
evaluate the outcome of energy projects and the achievements of energy
management programs. This report discusses under what circumstances
energy savings should or should not be evident in utility bills.

In the special case of a large ESPC project at the Army's Fort Polk, the
authors' analysis of utility bills does unequivocally confirm and
quantify savings. The data requirements and methods for arriving at
definitive answers through utility bill analysis are demonstrated in our
discussion of the Fort Polk project.

The following paragraphs address why the government generally should not
expect to see savings from ESPC projects in their utility bills. We also
provide an overview of related lessons learned about measurement and
verification (M&V) --- methods that practitioners have found to be more
practical, straightforward, and cost-effective than utility bill
analysis, and best practices that can assure best value for the
government.

## 2. When Savings Should or Should Not be Apparent in Utility Bills

The first problem with seeing energy savings in utility bills is a
matter of scale --- the magnitude of savings compared to the magnitude
of metered energy use. Consider a simplified example --- a typical
energy efficiency project that affects 25% of a facility's total load
and yields average annual energy savings of 20%. (Larger projects are
rare.) Assuming that all savings and load are electricity and one meter
measures the whole site (as is common), then an average change of only
5% in the utility bill would be expected. A simple comparison of utility
bills before and after the project may not show a savings of this
magnitude, because variations due to weather are generally in the same
range or larger.

In later years, it would become difficult or impossible to isolate the
small change attributable to this typical energy project from the normal
variation in utility bills, even using sophisticated analysis methods.
Over time, many factors in addition to weather contribute to changes in
a facility's energy use, such as occupancy rates, operating hours, and
acquisition and use of new energy-consuming equipment. These changes are
compounded over time, and the utility bill provides no information to
help distinguish between their effects.

"Utility bill analysis," which encompasses simple utility bill
comparison, is included under M&V Option C as defined in the
*International Performance Measurement & Verification Protocol* (IPMVP)
and the Federal Energy Management Program's (FEMP's) *M&V Guidelines*
(an application of IPMVP to federal energy projects). However, FEMP
recommends against depending on simple utility bill comparisons:

> ... energy savings evaluations using whole-building or facility-level
> metered data may be completed using techniques ranging from simple
> billing comparison to multivariate regression analysis. Utility bill
> comparison is the use of utility billing data ... and simple
> mathematical techniques to calculate annual energy savings. Utility
> bill comparison is a very simple and, typically, unreliable method. It
> is applicable only to very simple ECMs in which energy use changes are
> a direct result of ECM installation. Therefore, this method is not
> recommended for most federal ESPC projects. (FEMP 2000, p. 164-165)

Regression modeling of utility billing meter data is an acceptable
Option C method of calculating savings, but only when enough data is
available and savings represent a large proportion of metered energy
use, according to both the FEMP *M&V Guidelines* and ASHRAE Guideline
14-2002, "Measurement of Energy and Demand Savings." FEMP guidelines
specify that regression modeling is appropriate when predicted savings
are greater than about 10 to 20% of the site's energy use at the meter
on a monthly basis (p. 164). Also required are at least 12 and
preferably 24 months of pre-installation data to calculate a baseline
model, and at least 9 and preferably 12 months of post-installation data
to calculate first-year savings.

Even when utility bill analysis can reliably establish savings, it is
difficult to verify savings persistence over time using this technique
because buildings and facilitates are dynamic, and many factors, such as
weather, occupancy levels, operating hours, plug loads, new connected
loads, disconnected loads, energy rate changes, and others, affect
energy usage and cost. Rarely is it possible to track all of these
factors over time and adjust and correct the utility bill analysis in
order to isolate the savings attributable to an energy project.

The best chance of clearly seeing project impact in a utility bill
analysis occurs in cases of very large, comprehensive projects, where
the analysis compares periods immediately before and after the retrofit.

## 3. Case Study of Utility Bill Analysis: Fort Polk ESPC Project

The large ESPC project implemented at Fort Polk in 1995 -- 96 is an
instance where project savings should be apparent in the utility bills
when the analysis compares periods immediately before and after the
retrofit. The project was a comprehensive retrofit of 4003 family
housing units that before the retrofit accounted for about 42% of Fort
Polk's total electricity use of about 190 million kWh per year. Since
family housing represents such a large share of total electricity use,
and the ESPC project resulted in large reductions in electricity use in
family housing (32.5%), the post-wide savings of ∼14% resulting from the
project should be apparent in the utility bills.

The nature of the Fort Polk project itself, access to historical and
current data, and the authors' history with the project present an ideal
opportunity to answer these questions:

--- Are the ESPC electricity use and demand savings apparent in the
utility bills?

--- Have the ESPC savings persisted?

--- Are the ESPC cost savings apparent in the utility bills?

-   Why is utility bill analysis seldom used for M&V in ESPC projects?

-   What are the M&V best practices based on experience?

> --- What are the implications for agencies trying to reconcile ESPC
> savings and utility bills?

### 3.1 Fort Polk ESPC Project

In 1995 -- 1996, Fort Polk used an ESPC to complete a major energy
retrofit of its family housing units. An energy services company (ESCO)
converted space conditioning equipment in all 4003 of its family housing
units to geothermal heat pumps (GHPs). Original equipment consisted of
air-source heat pumps with electric water heaters in 81% of the
residences, and gas furnace/central air conditioner combinations with
gas water heaters in the remaining 19%. All of the gas water heaters
were replaced with electric water heaters, and in the majority of
residences, a desuperheater was installed with the GHP to supplement the
heating elements in the water heater. Other energy conservation measures
such as compact fluorescent lighting, low-flow shower heads, and some
insulation upgrades were installed at the same time.

### 3.2 ORNL's 1998 Evaluation of the Fort Polk Project

A detailed evaluation of the project, published in 1998, was carried out
by a team of Oak Ridge National Laboratory (ORNL) researchers led by the
authors (Hughes and Shonder 1998). For the evaluation, the team
collected data on electricity use at 15-minute intervals in family
housing for about one year before and one year after the retrofits (the
periods are approximate because of varying construction schedules in the
different housing areas). Based on this data, we estimated that in a
typical meteorological year (TMY), the project would result in annual
savings of 25.8 million kWh, and that summer peak electrical demand in
family housing would be reduced by an estimated 7.55 MW. These savings
correspond to 14% of the total base-wide pre-retrofit electricity use,
and 18% of post-wide summer peak demand.

### 3.3 Data Available for This Case Study

There are three main sources of data on Fort Polk's electrical energy
consumption: utility bills based on utility-maintained meters,
Army-maintained submeters for the housing areas, and the metering
equipment installed by ORNL for the original evaluation and reactivated
recently to address savings persistence.

The authors obtained records of Fort Polk's electric bills for the
121-month period from June 1993 to June 2003. Excluding the 18-month
construction period, the data set includes 21 months of pre-retrofit
data (June 1993 -- February 1995) and 82 months of post-retrofit data
(September 1996 -- June 2003). In addition to cost information, the
monthly bills provide the electrical consumption in kWh and the monthly
15-minute peak electrical demand in kW. The serving utility maintains
the electric meters and reads them remotely on the first day of each
month, therefore the billing periods correspond to the actual number of
days per month. The utility bills reflect all of the electricity used at
Fort Polk both for housing and non-housing loads.

Fort Polk actually receives two monthly utility bills, one each for the
North Fort and South Fort areas. In the body of this report, the Fort
Polk "totals" we refer to mean the sums from North and South Fort bills.

Fort Polk maintains submeters on electrical distribution feeders that
supply energy to the family housing areas. Data have been collected
manually from these meters monthly by Fort Polk personnel throughout the
121-month period from June 1993 to June 2003.

In 1995, ORNL installed data collection equipment on the housing feeder
submeters to collect electrical use data at 15-minute intervals. Use of
this equipment was discontinued after data needs to support the original
evaluation were satisfied. However, about 7 years later ORNL was able to
resume collection of 15-minute-interval data on 4 feeders to support an
investigation of savings persistence. During both the original and
recent data collection periods, ambient temperature and humidity
measurements were also made at 15-minute intervals at four locations
across the base.

The ESPC project reduced use of electricity and natural gas, but most of
the savings by far were associated with electricity. This study focuses
on electricity for the sake of simplicity and because all issues of
interest can be addressed in the context of electricity.

From a National Oceanic and Atmospheric Administration (NOAA) web site,
we obtained daily high and low temperatures at Fort Polk for the
121-month period from which we generated base-65°F heating and cooling
degree days for each month. Missing temperatures (about 150 out of 3682
days) were filled with data from our own monitoring equipment at the
site or with NOAA data from Lufkin, Texas, which is about 50 miles west
of Fort Polk.

Lufkin is the closest city to Fort Polk for which TMY data is available.
Based on thirty years of observations, TMY files provide hourly data on
ambient wet-bulb and dry-bulb temperatures, wind speed and direction,
and other meteorological phenomena for a typical year at the site. These
files drive building energy analysis programs such as TRNSYS, DOE-2,
BLAST and EnergyPlus. In making the original savings estimates for this
project, the ESCO used the Lufkin TMY file. In this study we also used
the Lufkin TMY data to normalize electricity use and demand to typical
weather values.

### 3.4 Resolution of Electricity Use Savings From Utility Bill Analysis

For this study, the authors conducted a utility bill analysis following
generally accepted practices (ASHRAE) to determine whether the reduction
in electricity use caused by the ESPC project is apparent in the utility
bills. (A detailed description of the analysis is provided in Appendix
A).

Analysis of utility bills for all of Fort Polk for about one year before
and one year after the retrofit yielded an estimate of savings in a TMY
of 24.3 million kWh, with a 95% confidence interval of ± 4.0 million
kWh. This result confirms the estimates of electricity savings published
in the original evaluation of the Fort Polk project (Hughes and Shonder
1998).

In the original study, we based our estimate of electricity savings on
15-minute-interval recordings from submeters on the electric
distribution feeders serving the housing areas for about one year before
and one year after the retrofit. That data indicated annual electricity
savings from the retrofit as 25.8 million kWh in a typical year.

One can presume that the 25.8 million kWh is the true mean of savings,
because this value is based on electricity consumption data for the
housing areas only (about 80 million kWh/yr before the retrofit), rather
than on the total Fort Polk pre-retrofit electricity consumption of
roughly 190 million kWh/yr. The 1.5 million kWh difference between the
means of the two estimates could be explained by an annual increase of
only 0.55% in the 110 million kWh/yr of non-housing electricity use over
the 2.5 years between the mid-points of the pre- and post-retrofit
periods used for the utility bill analysis. Nevertheless, the mean
savings value of 24.3 million kWh is used in the balance of this report
because it derives directly from the regression equations for the
utility bill analysis, and the authors desired to use the regression
analysis in order to illustrate important points.

![](media/image1.wmf){width="5.995138888888889in"
height="4.095138888888889in"}Figures 1 and 2 graph some of the data for
the utility bill analysis. Figure 1 displays actual monthly kWh from the
utility bills for June 1993 through June 2003. The bar graph shows that
energy use peaks in the summer months, falls to a minimum in the
shoulder months, and rises to a lower peak during the winter months. The
highest peaks in the post-retrofit period are lower than those in the
pre-retrofit and construction periods, but beyond this, not much
information can be discerned from the graph.

**Figure 1. Total billed electricity use for Fort Polk, June 1993 --
June 2003. Usage peaks are lower after the construction period than
before, but we can conclude very little other than that from this
graph.**

A clearer picture of the trend in electricity usage is obtained in
Figure 2, which shows weather-normalized (TMY) data, with each bar
representing the weather-normalized annual electricity use for the
12-month period ending with the plotted month. In this graph the 24.3
million kWh annual savings attributable to the ESPC project is clearly
visible. Beginning in the construction period, annual consumption began
to drop, and then was relatively stable for the next 24 months. In
August 1998 consumption began to rise, then fell again, but in July 2000
began a steady, steep climb, which continued until February 2003.
Section 3.6 of this report discusses this increase in consumption, along
with analysis results indicating that the Fort Polk ESPC project is
continuing to generate the expected savings.

![](media/image2.wmf){width="5.995138888888889in"
height="4.095138888888889in"} **Figure 2. Fort Polk's total billed
electricity use, normalized to TMY. Each bar represents
weather-normalized annual electricity use for the 12-month period ending
with the graphed month.**

### 3.5 Resolution of Reduced Peak Demand From Utility Bill Analysis

Our analysis showed that reductions in peak electricity demand caused by
the ESPC project, as well as usage reductions, are apparent in the
utility bills. (See Appendix B for a detailed description of the demand
reduction analysis.) Based on the utility bills for all of Fort Polk, a
demand reduction of 7.27 MW from the total Fort Polk peak demand of 42
MW is estimated, with a 95% confidence interval of ± 3.1 MW. Again, this
confirms our estimate of 7.55 MW in the original evaluation, which was
based on data from the housing submeters only. The 0.25 MW difference in
mean values of the two estimates could easily be explained by an annual
increase of only 0.42% in the roughly 24.4 MW of non-housing electrical
demand over the 2.5 years between the mid-points of the pre- and
post-retrofit periods.

Although we presume that 7.55 MW is the true mean of savings, the value
of 7.27 MW is used in this report so the regression analysis can be used
in order to illustrate important points.

![](media/image3.wmf){width="5.995275590551181in"
height="4.095275590551181in"}The trend in peak demand can be seen in
Figure 3, which plots the weather-normalized total Fort Polk peak
demand. Each bar represents the highest monthly peak demand (always in
summer) from the 12-month period ending with the plotted month. The
project's 7.27 MW reduction is clearly visible, but this graph gives us
no information about, for example, the increase in peak demand in 2001.

**Figure 3. Weather-normalized Fort Polk peak demand. Each column shows
highest peak demand (always in summer) from the 12-month period ending
with the plotted month.**

### 3.6 Persistence of Savings

The energy conservation measures installed in 1995 -- 1996 had a
definite impact on electricity use and demand at Fort Polk, shown
clearly in Figures 2 and 3. But starting in about August of 2000,
overall Fort Polk electricity use began a steady rise, and by June 2003
the weather-normalized annual consumption was half way back to
pre-retrofit levels. This section addresses the question of whether the
ESPC project savings are falling short by 50% or post-wide loads have
simply grown. Our analysis of submetered data from electrical feeders to
Fort Polk family housing indicates that electricity use in those areas
has increased very little since the ESPC project was completed. Our
analysis is reviewed below and discussed in detail in Appendix C.

The submeters on the feeders to the housing areas are read manually by
Army staff once per month to track family housing electricity use.
Figure 4 presents the sum of these meter readings for the same period
for which we analyzed Fort Polk's utility bills, June 1993 to June 2003.
As with electricity use for the entire facility, the raw data is not
very informative.

![](media/image4.wmf){width="5.995138888888889in"
height="4.095138888888889in"}

**Figure 4. Monthly sums of readings from the 16 electric submeters in
the family housing areas.**

In our original study of the Fort Polk retrofit project, we attached
data loggers to 14 of the 16 family housing electric submeters in order
to read them electronically at 15-minute intervals. All 14 feeders
exhibited electricity use patterns characteristic of pure housing loads.
In July 2003, we restarted automated data collection on 11 of our
original 14 feeders. We were able to verify proper operation of the
automated data collection equipment on four feeders (Feeders 3, 4, 6,
and 16) and that the loads on these feeders were from housing only. The
data from the four feeders, for both the 1996 -- 1997 and 2003 -- 2004
periods are plotted in Figure 5. These feeders represent about 12% of
the 5 million square feet of family housing at Fort Polk, a sufficient
sample size for representing all housing on the post.

![](media/image5.wmf){width="7.25in" height="4.9319444444444445in"}

**Figure 5. Electricity use on four submetered feeders to Fort Polk
family housing appears to have changed very little, based on comparison
of 1996-1997 and 2003 -- 2004 data sets.**

For each of the four feeders with usable data, we fit daily electricity
use data from 1996-1997 and 2003-2004 to a five-parameter function of
daily average temperature, using the technique described in our original
evaluation of the Fort Polk project (Hughes and Shonder 1998). This
regression analysis allowed us to predict annual energy use for a
typical year. If there have been no increases in energy use, then the
TMY predictions from the two data sets should be roughly the same. Table
1 shows the results of the regression analysis of 1996-1997 and
2003-2004 electricity use data for the four feeders.

The analysis indicates that energy use on some of the feeders has
decreased, and on some it has increased. Some of the changes are
undoubtedly due to changes in occupancy, for which we have no
information. Overall, energy use for the four feeders has increased by
2.2% over 7 years. This is an annual rate of increase of just 0.31%.
Based on this analysis of a large sample of Fort Polk's housing, we
conclude that there have been only small increases in electricity use in
Fort Polk's family housing since the retrofits were installed. These
increases are smaller than housing load growth due to plug load
increases predicted in the 1998 evaluation, based on national averages
at the time for plug load growth.

Since in ESPCs the ESCO is not accountable for plug load growth, the
mean values of 24.3 million kWh and 7.27 MW are used for ESPC project
savings in a TMY for all analysis years in the rest of this report.

Table 1. Annual electricity use predicted for a TMY by regression
analysis of 1996-1997 and 2003-2004 electricity use data for four
submetered Fort Polk family housing feeders.

  -------------- ----------------- -------------------------- -------------
  Feeder         TMY energy use    TMY energy use based on    Percentage
                 based on          2003-2004 data (kWh)       change
                 1996-1997 data                               
                 (kWh)                                        

  3              977,428           883,000 ± 9,400            --9.7%

  4              176,601           167,000 ± 2,100            --5.5%

  6              999,035           1,038,000 ± 9,400          3.9%

  16             4,763,891         4,980,000 ± 27,000         4.6%

  Total          6,916,955         7,070,000 ± 39,000         2.2%
  -------------- ----------------- -------------------------- -------------

### 3.7 Resolution of ESPC Cost Savings From Utility Bill Analysis

This study has shown that electricity use and demand savings from the
ESPC project are apparent in Fort Polk's weather-normalized utility
bills, when comparing periods immediately before and after the retrofit.
Analysis of submetered data also indicates that these savings appear to
have held up over the 7 years since the retrofit. We also analyzed Fort
Polk's utility bills to estimate true cost savings attributable to the
ESPC project. The analysis was structured to value the savings at their
true marginal rates, in accordance with the electricity rate structures.
The utility bills capture the net effect of all factors influencing
electricity use and demand, including the ESPC project. Total billed
Fort Polk electricity costs are shown in Figure 6.

Monthly total billed kWh and kW define the energy baseline for this
"actual in arrears" analysis, performed after the billing periods when
actual billed kWh and kW are known. The electricity cost baseline was
defined mathematically by implementing the electricity rate structures
in a spreadsheet and verified by entering the actual monthly kWh and kW
values to reproduce the billed electricity costs.

![](media/image6.wmf){width="5.5in" height="3.7597222222222224in"}Adding
kWh and kW savings to billed values yielded monthly kWh and kW that
would have been billed if the ESPC project had never been implemented.
These values were run through the electric rate schedules to calculate
electricity costs that Fort Polk would have incurred if the ESPC project
had not been done. Subtracting the actual billed costs yields the cost
savings attributable to the ESPC project.

**Figure 6. Total billed Fort Polk electricity costs.**

The relationships used to calculate pre- and post-retrofit kWh and kW
from weather data, and to arrive at savings by subtracting post-retrofit
from pre-retrofit values, are described in detail in Appendices A and B.
Actual weather data (not TMY) for the months during the analysis period
were used to yield realistic energy and demand values, and actual
electricity rate structures in effect during the period were the basis
for the cost estimates.

The monthly electricity cost savings to Fort Polk resulting from the
ESPC, as estimated from the analysis described above, are presented in
Figure 7. These monthly values are the basis for the annual savings
presented in Figure 8, where the bars represent the sum of cost savings
for the year preceding (and including) the plotted month. The first
annual savings is shown in the first month when all 12 months in the
period represented occurred after the ESPC project was fully installed.
The average annual electricity cost savings from the ESPC over the time
interval shown in Figure 8 is \$1,450,000.

Figure 9 shows the actual total annual electricity cost along with total
electricity costs that would have been incurred if the ESPC project had
never been implemented.

![](media/image7.wmf){width="5.75in" height="3.9256944444444444in"}

> **Figure 7. Electricity cost savings to Fort Polk resulting from the
> ESPC project, as estimated by analyzing Fort Polk's utility bills.**
>
> ![](media/image8.wmf){width="5.625in" height="3.8402777777777777in"}
> **Figure 8. Annual cost savings to Fort Polk from the ESPC project
> expressed as the sum of cost savings for the 12-month periods ending
> with the plotted months.**

![](media/image9.wmf){width="5.995138888888889in"
height="4.095138888888889in"}

> **Figure 9. Actual total annual Fort Polk electricity costs and costs
> that would have been incurred if the ESPC project had never been
> implemented.**

## 4. M&V Lessons Learned

The Fort Polk project was implemented relatively early in the history of
federal ESPC activity. The project was awarded based on a site-specific
procurement in 1994, before the first ESPC umbrella contracts
(indefinite-delivery indefinite-quantity, or IDIQ contracts) became
available for agency use. Since the advent of standardized and
streamlined ESPC umbrella contracts in 1998, federal buildings have been
improved with \$1.5 billion worth of ESPC investments, and M&V practices
have been significantly refined through experience. The M&V guidelines
used by today's performance contracting industry, such as IPMVP, FEMP
M&V Guidelines, and ASHRAE Guideline 14, were all developed and/or
refined between 1998 and 2002.

If the Fort Polk project were under development today, it is very
unlikely that its M&V would be based on utility bill analysis (referred
to as M&V Option C by IPMVP and FEMP), even though, because of the
project's size and comprehensiveness, the savings are apparent in the
utility bills for the period immediately after installation. Federal
ESPCs are long-term contracts, 15 years on average, and M&V is intended
to verify savings annually over the entire period, not just immediately
after installation. This section highlights some of the M&V best
practices developed over the years that enable savings to be verified at
modest expense, to an acceptable degree of certainty, and without
allocating unreasonable risks to the project provider, which would
inevitably burden projects with premium pricing and financing costs as
compensation for bearing those risks.

### 4.1 Risk and Responsibility Matrix

Delivery order awards under the DOE Super ESPC program include a risk
and responsibility matrix that documents the assignment of risks and
responsibilities to the ESCO and the government in the form of mutually
agreeable assumptions. The matrix extends beyond just the M&V approach
used to verify guaranteed savings.

Note that there are distinctive meanings assigned to categories of
savings in ESPCs, both in common usage and for the purposes of this
study:

> Estimated savings --- the ESCO's prediction of the annual savings the
> project is to generate, documented in the financial schedules of the
> delivery order (both in terms of energy units and dollars)
>
> Guaranteed savings --- The contractually guaranteed annual savings (in
> dollars) that the project must deliver, which must exceed annual
> payments to the ESCO
>
> Actual savings --- For the purposes of this study, savings (energy
> units or dollars) calculated in arrears based on analysis of data from
> monthly utility bills, monthly submeter readings, or
> 15-minute-interval data loggers.
>
> Contracted savings --- Savings (energy units or dollars) calculated
> for the M&V annual report following the algorithms defined in the M&V
> plan (part of the ESPC contract), which generally includes practical
> stipulations, agreements, and assumptions. In other words, contracted
> savings are those calculated in accordance with the contract.

The goal in assigning M&V-related risks and responsibilities is to
achieve a cost-effective ESPC project where the ESCO can be held
accountable for contracted savings equaling or exceeding the guarantee
each year, and contracted and actual cost savings remain reasonably
proximate to each other annually and converge over the contract term,
ensuring that savings exceed payments. Highlighted here are several
common, and indeed recommended, M&V-related risk and responsibility
assignments, which nonetheless potentially de-couple contracted and
actual cost savings in any given year.

It is important that any mutually agreed upon assumptions be carefully
selected. Risk and responsibility assignments related to the performance
of energy conservation measures (ECMs), weather, energy loads, and
energy price escalation are highlighted in the following sections.

### 4.2 ECM Performance 

Directly or through use of subcontractors, the ESCO designs the ECM
application, selects the equipment, installs the ECM, and commissions
the ECM. Therefore, the ESCO is in a position to bear the risk of fixed
ECM prices and guaranteed performance, and is assigned these
responsibilities under the contract. How does assigning the risk and
responsibility for ECM performance to the ESCO relate to the electric
utility bill? Essentially, the ESCO's equipment is expected to perform
so that specified levels of electricity use and demand savings are
achieved in comparison to the pre-project baseline under the mutually
agreed upon conditions for weather and energy load.

### 4.3 Weather 

Neither party can control the weather, so the government and ESCO
generally agree (stipulate) that typical weather (TMY data) will be used
for the purposes of calculating contracted electricity use savings in
kWh and demand savings in kW. This agreement clarifies the ESCO's
performance responsibilities in a practical manner, rather than forcing
the private-sector partner to shoulder an unmanageable risk which would
result in premium pricing and financing costs.

The consequence of using typical weather in calculating contracted
savings is that contracted and actual savings may differ in any given
year. These differences are summarized for Fort Polk in Table 2. Shown
are the typical year and actual kWh and kW savings for the post-retrofit
12-month periods (September through the following August) from 1996-97
to 2001-02.

The energy savings for a typical year at the site have been calculated
as 24.3 million kWh, which is the "contracted" savings. Note, however,
that the "actual" savings differ from "contracted" savings because of
weather variation in any given year. Also the "actual" savings at Fort
Polk differ from "contracted" savings because about 10% of the housing
was demolished beginning in 1998. Table 2 shows that if there had been
no housing demolitions, the average annual "actual" savings over the
1996-2002 period would have been 24.9 million kWh, slightly (about 2.5%)
higher than the "contracted" savings for the typical year. In no year
were the actual savings (without demolitions) more than 8% higher or
lower than the contracted savings.

A similar result holds for peak summer demand savings. Since these are
also affected by the demolitions beginning in 1998, Table 2 presents the
savings as if the demolitions had not occurred. We have already shown
that in a typical year, the peak summer demand savings would be 7.27 MW,
which corresponds to the "contracted" demand savings. In the
post-retrofit period, "actual" summer peak demand savings (adjusted to
take out the effect of the demolitions) is sometimes higher and
sometimes lower than the contracted amount depending on weather
conditions, but the variation is never more than 18% higher or lower
than the contracted amount. Over the period 1996-2002, the average
summer peak demand savings is 7.71 MW, about 6% higher than the
contracted amount.

Typically an ESCO would guarantee only a fraction of the energy and
demand savings calculated for a TMY. In this case we have the fraction
to be 95%. For both energy and demand, Table 2 presents the guaranteed
savings, the difference between the actual savings and the guaranteed
savings, and the cumulative difference between actual and guaranteed
savings.

Table 2. Fort Polk Savings --- Actual and Normalized to TMY

  -------------------------- ------------- --------- --------- --------- ----------- --------- ---------
  Post-retrofit year         Typical year  1996-97   1997-98   1998-99   1999-2000   2000-01   2001-02

                             Annual                                                            
                             electricity                                                       
                             use savings                                                       
                             (million kWh)                                                     

  Actual savings (actual                   23.6      26.1      24.4      23.7        23.5      22.3
  weather)                                                                                     

  Actual savings if no                     23.6      26.1      25.3      24.6        25.3      24.5
  demolitions                                                                                  

  Contracted savings         24.3          24.3      24.3      24.3      24.3        24.3      24.3
  (typical weather)                                                                            

  Actual minus contracted if               (0.7)     1.8       1.0       0.3         1.0       0.2
  no demolitions                                                                               

  Cumulative actual minus                  (0.7)     1.1       2.1       2.4         3.4       3.6
  contracted without                                                                           
  demolitions                                                                                  

  Guaranteed savings         23.1          23.1      23.1      23.1      23.1        23.1      23.1

  Actual minus guaranteed                  0.5       3         1.3       0.6         0.4       (0.8)
  without demolitions                                                                          

  Cumulative actual minus                  0.5       3.5       4.8       5.4         5.8       5
  guaranteed without                                                                           
  demolitions                                                                                  

                             Summer peak                                                       
                             electrical                                                        
                             demand                                                            
                             savings (peak                                                     
                             month MW)                                                         

  Actual savings (actual                   6.45      7.89      7.63      8.29        8.29      6.19
  weather)                                                                                     

  Actual savings if no                     6.45      7.89      7.89      8.61        8.61      6.81
  demolitions                                                                                  

  Contracted savings         7.27          7.27      7.27      7.27      7.27        7.27      7.27
  (typical weather)                                                                            

  Actual minus contracted if               (0.82)    0.62      0.62      1.34        1.34      (0.46)
  no demolitions                                                                               

  Cumulative actual minus                  (0.82)    (0.20)    0.42      1.76        3.10      2.64
  contracted if no                                                                             
  demolitions                                                                                  

  Guaranteed savings         6.91          6.91      6.91      6.91      6.91        6.91      6.91

  Actual minus guaranteed                  (0.46)    0.98      0.72      1.38        1.38      (0.72)
  without demolitions                                                                          

  Cumulative actual minus                  (0.46)    0.53      1.25      2.63        4.02      3.30
  guaranteed without                                                                           
  demolitions                                                                                  
  -------------------------- ------------- --------- --------- --------- ----------- --------- ---------

### 4.4 Energy Loads

Generally the government controls how many hours per day a building is
used, which is a prime contributor to energy load, so in a typical
retrofit isolation M&V approach, operating hours are measured during
baseline development, and a reasonable value is agreed on (stipulated)
as the basis for calculating savings. In the case of Fort Polk family
housing, in a utility bill M&V approach where the "utility meter" in
reality refers to only the submetering of electricity into the housing
areas, the analogous stipulations might be housing occupancy rate or
total population of the housing area, with some allowance for plug load
growth over time. Using data from the actual utility meters for M&V for
the Fort Polk housing project adds the complication of the non-housing
loads.

At Fort Polk, prior to the retrofit, family housing accounted for about
42% of the total base-wide electricity use of \~190 million kWh per
year. Obviously the ESCO cannot be held accountable for the growth in
the \~110 million kWh per year of non-housing load at the post. But even
in family housing, it would not be cost-effective to hold the ESCO
accountable for increases in energy use due to factors such as occupancy
rate and plug load growth.

Figure 10 helps us to visualize the magnitude of unaccounted for
electric load variation that occurs at a large post like Fort Polk. The
bars in Figure 10 show the weather-normalized total electricity use in
kWh by year. For post-retrofit years, higher bars indicate the estimated
total Fort Polk electricity use that would have occurred without the
improvements implemented under the ESPC project. Note that the
pre-retrofit energy use is comparable to the higher of the two 1996-97
post-retrofit bars, as would be expected, with the shorter 1996-97 bar
representing the billed electricity use after the ESPC project was
implemented.

![](media/image10.wmf){width="5.996527777777778in"
height="4.107638888888889in"} **Figure 10. Weather-normalized total
annual electricity use at Fort Polk, actual and estimated for if the
ESPC project had not been implemented.**

If post-retrofit electricity use were the same in all years, we might
conclude that Fort Polk's troop levels and war fighting mission were
stable and that new electric load connections were offset by other load
reductions, but they clearly are not. Compared to 1996-97, these loads
not related to the ESPC project are 12 million kWh higher in 2000-2001
and 4 million kWh lower in 1999-2000. The post is fully responsible for
these large variations in load, but without submetering or an energy
monitoring/management system, there is no way to tell where the new load
comes from. Furthermore, these large changes in load could have an
impact on the how the ESPC project's electricity use and demand savings
are valued as cost savings.

### 4.5 Energy Price Escalation 

It is common practice for the government and the ESCO to agree, for the
purpose of calculating contracted electricity cost savings, that current
electric rates will escalate at a constant rate. As in the case of
weather and load stipulations, this agreement clarifies the ESCO's
performance responsibilities in a practical manner that avoids burdening
the project with premium pricing and financing costs in exchange for
loading an unmanageable risk onto the private-sector partner.

The experience at Fort Polk illustrates why agreement to use constant
electricity price escalation factors is practical and common. During the
121-month period for which utility bills are available, Fort Polk has
been billed for electricity under four different rate structures. Within
these different rate structures, in addition to the valuation of
electrical usage and demand, is the fuel adjustment charge, which can
vary from month to month with the price of the utility's power
generation fuel and accounts for about 35% of Fort Polk's total
electricity bill, on average.

Further, changes in the weather and Fort Polk's war fighting mission
affect electric loads and the valuation of the ESPC project's savings,
which is at the margin when processed through the actual rate
structures. Obviously neither the ESCO nor Fort Polk can predict
weather, mission, or electricity rates. If the ESCO were required to
guarantee electricity cost savings based on whatever the weather,
mission, or rates turned out to be, excessive hedge, insurance or other
costs would have to be built into the price of the project, and Fort
Polk would end up paying them anyway.

Nevertheless, it is true that agreeing to the combination of typical
weather and constant energy price escalation for the calculation of
contracted electrical cost savings does have the consequence that
contracted and actual cost savings may differ in any given year. These
differences are summarized for Fort Polk in Table 3. Shown are the
typical year and actual cost savings for the 12-month periods (September
through August) from 1996-97 to 2001-02. Also shown are the guaranteed
cost savings (calculated by assuming that 95% of the energy and demand
savings are achieved, and calculating the resulting savings based on the
rate structure in effect at the time of contract award).

The first row in Table 3 shows actual energy cost savings per year. As
with energy savings, cost savings are complicated by the fact that 10%
of the housing was demolished beginning in 1998. Therefore in the second
row we have estimated what the savings would have been without the
demolitions. We have previously shown that the annual savings in a
typical year, based on the electricity rate that applied in the
pre-retrofit period, is \$1.55 million. Now, suppose Fort Polk and the
ESCO agree to escalate these cost savings at 0.5% per year. The third
row in Table 3 shows what the annual contracted savings would be through
2002. The fourth row shows the difference between actual savings
(without demolitions) and contracted savings. For the first three
post-retrofit years, the contracted savings are higher than actual
savings, but in 1999-2000 and again in 2000-2001, actual savings are
higher than contracted savings. By the end of 2001, the cumulative net
difference is zero. A drop in rates in 2001 brings actual savings below
contracted savings, but presumably in the long run things will even out.

Table 3. Potential for actual weather and price escalation assumptions\
to cause differences between actual and contracted savings

  -------------------------- --------------- --------- --------- --------- ----------- --------- ---------
  Annual electricity use                                                                         
  cost savings (\$ millions)                                                                     

                             Post-retrofit                                                       
                             year                                                                

                             Typical year    1996-97   1997-98   1998-99   1999-2000   2000-01   2001-02

  Actual savings (actual                     1.51      1.44      1.42      1.56        1.71      1.06
  weather and actual rates)                                                                      

  Actual savings if no                       1.51      1.44      1.46      1.62        1.84      1.16
  demolitions                                                                                    

  Contracted savings (TMY    1.55            1.56      1.57      1.57      1.58        1.59      1.60
  and price escalation of                                                                        
  0.5% per year)                                                                                 

  Actual minus contracted if                 (0.05)    (0.13)    (0.11)    0.04        0.25      (0.44)
  no demolitions                                                                                 

  Cumulative actual minus                    (0.05)    (0.17)    (0.29)    (0.25)      0.00      (0.43)
  contracted without                                                                             
  demolitions                                                                                    

  Guaranteed savings (ESCO   1.47            1.48      1.48      1.49      1.50        1.51      1.51
  payment)                                                                                       

  Actual savings minus ESCO                  0.03      (0.04)    (0.03)    0.12        0.33      (0.35)
  payment without                                                                                
  demolitions                                                                                    

  Cumulative actual savings                  0.03      (0.01)    (0.04)    0.08        0.41      0.05
  minus ESCO payment without                                                                     
  demolitions                                                                                    
  -------------------------- --------------- --------- --------- --------- ----------- --------- ---------

[]{.mark}

Figure 11 compares what Fort Polk's annual utility payment would have
been without the project: the post's actual utility bill, plus actual
savings from Table 3; to what they pay annually with the ESPC: the
post's actual utility bill, plus a payment to the ESCO. Note that in
2000-2001, the sum of payments (utility bill plus ESCO payment) is lower
than the utility bill Fort Polk would have paid without the project. On
the other hand, in 2001-2002 the sum of payments is higher than what the
utility bill would have been without the project.

The difference comes about because of the weather: 2000-2001 is a
relatively extreme year (as seen by the high utility bill compared with
other years) while 2001-2002 is a relatively mild year. As shown in
Appendix A, energy savings increase with increasing heating and cooling
degree days, meaning that savings are larger in more extreme years, and
smaller in milder years. Since the ESCO payment, or guarantee level, is
based on 95% of estimated savings in an average year, then indeed, in
some years the ESCO payment will exceed the actual savings. But this
will happen in years when the climate is milder than average. Since the
overall utility bill will be lower in those years, Fort Polk will have
room in its budget to meet its obligation to the ESCO.

![](media/image11.wmf){width="5.996527777777778in"
height="4.107638888888889in"} **Figure 11. Comparison of what payments
would have been without the project (total height of white bar) and with
the project (total height ofgray bar) with guaranteed savings escalated
at 0.5%. The height of the white bar is equal to the annual utility bill
with the project, plus the actual savings. The height of the gray bar is
equal to the annual utility bill with the project, plus the ESCO payment
(equal to the guaranteed savings).**

[]{.mark}

Table 3 and Figure 11 assume that contracted savings are based on an
electricity price escalation rate of 0.5% per year. Another way to
estimate the escalation rate of electrical savings is to use the latest
energy price forecasts from the U.S. Department of Energy's Energy
Information Administration (DOE/EIA) and the National Institute of
Standards and Technology (NIST), which are published annually. When the
Fort Polk ESPC was signed, the Annual Supplement was forecasting an
escalation rate of about 3.9% per year for commercial electricity prices
(Lippiat 1992). As shown in Table 4, these figures significantly
overestimated the true electricity price escalation that occurred from
1995 through 2001. If the ESCO guarantees only 95% of the savings,
payments to the ESCO exceed the actual savings in every year except
2000-2001 but over the five year period, the cumulative difference is
more than one million, meaning that had DOE/EIA escalation rates been
used, by the fifth year payments to the ESCO would have exceeded actual
savings by more than one million dollars.

Figure 12 shows the annual payments with the project, and without the
project. For every year except 2000-2001, payments with the project --
i.e., the utility bill plus the ESCO payment -- exceed what the utility
bill would have been had the project not been implemented. An obvious
conclusion is that escalation rates have a significant impact on project
cash flow, and therefore should be chosen with care.

![](media/image12.wmf){width="5.996527777777778in"
height="4.107638888888889in"} **Figure 12. Comparison of what payments
would have been without the project (total height of white bar) and with
the project (total height ofgray bar) with guaranteed savings escalated
at DOE/EIA escalation rates current for 1993. The height of the white
bar is equal to the annual utility bill with the project, plus the
actual savings. The height of the gray bar is equal to the annual
utility bill with the project, plus the ESCO payment (equal to the
guaranteed savings). Note that in this case, the payments with the
project exceed the payments without the project for evry year except the
fifth (2000-2001).**

Table 4. Potential for actual weather and EIA energy price escalation
assumptions\
to cause differences between actual and contracted savings

  -------------------------- --------------- --------- --------- --------- ----------- --------- ---------
  Annual electricity use                                                                         
  cost savings (\$ millions)                                                                     

                             Post-retrofit                                                       
                             year                                                                

                             Typical year    1996-97   1997-98   1998-99   1999-2000   2000-01   2001-02

  Actual savings (actual                     1.51      1.44      1.42      1.56        1.71      1.06
  weather and price                                                                              
  escalation based on actual                                                                     
  rates)                                                                                         

  Actual savings if no                       1.51      1.44      1.46      1.62        1.83      1.16
  demolitions                                                                                    

  Contracted savings         1.55            1.60      1.66      1.72      1.80        1.89      1.97
  (typical weather and                                                                           
  EIA-forecasted price                                                                           
  escalation)                                                                                    

  Actual minus contracted if                 (0.09)    (0.22)    (0.26)    (0.18)      (0.06)    (0.81)
  no demolitions                                                                                 

  Cumulative actual minus                    (0.09)    (0.31)    (0.57)    (0.75)      (0.80)    (1.61)
  contracted without                                                                             
  demolitions                                                                                    

  Guaranteed savings (ESCO   1.47            1.52      1.57      1.63      1.71        1.79      1.87
  payment)                                                                                       

  Actual savings minus                       (0.01)    (0.13)    (0.17)    (0.09)      0.04      (0.71)
  guaranteed savings if no                                                                       
  demolitions                                                                                    

  Cumulative actual savings                  (0.01)    (0.14)    (0.31)    (0.40)      (0.36)    (1.07)
  minus guaranteed savings                                                                       
  if no demolitions                                                                              
  -------------------------- --------------- --------- --------- --------- ----------- --------- ---------

### 4.6 Summary of M&V Best Practices

Because of the difficulty (or impossibility) of discerning the reasons
for changes in energy usage and peak demand reflected in utility bills,
especially over the time spanned by many ESPC contracts, it is very
unusual for ESPC project M&V to be based on utility bill analysis (M&V
Option C). The ECM-isolation methods (M&V Options A and B) are by far
the most commonly used, with some use of calibrated models (Option D) in
special circumstances, such as GHP projects where detailed models must
be developed anyway to properly size the ground heat exchanger.

Whether using Option A, B, or D to verify ESPC project savings,
generally the ESCO's M&V-related risk and responsibility assignments
extend only to the performance of the ECMs. Energy savings performance
is generally translated into contracted cost savings assuming typical
weather, pre-retrofit baseline load levels for the non-project-related
loads, and stipulated energy cost escalation rates. The ESCO's
contractual guarantees are met if contracted cost savings exceed the
guarantee level each year.

According to FEMP guidance, the responsibility matrix and M&V plan are
to document the agreements between the agency and ESCO on assumptions
and allocations of responsibilities and risks. Logic and
cost-effectiveness drive the decisions for each project, and common
sense dictates that the party that can control the risk should be
responsible for it.

Some patterns have emerged in the agreements reached in federal ESPC
projects that are recognized as best practices and are generally
followed, unless unusual circumstances warrant different arrangements:

> • The ESCO designs the ECM application, selects the equipment, and
> installs and commissions the ECM, and is therefore held responsible
> for the ECM's fixed price and performance.
>
> • Savings are normalized to typical weather (TMY).
>
> • Since the Government is responsible for mission-driven load
> fluctuations, non-project-related loads, for the purpose of
> calculating savings with marginal rate analysis, are set at
> pre-retrofit baseline levels.
>
> • Facility operating hours are stipulated for the purpose of
> determining savings.
>
> • Current energy rate structures, and the marginal prices used to
> value ESPC savings, which are derived from them, apply at the time of
> award; and agreed upon escalation rates apply for the rest of the
> contract term.

Engineering algorithms defined in the M&V plan consider all factors
(performance, weather, non-project-related load, operating hours, and
energy escalation rates) and, for the annual M&V report, are used to
determine contracted ECM cost savings as the difference between baseline
and post-retrofit values. While it is true the contracted ECM cost
savings may differ from actual ECM cost savings in a given year
(because, for example, the weather was not typical or energy rates were
not as forecasted), the fact remains that the government has remedies
against the ESCO if the verified contracted cost savings do not match or
exceed the guaranteed cost savings each year. When appropriate
assumptions and choices are made, annual contracted and actual savings
will be reasonably similar, and over the contract term contracted and
actual savings tend to converge because weather and operating hours and
other stipulated factors conform to the average over the long term.

According to FEMP guidance and in practice, the ESCO guarantees that the
project's bundle of ECMs will result in a specified level of annual
contracted cost savings, and the risk and responsibility matrix and the
M&V plan in the project's delivery order award document the agreed-upon
assumptions and allocations of responsibilities and risks. The
alternative to using simplifying assumptions for the purpose of
calculating savings --- having the ESCO take the risk that factors such
as the weather, future energy rates, and the government's own operating
hours and non-project-related loads will affect savings --- would be a
poor and expensive choice for the government.

## 5. Implications for Agencies Deciding to Reconcile ESPC Savings and Utility Bills

Agencies that decide to correlate the impact of individual ESPC projects
directly to changes in their utility bills, either at the site or agency
level, will probably need to consider implementing a system for tracking
what their utility bills would have been had the energy efficiency
projects never been implemented. Since most agencies have tracking
systems for their actual energy use and costs across all agency sites,
perhaps these same systems could be enhanced for this purpose.

Agencies would need to decide whether to track contracted savings,
actual savings, or both. While the calculation of contracted savings is
really the only practical option for annual guarantee verification in a
long-term ESPC contract, it is possible to estimate the actual savings
annually in arrears, for example, as part of the annual M&V report.
Going back and bringing all past projects into the system on an actual
savings basis may be a significant effort. Going forward, requiring that
both contracted and actual savings be reported by the ESCO in the annual
M&V report would be one option, and use of a third party for this
purpose would be another.

Contracted and actual savings are reasonably proximate to each other
annually, and converge over the contract term, if appropriate
assumptions and choices are made during project development. This being
the case, one could question the value of the extra effort required to
calculate actual savings. Potential benefits of going the extra mile
would be to provide better guidance on "appropriate assumptions and
choices" to use during project development, and to monitor related
issues such as utility rate schedule changes that appear motivated by
the desire to recover lost revenues as a result of these projects.

# 6. Conclusions

This study shows that under the right circumstances, analysis of utility
bills can resolve savings from an ESPC project. (Details of our analysis
are given in this report's appendices.) However, such circumstances are
rare, and more cost-effective and appropriate methods of verifying
savings are widely recommended.

Our analysis of Fort Polk's utility bills unequivocally confirms and
quantifies the savings resulting from the GHP retrofit accomplished at
the facility in 1995 -- 1996 under an ESPC. Using utility bills from 12
months immediately before and after the retrofits, we show that the
annual electricity savings were 24.3 ± 4.0 million kWh (a 95% confidence
interval) in the year after the retrofit, and that the utility bills
predict peak summer demand savings of 7.27 ± 3.1 MW. These results agree
with our 1998 evaluation of the project (Hughes and Shonder 1998).

We were also able to resolve cost savings from analysis of Fort Polk's
utility bills. Using actual weather data for the months during the
analysis period to yield realistic energy and demand values, and actual
electricity rate structures in effect during the period, we estimated
annual electricity cost savings from the ESPC to be \$1,450,000.

Whether utility bill data can be used to discern persistence of
electricity use and demand savings is another question. Fort Polk's
utility bills indicate that the facility's total annual electricity use
increased by about 13 million kWh, or about half of the ESPC project's
savings, between 1997 and 2003. Analysis of data from four feeders
serving about 12% of family housing --- a large sample for such an
analysis --- shows that during this period electricity use in family
housing increased about 2.2%, or 0.31% per year, which is far less than
the increase due to plug load growth that we predicted in 1998. The
increase in electricity use in family housing accounts for about 1
million of Fort Polk's total 13 million kWh increase since 1996,
indicating that savings due to the GHP retrofits have persisted and the
majority of the increase in electricity use has occurred outside family
housing.

Utility bill analysis is seldom used for M&V in federal ESPC projects
because other methods of verifying savings are more practical,
cost-effective, and reliable. FEMP M&V guidelines (FEMP 2002) recommend
against using simple utility bill analysis for federal ESPC projects.
More complex regression modeling techniques may be appropriate,
according to requirements specified by both ASHRAE and FEMP guidelines,
where predicted savings are greater than about 10 to 20% of the site's
energy use at the meter on a monthly basis (p. 164), and the site has at
least 12 and preferably 24 months of pre-installation data to calculate
a baseline model, and at least 9 and preferably 12 months of
post-installation data to calculate first-year savings.

As we have shown with our Fort Polk case study, given a project of
suitable scale in relation to the total facility energy use and
sufficient data, utility bill analysis can resolve savings when
comparing the periods immediately before and after the retrofits.
However, factors that affect energy use change constantly, and those
changes compound over the years. After the first year or two of the ESPC
term utility bill analysis is inadequate for verifying savings because
the method cannot be used to effectively distinguish between the
performance of the energy conservation measures installed in the energy
project and the effects of factors such as operating schedules,
occupancy levels, or new or disconnected loads. Further, cost
comparisons are complicated by changes in electricity rates, which
affect the dollar value of energy and demand savings.

The current state of the art in federal M&V practices enables the
government to cost-effectively verify savings to an acceptable degree of
certainty without allocating unmanageable risks to the ESCO that would
burden projects with premium pricing and financing costs as compensation
for bearing those risks. Several recognized best practices are generally
followed in federal ESPC projects, unless unusual circumstances warrant
different arrangements. Generally the ESCO's risks are limited to
guaranteeing the performance of the ECMs, and guaranteed performance is
translated into contracted cost savings assuming typical weather,
pre-retrofit baseline load levels for the non-project-related loads, and
stipulated energy cost escalation rates.

While contracted savings as calculated may differ from actual cost
savings in a given year, contracted and actual savings will be
reasonably similar and over the contract term the two values will tend
to converge (if appropriate assumptions and choices have been made)
because over time the actual weather (for example) will tend to conform
to the same average used for the experience-based stipulation.

An additional margin of safety (and cost savings) is afforded by the
fact that ESCOs universally guarantee less than 100% of estimated
savings to increase their certainty of meeting the guarantee. And the
fact remains that the government has remedies against the ESCO if the
verified contracted cost savings do not match or exceed the guaranteed
cost savings each year.

The alternative to using simplifying assumptions for the purpose of
calculating savings --- having the ESCO take the risk that factors such
as the weather, future energy rates, and the government's own operating
hours and non-project-related loads will affect savings --- would be a
poor and expensive choice for the government.

Agencies that decide to correlate the impact of individual ESPC projects
directly to changes in their utility bills, either at the site or agency
level, will probably need to consider implementing a system for tracking
what their utility bills would have been had the energy efficiency
projects never been implemented. Since most agencies have tracking
systems for their actual energy use and costs across all agency sites,
perhaps these same systems could be enhanced for this purpose. However,
agencies will have to weigh the value of the extra effort required to
calculate actual savings against its cost.

## 7. References / Bibliography

American Society of Heating, Refrigerating and Air-Conditioning
Engineers, Inc. (ASHRAE). 2002. *ASHRAE Guideline 14-2002, Measurement
of Energy and Demand Savings*.

Davidson, A. C., D. V. Hinkley, R. Gill, B. D. Ripley, S. Ross, M.
Stein, D. Williams. 1977. *Bootstrap Methods and Their Application*.
(Cambridge Series in Statistical and Probabilistic Mathematics, No. 1).
Cambridge University Press.

Federal Energy Management Program (FEMP). 2000. *Measurement and
Verification Guidelines for Federal Energy Projects, Version 2.2*,
DOE/GO-102000-0960. U.S. Department of Energy.

Hughes, P. J., and J. A. Shonder. 1998. *The Evaluation of a 4000-Home
Geothermal Heat Pump Retrofit at Fort Polk, Louisiana: Final Report,*
ORNL/CON-460, Oak Ridge National Laboratory, Oak Ridge, Tenn.

IPMVP Inc. 2001. *International Performance Measurement and Verification
Protocol*. www.ipmvp.org.

Lippiat, Barbara C. 1992. Energy Prices and Discount Factors for
Life-Cycle Cost Analysis 1993 --- Annual Supplement to NIST Handbook 135
and NBS Special Publication 709. NIST 85-3273-7 (Rev. 10.92). U.S.
Department of Commerce.

Reddy, T. A., N. F. Saman, D. E. Claridge, et al. 1977. "Baselining
Methodology for Facility-Level Monthly Energy Use --- Part 1:
Theoretical Aspects." ASHRAE Trans., 1977, vol. 103, part 2.