Energy Savings Performance Contracts: Additional Actions Needed to Improve Federal Oversight
Published: 2015-06-17T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: Constrained budgets and increasing energy efficiency goals have led federal agencies to explore innovative ways to fund energy improvements, including ESPCs. An expected increase in the use of ESPCs has raised questions about agencies' ability to ensure that the government's interests are protected. ESPCs can span up to 25 years and be valued at millions of dollars each.
GAO was asked to review federal use of ESPCs since 2005. This report examines the extent to which (1) agencies have used ESPCs and plan to use them; (2) projects have achieved their expected cost and energy savings; and (3) agencies have overseen and evaluated such projects. GAO compiled data on awarded ESPCs; reviewed agency guidance and files for a nongeneralizable sample of 20 ESPC projects that reflected a range of contract award dates, contract values, and other characteristics; and interviewed officials from the seven agencies with the highest energy usage and greatest facility square footage—the Air Force, Army, and Navy within the Department of Defense; the Departments of Energy, Justice, and Veterans Affairs; and the General Services Administration.
section_title: What GAO Found
paragraphs: The seven selected agencies in GAO's review awarded approximately $12.1 billion in energy savings performance contracts (ESPC) in fiscal years 1995 through 2014 and plan to continue using them to help meet federal energy directives and initiatives. Under ESPCs, private contractors finance the up-front costs of energy improvements. Agencies then repay contractors from the savings, such as those resulting from lower utility bills. The seven agencies GAO reviewed have used more than 500 ESPCs for projects, such as installing energy-efficient lighting or power generation projects. Agencies' plans to use ESPCs vary, particularly for data center consolidation projects, which could reduce a significant amount of energy.
Cost and energy savings that contractors reported to agencies for most ESPCs met or exceeded expectations, but some of these savings may be overstated. GAO's review of a nongeneralizable sample of 20 projects found that contractors' reports overstated cost and energy savings for 14 projects. Contractors calculate and report savings annually in accordance with plans agreed to in their contracts with agencies. These plans include assumptions about agencies' use of equipment, which may change over the life of the contract. If changes reduce project savings, such as when an agency does not operate or maintain the equipment as agreed, contractors are not required to reduce the amount of savings they report or measure the changes' effects. GAO evaluated the extent to which changes in agency operations or other factors within agencies' control may have reduced energy savings for a nongeneralizable sample of projects. Estimates agencies provided to GAO of savings that were reported but not achieved ranged from negligible to nearly half of a project's reported annual savings. For example, one agency removed equipment for a sewer system upgrade, which resulted in over $104,000 in annual savings that were reported but not achieved, or about 40 percent of the project's reported savings. Federal guidance states that when reviewing contractor reports, agencies should understand changes in project performance and savings levels, and what actions should be taken to address deficiencies. However, agencies were not always aware of how much savings were not achieved due to agency actions because contractors were not required to report this information. Without clearer reporting of savings that are not achieved, agencies may be unable to determine what, if any, corrective actions should be taken.
The seven agencies in GAO's review have conducted limited oversight and evaluation of their ESPC projects. In GAO's sample of 20 projects, agency representatives did not perform some oversight activities included in guidance because they were unaware of these duties or how to perform them, among other reasons. Without ensuring that training provides officials with the information needed to understand how to perform their oversight responsibilities, agencies may continue to inconsistently perform oversight. Moreover, most of the agencies in GAO's review have not systematically evaluated their ESPC portfolios to determine the effects of changing circumstances—such as facility use—on project performance because they do not have processes to do so. Without such oversight and evaluation, agency officials cannot make fully informed decisions about how best to strategically manage their ESPCs.
section_title: What GAO Recommends
paragraphs: GAO is making recommendations to improve oversight of ESPC projects through clearer reporting of savings, improved training, and systematic evaluations of portfolios, among other things. In general, the agencies partially concurred with GAO's recommendations. GAO has modified the report, as appropriate, to address the agencies' comments.

section_title: Letter
paragraphs: The federal government is the nation’s largest energy consumer. In fiscal  year 2013, the government spent about $6.8 billion on energy for over 3.1  billion square feet of buildings and facilities—an area about the size of  50,000 football fields. Several laws and executive orders have  established federal energy requirements and sustainability goals, such as  goals to reduce energy usage and conserve water. Implementing projects  to meet these requirements and goals can be costly, and obtaining up- front appropriations for such projects has been particularly challenging for  agencies in recent years because of constrained federal budgets.  Consequently, agencies have explored alternative methods to fund  energy conservation measures, such as more efficient lighting, heating, or   One of these alternative methods is using energy savings  other systems.performance contracts (ESPC), a type of share-in-savings contract under  which agencies use private funds to finance energy conservation  measures. Although usage of ESPCs has varied from year-to-year, about  23 percent of federal investments in energy efficiency and renewable  energy for fiscal years 2003 through 2013 have been made using ESPCs,  according to Department of Energy (DOE) data.
In 1986, Congress authorized federal agencies to use shared energy  savings contracts, now known as ESPCs, to privately finance energy  improvements. DOE’s Federal Energy Management Program (FEMP) is  the lead agency program for providing implementing rules and policies  regarding ESPCs. Under an ESPC, agencies enter into a long-term  contract—up to 25 years—with a private contractor to install energy  conservation measures and then make annual payments to the contractor  until the measures have been paid off. As part of an ESPC, the agency  and the contractor estimate the annual energy and cost savings expected  from the energy conservation measures outlined in the contract and  develop a plan to measure and verify that the savings are achieved over  the life of the contract. Cost savings are to be calculated as the difference  between the baseline costs for energy, water, and related expenses that  the agency would have incurred without the ESPC, and the actual costs  incurred with the energy conservation measures in place. ESPCs are  intended to shift the performance risks of conservation measures from the  agency to the contractor by making the annual payments to the contractor  contingent upon verifying that estimated savings have been realized. By  law, an agency’s aggregate annual payments may not exceed the amount  that the agency would have paid for utilities without the ESPC.contractors have been fully repaid for the costs of the energy  conservation measures, interest and other costs associated with financing  the energy conservation measures, and related costs, such as for any  operation and maintenance services the contractor provided, agencies   Once  retain any savings the energy conservation measures continue to  generate.
We have previously raised questions about federal agencies’ use of  ESPCs. Specifically, in our June 2005 report, we found that agency  officials generally believed that ESPCs’ savings cover the contract’s  costs, but we could not verify that conclusion using the agencies’ ESPC  data.contract terms, missing documentation, and other problems that called  into question how consistently savings cover contract costs. Additionally,  we found that agency officials often did not have the expertise and related  information needed to effectively develop and negotiate the terms of  ESPCs and to monitor contract performance once energy conservation  measures were installed. We recommended, among other things, that  agency officials responsible for ESPC decision making make use of  appropriate expertise when undertaking ESPCs and that agencies more  effectively collect and use ESPC-related data to help ensure that ESPCs’  savings cover their costs. The agencies generally concurred with these  recommendations and have taken steps to address them, as noted  throughout this report.
Our work and agency audits found ESPCs with unfavorable  You asked us to update our June 2005 report and review federal use of  ESPCs since 2005. This report examines the extent to which (1) selected  agencies have used ESPCs and plan to use them in the future, (2)  selected agencies’ ESPC projects have achieved their expected cost and  energy savings, and (3) selected agencies have overseen and evaluated  their ESPC projects.
To determine which federal agencies to include in our review, we selected  agencies with the highest energy usage and greatest facility square  footage based on government-wide data collected by FEMP. We selected  the following seven agencies based on the above criteria the Air Force,  Army, and Navy within the Department of Defense (DOD); DOE; the  Departments of Justice (Justice), and Veterans Affairs (VA); and the  General Services Administration (GSA). We refer to these organizations  as the “seven selected agencies.” As of fiscal year 2013, DOD, DOE,  Justice, VA, and GSA represented 78 percent of the federal government’s  total floor space and 80 percent of the government’s energy use. Our  findings from our reviews of these seven agencies cannot be generalized  to agencies we did not include. To provide information for all of our  objectives, we reviewed relevant agency and contractor reports and we  interviewed knowledgeable officials at the seven selected agencies, as  well as at FEMP and two industry associations representing contractors.  We also conducted site visits at two ESPC projects, which we selected  based on whether they were undertaken by federal agencies within our  review, involved a range of technologies, and were readily accessible.
To determine the extent to which selected agencies have used ESPCs  and plan to use them in the future, we collected and analyzed agency  data on ESPCs awarded in fiscal years 1995 through 2014—the span of  time available agency data cover. We assessed the reliability of selected  agencies’ ESPC data by (1) performing electronic testing of required data  elements, (2) reviewing existing information about the data and the  systems that produced them, and (3) interviewing agency officials  knowledgeable about the data. We assessed available ESPC data from  the Air Force, Army, Navy, DOE, and GSA and determined that the data  were sufficiently reliable for the purposes of this report. Because data  from Justice and VA on ESPCs were not sufficiently reliable for our  purposes, we relied on DOE’s data for these agencies. We also reviewed  relevant federal laws, executive orders, the selected agencies’ fiscal year  2014 Strategic Sustainability Performance Plans, and Office of  Management and Budget (OMB) guidance. To determine the extent to  which selected agencies’ ESPCs achieved their expected cost and  energy savings, we reviewed six studies by DOE’s Oak Ridge National  Laboratory analyzing ESPCs’ reported cost and energy savings. We  assessed the studies’ methodology and interviewed their authors, and we  determined the findings were sufficiently reliable for the purposes of our  report. To provide illustrative examples of the extent to which expected  energy and cost savings were achieved, we reviewed contractors’ annual  measurement and verification reports and other project documentation for  a nongeneralizable sample of 20 ESPC projects with a total contract  value of about $824 million. We selected these projects from among the  530 ESPC projects awarded by the seven selected agencies in fiscal  years 1995 through 2014. We selected projects that reflected a range of  contract award dates, contract values, and other characteristics. We also  interviewed officials from the seven selected agencies regarding their  processes for measuring and verifying ESPC savings. To determine the  extent to which selected agencies and FEMP have overseen and  evaluated ESPC projects, we reviewed contractors’ annual measurement  and verification reports and other documentation for the above  nongeneralizable sample of 20 ESPC projects and also interviewed  FEMP and other agency officials about oversight activities associated  with the sample projects. Additionally, we reviewed audit agencies’  reports issued from 2005 through 2014 and interviewed FEMP and  knowledgeable officials about general oversight procedures for and  evaluations of agency ESPC projects. See appendix I for more  information on the objectives, scope, and methodology of our review.
We conducted this performance audit from March 2014 to June 2015 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
subsections: section_title: Background
paragraphs: Several key laws and an executive order directly relate to federal  agencies’ use of ESPCs (see table 1).
In addition, in December 2011, the President challenged federal agencies  to enter into $2 billion in performance-based contracts, including ESPCs  and utility energy service contracts through the President’s Performance  Contracting Challenge. In May 2014, the President expanded this challenge to total $4 billion in performance-based contracts by the end of  2016.
The process that agencies and contractors generally follow for developing  and implementing an ESPC project spans five phases, from acquisition  planning—during which agencies identify project requirements and  assemble their acquisition team—to project performance—during which  energy conservation measures are in place and operating, and agencies  pay contractors. Figure 1 illustrates the general process for developing  and implementing an ESPC project.
During the process of developing and implementing an ESPC project,  agency officials often work with DOE’s or the U.S. Army Corps of  Engineers’ (Corps) federal contracting centers. Both DOE and the Corps  have awarded indefinite-delivery, indefinite-quantity ESPC contract  vehicles to a set of prequalified energy services contractors. Agencies  using these “umbrella” contract vehicles can award an ESPC for an  individual project to any of the prequalified contractors. Using one of  these contract vehicles allows agencies to develop and implement an  ESPC project in less time because the process of competitively selecting  qualified contractors has already been completed, and key aspects of  contracts have been broadly negotiated. In addition, both DOE and the  Corps provide contracting and technical support to agencies that use their  contract vehicles. For example, DOE’s FEMP provides facilitation  services, where a third party assists the agency and contractor in  agreeing on the terms of a contract. FEMP also issues guidance and  offers training for agencies on the various steps of developing and  implementing an ESPC project. The Corps provides technical support,  cost estimating services, and legal support to agencies using its contract  vehicle. As part of revisions that DOE and the Corps made to their  contract vehicles in 2008, both DOE and the Corps now require that  agencies use a qualified project facilitator when developing and  implementing an ESPC, which addresses the recommendation in our  June 2005 report to ensure that agencies use appropriate expertise when  undertaking an ESPC. Additionally, each of the seven agencies in our  review has established a central office to support individual sites with  developing and implementing ESPC projects, and several of these  agencies have increased the role of their central offices since our last  review to provide additional support, such as oversight of ESPCs.
An ESPC project’s expected cost and energy savings are established  during project development, finalized when the contract is awarded, and  measured and verified over the course of a project’s performance period.  These savings can include reductions in costs for energy, water,  operation and maintenance, and repair and replacement directly related  to the energy conservation measures. Agencies must pay contractors  from funds appropriated or otherwise made available to pay for such  utilities and related operational expenses. Payments to contractors  generally cover the costs associated with equipment and installation,  contractor-provided operation and maintenance services, financing  charges, and other costs. ESPC projects generally include two types of  expected savings: (1) proposed cost and energy savings, which  contractors estimate will result from the energy conservation measures  installed, and (2) guaranteed cost savings, which must be achieved for   Generally, contractors guarantee about  the contractor to be fully paid.95 percent of a project’s proposed cost savings, which gives them room  for some amount of proposed savings to not be achieved without a  reduction in their payments.
Energy and cost savings are the difference between a projected baseline  of the energy use without the energy conservation measures and with the  measures. The process used to determine ESPC savings is referred to as  measurement and verification. Most ESPC projects include the following  four key documents that outline how cost and energy savings are to be  measured and verified:
Measurement and verification plan. During the project development  phase, the contractor and agency develop a plan that establishes how  to measure and verify that savings are achieved. Measurement and  verification methods can include surveys, inspections, direct  measurements of energy use, and other activities to ensure that  equipment is operating correctly and has the potential to generate  expected savings.
Risk and responsibility matrix. During the contractor selection  phase, the contractor and agency develop a risk and responsibility  matrix that identifies key project risks and their potential effects, and  specifies whether the agency or the contractor will be responsible for  managing financial risks, such as changing interest rates; operational  risks, such as operating hours and weather; and performance risks,  such as equipment performance and preventative maintenance.
Postinstallation measurement and verification report. After the  energy conservation measures are installed, the contractor conducts  measurement and verification activities and presents the results in a  postinstallation measurement and verification report.
Annual measurement and verification report. Throughout an  ESPC’s performance period, the contractor conducts measurement  and verification activities and submits an annual report to the agency  to document the cost and energy savings achieved.
According to FEMP guidance, one of the primary purposes of  measurement and verification is to reduce the risk that expected savings  will not be achieved. FEMP guidance describes risks related to (1)  equipment use, which stem from uncertainty in operational factors, such  as the number of hours equipment is used or changes in the planned  operation of equipment, and (2) equipment performance, which stem from  uncertainty in projecting a specified level of performance. Contractors are  usually reluctant to assume risks related to equipment use because they  often have no control over operational factors. In contrast, according to  FEMP guidance, the contractor is ultimately responsible for the selection,  design, and installation of equipment and typically assumes responsibility  for performance risks.
FEMP guidance outlines a range of options that contractors may select to  measure and verify the cost and energy savings achieved by each energy  conservation measure. If certain factors that affect savings, such as  weather conditions, utility prices, and hours of agency operation, are  either too complex or costly to measure, agency and contractors may  choose to agree in advance on—or stipulate—the values for those factors  regardless of the actual behavior of those factors. For example, because  ESPCs can be long contracts, the contractor and agency typically  stipulate escalation rates to estimate future utility prices during the  performance period. If the savings that are achieved are less than the  savings calculated using stipulated values, the agency pays the  contractor for the savings calculated using stipulated values. If achieved  savings are greater than the savings calculated using stipulated values,  the agency retains the additional savings.
The measurement and verification options that FEMP guidance outlines  vary in their rigor and costs. The option that is generally the least rigorous  and costly involves measuring the key factors affecting energy use—such  as the number of lighting fixtures or efficiency of a heating unit—before  and after installation, but typically does not involve measuring such  factors over the term of the contract. In contrast, other options outlined in  FEMP guidance generally involve ongoing measurements of energy use,  or proxies of energy use, over the contract term. FEMP guidance helps  identify when each option should be used and states that the selection of  a measurement and verification method is based on project costs and  savings, complexity of the energy conservation measure, and the  uncertainty or risk of savings being achieved, among others. According to  FEMP guidance, costs for measurement and verification generally  increase with the level of accuracy required in energy savings analyses  and the number and complexity of variables that are analyzed, among  other factors. Moreover, the incremental value of additional measurement  and verification will at some point be less than its cost. For instance, the  energy consumed by a light fixture does not change appreciably over  time, and requiring contractors to measure fixtures annually would  increase the cost of measurement and verification for little benefit,  according to FEMP officials.
section_title: Selected Agencies Have Awarded About $12.1 Billion for ESPCs for a Variety of Projects, and Their Plans for Continued Use Vary
paragraphs: In fiscal years 1995 through 2014, the seven selected agencies in our  review awarded approximately $12.1 billion in ESPCs for a variety of  projects, such as constructing biomass facilities to heat federal buildings.  According to agency officials and documents, agencies plan to continue  using ESPCs to meet federal energy directives and initiatives, but some  agency officials said they are hesitant to use ESPCs to consolidate data  centers.
subsections: section_title: Selected Agencies Awarded About $12.1 Billion for ESPCs in Fiscal Years 1995 through 2014
paragraphs: In fiscal years 1995 through 2014, the seven selected agencies awarded  approximately $12.1 billion for more than 500 ESPC projects to help fund  energy conservation measures in federal facilities. The total amount  awarded in ESPCs varied by agency, with the Army awarding the most— approximately $3.7 billion. (See fig. 2.) Data on the ESPCs awarded by  each agency are included in appendix II.
The seven selected agencies awarded approximately 530 ESPCs in fiscal  years 1995 through 2014. The length of the contracts for these projects  ranged from approximately 2 to 25 years, with an average of about 16  years. Additionally, the projects for ESPCs awarded during this period  had a total guaranteed cost savings of roughly $12.4 billion and total  proposed energy savings of approximately 563 trillion British thermal units  (Btu).energy savings by agency.
section_title: Agencies Have Used ESPCs for a Variety of Projects
paragraphs: The seven agencies have used ESPCs for a variety of projects ranging  from smaller-scale projects to install more energy-efficient light bulbs or  water flow restrictors in toilets, to larger-scale projects, such as power  generation projects. For example, GSA officials used ESPCs for three  projects at its White Oak, Maryland, facility to install infrastructure and  equipment with cogeneration capabilities, which involves the  simultaneous production of electricity and heat from a single fuel source,  such as natural gas. Figure 3 shows some of the White Oak cogeneration  project components. Additionally, DOE installed a biomass facility at the  National Renewable Energy Laboratory in Golden, Colorado. The  biomass facility, the first of its kind for DOE, according to project officials,  uses wood chips from forest thinnings and trees killed by either beetles or  fire as fuel to generate heat that warms water for buildings at the campus.  Figure 3 shows some of the components of DOE’s biomass facility.
Some agencies have started to use ESPCs to develop larger and more  comprehensive projects to try to achieve greater cost and energy savings.
For example, in 2012, GSA began using ESPCs for its National Deep  Energy Retrofit program which, according to an analysis by the Oak  Ridge National Laboratory, achieved an average level of savings more  than twice that of other federal ESPC projects. Furthermore, to help  achieve these cost and energy savings, agencies have increasingly  turned to bundling energy conservation measures together under an  ESPC, which is more efficient than using separate contracts, according to  FEMP officials.
section_title: Agencies’ Plans for Using ESPCs in the Future Vary, Particularly for Data Center Consolidation Projects
paragraphs: Selected agencies’ plans to use ESPCs in the future vary. Officials from  five of the agencies we spoke with said their plans for continuing to use  ESPCs will help their agencies meet goals in federal executive orders and  other energy goals, including the President’s Performance Contracting  Challenge. For example, in response to federal energy goals, Army  officials said they plan to aggressively pursue using ESPCs, among other  financing options, to improve energy efficiency. Justice officials said they  plan to extensively use ESPCs at all of their Bureau of Prisons sites to  upgrade and repair many buildings that have aging infrastructure. VA  officials said ESPCs are one of many tools to meet energy goals and the  agency prioritizes ESPCs at all facilities where feasible. With regard to  the President’s Performance Contracting Challenge, agencies  government-wide had awarded approximately $1.9 billion in performance- based contracts out of the $4 billion goal as of January 2015, as shown in  table 3, with the seven selected agencies in our review awarding most of  these contracts. If agencies award the contracts they currently have  planned, they will meet the Challenge’s goal of awarding $4 billion in  performance-based contracts by the end of 2016 (see app. III for federal  agencies’ status in achieving their goals under the President’s  Performance Contracting Challenge).
Some agency officials we interviewed said they are interested in using  ESPCs to consolidate data centers, which consume significant amounts  of energy and can be costly to operate, but the agency officials are  hesitant to move forward with such projects because of concerns OMB   staff have raised about using ESPCs for such projects. By law, ESPCs  must be used “solely for the purpose of achieving energy savings and  benefits ancillary to that purpose.” However, the law does not specify  what qualifies as ancillary benefits—also referred to as energy-related  savings—or the proportion of an ESPC’s overall savings that can be  energy-related. OMB guidance on federal use of performance contracts  outlines some general criteria that projects must meet to be scored under  OMB’s annual budget scoring process but does not provide specific  guidance on energy-related savings. If an ESPC project does not meet  OMB’s criteria, then to pursue the project the agency would need to  obligate funding for the entire contract “up front” in its first year, rather  than annually. This can be an issue because agencies might not have the  funding for the entire contract during its first year, which would leave  agencies with the option of either canceling the contract or moving  funding from other agency efforts.
According to DOE officials, they nearly completed the project  development phase of the ESPC development and implementation  process in May 2011 for a project to use an ESPC to consolidate data  centers. However, they delayed awarding the contract in March 2013  because OMB staff raised concerns about the project. DOE officials said  the project, as originally proposed, would consolidate two data centers  and replace 5,000 desktop computers with computers that are more  energy efficient. The project was expected to save DOE approximately  $76 million, and 97 percent of the overall cost savings would come from  operations and maintenance such as maintaining computer hardware and  software, or energy-related savings, and the remaining 3 percent from  energy savings. According to DOE officials, the concerns that OMB staff  raised included (1) whether savings resulting from more efficient  information technology equipment qualify as energy-related savings and  (2) the project’s high proportion of cost savings resulting from the  reduction in operations and maintenance costs, rather than energy cost  savings. At the time of our review, DOE had resumed consideration of the  project and had not awarded a contract, but said that OMB staff had not  clarified their position regarding their concerns.
According to Army officials, the Army is also interested in using ESPCs to  consolidate data centers, but they are hesitant to move forward with any  projects because they have heard about OMB’s concerns and are waiting  to learn OMB’s position regarding DOE’s data center consolidation. Army  officials said they have not seen any information on OMB’s position  officially released, which they said they need before pursuing the use of  ESPCs for data center consolidation projects. Furthermore, DOD officials  who oversee Army and other DOD agencies said the agencies need  clarification on whether moving data to a more energy efficient off-site  storage facility (rather than storing it on servers in DOD facilities) or  eliminating help desk support and software licenses would qualify as  energy-related savings under OMB guidance.
According to federal standards for internal control, information should be  communicated to those who need it in a form and within a time frame that   Because OMB staff have  enables them to carry out their responsibilities.expressed concerns about but have not clarified their position on what  qualifies as energy-related savings and the allowable proportion of energy  and energy-related cost savings, DOE delayed its data center  consolidation project and some agencies, such as the Army, have been  hesitant to pursue using ESPCs for such projects. As a result, agencies  might be needlessly missing opportunities for potential energy and  energy-related cost savings. We solicited OMB staff’s comments during  our review regarding their position on DOE’s data center consolidation  ESPC project, as well as the use of ESPCs for data center consolidation  projects generally, regarding what qualifies as energy-related savings and  the proportion of cost savings resulting from operations and maintenance  and energy use reduction. In response, OMB staff said, in part, that it is  generally not appropriate for them to comment on the merits of specific  contracts.
section_title: Reported Savings Generally Exceeded Expectations, but Some Savings for Selected ESPC Projects Were Overstated
paragraphs: The cost and energy savings that contractors reported for most ESPCs  met or exceeded expected savings, according to studies by DOE’s Oak  Ridge National Laboratory, but some of these savings may be overstated.  Our review of a nongeneralizable sample of 20 projects found that  contractors overstated cost and energy savings for 14 projects by  reporting some savings that, due to agency actions, were not achieved.  Contractors must calculate and report savings in accordance with plans  agreed to in their contracts with agencies. If factors beyond contractors’  control reduce the savings achieved, contractors generally are not  required to reduce the amount of savings they report or measure the  effects of such factors on savings. Agencies were not always aware of the  amount of expected savings that were not achieved among their projects,  in part, because contractors generally do not provide this information in  measurement and verification reports.
subsections: section_title: Reported Cost and Energy Savings for Most ESPCs Met or Exceeded Expected Savings
paragraphs: DOE’s Oak Ridge National Laboratory found in its six studies of  contractor-reported savings for agencies that awarded ESPCs through  DOE’s contract vehicle, that the total cost and energy savings reported  for these ESPCs exceeded their expected savings. The total cost  savings reported in the 6 years of annual measurement and verification  reports that Oak Ridge analyzed was about 106 percent of the total  guaranteed cost savings for these ESPCs. Moreover, in each of the 6  years, total reported cost savings across all projects were at least 105  percent of total guaranteed savings. Similarly, the Oak Ridge studies  found that the total energy savings reported for ESPCs awarded through  DOE’s contract vehicle exceeded proposed energy savings. Specifically,  the total energy savings reported in all of the annual measurement and  verification reports analyzed over the 6 years was about 102 percent of  the total proposed energy savings for these ESPCs. Moreover, in each of  the 6 years, total reported energy savings across all projects were at least  equal to total proposed savings.
Most contractors reported cost savings that exceeded guaranteed  savings, but the Oak Ridge studies found that some contractors reported  cost savings below guaranteed amounts, also referred to as cost savings  shortfalls, in about 6 percent of the reports that Oak Ridge reviewed. The  average shortfall in cost savings for the small number of ESPCs with a  reported shortfall was 17 percent, meaning reported cost savings were 83  percent of guaranteed amounts for these ESPCs. However, these  shortfalls ranged widely, from 0.5 percent to 75 percent of guaranteed  cost savings. Appendix V provides additional information from the Oak  Ridge studies.
Similarly, we found that reported savings for 19 of the 20 projects in the  nongeneralizable sample we reviewed met or exceeded their guaranteed  cost savings for the year reviewed. The remaining project, DOE’s  National Renewable Energy Laboratory, had a reported cost savings  shortfall of about $76,000—about 18 percent of its guaranteed cost  savings—in its most recent measurement and verification report.  According to the project’s measurement and verification report, the  shortfall was primarily due to warmer weather, which reduced the number  of days the equipment was used, and there was an outage due to a failed  motor. In the measurement and verification report, the contractor  identified planned changes to the equipment that are expected to address  the performance deficiencies and savings shortfalls.
FEMP tracks cost savings shortfalls monthly for ESPCs that agencies  have awarded through DOE’s contract vehicle with its “dashboard report”  and has found that contractors are reporting that most ESPC projects are  meeting or exceeding their guaranteed savings. For instance, a sample  dashboard report from 2014 showed nine projects had reported cost  savings shortfalls, ranging from less than 1 percent of guaranteed savings  to more than 30 percent of guaranteed savings. The dashboard report  includes details on the reasons for shortfalls and actions for FEMP to take  to help agencies address shortfalls. According to FEMP officials, the  dashboard report, which FEMP developed in 2007, provides FEMP  management with a snapshot of key aspects of the ESPC projects and  has enabled them to more effectively monitor ESPC projects, which GAO  recommended in 2005.
section_title: Most Selected Measurement and Verification Reports We Reviewed Overstated Some Cost and Energy Savings
paragraphs: Measurement and verification reports for 14 of the 20 projects in the  nongeneralizable sample we reviewed—including projects from each of  the seven selected agencies—overstated some cost and energy savings  in that they reported savings that were not achieved. Contractors must  calculate and report annual savings in accordance with the measurement  and verification plans agreed to in their contracts with agencies. These  plans include measuring equipment performance. They also include  assumptions about factors that are beyond contractors’ control, such as  agencies’ use of energy-saving equipment and utility prices, which may  change over the life of the contract. If changes in such factors reduce  savings, contractors generally are not required to reduce the amount of  savings they report or measure the effects of such changes. For example,  contractors do not generally reduce the savings they report when an  agency alters the agreed upon hours of operation, thus reducing the  number of hours that energy-saving equipment is used. Conversely, if  savings increase because of changes in factors beyond contractors’  control, contractors generally do not increase the amount of savings they  report, and agencies generally retain any surplus savings and do not  increase payments to the contractor.
Measurement and verification reports for 14 projects in our sample  overstated some cost and energy savings in that they reported savings  that were not achieved because of agencies’ actions, including (1)  agencies not operating or maintaining equipment as agreed when the  ESPC was awarded and (2) agencies’ removal of equipment from or  closure of facilities where energy conservation measures had been  installed. For projects in our sample, contractors’ reports generally did not  quantify or estimate the effects of these factors on savings, although  some reports noted that savings were affected in some way. Because of  the large number of factors that can result in overstated or understated  savings, we did not determine the net effect of all factors on projects’  achieved savings. For example, some energy conservation measures in  the projects we reviewed outperformed expectations, which may have  offset the lower-than-expected savings of other energy conservation  measures in those projects. Table 4 shows the projects we reviewed and  the agencies’ actions that affected savings that were reported in the most  recent measurement and verification report, as of September 2014. (For  further detail on the effects of these factors on savings for these projects,  see app. VI.)
The following are examples from the ESPC projects we reviewed of  agency actions that resulted in reported savings that were not achieved:
Agency did not operate or maintain equipment as agreed  The  most common factor resulting in overstated savings for the ESPC  projects we reviewed was an agency making changes to operating  hours and temperature set points on programmable heating,  ventilation, and air conditioning (HVAC) equipment, which occurred in  8 of the 20 projects. According to available agency estimates, these  changes generally resulted in lower energy and associated cost  savings than expected, but contractors did not reflect these effects in  reported savings amounts because they were due to agency  actions. In other cases, agencies did not fulfill their responsibilities  for operating or maintaining equipment. For instance, the contractor  for a project at a Justice facility found that steam distribution  equipment the contractor installed had been damaged, reducing the  savings the equipment achieved. However, the contractor did not  reduce reported savings because it stated that the damage resulted  from improper operation by Justice staff. In some cases, agencies  took actions that reduced savings, such as changing operating hours  or temperature set points, to meet changing agency mission needs.
Agency removed or abandoned equipment  components of energy conservation measures or entire measures  were removed by the agency during the performance period, but  contractors did not reduce reported savings because these changes  were due to agency decisions. For instance, the Army closed a  section of an installation that had numerous buildings with energy  conservation measure equipment installed. As a result, savings were  not being generated by this equipment, but the contractor reported the  savings that would have been achieved for the year had the  equipment continued to operate. In some cases, agencies removed  or abandoned equipment to meet changing agency mission needs.
The amount of savings reported but not achieved ranged from negligible  to nearly half of an ESPC project’s reported savings for the year, based  on information provided by agencies and our analysis of available  information from the most recent measurement and verification reports for  selected projects. For example, where estimates were available, agency  changes to operating hours and temperature set points on programmable  HVAC equipment generally resulted in savings that were reported but not  achieved that were negligible as a percentage of the total savings  reported, according to agency officials. In contrast, the Air Force’s  removal of equipment associated with a sewer system upgrade resulted  in over $104,000 in annual savings that were reported but not achieved— about 40 percent of the annual savings reported for the project. (For a full  list of the projects we reviewed and information on the effects of factors  beyond contractors’ control on savings, see app. VI.) Officials from  several agencies noted that there are benefits to funding energy  conservation projects through ESPCs, as opposed to using up-front  appropriations. The officials noted that like ESPC projects, the expected  savings for projects funded with up-front appropriations may not be  achieved.  However, savings that are not achieved are more likely to be  identified for ESPC projects because savings must be measured and  verified.
Unlike changes in agencies’ use of equipment, agencies cannot control  changes in utility prices, but changes in utility prices compared with the  amounts stipulated in the contracts could affect the savings for ESPC  projects. Agencies commonly stipulate annual escalation rates for energy  costs based on projected utility prices published by the National Institute  of Standards and Technology and developed by DOE’s Energy  Information Administration. DOE has reported that the projected utility  prices for ESPCs awarded through its contract vehicle have generally  underestimated the actual increase in utility prices, and therefore ESPC  projects are generally saving more than expected. Specifically, in 2007,  DOE’s Oak Ridge National Laboratory analyzed 22 ESPC projects to  calculate savings using actual, rather than projected, utility prices. After  adjusting for actual utility prices, savings for 16 of the 22 ESPC projects  Oak Ridge examined were greater than the savings contractors reported,  while savings for the remaining 6 ESPC projects were lower than  reported.
Energy markets have changed significantly since 2007, and are likely to  change in the future. For example, improvements in horizontal drilling and  hydraulic fracturing led to large increases in the production of natural gas  from shale formations, which contributed to significant decreases in the  price of natural gas. Such changes likely affected the savings that certain  ESPC projects achieved, such as those whose savings were based  predominantly on reductions in natural gas use. However, it is not clear  whether the assumptions that agencies are using for utility prices are  reasonable because DOE has not conducted an analysis of ESPC  projects awarded under its contract vehicle since its 2007 report. As a  result, agencies may not have the information they need to know whether  their projects are achieving expected savings and achieved savings may  be significantly different—either higher or lower—than reported savings.  There are drawbacks of assumptions about utility prices being  consistently higher or lower than actual rates. DOE guidance states that  stipulating higher utility rates, which generally results in higher expected  savings, will provide better cash-flow for projects. However, the guidance  also states that overvaluing savings is a serious concern that can cause  budgetary problems for the agency. This is because contractor payments  must come from agency funds used to pay for energy, water, and related  expenses. Therefore, contractor payments that exceed achieved energy,  water, and related savings will limit the funds agencies can use to cover  these expenses.
We identified three projects in our nongeneralizable sample for which  achieved savings were lower than reported savings because utility prices  differed from those stipulated in the contract. Specifically, for one DOE  project and one Justice project, natural gas prices were significantly lower  than the amounts stipulated in the contracts, which led to achieved  savings that were about $147,000 and $477,000 less than the reported  cost savings for the year, respectively. These amounts represented about  44 percent of the reported cost savings for the DOE project and about 30  percent of the reported cost savings for the Justice project. Additionally,  an Air Force project that involved switching inefficient oil heating units to  natural gas units projected rates for natural gas that did not reflect  seasonal price increases in winter months. Because actual natural gas  prices were substantially higher than projected in the winter, the costs of  running the new natural gas units were higher than projected. This  resulted in achieved savings that were about $160,000 less than the  reported savings for the year, which was about 5 percent of the project’s  total reported savings for the year. Utility prices vary from year to year, so  it is to be expected that prices will differ from the stipulated values in  some years. Without a periodic analysis of utility prices over several years  and across projects, agencies may not have the information they need to  know whether examples like the three in our sample are typical and  indicative of problems with the assumptions or anomalies.
section_title: Agencies Were Not Always Aware of How Much Expected Savings Were Not Achieved
paragraphs: Agencies were not always aware of the amount of expected savings that  were not achieved, in part because contractors generally did not—and  were not required to—report this information when savings were not  achieved due to agency actions. FEMP guidance states that when  reviewing measurement and verification reports, agencies should  understand changes in project performance and savings levels from year  to year, and what corrective actions should be taken to address  deficiencies resulting in savings that are not achieved. In addition, the  DOE and Corps contract vehicles provide an outline for contractors to use  in writing annual reports, which includes sections detailing performance,  operating, and maintenance deficiencies that need to be addressed by  the contractor or the agency, and the effect of deficiencies on savings.  However, the DOE and Corps contract vehicles do not explicitly require  contractors to provide estimates of expected cost and energy savings that  have not been achieved due to factors beyond contractors’ control.
Most reports we reviewed did not contain information that would allow us  to estimate the amount of savings that were not achieved because of  agency actions. According to FEMP documentation, most contractors’  measurement and verification reports describe performance issues  related to agency actions, without providing information on the magnitude  of the effect on cost savings. During the course of our review, FEMP  drafted guidance for reporting on cost savings that are affected by factors  beyond contractors’ control. Specifically, the guidance includes tables to  be added to measurement and verification reports to provide specific  information on cost savings that are not achieved due to agency actions  and on the net cost savings to agencies from the projects after accounting  for the effects of these actions. However, FEMP had not provided this  guidance to agencies or incorporated it into DOE’s contract vehicle as of  December 2014. Without revising the reporting requirements in the DOE  and Corps contract vehicles to incorporate the updated guidance for  future contracts or providing the guidance to agencies, agencies may  continue to be unaware of the scale of savings that are not achieved, and  may therefore be unable to determine what corrective actions should be  taken. projects that have already been implemented under existing contracts,  their oversight of ongoing projects could be limited unless they work with  contractors to determine the best way to obtain such information.
DOE and the Corps could make such revisions during the planned process of  recompeting the contract vehicles. DOE issued a solicitation for a new contract vehicle in  March 2015, and officials said they plan to award the contract vehicle in early 2016. Corps  officials said they plan to award the Corps’ new contract vehicle by June 2015.
section_title: Agencies’ Oversight and Evaluation of ESPC Projects Is Limited
paragraphs: The seven agencies in our review have conducted limited oversight and  evaluation of their ESPC projects. Specifically, none of the agencies fully  implemented FEMP guidance regarding observing contractors’  measurement and verification activities or reviewing and certifying  contractors’ measurement and verification reports for individual ESPC  projects. Moreover, most of the agencies in our review have not  systematically evaluated their ESPC portfolios to determine the effects of  changing circumstances—such as facility use, utility prices, or interest  rates—on project performance because they do not have processes in  place to do so.
subsections: section_title: Selected Agencies Did Not Fully Implement FEMP Guidance on Project Oversight
paragraphs: Our review of a nongeneralizable sample of 20 ESPC projects across the  seven selected agencies found that agencies did not fully implement  FEMP’s guidance for observing contractors’ measurement and  verification activities or document that the agency had reviewed and  certified contractor’s most recent measurement and verification reports.  In 2007, FEMP issued guidance that identified practices to assist  agencies with overseeing contractors’ measurement and verification  activities. The guidance states, among other things, that an agency  representative should observe the contractor’s measurement and  verification activities, review the contractor’s measurement and  verification report, and certify in writing that the report is acceptable to the  agency. According to FEMP’s guidance, these activities are designed, in  part, to provide the agency assurance that the project is performing as  expected and to provide increased confidence that the expected savings  are achieved. FEMP has also issued guidance that provides a framework  for reviewing postinstallation and annual measurement and verification  reports and includes a template that agencies can use to document their  review of these reports.recommended, and, in some cases, required in agencies’ own guidance.
These oversight activities are also  More specifically, five of the seven selected agencies recommend or  require that agency representatives observe contractors’ measurement  and verification activities, and two agencies require that agency  representatives review the measurement and verification report and  certify acceptance of the report.
Agency representatives observed the contractors’ measurement and  verification activities for all energy conservation measures for 9 of the 20  projects in our nongeneralizable sample; observed measurement and  verification activities for some, but not all, energy conservation measures  for 4 projects; and did not observe these activities for any energy  conservation measures for 7 projects. Additionally, agency officials had  not reviewed the most recent measurement and verification report for 4 of  the 20 projects in our sample and did not certify acceptance of the report  for 11 projects. According to project officials, review was in process for 3  of the 4 reports that had not been reviewed, and officials were in the  process of approving reports for 5 of the 11 projects for which acceptance  was not certified at the time of our review.
Other audit agencies have also identified problems associated with  agency representatives’ observing of contractors’ measurement and  verification activities, reviewing reports, or certifying acceptance of the  reports. For example, a 2011 Naval Audit Service report found that  oversight practices were not sufficiently formalized to ensure that  contractors’ measurement and verification reports were reviewed by Navy  personnel and made 11 recommendations based on its findings. In  2013, the Naval Audit Service conducted a follow-up audit and found that,  among other things, Navy management did not provide sufficient  oversight to ensure that Navy personnel fully completed and clearly  stated on the standard measurement and verification review template  whether Navy personnel observed contractors’ measurement and  verification activities.
Similar findings regarding insufficient oversight  were also included in audit reports from the DOE Inspector General and  the Air Force Audit Agency.
Some project and agency officials told us that agency representatives did  not observe some measurement and verification activities or review and  approve the contractors’ reports because they were unaware of these  duties—or the steps they are supposed to take to perform them—or  believed them unnecessary. According to FEMP officials we interviewed  in December 2014, FEMP has expanded training related to ESPCs, some  of which discusses oversight activities. The officials also stated that there  was no specific training course dedicated to performing agency oversight  and that there would be benefits to having such a course. In commenting  on our report, DOE officials stated that FEMP hosted a webinar in  September 2014 that discussed agencies’ responsibilities during the  performance period. Additionally, DOE stated in its comments that the  webinar included a review of FEMP’s guidance on observing contractors’  measurement and verification activities and reviewing and certifying the  measurement and verification reports, among other issues. DOD officials  we interviewed in December 2014 suggested having additional training on  oversight; however, it is unclear whether they were aware of the webinar.  Because DOE provided information on the webinar late in our review, we  did not assess the webinar, but we believe, based on the analysis  conducted for this review, that issues related to training may have been a  factor in agencies’ inconsistent oversight of contractors’ measurement  and verification activities that we found in our sample of ESPC projects.  According to federal standards for internal control, all personnel need to  possess and maintain a level of competence that allows them to  accomplish their assigned duties, and management needs to identify  appropriate knowledge and skills needed for various jobs and provide  needed training. information needed to understand how to perform their oversight  responsibilities, agencies may continue to inconsistently perform these  oversight responsibilities. As a result, agencies may not be aware of  whether ESPC projects are achieving the expected savings.
GAO/AIMD-00-21.3.1.
FEMP officials said they were aware that agency officials are not always  observing the contractor’s measurement and verification activities or  reviewing and certifying the reports for all projects, but they do not know  the extent to which such oversight activities are occurring. The officials  said FEMP’s Life of Contract program, established in 2009, was an  attempt to ensure that agencies carry out their oversight responsibilities.  Under the program, FEMP calls agencies twice a year—once before  measurement and verification is supposed to be performed by the  contractor and once after measurement and verification has occurred—to  ensure that agencies have the assistance they need to perform their  oversight responsibilities. However, FEMP officials said they do not  know the extent to which agencies have witnessed the contractor’s  measurement and verification activities or reviewed and certified the  contractors’ measurement and verification reports because they do not  monitor whether agencies have carried out these oversight  responsibilities. According to the federal standards for internal control,  internal controls should generally be designed to assure ongoing  monitoring of their performance over time, and any identified deficiencies  should be communicated and corrected.relying on calls made through the Life of Contract program to ensure that  the oversight takes place, but that monitoring whether agencies  performed the oversight would be useful in light of ongoing concerns  about oversight. Because FEMP does not monitor whether agencies are  observing contractors’ measurement and verification activities and  reviewing and certifying contractors’ measurement and verification  reports, FEMP does not know whether its Life of Contract program or its  guidance is effective and cannot identify deficiencies, if any, in the  program or its guidance that need to be corrected.
section_title: Most Agencies Have Not Systematically Evaluated the Effects of Changing Circumstances on the Performance of Their ESPC Projects
paragraphs: Estimating future savings is inherently uncertain, and given the length of  ESPCs—those awarded under DOE’s contract vehicle last 17 years on  average and can last as long as 25 years—changes are likely to occur in  utility prices, agency mission needs, and other factors that affect cost and  energy savings. However, most of the seven selected agencies in our  review have not systematically evaluated their ESPC portfolios to  determine the effects of changing circumstances—such as facility use,  utility prices, or interest rates—on project performance, because they do  not have processes in place to do so.
One agency in our review, the Air Force, evaluated its ESPC portfolio  from 2009 through 2011, but has not established a process for agency- wide portfolio evaluations going forward. During this evaluation, the Air  Force identified over 50 projects that it determined were not economical  due to facility closures, high interest rates, or minimal measurement and  verification requirements, among other issues. In addition, during the  course of our review, FEMP established a process that would allow it to  identify changes in agencies’ use of energy conservation measures and  associated facilities and other agency actions that could negatively affect  savings for ESPCs awarded under the DOE contract vehicle. This  process is intended to help FEMP better advise and oversee agencies  implementing ESPCs. However, the process does not include comparing  expected energy prices to actual prices, or comparing interest rates for  ESPC projects to current market rates.
GAO/AIMD-00-21.3.1. or other project characteristics that have high potential for savings in  future projects. Conversely, if an agency determines that an ESPC  project’s achieved savings are less than expected, the agency can use  the information to inform decisions about future projects. Officials from  some agencies said such reviews could be tied to specific triggers. For  example, agencies could conduct reviews after a certain number of years,  or in response to specific events, such as changes in utility prices or  market interest rates, or appropriations becoming available that could be  used for terminations.
Officials at some agencies said that staff at project sites are generally  aware of performance deficiencies and savings shortfalls of their  individual projects. However, most agencies in our review did not have  processes in place for agency-wide reviews of ESPCs’ performance.  Without systematically reviewing agency-wide ESPC performance, such  as by reevaluating baseline assumptions in light of changing energy  prices or use of facilities, agency officials cannot make fully-informed  decisions about their portfolios of projects. For instance, limited  information on ESPC performance could hinder agencies’ ESPC program  managers in planning future ESPCs, and it could hinder facility managers  in determining how best to utilize facilities and operate and maintain  conservation measures. In addition, to the extent that changes beyond  contractors’ control cause projects not to achieve their guaranteed  savings, agencies’ payments to contractors may be greater than the  reductions in agencies’ utility costs, even though FEMP guidance states  that agencies must achieve savings that exceed payments to the  contractor. Furthermore, because contractor payments must come from  agency funds used to pay for energy, water, and related expenses,  contractor payments that exceed achieved energy, water, and related  savings will limit the funds agencies can use to cover these expenses.
section_title: Conclusions
paragraphs: Agencies have used ESPCs in a variety of ways and plan to continue to  do so to help meet various energy-related goals. However, some agency  officials are hesitant to develop projects to consolidate federal data  centers—which consume large amounts of energy—because the law  does not specify and OMB has not clarified its position on what qualifies  as energy-related savings and the allowable proportion of energy and  energy-related cost savings with regard to scoring ESPCs. OMB’s  position on these issues is important because it determines whether an  agency would need to obligate funding for the entire contract up front in  the first year of the contract or annually throughout the life of the contract.  Unless OMB clarifies its position on these issues, consistent with federal  standards for internal control, agencies may needlessly forego  opportunities to reduce energy consumption by developing ESPCs to  consolidate data centers.
Having access to the information needed to fully understand the cost and  energy savings that projects are—or are not—achieving is a key aspect in  overseeing ESPCs. Contractors reported some cost and energy savings  that were not achieved due to agency actions for 14 of the 20 projects in  our sample. Contracts typically do not require contractors to reduce the  amount of savings they report in such cases, but FEMP guidance, as well  as the DOE and Corps contract vehicles, encourages contractors to  identify deficiencies that may lead to savings that are not achieved.  However, unless DOE and the Corps revise their contract vehicles or  provide agencies with updated guidance that requires contractors to  provide estimates of cost and energy savings that are not achieved  because of agencies’ actions, agencies may not be able to identify the  extent to which expected savings are not achieved. In addition, even if  DOE and the Corps change the language in the contract vehicles to  require contractors to provide estimates of cost and energy savings that  are not achieved, the changes would likely not affect the contract  requirements for ongoing projects. To obtain this information for ongoing  projects, agencies could, for example, work with the contractors for  individual projects to determine the best way to obtain this information.  Without this information, agencies may not be able to determine what, if  any, corrective actions they should take. Further, changes in energy  markets in recent years have affected utility prices, but DOE has not  updated its analysis of utility prices for projects under its contract vehicle  since 2007. Without information on the accuracy of the assumptions  about utility rates, agencies may not have the information they need to  know whether their projects are achieving expected savings.
Agencies have implemented some changes to increase the oversight of  ESPC projects, such as establishing or strengthening central offices to  help manage ESPC projects. However, for the projects we reviewed,  agencies did not always implement practices identified in FEMP guidance  for overseeing the contractors’ measurement and verification activities.  Specifically, agencies did not consistently observe the contractors’  measurement and verification activities, review the most recent  measurement and verification reports, or certify that the reports were  acceptable to the agency. In some cases, officials did not know they were  responsible for this oversight or thought that it was not necessary, in part  because they may not have received specific training on this oversight.  Without ensuring that training provides officials with the information  needed to understand how to perform their oversight responsibilities,  agencies may continue to inconsistently perform their oversight  responsibilities. As a result, agencies may not be aware of whether ESPC  projects are achieving the expected savings. FEMP designed its Life of  Contract program to help agency officials carry out oversight called for in  ESPC guidance, but FEMP does not monitor whether agency officials are  witnessing contractors’ measurement and verification activities or  reviewing and certifying the contractors’ measurement and verification  report, as called for in guidance. Without such monitoring, FEMP does not  have information necessary to identify any deficiencies that need to be  corrected in its Life of Contract program or its guidance.
Furthermore, most agencies we reviewed have not systematically  evaluated the effects of changes to certain circumstances, like facility  use, utility prices, or interest rates, on their portfolios of ESPC projects  because they do not have processes in place to do so. Estimating future  savings is inherently uncertain and, if assumptions about facility use or  utility prices are not accurate, then agencies could be paying more for  projects than they are saving. There are challenges to and drawbacks of  frequent reviews. However, such evaluations could be tied to specific  triggers, such as passage of a certain number of years or certain events  such as changes in utility prices, market interest rates, or appropriations  becoming available that could be used for modifications or terminations. If  agencies do not systematically review the performance of ESPC projects  agency-wide compared with the assumptions developed when the  contract was signed, agency officials may be unaware of how changing  circumstances have affected the performance of their ESPCs and cannot  make fully-informed decisions about how to best strategically manage  their ESPC portfolios.
section_title: Recommendations for Executive Action
paragraphs: We are making six recommendations to help improve the oversight of  agencies’ ESPC projects.
To help agencies decide whether to use ESPCs to consolidate federal  data centers, we recommend that the Director of OMB document, for the  purposes of scoring ESPCs, (1) what qualifies as energy-related savings  and (2) the allowable proportion of energy and energy-related cost  savings.
To help ensure that agencies have sufficient information on ESPC  performance to oversee whether future and current contracts are  achieving their expected savings, we recommend that  the Secretaries of Defense and Energy specify in the scheduled  revisions to their ESPC contract vehicles or in guidance to agencies  that measurement and verification reports for future projects are to  include estimates of cost and energy savings that were not achieved  because of agency actions. Additionally, DOE may wish to consider  periodically analyzing data on other factors that may affect savings,  such as utility prices, to provide information on how savings achieved  by ESPCs awarded through its contract vehicle have been affected by  changing utility prices since its prior study in 2007.  the Secretaries of Defense, Energy, and Veterans Affairs; the  Attorney General; and the Administrator of the General Services  Administration work with contractors to determine the best way to  obtain estimates of cost and energy savings that are not achieved  because of agency actions in order to include these estimates in  future measurement and verification reports for existing contracts, in  accordance with DOE guidance, and where economically feasible.
To help agencies more consistently perform their oversight  responsibilities and oversee contractors’ measurement and verification  activities, we recommend that the Secretary of Energy direct FEMP to evaluate existing training and determine whether additional training is  needed on observing contractors’ measurement and verification  activities and reviewing and certifying measurement and verification  reports, and monitor agencies’ oversight of ESPC projects that agencies have  awarded using the DOE contract vehicle, including whether agencies  witnessed the contractors’ measurement and verification activities and  reviewed and certified acceptance of the measurement and  verification report.
To help ensure that agencies have sufficient information on the effects of  changing circumstances on the performance of their ESPC portfolios, we  recommend that the Secretaries of Defense, Energy, and Veterans  Affairs; the Attorney General; and the Administrator of the General  Services Administration establish a process to systematically evaluate  their ESPC projects—including baseline assumptions about facilities’  energy use, utility prices, and interest rates—to determine how their  ESPC portfolios are performing and the extent to which they are  achieving expected savings. Agencies could consider conducting such  evaluations either after a certain number of years, or in response to  events, such as changes in utility prices or market interest rates, or  appropriations becoming available that could be used for modifications or  terminations.
section_title: Agency Comments and Our Evaluation
paragraphs: We provided a draft of this report to the agencies in our review—DOD,  DOE, Justice, VA, and GSA. We also provided a draft to OMB. DOD,  DOE, VA, and GSA provided written comments, which are reproduced in  appendixes VII through X, respectively. DOD, DOE, Justice, GSA, and  OMB provided technical comments, which we incorporated, as  appropriate. Justice and GSA concurred with our findings and  recommendations, and the other agencies provided specific comments on  our findings and recommendations, which we discuss in more detail  below.
OMB did not comment on our first recommendation, which originally  called on OMB to clarify certain information about using ESPCs to  consolidate federal data centers. However, DOE commented that it has  the authority to administer the ESPC program and issue guidance  accordingly and that OMB issues guidance on the budget scoring  treatment of ESPCs. We agree with these statements and have clarified  our recommendation to specify that it pertains to OMB’s scoring of  ESPCs.
DOD concurred with our second recommendation, related to requiring  that measurement and verification reports for future contracts contain  estimates of savings that are not achieved due to various factors beyond  contractors’ control. DOE partially concurred with the recommendation.  DOE agreed that factors such as physical changes to buildings, which  were not contemplated prior to the contract, should be verified by annual  measurement and verification activities. DOE stated that FEMP is  addressing these issues through a revision of its measurement and  verification reporting template. Further, DOE said that FEMP will  investigate using the revised reporting template in future contracts. We  modified the recommendation to allow for use of an alternative  mechanism, such as the template, to implement the requirement. DOE  also stated that methodologies for dealing with risks, such as changes in  utility prices, are incorporated in the measurement and verification plan  that is part of the contract for each project. DOE stated that further action  is not warranted for factors, such as changes in utility prices, that are  beyond contractor and agency control because their variability is  accounted for at the time of contract formation. According to DOE,  attempting to evaluate the impact of such factors on savings would be  potentially costly and burdensome to agencies and contractors and would  have little benefit. Furthermore, DOE stated that there is evidence, based  on a 2007 Oak Ridge National Laboratory study, that ESPC projects have  underestimated utility prices and have achieved greater overall savings  than contractors reported. We recognize DOE’s concerns and have   modified our report and recommendation to focus on estimating savings  that were not achieved due to agency actions. Additionally, we have  modified the recommendation to include that DOE consider periodically  analyzing the impacts of utility prices on ESPC savings, given significant  changes in energy markets since Oak Ridge’s 2007 study.
DOD and DOE partially concurred, and VA did not concur, with our third  recommendation about working with contractors to determine the best  way to obtain estimates of savings that are not achieved for existing  contracts. DOD and VA suggested changes to the wording of the  recommendation, which we have incorporated. In its comments, DOE  reiterated that agencies would benefit from verifying factors, such as  physical changes to buildings that were not contemplated prior to contract  implementation, through annual measurement and verification activities.  DOE also stated that FEMP is addressing these issues through revision  of its measurement and verification reporting template and will investigate  the use of the revised reporting template for existing contracts. DOE  reiterated its concerns about reporting savings that were not achieved  due to factors, such as changes in utility prices, that are beyond  contractor and agency control. We recognize this concern, as discussed  in our response to the second recommendation above. We have modified  our report and revised the wording of the recommendation to (1) focus on  estimating savings that were not achieved due to agency actions, (2)  more clearly indicate that the estimates are to be obtained for inclusion in  future measurement and verification reports for existing contracts, and (3)  limit its implementation to instances where it is economically feasible.
DOE partially concurred with our fourth recommendation about providing  training on certain oversight activities. In December 2014, we interviewed  DOE officials, including FEMP program managers, about available  training pertaining to agencies’ oversight responsibilities. These officials  told us that there was no specific course dedicated to performing agency  oversight and that such a course would be beneficial. The need for  additional training on oversight activities was also suggested by DOD  officials during our review. In commenting on our report, however, DOE  stated that, in September 2014, FEMP added to its training courses a  webinar that addressed agency responsibilities for oversight during the  contract performance period. DOE also stated that FEMP would examine  available training and resources; make updates, as appropriate; and  investigate how to encourage their use among agencies. We have noted  this new information in the body of the report. However, because DOE  provided the information on the webinar late in our review, we did not  assess the webinar cited by DOE. We continue to believe, based on the  analysis conducted for this review, that issues related to training may  have been a factor in agencies’ inconsistent oversight of contractors’  measurement and verification activities that we found in our sample of  ESPC projects. We are encouraged by DOE’s plans and have modified  our recommendation to include evaluating its existing training and  determining whether additional training on oversight is needed.
DOE concurred with our fifth recommendation related to FEMP  monitoring of agencies’ oversight of ESPC projects awarded using the  DOE contract vehicle. DOE stated that FEMP will examine its Life of the  Contract program for an improved means of quantifying agencies’  compliance in observing measurement and verification activities and  reviewing and certifying the resulting reports.
DOD concurred with our sixth recommendation about establishing a  process to systematically evaluate its ESPC portfolio. DOE and VA  partially concurred with this recommendation. In its comments, DOE  stated that FEMP would review its process that addresses performance  issues and its process for engaging with agencies to determine whether  to modify or terminate a contract. DOE stated that its review process  includes evaluating cost savings that are not achieved as a result of  agency actions and evaluating interest rates to assist agencies in  determining the potential cost savings available through refinancing. In its  comments, VA concurred in principle with the recommendation but stated  that the agency would be limited in how it could use information obtained  from such evaluations and that the evaluations would not provide  significant value relative to the time and money required to conduct them.  We have modified our recommendation to be less prescriptive about how  the information is to be used. We continue to believe, as VA stated in its  comments, that such evaluations could make agency officials aware of  how changing circumstances have affected ESPC performance.  Moreover, our recommendation allows for agencies to consider  conducting such evaluations after a certain number of years or in  response to events, which should decrease the burden of such  evaluations.
In its technical comments, DOE stated that it disagreed with our use of  the term "actual savings," and that actual savings would be better  characterized as "the cost and energy savings that contractors measure  and verify in accordance with the plan the agency agreed to when  developing and awarding the contract." DOE stated doing so is consistent  with the ESPC authority, which authorizes a methodology to determine  energy savings using models and assumptions that the federal agency  and contractor agree on prior to contract formation. DOE also stated that  there are factors that could affect savings that cannot be known and that  analyzing only known factors will produce a skewed analysis of "actual"  savings. We acknowledge these limitations. However, we found that  savings that contractors reported, in accordance with the plan the agency  agreed to, sometimes included savings that were not achieved because  of agency actions, such as physical changes to buildings. DOE agrees  that such factors, which were not contemplated prior to contract  formation, should be verified by annual measurement and verification  activities. We continue to believe that it is important for agencies to obtain  information from contractors on savings that are not achieved because of  agency actions and have modified our report to discuss “achieved  savings” instead of “actual savings.”
In DOD’s technical comments, it provided some missing data for the  amount the Army awarded in ESPCs for fiscal years 2011, 2012, and  2014. DOD also provided updated guaranteed costs savings data for  some Army projects after it submitted comments on the draft report. We  updated the report with these data. DOD also requested that we list DOD  as the agency in tables 2 and 4 and list the Air Force, Army, and Navy as  components, but we did not do so because we did not include other DOD  components in the scope of our audit, and we wanted to highlight the  details specific to the Air Force, Army, and Navy. We recognize that the  Air Force, Army, and Navy are components of DOD, and we acknowledge  this in the beginning of the report.
We are sending copies of this report to the appropriate congressional  committees and the Secretaries of Defense, Energy, and Veterans  Affairs; the Attorney General; the Administrator of the General Services  Administration; and the Director of the Office of Management and Budget.
In addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or ruscof@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. Key contributors to this report are listed in  appendix XI.
section_title: Appendix I: Objectives, Scope, and Methodology
paragraphs: We were asked to review federal use of energy savings performance  contracts (ESPC) and process changes since 2005. This report examines  the extent to which (1) selected agencies have used ESPCs and plan to  use them in the future, (2) selected agencies’ ESPC projects have  achieved their expected cost and energy savings, and (3) selected  agencies have overseen and evaluated their ESPC projects.
To determine which federal agencies to include in our review, we selected  agencies with the highest energy usage and greatest facility square  footage, based on government-wide data collected by the Federal Energy  Management Program (FEMP). We chose the following seven agencies  based on the above criteria the Departments of Energy (DOE), Justice,  and Veterans Affairs (VA); the General Services Administration (GSA);  and the Army, Navy, and Air Force within the Department of Defense  (DOD). We refer to these agencies as the seven selected agencies. As of  fiscal year 2013, DOD, DOE, Justice, VA, and GSA represented 78  percent of the federal government’s total floor space and 80 percent of  the government’s energy use. Findings based on these agencies cannot  be generalized to other agencies. To provide information on all of our  objectives, we interviewed knowledgeable agency officials, reviewed  relevant agency and contractor reports, and conducted site visits to ESPC  projects in Golden, Colorado, and White Oak, Maryland. We selected  these sites based on whether they were undertaken by federal agencies  within our review; innovativeness, such as use of newer technology; and  proximity to locations of GAO staff. Findings from these site visits cannot  be generalized to other projects.
To determine the extent to which selected agencies used ESPCs, we  collected and analyzed available data on ESPCs awarded in fiscal years  1995 through 2014. We found that there is no source of comprehensive  data on federal agencies’ use of ESPCs, either in DOE, the contracting  centers, or the agencies. The seven selected agencies started collecting  data comprehensively and electronically at different points in time, and  they keep some contract data only in project files at the facilities where  the contracts are being implemented. We combined agencies’ available  data into the most consistent format available, deleted duplicate records,  performed basic tests to determine the reliability of the data, reviewed  existing information about the data and the systems that produced them,  and interviewed agency officials knowledgeable about the data. We found  that selected agencies were missing some data, but we found the data  used in this report to be sufficiently reliable for our purposes. We  compiled data into the following fields: project title, contractor, contract  vehicle, award date, agency, implementation price, total contract price,  guaranteed savings, contract term length, and annual energy savings. If  agencies did not provide data that were defined in the same way as other  agency data, we used the most comparable data available. We used  FEMP’s data as our primary ESPC data source for all seven selected  agencies, and we supplemented it with Air Force, Army, GSA, and Navy  data. Because data from Justice and VA on ESPCs were not sufficiently  reliable for the purposes of this report, we relied on FEMP’s data on these  agencies. To determine the extent to which agencies plan to use ESPCs  and challenges faced when using ESPCs to consolidate data centers, we  reviewed relevant federal laws, executive orders, the President’s  Performance Contracting Challenge, the seven selected agencies’ fiscal  year 2014 Strategic Sustainability Performance Plans, and Office of  Management and Budget guidance.
To determine the extent to which selected agencies’ ESPCs projects  have achieved their expected cost and energy savings, we reviewed six  annual studies by DOE’s Oak Ridge National Laboratory that analyzed  cost and energy savings reported by contractors in annual measurement  and verification reports for ESPCs awarded under DOE’s contract  vehicle. These ESPCs represent about 70 percent of federal ESPCs  awarded since 1995 by total contract value. Oak Ridge National  Laboratory’s first annual study was issued in 2007 and reflected savings  reported by contractors in calendar year 2005, and its most recent annual  study was issued in 2013 and it reflected savings reported by contractors  in calendar year 2012. Oak Ridge National Laboratory did not issue an  annual study for savings reported in 2006 or 2007. In addition, the years  for which savings were reported were approximated based on the  average start and end dates of the reporting periods covered by the  annual measurement and verification reports included in Oak Ridge  National Laboratory’s analysis. For instance, the reports included in the  2012 analysis had an average start date of January 4, 2012, and an  average end date of January 5, 2013, for an approximate reporting period  of calendar year 2012. Oak Ridge National Laboratory’s studies included  ESPCs for projects that were in their performance period and for which  the contractor had produced at least one measurement and verification  report in the year before the study. Projects in the planning or  construction phases, first year of the performance period, or  postperformance period were not reflected in a given year’s study. We  reviewed Oak Ridge National Laboratory’s methodology for these studies,  interviewed the authors of the studies, and determined the findings of the  studies were sufficiently reliable for purposes of our report. We did not  analyze trends in reported savings for ESPCs awarded through the  Corps’ contract vehicle because the Corps had not centrally tracked or  analyzed reported savings for these ESPCs.
In addition, to provide illustrative examples of the extent to which selected  agencies’ ESPC projects have achieved their expected savings, we  reviewed annual measurement and verification reports submitted by  contractors and other project documentation for a nongeneralizable  sample of 20 ESPC projects, with a total contract value of about $824  million. (See app. VI for a list of projects that we selected.) We selected  projects from among the 530 projects listed in DOE, Corps, and agency  data on ESPCs awarded by the seven selected agencies in fiscal years  1995 through 2014. We selected projects that reflected a range of award  dates, contract values, and other characteristics. We selected at least one  project at each of the seven selected agencies, and more projects at  agencies that had awarded more ESPCs. Our review was generally  limited to projects that had completed at least 1 year of the performance  period for which an annual measurement and verification report was  submitted. The one exception was VA’s Veterans Integrated Service  Network 22, Greater Los Angeles project, which was in the first year of its  performance period at the time of our review, and it did not yet have an  annual measurement and verification report. However, because VA did  not award any ESPCs in 2004 through 2011, this was the only VA project  available for our sample that was in its performance period. Therefore, in  order to include VA in our sample, we reviewed the postinstallation  measurement and verification report for the project, which included  information on projected savings for the first year of the performance  period based on measurement and verification activities conducted after  project installation. For all 20 projects in our sample, we reviewed  measurement and verification reports and other documentation to identify  instances where contractors noted changes in the performance or  operation of equipment that could have affected the savings they  generated. We also reviewed information in the documents on projected  utility rates. We contacted agency officials directly involved with the  projects to obtain additional information, such as estimates of the savings  that are not achieved due to changes in equipment performance or  operation, the reasons for those changes, and actual utility rates for the  most recent year. The findings from our review of these projects are not  generalizable to other projects. To inform our review of the projects, we  reviewed FEMP’s measurement and verification guidance, which  includes information on procedures and guidelines for quantifying the  savings resulting from ESPCs, and is intended for agency staff and  contractors. We also reviewed supplemental measurement and  verification guidance from the seven selected agencies in our review and  interviewed officials from these agencies regarding their processes for  measuring and verifying ESPC savings.
To determine the extent to which selected agencies have overseen and  evaluated ESPC projects, we reviewed and analyzed annual  measurement and verification reports submitted by contractors and other  project documentation for a nongeneralizable sample of 20 ESPCs at the  seven selected agencies to determine the extent that agencies observed  the contractor’s measurement and verification activities and reviewed and  approved the latest measurement and verification report. We conducted  follow-up inquiries with agency officials to obtain any missing data in the  project files. We also interviewed FEMP and other agency officials about  the results associated with the sample projects. We also reviewed audit  agency reports conducted on ESPCs since 2005. Furthermore, we  interviewed agency officials about internal procedures for evaluating  agency ESPC projects and analyzed agency documents related to these  evaluations.
We conducted this performance audit from March 2014 to June 2015 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.
section_title: Appendix II: Value of Annually Awarded ESPCs by Selected Agencies, in Fiscal Years 1995 through 2014
subsections: section_title: Year 1995
section_title: Appendix III: Federal Agencies’ Awarded and Planned Contracts for the President’s Performance Contracting Challenge
paragraphs: Table 5 shows the amount agencies have awarded in performance-based  contracts, including energy savings performance contracts (ESPC) and  utility energy service contracts, and the amounts that agencies plan to  award by December 2016. Planned awards have not yet been awarded,  which means these data are subject to change and contracts might not be  awarded.
section_title: Appendix IV: Budgetary Treatment of ESPCs
paragraphs: In recent years, members of Congress and industry officials have raised  questions about how energy savings performance contracts’ (ESPC)  costs and savings should be reflected in the federal budget. The full  amount of the government’s financial commitment under an ESPC is not  reflected—”scored”—up front in the budget when the contract is signed.  Moreover, federal budget agencies disagree about whether this should be  the case. The Office of Management and Budget’s (OMB) scoring  treatment is based on the contingent nature of the contract—payments  are contingent on achieving expected cost savings and, therefore, the  government is not fully committed to the entire long-term cost of the  ESPC at the time it is signed. Under OMB’s scoring treatment, an agency  must obligate, at the time the contract is executed, sufficient budgetary  resources to cover the agency’s contract payments for the fiscal year in  which the contract is signed. For each subsequent fiscal year during the  contract period, the agency must obligate funds to cover the contract  payments the agency is required to make for that year. OMB has not  changed its approach to scoring ESPCs since it first issued formal  guidance in 1998, and OMB staff said they have no plans to do so. The  Congressional Budget Office (CBO), on the other hand, scores the full  cost of ESPCs up front in its cost estimates of legislation authorizing  agencies to enter into ESPCs. It views this treatment as consistent with  government-wide principles that the budget should reflect the  government’s full commitment at the time decisions are made. In the case  of an ESPC, this means a new obligation would be made at the time the  ESPC is signed. CBO’s cost estimates of legislation authorizing agencies  to enter into ESPCs reflect the annual net effects of such obligations in  the current fiscal year and 10 subsequent years.
CBO has developed several cost estimates for legislation affecting  ESPCs and, in a recent estimate, changed how it reflects the cost savings  that may result from ESPCs. estimate for legislation that, among other things, expanded the definition  of allowable energy conservation measures under an ESPC, CBO  showed an increase in spending resulting from increased ESPC use.
CBO, S. 1321 Energy Savings Act of 2007 (Washington, D.C.: June 11, 2007); CBO, S.  761 Energy Savings and Industrial Competitiveness Act of 2013 (Washington, D.C.: May  21, 2013); CBO, H.R. 2689 Energy Savings Through Public-Private Partnerships Act of  2014 (Washington, D.C.: Sept. 24, 2014).
CBO estimated that contractual commitments to pay vendors for energy  conservation measures implemented pursuant to the legislation would  amount to $450 million over 10 years. This treatment is consistent with  CBO’s previous estimates of ESPC-related legislation. However, unlike  previous estimates for such legislation, the 2014 cost estimate also  factored in reductions in spending due to anticipated reductions in energy  costs. CBO estimated that reductions in federal costs attributable to  contracts implemented pursuant to the legislation would total $210 million  over 10 years, with additional reductions in subsequent years. In addition,  CBO issued a report in February 2015 with further information on its new  scoring treatment of ESPCs, including how it accounts for reductions in  agencies’ energy costs.
section_title: Appendix V: Oak Ridge National Laboratory Analysis of Reported Savings for ESPCs Awarded through DOE’s Contract Vehicle
paragraphs: The following tables provide information on the reported energy and cost  savings for energy savings performance contracts (ESPC) awarded  through the Department of Energy’s (DOE) contract vehicle, based on  analysis by DOE’s Oak Ridge National Laboratory. Oak Ridge National  Laboratory has issued six studies of contractor-reported savings for  ESPCs awarded through DOE’s contract vehicle. Table 6 shows the  reported and guaranteed cost savings for these ESPCs in the 6 years  analyzed by Oak Ridge.
Table 7 shows the reported and proposed energy savings for ESPCs  awarded through DOE’s contract vehicle in the 6 years analyzed by Oak  Ridge National Laboratory.
Table 8 provides information on the extent to which contractors reported  cost and energy savings below expected amounts—also referred to as  savings shortfalls—for ESPCs awarded through DOE’s contract vehicle in  the 6 years analyzed by Oak Ridge National Laboratory.
section_title: Appendix VI: Effects of Agency Actions on Savings for Selected ESPC Projects
paragraphs: Table 9 shows the energy savings performance contract (ESPC) projects  we reviewed, and the agency actions that affected savings and resulted in  overstatements of reported savings in the most recent measurement and  verification report.
section_title: Appendix VII: Comments from the Department of Defense
section_title: Appendix VIII: Comments from the Department of Energy
section_title: Appendix IX: Comments from the Department of Veterans Affairs
section_title: Appendix X: Comments from the General Services Administration
section_title: Appendix XI: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Staff Acknowledgments
paragraphs: In addition to the individual named above, Hilary Benedict (Assistant  Director), Joshua Becker, John Delicath, Keesha Egebrecht, Cindy  Gilbert, Carol Henn, Miles Ingram, John Johnson, Brian Lepore, Armetha  Liles, Cynthia Norris, Barbara Timmerman, and Bill Woods made  significant contributions to this report.