Energy Savings Performance Contracts: Additional Actions Needed to Improve Federal Oversight
Released: 2015-06-17T00:00:00

Letter

Background

Several key laws and an executive order directly relate to federal agencies’ use of ESPCs (see table 1).

In addition, in December 2011, the President challenged federal agencies to enter into $2 billion in performance-based contracts, including ESPCs and utility energy service contracts through the President’s Performance Contracting Challenge. In May 2014, the President expanded this challenge to total $4 billion in performance-based contracts by the end of 2016.

The process that agencies and contractors generally follow for developing and implementing an ESPC project spans five phases, from acquisition planning—during which agencies identify project requirements and assemble their acquisition team—to project performance—during which energy conservation measures are in place and operating, and agencies pay contractors. Figure 1 illustrates the general process for developing and implementing an ESPC project.

During the process of developing and implementing an ESPC project, agency officials often work with DOE’s or the U.S. Army Corps of Engineers’ (Corps) federal contracting centers. Both DOE and the Corps have awarded indefinite-delivery, indefinite-quantity ESPC contract vehicles to a set of prequalified energy services contractors. Agencies using these “umbrella” contract vehicles can award an ESPC for an individual project to any of the prequalified contractors. Using one of these contract vehicles allows agencies to develop and implement an ESPC project in less time because the process of competitively selecting qualified contractors has already been completed, and key aspects of contracts have been broadly negotiated. In addition, both DOE and the Corps provide contracting and technical support to agencies that use their contract vehicles. For example, DOE’s FEMP provides facilitation services, where a third party assists the agency and contractor in agreeing on the terms of a contract. FEMP also issues guidance and offers training for agencies on the various steps of developing and implementing an ESPC project. The Corps provides technical support, cost estimating services, and legal support to agencies using its contract vehicle. As part of revisions that DOE and the Corps made to their contract vehicles in 2008, both DOE and the Corps now require that agencies use a qualified project facilitator when developing and implementing an ESPC, which addresses the recommendation in our June 2005 report to ensure that agencies use appropriate expertise when undertaking an ESPC. Additionally, each of the seven agencies in our review has established a central office to support individual sites with developing and implementing ESPC projects, and several of these agencies have increased the role of their central offices since our last review to provide additional support, such as oversight of ESPCs.

An ESPC project’s expected cost and energy savings are established during project development, finalized when the contract is awarded, and measured and verified over the course of a project’s performance period. These savings can include reductions in costs for energy, water, operation and maintenance, and repair and replacement directly related to the energy conservation measures. Agencies must pay contractors from funds appropriated or otherwise made available to pay for such utilities and related operational expenses. Payments to contractors generally cover the costs associated with equipment and installation, contractor-provided operation and maintenance services, financing charges, and other costs. ESPC projects generally include two types of expected savings: (1) proposed cost and energy savings, which contractors estimate will result from the energy conservation measures installed, and (2) guaranteed cost savings, which must be achieved for Generally, contractors guarantee about the contractor to be fully paid.95 percent of a project’s proposed cost savings, which gives them room for some amount of proposed savings to not be achieved without a reduction in their payments.

Energy and cost savings are the difference between a projected baseline of the energy use without the energy conservation measures and with the measures. The process used to determine ESPC savings is referred to as measurement and verification. Most ESPC projects include the following four key documents that outline how cost and energy savings are to be measured and verified:

Measurement and verification plan. During the project development phase, the contractor and agency develop a plan that establishes how to measure and verify that savings are achieved. Measurement and verification methods can include surveys, inspections, direct measurements of energy use, and other activities to ensure that equipment is operating correctly and has the potential to generate expected savings.

Risk and responsibility matrix. During the contractor selection phase, the contractor and agency develop a risk and responsibility matrix that identifies key project risks and their potential effects, and specifies whether the agency or the contractor will be responsible for managing financial risks, such as changing interest rates; operational risks, such as operating hours and weather; and performance risks, such as equipment performance and preventative maintenance.

Postinstallation measurement and verification report. After the energy conservation measures are installed, the contractor conducts measurement and verification activities and presents the results in a postinstallation measurement and verification report.

Annual measurement and verification report. Throughout an ESPC’s performance period, the contractor conducts measurement and verification activities and submits an annual report to the agency to document the cost and energy savings achieved.

According to FEMP guidance, one of the primary purposes of measurement and verification is to reduce the risk that expected savings will not be achieved. FEMP guidance describes risks related to (1) equipment use, which stem from uncertainty in operational factors, such as the number of hours equipment is used or changes in the planned operation of equipment, and (2) equipment performance, which stem from uncertainty in projecting a specified level of performance. Contractors are usually reluctant to assume risks related to equipment use because they often have no control over operational factors. In contrast, according to FEMP guidance, the contractor is ultimately responsible for the selection, design, and installation of equipment and typically assumes responsibility for performance risks.

FEMP guidance outlines a range of options that contractors may select to measure and verify the cost and energy savings achieved by each energy conservation measure. If certain factors that affect savings, such as weather conditions, utility prices, and hours of agency operation, are either too complex or costly to measure, agency and contractors may choose to agree in advance on—or stipulate—the values for those factors regardless of the actual behavior of those factors. For example, because ESPCs can be long contracts, the contractor and agency typically stipulate escalation rates to estimate future utility prices during the performance period. If the savings that are achieved are less than the savings calculated using stipulated values, the agency pays the contractor for the savings calculated using stipulated values. If achieved savings are greater than the savings calculated using stipulated values, the agency retains the additional savings.

The measurement and verification options that FEMP guidance outlines vary in their rigor and costs. The option that is generally the least rigorous and costly involves measuring the key factors affecting energy use—such as the number of lighting fixtures or efficiency of a heating unit—before and after installation, but typically does not involve measuring such factors over the term of the contract. In contrast, other options outlined in FEMP guidance generally involve ongoing measurements of energy use, or proxies of energy use, over the contract term. FEMP guidance helps identify when each option should be used and states that the selection of a measurement and verification method is based on project costs and savings, complexity of the energy conservation measure, and the uncertainty or risk of savings being achieved, among others. According to FEMP guidance, costs for measurement and verification generally increase with the level of accuracy required in energy savings analyses and the number and complexity of variables that are analyzed, among other factors. Moreover, the incremental value of additional measurement and verification will at some point be less than its cost. For instance, the energy consumed by a light fixture does not change appreciably over time, and requiring contractors to measure fixtures annually would increase the cost of measurement and verification for little benefit, according to FEMP officials.

Selected Agencies Have Awarded About $12.1 Billion for ESPCs for a Variety of Projects, and Their Plans for Continued Use Vary

In fiscal years 1995 through 2014, the seven selected agencies in our review awarded approximately $12.1 billion in ESPCs for a variety of projects, such as constructing biomass facilities to heat federal buildings. According to agency officials and documents, agencies plan to continue using ESPCs to meet federal energy directives and initiatives, but some agency officials said they are hesitant to use ESPCs to consolidate data centers.

Selected Agencies Awarded About $12.1 Billion for ESPCs in Fiscal Years 1995 through 2014

In fiscal years 1995 through 2014, the seven selected agencies awarded approximately $12.1 billion for more than 500 ESPC projects to help fund energy conservation measures in federal facilities. The total amount awarded in ESPCs varied by agency, with the Army awarding the most— approximately $3.7 billion. (See fig. 2.) Data on the ESPCs awarded by each agency are included in appendix II.

The seven selected agencies awarded approximately 530 ESPCs in fiscal years 1995 through 2014. The length of the contracts for these projects ranged from approximately 2 to 25 years, with an average of about 16 years. Additionally, the projects for ESPCs awarded during this period had a total guaranteed cost savings of roughly $12.4 billion and total proposed energy savings of approximately 563 trillion British thermal units (Btu).energy savings by agency.

Agencies Have Used ESPCs for a Variety of Projects

The seven agencies have used ESPCs for a variety of projects ranging from smaller-scale projects to install more energy-efficient light bulbs or water flow restrictors in toilets, to larger-scale projects, such as power generation projects. For example, GSA officials used ESPCs for three projects at its White Oak, Maryland, facility to install infrastructure and equipment with cogeneration capabilities, which involves the simultaneous production of electricity and heat from a single fuel source, such as natural gas. Figure 3 shows some of the White Oak cogeneration project components. Additionally, DOE installed a biomass facility at the National Renewable Energy Laboratory in Golden, Colorado. The biomass facility, the first of its kind for DOE, according to project officials, uses wood chips from forest thinnings and trees killed by either beetles or fire as fuel to generate heat that warms water for buildings at the campus. Figure 3 shows some of the components of DOE’s biomass facility.

Some agencies have started to use ESPCs to develop larger and more comprehensive projects to try to achieve greater cost and energy savings.

For example, in 2012, GSA began using ESPCs for its National Deep Energy Retrofit program which, according to an analysis by the Oak Ridge National Laboratory, achieved an average level of savings more than twice that of other federal ESPC projects. Furthermore, to help achieve these cost and energy savings, agencies have increasingly turned to bundling energy conservation measures together under an ESPC, which is more efficient than using separate contracts, according to FEMP officials.

Agencies’ Plans for Using ESPCs in the Future Vary, Particularly for Data Center Consolidation Projects

Selected agencies’ plans to use ESPCs in the future vary. Officials from five of the agencies we spoke with said their plans for continuing to use ESPCs will help their agencies meet goals in federal executive orders and other energy goals, including the President’s Performance Contracting Challenge. For example, in response to federal energy goals, Army officials said they plan to aggressively pursue using ESPCs, among other financing options, to improve energy efficiency. Justice officials said they plan to extensively use ESPCs at all of their Bureau of Prisons sites to upgrade and repair many buildings that have aging infrastructure. VA officials said ESPCs are one of many tools to meet energy goals and the agency prioritizes ESPCs at all facilities where feasible. With regard to the President’s Performance Contracting Challenge, agencies government-wide had awarded approximately $1.9 billion in performance- based contracts out of the $4 billion goal as of January 2015, as shown in table 3, with the seven selected agencies in our review awarding most of these contracts. If agencies award the contracts they currently have planned, they will meet the Challenge’s goal of awarding $4 billion in performance-based contracts by the end of 2016 (see app. III for federal agencies’ status in achieving their goals under the President’s Performance Contracting Challenge).

Some agency officials we interviewed said they are interested in using ESPCs to consolidate data centers, which consume significant amounts of energy and can be costly to operate, but the agency officials are hesitant to move forward with such projects because of concerns OMB staff have raised about using ESPCs for such projects. By law, ESPCs must be used “solely for the purpose of achieving energy savings and benefits ancillary to that purpose.” However, the law does not specify what qualifies as ancillary benefits—also referred to as energy-related savings—or the proportion of an ESPC’s overall savings that can be energy-related. OMB guidance on federal use of performance contracts outlines some general criteria that projects must meet to be scored under OMB’s annual budget scoring process but does not provide specific guidance on energy-related savings. If an ESPC project does not meet OMB’s criteria, then to pursue the project the agency would need to obligate funding for the entire contract “up front” in its first year, rather than annually. This can be an issue because agencies might not have the funding for the entire contract during its first year, which would leave agencies with the option of either canceling the contract or moving funding from other agency efforts.

According to DOE officials, they nearly completed the project development phase of the ESPC development and implementation process in May 2011 for a project to use an ESPC to consolidate data centers. However, they delayed awarding the contract in March 2013 because OMB staff raised concerns about the project. DOE officials said the project, as originally proposed, would consolidate two data centers and replace 5,000 desktop computers with computers that are more energy efficient. The project was expected to save DOE approximately $76 million, and 97 percent of the overall cost savings would come from operations and maintenance such as maintaining computer hardware and software, or energy-related savings, and the remaining 3 percent from energy savings. According to DOE officials, the concerns that OMB staff raised included (1) whether savings resulting from more efficient information technology equipment qualify as energy-related savings and (2) the project’s high proportion of cost savings resulting from the reduction in operations and maintenance costs, rather than energy cost savings. At the time of our review, DOE had resumed consideration of the project and had not awarded a contract, but said that OMB staff had not clarified their position regarding their concerns.

According to Army officials, the Army is also interested in using ESPCs to consolidate data centers, but they are hesitant to move forward with any projects because they have heard about OMB’s concerns and are waiting to learn OMB’s position regarding DOE’s data center consolidation. Army officials said they have not seen any information on OMB’s position officially released, which they said they need before pursuing the use of ESPCs for data center consolidation projects. Furthermore, DOD officials who oversee Army and other DOD agencies said the agencies need clarification on whether moving data to a more energy efficient off-site storage facility (rather than storing it on servers in DOD facilities) or eliminating help desk support and software licenses would qualify as energy-related savings under OMB guidance.

According to federal standards for internal control, information should be communicated to those who need it in a form and within a time frame that Because OMB staff have enables them to carry out their responsibilities.expressed concerns about but have not clarified their position on what qualifies as energy-related savings and the allowable proportion of energy and energy-related cost savings, DOE delayed its data center consolidation project and some agencies, such as the Army, have been hesitant to pursue using ESPCs for such projects. As a result, agencies might be needlessly missing opportunities for potential energy and energy-related cost savings. We solicited OMB staff’s comments during our review regarding their position on DOE’s data center consolidation ESPC project, as well as the use of ESPCs for data center consolidation projects generally, regarding what qualifies as energy-related savings and the proportion of cost savings resulting from operations and maintenance and energy use reduction. In response, OMB staff said, in part, that it is generally not appropriate for them to comment on the merits of specific contracts.

Reported Savings Generally Exceeded Expectations, but Some Savings for Selected ESPC Projects Were Overstated

The cost and energy savings that contractors reported for most ESPCs met or exceeded expected savings, according to studies by DOE’s Oak Ridge National Laboratory, but some of these savings may be overstated. Our review of a nongeneralizable sample of 20 projects found that contractors overstated cost and energy savings for 14 projects by reporting some savings that, due to agency actions, were not achieved. Contractors must calculate and report savings in accordance with plans agreed to in their contracts with agencies. If factors beyond contractors’ control reduce the savings achieved, contractors generally are not required to reduce the amount of savings they report or measure the effects of such factors on savings. Agencies were not always aware of the amount of expected savings that were not achieved among their projects, in part, because contractors generally do not provide this information in measurement and verification reports.

Reported Cost and Energy Savings for Most ESPCs Met or Exceeded Expected Savings

DOE’s Oak Ridge National Laboratory found in its six studies of contractor-reported savings for agencies that awarded ESPCs through DOE’s contract vehicle, that the total cost and energy savings reported for these ESPCs exceeded their expected savings. The total cost savings reported in the 6 years of annual measurement and verification reports that Oak Ridge analyzed was about 106 percent of the total guaranteed cost savings for these ESPCs. Moreover, in each of the 6 years, total reported cost savings across all projects were at least 105 percent of total guaranteed savings. Similarly, the Oak Ridge studies found that the total energy savings reported for ESPCs awarded through DOE’s contract vehicle exceeded proposed energy savings. Specifically, the total energy savings reported in all of the annual measurement and verification reports analyzed over the 6 years was about 102 percent of the total proposed energy savings for these ESPCs. Moreover, in each of the 6 years, total reported energy savings across all projects were at least equal to total proposed savings.

Most contractors reported cost savings that exceeded guaranteed savings, but the Oak Ridge studies found that some contractors reported cost savings below guaranteed amounts, also referred to as cost savings shortfalls, in about 6 percent of the reports that Oak Ridge reviewed. The average shortfall in cost savings for the small number of ESPCs with a reported shortfall was 17 percent, meaning reported cost savings were 83 percent of guaranteed amounts for these ESPCs. However, these shortfalls ranged widely, from 0.5 percent to 75 percent of guaranteed cost savings. Appendix V provides additional information from the Oak Ridge studies.

Similarly, we found that reported savings for 19 of the 20 projects in the nongeneralizable sample we reviewed met or exceeded their guaranteed cost savings for the year reviewed. The remaining project, DOE’s National Renewable Energy Laboratory, had a reported cost savings shortfall of about $76,000—about 18 percent of its guaranteed cost savings—in its most recent measurement and verification report. According to the project’s measurement and verification report, the shortfall was primarily due to warmer weather, which reduced the number of days the equipment was used, and there was an outage due to a failed motor. In the measurement and verification report, the contractor identified planned changes to the equipment that are expected to address the performance deficiencies and savings shortfalls.

FEMP tracks cost savings shortfalls monthly for ESPCs that agencies have awarded through DOE’s contract vehicle with its “dashboard report” and has found that contractors are reporting that most ESPC projects are meeting or exceeding their guaranteed savings. For instance, a sample dashboard report from 2014 showed nine projects had reported cost savings shortfalls, ranging from less than 1 percent of guaranteed savings to more than 30 percent of guaranteed savings. The dashboard report includes details on the reasons for shortfalls and actions for FEMP to take to help agencies address shortfalls. According to FEMP officials, the dashboard report, which FEMP developed in 2007, provides FEMP management with a snapshot of key aspects of the ESPC projects and has enabled them to more effectively monitor ESPC projects, which GAO recommended in 2005.

Most Selected Measurement and Verification Reports We Reviewed Overstated Some Cost and Energy Savings

Measurement and verification reports for 14 of the 20 projects in the nongeneralizable sample we reviewed—including projects from each of the seven selected agencies—overstated some cost and energy savings in that they reported savings that were not achieved. Contractors must calculate and report annual savings in accordance with the measurement and verification plans agreed to in their contracts with agencies. These plans include measuring equipment performance. They also include assumptions about factors that are beyond contractors’ control, such as agencies’ use of energy-saving equipment and utility prices, which may change over the life of the contract. If changes in such factors reduce savings, contractors generally are not required to reduce the amount of savings they report or measure the effects of such changes. For example, contractors do not generally reduce the savings they report when an agency alters the agreed upon hours of operation, thus reducing the number of hours that energy-saving equipment is used. Conversely, if savings increase because of changes in factors beyond contractors’ control, contractors generally do not increase the amount of savings they report, and agencies generally retain any surplus savings and do not increase payments to the contractor.

Measurement and verification reports for 14 projects in our sample overstated some cost and energy savings in that they reported savings that were not achieved because of agencies’ actions, including (1) agencies not operating or maintaining equipment as agreed when the ESPC was awarded and (2) agencies’ removal of equipment from or closure of facilities where energy conservation measures had been installed. For projects in our sample, contractors’ reports generally did not quantify or estimate the effects of these factors on savings, although some reports noted that savings were affected in some way. Because of the large number of factors that can result in overstated or understated savings, we did not determine the net effect of all factors on projects’ achieved savings. For example, some energy conservation measures in the projects we reviewed outperformed expectations, which may have offset the lower-than-expected savings of other energy conservation measures in those projects. Table 4 shows the projects we reviewed and the agencies’ actions that affected savings that were reported in the most recent measurement and verification report, as of September 2014. (For further detail on the effects of these factors on savings for these projects, see app. VI.)

The following are examples from the ESPC projects we reviewed of agency actions that resulted in reported savings that were not achieved:

Agency did not operate or maintain equipment as agreed The most common factor resulting in overstated savings for the ESPC projects we reviewed was an agency making changes to operating hours and temperature set points on programmable heating, ventilation, and air conditioning (HVAC) equipment, which occurred in 8 of the 20 projects. According to available agency estimates, these changes generally resulted in lower energy and associated cost savings than expected, but contractors did not reflect these effects in reported savings amounts because they were due to agency actions. In other cases, agencies did not fulfill their responsibilities for operating or maintaining equipment. For instance, the contractor for a project at a Justice facility found that steam distribution equipment the contractor installed had been damaged, reducing the savings the equipment achieved. However, the contractor did not reduce reported savings because it stated that the damage resulted from improper operation by Justice staff. In some cases, agencies took actions that reduced savings, such as changing operating hours or temperature set points, to meet changing agency mission needs.

Agency removed or abandoned equipment components of energy conservation measures or entire measures were removed by the agency during the performance period, but contractors did not reduce reported savings because these changes were due to agency decisions. For instance, the Army closed a section of an installation that had numerous buildings with energy conservation measure equipment installed. As a result, savings were not being generated by this equipment, but the contractor reported the savings that would have been achieved for the year had the equipment continued to operate. In some cases, agencies removed or abandoned equipment to meet changing agency mission needs.

The amount of savings reported but not achieved ranged from negligible to nearly half of an ESPC project’s reported savings for the year, based on information provided by agencies and our analysis of available information from the most recent measurement and verification reports for selected projects. For example, where estimates were available, agency changes to operating hours and temperature set points on programmable HVAC equipment generally resulted in savings that were reported but not achieved that were negligible as a percentage of the total savings reported, according to agency officials. In contrast, the Air Force’s removal of equipment associated with a sewer system upgrade resulted in over $104,000 in annual savings that were reported but not achieved— about 40 percent of the annual savings reported for the project. (For a full list of the projects we reviewed and information on the effects of factors beyond contractors’ control on savings, see app. VI.) Officials from several agencies noted that there are benefits to funding energy conservation projects through ESPCs, as opposed to using up-front appropriations. The officials noted that like ESPC projects, the expected savings for projects funded with up-front appropriations may not be achieved. However, savings that are not achieved are more likely to be identified for ESPC projects because savings must be measured and verified.

Unlike changes in agencies’ use of equipment, agencies cannot control changes in utility prices, but changes in utility prices compared with the amounts stipulated in the contracts could affect the savings for ESPC projects. Agencies commonly stipulate annual escalation rates for energy costs based on projected utility prices published by the National Institute of Standards and Technology and developed by DOE’s Energy Information Administration. DOE has reported that the projected utility prices for ESPCs awarded through its contract vehicle have generally underestimated the actual increase in utility prices, and therefore ESPC projects are generally saving more than expected. Specifically, in 2007, DOE’s Oak Ridge National Laboratory analyzed 22 ESPC projects to calculate savings using actual, rather than projected, utility prices. After adjusting for actual utility prices, savings for 16 of the 22 ESPC projects Oak Ridge examined were greater than the savings contractors reported, while savings for the remaining 6 ESPC projects were lower than reported.

Energy markets have changed significantly since 2007, and are likely to change in the future. For example, improvements in horizontal drilling and hydraulic fracturing led to large increases in the production of natural gas from shale formations, which contributed to significant decreases in the price of natural gas. Such changes likely affected the savings that certain ESPC projects achieved, such as those whose savings were based predominantly on reductions in natural gas use. However, it is not clear whether the assumptions that agencies are using for utility prices are reasonable because DOE has not conducted an analysis of ESPC projects awarded under its contract vehicle since its 2007 report. As a result, agencies may not have the information they need to know whether their projects are achieving expected savings and achieved savings may be significantly different—either higher or lower—than reported savings. There are drawbacks of assumptions about utility prices being consistently higher or lower than actual rates. DOE guidance states that stipulating higher utility rates, which generally results in higher expected savings, will provide better cash-flow for projects. However, the guidance also states that overvaluing savings is a serious concern that can cause budgetary problems for the agency. This is because contractor payments must come from agency funds used to pay for energy, water, and related expenses. Therefore, contractor payments that exceed achieved energy, water, and related savings will limit the funds agencies can use to cover these expenses.

We identified three projects in our nongeneralizable sample for which achieved savings were lower than reported savings because utility prices differed from those stipulated in the contract. Specifically, for one DOE project and one Justice project, natural gas prices were significantly lower than the amounts stipulated in the contracts, which led to achieved savings that were about $147,000 and $477,000 less than the reported cost savings for the year, respectively. These amounts represented about 44 percent of the reported cost savings for the DOE project and about 30 percent of the reported cost savings for the Justice project. Additionally, an Air Force project that involved switching inefficient oil heating units to natural gas units projected rates for natural gas that did not reflect seasonal price increases in winter months. Because actual natural gas prices were substantially higher than projected in the winter, the costs of running the new natural gas units were higher than projected. This resulted in achieved savings that were about $160,000 less than the reported savings for the year, which was about 5 percent of the project’s total reported savings for the year. Utility prices vary from year to year, so it is to be expected that prices will differ from the stipulated values in some years. Without a periodic analysis of utility prices over several years and across projects, agencies may not have the information they need to know whether examples like the three in our sample are typical and indicative of problems with the assumptions or anomalies.

Agencies Were Not Always Aware of How Much Expected Savings Were Not Achieved

Agencies were not always aware of the amount of expected savings that were not achieved, in part because contractors generally did not—and were not required to—report this information when savings were not achieved due to agency actions. FEMP guidance states that when reviewing measurement and verification reports, agencies should understand changes in project performance and savings levels from year to year, and what corrective actions should be taken to address deficiencies resulting in savings that are not achieved. In addition, the DOE and Corps contract vehicles provide an outline for contractors to use in writing annual reports, which includes sections detailing performance, operating, and maintenance deficiencies that need to be addressed by the contractor or the agency, and the effect of deficiencies on savings. However, the DOE and Corps contract vehicles do not explicitly require contractors to provide estimates of expected cost and energy savings that have not been achieved due to factors beyond contractors’ control.

Most reports we reviewed did not contain information that would allow us to estimate the amount of savings that were not achieved because of agency actions. According to FEMP documentation, most contractors’ measurement and verification reports describe performance issues related to agency actions, without providing information on the magnitude of the effect on cost savings. During the course of our review, FEMP drafted guidance for reporting on cost savings that are affected by factors beyond contractors’ control. Specifically, the guidance includes tables to be added to measurement and verification reports to provide specific information on cost savings that are not achieved due to agency actions and on the net cost savings to agencies from the projects after accounting for the effects of these actions. However, FEMP had not provided this guidance to agencies or incorporated it into DOE’s contract vehicle as of December 2014. Without revising the reporting requirements in the DOE and Corps contract vehicles to incorporate the updated guidance for future contracts or providing the guidance to agencies, agencies may continue to be unaware of the scale of savings that are not achieved, and may therefore be unable to determine what corrective actions should be taken. projects that have already been implemented under existing contracts, their oversight of ongoing projects could be limited unless they work with contractors to determine the best way to obtain such information.

DOE and the Corps could make such revisions during the planned process of recompeting the contract vehicles. DOE issued a solicitation for a new contract vehicle in March 2015, and officials said they plan to award the contract vehicle in early 2016. Corps officials said they plan to award the Corps’ new contract vehicle by June 2015.

Agencies’ Oversight and Evaluation of ESPC Projects Is Limited

The seven agencies in our review have conducted limited oversight and evaluation of their ESPC projects. Specifically, none of the agencies fully implemented FEMP guidance regarding observing contractors’ measurement and verification activities or reviewing and certifying contractors’ measurement and verification reports for individual ESPC projects. Moreover, most of the agencies in our review have not systematically evaluated their ESPC portfolios to determine the effects of changing circumstances—such as facility use, utility prices, or interest rates—on project performance because they do not have processes in place to do so.

Selected Agencies Did Not Fully Implement FEMP Guidance on Project Oversight

Our review of a nongeneralizable sample of 20 ESPC projects across the seven selected agencies found that agencies did not fully implement FEMP’s guidance for observing contractors’ measurement and verification activities or document that the agency had reviewed and certified contractor’s most recent measurement and verification reports. In 2007, FEMP issued guidance that identified practices to assist agencies with overseeing contractors’ measurement and verification activities. The guidance states, among other things, that an agency representative should observe the contractor’s measurement and verification activities, review the contractor’s measurement and verification report, and certify in writing that the report is acceptable to the agency. According to FEMP’s guidance, these activities are designed, in part, to provide the agency assurance that the project is performing as expected and to provide increased confidence that the expected savings are achieved. FEMP has also issued guidance that provides a framework for reviewing postinstallation and annual measurement and verification reports and includes a template that agencies can use to document their review of these reports.recommended, and, in some cases, required in agencies’ own guidance.

These oversight activities are also More specifically, five of the seven selected agencies recommend or require that agency representatives observe contractors’ measurement and verification activities, and two agencies require that agency representatives review the measurement and verification report and certify acceptance of the report.

Agency representatives observed the contractors’ measurement and verification activities for all energy conservation measures for 9 of the 20 projects in our nongeneralizable sample; observed measurement and verification activities for some, but not all, energy conservation measures for 4 projects; and did not observe these activities for any energy conservation measures for 7 projects. Additionally, agency officials had not reviewed the most recent measurement and verification report for 4 of the 20 projects in our sample and did not certify acceptance of the report for 11 projects. According to project officials, review was in process for 3 of the 4 reports that had not been reviewed, and officials were in the process of approving reports for 5 of the 11 projects for which acceptance was not certified at the time of our review.

Other audit agencies have also identified problems associated with agency representatives’ observing of contractors’ measurement and verification activities, reviewing reports, or certifying acceptance of the reports. For example, a 2011 Naval Audit Service report found that oversight practices were not sufficiently formalized to ensure that contractors’ measurement and verification reports were reviewed by Navy personnel and made 11 recommendations based on its findings. In 2013, the Naval Audit Service conducted a follow-up audit and found that, among other things, Navy management did not provide sufficient oversight to ensure that Navy personnel fully completed and clearly stated on the standard measurement and verification review template whether Navy personnel observed contractors’ measurement and verification activities.

Similar findings regarding insufficient oversight were also included in audit reports from the DOE Inspector General and the Air Force Audit Agency.

Some project and agency officials told us that agency representatives did not observe some measurement and verification activities or review and approve the contractors’ reports because they were unaware of these duties—or the steps they are supposed to take to perform them—or believed them unnecessary. According to FEMP officials we interviewed in December 2014, FEMP has expanded training related to ESPCs, some of which discusses oversight activities. The officials also stated that there was no specific training course dedicated to performing agency oversight and that there would be benefits to having such a course. In commenting on our report, DOE officials stated that FEMP hosted a webinar in September 2014 that discussed agencies’ responsibilities during the performance period. Additionally, DOE stated in its comments that the webinar included a review of FEMP’s guidance on observing contractors’ measurement and verification activities and reviewing and certifying the measurement and verification reports, among other issues. DOD officials we interviewed in December 2014 suggested having additional training on oversight; however, it is unclear whether they were aware of the webinar. Because DOE provided information on the webinar late in our review, we did not assess the webinar, but we believe, based on the analysis conducted for this review, that issues related to training may have been a factor in agencies’ inconsistent oversight of contractors’ measurement and verification activities that we found in our sample of ESPC projects. According to federal standards for internal control, all personnel need to possess and maintain a level of competence that allows them to accomplish their assigned duties, and management needs to identify appropriate knowledge and skills needed for various jobs and provide needed training. information needed to understand how to perform their oversight responsibilities, agencies may continue to inconsistently perform these oversight responsibilities. As a result, agencies may not be aware of whether ESPC projects are achieving the expected savings.

GAO/AIMD-00-21.3.1.

FEMP officials said they were aware that agency officials are not always observing the contractor’s measurement and verification activities or reviewing and certifying the reports for all projects, but they do not know the extent to which such oversight activities are occurring. The officials said FEMP’s Life of Contract program, established in 2009, was an attempt to ensure that agencies carry out their oversight responsibilities. Under the program, FEMP calls agencies twice a year—once before measurement and verification is supposed to be performed by the contractor and once after measurement and verification has occurred—to ensure that agencies have the assistance they need to perform their oversight responsibilities. However, FEMP officials said they do not know the extent to which agencies have witnessed the contractor’s measurement and verification activities or reviewed and certified the contractors’ measurement and verification reports because they do not monitor whether agencies have carried out these oversight responsibilities. According to the federal standards for internal control, internal controls should generally be designed to assure ongoing monitoring of their performance over time, and any identified deficiencies should be communicated and corrected.relying on calls made through the Life of Contract program to ensure that the oversight takes place, but that monitoring whether agencies performed the oversight would be useful in light of ongoing concerns about oversight. Because FEMP does not monitor whether agencies are observing contractors’ measurement and verification activities and reviewing and certifying contractors’ measurement and verification reports, FEMP does not know whether its Life of Contract program or its guidance is effective and cannot identify deficiencies, if any, in the program or its guidance that need to be corrected.

Most Agencies Have Not Systematically Evaluated the Effects of Changing Circumstances on the Performance of Their ESPC Projects

Estimating future savings is inherently uncertain, and given the length of ESPCs—those awarded under DOE’s contract vehicle last 17 years on average and can last as long as 25 years—changes are likely to occur in utility prices, agency mission needs, and other factors that affect cost and energy savings. However, most of the seven selected agencies in our review have not systematically evaluated their ESPC portfolios to determine the effects of changing circumstances—such as facility use, utility prices, or interest rates—on project performance, because they do not have processes in place to do so.

One agency in our review, the Air Force, evaluated its ESPC portfolio from 2009 through 2011, but has not established a process for agency- wide portfolio evaluations going forward. During this evaluation, the Air Force identified over 50 projects that it determined were not economical due to facility closures, high interest rates, or minimal measurement and verification requirements, among other issues. In addition, during the course of our review, FEMP established a process that would allow it to identify changes in agencies’ use of energy conservation measures and associated facilities and other agency actions that could negatively affect savings for ESPCs awarded under the DOE contract vehicle. This process is intended to help FEMP better advise and oversee agencies implementing ESPCs. However, the process does not include comparing expected energy prices to actual prices, or comparing interest rates for ESPC projects to current market rates.

GAO/AIMD-00-21.3.1. or other project characteristics that have high potential for savings in future projects. Conversely, if an agency determines that an ESPC project’s achieved savings are less than expected, the agency can use the information to inform decisions about future projects. Officials from some agencies said such reviews could be tied to specific triggers. For example, agencies could conduct reviews after a certain number of years, or in response to specific events, such as changes in utility prices or market interest rates, or appropriations becoming available that could be used for terminations.

Officials at some agencies said that staff at project sites are generally aware of performance deficiencies and savings shortfalls of their individual projects. However, most agencies in our review did not have processes in place for agency-wide reviews of ESPCs’ performance. Without systematically reviewing agency-wide ESPC performance, such as by reevaluating baseline assumptions in light of changing energy prices or use of facilities, agency officials cannot make fully-informed decisions about their portfolios of projects. For instance, limited information on ESPC performance could hinder agencies’ ESPC program managers in planning future ESPCs, and it could hinder facility managers in determining how best to utilize facilities and operate and maintain conservation measures. In addition, to the extent that changes beyond contractors’ control cause projects not to achieve their guaranteed savings, agencies’ payments to contractors may be greater than the reductions in agencies’ utility costs, even though FEMP guidance states that agencies must achieve savings that exceed payments to the contractor. Furthermore, because contractor payments must come from agency funds used to pay for energy, water, and related expenses, contractor payments that exceed achieved energy, water, and related savings will limit the funds agencies can use to cover these expenses.

Conclusions

Agencies have used ESPCs in a variety of ways and plan to continue to do so to help meet various energy-related goals. However, some agency officials are hesitant to develop projects to consolidate federal data centers—which consume large amounts of energy—because the law does not specify and OMB has not clarified its position on what qualifies as energy-related savings and the allowable proportion of energy and energy-related cost savings with regard to scoring ESPCs. OMB’s position on these issues is important because it determines whether an agency would need to obligate funding for the entire contract up front in the first year of the contract or annually throughout the life of the contract. Unless OMB clarifies its position on these issues, consistent with federal standards for internal control, agencies may needlessly forego opportunities to reduce energy consumption by developing ESPCs to consolidate data centers.

Having access to the information needed to fully understand the cost and energy savings that projects are—or are not—achieving is a key aspect in overseeing ESPCs. Contractors reported some cost and energy savings that were not achieved due to agency actions for 14 of the 20 projects in our sample. Contracts typically do not require contractors to reduce the amount of savings they report in such cases, but FEMP guidance, as well as the DOE and Corps contract vehicles, encourages contractors to identify deficiencies that may lead to savings that are not achieved. However, unless DOE and the Corps revise their contract vehicles or provide agencies with updated guidance that requires contractors to provide estimates of cost and energy savings that are not achieved because of agencies’ actions, agencies may not be able to identify the extent to which expected savings are not achieved. In addition, even if DOE and the Corps change the language in the contract vehicles to require contractors to provide estimates of cost and energy savings that are not achieved, the changes would likely not affect the contract requirements for ongoing projects. To obtain this information for ongoing projects, agencies could, for example, work with the contractors for individual projects to determine the best way to obtain this information. Without this information, agencies may not be able to determine what, if any, corrective actions they should take. Further, changes in energy markets in recent years have affected utility prices, but DOE has not updated its analysis of utility prices for projects under its contract vehicle since 2007. Without information on the accuracy of the assumptions about utility rates, agencies may not have the information they need to know whether their projects are achieving expected savings.

Agencies have implemented some changes to increase the oversight of ESPC projects, such as establishing or strengthening central offices to help manage ESPC projects. However, for the projects we reviewed, agencies did not always implement practices identified in FEMP guidance for overseeing the contractors’ measurement and verification activities. Specifically, agencies did not consistently observe the contractors’ measurement and verification activities, review the most recent measurement and verification reports, or certify that the reports were acceptable to the agency. In some cases, officials did not know they were responsible for this oversight or thought that it was not necessary, in part because they may not have received specific training on this oversight. Without ensuring that training provides officials with the information needed to understand how to perform their oversight responsibilities, agencies may continue to inconsistently perform their oversight responsibilities. As a result, agencies may not be aware of whether ESPC projects are achieving the expected savings. FEMP designed its Life of Contract program to help agency officials carry out oversight called for in ESPC guidance, but FEMP does not monitor whether agency officials are witnessing contractors’ measurement and verification activities or reviewing and certifying the contractors’ measurement and verification report, as called for in guidance. Without such monitoring, FEMP does not have information necessary to identify any deficiencies that need to be corrected in its Life of Contract program or its guidance.

Furthermore, most agencies we reviewed have not systematically evaluated the effects of changes to certain circumstances, like facility use, utility prices, or interest rates, on their portfolios of ESPC projects because they do not have processes in place to do so. Estimating future savings is inherently uncertain and, if assumptions about facility use or utility prices are not accurate, then agencies could be paying more for projects than they are saving. There are challenges to and drawbacks of frequent reviews. However, such evaluations could be tied to specific triggers, such as passage of a certain number of years or certain events such as changes in utility prices, market interest rates, or appropriations becoming available that could be used for modifications or terminations. If agencies do not systematically review the performance of ESPC projects agency-wide compared with the assumptions developed when the contract was signed, agency officials may be unaware of how changing circumstances have affected the performance of their ESPCs and cannot make fully-informed decisions about how to best strategically manage their ESPC portfolios.

Recommendations for Executive Action

We are making six recommendations to help improve the oversight of agencies’ ESPC projects.

To help agencies decide whether to use ESPCs to consolidate federal data centers, we recommend that the Director of OMB document, for the purposes of scoring ESPCs, (1) what qualifies as energy-related savings and (2) the allowable proportion of energy and energy-related cost savings.

To help ensure that agencies have sufficient information on ESPC performance to oversee whether future and current contracts are achieving their expected savings, we recommend that the Secretaries of Defense and Energy specify in the scheduled revisions to their ESPC contract vehicles or in guidance to agencies that measurement and verification reports for future projects are to include estimates of cost and energy savings that were not achieved because of agency actions. Additionally, DOE may wish to consider periodically analyzing data on other factors that may affect savings, such as utility prices, to provide information on how savings achieved by ESPCs awarded through its contract vehicle have been affected by changing utility prices since its prior study in 2007. the Secretaries of Defense, Energy, and Veterans Affairs; the Attorney General; and the Administrator of the General Services Administration work with contractors to determine the best way to obtain estimates of cost and energy savings that are not achieved because of agency actions in order to include these estimates in future measurement and verification reports for existing contracts, in accordance with DOE guidance, and where economically feasible.

To help agencies more consistently perform their oversight responsibilities and oversee contractors’ measurement and verification activities, we recommend that the Secretary of Energy direct FEMP to evaluate existing training and determine whether additional training is needed on observing contractors’ measurement and verification activities and reviewing and certifying measurement and verification reports, and monitor agencies’ oversight of ESPC projects that agencies have awarded using the DOE contract vehicle, including whether agencies witnessed the contractors’ measurement and verification activities and reviewed and certified acceptance of the measurement and verification report.

To help ensure that agencies have sufficient information on the effects of changing circumstances on the performance of their ESPC portfolios, we recommend that the Secretaries of Defense, Energy, and Veterans Affairs; the Attorney General; and the Administrator of the General Services Administration establish a process to systematically evaluate their ESPC projects—including baseline assumptions about facilities’ energy use, utility prices, and interest rates—to determine how their ESPC portfolios are performing and the extent to which they are achieving expected savings. Agencies could consider conducting such evaluations either after a certain number of years, or in response to events, such as changes in utility prices or market interest rates, or appropriations becoming available that could be used for modifications or terminations.

Agency Comments and Our Evaluation

We provided a draft of this report to the agencies in our review—DOD, DOE, Justice, VA, and GSA. We also provided a draft to OMB. DOD, DOE, VA, and GSA provided written comments, which are reproduced in appendixes VII through X, respectively. DOD, DOE, Justice, GSA, and OMB provided technical comments, which we incorporated, as appropriate. Justice and GSA concurred with our findings and recommendations, and the other agencies provided specific comments on our findings and recommendations, which we discuss in more detail below.

OMB did not comment on our first recommendation, which originally called on OMB to clarify certain information about using ESPCs to consolidate federal data centers. However, DOE commented that it has the authority to administer the ESPC program and issue guidance accordingly and that OMB issues guidance on the budget scoring treatment of ESPCs. We agree with these statements and have clarified our recommendation to specify that it pertains to OMB’s scoring of ESPCs.

DOD concurred with our second recommendation, related to requiring that measurement and verification reports for future contracts contain estimates of savings that are not achieved due to various factors beyond contractors’ control. DOE partially concurred with the recommendation. DOE agreed that factors such as physical changes to buildings, which were not contemplated prior to the contract, should be verified by annual measurement and verification activities. DOE stated that FEMP is addressing these issues through a revision of its measurement and verification reporting template. Further, DOE said that FEMP will investigate using the revised reporting template in future contracts. We modified the recommendation to allow for use of an alternative mechanism, such as the template, to implement the requirement. DOE also stated that methodologies for dealing with risks, such as changes in utility prices, are incorporated in the measurement and verification plan that is part of the contract for each project. DOE stated that further action is not warranted for factors, such as changes in utility prices, that are beyond contractor and agency control because their variability is accounted for at the time of contract formation. According to DOE, attempting to evaluate the impact of such factors on savings would be potentially costly and burdensome to agencies and contractors and would have little benefit. Furthermore, DOE stated that there is evidence, based on a 2007 Oak Ridge National Laboratory study, that ESPC projects have underestimated utility prices and have achieved greater overall savings than contractors reported. We recognize DOE’s concerns and have modified our report and recommendation to focus on estimating savings that were not achieved due to agency actions. Additionally, we have modified the recommendation to include that DOE consider periodically analyzing the impacts of utility prices on ESPC savings, given significant changes in energy markets since Oak Ridge’s 2007 study.

DOD and DOE partially concurred, and VA did not concur, with our third recommendation about working with contractors to determine the best way to obtain estimates of savings that are not achieved for existing contracts. DOD and VA suggested changes to the wording of the recommendation, which we have incorporated. In its comments, DOE reiterated that agencies would benefit from verifying factors, such as physical changes to buildings that were not contemplated prior to contract implementation, through annual measurement and verification activities. DOE also stated that FEMP is addressing these issues through revision of its measurement and verification reporting template and will investigate the use of the revised reporting template for existing contracts. DOE reiterated its concerns about reporting savings that were not achieved due to factors, such as changes in utility prices, that are beyond contractor and agency control. We recognize this concern, as discussed in our response to the second recommendation above. We have modified our report and revised the wording of the recommendation to (1) focus on estimating savings that were not achieved due to agency actions, (2) more clearly indicate that the estimates are to be obtained for inclusion in future measurement and verification reports for existing contracts, and (3) limit its implementation to instances where it is economically feasible.

DOE partially concurred with our fourth recommendation about providing training on certain oversight activities. In December 2014, we interviewed DOE officials, including FEMP program managers, about available training pertaining to agencies’ oversight responsibilities. These officials told us that there was no specific course dedicated to performing agency oversight and that such a course would be beneficial. The need for additional training on oversight activities was also suggested by DOD officials during our review. In commenting on our report, however, DOE stated that, in September 2014, FEMP added to its training courses a webinar that addressed agency responsibilities for oversight during the contract performance period. DOE also stated that FEMP would examine available training and resources; make updates, as appropriate; and investigate how to encourage their use among agencies. We have noted this new information in the body of the report. However, because DOE provided the information on the webinar late in our review, we did not assess the webinar cited by DOE. We continue to believe, based on the analysis conducted for this review, that issues related to training may have been a factor in agencies’ inconsistent oversight of contractors’ measurement and verification activities that we found in our sample of ESPC projects. We are encouraged by DOE’s plans and have modified our recommendation to include evaluating its existing training and determining whether additional training on oversight is needed.

DOE concurred with our fifth recommendation related to FEMP monitoring of agencies’ oversight of ESPC projects awarded using the DOE contract vehicle. DOE stated that FEMP will examine its Life of the Contract program for an improved means of quantifying agencies’ compliance in observing measurement and verification activities and reviewing and certifying the resulting reports.

DOD concurred with our sixth recommendation about establishing a process to systematically evaluate its ESPC portfolio. DOE and VA partially concurred with this recommendation. In its comments, DOE stated that FEMP would review its process that addresses performance issues and its process for engaging with agencies to determine whether to modify or terminate a contract. DOE stated that its review process includes evaluating cost savings that are not achieved as a result of agency actions and evaluating interest rates to assist agencies in determining the potential cost savings available through refinancing. In its comments, VA concurred in principle with the recommendation but stated that the agency would be limited in how it could use information obtained from such evaluations and that the evaluations would not provide significant value relative to the time and money required to conduct them. We have modified our recommendation to be less prescriptive about how the information is to be used. We continue to believe, as VA stated in its comments, that such evaluations could make agency officials aware of how changing circumstances have affected ESPC performance. Moreover, our recommendation allows for agencies to consider conducting such evaluations after a certain number of years or in response to events, which should decrease the burden of such evaluations.

In its technical comments, DOE stated that it disagreed with our use of the term "actual savings," and that actual savings would be better characterized as "the cost and energy savings that contractors measure and verify in accordance with the plan the agency agreed to when developing and awarding the contract." DOE stated doing so is consistent with the ESPC authority, which authorizes a methodology to determine energy savings using models and assumptions that the federal agency and contractor agree on prior to contract formation. DOE also stated that there are factors that could affect savings that cannot be known and that analyzing only known factors will produce a skewed analysis of "actual" savings. We acknowledge these limitations. However, we found that savings that contractors reported, in accordance with the plan the agency agreed to, sometimes included savings that were not achieved because of agency actions, such as physical changes to buildings. DOE agrees that such factors, which were not contemplated prior to contract formation, should be verified by annual measurement and verification activities. We continue to believe that it is important for agencies to obtain information from contractors on savings that are not achieved because of agency actions and have modified our report to discuss “achieved savings” instead of “actual savings.”

In DOD’s technical comments, it provided some missing data for the amount the Army awarded in ESPCs for fiscal years 2011, 2012, and 2014. DOD also provided updated guaranteed costs savings data for some Army projects after it submitted comments on the draft report. We updated the report with these data. DOD also requested that we list DOD as the agency in tables 2 and 4 and list the Air Force, Army, and Navy as components, but we did not do so because we did not include other DOD components in the scope of our audit, and we wanted to highlight the details specific to the Air Force, Army, and Navy. We recognize that the Air Force, Army, and Navy are components of DOD, and we acknowledge this in the beginning of the report.

We are sending copies of this report to the appropriate congressional committees and the Secretaries of Defense, Energy, and Veterans Affairs; the Attorney General; the Administrator of the General Services Administration; and the Director of the Office of Management and Budget.

In addition, the report is available at no charge on the GAO website at http://www.gao.gov.

If you or your staff members have any questions about this report, please contact me at (202) 512-3841 or ruscof@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are listed in appendix XI.

Appendix I: Objectives, Scope, and Methodology

We were asked to review federal use of energy savings performance contracts (ESPC) and process changes since 2005. This report examines the extent to which (1) selected agencies have used ESPCs and plan to use them in the future, (2) selected agencies’ ESPC projects have achieved their expected cost and energy savings, and (3) selected agencies have overseen and evaluated their ESPC projects.

To determine which federal agencies to include in our review, we selected agencies with the highest energy usage and greatest facility square footage, based on government-wide data collected by the Federal Energy Management Program (FEMP). We chose the following seven agencies based on the above criteria the Departments of Energy (DOE), Justice, and Veterans Affairs (VA); the General Services Administration (GSA); and the Army, Navy, and Air Force within the Department of Defense (DOD). We refer to these agencies as the seven selected agencies. As of fiscal year 2013, DOD, DOE, Justice, VA, and GSA represented 78 percent of the federal government’s total floor space and 80 percent of the government’s energy use. Findings based on these agencies cannot be generalized to other agencies. To provide information on all of our objectives, we interviewed knowledgeable agency officials, reviewed relevant agency and contractor reports, and conducted site visits to ESPC projects in Golden, Colorado, and White Oak, Maryland. We selected these sites based on whether they were undertaken by federal agencies within our review; innovativeness, such as use of newer technology; and proximity to locations of GAO staff. Findings from these site visits cannot be generalized to other projects.

To determine the extent to which selected agencies used ESPCs, we collected and analyzed available data on ESPCs awarded in fiscal years 1995 through 2014. We found that there is no source of comprehensive data on federal agencies’ use of ESPCs, either in DOE, the contracting centers, or the agencies. The seven selected agencies started collecting data comprehensively and electronically at different points in time, and they keep some contract data only in project files at the facilities where the contracts are being implemented. We combined agencies’ available data into the most consistent format available, deleted duplicate records, performed basic tests to determine the reliability of the data, reviewed existing information about the data and the systems that produced them, and interviewed agency officials knowledgeable about the data. We found that selected agencies were missing some data, but we found the data used in this report to be sufficiently reliable for our purposes. We compiled data into the following fields: project title, contractor, contract vehicle, award date, agency, implementation price, total contract price, guaranteed savings, contract term length, and annual energy savings. If agencies did not provide data that were defined in the same way as other agency data, we used the most comparable data available. We used FEMP’s data as our primary ESPC data source for all seven selected agencies, and we supplemented it with Air Force, Army, GSA, and Navy data. Because data from Justice and VA on ESPCs were not sufficiently reliable for the purposes of this report, we relied on FEMP’s data on these agencies. To determine the extent to which agencies plan to use ESPCs and challenges faced when using ESPCs to consolidate data centers, we reviewed relevant federal laws, executive orders, the President’s Performance Contracting Challenge, the seven selected agencies’ fiscal year 2014 Strategic Sustainability Performance Plans, and Office of Management and Budget guidance.

To determine the extent to which selected agencies’ ESPCs projects have achieved their expected cost and energy savings, we reviewed six annual studies by DOE’s Oak Ridge National Laboratory that analyzed cost and energy savings reported by contractors in annual measurement and verification reports for ESPCs awarded under DOE’s contract vehicle. These ESPCs represent about 70 percent of federal ESPCs awarded since 1995 by total contract value. Oak Ridge National Laboratory’s first annual study was issued in 2007 and reflected savings reported by contractors in calendar year 2005, and its most recent annual study was issued in 2013 and it reflected savings reported by contractors in calendar year 2012. Oak Ridge National Laboratory did not issue an annual study for savings reported in 2006 or 2007. In addition, the years for which savings were reported were approximated based on the average start and end dates of the reporting periods covered by the annual measurement and verification reports included in Oak Ridge National Laboratory’s analysis. For instance, the reports included in the 2012 analysis had an average start date of January 4, 2012, and an average end date of January 5, 2013, for an approximate reporting period of calendar year 2012. Oak Ridge National Laboratory’s studies included ESPCs for projects that were in their performance period and for which the contractor had produced at least one measurement and verification report in the year before the study. Projects in the planning or construction phases, first year of the performance period, or postperformance period were not reflected in a given year’s study. We reviewed Oak Ridge National Laboratory’s methodology for these studies, interviewed the authors of the studies, and determined the findings of the studies were sufficiently reliable for purposes of our report. We did not analyze trends in reported savings for ESPCs awarded through the Corps’ contract vehicle because the Corps had not centrally tracked or analyzed reported savings for these ESPCs.

In addition, to provide illustrative examples of the extent to which selected agencies’ ESPC projects have achieved their expected savings, we reviewed annual measurement and verification reports submitted by contractors and other project documentation for a nongeneralizable sample of 20 ESPC projects, with a total contract value of about $824 million. (See app. VI for a list of projects that we selected.) We selected projects from among the 530 projects listed in DOE, Corps, and agency data on ESPCs awarded by the seven selected agencies in fiscal years 1995 through 2014. We selected projects that reflected a range of award dates, contract values, and other characteristics. We selected at least one project at each of the seven selected agencies, and more projects at agencies that had awarded more ESPCs. Our review was generally limited to projects that had completed at least 1 year of the performance period for which an annual measurement and verification report was submitted. The one exception was VA’s Veterans Integrated Service Network 22, Greater Los Angeles project, which was in the first year of its performance period at the time of our review, and it did not yet have an annual measurement and verification report. However, because VA did not award any ESPCs in 2004 through 2011, this was the only VA project available for our sample that was in its performance period. Therefore, in order to include VA in our sample, we reviewed the postinstallation measurement and verification report for the project, which included information on projected savings for the first year of the performance period based on measurement and verification activities conducted after project installation. For all 20 projects in our sample, we reviewed measurement and verification reports and other documentation to identify instances where contractors noted changes in the performance or operation of equipment that could have affected the savings they generated. We also reviewed information in the documents on projected utility rates. We contacted agency officials directly involved with the projects to obtain additional information, such as estimates of the savings that are not achieved due to changes in equipment performance or operation, the reasons for those changes, and actual utility rates for the most recent year. The findings from our review of these projects are not generalizable to other projects. To inform our review of the projects, we reviewed FEMP’s measurement and verification guidance, which includes information on procedures and guidelines for quantifying the savings resulting from ESPCs, and is intended for agency staff and contractors. We also reviewed supplemental measurement and verification guidance from the seven selected agencies in our review and interviewed officials from these agencies regarding their processes for measuring and verifying ESPC savings.

To determine the extent to which selected agencies have overseen and evaluated ESPC projects, we reviewed and analyzed annual measurement and verification reports submitted by contractors and other project documentation for a nongeneralizable sample of 20 ESPCs at the seven selected agencies to determine the extent that agencies observed the contractor’s measurement and verification activities and reviewed and approved the latest measurement and verification report. We conducted follow-up inquiries with agency officials to obtain any missing data in the project files. We also interviewed FEMP and other agency officials about the results associated with the sample projects. We also reviewed audit agency reports conducted on ESPCs since 2005. Furthermore, we interviewed agency officials about internal procedures for evaluating agency ESPC projects and analyzed agency documents related to these evaluations.

We conducted this performance audit from March 2014 to June 2015 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Value of Annually Awarded ESPCs by Selected Agencies, in Fiscal Years 1995 through 2014

Year 1995

Appendix III: Federal Agencies’ Awarded and Planned Contracts for the President’s Performance Contracting Challenge

Table 5 shows the amount agencies have awarded in performance-based contracts, including energy savings performance contracts (ESPC) and utility energy service contracts, and the amounts that agencies plan to award by December 2016. Planned awards have not yet been awarded, which means these data are subject to change and contracts might not be awarded.

Appendix IV: Budgetary Treatment of ESPCs

In recent years, members of Congress and industry officials have raised questions about how energy savings performance contracts’ (ESPC) costs and savings should be reflected in the federal budget. The full amount of the government’s financial commitment under an ESPC is not reflected—”scored”—up front in the budget when the contract is signed. Moreover, federal budget agencies disagree about whether this should be the case. The Office of Management and Budget’s (OMB) scoring treatment is based on the contingent nature of the contract—payments are contingent on achieving expected cost savings and, therefore, the government is not fully committed to the entire long-term cost of the ESPC at the time it is signed. Under OMB’s scoring treatment, an agency must obligate, at the time the contract is executed, sufficient budgetary resources to cover the agency’s contract payments for the fiscal year in which the contract is signed. For each subsequent fiscal year during the contract period, the agency must obligate funds to cover the contract payments the agency is required to make for that year. OMB has not changed its approach to scoring ESPCs since it first issued formal guidance in 1998, and OMB staff said they have no plans to do so. The Congressional Budget Office (CBO), on the other hand, scores the full cost of ESPCs up front in its cost estimates of legislation authorizing agencies to enter into ESPCs. It views this treatment as consistent with government-wide principles that the budget should reflect the government’s full commitment at the time decisions are made. In the case of an ESPC, this means a new obligation would be made at the time the ESPC is signed. CBO’s cost estimates of legislation authorizing agencies to enter into ESPCs reflect the annual net effects of such obligations in the current fiscal year and 10 subsequent years.

CBO has developed several cost estimates for legislation affecting ESPCs and, in a recent estimate, changed how it reflects the cost savings that may result from ESPCs. estimate for legislation that, among other things, expanded the definition of allowable energy conservation measures under an ESPC, CBO showed an increase in spending resulting from increased ESPC use.

CBO, S. 1321 Energy Savings Act of 2007 (Washington, D.C.: June 11, 2007); CBO, S. 761 Energy Savings and Industrial Competitiveness Act of 2013 (Washington, D.C.: May 21, 2013); CBO, H.R. 2689 Energy Savings Through Public-Private Partnerships Act of 2014 (Washington, D.C.: Sept. 24, 2014).

CBO estimated that contractual commitments to pay vendors for energy conservation measures implemented pursuant to the legislation would amount to $450 million over 10 years. This treatment is consistent with CBO’s previous estimates of ESPC-related legislation. However, unlike previous estimates for such legislation, the 2014 cost estimate also factored in reductions in spending due to anticipated reductions in energy costs. CBO estimated that reductions in federal costs attributable to contracts implemented pursuant to the legislation would total $210 million over 10 years, with additional reductions in subsequent years. In addition, CBO issued a report in February 2015 with further information on its new scoring treatment of ESPCs, including how it accounts for reductions in agencies’ energy costs.

Appendix V: Oak Ridge National Laboratory Analysis of Reported Savings for ESPCs Awarded through DOE’s Contract Vehicle

The following tables provide information on the reported energy and cost savings for energy savings performance contracts (ESPC) awarded through the Department of Energy’s (DOE) contract vehicle, based on analysis by DOE’s Oak Ridge National Laboratory. Oak Ridge National Laboratory has issued six studies of contractor-reported savings for ESPCs awarded through DOE’s contract vehicle. Table 6 shows the reported and guaranteed cost savings for these ESPCs in the 6 years analyzed by Oak Ridge.

Table 7 shows the reported and proposed energy savings for ESPCs awarded through DOE’s contract vehicle in the 6 years analyzed by Oak Ridge National Laboratory.

Table 8 provides information on the extent to which contractors reported cost and energy savings below expected amounts—also referred to as savings shortfalls—for ESPCs awarded through DOE’s contract vehicle in the 6 years analyzed by Oak Ridge National Laboratory.

Appendix VI: Effects of Agency Actions on Savings for Selected ESPC Projects

Table 9 shows the energy savings performance contract (ESPC) projects we reviewed, and the agency actions that affected savings and resulted in overstatements of reported savings in the most recent measurement and verification report.

Appendix VII: Comments from the Department of Defense

Appendix VIII: Comments from the Department of Energy

Appendix IX: Comments from the Department of Veterans Affairs

Appendix X: Comments from the General Services Administration

Appendix XI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, Hilary Benedict (Assistant Director), Joshua Becker, John Delicath, Keesha Egebrecht, Cindy Gilbert, Carol Henn, Miles Ingram, John Johnson, Brian Lepore, Armetha Liles, Cynthia Norris, Barbara Timmerman, and Bill Woods made significant contributions to this report.