Data Center Consolidation: Reporting Can Be Improved to Reflect Substantial Planned Savings
Released: 2014-09-25T00:00:00

Letter

Background

The federal government’s increasing demand for IT led to a dramatic rise in the number of federal data centers and a corresponding increase in operational costs. According to OMB, the federal government had 432 data centers in 1998 and more than 1,100 in 2009. Operating such a large number of centers has been and continues to be a significant cost to the federal government, including costs for hardware, software, real estate, and cooling. For example, in 2007, the Environmental Protection Agency (EPA) estimated that the electricity cost to operate federal servers and data centers across the government was about $450 million annually. According to the Department of Energy (Energy), data center spaces can consume 100 to 200 times more electricity than a standard that server utilization rates as low office space. In 2009, OMB reportedas 5 percent across the federal government’s estimated 150,000 servers were a driving factor in the need to establish a coordinated, government- wide effort to improve the efficiency, performance, and environmental footprint of federal data center activities.

OMB and the Federal CIO Established the Federal Data Center Consolidation Initiative

Concerned about the size of the federal data center inventory and the potential to improve the efficiency, performance, and the environmental footprint of federal data center activities, OMB, under the direction of the Federal CIO, established FDCCI in February 2010. This initiative’s four high-level goals are to promote the use of “green IT” by reducing the overall energy and real estate footprint of government data centers; reduce the cost of data center hardware, software, and operations; increase the overall IT security posture of the government; and shift IT investments to more efficient computing platforms and technologies.

As part of FDCCI, OMB required the 24 agencies to identify a data center consolidation program manager to lead the agency’s consolidation efforts. In addition, agencies were required to submit an asset inventory baseline and other documents that would result in a plan for consolidating their data centers. The asset inventory baseline was to contain detailed information on each data center and identify the consolidation approach to be taken for each one. It would serve as the foundation for developing the final data center consolidation plan. The data center consolidation plan would serve as a technical road map and approach for achieving the targets for infrastructure utilization, energy efficiency, and cost efficiency and was to be incorporated into the agency’s fiscal year 2012 budget.

In October 2010, OMB reported that all of the agencies had submitted an inventory and plan. In addition, in a series of memorandums, OMB described plans to monitor agencies’ consolidation activities on an ongoing basis. Starting in fiscal year 2011, OMB required agencies to provide an updated data center asset inventory at the end of every third quarter and an updated consolidation plan (including any missing elements) at the end of every fourth quarter. Further, starting in fiscal year 2012, OMB required agencies to provide a consolidation progress report at the end of every quarter. This progress information has subsequently been made available on the federal website dedicated to providing the public with access to datasets developed by federal agencies, http://data.gov.

Pursuant to requirements of the Government Performance and Results Act Modernization Act of 2010 (GPRAMA), in February 2012, OMB designated data center consolidation as 1 of its 14 priority goals (now known as cross-agency priority goals) because of its importance to improving management across the federal government. These goals are designed to cover areas where increased cross-agency collaboration is needed to improve progress towards the achievement of goals shared by multiple contributing agencies. In March 2014, OMB announced the creation of a new set of goals in its submission of the President’s fiscal year 2015 budget, which did not include data center consolidation. According to OMB, although the updated set of goals did not include the data center consolidation goal because it had reached the end of its goal cycle time frame under GPRAMA, the effort will remain an administration priority.

While OMB is primarily responsible for FDCCI, the agency designated the Federal CIO Council—the principal interagency forum to improve IT- related practices across the federal government—to lead the effort. In addition, OMB originally identified two additional organizations to assist in managing and overseeing the initiative:

The GSA FDCCI Program Management Office to support OMB in planning, execution, management, and communications.

The Task Force is comprised of the data center consolidation program managers from each agency. According to its charter, the Task Force is critical to supporting collaboration across agencies, including identifying and disseminating key information, solutions, and processes that will help agencies in their consolidation efforts.

However, in December 2013, GSA and Task Force officials stated that the GSA’s Program Management Office would no longer be supporting FDCCI and its responsibilities were being transitioned to OMB and the Task Force.

With an Expanded Definition, OMB’s Inventory of Federal Data Centers Has Grown

“…a data center is…a closet, room, floor or building for the storage, management, and dissemination of data and information and computer systems and associated components, such as database, application, and storage systems and data stores [excluding facilities exclusively devoted to communications and network equipment (e.g., telephone exchanges and telecommunications rooms)]. A data center generally includes redundant or backup power supplies, redundant data communications connections, environmental controls…and special security devices housed in leased, owned, collocated, or stand-alone facilities.”

Under the first definition, OMB identified 2,094 data centers in July 2010. Using the new definition from October 2011, OMB estimated that there were a total of 3,133 federal data centers in December 2011, and its goal was to consolidate approximately 40 percent, or 1,253 data centers, for a savings of approximately $3 billion by the end of 2015.

OMB, Implementation Guidance for the Federal Data Center Consolidation Initiative (Washington, D.C.: Mar. 19, 2012).

Since 2011, the number of federal data centers reported by agencies has continued to grow. In July 2013, we testified that 22 of the 24 FDCCI agencies had collectively reported 6,836 data centers in their inventories—an increase of about 3,700 compared to OMB’s previous estimate from December 2011. According to the Federal CIO, the increase in data centers was primarily due to the expanded definition of a data center and improved inventory reporting by the agencies.

More recently, our analysis of agencies’ May 2014 data center inventories indicated that agencies collectively reported a total of 9,658 data centers. Of the total reported data centers, 242 were reported by agencies as “core” data centers—meaning that they are primary consolidation points for agency enterprise IT services and not planned for closure, while the remaining 9,416 were reported as “non-core.”memorandum states that the goal is for agencies to close 40 percent of the total non-core data centers, or 3,766 data centers based on the May 2014 inventory data, by the end of fiscal year 2015.

OMB’s March 2013 Since 2011, agencies have reported their data center closures and planned closures on http://data.gov. As of May 2014, agencies collectively reported that they had closed a total of 976 data centers, and were planning to close an additional 2,689 data centers—for a total of 3,655—by the end of September 2015. See figure 1 for a summary of the total number of the federal data centers reported in agencies’ inventories and closures as reported by agencies on http://data.gov over time, and table 1 for a depiction of the total number of data centers (including a breakdown of core and non-core centers) reported in agencies’ May 2014 inventory submissions, and reported and planned closures.

OMB Integrated FDCCI with Its PortfolioStat Initiative

In March 2012, OMB launched the PortfolioStat initiative, which requires agencies to conduct an annual agency-wide IT portfolio review to, among other things, reduce commodity IT spending and demonstrate how its IT investments align with the agency’s mission and business functions. PortfolioStat is designed to assist agencies in assessing the current maturity of their IT portfolio management process, make decisions on eliminating duplication, and move to shared solutions in order to maximize the return on IT investments across the portfolio.

In September 2012, the Federal CIO wrote in an e-mail to agencies that OMB was planning to integrate FDCCI with the PortfolioStat initiative to allow agencies to focus on an enterprise-wide approach to address all commodity IT, including data centers, in an integrated, comprehensive plan and that agencies should continue to focus on optimizing those data centers that are essential to delivering taxpayer services, while continuing to close those that are duplicative. In addition, the e-mail directed agencies to delay submitting their October 1, 2012 submissions of updated consolidation plans until further guidance could be provided. However, agencies were still to report quarterly updates on their data center closures.

In March 2013, OMB issued a memorandum documenting the integration of FDCCI with PortfolioStat. Among other things, the memorandum discussed OMB’s efforts to further the PortfolioStat initiative by incorporating several changes, such as consolidating previously collected IT-related plans, reports, and data submissions. The guidance also stated that, to more effectively measure the efficiency of an agency’s data center assets, agencies would also be measured by the extent to which their data centers are optimized for total cost of ownership by incorporating metrics for data center energy, facility, labor, and storage, among other things. OMB indicated in its memorandum that these metrics would be developed by the Task Force.

This March 2013 memorandum also established new agency reporting requirements and related time frames. Specifically, agencies were no longer required to submit the data center consolidation plans previously required under FDCCI. Rather, agencies were to submit information to OMB via three primary means—an information resources management strategic plan, an enterprise road map, and an integrated data collection channel. In addition, agencies were still required to update their data center inventories yearly and report quarterly on http://data.gov regarding their consolidation progress.

More recently, in May 2014, OMB issued a memorandum updating its PortfolioStat guidance for fiscal year 2014. As in past PortfolioStat guidance, the memorandum discussed the importance of PortfolioStat sessions—data-driven reviews of agency portfolio management between the Federal CIO, Agency Deputy Secretary, and other senior agency officials—as a means to continue to drive cost savings. OMB’s guidance also reinforced the need for agencies to continue to consolidate their non- core data centers while optimizing their core data centers using metrics established by the Task Force, and documented in OMB’s memorandum. These metrics are discussed in more detail later in this report.

GAO Has Reported and Testified on Issues Related to Consolidating Data Centers

We have previously reported on OMB’s efforts to consolidate federal data centers. In March 2011, we identified data center consolidation as one of the 81 areas within the federal government with opportunities to reduce potential duplication, overlap, and fragmentation. In this regard, we reported on the status of FDCCI and noted that data center consolidation made sense economically and was a way to achieve more efficient IT operations, but that challenges existed. For example, agencies reported facing challenges in ensuring the accuracy of their inventories and plans, providing upfront funding for the consolidation effort before any cost savings accrue, and overcoming cultural resistance to such major organizational changes, among other things.

In July 2011, we issued a report on the status of FDCCI and found that only 1 of the 24 agencies had submitted a complete inventory and no agency had submitted complete plans. agencies to document the steps they had taken, if any, to verify the inventory data. We concluded that until these inventories and plans were complete, agencies would not be able to implement their consolidation activities and realize expected cost savings. Moreover, without an understanding of the validity of agencies’ consolidation data, OMB could not be assured that agencies were providing a sound baseline for estimating consolidation savings and measuring progress against those goals. Accordingly, we made several recommendations to OMB, including that the Federal CIO require that agencies, when updating their data center inventory, state what actions were taken to verify the information in the inventory and to identify any associated limitations on the data, and to complete the missing elements in their inventories and consolidation plans.

Further, OMB had not required OMB generally agreed with our report and has since taken actions to address our recommendations. For example, in July 2011, OMB required agency CIOs to submit a letter that identified steps taken to verify their data center inventory information and attest to the completeness of their consolidation plan. In addition, in March 2012, OMB required that all agencies complete all elements missing from their consolidation plans by the end of the fourth quarter of every fiscal year.

GAO-11-565.

Additionally, in July 2012, we updated our review of FDCCI’s status and found that, while agencies’ 2011 inventories and plans had improved as compared to their 2010 submissions, only 3 agencies had submitted a complete inventory and only 1 agency had submitted a complete consolidation plan. In addition, we noted that 3 agencies had submitted their inventory using an outdated format, in part, because OMB had not publicly posted its revised guidance. Notwithstanding these weaknesses, we noted that 19 agencies reported anticipating about $2.4 billion in cost savings between 2011 and 2015.

We also reported that none of five selected agencies had a master program schedule or cost-benefit analysis that was fully consistent with best practices. To assist agencies with their data center consolidation efforts, OMB had sponsored the development of a FDCCI total cost of ownership estimated costs for consolidation; however, agencies were not required to use the cost model as part of their cost estimating efforts. Accordingly, we reiterated our prior recommendation that agencies complete missing plan and inventory elements and made new recommendations to OMB to publically post guidance updates on the FDCCI website and to require agencies to use its cost model. model that was intended to help agencies refine their OMB generally agreed with our recommendations and has since taken steps to address them. More specifically, OMB posted its 2012 guidance for updating data center inventories and plans, as well as guidance for reporting consolidation progress, to the FDCCI public website. Further, the website has been updated to provide prior guidance documents and OMB memorandums. In addition, OMB’s 2012 consolidation plan guidance required agencies to use the cost model as they developed their 2014 budget request.

OMB refers to total cost of ownership as all associated data center-related activities and costs without regard to ownership, project association, or funding line. had not measured agencies’ progress against key performance measures, including its cost savings goal, or ensured that other key oversight responsibilities, such as approving agencies consolidation plans on the basis of their completeness, were being fully executed. We reported that OMB had not determined agencies’ progress against its cost savings goal because, according to OMB staff, the agency had not determined a consistent and repeatable method for tracking cost savings and that the weaknesses in oversight were due, in part, to OMB not ensuring that assigned responsibilities were being executed. Accordingly, we recommended that OMB track and report on key performance measures, including cost savings, and improve the execution of important oversight responsibilities. OMB generally agreed with our recommendations and has since taken some initial actions to implement them, including tracking and reporting on data center consolidation cost savings on a quarterly basis.

Finally, between May 2013 and June 2014, we testified on the status of FDCCI. Notably, in July 2013, we testified that, while agencies continued to make progress by closing an additional 64 data centers compared to the total number reported through the end of December 2012, the number of federal data centers had grown significantly since OMB’s December 2011 estimate of approximately 3,133 data centers. Specifically, 22 of the 24 FDCCI agencies had collectively reported 6,836 data centers in their inventories—an increase of about 3,700 as compared to OMB’s previous We concluded that it would be important estimate from December 2011.for OMB to be transparent on agencies’ progress against its performance metrics going forward.

Agencies Have Planned Substantial Consolidation Savings, but Reporting Can Be Improved

For FDCCI, OMB originally established a goal of achieving $3 billion in cost savings by the end of 2015. Pursuant to this goal, agencies have reported achieving more than a billion dollars in savings and avoidances through fiscal year 2013 and are planning a total of about $3.3 billion in savings and avoidances by the end of fiscal year 2015—an amount that is approximately $300 million higher than OMB’s goal. Between fiscal years 2011 and 2017, agencies reported planning approximately $5.3 billion in total savings and avoidances. However, planned cost savings may be higher because six agencies with as many as 67 data center closures each have been limited in their abilities to fully report their savings. In addition, slightly more than half of agencies with planned cost savings are underreporting their fiscal years 2012 through 2015 figures to OMB by approximately $2.2 billion. While several agencies noted internal agency communication issues as the reasons for not reporting savings to OMB, other agencies were unable to provide a reason. Until agencies fully report their savings, the total planned cost savings and avoidances of $5.3 billion will be understated.

Agencies’ Reported Savings Exceed More than a Billion Dollars through Fiscal Year 2013; Significant Additional Savings Planned through Fiscal Year 2017

Since launching FDCCI in 2010, achieving cost savings has been a primary goal of the initiative. As previously discussed, one of the original high-level objectives was to reduce the costs of data center hardware, software, and operations. OMB subsequently expanded on this goal and, in February 2012, stated that data center consolidation had the potential to achieve $3 billion dollars in savings by the end of 2015. Pursuant to these goals, OMB required agencies to describe year-by-year investments and cost savings in their 2010 and 2011 consolidation plans and, beginning in August 2013, has required agencies to identify and report all cost savings and avoidances related to data center consolidation, among other areas, to OMB as part of a quarterly data collection process known as the integrated data collection.

Most of the 24 agencies are achieving cost savings or avoidances from their data center consolidation efforts. Specifically, between fiscal years 2011 and 2013, 19 agencies collectively reported achieving an estimated $1.1 billion in cost savings and avoidances. Notably, Defense, the Department of Homeland Security (DHS), and the Department of the Treasury (Treasury) account for approximately $850 million (or 74 percent) of the reported estimated savings through fiscal year 2013. The remaining 5 agencies that did not report savings between fiscal years 2011 and 2013 cited varied reasons for not being able to do so, which included difficulties in determining baseline data center costs, upfront costs that have exceeded savings to date, and a lack of electrical metering to determine power usage savings.

The methodologies used to calculate savings varied across the 19 agencies that reported estimated or actual savings and avoidances through fiscal year 2013; however, most of these agencies estimated their figures. Specifically, 3 agencies—the Department of Education (Education), EPA, and the National Science Foundation (NSF)—reported actual cost savings and avoidances, which they determined by calculating differences in executed budget or contract amounts over time. The remaining 16 agencies estimated their cost savings and avoidances. As examples, GSA estimated its savings using the department’s total cost of ownership model; the Department of the Interior (Interior) used post- consolidation forms collected from its component bureaus and offices to estimate cost savings related to areas such as rent, utilities, and personnel after a consolidation activity was completed; and Treasury estimated savings resulting from reductions in the percentage of IT infrastructure investment spending as compared to total IT spending over time. Officials at these agencies stated that they were limited to reporting estimating savings because of challenges in determining actual savings, including the lack of electrical metering to calculate power usage savings, budget and accounting systems that are not structured to account for the costs of individual data centers, and difficulties in determining costs and savings when data centers are located in multipurpose facilities. These issues are discussed in more detail later in this report.

See table 2 for a listing of agencies’ data center closures, cost savings and cost avoidances between fiscal years 2011 and 2013, and whether the agency savings are estimated.

As prescribed by OMB’s initial guidance on data center consolidation, the 19 agencies that reported achieving cost savings and avoidances did so using a variety of approaches. While these approaches can be grouped into four key areas—decommissioning, consolidation, cloud computing, and virtualization—agencies generally employed, and achieved cost savings and avoidances using, multiple approaches at the same time. For example, NSF officials stated that in order to reduce the agency’s dependence on onsite infrastructure, the agency has been focused on increasing virtualization and consolidation of servers and storage, while continuing to adopt cloud computing technologies. See table 3 for a description of the four approaches and key examples of agency-reported savings or avoidances in each.

In addition to savings through fiscal year 2013, our analysis of estimated future savings shows that, collectively, agencies are reporting that they expect to exceed OMB’s cost savings goal by the end of fiscal year 2015 and continue to achieve significant savings in future years. Specifically, 21 agencies collectively reported planning a total of about $3.3 billion in savings and avoidances by the end of 2015—an amount that is approximately $300 million higher than OMB’s original $3 billion goal. Further, through fiscal year 2017, these agencies collectively reported planning an additional $2.1 billion in cost savings and avoidances, for a total of approximately $5.3 billion. Five agencies—the Department of Agriculture (Agriculture), Defense, DHS, the Department of Transportation (Transportation), and Treasury—account for about $4.9 billion (or approximately 91 percent) of the total savings reported. See table 4 for a listing of agencies’ total cost savings and cost avoidances between fiscal years 2011 and 2017.

The extent of cost savings and avoidances being reported by agencies beyond fiscal year 2015 highlights the importance of OMB continuing to track and report on such savings beyond the time frame of its initial goal. Further, with many agencies having not yet reported on their planned savings, the savings beyond fiscal year 2015 may be higher than previously discussed. In this regard, we have previously recommended that OMB extend the horizon for realizing cost savings from FDCCI, as doing so could provide OMB and FDCCI stakeholders with input and information on the benefits of consolidation beyond OMB’s initial goal.OMB neither agreed nor disagreed with our recommendation but stated that, as the FDCCI and PortfolioStat initiatives proceed and continue to generate savings, OMB would consider whether updates to the current time frame are appropriate.

Savings May Be Higher than Current Amounts Due to Underreporting of Planned Savings by Many Agencies

As previously mentioned, OMB’s March 2013 memorandum identified the requirements for reporting cost savings from data center consolidation. Specifically, the memorandum stated that agencies are required to report their data center consolidation cost savings and avoidances, among other areas, to OMB as part of a quarterly integrated data collection process. OMB’s May 2014 memorandumrequirements for integrated data collection submissions. Agencies can currently input cost savings and avoidances for fiscal years 2012 through 2015 into the web-based portal used to submit their integrated data collection submissions. Finally, standards for internal control emphasize the need for federal agencies to establish plans to help ensure goals and reiterated the objectives can be met, including compliance with applicable laws and regulations.

Although agencies are already collectively reporting approximately $5.3 billion in planned cost savings and avoidances from their consolidation efforts, these savings may be higher because 6 of the 24 agencies, claiming between 11 and 67 data center closures each, have been limited in their abilities to report savings. For example, although Interior reported closing 65 data centers as of May 2014, the agency cited significant challenges in obtaining costs and related savings information from its component agencies. In addition, the National Aeronautics and Space Administration (NASA) reported that, as of May 2014, it had closed 25 data centers; however, while the agency has been able to report $1.3 million in savings through fiscal year 2013, agency officials stated that NASA has been otherwise limited in its ability to identify cost savings and avoidances because of the agency’s complex organizational structure, which includes multiple centers with multiple missions and multiple IT contractors utilizing data centers within multipurpose facilities. Similar challenges were also identified by other agencies, as discussed later in this report. Table 5 shows the agencies with limited or no savings relative to their consolidation efforts and their reasons for not being able to fully report savings.

Considering that cost savings is one of OMB’s original high-level goals of FDCCI and reporting such savings is currently required on a quarterly basis, OMB has a responsibility for ensuring that agencies are identifying the full extent of cost savings from their consolidation efforts. Further, as previously mentioned, OMB’s PortfolioStat guidance requires yearly review sessions of agency portfolio management (including data center consolidation) with the Federal CIO and senior agency officials and notes that these reviews are critical to driving cost savings. We previously found that all agencies held PortfolioStat sessions with OMB in fiscal year 2012. In addition, agencies were required to hold sessions again in 2013. However, after 2 years of PortfolioStat sessions, the six agencies identified in the table have been limited in their ability to report savings from their data center consolidation efforts.

In addition, slightly more than half of the agencies with cost savings and avoidances did not fully report them to OMB—a requirement of OMB’s quarterly integrated data collection process. Specifically, of the 21 agencies with actual and estimated fiscal years 2012 through 2015 cost savings and avoidances, 10 agencies fully reported their savings and avoidances to OMB through the integrated data collection process, 8 agencies partially reported this information, and 3 agencies did not report it. As a result, agencies collectively reported savings for fiscal years 2012 through 2015 of approximately $3.1 billion to us, as compared to only about $876 million that agencies reported to OMB, meaning that the savings have been underreported to OMB by approximately $2.2 billion. See table 6 for a listing of agencies and a comparison of their data center consolidation savings as reported to GAO and through OMB’s integrated data collection process.

While several agencies noted internal agency communication issues as a reason that their savings and avoidances were not fully reported, other agencies were not able to provide a reason. These shortcomings in agency reporting have resulted in OMB not being able to fully report agencies’ data center consolidation cost savings and avoidances in its quarterly reports to Congress on the status of federal IT reform efforts, in accordance with its responsibilities as set forth in law. For example, OMB’s May 2014 report to Congress noted total fiscal years 2012 and 2013 data center consolidation cost savings and avoidances of slightly less than $329 million, as compared to the approximately $951 million that agencies reported to us that they achieved over that same time period—a difference of approximately $622 million.

Until OMB assists those agencies with limited or no cost savings reported, agencies may not be able to identify the full extent of savings from their consolidation efforts and the total planned cost savings and avoidances of approximately $5.3 billion will be understated. Further, until agencies fully report their cost savings and avoidances to OMB, Congress may be limited in its ability to oversee agencies’ progress against key initiative goals.

Agencies Reported Successes and Challenges in Achieving Cost Savings

Agencies Reported That Efforts to Achieve Consolidation Cost Savings Face Challenges Similar to Those Previously Reported

In 2011 and 2012, we reported on the broad challenges that agencies were facing during data center consolidations. These included FDCCI- related, cultural, funding-related, operational, and technical challenges. In 2014, agencies reported that many of the same challenges still exist and impacted their ability to achieve cost savings through consolidation efforts. In addition, agencies identified many new challenges that were specific to achieving cost savings. As we found previously, some challenges are more common than others, with the most-reported challenge being faced by a total of eight agencies. One agency—HUD— did not report any challenges. Table 8 details the reported challenges and the numbers of agencies experiencing that challenge. The table is followed by a discussion of the most prevalent challenges.

Agencies reported that the most significant operational challenges included difficulty in obtaining information (such as data center inventory and cost savings data) from component organizations and determining costs and realizing savings when data centers were located in shared, multipurpose facilities. Specifically, 6 agencies reported that obtaining consolidation-related data from component organizations as a challenge to achieving cost savings which is similar to, but not as prevalent as, the 10 agencies we found having difficulty providing good quality asset inventories in 2012. For example, Defense’s Data Center Consolidation Lead noted that getting component organizations to report all of their data centers remains a challenge in achieving cost savings, particularly in the case of smaller, single-server data centers (e.g., research stations or computers that are only used by a few individuals and are often not reported until replacement or enhancement is needed). Additionally, Agriculture’s Associate CIO for Data Center Operations stated that it was often difficult to determine actual cost savings because the department’s 32 component agencies did not track their total cost of IT, as portions are funded from many different sources versus being under the control of the component agency or office CIO (e.g., building rent and utilities, salaries, construction, etc.).

In addition, whereas we found in 2012 that one agency had difficulty with identifying and quantifying actual costs associated with data center facilities, we found that five agencies reported that it was difficult to determine costs and realized savings when data centers were located in shared, multipurpose facilities. For example, Energy reported that because several of its data centers were located in shared-use facilities, it was difficult for the department to determine the centers’ total operating costs without an additional investment in advanced electricity metering. Energy also noted that it is difficult to determine the total cost savings and avoidances associated with the closure of these data centers until they are decommissioned and the vacated floor space is repurposed. EPA also reported that its data centers and server rooms were housed within mixed-use facilities, which generally cannot be discarded. Further, EPA expects that most former data center spaces will continue to provide local telecommunications and building access support services, which reduces potential substantive building operational cost reductions due to room decommissioning.

Technical Challenges

Agencies reported that the most significant technical challenges included a lack of electricity metering to determine power usage information and increased telecommunication costs after relocating small data centers or applications. The lack of electricity metering is similar to the difficulty we previously found for 15 agencies in 2012 with obtaining power usage information. Our current work found that 8 agencies reported that a lack of electricity metering to determine power usage was a challenge. For example, the Department of Labor (Labor) reported that a lack of electricity metering at many data centers prevented the department from accurately reporting energy savings attributed to the consolidation effort. Labor officials added that it was difficult to perform power usage efficiency calculations because the data to feed the calculations were not available.

As other examples, Transportation reported that its smaller data centers were operated in GSA-owned buildings which did not have electricity metering for the data center spaces. Transportation also noted that many of these spaces also contain telecommunications equipment which would remain after the data center equipment is relocated or decommissioned, which would mean the closures are not expected to produce significant savings. Further, NASA reported that it had encountered difficulties in metering its older, less-efficient facilities and that the modifications needed to make the facilities more efficient would require a significant amount of resources and yield a low return on investment. In addition, those modifications would adversely impact current operations due to the requisite power outages to install metering equipment. Commerce, Defense, Interior, NSF, and the Social Security Administration (SSA) also reported a lack of electricity metering as a challenge.

Regarding increased telecommunication costs, Interior and Transportation both reported this area as a challenge. Interior noted that the higher telecommunication premiums realized from its effort often offset the savings from consolidating a large number of small and closet- sized data centers. Transportation officials also stated that that the relocation of locally run applications to a consolidated data center may lead to increased telecommunications costs.

Financial Challenges

Agencies reported two financial challenges related to obtaining the funding required within their agency for consolidation efforts, as well as budget and accounting system issues that impacted their ability to achieve cost savings. In 2012, we found that nine agencies considered obtaining the funding required for consolidation and migration efforts to be a challenge. In 2014, eight agencies identified this challenge. For example, VA reported that the investment funding for all phases of its consolidation plan has not been available from the department as initially scheduled and, as a result has had to evolve its plan to address the risk of continued investment funding shortfalls so that it can continue to make progress towards its consolidation goals. In addition, SBA officials stated that a lack of funding allocated to implementing its data center consolidation strategy has been the primary challenge in achieving data center consolidation cost savings. Officials added that, in light of this challenge, the agency continues to examine federal cloud and GSA e- mail-as-a-service offerings, and initiated a request to the SBA investment governance process and a fiscal year 2015 request to pilot and migrate agency e-mail to a GSA vendor-managed cloud provider.

In addition, six agencies reported having budget and accounting systems that were not structured to account for individual data centers. For example, Energy officials stated that data centers have generally operated from separate facility and IT operations budgets and specific facility cost elements have not been tracked. In addition, officials added that different budgets are used to support different data centers, resulting in a lack of a consolidated budget for data centers, and has made documenting costs and related savings difficult. In addition, VA officials stated that IT costs often encompass multiple data centers and user facilities, making it challenging to parse the costs to the individual data centers and determine related savings.

Cultural Challenges

Agencies reported that the most significant cultural challenges included having a decentralized organizational structure that was not geared toward consolidation and accepting the cultural changes that were part of consolidation. In 2012, we found that two agencies encountered cultural challenges related to having a decentralized organization structure and five agencies had difficulty accepting cultural change as part of the consolidation effort. Our current work showed six agencies encountered cultural challenges related to having a decentralized organization structure. For example, GSA reported that its foremost challenge was that its data center costs and expenses were distributed across a federated organization. The agency indicated that the costs for rents; leases; personnel; and equipment repair, replacement, and service contracts were distributed across 11 regions, six components, and their subordinate organizations. Further, the Department of Justice (Justice) reported that its federated organizational structure had made it especially challenging to implement enterprise-wide initiatives such as data center consolidation. Officials noted that this was due, in part, to the need to build consensus for, plan, and then implement the consolidation changes, which does not happen quickly in a decentralized environment.

In addition, three agencies stated that accepting the cultural changes required to implement their consolidation efforts impacted their ability to achieve cost savings. For example, Justice also noted that its federated environment made it difficult for people to accept the cultural changes that are part of consolidation. Further, officials from Agriculture reported the challenge of accepting culture change as it encountered resistance to consolidation-related changes, including the use of cloud computing, from component agency personnel.

In any significant IT initiative, it is important that both successes and challenges be highlighted. In the case of FDCCI, a success highlights approaches and strategies that have helped agencies to achieve cost savings and fulfill the intent of the initiative. Conversely, a challenge identifies an area that was impacting an agency’s ability to achieve cost savings and meet the intent of this government-wide effort. In light of how closely the successes and challenges reported by agencies relate to achieving cost savings—a key OMB goal for FDCCI—it will be important for OMB to continue to provide leadership and guidance to the initiative. This includes, as we have previously recommended, utilizing the Task Force—the primary organization responsible for supporting collaboration and knowledge transfer across the FDCCI agencies—to monitor and assist with agencies’ consolidation efforts.

Data Center Optimization Metrics and Related Targets Are Largely Established, but Key Area Not Addressed

Leading practices have established the need for initiatives to develop performance measures to gauge progress. According to government and industry leading practices, performance measures should be measurable, outcome-oriented (i.e., identify targets for improving performance), and actively tracked and reported. In accordance with these principles, OMB’s March 2013 memorandumcenter metrics for energy, facility, labor, storage, virtualization and cost per operating system to enable the measurement of the extent to which federal agency core data centers are optimized for total cost of ownership. directed the Task Force to develop data In May 2014, OMB released a set of 11 data center consolidation optimization metrics established by the Task Force. These metrics address all of the categories defined in the March 2013 memorandum. In addition, related targets to be achieved by the end of fiscal year 2015 have been established for all the metrics except for the cost-per- operating-system metric, which provides for measuring progress on optimizing data center costs. According to a Task Force official, current data center inventory data (already required to be submitted by agencies on at least a yearly basis) will be used to calculate agencies’ progress using the metrics and related targets. See table 9 for a list of the metrics, including the related category, a brief description, and the established target for each metric.

According to OMB staff from the Office of E-Government and Information Technology, there have been challenges in reaching consensus on the cost-per-operating-system target. Specifically, the staff stated that the Task Force has had difficulty with developing a baseline for cloud computing costs that could be used to establish an appropriate target because private sector cloud providers are continually cutting the prices for their services. Development of the cost-per-operating-system target is expected to continue and OMB staff stated that the Task Force expects to finalize the target in the fall of 2014.

In addition, although low server utilization rates were a driving force cited by OMB in launching FDCCI, the new data center optimization metrics do not address this key issue. As previously mentioned, in 2009, OMB reported that server utilization rates were as low as 5 percent across the federal government’s servers. OMB subsequently required agencies to report on server utilization percentage as part of their 2011 and 2012 consolidation plans and included a suggested target of 60 to 70 percent server utilization in its 2011 and 2012 FDCCI consolidation plan guidance. OMB later eliminated the requirement for agencies to continue to update their consolidation plans, but indicated in its March 2013 memorandum that it would continue tracking agencies’ progress through other means, including the data center optimization metrics.

However, a metric for server utilization was not included in the final metrics established by the Task Force. According to an official from the GSA FDCCI Program Management Office that led initial efforts to establish the metrics, server utilization was not included as a metric for a variety of reasons, including that agencies have not traditionally collected the necessary data to be able to calculate server utilization, agencies do not have the server monitoring capabilities required to collect such data, and improvements in other areas of the metrics (such as virtualization) would likely result in higher server utilization. However, as previously mentioned, with server utilization a driving factor for FDCCI and measuring as low as 5 percent as recently as 2009, determining progress against this metric is critical to improving the efficiency, performance, and environmental footprint of federal data center activities.

Without an established target for one of its key cost metrics, the cost per operating system, OMB may not be getting complete information about agencies’ progress in their data center optimization efforts and, therefore, may be lacking important insight that limits its ability to take corrective actions as needed. In addition, without a specific metric for server utilization, OMB may not be fully aware of agencies’ progress on a key metric that was a driving force in launching FDCCI.

Conclusions

After slightly more than 4 years into FDCCI, agencies have begun to report significant savings from their consolidation efforts—most notably, Defense, DHS, and Treasury, which account for 74 percent of the reported savings to date. Furthermore, with approximately $3.3 billion in total planned savings being reported by agencies through fiscal year 2015, meeting OMB’s savings goal is increasingly more likely and, if executed as planned, would represent a significant accomplishment for OMB and the FDCCI agencies. However, limited or no savings achieved at agencies with major consolidation efforts underway suggests that additional actions are necessary. OMB’s and these agencies’ continued efforts to address challenges and identify cost savings opportunities, through the use of such existing mechanisms as PortfolioStat sessions, will result in even more savings. Additionally, agencies’ continued underreporting of consolidation savings will limit OMB’s ability to accurately track agencies’ progress and report to Congress, a point highlighted by the significant understatement of agency-reported savings—by approximately $622 million through fiscal year 2013—in OMB’s recent congressional submission, and—by over $2.2 billion through fiscal year 2015—in agency data submissions to OMB.

As the federal consolidation effort has matured over the past few years, agencies have reported noteworthy successes in achieving cost savings—particularly in leveraging virtualization and cloud computing as a means to achieve such savings. These constructive experiences, which stem from OMB’s recommended consolidation strategies, indicate that FDCCI is moving in the right direction. However, as agencies work toward achieving their cost savings goals, many continue to report challenges related to gathering the necessary technical information from which to calculate savings and funding the consolidation itself. While these challenges are consistent with those reported in the past, others, such as determining savings when data centers are located in multipurpose facilities, have become more prominent. Such a dynamic environment reinforces the need for agencies to remain in communication with OMB in order to facilitate knowledge sharing and transfer and for OMB to continue to provide leadership and guidance, as we have previously recommended.

OMB’s May 2014 publication of the data center optimization metrics is a considerable step forward in helping OMB provide better oversight of agencies’ efforts to optimize their core data centers. Furthermore, targets established for nearly all the metrics will provide agencies with clear and transparent goals to guide their data center optimization efforts. However, the continued absence of a metric for server utilization, despite OMB’s previously-reported concerns about low average utilization rates, represents a missed opportunity to track agencies’ progress on this metric. In the absence of such a metric, OMB will be challenged in demonstrating agencies’ improvement in an area that was a driving force in starting FDCCI and which is critical to improving the efficiency, performance, and environmental footprint of federal data center activities.

Recommendations for Executive Action

To better ensure that FDCCI improves governmental efficiency and achieves cost savings, we are making two recommendations to OMB. We recommend that the Director of OMB direct the Federal CIO to: utilize the existing PortfolioStat review sessions to assist HHS, Interior, Justice, Labor, GSA, and NASA in identifying data center consolidation cost savings opportunities; and as part of any future evaluation of the data center optimization metrics, develop and implement a metric for server utilization.

We also recommend that the Secretaries of HHS, the Interior, Justice, and Labor, and the Administrators of GSA and NASA complete action plans for addressing their challenges in reporting cost savings, as discussed in this report.

Finally, we recommend that the Secretaries of Agriculture, Commerce, Defense, Energy, the Interior, Transportation, the Treasury, and VA; the Administrators of EPA and NASA; and the Director of the Office of Personnel Management direct responsible officials to report all data center consolidation cost savings and avoidances to OMB in accordance with established guidance.

Agency Comments and Our Evaluation

We received comments on a draft of our report from OMB, the 15 agencies to which we made recommendations, and the other 9 agencies mentioned in the report. Specifically, OMB and 12 agencies agreed with our recommendations, 1 agency did not state whether it agreed or disagreed, 1 agency had no comments, and 1 agency—NASA—agreed with one of our recommendations but partially agreed with the other. The other 9 agencies had no specific comments on our recommendations.

Multiple agencies also provided technical comments, which we incorporated as appropriate. Each of the agency’s comments are discussed in more detail below.

In comments provided by e-mail on July 30, 2014, a policy analyst from OMB’s Office of E-Government and Information Technology stated that OMB agreed with the findings and recommendations of the report. OMB also provided technical comments, which we have incorporated as appropriate.

In comments provided by e-mail, a liaison officer from Agriculture’s Office of the CIO stated that the department agreed with the report’s recommendation and noted steps planned to address the recommendation, including engaging with Agriculture agencies to collect actual cost savings and avoidance information realized through their internal consolidation efforts. In addition, the department noted that the Office of the CIO is drafting a Cloud Computing Departmental Directive that, in addition to other requirements, is expected to standardize the process by which IT investments are evaluated for cloud services, including projected and actual cost savings and avoidances.

In written comments, Commerce’s Deputy Secretary stated that the department concurred with the general findings of the report as they applied to Commerce. The department did not state whether it agreed or disagreed with our recommendation, but noted that department would ensure that all savings and avoidances identified by its component bureaus are reported through OMB’s integrated data collection. Commerce’s written comments are provided in appendix II.

Our draft report provided to Defense for comment included a recommendation that the department complete an action plan for addressing their challenges in reporting cost savings. This was based on the department withdrawing its original savings figures—totaling approximately $4.7 billion between fiscal years 2011 through 2017— reported earlier in our review, and submitting revised figures using a new methodology that did not result in planned cost savings estimates beyond fiscal year 2014. Subsequently, Defense provided additional documentation of its planned savings between fiscal years 2015 and 2017, which resulted in an updated total planned cost savings figure of approximately $2.6 billion between fiscal years 2011 and 2017. As a result of Defense’s action, we have removed this recommendation from our report. We have also made changes to the report to reflect these newly-reported numbers. However, in reviewing the additional cost savings information provided by the department, we found that Defense had not fully reported its fiscal years 2012 through 2015 cost savings to OMB, consistent with OMB guidance. As a result, we have added a recommendation for Defense to report all data center consolidation cost savings and avoidances to OMB in accordance with established guidance.

In written comments, Defense’s Acting Principal Deputy CIO stated that the department agreed with the amended report and recommendation. Defense’s written comments are provided in appendix III.

In written comments, Energy’s CIO stated that the department concurred with the report’s recommendation and noted steps being taken by the department to address the discrepancies in its reporting of estimated cost savings and avoidances. For example, the CIO stated that, in order to improve the accuracy and completeness of the data center cost savings and avoidance data, the Energy will clarify its guidance for the integrated data collection data call to better ensure that the department’s organizations report on all data center optimization and consolidation activities. Energy’s written comments are provided in appendix IV.

In comments provided by e-mail on August 25, 2014, an official from HHS’s Division for Oversight and Investigations, Assistant Secretary for Legislation, stated that the department concurred with the report’s recommendation.

In comments provided by e-mail on August 11, 2014, an Interior OIG/GAO Audit Liaison stated that the department concurred with the report’s findings and recommendations.

In comments provided by e-mail on August 20, 2014, a Justice audit liaison stated that the department concurred with the report’s recommendation.

In written comments, Labor’s Assistant Secretary for Administration and Management and CIO stated that the department concurred with the report’s recommendations. The department also provided technical comments, including stating that Labor’s data center inventory figures in our draft report were incorrect. Specifically, the department asserted that its total number of data centers was lower than the number cited in our report and that its number of closed data centers was higher. However, the department did not provide supporting documentation for these changes. We have incorporated Labor’s other technical comment related to its challenges in achieving cost reductions. Labor’s written comments are provided in appendix V.

In written comments, Transportation’s Assistant Secretary for Administration stated that the department agreed with the recommendation related to reporting all of its data center consolidation cost savings and avoidances to OMB, but asserted that the department’s current reporting of this information satisfied our recommendation. While we acknowledge in our report that Transportation has reported a portion of its cost savings and avoidances to OMB, we identified discrepancies between that information and the cost savings and avoidance information that the department reported to us. As a result, we determined that Transportation’s savings and avoidances were not being fully reported to OMB. Therefore, we continue to believe that our recommendation remains valid. Transportation’s written comments are provided in appendix VI.

In written comments, Treasury’s Acting Deputy Assistant Secretary for Information Systems and CIO stated that Treasury had no comments on the report. Treasury’s written comments are provided in appendix VII.

In written comments, VA’s Chief of Staff stated that the department concurred with our recommendation to report all data center consolidation cost savings and avoidances to OMB, stating that it plans to begin reporting this information by the end of 2014, but strongly disagreed with our recommendation that OMB include server utilization in the FDCCI metrics. In our report, we acknowledge the reasons that the server utilization metric was not included when OMB issued the data center optimization metrics in May 2014, such as lack of agency data to calculate utilization and lack of utilization monitoring capabilities. However, because low server utilization rates were a driving force in launching FDCCI, we believe that tracking this metric can provide useful information in assessing agencies’ progress in optimizing their data centers. As previously mentioned, OMB agreed with our findings and recommendations related to this area. Accordingly, we continue to believe our recommendation remains valid. VA’s written comments are provided in appendix VIII.

In written comments, EPA’s Acting Assistant Administrator and CIO stated that the agency agreed with our findings, conclusions, and recommendation, and noted processes in place to address the recommendation. EPA’s written comments are provided in appendix IX.

In written comments, GSA’s Administrator stated that the agency agreed with the report’s findings and recommendation and would take appropriate actions to address the recommendation. GSA’s written comments are provided in appendix X.

In written comments, NASA’s CIO stated that the agency concurred with one of two of our recommendations and partially concurred with the other. Specifically, NASA agreed with our recommendation related to reporting all of its data center consolidation cost savings and avoidances to OMB, stating that it would issue a directive by October 2014. The agency partially concurred with our recommendation to complete an action plan for addressing challenges in reporting cost savings. Specifically, NASA stated that, while it plans to develop and finalize revisions of existing action plans by December 2014, execution of those plans remains a challenge due to difficulties in power metering, particularly in older multipurpose buildings, and measuring facility savings. While we acknowledge the challenges described by NASA in our report, we believe that completing an action plan to address these challenges, as we recommended, could serve as a valuable tool in defining a road map toward overcoming these issues. We therefore continue to believe our recommendation remains valid. NASA’s written comments are provided in appendix XI.

In written comments, OPM’s CIO stated that the agency concurred with our recommendation and described planned actions to address our recommendation. For example, the CIO stated that OPM is preparing its data center consolidation plan to include consideration for shared services and cloud technologies and that any related cost savings will be reported once the consolidation plan is implemented. OPM’s written comments are provided in appendix XII.

In comments provided via e-mail on August 5, 2014, a policy analyst from Education’s Office of the Secretary/Executive Secretariat stated that the department had no comments on the report.

In comments provided via e-mail on August 18, 2014, a program analyst from DHS’s Departmental GAO-OIG Liaison Office stated that the department had no technical comments on the report.

In written comments, HUD’s CIO stated that the agency had no comments on the report. HUD’s written comments are provided in appendix XIII.

In comments provided by e-mail on August 8, 2014, a senior management analyst from State’s Bureau of the Comptroller and Global Financial Services stated that the agency had no comments on the report.

In written comments, NSF’s CIO stated that the agency had no comments on the report. NSF’s written comments are provided in appendix XIV.

In comments provided via e-mail on August 19, 2014, an executive technical assistant from NRC’s Office of the Executive Director for Operations stated that the agency had no comments on the report.

In comments provided via e-mail on August 12, 2014, the program manager for SBA’s Office of Congressional and Legislative Affairs stated that the agency had no comments on the report.

In written comments, the Deputy Chief of Staff from SSA’s Office of the Commissioner stated that the agency had no comments on the report. SSA’s written comments are provided in appendix XV.

In comments provided via e-mail on August 11, 2014, a systems accountant from USAID’s Office of the Chief Financial Officer, Audit, Performance and Compliance Division, stated that the agency had no comments on the report.

We are sending copies of this report to interested congressional committees, the Director of OMB, the secretaries and agency heads of the departments and agencies addressed in this report, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this report, please contact me at (202) 512-9286 or pownerd@gao.gov.

Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix XVI.

Appendix I: Objectives, Scope, and Methodology

Our objectives were to (1) evaluate the extent to which agencies have achieved cost savings to date and identified future savings through their consolidation efforts, (2) identify agencies’ notable consolidation successes and challenges in achieving cost savings, and (3) evaluate the extent to which data center optimization metrics have been established.

To evaluate the extent to which agencies have achieved cost savings to date and identified future savings through their consolidation efforts, we obtained and analyzed cost savings and avoidance documentation, relative to requirements of the Office of Management and Budget’s (OMB) March 2013 memorandum, from the 24 departments and agencies (agencies) in our review. This documentation included, but was not limited to, agencies’ quarterly reports of cost savings and avoidances submitted to OMB, total cost of ownership models, contract and budget documentation, and internal agency status reports. To determine cost savings achieved to date, we totaled agency reported savings and avoidances from fiscal years 2011 through 2013, and to identify future planned savings we totaled agency projected savings and avoidances from fiscal years 2014 through 2017. We also compared agencies’ cost savings and avoidance information to key requirements for identifying and reporting data center consolidation cost savings and avoidances, as outlined in OMB’s March 2013 memorandum.

To assess the reliability of agencies’ cost savings and avoidance data, we reviewed related documentation provided by agency data center program managers and other cognizant officials, such as agency total cost of ownership models, agency-developed spreadsheets, agencies’ quarterly data submissions to OMB, among other sources. We also compared the cost savings and avoidances reported to us by agencies with cost savings identified in OMB’s quarterly reports to Congress on the status of information technology reform efforts. In addition, we reviewed agency documentation for missing data or other errors (e.g., incorrect calculations). Finally, we interviewed agency officials to obtain additional supporting information regarding how their cost savings and avoidance figures were determined, the processes and methods to recalculate the figures, and the steps that the agency took to ensure the reliability of their figures and validate their figures. We also discussed with agency officials any discrepancies or potential errors identified during our review of their supporting documentation to determine the cause or request additional information. We determined that the data were sufficiently reliable to report on agencies’ cost savings achieved to date and identified future savings. However, as part of our reliability assessment, we identified issues with the reliability of OMB’s quarterly reports to Congress, including that agencies’ data center consolidation cost savings were not being fully reflected in OMB’s report. We have highlighted this issue in our report.

Lastly, we reviewed agencies’ data center facility reductions as reported on http://data.gov and compared the information to agencies’ cost savings and avoidances achieved to date, taking into consideration the challenges in achieving savings identified by agencies. To assess the reliability of agencies’ data center reductions, we reviewed prior reporting of data center closures to check for anomalies in the data, such as fewer closures for agencies in more recent data sets than previously reported. We also checked for missing data, outliers, and other obvious errors, such as missing closure status information. Finally, we interviewed OMB staff from the Office of E-Government and Information Technology regarding actions taken to verify the data. We determined that the data were sufficiently reliable to report on agencies’ consolidation progress.

To identify notable consolidation successes and challenges in achieving cost savings, we reviewed agencies’ cost savings documentation, including quarterly reports on cost savings and avoidances submitted to OMB, total cost of ownership models, contract and budget documentation, internal agency status reports, and other documentation, and interviewed agency officials. To determine the types of successes experienced, we identified areas reported in agencies’ documentation with directly attributable cost savings or avoidances. We also interviewed agency officials to identify additional successes in achieving cost savings, including areas where the agency may not have been able to quantify the savings. To determine challenges in achieving cost savings, we interviewed agency officials to obtain information regarding challenges faced, as well as to discuss any steps taken, or planned, to address the challenges identified. We then determined which successes and challenges were encountered most often. In some cases, agencies’ cost savings and avoidance data were used to highlight the impact of a particular success. As a result of the reliability assessment performed for our first objective, we determined these data to be sufficiently reliable for reporting on agencies’ cost savings and avoidances achieved to date and planned.

To evaluate the extent to which data center optimization metrics have been established, we analyzed OMB’s March 2013 memorandum to determine OMB’s requirements for such metrics, including the responsibilities for completing the metrics and the key areas or categories that were to be addressed by the metrics. We then compared OMB’s requirements for the metrics to the final metrics, as documented in a May 2014 OMB memorandum. We also reviewed previous data center consolidation-related OMB memorandums and consolidation plan guidance to identify metrics that had previously been identified by OMB as indicators of data center optimization success and determined the extent to which the metrics addressed these areas. Finally, we interviewed relevant OMB, General Services Administration, and Data Center Consolidation Task Force officials to discuss the process by which the metrics were established and to determine the extent that related targets, or goals, for the metrics had been established.

We conducted this performance audit from October 2013 to September 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the Department of Commerce

Appendix III: Comments from the Department of Defense

Appendix IV: Comments from the Department of Energy

Appendix V: Comments from the Department of Labor

Appendix VI: Comments from the Department of Transportation

Appendix VII: Comments from the Department of the Treasury

Appendix VIII: Comments from the Department of Veterans Affairs

Appendix IX: Comments from the Environmental Protection Agency

Appendix X: Comments from the General Services Administration

Appendix XI: Comments from the National Aeronautics and Space Administration

Appendix XII: Comments from the Office of Personnel Management

Appendix XIII: Comments from the Department of Housing and Urban Development

Appendix XIV: Comments from the National Science Foundation

Appendix: XV: Comments from the Social Security Administration

Appendix XVI: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, individuals making contributions to this report included Dave Hinchman (Assistant Director), Justin Booth, Rebecca Eyler, Brandon Sanders, and Jonathan Ticehurst.