Data Center Optimization: Agencies Report Progress, but Oversight and Cybersecurity Risks Need to Be Addressed
Released: 2020-03-05T00:00:00

Letter

Background

According to OMB, federal agencies reported that they operated 432 data centers in 1998, 2,094 in July 2010, 5,607 in August 2016, and 5,916 in August 2018. As previously mentioned, operating such a large number of centers has been, and continues to be, a significant cost to federal agencies. For example, in 2007, the Environmental Protection Agency (EPA) estimated that the annual cost for electricity to operate federal servers and data centers across the government was about $450 million.

Further, according to the Department of Energy (Energy), a typical government data center has 100 to 200 times the energy use intensity of a commercial building. However, in 2009, OMB reported server utilization rates as low as 5 percent across the federal government’s estimated 150,000 servers. These factors contributed to OMB recognizing the need to establish a coordinated, government-wide effort to improve the efficiency, performance, and environmental footprint of federal data center activities.

Subsequently, OMB launched the Federal Data Center Consolidation Initiative in 2010 to reduce the growing number of federal data centers and we have reported extensively on federal agencies’ efforts to implement the initiative’s requirements. Among other things, OMB required agencies to consolidate inefficient infrastructure, optimize existing facilities, improve their security posture, and achieve cost savings. For example, each agency was required to maintain a complete inventory of all data center facilities owned, operated, or maintained by or on its behalf, and measure progress toward defined optimization performance metrics on a quarterly basis as part of its data center inventory submission.

IT Acquisition Reform Law Enhanced Data Center Consolidation and Optimization Efforts

Recognizing the importance of reforming the government-wide management of IT, Congress enacted FITARA in December 2014. Among other things, the law required agencies to:

Submit to OMB a comprehensive inventory of the data centers owned, operated, or maintained by or on behalf of the agency.

Submit, by the end of fiscal year 2016, a multi-year strategy to achieve the consolidation and optimization of the agency’s data centers. The strategy was to include performance metrics that were consistent with the government-wide data center consolidation and optimization metrics.

Report progress toward meeting government-wide data center consolidation and optimization metrics on a quarterly basis to OMB’s Administrator of the Office of Electronic Government.

In addition, according to FITARA, the Office of Electronic Government at OMB was to:

Establish metrics applicable to the consolidation and optimization of data centers (including server efficiency), ensure that information related to agencies’ progress toward meeting government-wide data center consolidation and optimization metrics was made available to the public in a timely manner, review agencies’ inventories and strategies to determine whether they were comprehensive and complete, and monitor the implementation of each agency’s strategy.

Develop and make publicly available not later than December 19, 2015, a goal broken down by year for the amount of planned cost savings and optimization improvements that were to be achieved through the FDCCI; and, for each year thereafter until October 1, 2020, compare reported cost savings and optimization improvements against those goals.

OMB Established DCOI to Provide Oversight of FITARA Data Center Consolidation and Optimization Requirements

In August 2016, OMB issued Memorandum M-16-19, which established DCOI and included guidance on how to implement the data center consolidation and optimization provisions of FITARA. The memorandum directed each agency to develop a DCOI strategic plan that defined its data center strategy. Among other things, this strategy was to include a timeline for agency consolidation and optimization activities, with an emphasis on cost savings and optimization performance benchmarks that the agency could achieve between fiscal years 2016 and 2018. For example, each agency was required to develop cost savings targets due to consolidation and optimization actions and report any realized cost savings. OMB required each agency to publicly post its DCOI strategic plan to its agency-owned digital strategy website.

In addition, OMB’s memorandum included a series of performance metrics in the areas of data center closures, cost savings, and optimization progress. The guidance further noted that agency progress was to be measured by OMB on a quarterly basis, using agencies’ data center inventory submissions and OMB-defined closures, cost savings, and optimization targets.

Further, the memorandum stated that OMB was to maintain a public dashboard (the IT Dashboard) to display government-wide and agency- specific data center consolidation and optimization progress. In this regard, OMB began including such progress information on the IT Dashboard in August 2016.

GAO Previously Made Recommendations on Agencies’ Consolidation and Optimization Efforts

Since the enactment of FITARA in December 2014, we have reviewed and verified the quality and completeness of each covered agency’s inventory and DCOI strategy annually. We have also published reports documenting the findings from each of these reviews. In addition, we have examined and reported on agencies’ efforts to optimize their data centers, as well as the challenges encountered and successes achieved. As of December 2019, 75 of the 117 recommendations from these reports had not been fully addressed. The results and recommendations of our previous reviews are detailed in appendix II.

OMB Updated DCOI in 2019 and Revised the Definition of a Data Center

In June 2019, OMB issued a memorandum, M-19-19, that updated DCOI and redefined a data center as a purpose-built, physically separate, dedicated space that meets certain criteria. The memorandum also revised the priorities for consolidating and optimizing the federal data centers. Specifically, OMB directed agencies to focus their efforts on their tiered data centers and to stop reporting on spaces not designed to be data centers (i.e., non-tiered data centers) as part of their inventory. The guidance outlined a process by which agencies could request, and OMB would approve, that these facilities be dropped from reporting.

The guidance also noted that OMB would set agency-specific data center closure and cost savings targets in collaboration with each agency and in alignment with that agency’s mission and budget. In addition, OMB described criteria for designating certain data centers as mission critical facilities, which would be exempt from new agency-specific closure targets. Those mission critical designations are to be assumed to be granted unless OMB specifically overturns them.

OMB’s revised June 2019 DCOI guidance also directed agencies to stop reporting on spaces not designed to be a data center as part of their inventory, and to focus their efforts on their remaining purpose-built data centers. This is a change from the previous DCOI guidance, which required agencies to report on a much wider range of facilities.

OMB’s new memorandum also replaced the previous optimization metrics with revised measures that focused on (1) reporting the number of agencies’ virtualized hosts, underutilized servers, and data centers with advanced energy metering; and (2) the percentage of time that data centers were expected to be available to provide services. In contrast to the previous DCOI guidance, the new memorandum did not specify government-wide performance targets for the optimization metrics, such as setting a target for server utilization of 65 percent for all agencies. Instead, OMB worked with agencies to establish agency-specific targets that were also identified in agency DCOI strategic plans and on the IT Dashboard. In addition, the guidance described how agencies could apply for an optimization performance exemption for data centers where typical optimization activities (consolidation of data collection, storage, and processing to a central location) were technically possible but increased the response time for systems beyond a reasonable limit.

Agencies Have Continued to Close Data Centers and Achieve Cost Savings, but Oversight and Cybersecurity Risks Need to be Addressed

As in previous years, the 24 agencies participating in DCOI continued to report progress in closing unneeded data centers and achieving related additional cost savings. The agencies reported closing a total of 102 data centers in fiscal year 2019, as of August 2019, and reported plans to close an additional 184 data centers by the end of fiscal year 2019. According to agencies’ data center inventories, almost all of the 24 agencies met or planned to meet their fiscal year 2019 closure targets. In addition, agencies reported that their DCOI-related activities had either achieved, or planned to achieve, the $241.5 million in total planned savings for fiscal year 2019. However, recent OMB DCOI policy changes will reduce the number of data centers covered by the policy and both OMB and agencies may lose important visibility over the security risks posed by these facilities.

Almost All 24 Agencies Met, or Planned to Meet, OMB’s Fiscal Year 2019 Targets for Data Center Closures

For fiscal year 2019, 23 of the 24 agencies reported that they met or planned to meet their fiscal year data center closure targets, as established under OMB’s June 2019 guidance. Of those 23 agencies: three agencies reported that they did not have any agency-owned data centers and had a target of zero closures; these agencies were listed on the IT Dashboard as having completed their closure efforts; five agencies were not expected to close any of their operating data centers during the fiscal year, and their target was zero;

13 agencies reported meeting or exceeding their target closures by two agencies—the Departments of Defense (Defense) and Veterans Affairs (VA)—reported closing a number of data centers and had additional closures planned that were expected to meet their respective fiscal year targets.

In addition, one agency—the Office of Personnel Management (OPM)— did not submit a DCOI strategic plan and, consequently, did not report a data center closure target.

Table 1 details, for each of the 24 agencies, the number of data centers open at the start of fiscal year 2019, the agency’s fiscal year 2019 closure target, the number of data centers closed, and the number planned for closure during the remainder of the fiscal year, as of August 31, 2019.

Agencies reported a total of 102 fiscal year 2019 data center closures through August 31, 2019, with an additional 184 planned closures by the end of that fiscal year. Figure 1 aggregates this information to show agencies’ overall fiscal year 2019 progress against the reported total number of federal data centers.

In regard to the remaining data centers, as of August 2019, 12 of the 24 agencies reported plans to close 37 data centers in fiscal year 2020 and beyond. Specifically, 10 agencies reported plans to close 31 additional data centers in fiscal year 2020. Further, two agencies—Energy and the Social Security Administration (SSA)—reported plans to close a total of five data centers in 2021, and one agency—the Department of Homeland Security (DHS)—reported plans to close one data center in 2022. Based on our past work reviewing agencies’ DCOI strategic plans, this total number of planned closures is likely to increase when agencies submit their annual DCOI strategic plans in the spring of 2020.

However, the ability to track agencies’ progress against their goals is hampered because the agencies are not reporting their planned and achieved closures on a fiscal year basis, and in one case, the agency had not submitted a plan. As of September 2019, neither the agencies’ strategic plans nor the IT Dashboard provided a specific breakdown of the planned and achieved closures for each fiscal year. OMB’s guidance on DCOI strategic plans only requires reporting cumulative numbers, and staff in OMB’s Office of the Federal CIO confirmed that the IT Dashboard is now intended to report agencies’ cumulative numbers of actual and planned data center closures, rather than numbers broken out by fiscal year. This lack of visibility into exactly how many closures the agencies expect to achieve every fiscal year jeopardizes OMB’s and Congress’ ability to effectively oversee agencies’ data center consolidation efforts.

OMB’s Policy Changes Will Reduce Oversight of Certain Key Data Centers

In August 2016, OMB expanded its definition of a data center to include many smaller facilities that OMB cited as consuming significant amounts of resources. Specifically, OMB included rooms with at least one server, providing IT-related services, and categorized data centers into two groups: tiered (which had to meet specific characteristics defined by OMB) and non-tiered. We previously reported that, based on this definition, as of August 2018, the 24 agencies planned to have a total of 4,907 operating data centers at the beginning of fiscal year 2019.

However, OMB’s June 2019 revised DCOI reporting requirements further changed the definition of a data center, including no longer requiring agencies to report most of the facilities previously categorized as non- tiered data centers. As noted previously, OMB directed agencies to stop reporting on spaces not designed to be data centers as part of their inventory. As a result, agencies are no longer required to report on about 2,000 facilities, some of which are considerable in size and will continue to operate. Based on OMB’s revised definition of a data center, agencies revised their data center inventory counts and now reported 2,727 operating data centers at the beginning of fiscal year 2019.

Specifically, our analysis identified 20 data centers of more than 1,000 square feet that agencies had previously reported as planned for closure, but will not be reported under the current definition. In addition, our analysis found 260 data centers over 1,000 square feet, previously categorized as non-tiered, that agencies plan to continue operating, but which will no longer be reported as part of DCOI. This includes SSA, which plans to no longer report on, but to continue operating, five data centers that are each over 8,000 square feet. Similarly, the Department of State (State) plans to no longer report on, but to keep operating, two facilities that are each at least 10,000 square feet in size.

Further, many of the smaller facilities that are now exempt from DCOI reporting represent what OMB has said in the past are the types of data centers that should be included in DCOI because of the risks they posed. Specifically, in its 2016 guidance memorandum, OMB stated that these smaller facilities posed a cybersecurity risk, and consequently, identified them as data centers that needed to be included in consolidation efforts under DCOI. In particular, OMB called out server rooms and closets as security risks that should be targeted for closure. However, while OMB’s 2019 guidance noted the need to address security at these locations and encouraged agencies to continue working to consolidate and optimize them, there is no requirement for agencies to continue to track and report on their progress in closing these smaller facilities.

In July 2019, we found that IT systems supporting federal agencies, such as those found in the government’s data centers, are inherently at risk. Specifically, we reported that because these systems can be highly complex and dynamic, technologically diverse, and often geographically dispersed, these factors increase the difficulty of protecting their security. Since each physical location represents a potential access point to an agency’s interconnection with other internal and external systems and networks, each location also poses a risk as a point of potential attack. We also noted that IT systems are often riddled with security vulnerabilities—both known and unknown. Cybersecurity vulnerabilities, such as unsecured access points, can facilitate security incidents and cyberattacks that disrupt critical operations; lead to inappropriate access to and disclosure, modification, or destruction of sensitive information; and threaten national security, economic well-being, and public health and safety. Because of OMB’s decision to remove these types of data centers from DCOI reporting, agencies may lose track of the security vulnerabilities that these facilities present due to the consequent reduction in overall visibility and oversight into all data centers.

In its June 2019 guidance, OMB also outlined a process by which agencies could request, and OMB approve, that specific facilities be removed from reporting. As part of this process, agencies were allowed to identify data centers to be removed in one reporting period and then actually remove them in the next, unless OMB provided a written denial within 30 days of the original request. Similarly, agencies could request an exemption for mission critical facilities from their closure target; that request also allows 30 days for OMB to object to the request before an agency should consider the request approved.

However, there is currently no documentation of OMB’s decisions on requests to remove specific data centers from reporting, or to exempt the data centers from closure targets because the facility is mission critical. Although an agency’s data center inventory included fields for documenting OMB’s decisions with regard to potential exemptions to optimization, there is no requirement or mechanism to document OMB’s approval that a data center could be dropped from reporting or exempt from closure. There is also no mechanism that would allow a third party to determine whether OMB is providing any denials within the 30 days specified in the DCOI guidance. Staff in OMB’s Office of the Federal CIO acknowledged that someone without access to OMB’s repository of agencies’ data center inventories could not determine whether OMB completed its review within the required time period.

We recognize that OMB’s data center definition and reporting revisions are an effort to focus agency closure and optimization efforts on certain types of facilities. However, OMB’s own past guidance has acknowledged the security risks posed by the types of facilities that agencies can now exclude from DCOI. While agencies are best positioned to determine whether these locations should be closed or optimized, it is important that these facilities, previously covered by DCOI, continue to be reported on quarterly, regardless of whether they are subject to closure or optimization. Further, the lack of transparency into OMB’s approval process for removing certain facilities from reporting due to a lack of documentation hinders its ability to understand how and why those decisions are made. This, in turn, jeopardizes OMB’s and Congress’ ability to effectively oversee agencies’ data center consolidation and optimization efforts.

Almost All DCOI Agencies Met, or Planned to Meet, OMB Fiscal Year 2019 Cost Savings Targets, with More Savings Planned in 2020

Since 2013, federal agencies have been required to report on data center cost savings. In this regard, OMB provided guidance regarding how agencies were to report cost savings and avoidances. Specifically, it required agencies to report both data center consolidation cost savings and avoidances, among other areas, as part of a quarterly data collection process known as the integrated data collection.

FITARA also called for each agency to submit a multi-year strategy for achieving the consolidation and optimization of data centers that includes year-by-year calculations of investment and cost savings through fiscal year 2018, which has now been extended to 2020. In addition, OMB’s June 2019 memorandum, M-19-19, noted that agency-specific targets would be set in collaboration with each agency and aligned to that agency’s mission and budget.

In their fiscal year 2019 DCOI strategic plans, agencies identified a collective goal of achieving $241.5 million in savings. As of August 2019, the 24 DCOI participating agencies had collectively identified in their quarterly reports to OMB a total of $202.36 million in data center-related cost savings for fiscal year 2019, with an additional $39.14 million expected to be realized in the remaining month of the fiscal year. Specifically, 18 agencies reported that they had met or exceeded their cost savings targets, including seven agencies that did not have a cost savings target and did not report achieving any cost savings. Further, 12 agencies reported plans to achieve about $264 million in data center- related cost savings for fiscal year 2020.

Five agencies that had cost savings targets—the Departments of Agriculture (Agriculture), Commerce (Commerce), DHS, and State; and the National Aeronautics and Space Administration (NASA)—reported that they had not yet met their targets, but planned to do so. Additionally, as noted previously, OPM had not submitted its DCOI strategic plan as of August 2019 and, therefore, did not identify cost savings targets for fiscal year 2019 and beyond.

Table 2 provides a breakdown of each agency’s planned and achieved cost savings for fiscal year 2019, as of August 2019, and planned savings for fiscal year 2020, according to their DCOI strategic plans and quarterly reporting.

Agencies that did not report achieving any cost savings provided a variety of reasons for why they had not done so. For example, officials in the Department of Veterans Affairs’ (VA) Office of the CIO reported 12 data center closures, but said they did not report any achieved cost savings because the majority of those data centers were within multi-use facilities that were still owned and maintained by the agency. However, according to VA’s DCOI strategic plan, the agency plans to achieve cost savings in fiscal year 2020 because it expects to stop leasing two data centers, which is expected to reduce data center spending.

In addition, officials from three agencies—the Department of Housing and Urban Development (HUD), the General Services Administration (GSA), and the United States Agency for International Development (USAID)— reported that they did not have any agency-owned data centers and had limited opportunity to achieve cost savings related to closing and optimizing their data centers. According to OPM officials, the agency did not have a savings target due to the lack of a fiscal year 2019 DCOI strategic plan, which was attributed by the officials to an oversight that resulted from changes in OPM CIO leadership at the time the plan was due. The officials reported that the agency continued to execute on a plan that was already in place and they did not anticipate any meaningful changes in the agency’s DCOI strategy for 2020. The officials said they expect OPM to submit its fiscal year 2020 strategic plan on time in April 2020.

Overall, the 24 participating DCOI agencies have reported a total of $4.7 billion in cost savings and avoidances from fiscal years 2012 through 2019. We have previously stressed that identifying and reporting the savings resulting from agencies’ data center consolidations was an important indicator for monitoring the progress of DCOI. Until OPM submits a plan that identifies its cost savings targets to OMB, the agency’s ability to plan how to achieve DCOI’s expected benefits will be limited. In addition, until the five agencies that still expect to achieve savings establish and meet their cost savings targets, DCOI may not deliver the expected financial benefits.

Agencies Reported Progress against DCOI’s Revised Optimization Metrics, but Metrics Lacked Appropriate Information on Performance Parameters

FITARA required OMB to establish metrics to measure the optimization of data centers, including server efficiency, and to ensure that agencies’ progress toward meeting those metrics is made available to the public. Pursuant to this requirement, OMB has used several different sets of performance measures that have changed over time. Most recently, and as previously noted, OMB issued revised DCOI guidance in June 2019 that defined a set of three revised and one new data center optimization metrics to replace the five previous metrics. According to the OMB memorandum that published these changes, the current metrics were intended to focus optimization efforts in key areas where agencies can make meaningful improvements and achieve further cost savings through optimization. Table 3 provides a description of the four data center optimization metrics and how each metric is to be calculated.

According to the June 2019 revised DCOI guidance, agencies are to focus their optimization efforts on their remaining open, agency-owned, tiered data centers. OMB also included in the guidance its plans to work with the agencies to set agency-specific optimization performance targets for each fiscal year. According to staff in OMB’s Office of the Federal CIO, these targets are to be established by fiscal year and progress toward meeting them is expected to be provided via the IT Dashboard.

For three of OMB’s June 2019 optimization metrics, 19 of the 24 DCOI agencies reported progress in meeting OMB’s fiscal year 2019 data center optimization targets identified on the IT Dashboard. Specifically, as of September 2019:

11 reported that they had met their target for virtualization,

11 reported that they had met their advanced metering target, and

18 reported that they had met their server utilization target.

Of the remaining five agencies, OPM had not submitted a DCOI strategic plan as of September 2019 and consequently, did not have established optimization targets or a basis to measure and report optimization progress. The remaining four agencies—the Department of Education (Education), HUD, GSA, and USAID—reported that they did not have any agency-owned data centers in their inventory and, therefore, the optimization metrics were not applicable. In addition, Justice had not established a target for the server utilization metric and, therefore, did not have a basis to measure and report progress. Figure 2 summarizes the DCOI agencies’ progress in meeting each optimization target, as of September 2019.

Of the 19 agencies with a basis to report against OMB’s optimization targets, eight agencies—Energy, DHS, the Department of the Interior, State, NASA, NSF, NRC, and SSA—reported meeting three targets as of September 2019. Also, five agencies reported that they had met two targets, and six agencies reported meeting one target.

Table 4 lists the DCOI agencies and their status on meeting their OMB optimization performance targets.

Of the current DCOI metrics, as shown in table 4, agencies reported greater success in meeting their agency-specific optimization targets than we had reported in our previous reviews, as detailed in appendix II. As of September 2019, the IT Dashboard reported that four agencies had fully completed their overall DCOI optimization efforts for all of their data centers and had no further work to do. The IT Dashboard further reported that another four agencies had met their optimization targets for fiscal year 2019.

However, eight agencies had not met their fiscal year 2019 virtualization target. The reasons agencies provided for not meeting the target varied. For example, officials in the Department of Agriculture’s Office of the CIO reported that the department did not meet the virtualization target because the closure date for one of its data centers was moved to fiscal year 2020, which resulted in fewer virtualized hosts for 2019 under OMB’s new definition. Additionally, although EPA did not meet its virtualization target, its DCOI strategic plan described the agency’s intention to meet its goals by expanding its virtualization strategy agency-wide, which would increase the agency’s virtualization performance.

In addition, OMB required agencies to report the number of agency- owned data centers with advanced energy metering. As of September 2019, of the 19 agencies with the basis to report, eight reported that they did not reach their target for having such metering in their data centers. For example, officials at the Department of Veteran Affairs reported that they did not meet their advanced energy metering target due to difficulties in getting a contract in place to install the metering.

Further, for the new availability metric, there were unexpected variances in how agencies reported information—thus rendering the data for this metric unreliable. Specifically, according to OMB’s quarterly reporting instructions, agencies were to report the number of hours, in the 3-month reporting period, that each data center was expected to be available to provide services. However, several agencies reported information based on annual, instead of quarterly, calculations. In addition, Department of Agriculture officials stated that, for one data center, they reported the total number of availability hours for multiple instances where they provided data center services to other agencies. Based on the various instances of erroneous agency reporting that we identified, we determined that the data for this metric was not sufficiently reliable for us to use.

When the problems with these data were brought to agencies’ attention, many agreed that their reporting needed to be updated; in some cases, the agencies updated their information, but not in time for it to be analyzed and addressed in this report. Based on our discussions with agencies, we will continue to monitor their progress in improving the accuracy of their reporting for this metric through our follow-up efforts for this report, as well as our future mandated reviews of DCOI progress.

Additionally, and as mentioned previously, Justice had not established a target for server utilization. Officials in the department’s Justice Management Division stated that this was due to OMB’s issuing the revised DCOI guidance and metrics in June 2019. Once they can track server utilization for a few reporting periods, the officials stated that the agency will finalize its definition for underutilized severs and establish an appropriate target for the metric.

Overall, while agencies reported more success in meeting the current optimization metrics, most agencies did not meet all of their metric targets for fiscal year 2019. Until these agencies take the steps necessary to meet their optimization targets, it is unlikely that these agencies will achieve the expected benefits of optimization and the resulting cost savings. Given that our April 2019 report included recommendations for all of the agencies except Commerce that missed an optimization target to take action to meet the data center optimization metric targets established under DCOI by OMB, we are not making new optimization- related recommendations to those agencies.

OMB’s New Optimization Metric Definitions Lack Key Characteristics of Effective Performance Measures

GAO’s Green Book provides the standards for internal control in the federal government and an overall framework for establishing and maintaining an effective internal control system. Such a control system addresses, in part, the attainment of a federal entity’s objectives, which is accomplished through monitoring specific performance measures. Such monitoring is also expected to assess the quality of performance over time.

In addition, the Green Book discusses the importance of clearly defining an entity’s objectives in order to determine what is trying to be achieved and to establish related performance measures. According to the Green Book, the controls represented by an agency’s performance metrics should include these key characteristics. The controls should be:

Clearly defined in measurable terms that are easily understood.

Objective and free of bias, rather than subjective.

Defined by appropriate parameters that allow for evaluating performance.

Understood by all levels of the organization, including what is being achieved with the metric, who is primarily responsible for achieving the metric, how the metric will be achieved, and when the metric will be achieved.

Aligned with internal and external requirements, including applicable legislation, regulations, and standards.

We found that all four of OMB’s current optimization performance metrics met three of these five characteristics—that is, each was clearly defined, objectively measurable, and aligned with internal and external requirements. However, the performance metrics did not fully meet the two other characteristics—namely they did not include appropriate performance parameters and did not fully include all the information that would allow them to be understood at all levels of the organization.

Table 5 provides our assessment of the extent to which the OMB metrics aligned with the characteristics of an effective metric. In addition, appendix III provides additional detail of our assessment of the characteristics of each metric.

While all four of OMB’s metrics met three of the five characteristics of an effective metric, none of the metrics addressed the fourth characteristic of providing appropriate performance parameters. Specifically, none of the metrics included statistical universe parameters that would enable a determination of progress against goals. For example, the virtualization metric requires an agency to report the number of its virtual hosts, but does not relate that to the overall number of servers and mainframes at the agency. As a result, the metric does not indicate whether an agency’s reported number of virtual hosts is almost all of that agency’s servers and mainframes, or very few. Similarly, the server utilization metric identifies how many underutilized servers an agency has, but does not give the context of how that relates to the agency’s total population of servers. In both these cases, percentages cannot be calculated to determine progress. For instance, while the number of an agency’s virtualized servers may increase, if the universe of servers were to increase at a higher rate, then progress would actually be negative. In the June 2019 DCOI revised guidance, OMB acknowledged removing targeted averages for its metric targets. However, by doing so, OMB also removed important information that provided a relative sense of the progress indicated by the data.

Further, the lack of performance parameters in defining the metrics had an impact on OMB’s public reporting of agencies’ progress. The IT Dashboard displays agencies’ consolidation and progress information through a DCOI Optimization Summary that displays data about the number of agency data center facilities, achieved and planned closures, achieved and planned IT cost savings, and progress of the current performance metrics against the related targets. However, the IT Dashboard does not provide important information, such as in which fiscal year the targets are to be achieved and how the metric information being reported relates to an agency’s operations. For example, the IT Dashboard reports the number of servers and mainframes serving as virtual hosts in agency-managed data centers, but does not provide the total number of servers and mainframes to give the context of how well agencies are managing the number of their virtual hosts.

Staff in OMB’s Office of the Federal CIO stated that the lack of performance parameters for the metrics is due to OMB and the agencies needing time to collect baseline data before making changes to the metrics. However, until OMB addresses missing information from the optimization metric definitions, the metrics will lack important and meaningful information about agencies’ DCOI performance that would assist OMB and Congress in their oversight roles. In addition, unless OMB takes action to update the metrics’ definitions to include missing key metric characteristics, agencies’ reporting may not provide an accurate view of their data center optimization progress. Further, without this information on the IT Dashboard, Congress lacks the information needed to inform its decision making and oversight responsibilities.

Conclusions

Federal data center consolidation efforts have been underway since 2010, and agencies continue to report progress towards meeting their goals for data center closures and achieving related savings. Specifically, almost all of the 24 DCOI agencies met, or planned to meet, their goals for data center closures in fiscal year 2019. Additionally, in fiscal year 2019, almost all of the agencies met or planned to meet their $249 million total savings target. Agencies’ efforts in both respects have made an important contribution to achieving the overall goals of DCOI. However, agencies’ annual closure goals are not currently reported in their DCOI strategic plans or tracked on the IT Dashboard, requiring us to manually calculate those targets. Unless agencies’ annual closure goals are fully reported and tracked, oversight of DCOI will be hampered. Further, the six agencies without plans to meet their fiscal year data center closure or cost savings targets will continue to be challenged to realize the full benefits of DCOI.

As part of the 2019 changes to DCOI, OMB significantly reduced the scope of what is considered a data center, and, in doing so, excluded about 2,000 smaller facilities that were previously reported by agencies in 2018. While OMB previously acknowledged that these types of facilities inefficiently consume resources and pose security risks, agencies are no longer required to report these locations in their inventories. Further, there is currently no documentation of OMB’s decisions on agency requests to remove data centers from reporting, or to exempt mission critical data centers from closure targets. By no longer reporting key facilities as part of DCOI and by not documenting decisions on which facilities are exempt from DCOI, oversight of agencies’ consolidation and optimization efforts may be impaired, and agencies may remain exposed to the related vulnerabilities.

Agencies’ progress against OMB’s three revised metrics was mixed, and, for one new metric, agencies reported data that varied so widely, we concluded the data for this metric were not sufficiently reliable for us to report on. However, in comparing OMB’s four metrics against the characteristics of an effective metric, we most notably found that none of the metrics included appropriate performance parameters for evaluating agencies’ progress against goals. Metrics that include more robust and informative agency performance data can play an important role in both achieving the optimization goals and mission of DCOI and allowing for stronger oversight of those efforts.

Recommendations for Executive Action

In addition to reiterating our prior open recommendations to the agencies in our review regarding their need to meet DCOI’s closure and savings goals and optimization metrics, we are making a total of eight new recommendations—four to OMB and four to three of the 24 agencies. Specifically: The Director of the Office of Management and Budget should (1) require that agencies explicitly document annual data center closure goals in their DCOI strategic plans and (2) track those goals on the IT Dashboard. (Recommendation 1)

The Director of the Office of Management and Budget should require agencies to report in their quarterly inventory submissions those facilities previously reported as data centers, even if those facilities are not subject to the closure and optimization requirements of DCOI. (Recommendation 2)

The Director of the Office of Management and Budget should document OMB’s decisions on whether to approve individual data centers when designated by agencies as either a mission critical facility or as a facility not subject to DCOI. (Recommendation 3)

The Director of the Office of Management and Budget should take action to address the key performance measurement characteristics missing from the DCOI optimization metrics, as identified in this report. (Recommendation 4)

The Secretary of Agriculture should take action to achieve its data center- related cost savings target established under DCOI by OMB. (Recommendation 5)

The Secretary of Commerce should take action to achieve its data center- related cost savings target established under DCOI by OMB. (Recommendation 6)

The Secretary of Commerce should take action to meet its data center optimization metric targets established under DCOI by OMB. (Recommendation 7)

The Administrator of the National Aeronautics and Space Administration should take action to achieve its data center-related cost savings target established under DCOI by OMB. (Recommendation 8)

Agency Comments and Our Evaluation

We provided a draft of this report to OMB and the 24 agencies for their review and comment. In response, of the seven agencies to which we made recommendations, five agencies stated that they agreed with the recommendations and two agencies did not state whether they agreed or disagreed with the recommendations.

In addition, of the 18 agencies to which we did not make recommendations, three agencies stated that they concurred with the information presented in the report, three other agencies did not state whether they agreed or disagreed with the report, and 12 agencies stated that they had no comments on the report. Further, four agencies provided technical comments on the report, which we incorporated as appropriate.

Of the agencies to which we made recommendations, five agreed with the recommendations.

In an email, a Director for Strategic Planning, Egovernment, and Audits in the Office of the CIO at Agriculture stated that the department agreed with our recommendation to achieve its data center-related cost savings target established under DCOI and that it planned to meet the cost savings target in 2020. Agriculture also included technical comments, which we have incorporated as appropriate.

In written comments, Commerce agreed with our recommendations to achieve its data center-related cost savings target established under DCOI and to meet its data center optimization metric targets established under DCOI by OMB. The department also described actions that they planned to take in order to address the recommendations. Commerce’s comments are reprinted in appendix IV.

In written comments, DHS agreed with our recommendation to achieve its data center-related cost savings target established under DCOI. Further, the department stated that, in its November 2019 DCOI data submission, it reported $354.97 million in cumulative DCOI cost savings through fiscal year 2019. Subsequent to reviewing our draft report, the department provided documentation of the savings claimed in their response. In reviewing this data, we confirmed that these cumulative savings included the $33.8 million savings the department had planned for fiscal year 2019. As a result, we consider our recommendation to have been addressed and therefore removed it from the final report. DHS also provided technical comments, which we have incorporated as appropriate. DHS's comments are reprinted in appendix V.

In written comments, NASA agreed with our recommendation to achieve its data center-related cost savings target established under DCOI and described actions that the agency planned to take to address the recommendation. NASA stated that it expects to complete these actions by March 31, 2020. NASA's comments are reprinted in appendix VI.

In written comments, OPM agreed with our recommendation to develop and submit to OMB a complete DCOI strategic plan. Subsequent to reviewing our draft report, OPM informed us that the agency had published its fiscal year 2019 plan, and that the agency was on track to meet the OMB reporting deadline for fiscal year 2020. We confirmed that OPM's fiscal year 2019 strategic plan was published and publicly available through the agency's website. As a result, we consider our recommendation to have been addressed and therefore removed it from the final report. OPM's comments are reprinted in appendix VII.

In addition, two agencies did not state whether they agreed or disagreed with their recommendations.

In an email, a GAO liaison on OMB’s Ethics Team provided an annotated copy of our draft report. In OMB’s comments in that copy of the draft, OMB did not agree or disagree with our recommendations. However, OMB took issue with the report’s findings that the removal of facilities from DCOI oversight posed cybersecurity-related risks represented by those facilities. OMB’s comments further recommended that we remove references to cybersecurity from our report’s title and from the body of the report.

In raising these objections, OMB’s comments stated that DCOI is focused on consolidating and optimizing the federal data center portfolio and that cybersecurity is not a primary driver of the initiative. OMB added that DCOI was never designed to track or directly address cybersecurity risks. Specifically, OMB’s comments took issue with our finding that data centers not tracked within DCOI are at a greater risk for a cybersecurity incident. These comments noted that many other laws, policies, and procedures directly deal with the cybersecurity posture of all federal IT systems, and that OMB’s DCOI guidance does not affect the applicability of those requirements. The comments also acknowledged that, while past DCOI guidance has stated that the reduction of data centers may improve the cybersecurity posture of federal agencies, this was because agency CIOs could better allocate constrained resources across a smaller portfolio of devices.

We agree that agencies are subject to numerous cybersecurity requirements external to DCOI. We also agree that a reduced portfolio of data centers may improve the cybersecurity of an agency. However, our report focuses on OMB’s recent DCOI policy changes that allow agencies to stop tracking and reporting on over 2,000 data centers. In this discussion, we cite our July 2019 report which found that, facilities such as these, represent a potential access point to an agency’s systems and networks and pose a risk as points of potential attack. OMB’s policy changes do not require agencies to continue to close these points of access, nor do they yield the smaller portfolio of devices that OMB referenced in its comments on our draft report.

Our report notes that OMB’s policy change to remove those data centers from DCOI reporting may contribute to agencies losing track of the security vulnerabilities that those facilities present because DCOI has provided a mechanism for ongoing visibility and oversight of these facilities separate from the federal government’s cybersecurity framework. As such, we maintain our report accurately characterizes the increased potential for cybersecurity risk that could be posed by these now-unreported physical locations. We also affirm that our related recommendation to OMB to require agencies to report in their quarterly inventory submissions, those facilities previously reported as data centers, even if those facilities are not subject to the closure and optimization requirements of DCOI, is still appropriate.

In written comments, State did not say whether it agreed or disagreed with our recommendation to achieve its data center-related cost savings target established under DCOI by OMB. Subsequent to reviewing our draft report, the department informed us of $61.1 million in fiscal year 2019 optimization and consolidation cost savings and avoidances, an amount in excess of its $58.9 million fiscal year 2019 target, and provided documentation to support this claim. The department also stated that this information would be reported in the department's annual DCOI strategic plan update in the second quarter of fiscal year 2020. In reviewing the documentation provided by the department, we confirmed State’s reported $61.1 million in fiscal year 2019 savings. As a result, we consider our recommendation to have been addressed and therefore removed it from the final report. State's comments are reprinted in appendix VIII.

Further, of the 18 agencies to which we did not make recommendations, three agencies agreed with the information presented in the report.

Via emails, audit liaisons in the Office of the CIO at Justice, the Office of the Assistant Secretary for Policy at Labor, and the Office of Congressional and Legislative Affairs at VA agreed with the findings in the draft report.

In addition, three agencies did not state whether they agreed or disagreed with the report.

In written responses, Defense and USAID did not state whether they agreed or disagreed with the draft report. The agencies' responses are reprinted in appendices IX and X respectively.

In an email, an audit liaison in the OIG-GAO Audit Liaison Office at Interior did not state whether the department agreed or disagreed with the draft report. The department also provided technical comments, which we have incorporated as appropriate.

Finally, 12 agencies stated that they had no comment on the report.

In written responses, HUD and SSA stated that they had no comments on the draft report. The agencies' responses are reprinted in appendices XI and XII respectively.

We also received emails from officials of Education, Energy, HHS, Transportation, Treasury, EPA, GSA, NSF, NRC, and SBA, which stated that the agencies had no comment on the report. EPA also provided technical comments, which we have incorporated as appropriate.

We are sending copies of this report to interested congressional committees, the Director of OMB, the secretaries and heads of the departments and agencies addressed in this report, and other interested parties. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

If you or your staffs have any questions about this report, please contact me at (202) 512-4456 or harriscc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix XIII.

Appendix I: Objectives, Scope, and Methodology

This report addresses (1) agencies’ progress on data center closures and the related savings that have been achieved, and agencies’ plans for future closures and savings and (2) agencies’ progress against the Office of Management and Budget’s (OMB) data center optimization targets.

To address the first objective, for data center closures, we obtained and analyzed August 2019 data center inventory documentation from the 24 departments and agencies (agencies) that participate in OMB’s Data Center Optimization Initiative (DCOI). To determine data center closures to date, we totaled their reported closures for fiscal year 2019 through August 31, 2019, and, to identify future closures, we totaled their reported planned closures for fiscal years 2019 through 2022. We also compared agencies’ completed and planned closures to the planned fiscal year 2019 consolidation goals, as documented in their DCOI strategic plans. OMB’s guidance for developing agencies’ DCOI strategic plans required agencies to report cumulative numbers for their planned and achieved data center closures; as a result, we calculated agencies’ fiscal year 2019 targets from the data reported in DCOI plans.

To verify the quality, completeness, and reliability of each agency’s data center inventory, we compared information on completed and planned data center closures to similar information reported on OMB’s IT Dashboard—a public website that provides information on federal agencies’ major IT investments. We also checked for missing data and other errors, such as missing closure status information. In some cases identified, we followed up with agency officials to obtain further information. We determined that the data were sufficiently complete and reliable to report on their consolidation progress and planned closures.

For cost savings and avoidances, we obtained and analyzed documentation from the 24 DCOI agencies. This documentation is required by OMB’s March 2013, August 2016, and June 2019 memorandums and included the agencies’ quarterly reports of cost savings and avoidances posted to their digital services websites and their DCOI strategic plans. To determine cost savings achieved, we totaled agencies’ reported savings and avoidances from the start of fiscal year 2012 through August 2019, as found in the August 2019 quarterly reports posted to the agencies’ digital services websites. To identify future planned savings, we totaled the agencies’ projected savings and avoidances from fiscal years 2019 through 2020, as reported in their DCOI strategic plans.

To assess the quality, completeness, and reliability of each agency’s data center consolidation cost savings information, we used the latest version of each agency’s quarterly cost savings report and DCOI strategic plan as of August 31, 2019. We also reviewed the quarterly reports and DCOI strategic plans for errors and missing data, such as missing cost-savings information. In addition, we compared agencies’ cost savings and avoidances with data from our most recent data center consolidation report. Further, we obtained written responses from agency officials regarding the steps they took to ensure the accuracy and reliability of their cost savings data. As a result, we determined that the data were sufficiently complete and reliable to report on agencies’ data center consolidation cost-savings information.

For our second objective, we analyzed the September 2019 data center optimization progress information of the 20 DCOI agencies. This progress information was obtained from the IT Dashboard. We then compared the agencies’ current optimization progress information to agencies’ fiscal year 2019 optimization targets, as documented on the IT Dashboard.

In addition, to assess the reliability of agencies’ optimization progress information on OMB’s IT Dashboard, we reviewed the information for errors or missing data, such as progress information that was not available for certain metrics. We also compared agencies’ optimization progress information across two reporting quarters to identify any inconsistencies in agencies’ reported progress. We also followed up with the agencies to understand the steps they took to insure that what they reported to OMB was accurate and reliable. We determined that the data were sufficiently complete and reliable to report on agencies’ progress information for virtualization, advanced energy metering, and server utilization.

However, for the fourth metric—data center availability—our analysis identified variances in how agencies reported their data. According to OMB’s quarterly reporting instructions, agencies were to report the number of hours, in the 3-month reporting period, that each data center was expected to be available to provide services. Instead, several agencies reported information based on annual, instead of quarterly, calculations. In addition, Department of Agriculture officials stated that, for one data center, they reported the total number of availability hours for multiple instances where they provided data center services to other agencies. Because of these variances and the impact they had on the reported information, we determined that the availability metric data were insufficiently reliable to report on agencies’ progress.

To assess whether OMB’s new performance metrics met key characteristics of an effective performance measure, we adapted principles from the Green Book that described characteristics of effective performance measures. The Green Book provides an overall framework for establishing and maintaining an effective internal control system that includes monitoring through performance measures. We then compared each OMB optimization performance metric, as defined in the revised DCOI guidance and reported on OMB’s IT Dashboard, to the criteria we identified from the Green Book to determine the extent to which each metric met each characteristic.

We conducted this performance audit from April 2019 to March 2020 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: GAO Previously Made Recommendations on Agencies’ DCOI- related Efforts

Since the enactment of FITARA in December 2014, we have reviewed and verified the quality and completeness of each covered agency’s inventory and Data Center Optimization Initiative (DCOI) strategy annually. Accordingly, we have published reports documenting the findings and recommendations from each of these reviews. In addition, we have examined and reported on agencies’ efforts to optimize their data centers, as well as the challenges encountered and successes achieved. As of December 2019, 75 of the 117 recommendations from these reports had not been fully implemented.

In a report that we issued in March 2016, we noted that agencies had reported significant data center closures—totaling more than 3,100 through fiscal year 2015—but fell short of the Office of Management and Budget’s (OMB) fiscal year 2015 consolidation goal. Agencies also reported significant consolidation cost savings and avoidances—totaling about $2.8 billion through fiscal year 2015. However, we pointed out that many agencies lacked complete cost savings goals for the next several years despite having closures planned.

In addition, we reported that 22 agencies had made limited progress against OMB’s fiscal year 2015 data center optimization performance metrics, such as the utilization of data center facilities. Accordingly, we recommended that the agencies take actions to complete their cost savings targets and improve optimization progress. As of December 2019, 17 of the 32 recommendations from this report had yet to be fully addressed.

In May 2017, we reported that the agencies continued to report significant data center closures—totaling more than 4,300 through August 2016— with more than 1,200 additional centers planned for closure through fiscal year 2019. The agencies also reported achieving about $2.3 billion in cost savings through August 2016. However, agencies’ total planned cost savings for fiscal years 2016 through 2018 were more than $2 billion less than OMB’s fiscal year 2018 cost savings goal of $2.7 billion.

In addition, our May 2017 report identified weaknesses in agencies’ DCOI strategic plans. Of the 23 agencies that had submitted their strategic plans at the time of our review, seven had addressed all of the five required elements of a strategic plan, as identified by OMB (such as providing information related to data center closures and cost savings metrics). The remaining 16 agencies that submitted their plans either partially met or did not meet the requirements.

Given these findings, we recommended that OMB improve its oversight of agencies’ DCOI strategic plans and their reporting of cost savings and avoidances. We also recommended that 17 agencies complete the missing elements in their strategic plans, and that 11 agencies ensure the reporting of consistent cost savings and avoidance information to OMB. As of December 2019, five of the 30 recommendations had not been fully addressed.

In a subsequent report that we issued in August 2017, we noted that 22 of the 24 agencies required to participate in the OMB DCOI had collectively reported limited progress against OMB’s fiscal year 2018 performance targets for the five optimization metrics. Specifically, for each of the five targets, no more than five agencies reported that they had met or exceeded that specific target.

In addition, we noted in the report that most agencies had not implemented automated monitoring tools to measure server utilization, as required by the end of fiscal year 2018. Specifically, four agencies reported that they had fully implemented such tools and 18 reported that they had not done so. Two agencies did not have a basis to report on progress because they did not have any agency-owned data centers.

Accordingly, we recommended that OMB formally document a requirement for agencies to include plans, as part of existing OMB reporting mechanisms, to implement automated monitoring tools at their agency-owned data centers. We also recommended that the 18 agencies without fully documented plans take action within existing OMB reporting mechanisms to complete plans describing how they intended to achieve OMB’s requirement to implement automated monitoring tools at all agency-owned data centers by the end of fiscal year 2018. As of December 2019, two of the 19 recommendations had been fully addressed.

In May 2018, we noted that the 24 agencies participating in DCOI reported mixed progress toward achieving OMB’s goals for closing data centers by September 2018. Over half of the agencies reported that they had either already met, or planned to meet, all of their OMB-assigned closure goals by the deadline. However, four agencies reported that they did not have plans to meet all of their assigned goals and two agencies were working with OMB to establish revised targets. With regard to agencies’ progress in achieving cost savings, 20 agencies reported planned and achieved savings that totaled $1.62 billion for fiscal years 2016 through 2018. However, this total was approximately $1.12 billion less than OMB’s DCOI savings goal of $2.7 billion.

In addition, the 24 agencies continued to report limited progress against OMB’s five data center optimization targets, with one agency meeting four targets, one meeting three targets, six meeting either one or two targets, and 14 meeting none of their targets. Further, as of August 2017, most agencies were not planning to meet OMB’s fiscal year 2018 optimization targets.

Because we had previously made a number of recommendations to OMB and the 24 DCOI agencies to help improve the reporting of data center- related cost savings and to achieve optimization targets, we did not make new recommendations in our May 2018 report, but indicated that we would continue to monitor the agencies’ progress toward meeting OMB’s DCOI goals.

Most recently, in April 2019, we reported that the 24 DCOI agencies continued to report mixed progress toward achieving OMB’s goals for closing data centers and realizing the associated savings by September 2018. Thirteen agencies reported that they had met, or had plans to meet, all of their OMB-assigned closure goals by the deadline. However, 11 agencies reported that they did not have plans to meet their goals.

In addition, 16 agencies reported that they had met, or planned to meet, their cost savings targets, for a total of $2.36 billion in cost savings for fiscal years 2016 through 2018. This is about $0.38 billion less than OMB’s DCOI savings goal of $2.7 billion. This shortfall is the result of five agencies reporting less in planned cost savings and avoidances in their DCOI strategic plans as compared to the savings targets established for them by OMB. Three agencies did not have a cost savings target and did not report any achieved savings.

Regarding data center optimization, the 24 agencies reported limited progress in fiscal year 2018 against OMB’s five optimization targets. In this regard, 12 agencies reported that they had met at least one target, while 10 reported that they had not met any of the targets. Two agencies stated that they did not have a basis to report on progress as they did not own any data centers.

Further, 20 agencies did not plan to meet all of OMB’s fiscal year 2018 optimization goals. Specifically, only two agencies reported plans to meet all applicable targets, while six reported that they did not plan to meet any of the targets.

As a result of these findings, we recommended that 22 agencies take actions to meet the data center closure, cost savings, and optimization performance metrics targets, as appropriate. As of December 2019, none of the 36 recommendations had been fully addressed.

Appendix III: Detailed Analysis of Optimization Metrics

As noted previously in this report, the Office of Management and Budget (OMB) issued revised Data Center Optimization Initiative (DCOI) performance metrics in June 2019 as part of its revised DCOI guidance. According to OMB, the four current data center optimization metrics were intended to focus targeted improvements in key areas where agencies can make meaningful improvements and achieve further cost savings through optimization. OMB’s intent was to avoid using averages for metrics and instead identify metrics where agencies could demonstrate continuous improvement beyond the performance period of the June 2019 memorandum. OMB stated this would provide a more accurate measure of the agencies’ data center performance.

GAO published the Green Book, which provides the standards for internal control in the federal government and an overall framework for establishing and maintaining an effective internal control system. Such a control system addresses, in part, the attainment of a federal entity’s objectives, which is accomplished through monitoring specific performance measures. Such monitoring is also expected to assess the quality of performance over time.

In addition, the Green Book discusses the importance of clearly defining an entity’s objectives in order to determine what is to be achieved and to establish related performance measures. According to the Green Book, the controls represented by an agency’s performance metrics should include several key characteristics.

Clearly defined in measurable terms that are easily understood.

Objective and free of bias, rather than subjective.

Defined by appropriate parameters that allow for evaluating performance.

Understood by all levels of the organization, including what is being achieved with the metric, who is primarily responsible for achieving the metric, how the metric will be achieved, and when the metric will be achieved.

Aligned with internal and external requirements, including applicable legislation, regulations, and standards.

We compared each OMB optimization performance metric, as defined in the revised DCOI guidance and reported on OMB’s IT Dashboard, to the key effective metric characteristics identified in the Green Book. In assessing each of the OMB metrics against the key characteristics, we assigned one of three categories:

Met. The metric definition aligned with the characteristics of an effective metric.

Partially met. The metric definition aligned with some, but not all, the characteristics of an effective metric.

Not met. The metric definition did not align with the effective metric characteristics.

Virtualization

OMB’s virtualization metric counted the number of servers and mainframes serving as a virtual host in an agency-managed data center. We found that the virtualization metric met three characteristics, met two of four parts of one characteristic, and didn’t meet one. Table 6 provides our evaluation of the extent to which this OMB metric aligns with key characteristics of an effective metric.

Advanced Energy Metering

OMB’s advanced energy metering metric counted the data centers with advanced energy metering covering the majority of their floor space. We found that the advanced energy metering metric met two characteristics, met three of four parts of one characteristic, and did not meet two. Table 7 provides our evaluation of the extent to which this OMB metric aligned with key characteristics of an effective metric.

Server Utilization

OMB’s server utilization metric counts the number of underutilized production servers in federal data centers. We found that the underutilized servers metric met three characteristics, met two of four parts of one characteristic, and did not meet one. Table 8 provides our evaluation of the extent to which this OMB metric aligned with key characteristics of an effective metric.

Data Center Availability

OMB’s data center availability metric calculated the ratio of uptime (when the data center services were available) to unexpected downtime (unplanned service outages) in data centers. We found that the data center availability metric met two characteristics, met two of four parts of one characteristic, and did not meet two. Table 9 provides our evaluation of the extent to which the OMB metric aligned with key characteristics of an effective metric.

Appendix IV: Comments from the Department of Commerce

Appendix V: Comments from the Department of Homeland Security

Appendix VI: Comments from the National Aeronautics and Space Administration

Appendix VII: Comments from the Office of Personnel Management

Appendix VIII: Comments from the Department of State

Appendix IX: Comments from the Department of Defense

Appendix X: Comments from the U.S. Agency for International Development

Appendix XI: Comments from the Department of Housing and Urban Development

Appendix XII: Comments from the Social Security Administration

Appendix XIII: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the contact named above, individuals making contributions to this report included Dave Hinchman (Assistant Director), Justin Booth (Analyst-in-Charge), Lamis Alabed, Chris Businsky, Nancy Glover, Gina Hoover, and Jonathan Wall.