Cloud Computing: Agencies Have Increased Usage and Realized Benefits, but Cost and Savings Data Need to Be Better Tracked
Released: 2019-05-06T00:00:00

Letter

Background

Investments in federal IT have the potential to make agencies more efficient in fulfilling their missions by reducing costs and improving operational efficiencies. Each year, the federal government invests approximately $90 billion in IT, with about 75 percent reportedly spent on operating and maintaining existing systems. However, as we have previously testified, federal IT investments have too frequently failed or incurred cost overruns and schedule slippages while contributing little to mission-related outcomes. As a result, the federal government has spent billions of dollars on failed and poorly performing IT investments. These investments have often suffered from ineffective management of project planning, requirements definition, and program oversight and governance tasks.

Accordingly, in February 2015, we added improving the management of IT acquisitions and operations to our high-risk list—a list of agencies and program areas that have a higher potential for fraud, waste, abuse, and mismanagement, or are in need of transformation. In introducing this high risk area, we specifically noted that agencies spend a significant portion of their budgets on the operations and maintenance of IT systems and need to effectively manage these investments in order to ensure they continue to meet agencies’ needs and deliver value. We issued an update to our high-risk report in February 2017 and noted that, while progress has been made in addressing the IT acquisitions and operations high-risk area, significant work remains to be completed, including establishing action plans to modernize or replace obsolete investments.

In addition, over the last 3 decades, Congress has enacted several laws to assist agencies and the federal government in managing IT investments. For example, Congress enacted the Clinger-Cohen Act of 1996 to assist agencies in managing their investments. This act requires OMB to establish processes to analyze, track, and evaluate the risks and results of major capital investments in information systems made by federal agencies and report to Congress on the net program performance benefits achieved as a result of these investments.

Further, in December 2014, Congress enacted Federal Information Technology Acquisition Reform provisions (commonly referred to as FITARA) as a part of the Carl Levin and Howard P. ‘Buck’ McKeon National Defense Authorization Act for Fiscal Year 2015. The act requires OMB, among other things, to develop standardized performance metrics, including for cost savings and cost avoidances, and to submit quarterly reports to Congress on cost savings and reductions in duplicative information technology investments.

More recently, recognizing the challenges in modernizing government IT systems, in December 2017, Congress enacted the Modernizing Government Technology Act as part of the National Defense Authorization Act for Fiscal Year 2018. This law authorizes all covered agencies to establish an IT system modernization and working capital fund to, among other things, transition legacy systems to commercial cloud computing and other innovative commercial platforms and technologies using agency reprogrammed funds. The act also establishes a Technology Modernization Fund administered by the Administrator of General Services, in consultation with the CIO Council, which will provide funds to federal agencies for modernization efforts.

As of March 2019, the board that oversees the Technology Modernization Fund had awarded $60.87 million to four projects that plan to migrate or deploy systems to cloud services. Specifically,

GSA’s project, which received an award of $20.65 million, is intended to expedite the completion of a new software as a service solution for the agency’s payroll and work schedule and leave management within 2 years.

The Department of Housing and Urban Development’s project, which received an award of $20 million, is expected to accelerate the migration of five of the agency’s most critical business systems from an on-premise mainframe database to the cloud within the next 2 years.

Energy’s project, which received an award of $15.2 million, is intended to help the agency move 45 separate on-premise email systems to the cloud within the next 3 years.

Agriculture’s project, which received an award of $5 million, is intended to help the agency migrate 10 applications to a shared services cloud platform model.

Overview of Cloud Services

One approach to improving the government’s management of IT services is through cloud computing. As mentioned previously, cloud computing is a means for enabling on-demand access to shared pools of configurable computing resources (e.g., networks, servers, storage applications, and services) that can be rapidly provisioned. More specifically, purchasing IT services through a cloud service provider enables agencies to avoid paying for all the computing resources that would typically be needed to provide such services. This approach offers federal agencies a means to buy services more quickly and possibly at a lower cost than building, operating, and maintaining these computing resources themselves.

According to NIST, cloud computing offers federal agencies a number of benefits:

On-demand self-service. Agencies can, as needed, provision computing capabilities, such as server time and network storage, from the service provider automatically and without human interaction.

Broad network access. Agencies can access needed capabilities over the network through workstations, laptops, or other mobile devices.

Resource pooling. Agencies can use pooled resources from the cloud provider, including storage, processing, memory, and network bandwidth.

Rapid elasticity. Agencies can provision the resources that are allocated to match what actual resources are needed according to demand. This is done by scaling resources up or down by adding or removing processing or memory capacity, or both, according to demand.

Measured service. Agencies can pay for services based on usage.

This allows agencies to monitor, control, and generate reports, providing greater transparency into the agency’s use of cloud services.

As noted in NIST guidance, cloud service providers have established three types of service models that are offered to consumers: Infrastructure as a service. The service provider delivers and manages the basic computing infrastructure of servers, software, storage, and network equipment. The consumer provides the operating system, programming tools and services, and applications.

Platform as a service. The service provider delivers and manages the infrastructure, operating system and programming tools and services, which the consumer can use to create applications.

Software as a service. The service provider delivers one or more applications and all the resources (operating system and programming tools) and underlying infrastructure to run them for use on demand.

NIST has also defined four types of cloud deployment models, including:

Private cloud. Service is set up specifically for one organization, although there may be multiple customers within that organization and the cloud may exist on or off the customer’s premises.

Community cloud. Service is set up for organizations with similar requirements. The cloud may be managed by the organizations or a third party and may exist on or off the organization’s premises.

Public cloud. Service is available to the general public and is owned and operated by the service provider.

Hybrid cloud. Service is a composite of two or more of the three deployment models (private, community, or public) that are bound together by technology that enables data and application portability.

According to NIST guidance, these deployment models impact the number of consumers and the nature of other consumers’ data that may be present in the cloud environment. A public cloud should not allow a consumer to know or control other consumers of a cloud service provider’s environment. However, a private cloud can allow for ultimate control in selecting who has access to a cloud environment. Community clouds and hybrid clouds allow for a mixed degree of control and knowledge of other consumers. Additionally, the cost for cloud services typically increases as control over other consumers and knowledge of these consumers increase.

OMB and Past and Current Administrations Have Undertaken Efforts to Increase Use of Cloud Services

In December 2010, OMB made cloud computing an integral part of its 25 Point Implementation Plan to Reform Federal Information Technology Management. The plan called for the development of a government- wide strategy to hasten the adoption of cloud services. To accelerate the shift, OMB required agencies to identify three systems to migrate to cloud services, create a project plan for migration, and migrate all three systems by June 2012.

In February 2011, OMB issued the Federal Cloud Computing Strategy, as called for in its 25-point plan. The strategy provided definitions of cloud services; benefits of cloud services, such as accelerating data center consolidations; a decision framework for migrating services to a cloud environment; case studies to support agencies’ migration to cloud services; and roles and responsibilities for federal agencies. For example, the strategy states that NIST’s role is to lead and collaborate with federal, state, and local government agency CIOs, private sector experts, and international bodies to identify standards and guidance and prioritize the adoption of cloud services.

Subsequently, in December 2011, OMB established the Federal Risk and Authorization Management Program (FedRAMP), a government-wide program to provide joint authorizations and continuous security monitoring services for cloud services for all federal agencies. GSA initiated FedRAMP operations, which the agency referred to as initial operational capabilities, in June 2012.

In 2012, OMB began requiring agencies to evaluate each investment, or components or systems within the investment, for cloud services, regardless of the overall life-cycle stage of the investment. Agencies were required to report the status of each investment’s evaluation as part of the annual budget submission, as noted in OMB’s annual capital planning guidance. Specifically, OMB required agencies to select an option regarding whether they had evaluated a cloud alternative and chosen a cloud alternative with a particular cloud deployment model or indicate that they had not yet evaluated the investment for cloud services. Starting in fiscal year 2018, OMB revised the options that agencies were to select from and required agencies to select an option regarding whether the investment, or a portion of the investment, was leveraging cloud computing, or indicate that cloud computing had not been considered for the investment.

In 2012, OMB began requiring agencies to report associated cloud spending, as called for in its annual capital planning guidance. For fiscal years 2015 through 2018, OMB’s capital planning guidance required agencies to report their total cloud spending at the agency level based on the cloud deployment model, rather than by individual investment.

Starting in fiscal year 2019, OMB will require agencies to report total cloud spending by investment and use the Technology Business Management Framework. The Framework provides a cost taxonomy for agencies to use to manage the cost, quality, and value of their IT services. Specifically, agencies will be required to use a standard set of cost categories to group IT spending, including cloud-related spending. This new model is intended to increase the granularity in reporting of agency IT budget and spending data.

In addition, in May 2017, the administration established the American Technology Council to help transform and modernize federal IT and how the government uses and delivers digital services. The President is the chairman of this council, and the Federal CIO and the United States Digital Service Administrator are among its members.

Subsequently, in December 2017, the American Technology Council issued a Report to the President on Federal IT Modernization and made eight cloud computing-related recommendations that are relevant to the focus of our review. For example, the report recommended that OMB issue two data calls to agencies in order to: (1) obtain a list of agency in- progress and pending projects for cloud migration; and (2) have agencies identify systems that have not yet migrated due to perceived or encountered difficulties. Based on the information provided, OMB would then assist agencies in making transition plans and work to remove obstacles in order to accelerate cloud adoption. In addition, the report recommended that OMB take action to update its guidance related to cloud computing and revise the Federal Cloud Computing Strategy that was previously issued in 2011.

According to staff in OMB’s Office of E-Government and Information Technology, OMB has taken action to address these recommendations. For example, the staff reported that the two data calls were issued in December 2017 and staff are currently reviewing the information provided by agencies in response. In addition, OMB issued its draft strategy revision, the 2018 Federal Cloud Computing Strategy, for comment on September 24, 2018. This proposed Cloud Smart policy outlines a strategy for agencies to adopt cloud solutions that streamline transformation and embrace modern capabilities.

According to the draft strategy, Cloud Smart focuses on equipping agencies with the tools needed to make informative technology decisions in accordance with their mission needs. In addition, the draft strategy indicates that OMB intends to leverage private-sector solutions to provide the best services to the American people. The strategy also notes that the CIO Council and Chief Financial Officer Council are to work with OMB, GSA, DHS, and other federal entities to develop a work plan of actions and targeted policy updates that are to be delivered over the next 18 months. For more information about the current status of each of these eight cloud recommendations, as reported by OMB, please see appendix II.

Prior GAO Reports on Efforts to Implement Cloud Services

During the past several years, we reported on federal agencies’ efforts to implement cloud services, and on the progress that oversight agencies have made to help federal agencies in those efforts. For example, in July 2012, we reported that the seven federal agencies we reviewed had made progress in meeting OMB’s requirement to implement three cloud services by June 2012. Specifically, the seven agencies had implemented 21 cloud services and spent a total of $307 million for cloud computing in fiscal year 2012—about 1 percent of their total IT budgets. In addition, while all seven agencies had submitted plans to OMB for implementing cloud solutions, all but one plan were missing key required elements. We made 14 recommendations to the seven agencies to develop planning information, such as estimated costs and legacy IT systems’ retirement plans for existing and planned services. The agencies generally agreed with, and implemented, 13 out of 14 of our recommendations.

In September 2014, we reviewed the efforts of the same seven federal agencies again and found that each of them had implemented additional cloud services subsequent to our July 2012 report. In particular, the total number of cloud services implemented by the seven agencies had increased by 80 services, from 21 to 101. The seven agencies’ reported spending on cloud services had also increased by $222 million, from $307 million in 2012 to $529 million in 2014. However, this relatively small increase in cloud spending was attributed, in part, to the fact that these agencies had not considered cloud services for 67 percent of their investments. Accordingly, we recommended that the seven agencies assess their IT investments for suitability for cloud services. The agencies generally agreed with our recommendations and 6 of the agencies (Agriculture, DHS, GSA, HHS, SBA, and State) implemented all of our recommendations.

Further, in April 2016, we identified 10 key practices that federal and private-sector guidance noted should be included in service-level agreements in a contract when acquiring IT services though a cloud services provider. However, our review of five agencies’ (Defense, DHS, HHS, Treasury, and VA) cloud service contracts found that not all 10 key practices were included in these contracts. We therefore made recommendations to OMB to include all 10 key practices in future guidance to agencies. We also recommended that the five agencies incorporate these key practices as their contract and service level agreements expire. The agencies generally agreed with our recommendations and, to date, Defense and DHS have taken action to implement the recommendations.

More recently, in April 2017, we highlighted the results of a forum, convened by the Comptroller General on September 14, 2016, to explore challenges and opportunities for CIOs to improve federal IT acquisitions and operations—with the goal of better informing policymakers and government leadership. Thirteen current and former federal agency CIOs, members of Congress, and private-sector IT executives who participated in the forum noted challenges with agency operations that could be addressed by migrating more services to the cloud. In their view, this approach would offer agencies a means to buy the services faster and possibly at a lower cost than through the traditional methods of building and maintaining systems.

In addition, forum participants noted the importance of federal agencies’ IT procurement offices and processes evolving to align with new technologies, as agencies are not always set up to take advantage of cloud services. Lastly, forum participants said that, as the federal government is expected to increase its purchase of IT as a service with the move toward cloud computing, more oversight is needed to ensure that appropriate contracts are in place and appropriate oversight of performance occurs.

Most of the Selected Agencies Reported Making Progress in Implementing Cloud Services

The16 selected agencies reported making progress in implementing cloud services—namely, they established guidance for assessing investments for cloud services, performed those assessments, and implemented cloud services for their investments. However, the extent of these agencies’ progress varied. Specifically, 10 of the 16 agencies established guidance for assessing all new and existing investments for cloud services, while six agencies did not. In addition, while these agencies had assessed the majority of their investments for cloud services planned for fiscal year 2019, 12 agencies had not completed an assessment of 10 or more IT investments for cloud services. Lastly, 10 of the agencies reported a percentage increase in the use of cloud services from fiscal year 2016 through fiscal year 2019, while two agencies reported no percentage change and four agencies reported a decrease during this 4-year period.

About Two-thirds of the Selected Agencies Had Established Guidance for Assessing Investments for Cloud Services Suitability

OMB’s Cloud First policy, issued in February 2011, requires each agency’s CIO to implement a cloud service whenever there is a secure, reliable, cost-effective option to do so. Further, subsequent OMB capital planning guidance, issued in 2014, requires agencies to evaluate each investment, or components or systems within the investment, for cloud services, regardless of the overall life-cycle stage of the investment.

While OMB’s guidance is not specific on how agencies should conduct these evaluations, GAO’s Information Technology Investment Management framework notes that organizations should have documented policies and procedures for the management oversight of IT investments, including the selection of investments and the evaluation of information technologies that have the potential to improve the organization’s business.

Ten of the 16 agencies we reviewed had established guidance in accordance with OMB’s requirement to assess new and existing IT investments for suitability for cloud services, as of August 2018. In particular, all 10 agencies’ guidance required assessments of cloud service suitability for both new and existing IT investments.

However, the remaining six agencies did not have such comprehensive guidance in place. Rather, the guidance either required assessments of new or existing systems for cloud services but not both, or the guidance had not yet been established. Specifically, Labor’s and SSA’s guidance required assessments of new investments for cloud services but did not address assessments of existing investments. In addition, Energy’s guidance required assessments of existing systems but not new acquisitions. Further, three agencies (Education, HHS, and Transportation) had not established guidance for assessing investments for cloud services.

The results of our analysis of agencies’ guidance on assessing IT investments for cloud services are shown in figure 1.

Agency officials in the Office of the CIO at the six agencies provided a variety of reasons for why they did not have guidance for assessing all investments for cloud services. Specifically, Labor officials reported that they had not included legacy applications in their guidance because not all applications should or could be migrated to the cloud. In addition, SSA officials reported that they were planning to assess all existing systems for cloud services, but had not determined a time frame for this review. Further, Energy officials reported that, while the agency was following OMB’s Cloud First policy, it would need to establish guidance for assessing new investments for cloud services; however, a date for doing so had not been determined.

Transportation officials reported that they believed their guidance on managing cloud computing efforts was consistent with OMB’s Cloud First policy and stated that they had no plans to develop additional guidance. However, our review of the agency’s guidance found that it did not include any information regarding the assessment of investments for cloud services. Instead, the guidance only required that investments intending to use cloud services provide procurement and other cost information as part of the business case and use specified language in contracts with cloud service providers. Therefore, we believe that the guidance is not consistent with OMB’s guidance requiring agencies to assess investments for cloud services.

Education officials reported that they were in the process of finalizing a policy and hoped to have it completed by the end of the year. In addition, HHS officials reported that they had explored developing some guidance regarding cloud services, but had not established any plans to do so.

As previously discussed, assessing all new and existing IT investments to determine whether they are suitable for cloud services is an important component of OMB’s Cloud First policy. Until the six identified agencies update or establish guidance for assessing both new and existing investments for cloud services, they will not be positioned to ensure adequate implementation of OMB’s Cloud First policy. Further, these agencies increase the risk that they will not be able to take advantage of cloud services to improve operational efficiencies and minimize costs.

Selected Agencies Assessed the Majority of Their IT Investments for Cloud Services

As noted previously, OMB’s fiscal year 2016 IT capital planning guidance requires agencies to evaluate each investment for cloud services and report the status of this evaluation as part of the annual budget submission. Specifically, agencies were to respond to a question regarding whether they had selected cloud services for the investment, or components or systems within the investment, or, for example, report that the investment had not yet been assessed for cloud services. OMB publicly reports agencies’ responses to this question on the IT Dashboard.

As of October 9, 2018, the 16 agencies in our review reported on the IT Dashboard that they had completed cloud assessments for 84 percent of their IT investments (5,180 out of a total of 6,157) planned for fiscal year 2019. Of these, two agencies (GSA and State) had completed an assessment of all investments. However, 12 agencies had not completed an assessment of 10 or more IT investments for cloud services.

Table 1 lists the number of IT investments at the 16 selected agencies for fiscal year 2019 that had been assessed for cloud services. The table also shows the number and percentage of investments that remained to be assessed.

Officials in the Office of the CIO at the 12 agencies provided a variety of reasons for why they had not assessed all investments for cloud services. For example, Agriculture officials reported that 21 of their 53 investments did not need assessments because the investments were not suitable for cloud services. The officials said they intended to update the IT Dashboard to reflect this change. Further, these officials stated that they planned to assess the remaining 32 investments by April 30, 2019.

Defense officials reported that the agency was in the process of adjusting its cloud strategy that was issued in January 2018 and intends to address investments that have not yet been evaluated. However, the officials stated that they had not established time frames for the evaluations. In addition, DHS officials reported that they were in the process of implementing their guidance and putting in place a new process for identifying planned acquisitions based on the phase in the acquisition life cycle. However, the officials had not identified a time frame for when the new process would be finalized or when all assessments of the investments would be completed.

Justice officials reported that, as they began the budget process, the agency planned to look at performing additional assessments of investments for cloud services. However, the officials provided no time frames for when these assessments would be completed. In addition, SSA officials stated that the agency planned to perform an assessment of current investments for cloud services. However, the officials reported that they had not established a time frame for completing these assessments.

Further, Treasury officials reported that, while the agency had established a process for assessing investments for cloud services, it did not set specific dates for when the assessments were to be conducted. These officials reported that they only conducted a cloud assessment if the agency determined that it would replace, redevelop, or retire an investment. However, Treasury’s guidance is not consistent with OMB’s requirement that agencies conduct an annual assessment of all investments, regardless of the overall life-cycle stage of the investment.

Many of the 16 agencies in our review have made progress in implementing cloud services by establishing guidance for assessing investments for cloud services and performing assessments. Even agencies that lacked formal guidance for performing an assessment have made progress in increasing the use of cloud services when the assessment was completed. Nevertheless, 12 agencies still need to assess a large number of their investments. Until these agencies assess their investments that have yet to be evaluated for cloud services, they may not know which investments are likely candidates for migration to cloud services. Moreover, these agencies will not be positioned to take advantage of operational efficiencies, cost savings, and other benefits from the use of cloud services.

Selected Agencies Have Increased Their Use of Cloud Services

As of October 9, 2018, the 16 agencies in our review reported on the IT Dashboard that 11 percent of their IT investments were projected to use cloud services for fiscal year 2019—an increase of 3 percentage points from fiscal year 2016 to fiscal year 2019. In addition, 13 out of the 16 agencies reported that they planned to increase their use of cloud services, in some cases, by as much as 20 percentage points or more, between fiscal years 2018 and 2019.

Table 2 lists the percentage of the selected agency IT investments that used cloud services for fiscal years 2016 through 2018 and are projected for 2019. (For additional details on the number of cloud investments and the total investments reported by each of the selected agencies for fiscal years 2016 through 2019, see appendix III.)

In addition, while the majority of agencies made progress in implementing cloud services between fiscal years 2016 and 2019, the extent of agencies’ progress varied. Specifically, 10 of the 16 agencies reported an increase in the use of cloud services, with the percentage of increase varying from up to 10 percentage points to 20 or more percentage points.

For the remaining six agencies, two reported no change in the percentage of investments using cloud services and four reported a decrease in the overall percentage of cloud usage. Figure 2 shows the breakdown in the range of percentage point changes in the use of cloud services for agency investments for fiscal years 2016 through 2019, as reported on the IT Dashboard.

Officials in the Offices of the CIO, and Office of Information Technology, at the six agencies that reported no change or a decrease in their cloud investment percentages during this 4-year period provided a variety of reasons for why this was the case, or had no comments regarding the lack of change in their cloud investment percentages. Specifically, Energy officials reported that the agency had not shown an increase in the percentage of its cloud investments due to an IT portfolio optimization effort designed to consolidate the agency’s cloud investments. According to the officials, this optimization effort was designed to reduce the total number of these investments during the 4-year period. As a result, this optimization effort affected the overall percentage of cloud investments. As for Labor, its officials did not offer any comments regarding the lack of an increase in cloud use during this period.

In addition, Defense, DHS, and Education officials reported that staff in their agencies had inconsistently applied the definition of cloud computing, which had led to differences in identifying and reporting the number of cloud investments within their agencies during this period. Further, DHS officials noted that the ongoing addition, combination, completion, and cancellation of investments had contributed to the fluctuation in the number of cloud investments within their agency. Finally, VA officials reported that their cloud identification processes were maturing during this period and, as such, had resulted in different cloud investment counts.

Some of the inconsistencies reported by agencies regarding the types of investments that they identified as being cloud investments may also be a result of OMB’s changes to its guidance during this 4-year period. Specifically, OMB changed how agencies were required to report their use of cloud services in fiscal year 2018, and revised the options that agencies were to select from in order to identify and report the use of cloud services for each investment.

Going forward, several agencies reported that they intended to continue making progress in their implementation of cloud services beyond fiscal year 2019. For example:

Education officials reported that the agency expected to increase cloud use significantly in 2019 and beyond due to an IT services contract award that is to support the migration of the agency’s primary hosting infrastructure to the cloud.

DHS officials reported that the agency had set aggressive goals for acquiring cloud services. Toward this end, the agency had initiated a cloud steering group and created a team with staff from all components. In addition, the officials reported that they planned to consolidate space in one data center and eliminate another data center, which would allow the agency to accelerate its migration to the cloud.

Justice officials reported that they expected to increase spending on cloud services as the agency completed ongoing initiatives in 2019. In addition, the officials reported that they anticipated migrating the majority of the agency’s unclassified data to the cloud in the next few years.

VA officials reported that the agency planned to migrate at least 350 applications to the cloud by 2024.

Agencies’ efforts to acquire additional cloud services and take advantage of improved efficiencies and cost savings should help to further improve their management of IT acquisitions and operations.

Agencies Have Increased Spending and Realized Savings from Using Cloud Services, but Spending and Savings Figures Are Underreported

The 16 agencies in our review made progress in implementing cloud services. Specifically, the 16 agencies reported that their spending on cloud investments had increased by over $1 billion between fiscal years 2015 and 2018 for investments with total life-cycle costs of $1 million or more. Nevertheless, the agencies reported that factors such as inconsistent tracking of spending data, along with confusion in interpreting OMB guidance, impacted the accuracy of their reported cloud spending data. In addition, 13 of the 16 agencies provided savings data indicating that they had saved hundreds of millions of dollars on cloud services, but agencies reported that they had problems with tracking this data. Further, six agencies reported that they had reinvested cloud savings into other IT modernization efforts or other improvements to IT services.

Selected Agencies Are Spending More on Cloud Services, but Do Not Have Complete Spending Data

OMB requires agencies to report spending on cloud services. Specifically, OMB’s annual capital planning guidance for fiscal years 2015 through 2018 required agencies to report their total cloud spending on the IT Dashboard, although it did not require the information to be reported by investment.

While the selected agencies’ reporting on the IT Dashboard indicated that their percentage of total spending on cloud services generally remained constant during fiscal years 2015 through 2017. Specifically, the 16 agencies reported on the IT Dashboard that approximately 3 percent of their total IT spending each fiscal year during this period was spent on cloud services. For fiscal year 2018, agency-reported cloud spending through March 2018 was at 2 percent. Table 3 identifies the percentage of the selected agency spending on cloud services for fiscal years 2015 through 2018, as reported on the IT Dashboard. (For additional details on total agency cloud spending and total IT spending reported by each of the selected agencies for fiscal years 2015 through 2018, see appendix IV.)

However, the breakdown in spending by investment for cloud services with $1 million or more in life-cycle costs that the 16 agencies provided to us, showed that their spending on cloud investments had increased during fiscal years 2015 through 2018, and beyond (agencies generally submitted data on planned spending for one or more fiscal years beyond 2018). Specifically, the agencies’ provided data showed that total cloud spending for these investments was approximately $1.38 billion in fiscal year 2017—an increase of over $1 billion since fiscal year 2015. In addition, the 16 agencies’ data indicated that they plan to spend over $3.2 billion on cloud services in fiscal year 2018 and beyond for these investments.

Table 4 summarizes the information provided to us on a breakdown of the 16 selected agencies’ total spending for investments with $1 million or more in life-cycle costs for cloud services, from fiscal years 2015 through 2018 and beyond through fiscal year 2024, that was submitted to us. (For a list of the investments that have spent $1 million or more in life-cycle costs for cloud services, provided by each of the 16 selected agencies for fiscal year 2018, see appendix V.)

Officials in the Office of the CIO at all of the agencies in our review identified three factors that could affect the completeness of the cloud spending data provided to us and on the IT Dashboard: (1) spending data were not consistently tracked; (2) different methods were used to calculate cloud spending costs; and (3) interpreting changes in OMB and related guidance created confusion regarding what spending data should be tracked.

Spending data were not consistently tracked. Defense officials reported, for example, that the agency had only begun tracking cloud spending in fiscal year 2016 and, therefore, spending data were not available for fiscal year 2015. In addition, VA officials reported that they were in the process of maturing their tracking of cloud spending data and, therefore, the agency did not have spending data available for the majority of their investments prior to fiscal year 2019. Further, Justice officials reported that the agency had been challenged to track cloud costs because the costs are based on fluctuating usage, rather than a flat rate.

Different methods were used to calculate cloud spending costs.

Some agencies reported that the data they provided to us included costs for items such as power usage and staff full-time equivalents, while other agencies told us that they only provided contract costs. In addition, some agency officials noted that they included in the provided spending figure, the additional costs for migrating the application to cloud services, while other officials said that these costs had not been included in their spending totals.

Interpreting changes in OMB and related guidance created confusion regarding what spending data should be tracked. Agencies noted that OMB made changes to its guidance since 2015, including clarifications to the definition of cloud computing, changes to the definition and scope of cloud services and cloud spending, and changes to the guidance regarding what applicable costs should be included in spending totals—all of which created confusion regarding what investments and what costs should be tracked for cloud services. Defense officials also reported that the agency misinterpreted the NIST definition of cloud computing, and, as a result, Defense misreported that certain IT investments were using cloud services, when these investments were not using these services. Defense officials reported that the agency had corrected this issue but it affected the total cloud spending reported during this period and led to the decrease in spending noted.

Based on our review of these factors reported by the 16 selected agencies, we identified issues with the completeness of the reported cloud spending data. Specifically, these factors increase the likelihood that all costs associated with spending on cloud services may have been incompletely captured by the 16 selected agencies in our review. As a result, agencies’ reported total cloud spending on the IT Dashboard and the data provided to us is likely underreported.

Staff in OMB’s Office of E-Government and Information Technology stated that agencies have previously reported challenges in breaking out cloud costs, particularly when the cloud acquisition is part of a larger contract. Given these challenges, the staff acknowledged that agency- reported cloud spending data are underreported and stated that the IT Dashboard reflects only a fraction of actual federal spending on cloud services. However, the staff stated that OMB’s changes to its guidance, beginning in fiscal year 2019, should help to improve the reporting of cloud spending data. Specifically, beginning in fiscal year 2019, agencies will be required to report total cloud costs by investment, per OMB’s IT capital planning guidance, and use the Technology Business Management framework.

Having complete data on spending for cloud services is critical in order to ensure that agencies can provide effective management and oversight of their cloud use, and that OMB and lawmakers can hold CIOs accountable for the performance of these cloud investments. The changes to OMB’s guidance for fiscal year 2019 provide a key improvement for ensuring that agencies establish more consistent processes for reporting on cloud- spending and should help agencies improve the completeness of the cloud-spending data that they report to OMB.

Selected Agencies Report Saving Approximately $291 Million from the Use of Cloud Services but Acknowledge Data Are Incomplete

Since 2013, OMB has required agencies to report quarterly on their total savings and cost avoidances from implementing OMB’s IT reform initiatives, including savings realized from the migration to cloud services. Specifically, agencies are required to report actual and planned savings from implementing these initiatives in a quarterly submission and identify which implementation of an OMB initiative resulted in the reported savings. Despite this, in the reporting mechanism, agencies can only associate specific savings with certain OMB initiatives, a list that does not include the migration to cloud services. Standards for Internal Control in the Federal Government emphasizes that management should track the actual performance of key initiatives in order to ensure that these activities are meeting plans, goals, and objectives, and in doing so, management should use quality information.

Thirteen of the 16 agencies in our review provided savings data to us for at least one cloud investment with life-cycle costs of $1 million or more for cloud services during fiscal years 2014 through 2018. In total, the 13 agencies’ provided data showed that they had accrued approximately $291 million in savings or cost avoidances using cloud services since 2014. In addition, the agencies’ data indicated that they planned to save at least $150 million in fiscal year 2018 and beyond (agencies generally submitted data on planned savings for one or more fiscal years beyond 2018).

However, agency officials from the 13 agencies stated that, while they were able to provide some savings data, these data are only tracked on an ad hoc basis for certain cloud investments. In addition, officials from three agencies (Defense, State, and SSA) stated that they could not provide savings data for any of their cloud investments. As a result, the 16 agencies were unable to provide savings or avoidance data for 411 out of 488 investments (84 percent) that we reviewed.

Table 5 shows, for the selected agencies in our review, the breakdown in total agency savings and cost avoidances for fiscal years 2014 through 2018 and beyond for investments with $1 million or more in life-cycle costs for cloud services.

Officials in the Office of the CIO at the 16 agencies identified three factors that impacted their efforts to provide data on savings or cost avoidances for cloud computing investments: (1) savings data were not systematically tracked or were hard to track; (2) deploying or migrating systems to the cloud had resulted in no cost savings; and (3) OMB does not require agencies to identify savings associated with cloud services as part of reported savings.

Savings data were not systematically tracked or were hard to track. Defense, Treasury, and VA officials reported that their investment management systems did not have the capability to track cloud savings or avoidance data. In addition, GSA officials reported that, while their system had the capability to track cost savings data, the agency did not capture and track realized cloud savings in a consistent format. GSA officials stated that they were in the process of implementing the Technology Business Management Framework, which they expected would improve the collection of these data. However, the officials did not identify a time frame for when this framework was to be implemented.

Education officials reported that the agency did not provide cost savings data for those investments where cost savings targets had not been established or anticipated. SBA officials reported that investments with two cloud providers had only been recently made so the agency could not yet make a reasonable assessment of savings. Further, Agriculture officials reported that the agency had a hard time tracking the savings from certain investments because the process for formulating the overall agency budget was different than the process for determining savings from cloud implementation.

State officials reported that they were in the process of developing the capability to collect and track savings data from using cloud services but did not have any reliable data to provide during our review. In addition, Energy officials reported that their agency intended to establish review processes in the coming year to ensure that costs, cost savings, and cost avoidances were tracked for all cloud investments. As part of this process, the agency intended to work closely with its components to ensure that there was a consistent application of definitions for cloud spending and savings. However, the officials did not identify a specific time frame for when the agency expected the new process to be completed.

Lastly, HHS officials reported that the agency did not expect to track cost savings beyond the FITARA requirements. FITARA requires the reporting of savings associated with two OMB initiatives—data center consolidation and PortfolioStat. However, per M-13-09, OMB requires agencies to report savings associated with all of its initiatives. As such, HHS’s tracking of savings is not consistent with OMB’s guidance.

Deploying or migrating systems to the cloud resulted in no cost savings. Treasury officials reported that their agency had not realized any cost savings from the migration of certain investments because the acquisition of cloud services either had allowed the agency to purchase additional capabilities that the previous system did not have, or the agency had continued to operate the previous system at the same time as the new cloud system for a period of time. In addition, Commerce officials reported that their agency had not realized any cost savings for some investments because acquiring cloud services required that new business and performance requirements be put in place, which resulted in no overall savings for these investments. Further, DHS and SSA officials reported that a number of their investments were new applications that were developed and deployed in the cloud. As such, there were no costs from a prior system that could be compared with the costs to maintain the new system using cloud services; thus, there were no associated cost savings or avoidances.

OMB does not require agencies to identify savings associated with cloud services as part of reported savings. Officials from Agriculture, Justice and Transportation noted that, while OMB requires agencies to report savings, current reporting instructions do not specifically require the identification and reporting of cloud savings as a separate category of cost savings and avoidance. In this regard, OMB’s guidance requires agencies to identify which OMB initiative resulted in the reported savings, but the available options for agencies to choose from do not include cloud services. Accordingly, officials from these agencies stated that they either reached out to their components to try and collect this information or had to review their investments to determine whether there were any cloud savings, to be able to provide this information to us.

Based on our review of the factors that impacted the selected agencies’ efforts to provide savings data, we identified issues with the completeness of the savings data. Specifically, challenges identified by the selected agencies in systematically tracking savings data, and the lack of a specific OMB requirement to report savings associated with cloud services, increase the likelihood that all savings associated with cloud services may have been incompletely captured by the agencies that provided these data. As a result, agencies’ reported cloud savings data on the IT Dashboard and the data provided to us is likely underreported.

Staff in the Office of E-Government and Information Technology stated that, while agencies are required to report total savings related to OMB initiatives, the format is left to agency discretion. In addition, OMB staff confirmed that agencies do not have to specifically identify savings related to cloud computing unless they choose to do so. OMB staff further said that they do not require a specific format for reporting savings in order to minimize the burden on agencies in reporting this information.

While OMB’s effort to minimize the reporting burden on agencies is appropriate, the lack of an explicit requirement to identify reported savings associated with cloud services has contributed to agencies not consistently tracking these savings. In addition, while OMB has taken steps to ensure more accuracy and granularity in agency reporting of cloud investment spending data in fiscal year 2019, there has not been a corresponding effort to ensure better reporting of cloud savings data. As a result, OMB and Congress may not have sufficient data to see the results of key initiatives, like Cloud Smart, and understand whether agencies are achieving savings using cloud services.

Since 2013, OMB has required agencies to report on the savings resulting from implementation of its key IT reform initiatives. Although OMB does not provide the means for agencies to explicitly identify cloud- related savings, it is nevertheless important for agencies to take steps to fully track savings and cost avoidances from cloud computing acquisitions in order to ensure effective oversight and management of these initiatives. However, until OMB establishes a specific cloud savings reporting requirement, and until these agencies establish a consistent and repeatable mechanism to track these savings and cost avoidances, the agencies may lack sufficient information on the results of cloud acquisitions to date and the data necessary to make decisions regarding future cloud acquisitions.

Agencies Reported Reinvesting Cloud Implementation Savings into IT Modernization or Other Improvement Efforts

In 2017, Congress enacted what is known as the Modernizing Government Technology Act, which authorized covered agencies to establish an IT system modernization and working capital fund. This fund was to be used to, among other things, transition legacy IT systems to commercial cloud computing and other innovative commercial platforms and technologies using agency reprogrammed funds or amounts made available to the IT working capital fund through discretionary appropriations.

Regardless of the extent of agencies’ processes for tracking savings obtained from using cloud services, officials in the Office of the CIO at six agencies in our review (Education, GSA, Labor, SBA, SSA, and Treasury) reported that they have reinvested these savings into other IT modernization efforts or other improvements to IT services. For example:

Education officials reported that $498,000 in fiscal year 2018 cloud savings was used to modernize the agency’s network infrastructure in order to provide increased multipath bandwidth and software that automatically routes traffic if network issues occur.

GSA officials reported that the agency used the savings from replacing the agency’s legacy on-premises email program with a cloud-based email system to implement a modern enterprise collaboration platform, email, and document storage system. According to the officials, the move to the cloud helped improve the agency’s flexibility (the new system is accessible from any device, at any time, and from any location), productivity, and cost-effectiveness. As part of this effort, the officials reported that the savings were managed using the agency’s working capital fund.

Labor officials reported that their agency is using savings and cost avoidances to partially fund an initiative to consolidate cloud services within the agency in order to provide future secure cloud services and establish an enterprise contract vehicle to obtain cloud services. The officials noted that this investment is intended to allow the agency’s components to leverage a cloud authority to operate, obtain competitive pricing for, and establish communications to cloud service providers. In addition, Labor officials reported that the agency has established a working capital fund that is to be used to manage the savings from cloud and shared services.

SBA’s CIO reported that the agency reinvested $7.8 million in savings from efforts to consolidate data centers to the cloud toward the implementation of other enterprise-wide modernization efforts. In addition, the agency had used the savings for the deployment and migration of additional applications to cloud services. Specifically, the CIO reported that the savings were used to design and architect cloud services, roll out the agency’s update of key operating system and office applications, decommission obsolete data center assets, reduce overlapping technologies, and enhance security and compliance capabilities with new enterprise tools and network monitoring. In addition, the CIO stated that, by using the savings from the data center consolidation, SBA has been able to undertake all of the agency’s cloud modernization efforts with no additional budgeted funding.

We have previously reported that significant work remains to ensure that agencies improve their management of IT acquisitions and operations, including modernizing or replacing obsolete IT investments. It is encouraging that several agencies have reinvested savings from cloud initiatives into other IT modernization efforts and, in some cases, have taken advantage of working capital funds authorized by Congress to do so. Having complete information on the savings or avoidances that result from cloud initiatives and using those savings to further IT modernization efforts is critical to ensuring the transformation of IT services across the federal government in the future.

Selected Agencies Have Realized Benefits from Cloud Services

Officials from 15 of the 16 agencies in our review reported that they had realized several significant benefits from the adoption of cloud services, ranging from improvements in the delivery of IT services to increasing the efficiency of operations and systems. In addition, the 15 agencies noted that certain key practices enabled them to realize these benefits through the successful implementation of cloud services. These practices included establishing new governance planning activities and policies, reorganizing the management of agency IT resources, and having executive leadership involved to help drive acquisition efforts.

Cloud Services Aid with IT Efficiency, Cost Savings, and System Modernization

Officials in charge of cloud services at 15 of the 16 agencies in our review reported that they had identified five significant or notable benefits as a result of acquiring cloud services. Specifically, the 13 agencies reported that they had improved customer experiences through better design and performance of business systems and customer websites. In addition, all 15 agencies reported that they were able to procure more flexible and scalable IT resources, and reduce the cost of provisioning infrastructure and managing services. Table 6 lists the five significant or notable benefits reported by the 15 agencies and the number of agencies that reported each benefit. The discussion that follows the table provides examples of each of the five agency-reported benefits from the acquisition of cloud services.

Officials in the Office of the CIO at the 15 agencies reported that acquiring cloud services had allowed them to procure IT resources that were more flexible and scalable than the prior legacy infrastructure. For example, officials in Labor’s Office of the CIO reported that they had acquired cloud services to address seasonal demands for system processing. By eliminating the need to purchase additional servers and other equipment that would go unused during the rest of the year, Labor officials reported that cloud services allow the agency to scale resources up during these periods of increased processing and then scale the resources back down when the excess capacity is no longer needed.

In addition, officials in DHS’s Office of the CIO reported that, in 2012, they had acquired software as a service for the agency’s virtual desktop solution. This new service provided six agency components access to virtual secure desktop operating systems and applications. By eliminating the need for users to be physically present in a specified location in order to perform work activities, DHS officials reported that cloud services had improved the ability to quickly respond to the agency’s mission needs and provided teleworking capabilities. In addition, the officials reported that the solution streamlined the process of provisioning network access between agency components and other external agencies.

Acquiring Cloud Services Helped Agencies Reduce the Cost of IT Services

Officials in the Office of the CIO at the 15 agencies reported that acquiring cloud services had allowed them to procure more cost-effective options for provisioning IT infrastructure and managing IT services. For example, officials in Education’s Office of the CIO reported that, by migrating the Institute of Education Sciences’ data center to the cloud in 2014, the agency had saved approximately $3.3 million in cost avoidances annually for the last 3 years from not having to pay prior data center hosting charges. In addition, Education officials reported that the agency had saved $11.6 million between fiscal years 2013 and 2018 by eliminating contractor website hosting.

In addition, officials in Energy’s Office of the CIO reported that the agency saved $900,000 in fiscal years 2013 to 2014 by transitioning to a cloud- based platform for managing IT services, such as asset management. Acquiring the software and platform as a service reduced or eliminated the costs of administering the agency’s on-premise legacy infrastructure and associated software licensing fees.

Using Cloud Services Increased the Efficiency of Agency Operations and Systems

Officials in the 15 agencies’ Offices of the CIO reported that acquiring cloud services had allowed them to streamline or improve systems and automate business processes and other functions. For example, officials in State’s Office of the CIO reported that the agency had previously relied on paper-based and manual processes for completing employee requests for, among other things, leave, training, personal identification cards, and other general services. By acquiring software and platform as a service, State implemented an electronic application that replaced over 800 paper forms used to make these requests, without the time and cost of developing an application themselves. As a result, the officials reported that they estimate the application has saved more than 50,000 hours of staff time since its deployment by streamlining the request process, automatically populating common data fields, and improving support options.

In addition, officials in Treasury’s Community Development Financial Institutions Fund reported that their office acquired software as a service, which will enable them to reduce the number of legacy systems related to awards management from 17 to 2. These legacy systems had required staff to enter the same data in different systems and manually complete certification work tasks. By automating many of the manual review and compliance processes, the officials reported that the office saved approximately 650 staff hours in 2017.

Cloud Services Helped Agencies Enhance Their Customer Service

Officials in the Office of the CIO at 13 agencies reported that acquiring cloud services had allowed their agencies to improve system design and usability, which helped to enhance their customer service. For example, VA officials reported that they had deployed a website in the cloud, Access to Care, which included detailed data on the wait times and quality-care metrics at local hospitals. Doing so enabled veterans to be able to make better decisions about their health care options. By acquiring cloud services, VA officials reported that they had developed and deployed the new Access to Care website in approximately 30 days, incorporating information from 130 components of VA’s electronic health records system that were previously available on disparate legacy websites into one website. Further, the officials reported that the new website increased the transparency of health care information for the veteran community, empowered veterans, and promoted competition for health care services.

In addition, a Defense official from the Army’s Total Ammunition Management Information System reported that the office had acquired infrastructure as a service in order to improve the processing and reporting of ammunition requests that the Army receives from users worldwide. Defense staff reported that, previously, they had received complaints from customers regarding system pauses and delays when entering requests for ammunition and generating reports due to legacy infrastructure. Defense officials stated that using infrastructure as a service improved system processing and reporting times—from minutes to seconds—by providing scalable technology resources that can meet worldwide performance demands. As a result, customers can more quickly enter their orders into the system.

Acquiring Cloud Services Strengthened Mission Assurance

Officials in the Office of the CIO at nine agencies reported that acquiring cloud services had allowed them to achieve greater levels of mission assurance by streamlining security resources and improving backup capabilities that were not available previously. For example, officials at Defense’s North American Aerospace Defense Command and U.S. Northern Command reported that they had acquired cloud services for the Situational Awareness Geospatial Enterprise system in order to improve mission assurance and address Defense cybersecurity requirements.

According to the officials, cloud services improved mission assurance by allowing them to more quickly correct problems such as malware and the loss of network connectivity in order to ensure the near continuous availability of data from different access points. In addition, the system required extensive storage and backup capabilities due to the need to ensure the system’s data were available continuously from different access points. The officials reported that the acquisition of cloud services has reduced the costs required to maintain continuous backup and storage capabilities. They added that the system also complies with Defense requirements that investments use an approved cloud service provider. In addition, the officials said acquiring cloud services has provided the capability to scale resources, as needed, to meet demands during special events, such as the State of the Union address and the World Series, which require additional security.

Further, Federal Transit Administration officials reported that migrating two systems to the cloud had allowed the agency to enhance system security by managing access to the systems through a single portal rather than managing access to each system individually. As a result, the officials reported that they were able to shift some responsibilities of systems security management to the cloud vendor, which reduced the number of security risks and consolidated the number of security tools used. Further, according to the officials, an additional benefit is that users are required to only remember a single password rather than different passwords required for the multiple systems.

Separately, from information provided by the 15 selected agencies, we identified nine cloud computing investments that illustrate the variety of examples of benefits that had been realized by these agencies from the acquisition of cloud services. Table 7 identifies these investments and additional details regarding the nature and sources of the benefits achieved from them are profiled in appendix VI.

Selected Agencies Identified Key Practices That Enabled Cloud Services

In addition to the examples of significant benefits reported from acquiring cloud services, officials at the 15 agencies reported that six key practices had enabled them to realize these benefits through the successful implementation of cloud services. For example, 12 agencies reported that they implemented new governance planning activities, policies, or processes in order to help ensure that cloud acquisition efforts were managed enterprisewide. In addition, 12 agencies reported that they had reorganized the management of agency IT resources to help increase operational efficiency. Further, six agencies reported that having executive leadership involved in driving the acquisition or sponsoring efforts to use cloud services was critical for the successful adoption of cloud services across the agency. Table 8 lists these key practices and the number of agencies that reported each key practice, ranked by the number of agencies reporting the practice.

In addition, many of these six key practice areas identified by agencies are consistent with requirements outlined in FITARA and recommendations from our prior work made to agencies to address longstanding issues with the management of IT acquisitions and operations. Specifically, we previously have noted the importance of strengthening the authority of CIOs, improving the portfolio review process and the transparency of major investment data, ensuring the use of incremental development methodologies, and updating human capital plans.

Selected Agencies Implemented New Policies and Processes to Guide Governance of Cloud Acquisition

Officials in the Office of the CIO at 12 agencies reported that they had implemented new governance activities or drafted new policies and processes to help ensure the successful implementation of cloud services. For example, SSA officials reported that they had drafted several new policies to simplify the management of cloud resources and provide better oversight for new cloud service acquisitions. Specifically, the officials reported that the new policies established a request-and- approval governance process to address which staff can initiate cloud solutions and what types of projects can receive funding.

In addition, Energy officials reported that they had formalized policies and governance processes on how to perform cloud migrations, including establishing a documented, repeatable process to help offices migrate to cloud services more efficiently. Further, Treasury officials reported that they had focused on strengthening cloud-governance activities, including planning and identifying requirements, because changes to enterprise cloud systems may impact multiple programs. As a result, these officials reported that they had implemented a cross-cutting steering committee to help better plan and assess the impact of changes to enterprise cloud systems that support multiple programs.

Selected Agencies Modified Procurement and Contract Oversight Practices to Strengthen Cloud Acquisition Processes

Officials in the Office of the CIO at 12 agencies reported that they had modified their procurement and contract oversight practices in order to accommodate the differences in how cloud services are acquired from traditional acquisitions. For example, Commerce officials emphasized the importance of developing standardized requirements to ensure that when bureaus award contracts, they use standardized language. The officials stated that these requirements help to ensure that contracts with cloud service providers are comprehensive, legally adequate, and include specific details regarding all of the activities the agency will need the contractor to perform.

In addition, officials of the U.S. Trustee Program at Justice reported that they had used existing project and financial management resources to monitor the use of cloud services and associated spending to help control costs and ensure the accuracy of cloud vendor charges. For example, the officials reported that the program used the cloud vendor’s administrative and business intelligence tools to create reports to verify cloud charges. Also, Labor officials reported that they had worked with the agency’s acquisition team to ensure the agency is only billed for its actual cloud usage. This required the agency to transition from a fixed-price contract model to a time and materials-based contract model, which included a clause that limited the maximum costs the agency would have to pay for cloud services.

Selected Agencies Addressed Changes in Workforce Needs for Managing Cloud Services

Officials in the Office of the CIO at 12 agencies reported that they had taken several steps to address changes in workforce needs for managing cloud services. Specifically, these officials reported that they had conducted inventories of staff skills, transitioned staff into new roles, and ensured that staff acquired training. For example, VA officials reported that they had conducted a staff skills inventory to identify future IT workforce training needs and transition staff from managing legacy technologies to managing cloud services. In addition, Energy officials reported that they were preparing staff to transition from managing data center resources to managing the agency’s service level agreements with cloud providers. The officials reported that moving to cloud services allowed staff to spend more time improving existing applications and identifying other efforts to innovate IT services rather than managing on- premise infrastructure. Further, a Defense official lead for cloud computing in the Navy’s Office of the CIO reported that the Navy had developed an enterprise cloud working group consisting of key members from major offices and security groups to help determine the appropriate training and certification needs for staff and to conduct training seminars. In addition, SBA officials said that the agency took advantage of a contract option offered by the cloud vendor to acquire free cloud classes and training, thereby avoiding the need to spend approximately $380,000 on training.

Selected Agencies Reorganized the Management of IT Resources to Increase Operational Efficiency

Office of the CIO officials at 12 agencies reported that acquiring cloud services had led them to change how they organized and managed the agency’s IT resources. For example, GSA officials reported that they had transitioned from letting individual components within the agency acquire their own application to using an enterprise approach in which software as a service applications are acquired and made available to the entire agency. As a result, the officials reported that this approach allows the agency to further optimize their software purchases and improve their monitoring and tracking of software application usage enterprise-wide.

In addition, officials at the Agricultural Research Service reported that acquiring software as a service had promoted opportunities to share customizations of the acquired software between Agriculture’s components rather than having each component develop a separate customization. Specifically, these officials reported that they were able to take a software feature developed by another Agriculture component and implement it for their customer service portal, rather than having to develop it themselves.

Lastly, Education officials reported that they were in the process of beginning an assessment to consolidate the agency’s existing cloud services across federal and commercial environments. The officials said that they hoped to reduce the number of commercial cloud providers used from twenty-five to eight, and to consolidate two of the agency’s cloud environments into a single environment within the next 3 years.

Selected Agencies Streamlined Cloud Services to Address Security Needs in a More Efficient Manner

Office of the CIO officials at eight agencies reported that they were able to streamline the management of IT security by leveraging cloud services. For example, SBA officials reported that they used security tools from their cloud vendor in order to meet DHS’s requirements for continuous diagnostics and mitigation and improve the agency’s security posture. The officials reported that they had performed a requirements analysis and found that, compared to acquiring costly hardware solutions to manage this capability internally, their existing cloud vendor provided security capabilities that actually exceeded DHS’s recommended continuous diagnostics and mitigation requirements. As a result, SBA adopted the cloud vendor’s security tools and avoided $300,000 in initial hardware purchases, as well as subsequent hardware technology refreshes every 3 years.

In addition, GSA officials reported that choosing a FedRAMP-approved cloud service provider had expedited the agency’s adoption of cloud services. Specifically, the agency did not have to visit and review each vendor’s facility as part of the vendor approval process, which shortened the time frame needed to approve a system for use. The officials also reported that using cloud services streamlined the deployment process for new systems because using a cloud platform that had previously been granted the authority to operate allowed the agency to avoid undertaking a separate authorization process, which saved time and resources.

Selected Agencies Engaged Executive Leadership Support during Cloud Acquisition to Help Ensure Successful Implementation

Officials in the Office of the CIO at six agencies reported that having executive leadership involved in driving acquisitions or sponsoring efforts to use cloud services was critical to the successful adoption of cloud services across the agency. For example, SBA officials reported that their agency CIO’s commitment to acquiring cloud services and the deputy CIO’s attendance at daily cloud meetings were critical for the successful adoption of cloud services at the agency. Similarly, Energy officials reported that the agency had established a team with representatives from offices of the CIO, chief financial officer, and chief acquisition officer, to coordinate IT expenditures, including cloud investments, across the agency.

Further, Defense officials from the U.S. Transportation Command reported that establishing a cloud center of excellence team that reported directly to the Commander of U.S. Transportation Command had empowered the team to engage directly with users to help break down barriers that might impact the migration to cloud services. In addition, the officials said that the team helped streamline the processes—specifically, the design, contracting, funding, transition planning, and implementation processes—necessary for the successful migration of all of the command’s systems to the cloud.

Conclusions

Since 2011, when OMB began requiring agencies to adopt a Cloud First strategy, agencies have made progress in implementing cloud services and, in doing so, have saved hundreds of millions of dollars and realized notable benefits. However, six agencies still lack guidance for assessing IT investments for cloud services and 12 agencies still have not performed assessments for a number of their IT investments. In addition, all of the agencies in our review do not have sufficient mechanisms or approaches to track and report the savings data associated with these cloud initiatives.

Although agencies have reported spending $1 billion or more on cloud services in just the past 2 years, and identified hundreds of millions of dollars in related savings, these figures are not consistently reported. To its credit, beginning in fiscal year 2019, OMB will require more accuracy and granularity in how agencies report cloud investment spending data. However, there has not been a corresponding effort to improve the reporting of cloud savings data. An important aspect to the success of key OMB cloud initiatives, like Cloud Smart and the associated drive for greater agency adoption of cloud services, will be the ability for key stakeholders to access complete information on the savings that agencies are achieving under these efforts.

Recommendations for Executive Action

We are making a total of 35 recommendations—1 recommendation to OMB and 34 recommendations to the 16 agencies in our review.

The Director of the Office of Management and Budget should require agencies to explicitly report, at least on a quarterly basis, the savings and cost avoidance associated with cloud computing investments. (Recommendation 1)

The Secretary of Agriculture should ensure that the CIO of Agriculture completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 2)

The Secretary of Agriculture should ensure that the CIO of Agriculture establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 3)

The Secretary of Commerce should ensure that the CIO of Commerce establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 4)

The Secretary of Defense should ensure that the CIO of Defense completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 5)

The Secretary of Defense should ensure that the CIO of Defense establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 6)

The Secretary of Education should ensure that the CIO of Education establishes guidance on assessing new and existing IT investments for suitability for cloud computing services, in accordance with OMB guidance. (Recommendation 7)

The Secretary of Education should ensure that the CIO of Education completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 8)

The Secretary of Education should ensure that the CIO of Education establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 9)

The Secretary of Energy should ensure that the CIO of Energy updates the agency’s guidance on assessing IT investments for suitability for cloud computing services to include a requirement to assess new acquisitions for these services. (Recommendation 10)

The Secretary of Energy should ensure that the CIO of Energy completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 11)

The Secretary of Energy should ensure that the CIO of Energy establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 12)

The Secretary of Health and Human Services should ensure that the CIO of HHS establishes guidance on assessing new and existing IT investments for suitability for cloud computing services, in accordance with OMB guidance. (Recommendation 13)

The Secretary of Health and Human Services should ensure that the CIO of HHS completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 14)

The Secretary of Health and Human Services should ensure that the CIO of HHS establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 15)

The Secretary of Homeland Security should ensure that the CIO of DHS completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 16)

The Secretary of Homeland Security should ensure that the CIO of DHS establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 17)

The Attorney General of the United States should ensure that the CIO of Justice completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 18)

The Attorney General of the United States should ensure that the CIO of Justice establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 19)

The Secretary of Labor should ensure that the CIO of Labor updates the agency’s guidance on assessing IT investments for suitability for cloud computing services to include a requirement to assess existing investments for these services. (Recommendation 20)

The Secretary of Labor should ensure that the CIO of Labor completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 21)

The Secretary of Labor should ensure that the CIO of Labor establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 22)

The Secretary of State should ensure that the CIO of State establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 23)

The Secretary of the Treasury should ensure that the CIO of Treasury completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 24)

The Secretary of the Treasury should ensure that the CIO of Treasury establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 25)

The Secretary of Transportation should ensure that the CIO of Transportation establishes guidance on assessing new and existing IT investments for suitability for cloud computing services. (Recommendation 26)

The Secretary of Transportation should ensure that the CIO of Transportation completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 27)

The Secretary of Transportation should ensure that the CIO of Transportation establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 28)

The Secretary of Veterans Affairs should ensure that the CIO of VA completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 29)

The Secretary of Veterans Affairs should ensure that the CIO of VA establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 30)

The Administrator of General Services should ensure that the CIO of GSA establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 31)

The Administrator of the Small Business Administration should ensure that the CIO of SBA establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 32)

The Commissioner of the Social Security Administration should ensure that the CIO of SSA updates the agency’s guidance on assessing IT investments for suitability for cloud computing services to include a requirement to assess existing investments for these services. (Recommendation 33)

The Commissioner of the Social Security Administration should ensure that the CIO of SSA completes an assessment of all IT investments for suitability for migration to a cloud computing service, in accordance with OMB guidance. (Recommendation 34)

The Commissioner of the Social Security Administration should ensure that the CIO of SSA establishes a consistent and repeatable mechanism to track savings and cost avoidances from the migration and deployment of cloud services. (Recommendation 35)

Agency Comments and Our Evaluation

We provided a draft of this report to OMB and the 16 agencies for their review and comment. In response, 14 agencies provided comments stating that they agreed with our recommendations; one agency stated that it agreed with one recommendation but disagreed with another; and two agencies did not state whether they agreed or disagreed with the recommendations. In addition, multiple agencies provided technical comments, which we incorporated into the report, as appropriate.

The following 14 agencies agreed with our recommendations: In written comments from Commerce, Education, Energy, HHS, DHS, State, Transportation, VA, and GSA, these agencies stated that they agreed with the recommendations directed to them. In addition, each of the agencies indicated that it planned, or already had begun taking actions, to address the recommendations. The agencies’ comments are reprinted in appendixes VII through XV, respectively.

In emails received from Agriculture’s Director of Strategic Planning, Policy, Egovernment and Audits in the Office of the CIO on February 11, 2019, and from Justice’s Audit Liaison Specialist in the Internal Review and Evaluation Office on February 15, 2019, both of these departments stated that they agreed with our recommendations.

In written comments from Labor, the department stated that it agreed with our recommendations. The department also described actions taken to address our recommendation that it update its guidance on assessing IT investments for suitability for cloud computing services to include a requirement to assess existing investments for these services. Specifically, Labor stated that it had taken steps to ensure that its agencies included an assessment of cloud computing suitability as they moved forward with their investments and that this process had been integrated into Labor’s budget process. We followed up with the department and obtained a copy of Labor’s guidance.

However, in examining this guidance, we found it to be the same as what Labor had previously provided to us during the course of our audit. Further, as we mentioned earlier regarding our analysis of the department’s guidance for assessing investments for suitability for cloud services, Labor had required existing investments that were already using cloud services to migrate to the department’s new consolidated cloud environment; however, it did not require existing systems not using cloud services to be assessed for these services. Without receiving any additional information from the department that supported its actions to address our recommendation prior to this report’s issuance, we believe our recommendation to Labor is still appropriate. The department’s comments are reprinted in appendix XVI.

In written comments from SBA, the agency agreed with our recommendation. Also, in additional comments sent by a GAO liaison in the Office of Congressional and Legislative Affairs via email on March 11, 2019, SBA provided updated information regarding the benefits that the agency had realized from using cloud services for its system that was profiled in appendix VI of the draft report. Specifically, SBA officials in charge of the system provided a revised list of realized benefits from the cloud services. However, the officials did not provide any supporting documentation regarding the revised benefits; therefore, we were not able to validate the revised list of benefits prior to the issuance of this report. As a result, we removed the profile from the report in order to be consistent with our methodology for reporting examples of systems that had realized benefits from the acquisition of cloud services, and notified SBA of this decision. SBA’s comments are reprinted in appendix XVII.

I) Application and Service Migration to DOL Cloud, version 1.5 (March 2018). report. However, during further discussion with SSA officials in charge of the system on January 17, 2019, the officials confirmed that the agency had not yet identified all of the potential benefits related to the use of cloud services as a result of a change in their vendor solution. Thus, we removed the profile from our report in order to be consistent with our methodology for reporting examples of systems that had realized benefits from the acquisition of cloud services, and notified SSA of this decision. SSA’s comments are reprinted in appendix XVIII.

One agency agreed with one recommendation and disagreed with a second recommendation:

Defense provided written comments in which it agreed with our recommendation to complete an assessment of all IT investments for suitability for migration to a cloud computing service. However, the agency did not agree with our recommendation that it establish a mechanism to track savings and cost avoidances from the migration and deployment of cloud services. Specifically, Defense stated that it did not agree with our recommendation because there was no standard, consistent way to capture such savings or cost avoidance. The department stated that it would work with OMB on whether or how to collect such information, and, if practical, report such information in accordance with OMB guidance.

However, as we noted in our report, for the past 6 years, OMB has required agencies to report on savings and cost avoidances from implementing IT reform initiatives, including savings realized from the migration to cloud services. Tracking savings and cost avoidances for cloud initiatives is important in order to ensure that Defense is effectively managing and overseeing its cloud initiatives. In addition, it is essential that OMB and Congress have sufficient data to see the results of Defense’s cloud initiatives and understand whether the department is achieving savings using cloud services. Consequently, we believe our recommendation to track savings and cost avoidances from the migration and deployment of cloud services is still warranted. The department’s comments are reprinted in appendix XIX.

Finally, we received comments via email from Treasury’s Supervisory IT Specialist in the Office of the CIO on February 22, 2019, and OMB’s Liaison to GAO on February 25, 2019. In these comments these two agencies did not state whether they agreed or disagreed with the recommendations that we directed to them.

As agreed with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days from the report date. At that time, we will send copies of this report to the Director of the Office of Management and Budget, the Secretaries and agency heads of the departments and agencies in this report, and other interested parties. This report will also be available at no charge on our website at http://www.gao.gov.

If you or your staffs have any questions on matters discussed in this report, please contact me at (202) 512-4456 or harriscc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made major contributions to this report are listed in appendix XX.

Appendix I: Objectives, Scope, and Methodology

Our objectives for this engagement were to (1) evaluate selected agencies’ progress in implementing cloud services, (2) review the extent to which selected agencies have increased spending on cloud services and achieved cost savings or avoidances, and (3) describe examples of cloud investments with significant or notable benefits that have been identified by selected agencies.

For this review, we selected a sample of agencies based on the size of their total information technology (IT) budget for fiscal year 2017. Specifically, we categorized each of the 24 Chief Financial Officers Act agencies by the size of its IT budget: large (more than $3 billion), medium ($1 billion to $3 billion), and small (less than $1 billion), as reported on the Office of Management and Budget’s (OMB) IT Dashboard These agencies were the Department of Agriculture (Agriculture), Department of Commerce (Commerce), Department of Defense (Defense), Department of Education (Education), Department of Energy (Energy), Department of Health and Human Services (HHS), Department of Homeland Security (DHS), Department of Justice (Justice), Department of Labor (Labor), Department of State (State), Department of Transportation (Transportation), Department of the Treasury (Treasury), Department of Veterans Affairs (VA), General Services Administration (GSA), Small Business Administration (SBA), and Social Security Administration (SSA).

OMB, IT Dashboard, 2017 (https://itdashboard.gov).

To address the first objective, we obtained and analyzed IT Dashboard data related to the 16 selected agencies’ use of cloud services for fiscal years 2016 through 2018, and their projected use in 2019. We chose to begin with fiscal year 2016 because we had previously reported on federal agencies’ use of cloud services through fiscal year 2014 and fiscal year 2015 data was not available. Specifically, the Dashboard includes agency responses to a cloud-related question from OMB’s capital planning guidance. The question asks whether a cloud alternative was evaluated for the investment, or components or systems within the investment. We reviewed agency responses that were submitted for fiscal years 2016 through 2019 as part of the annual budget submission process in order to determine whether a specific investment was using cloud services.

During this 4-year period, OMB made changes to the options that agencies were required to choose from which indicated whether an investment was using cloud services. For OMB’s capital planning guidance for fiscal years 2016 and 2017, we selected responses that indicated that the agency “had evaluated a cloud alternative and chose a cloud alternative” with a particular cloud deployment model. In addition, for OMB’s capital planning guidance for fiscal years 2018 and 2019, we selected responses that indicated the “investment or a portion of the investment is leveraging cloud computing”. We then determined the total number of investments using cloud services and calculated the percentage of investments using these services based on the total number of reported investments by each agency for each fiscal year.

To ensure the accuracy and completeness of the selected agencies’ data on the use of cloud services, we downloaded this data from the IT Dashboard on October 3, 2017, March 7, 2018, and October 9, 2018. We took this step because agencies may update their data on a quarterly basis throughout the fiscal year. In addition, we presented the results of our analysis to officials in charge of cloud services within the Office of the Chief Information Officer (CIO) at each selected agency. We asked these officials to verify the completeness and accuracy of this data and provide any updates as appropriate. Officials at all 16 agencies confirmed the total number of investments using cloud services for fiscal years 2016 through 2018 and their projected use for fiscal year 2019. Based on these steps, we determined that these data were sufficiently reliable to report on agencies’ progress in using cloud services.

In addition, we compared each selected agency’s cloud guidance to OMB’s Cloud First guidance. We interviewed Office of the CIO officials in charge of cloud services at each agency regarding their guidance. In addition, we interviewed OMB staff from the Office of E-Government and Information Technology regarding its guidance. Because of the wide variety of responses and documents we received from the agencies related to their guidance for assessing investments for cloud computing services, we conducted a content analysis of the information in order to determine compliance with OMB’s guidance. In doing so, team members individually reviewed agencies’ responses and documents and assigned them to various categories and subcategories. Team members then compared their categorization schemes, discussed the differences, and reached agreement on the final characterization of compliance with OMB guidance. In cases where agencies provided multiple policies or documents, we followed up to clarify which portions were considered by the agency to support the requirement to assess all investments for cloud services.

In analyzing whether the agencies’ guidance on assessing investments for cloud services met OMB criteria, we assessed whether the guidance clearly identified a requirement for evaluating both new and existing investments for cloud services. Agencies found to not have guidance which clearly defined the assessment process were evaluated as such for one of two reasons: either the agency’s formal guidance did not completely address our assessment criteria or the agency’s guidance had not yet been established or finalized.

In analyzing whether agencies had met OMB’s requirement to evaluate each investment for cloud services, we assessed the number of investments that had completed assessments based on the fiscal year 2019 budget submission. Agencies found not to have met the requirement were evaluated as such if the agency had 10 or more investments that had not yet been evaluated for cloud services. We set this threshold based on a reasonable interpretation of the intent of OMB’s guidance requiring assessments of all investments.

For our second objective, we obtained and analyzed IT Dashboard data related to the 16 agencies’ spending on cloud services for fiscal years 2015 through 2018. We chose to begin with fiscal year 2015 because we had previously reported on federal agencies’ spending on cloud services through fiscal year 2014. Agencies report actual spending costs by fiscal year on the IT Dashboard as part of the next fiscal year reporting. To determine actual cloud spending costs for each fiscal year, we used agency spending data reported each subsequent fiscal year (from fiscal years 2017 through 2018) as of October 5, 2018.

In addition, we administered a data collection instrument to each of the 16 agencies to obtain and analyze spending and savings data by the 16 selected agencies for fiscal years 2014 through 2018 and plans for future planned costs. We requested that these agencies provide spending and savings data broken down by investment, as OMB only requires federal agencies to report total spending by cloud deployment model on the IT Dashboard, and agencies were not required to identify whether any reported savings were cloud-related. This instrument was administered from November 2017 to January 2018.

In the data collection instrument, we asked the selected agencies to complete information on each of their cloud investments, including the title of the application or system leveraging cloud, the cloud deployment and service models, and the associated cloud spending and net cloud savings or avoidances from fiscal year 2014 through fiscal year 2018 and beyond, as agencies generally submitted data on planned spending for one or more fiscal years beyond 2018. Due to the varied scale of cloud implementation efforts ongoing at these agencies, we asked agencies to only provide all applications, systems, or investments leveraging cloud with total life-cycle costs of $1 million or more. We also asked agencies to provide spending and savings or cost avoidances figures in whole numbers in order to avoid errors in rounding numbers when we calculated the reported figures in millions.

We took the following steps to help ensure the reliability of the data we collected. First, to minimize errors that might occur from respondents interpreting our instrument differently from our intended purpose, we reviewed the data collection instrument with agency officials who would be completing the instrument during meetings in October and November 2017. Second, we reviewed the completed spreadsheets to identify missing data or other errors, and consulted with our data quality expert about these issues as appropriate.

All agencies completed the data collection instrument by May 2018. For those agencies that provided rounded (rather than exact) spending or savings figures, we recalculated the data into whole numbers and confirmed our calculations with the agencies. In addition, one agency broke down its savings data into savings and cost avoidances; we combined these reported figures for each investment and, after consultation with a GAO data subject matter expert, confirmed with all the other agencies in our review that their information on savings also included cost avoidances.

We also reviewed the associated notes regarding agencies’ qualifications of the provided data and followed up with agency officials to clarify the responses as appropriate. These notes included information on whether certain spending or savings data were unavailable, whether certain costs were excluded from the spending information provided to us or whether there were other qualifications of the provided data.

Lastly, we presented the results of our analysis of IT Dashboard data and the data obtained from the data collection instrument to each of the selected agencies between June and August 2018. We asked the agencies to verify the completeness and accuracy of these data and provide any updates as appropriate. All 16 agencies provided updated information regarding the list of investments using cloud services with life- cycle costs of $1 million or more and six agencies (Agriculture, Commerce, Justice, Transportation, Treasury, and VA) provided updated information related to spending and savings for these investments, which we have incorporated as appropriate. Based on the measures we took to ensure the reliability of the data provided by the agencies and reported on the IT Dashboard, we determined that the data were sufficiently reliable for the purpose of this report.

For the third objective, we obtained and reviewed available documentation discussing examples of cloud computing investments reported by the selected agencies as having produced notable benefits and key practices that ensure the effort was successful. We also interviewed officials from the Office of the CIO and other components in charge of cloud services regarding these benefits.

In order to develop our list of questions for these meetings, we first conducted research to identify the range of benefits that could be achieved from acquiring cloud services. We reviewed OMB, GSA, and CIO Council cloud guidance; our prior work; and key leading cloud practices from GSA’s Federal Cloud Computing Center of Excellence. Based on this work, we developed a list of seven key areas of benefits: (1) improving efficiency and operations; (2) promoting agility and responsiveness; (3) achieving business growth; (4) reducing cost; (5) meeting regulatory requirements; (6) enhancing customer experience; and (7) ensuring mission outcomes. During meetings with agency officials in the Office of the CIO and other components in charge of cloud services, we asked officials whether they had identified any significant or notable benefits in these seven areas. As these seven areas might not represent all potential benefits, we also asked officials to describe any additional benefits not included in these areas.

In addition, as part of these meetings, we asked officials from the Office of the CIO at each selected agency to identify up to three examples of investments that benefited from the acquisition of cloud services. We asked agencies to exclude examples of email deployments to the cloud to ensure a wider variety of examples of investments with benefits. Fifteen of the 16 agencies in our review identified at least one or more examples of cloud investments that had produced significant or notable benefits, while one agency—HHS—reported that it did not have any such examples because it did not have any completed migration efforts. Because of the open-ended nature of the 15 agencies’ responses to our questions, we conducted a content analysis of the information we received in order to identify and summarize the benefits and key practices that were identified by the 15 agencies. We reviewed the benefits and key practices reported by the agencies and grouped them using the seven key benefit areas that our prior research had identified. We discussed the groupings of the reported benefits, and reached agreement on these categories. We grouped the benefit categories together based on commonalities such as purpose, impact, and capabilities, and summarized the benefits reported. Based on discussion, we confirmed a list of benefits and key practices and totaled the number of agencies that reported each of these.

In addition, to select systems or investments to profile, we reviewed the 34 examples provided by the 15 agencies and narrowed the list to 11 examples. We selected these examples using the following factors: the type of system, whether the system supported the mission or business operations of the agency or component, and the availability of information related to the benefits achieved from acquiring cloud services. In doing so, we sought to have a mix of systems that provided mission critical services to the agency or the public, illustrated a range of cloud computing benefits, and included detailed information on the benefits achieved from using cloud services.

In technical comments received on a draft of this report, two agencies provided new information regarding the use of cloud services for their systems that were profiled in appendix VI of the draft report. Based on the additional information provided by the two agencies, we determined there was no longer sufficient detail regarding what benefits were realized for these systems. Therefore, we removed the two agencies’ profiled examples from the report in order to be consistent with our methodology for reporting examples of systems that had realized benefits from the acquisition of cloud services. We then notified both agencies of this decision.

We conducted this performance audit from September 2017 to April 2019 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Status of Cloud-related Recommendations in the 2017 American Technology Council Report

In May 2017, the Administration established the American Technology Council to help transform and modernize federal IT and how the government uses and delivers digital services. Subsequently, in December 2017, the American Technology Council issued a Report to the President on Federal IT Modernization and made eight cloud computing- related recommendations that are relevant to the focus of our review. Table 9 outlines the cloud-related recommendations contained in the report and the current status of these recommendations as of July 2018, according to Office of Management and Budget (OMB) staff from the Office of E-Government and Information Technology.

Appendix III: Selected Agency Cloud Investments for Fiscal Years 2016 through 2019

The Office of Management and Budget (OMB) requires federal agencies to evaluate each investment, or components or systems within the investment, for cloud services, regardless of the overall life cycle stage of the investment. Agencies are required to report the status of each investment’s evaluation as part of the annual budget submission, as noted in OMB’s annual capital planning guidance. Table 10 lists the total number of investments using cloud services and the total number of all IT investments for fiscal years 2016 through 2019 for 16 selected agencies, as reported on the IT Dashboard as of October 9, 2018.

Appendix IV: Selected Agency Cloud Spending for Fiscal Years 2015 through 2018

The Office of Management and Budget (OMB) requires federal agencies to report total cloud spending based on the cloud deployment model as part of the annual budget submission, as noted in OMB’s annual capital planning guidance for fiscal years 2015 through 2018. Table 11 lists the total agency-reported cloud spending and total IT spending for fiscal years 2015 through 2018 for the 16 agencies in our review, as reported on the IT Dashboard as of October 5, 2018.

Appendix V: Description of Cloud Computing Investments Provided by Selected Agencies for Fiscal Year 2018

Sixteen selected agencies provided us with information on their investments related to spending on cloud services of at least $1 million or more in life-cycle costs. Table 12 identifies the investments for fiscal year 2018 that these agencies submitted to GAO as of October 2018. This list includes the name of the investment, the cloud deployment model, and the cloud service model.

Appendix VI: Profiles of Selected Cloud Computing Acquisitions

The following nine cloud investment profiles illustrate the variety of benefits that the selected agencies in our review had realized from the acquisition of cloud services. These profiles describe the cloud investment, costs, key benefits, and the savings or avoidances associated with implementation of cloud services. In addition, the profiles detail how, among other things, the acquisition of cloud services enabled the agency to overcome previous challenges with legacy systems and acquire more cost-effective, efficient, and responsive IT resources in order to meet mission needs.

Treasury’s Cloud Acquisition Supports Enterprise Resource Planning

Each year, the Department of the Treasury’s (Treasury) Bureau of Engraving and Printing prints billions of dollars—referred to as Federal Reserve notes—for delivery to the Federal Reserve System. According to the Chief of the Bureau of Engraving and Printing’s Office of Enterprise Solutions, in October 2012, the bureau’s CIO deployed an enterprise resource planning system to the cloud to improve efficiency and operations, and enhance the availability, security, and performance of its systems that manage the daily business activities of the bureau.

Previously, the bureau had used 16 separate IT legacy systems that were facing technological obsolescence and required heavy customization using old programming languages. Officials in the bureau reported that they spent time updating and maintaining the hardware and software for these systems to minimize downtime that led to the need for staff to work overtime and decreased customer satisfaction with the system. In addition, the legacy systems were not integrated and certain tasks were performed manually, including data entry, aggregation, and product quality checks. Further, the legacy systems lacked robust data validation features; thus, database administrators had to spend time making corrections to data submitted by users.

By moving to the cloud, the bureau replaced the 16 systems with software as a service that required no customization. As a result, officials in the bureau reported that the bureau was able to significantly improve its operations and decision making. Further, officials stated that the bureau’s use of software as a service enabled it to procure cloud services which helped to ensure that the new system had increased availability, performance, and security to prevent delays and downtime in operations.

The bureau also implemented features to improve data entry to reduce user errors. Figure 3 provides a summary of Treasury’s cloud acquisition.

Acquiring software as a service improved operations and production decisions. According to Bureau of Engraving and Printing officials, by consolidating the bureau’s 16 legacy systems with a single system in the cloud, the bureau made significant improvements to its operations through automation, improved data quality, and increased availability, performance, and security. Specifically, acquiring software as a service enabled the bureau to purchase capabilities that its previous systems did not have. By doing so, this eliminated some manual data entry and improved the bureau’s ability to, among other things, automate production decisions which allowed users to focus on more critical tasks.

For example, bureau staff reported that now they enter data from monthly Federal Reserve Bank orders into the Manufacturing Support Suite to determine the denominations and quantities of currency to produce, along with what banks will receive it. In addition, the new system uses current production times and data to determine when to replenish existing inventory and supplies, such as ink and paper, in order to prevent operational delays and downtime. Furthermore, staff stated that they now make compliance decisions using automated alerts and triggers, which help to prevent the release of products that do not meet bureau standards.

According to bureau officials, the bureau has also improved data quality and reduced the amount of time that database administrators have to spend correcting errors. For example, when users enter data into the system, real-time data validation checks prevent common errors and prompt users to make corrections before submission. In addition, staff stated that the bureau implemented bureau-specific data checks, including which accounts could be associated with item categories and cost centers, which has reduced user errors and improved the reliability of the data.

Finally, the bureau has improved system availability, security, and performance by acquiring software as a service. For example, bureau officials stated that they selected a FedRAMP-approved provider and established service level agreements with the provider to help ensure system availability, security, and performance, including disaster recovery capabilities that were not available for the legacy systems. In addition, bureau staff said that they no longer need to update and maintain IT software and hardware, which has saved time and resources, and decreased system downtime.

Transportation’s Cloud Acquisition Stores Public Transit Data

The Department of Transportation’s (Transportation) Federal Transit Administration provides financial and technical assistance to local public transit systems, including buses, subways, light rail, commuter rail, trolleys and ferries. According to the Federal Transit Administration IT Director, in October 2014, the National Transit Database was migrated to the cloud in order to improve customer experience, mission assurance, agility and responsiveness. Previously, the legacy database had several challenges, including the use of obsolete technology, poor usability, and problems with data accuracy. In addition, developing new functionality for the legacy system was a lengthy process, which decreased the ability of developers to respond to other user needs.

By transitioning to the cloud, the Federal Transit Administration established a centralized access portal for users, which consolidated systems, eliminating the need to remember multiple passwords for external users, and added a single sign on feature for internal users. Staff in the Federal Transit Administration also reported that they improved the database’s user interface by implementing improved system validation functionality for transit data. In addition, the cloud provided software developers with tools to develop functionality quicker to help improve the database’s responsiveness to user needs. Figure 4 provides a summary of Transportation’s cloud acquisition.

Automating validation features improved customer experience. According to Federal Transit Administration officials, by moving to the cloud, developers established automated validation features which use historical data to identify outliers and prevent potential user data entry errors. Officials reported that analysts previously performed manual data validation to ensure the accuracy of customer-entered data. The new cloud version of the National Transit Database uses historical data to identify errors and leverages cross-form data validation for the current reporting year, which has reduced the time it takes to validate the data.

Faster development methods improved the responsiveness to user needs. According to Federal Transit Administration officials, with the deployment to the cloud, the agency adopted faster development processes, which led to more frequent releases of functionality. For example, the database’s developers regularly receive requests from transit customers for enhancements that would traditionally take longer to implement in the prior legacy environment. By leveraging the cloud framework and improved Agile development procedures, officials reported that developers can now engage users earlier to make adjustments based on their feedback, thereby focusing more directly on meeting business needs.

GSA’s Cloud Acquisition Enhances Federal Data Analytics

The General Services Administration (GSA) helps federal agencies build and acquire office space, products and other workspace services, and oversees the preservation of historic federal properties. According to GSA’s Chief Data Officer, in 2015, GSA began developing an enterprise platform pilot program, Data to Decisions, in order to improve the agility, responsiveness, efficiency, and operations of the agency’s data analytics capabilities. Previously, GSA’s data analytics operations had redundancy and overlap, including similar contracts and data sources, negatively affecting data sharing efforts across the agency.

Subsequently, in October 2015, GSA’s CIO, Chief Data Officer, and Chief Technology Officer moved the program to the cloud while consolidating existing contracts to create a centralized web portal. GSA officials reported that the new portal provides new data analytics capabilities for staff to use in generating analyses to advise decision makers at the agency and across other federal agencies, while also saving staff time in producing these analyses. For example, the centralized web portal allows the agency’s 400 data practitioners to, among other things, build data models, understand business operations through analytics and visualizations, and publish dashboards and reporting. Figure 5 provides a summary of GSA’s cloud acquisition.

Providing analytical capabilities and tools to the federal government improved the management of resources. According to GSA officials in the Office of the CIO, cloud deployment has enabled the sharing of data and other analytical tools across the federal government to help agencies better manage their resources and create efficiencies in data management. For example, previously, agencies did not have access to detailed data regarding agency-owned and GSA-managed property in their asset portfolios. By moving to the cloud, GSA officials reported that they developed two tools called the Real Property Management Tool and the Asset Consolidation Tool. These tools were deployed to between 30 and 40 federal agencies, which enabled these agencies to identify potential opportunities to consolidate their building properties or co-locate office spaces to help save resources. Specifically, the tools provided dashboards that showed expiring leases and occupancy agreements, as well as excess and underutilized space. Further, the program provided data to federal agencies that were not previously available. By doing so, officials said that smaller agencies did not have to invest in their own data analytics capabilities or acquire additional staff resources for data analytics.

Flexible and scalable technology addressed an increased demand for data services. According to GSA officials in the Office of the CIO, as the program has expanded its data analytics capabilities, program usage has grown over time. In particular, in 2018, of the program’s 7,200 users, more than 80 percent were from federal agencies other than GSA. Cloud deployment has allowed GSA to easily scale resources to manage changes in user traffic and enabled agency personnel to focus on the mission rather than managing a data center to respond to these changes in demand. For example, in 2017, GSA officials said that they sent out a notice to approximately 1 million federal employees who had completed its annual tenant satisfaction survey notifying them that the survey’s results were available. As a result, several thousand users tried to access the report on the program’s portal, affecting the program’s operations. Officials said that the agency was able to scale up the portal’s resources and capabilities to handle the demand and then scale the resources back once user traffic returned to normal levels.

VA’s Cloud Acquisition Improves Veteran Benefits and Services

The Department of Veterans Affairs (VA), among other duties, administers a variety of benefits and services that provide financial and other forms of assistance to veterans, their dependents, and survivors. According to the Deputy Assistant Secretary for Enterprise Program Management, in March 2016, the VA CIO deployed the Vets.gov web portal to the cloud in order to improve veterans’ customer experience and scale resources to meet demand. Previously, VA officials reported that they had experienced challenges with its legacy websites. Specifically, the websites were not designed using federal government web standards, including browser compatibility and accommodations for the needs of individuals with disabilities. In addition, the websites required users to remember several sets of login information to access many features on approximately 500 websites.

By moving to the cloud, VA officials stated that the agency has been able to better address veterans’ needs by consolidating access to over 500 of the agency’s websites for benefits and services. The new easier, mobile- friendly web portal requires only one login for all 500 websites, and incorporates features for users with disabilities, such as blind veterans.

Further, the program was able to scale up the portal’s resources to meet the increased demand for online benefits and services, while adopting a design approach that better incorporated the needs of veterans and delivered functionality more quickly. In November 2018, the Vets.gov cloud platform became the building block for the agency’s new homepage at VA.gov. Figure 6 provides a summary of VA’s cloud acquisition.

Consolidating website access to benefits and services and incorporating veterans’ feedback improved customer service to veterans and reduced costs. According to VA officials in the Office of the CIO, by moving to the cloud, VA has worked to improve veterans’ access to benefits and services through its websites in several key areas. For example, Vets.gov is intended to be mobile-friendly and work on any computing device with a compliant web browser, avoiding the need to install separate software to apply for benefits. In addition, officials stated that the agency intends the portal to be easier for veterans to search for services. For instance, VA had previously developed an application to help veterans schedule medical appointments but VA officials reported that veterans could not easily locate the application after searching across multiple VA websites.

In addition, VA officials stated that the agency was also able to reduce costs because, in moving to the cloud, Vets.gov cost the agency 85 percent less than it would have cost to build a traditionally hosted service with the same features. VA also retired a legacy application, which saved an estimated $1 million in annual contract costs.

Scalable technology and a faster veteran-centered development approach increased agility and responsiveness. According to VA officials in the Office of the CIO, moving to the cloud allowed VA to acquire more flexible and scalable technologies in order to scale resources up and down to meet demand, while incorporating a faster, more user-friendly design approach. For example, after its launch, officials said that Vets.gov received a spike in the number of veterans that chose to submit online applications for healthcare, which the agency was able to handle by scaling up resources to meet the spike in demand.

In addition, VA officials reported that the agency adopted a design approach in the cloud that, among other things, allowed it to adopt Agile methods to more quickly deliver releases. For example, based on feedback, VA incorporated mobile friendly design features—40 percent of Vets.gov users access benefits and services through a mobile device. Officials said that the agency has made efforts to focus on the needs of veterans first by using an iterative design approach that incorporates user feedback into the design process so that no features in the portal are deployed without a final usability test with a veteran. VA officials also reported that using the cloud has allowed the agency to deploy new features as soon as they are ready, in small incremental daily releases. Further, the officials noted that VA developers have worked with veterans on the portal’s healthcare claims status tracker. Specifically, veterans can access the status of their healthcare claims that may be experiencing a backlog in processing, along with an estimated decision date. Lastly, officials reported that by incorporating an online application, Vets.gov reduced the number of paper-based healthcare applications submitted by veterans. In fiscal year 2018, users submitted over 750,000 digital forms for benefits through Vets.gov.

Justice’s Cloud Acquisition Hosts Data Center

The Department of Justice’s (Justice) U.S. Trustee Program (USTP) is responsible for overseeing the administration of bankruptcy cases and private trustees within the United States. According to USTP’s Chief Technology Officer, in June 2016, executives in the Program, including the CIO, decided to migrate USTP’s operations to the cloud to meet regulatory requirements, reduce costs, and improve agility, efficiency, and responsiveness. Officials said that their office had conducted an evaluation and determined that, in order to fulfill OMB’s mandate to consolidate agency data centers, USTP would have to spend at least $1 million for an on-premise consolidation. Officials reported that USTP also faced challenges with having adequate backup capabilities and implementing new technological solutions due to its legacy computing environment and the time it took to purchase and install new hardware and software. Subsequently, in March 2017, the Program moved its operations to the cloud and avoided the cost of consolidating its data centers. In addition, officials in USTP said that the move to the cloud helped them address backup issues, and speed up the development and testing of new applications. Figure 7 provides a summary of Justice’s cloud acquisition.

Avoiding an on-premise data center consolidation and streamlining IT operations reduced costs. According to USTP officials, by moving to the cloud, their office avoided at least $1 million in costs, while resolving internal performance issues, and streamlining the management of its contracts. Specifically, officials said that USTP shut down 1 of its 2 data centers and reduced its server inventory from 140 to 75 and the number of vendors from 21 to 9. In addition, the office eliminated an estimated 50- 70 monthly IT staff hours dedicated to resolving backup issues.

Flexible technology resources sped up the development of functionality. According to USTP officials, acquisitions of new technology previously took several months because of the time needed to estimate requirements and wait for officials to purchase and install hardware and software. By moving to the cloud, USTP officials stated that the intention is to be able to develop and test new applications faster, and determine their viability, with minimal time and costs. Specifically, officials reported that they have set up a cloud test lab to better understand system requirements by scaling up and down resources as needed and experimenting with new capabilities. In addition, while USTP’s legacy monitoring solution required consulting assistance and took months to implement, officials noted that they were able to set up a similar solution in the cloud within 1 week.

DHS’s Cloud Acquisition Supports Information Sharing and Collaboration

The Department of Homeland Security (DHS) collaborates with a variety of agencies and organizations to share information related to homeland security. According to the program’s Service Operations Manager, DHS’s CIO migrated the Homeland Security Information Network to the cloud in July 2017 in order to improve the system’s availability and operational efficiency, while reducing costs. Officials stated that, previously, the agency had faced challenges in ensuring the system’s redundancy and deploying new network enhancements quickly. This was due to the costs and time frames associated with acquiring new infrastructure and maintaining and upgrading current infrastructure. In addition, the agency was not able to quickly develop and deploy new capabilities to meet user needs.

By moving to the cloud, officials stated that the agency was able to implement a disaster response capability and improve the system’s operational efficiency, while also establishing more efficient environments for software development and testing. In addition, the agency was able to shut down an existing data center, which achieved cost savings of at least 30 percent from hosting the network in the data center. Figure 8 provides a summary of DHS’s cloud acquisition.

Acquiring infrastructure as a service improved system availability and operational efficiency. According to DHS officials in the Office of the CIO, migrating to the cloud has improved the system’s availability and operational efficiency, which cost less money than the prior hosting solution. For example, acquiring infrastructure as a service provided increased redundancy over the old solution and has helped to ensure the network remains continuously available for daily operations and emergency response. The officials stated that, previously, the agency had not been able to implement a disaster recovery capability because it would cost over $1.5 million to build and maintain a second active network environment. Moving to the cloud enabled the agency to implement this capability for significantly less cost.

In addition, officials in the CIO’s office said that the acquisition of infrastructure as a service has enabled the agency to improve the operational efficiency of the system. For example, network managers can easily stand up new virtual hardware, networking, and storage capabilities, or make changes to existing infrastructure, in less than a day. Officials said that, previously, it used to take staff several months to make these changes manually. This allows network managers to respond very rapidly to changes in user demand, particularly if there are emergencies or natural disasters, and then scale down resources during non-use periods. For example, officials said that managers scaled up resources to support first responders from federal, state, and local governments to share weather, response and recovery information during Hurricanes Harvey, Irma, and Jose, and the West Coast wildfires. In addition, the officials noted that network managers now have access to the latest virtual hardware and the agency does not have to pay for hardware refreshments.

Flexible technology resources strengthened the development of functionality. According to DHS officials in the Office of the CIO, moving to the cloud enabled the agency to very inexpensively build multiple environments in the system for software development, testing, and production, which has improved the development and deployment of new services. Software developers now have consistent and standardized environments, which helps to reduce the risk of errors and security vulnerabilities, as well as configuration issues. DHS officials stated that all of these issues would previously require staff time and funding to resolve. The developers can also now use automation tools to deploy new code from development into production more quickly to help meet user needs for new functionality. In addition, officials noted that cloud providers are constantly adding new services that users can leverage to do their work more efficiently, without the time and cost of the agency having to develop or procure this capability separately.

Agriculture’s Cloud Acquisition Improves Enterprise Content and Electronic Records Management

The Department of Agriculture’s (Agriculture) U.S. Forest Service manages 193 million acres of federal land in order to sustain the health, diversity, and productivity of the nation’s forests and grasslands for present and future generations. According to the Acting Assistant Forest Service CIO for Natural Resources and Environment, in August 2017, the Forest Service began deploying a new enterprise content management and electronic records management system, called Pinyon, to the cloud to help improve operations and the management of electronic records. The move also addressed federal requirements related to electronic records management. Officials stated that, previously, the Forest Service relied on a shared storage drive for enterprise content management. Officials reported that this drive was highly proprietary, slow, unreliable, and a security vulnerability because it could not be easily maintained. In addition, officials reported that the shared storage drive was on the verge of failure because the vendor no longer supported and upgraded the system.

By acquiring two software as a service solutions for enterprise content and electronic records management, officials said that the Forest Service was able to quickly deploy a new system with only some limited software customization for the integration of the two solutions. The Forest Service completed this in two phases; officials deployed the enterprise content management solution in August 2017 and the electronic records management solution began deployment in August 2018. Officials reported that they plan to fully deploy the system by December 2018. Figure 9 provides a summary of Agriculture’s cloud acquisition.

Acquisition of software as a service improved operations. According to Forest Service officials in the Office of the CIO, by acquiring software as a service, the Forest Service was able to implement new enterprise content management capabilities and collaboration tools quickly without the costs and risks associated with software development. Officials said that, previously, users did not have capabilities for managing their own content such as setting permissions, granting access privileges to documents, or easily managing different document versions. In addition, officials noted that users relied heavily on email to collaborate on daily work activities as other collaboration tools were not available. By acquiring software as a service, officials said that the Forest Service was able to quickly implement enhanced workflow and document management capabilities and add new tools for collaboration, which has increased staff productivity. Furthermore, acquiring software as a service allowed the Forest Service to integrate their new system with Agriculture’s electronic authentication system, which the agency could not previously accomplish with the legacy system. By integrating these systems, Forest Service officials said that the agency has increased the accessibility of the Forest Service’s information by allowing staff to securely access files regardless of physical location.

Going forward, officials in the Forest Service said that they are exploring other features and capabilities offered by the cloud vendor to help better meet mission needs. For example, the Forest Service regularly collaborates with a variety of other agencies, state and local governments, educational institutions and other organizations on issues related to managing federal lands and responding to natural disasters, such as wildfires. Officials noted that the Forest Service hopes to use shared virtual workspaces and other collaboration tools to engage these partners.

In addition, by acquiring software as a service, Forest Service officials said that they have ensured that there is a system in place that the vendor will automatically upgrade with new enhancements, capabilities, and the latest technology. For example, in order to meet new federal cybersecurity requirements, Forest Service officials said that they have been able to work with the cloud vendor to ensure the vendor incorporates software changes to meet these requirements.

Flexible and scalable technology enhanced the management and storage of electronic records. According to Forest Service officials in the Office of the CIO, by moving to the cloud, the Forest Service was able to acquire new storage capabilities that are easily scalable as its volume of electronic records grows over time. Officials said that, previously, the Forest Service used both paper-based records and a shared storage drive for storing work documents and other operational records. Paper- based records were stored in file cabinets and warehouses while the shared storage drive maintained approximately 320 million files and 250 terabytes of data. In addition, the agency previously used tape backups for the shared storage drive. By moving to the cloud, officials in the Forest Service said that they gained unlimited storage and electronic backup capabilities. Further, Forest Service officials said the new system is intended to be able to easily scale up storage resources as needed for the digitization of its paper-based records and handle the future volumes of electronic records.

In addition, by acquiring software as a service, officials in the CIO’s office reported that the Forest Service was able to meet the federal requirement for electronic records management more than a year before the December 2019 deadline, which the prior shared drive could not meet.

Commerce’s Cloud Acquisition Enhances Access to Weather Data

The Department of Commerce’s (Commerce) National Oceanic and Atmospheric Administration works to understand and predict changes in climate, weather, oceans, and coasts, and shares that knowledge and information with others. According to Commerce’s Acting Chief Information Officer, in September 2017, the National Oceanic and Atmospheric Administration’s CIO and National Weather Service leadership decided to deploy its public weather websites to the cloud in order to improve the agility and responsiveness of these websites in a cost-effective manner. Officials stated that, previously, in 2016, as a result of Hurricane Matthew, hundreds of millions of web requests led to failures with the program’s on-premise infrastructure, causing websites to become unavailable to the public for a period of time. Subsequently, in September 2017, prior to the landfall of Hurricane Irma, officials stated that the agency launched its weather cloud content delivery network. This new network is intended to ensure the availability of weather-related information, while avoiding the additional expenses for infrastructure that would likely go unused during normal business operations. Figure 10 provides a summary of Commerce’s cloud acquisition.

Increased service availability ensured the public’s timely access to extreme weather-related information. According to National Oceanic and Atmospheric Administration officials, the deployment of the weather cloud content delivery network in September 2017 helped websites handle the web requests for data on Hurricanes Irma and Maria by scaling up the resources needed to handle the increased requests. Normally, the weather websites receive approximately 26 million daily web requests from the public. However, officials noted that the number of requests increases dramatically during adverse weather events, such as hurricanes. For example, officials said that in August 2017, the websites began experiencing delays because of the high volume of hurricane- related requests from Hurricane Harvey—including approximately 218 million web requests on August 31, 2017 alone. After deployment to the cloud in September 2017, officials reported that over the course of two days, the weather cloud content delivery network successfully scaled up its resources and handled approximately two billion web requests received through the administration websites.

On-demand capabilities decreased costs. According to National Oceanic and Atmospheric Administration officials, by acquiring software as a service, it avoided the cost of expanding existing on-premise infrastructure to handle sudden surges in demand that only last a short period of time, as well as associated maintenance costs. Officials said that the program can now scale up the resources supporting the weather cloud content delivery network whenever it anticipates an adverse weather event that would lead to greater demand for website information.

Defense’s Cloud Acquisition Enhances Transportation Command Systems

The Department of Defense’s (Defense) U.S. Transportation Command (USTRANSCOM) provides common user and commercial air, land, and sea transportation, as well as terminal management and air refueling, in support of the military’s deployment, employment, sustainment, and re- deployment efforts. USTRANSCOM’s Chief of Cyber Operations and Readiness Division reported that in January 2017, the USTRANSCOM Commander made the decision to migrate all of the command’s systems to the cloud in order to improve mission assurance, agility, responsiveness, efficiency, and operations. Officials reported that, previously, the command had experienced a massive power outage affecting the availability of approximately 25 legacy systems that lacked the capability to quickly recover from network failures. In addition, officials noted that the command’s system, used to manage world-wide moves of Defense personnel property, was not user-friendly, and was difficult to maintain because the agency built the system using waterfall software development methods. Lastly, officials said that the command had largely relied on manual reporting activities that took numerous staff hours to produce to make financial, operational, planning, and support decisions.

By beginning to transition to the cloud in January 2018, USTRANSCOM officials said that the command is in the process of ensuring its systems are secure and continuously available, and is developing capabilities to improve the usability of its legacy systems. In addition, officials reported that the command is streamlining its tracking and reporting mechanisms to allow users to automatically generate key reports, which will give decision makers access to more current and accurate information to help improve program operations. USTRANSCOM officials said that executive sponsorship is absolutely critical for migrating to the cloud to overcome culture change by bringing together people throughout the enterprise. In addition, the command’s cloud center of excellence team facilitates the command’s adoption of cloud by, among other things, training users and addressing governance issues. Figure 11 provides a summary of Defense’s cloud acquisition.

Incorporating automated recovery from network failures and streamlining security increased mission assurance. According to USTRANSCOM officials, by moving the command’s network to the cloud, the command has been able to design and build its new network with higher levels of availability. For example, officials said that if a network segment becomes unavailable, the cloud technology has the capability to automatically reroute traffic to help reduce the amount of delay that users experience. In addition, officials reported that developers have been working to automate several hundred security checks that are part of Defense’s security technical implementation guides by implementing a repeatable, automated process instead of doing manual checks. Officials noted that, previously, manually checking of the status of configurations would take hundreds of man hours to complete. Eventually, the command anticipates that automation will save these hours of manual checks. The command plans to implement the new capability in May 2019.

Replacing a legacy system with a cloud-based system developed using Agile software development methodologies will enhance the shipment of personnel property. According to USTRANSCOM officials, moving to the cloud has assisted the command by replacing the legacy system that manages moves of Defense personnel property, like household goods, with a mobile prototype built in the cloud. Officials reported that the legacy system currently uses a variety of commercial products that are difficult to maintain and do not efficiently address the command’s complex business processes for personnel property moves, all of which affects the usability of the system. Currently, the command is using Agile software development methodologies to reengineer its business processes to develop a solution that is mobile and user-friendly. The new mobile prototype is intended to allow personnel to request access in order to manage the moves of certain household goods. Officials reported that the command initially planned to deploy the prototype in June 2018 but deployment was delayed and a new date had not yet been identified.

Automating reporting and tracking mechanisms will help eliminate manual processes. According to USTRANSCOM officials, the command is in the process of automating its processes for reporting and tracking cargo shipments utilizing cloud technologies. Currently, the command employs manual processes to track and monitor a variety of its cargo shipments. For example, officials reported that five analysts typically spend one day compiling a status report that details delays with food shipments for Defense military exercises and operations. In addition, analysts currently have to query up to 11 Defense and commercial carrier systems to compile a report on high-priority shipments across the combatant commands. Officials noted that these analysts often experience delays getting access to timely information and must also resolve conflicting information in various transportation systems. However, with the transition to cloud services, officials in USTRANSCOM reported that analysts will have the capability to automatically generate reports based on defined criteria, such as shipment method or destination, and use data feeds that officials can continuously update. By developing phase one of the system in the cloud in fiscal year 2018, officials reported that they will be able to monitor delays in a shipment and immediately take action to change the mode of transportation or source shipments from alternate suppliers. The command plans to release the full operational capability in fiscal year 2020, which, officials noted, will give authorized users near real-time access to shipment information, including estimates of whether a shipment will arrive on time.

Appendix VII: Comments from the Department of Commerce

Appendix VIII: Comments from the Department of Education

Appendix IX: Comments from the Department of Energy

Appendix X: Comments from the Health and Human Services

Appendix XI: Comments from the Department of Homeland Security

Appendix XII: Comments from the Department of State

Appendix XIII: Comments from the Department of Transportation

Appendix XIV: Comments from the Department of Veterans Affairs

Appendix XV: Comments from the General Services Administration

Appendix XVI: Comments from the Department of Labor

Appendix XVII: Comments from the Small Business Administration

Appendix XVIII: Comments from the Social Security Administration

Appendix XIX: Comments from the Department of Defense

Appendix XX: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

In addition to the individual named above, the following staff made key contributions to this report: Dave Powner (Director), Dave Hinchman (Assistant Director), Chris Businsky, Nancy Glover, Valerie Hopkins (Analyst-in-Charge), Sandra Kerr, James MacAulay, Jamelyn Payan, and Priscilla Smith.