CLOUD COMPUTING

Agencies Need to 

Incorporate Key 

Practices to Ensure 

Effective Performance

Report to Congressional Requesters

April 2016

GAO-16-325

United States Government Accountability Office
United States Government Accountability Office

Highlights of GAO-16-325, a report to 

congressional requesters

April 2016

CLOUD COMPUTING

Agencies Need to Incorporate Key Practices to 

Ensure Effective Performance

Why GAO Did This Study

Cloud computing is a means for 

delivering computing services via IT

networks. When executed effectively, 

cloud-based services can allow 

agencies to pay for only the IT services 

used, thus paying less for more 

services. An important element of 

acquiring cloud services is a service 

level agreement that specifies, among 

other things, what services a cloud 

provider is to perform and at what 

level. 

GAO was asked to examine federal 

agencies’ use of SLAs. GAO’s 

objectives were to (1) identify key 

practices in cloud computing SLAs and 

(2) determine the extent to which 

federal agencies have incorporated 

such practices into their SLAs. GAO 

analyzed research, studies, and 

guidance developed by federal and 

private entities to develop a list of key

practices to be included in SLAs. GAO 

validated its list with the entities,

including OMB, and analyzed 21 cloud 

service contracts and related 

documentation of five agencies (with 

the largest fiscal year 2015 IT budgets) 

against the key practices to identify 

any variances, their causes, and 

impacts.

What GAO Recommends

GAO recommends that OMB include 

all ten key practices in future guidance 

to agencies and that Defense, Health 

and Human Services, Homeland 

Security, Treasury, and Veterans 

Affairs implement SLA guidance and 

incorporate applicable key practices 

into their SLAs. In commenting on a 

draft of this report, OMB and one 

agency had no comment, the 

remaining four agencies concurred 

with GAO’s recommendations.

What GAO Found

Federal and private sector guidance highlights the importance of federal 

agencies using a service level agreement (SLA) in a contract when acquiring

information technology (IT) services through a cloud computing services

provider. An SLA defines the level of service and performance expected from a

provider, how that performance will be measured, and what enforcement 

mechanisms will be used to ensure the specified performance levels are 

achieved. GAO identified ten key practices to be included in an SLA, such as

identifying the roles and responsibilities of major stakeholders, defining 

performance objectives, and specifying security metrics. The key practices, if 

properly implemented, can help agencies ensure services are performed 

effectively, efficiently, and securely. Under the direction of the Office of 

Management and Budget (OMB), guidance issued to agencies in February 2012

included seven of the ten key practices described in this report that could help 

agencies ensure the effectiveness of their cloud services contracts.

GAO determined that the five agencies and the 21 cloud service contracts it 

reviewed had included a majority of the ten key practices. Specifically, of the 21 

cloud service contracts reviewed from the Departments of Defense, Health and 

Human Services, Homeland Security, Treasury, and Veterans Affairs, 7 had 

fulfilled all 10 of the key practices, as illustrated in the figure. The remaining 13 

contracts had incorporated 5 or more of the 10 key practices and 1 had not 

included any practices.

Figure 1: Number of Cloud Service Contracts That Met All 10 Key Practices

Agency officials gave several reasons for why they did not include all elements of

the key practices into their cloud service contracts, including that guidance 

directing the use of such practices had not been created when the cloud services 

were acquired. Unless agencies fully implement SLA key practices into their 

SLAs, they may not be able to adequately measure the performance of the 

services, and, therefore, may not be able to effectively hold the contractors 

accountable when performance falls short.

View GAO-16-325. For more information, 

contact David A. Powner at (202) 512-9286 or 

pownerd@gao.gov.
Page i GAO-16-325 Cloud Computing

Letter 1

Background 3

Key Practices for Cloud Computing Service Level Agreements 

Can Help Agencies Manage Services More Effectively 12

Selected Agencies Incorporated Most of the Key Practices, but 

Differed in Addressing Them 15

Conclusions 19

Recommendations for Executive Action 20

Agency Comments and Our Evaluation 20

Appendix I Objectives, Scope, and Methodology 22

Appendix II Analysis of Agencies’ Cloud Service SLAs against Key Practices 25

Appendix III Comments from the Department of Defense 32

Appendix IV Comments from the Department of Homeland Security 34

Appendix V Comments from the Department of Health & Human Services 36

Appendix VI Comments from the Department of Veterans Affairs 38

Appendix VII GAO Contact and Staff Acknowledgments 40

Tables

Table 1: Key Practices for a Cloud Computing Service Level 

Agreement 12

Table 2: Analysis of DOD’s Cloud Service SLAs against Key 

Practices 25

Contents
Page ii GAO-16-325 Cloud Computing

Table 3: Analysis of DHS’s Cloud Service SLAs against Key 

Practices 27

Table 4: Analysis of HHS’s Cloud Service SLAs against Key 

Practices 28

Table 5: Analysis of Treasury’s Cloud Service SLAs against Key 

Practices 30

Table 6: Analysis of VA’s Cloud Service SLAs against Key 

Practices 31

Figures

Figure 1: Cloud Service Provider and Consumer Responsibilities 

for the Three Service Models 5

Figure 2: Number of Cloud Service Contracts That Met All 10 Key 

Practices at Selected Agencies 16

Figure 3: Number of 10 Key Practices Met on 21 Cloud Service 

Contracts at Selected Agencies 17
Page iii GAO-16-325 Cloud Computing

Abbreviations

CIO chief information officer

DOD Department of Defense

DHS Department of Homeland Security

FAR Federal Acquisition Regulation

GSA General Services Administration

HHS Department of Health and Human Services

IT information technology

IRS Internal Revenue Service

NIST National Institute of Standards and Technology

OMB Office of Management and Budget

SLA service level agreement

Treasury Department of the Treasury

VA Department of Veterans Affairs

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-16-325 Cloud Computing

441 G St. N.W.

Washington, DC 20548

April 7, 2016

The Honorable Ron Johnson

Chairman

The Honorable Thomas R. Carper

Ranking Member

Committee on Homeland Security and Governmental Affairs

United States Senate

The Honorable Jason Chaffetz

Chairman

Committee on Oversight and Government Reform

House of Representatives

Cloud computing is a process for acquiring and delivering computing 

services via internal or external information technology (IT) networks. 

According to the National Institute of Standards and Technology (NIST), 

cloud computing is “a means for enabling on-demand access to shared 

and scalable pools of computing resources with the goal of minimizing 

management effort or service provider interaction.” More specifically, 

purchasing IT services through a provider enables agencies to avoid 

paying for all the assets (e.g., hardware, software, networks) that would 

typically be needed to provide such services. This approach offers federal 

agencies a means to buy the services faster and possibly cheaper than 

through the traditional methods they have used, such as keeping it all inhouse. To take advantage of these potential benefits, agencies have 

reported that they plan to spend more than $2 billion on cloud computing 

services in fiscal year 2016.1

An important part of acquiring IT cloud computing services is 

incorporating a service level agreement (SLA) into the contract. An SLA 

defines levels of service and performance that the agency expects the 

contractor to meet and the agency uses the information to measure the 

effectiveness of its cloud services. To encourage the use of SLAs, the 

Office of Management and Budget (OMB) directed subject matter experts 

to issue guidance that highlighted SLAs as a key factor to be addressed 

 1

https://www.itdashboard.gov.

Letter
Page 2 GAO-16-325 Cloud Computing

in developing cloud computing contracts.2 You asked us to examine 

federal agencies’ use of SLAs. Specifically, our objectives were to (1) 

identify key practices used in cloud computing service level agreements 

to ensure service is performed at specified levels and (2) determine the 

extent to which federal agencies have incorporated such practices into 

their cloud computing service level agreements.

To identify key practices, we analyzed SLA research, studies, and 

guidance developed and used by federal agencies and private entities 

and performed a comparative analysis of the practices to identify the most 

effective ones. Specifically, we analyzed information from publications 

and related documentation issued by the following ten public and private 

organizations to determine key SLA practices:

• Federal Chief Information Officers Council

• Chief Acquisitions Officers Council

• National Institute of Standards and Technology

• European Commission Directorate General for Communications 

Networks, Content and Technology

• OMB

• Gartner

• MITRE Corporation

• Cloud Standards Customer Council

• International Organization for Standardization

• International Electrotechnical Commission

We then validated our analysis through interviews with experts from these 

organizations. We also had officials from OMB review and validate that 

the practices we identified are those the office expects federal agencies 

to follow.

To determine the extent to which federal agencies have incorporated key 

practices into their cloud computing contracts, we selected five agencies 

to review based, in part, on the size of their largest fiscal year 2015 IT 

budgets and planned spending on cloud computing services. The 

agencies selected were the Departments of

 2

Chief Information Officer’s Council and Chief Acquisition Officers Council, Creating 

Effective Cloud Computing Contracts for the Federal Government, Best Practices for 

Acquiring IT as a Service (Feb. 24, 2012). The guidance was written in coordination with 

the Federal Cloud Compliance Committee.
Page 3 GAO-16-325 Cloud Computing

• Defense,

• Health and Human Services,

• Homeland Security,

• Treasury, and

• Veterans Affairs.

To select and review the cloud services used by the agencies, we 

obtained a list of each agency’s cloud services. We listed the cloud 

services for each agency and selected two for each of three major cloud 

service models (infrastructure, platform, or software).3 In certain cases, 

the agency did not have two cloud services for a service model, so the 

number chosen for that service model was less than two.

For each of the selected cloud services, we compared its contract (if one 

existed), any supporting SLA, and other documentation to our list of key 

practices to determine the extent to which the agency had adhered to the 

key practices or if there were variances.4 We also interviewed agency 

officials to corroborate our analysis and identify the causes and impacts 

of any variances from the practices. (Further details of our scope and 

methodology are in app. I.)

We conducted this performance audit from January 2015 to April 2016 in 

accordance to generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on our audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions 

based on our audit objectives.

The federal government spends more than $80 billion dollars on IT 

annually, with more than $2 billion of that amount spent on acquiring 

cloud-based services. This amount is expected to rise in coming fiscal 

years, according to OMB. A goal of these investments is to improve 

 3

An expanded description of each of these types of service is included in the following 

section.

4

Types of documentation included contract modifications, subscription agreement, terms 

of use, terms of condition, performance work statement, support agreement, service level 

compliance report, application management services, technical proposal, and customer 

agreements.

Background
Page 4 GAO-16-325 Cloud Computing

federal IT systems by replacing aging and duplicative infrastructure and 

systems that are costly and difficult to maintain. Cloud computing helps 

do this by giving agencies the ability to purchase a broad range of IT 

services in a utility-based model that allows an agency to pay for only the 

IT services it uses.5

According to NIST, an application should possess five essential 

characteristics to be considered cloud computing: on-demand selfservice, broad network access, resource pooling, rapid elasticity, and 

measured service.6 Essentially, cloud computing applications are 

network-based and scalable on demand.

According to OMB, cloud computing is economical, flexible, and fast:

• Economical: cloud computing can be a pay-as-you-go approach, in 

which a low initial investment is required to begin and additional 

investment is needed only as system use increases.

• Flexible: IT departments that anticipate fluctuations in user demand 

no longer need to scramble for hardware and software to meet 

increasing need. With cloud computing, capacity can be added or 

subtracted quickly.

• Fast: cloud computing eliminates long procurement and certification 

processes, while providing a wide selection of services.

In addition, according to NIST, cloud computing offers three service 

models:

• Infrastructure as a service—the agency has the capability to provision 

processing, storage, networks, and other fundamental computing 

 5

Chief Information Officer’s Council and the Chief Acquisition Officers Council, Creating 

Effective Cloud Computing Contracts for the Federal Government (Feb. 24, 2012).

6

NIST defines these characteristics further. On-demand self-service allows consumers to 

acquire computing capabilities automatically and as needed. Broad network access 

provides capabilities over a network, which is accessed through standard mechanisms 

(e.g., a mobile phone, tablet, laptop, and workstation). Resource pooling means the 

vendor’s combined computing resources serve multiple consumers. Rapid elasticity refers 

to the ability to vary resources commensurate with demand. Measured services are

incrementally valued, typically on a pay-per-use, or charge-per-use, basis.
Page 5 GAO-16-325 Cloud Computing

resources and run its own software, including operating systems and 

applications. The agency does not manage or control the underlying 

infrastructure but controls and configures operating systems, storage, 

deployed applications, and possibly, selected networking components 

(e.g., host firewalls).

• Platform as a service—the agency deploys its own or acquired 

applications created using programming languages and tools 

supported by the provider. The agency does not manage or control 

the underlying infrastructure, but controls and configures the deployed 

applications.

• Software as a service—the agency uses the service provider’s 

applications, which are accessible from various client devices through 

an interface such as a Web browser (e.g., Web-based e-mail system). 

The agency does not manage or control the underlying infrastructure 

or the individual application capabilities.

As can be seen in figure 1, each service model offers unique functionality, 

with consumer control of the environment decreasing from infrastructure 

to platform to software.

Figure 1: Cloud Service Provider and Consumer Responsibilities for the Three 

Service Models
Page 6 GAO-16-325 Cloud Computing

NIST has also defined four deployment models for providing cloud 

services: private, community, public, and hybrid.

• In a private cloud, the service is set up specifically for one 

organization, although there may be multiple customers within that 

organization and the cloud may exist on or off the customer’s 

premises.

• In a community cloud, the service is shared by organizations with 

similar requirements. The cloud may be managed by the 

organizations or a third party and may exist on or off an organization’s 

premises.

• A public cloud is available to the general public and is owned and 

operated by the service provider.

• A hybrid cloud is a composite of two or more other deployment 

models (private, community, or public) that are bound together by 

standardized or proprietary technology.

According to federal guidance, these deployment models determine the 

number of consumers and the nature of other consumers’ data that may 

be present in a cloud environment. A public cloud should not allow a 

consumer to know or control other consumers of a cloud service 

provider’s environment. However, a private cloud can allow for ultimate 

control in selecting who has access to a cloud environment. Community 

clouds and hybrid clouds allow for a mixed degree of control and 

knowledge of other consumers.

According to OMB, the federal government needs to shift from building 

custom computer systems to adopting cloud technologies and shared 

services, which will improve the government’s operational efficiencies and 

result in substantial cost savings. To help agencies achieve these 

benefits, OMB required agencies in 2010 to immediately shift to a “Cloud 

First” policy and increase their use of available cloud and shared services 

whenever a secure, reliable, and cost-effective cloud service exists.7

 7

OMB, 25 Point Implementation Plan to Reform Federal Information Technology 

Management (Washington, D.C.: Dec. 9, 2010).

OMB Has Undertaken 

Initiatives and Issued 

Guidance to Increase 

Agency Adoption of Cloud 

Computing Services
Page 7 GAO-16-325 Cloud Computing

In February 2011, OMB issued the Federal Cloud Computing Strategy,

8

as called for in its 25-Point Plan. The strategy provided definitions of 

cloud computing services; benefits of cloud services, such as accelerating 

data center consolidations; a decision framework for migrating services to 

a cloud environment;9 case studies to support agencies’ migration to 

cloud computing services; and roles and responsibilities for federal 

agencies. For example, the strategy stated that NIST’s role is to lead and 

collaborate with federal, state, and local government agency chief 

information officers, private sector experts, and international bodies to 

identify standards and guidance and prioritize the adoption of cloud 

computing services. In addition, the strategy stated that agency cloud 

service contracts should include SLAs designed to meet agency 

requirements.

In a December 2011 memo, OMB established the Federal Risk and 

Authorization Management Program (FedRAMP),10 a government-wide 

program intended to provide a standardized approach to security 

assessment, authorization,11 and continuous monitoring for cloud 

computing products and services.12 All federal agencies must meet 

FedRAMP requirements when using cloud services and the cloud service 

providers must implement the FedRAMP security requirements in their 

cloud environment. To become authorized, cloud service providers 

provide a security assessment package to be reviewed by the FedRAMP 

 8

OMB, Federal Cloud Computing Strategy (Washington, D.C.: Feb. 8, 2011).

9

The decision framework, among other things, identifies several key areas for determining 

the readiness for moving to a cloud environment, including the ability of the cloud service 

provider to address government security requirements.

10OMB, Security Authorization of Information Systems in Cloud Computing Environments

(Washington, D.C.: Dec. 8, 2011).

11Security authorization is the official management decision given by a senior official of an 

organization to authorize operation of an information system and to explicitly accept the 

risk to organizational operations and assets, individuals, other organizations, and the 

nation, based on the implementation of an agreed-on set of security controls.

12FedRAMP’s security assessment framework encompasses four process areas 

(document, assess, authorize, and monitor) that are based on the six steps within the 

framework described in NIST’s Guide for Applying the Risk Management Framework to 

Federal Information Systems: A Security Life Cycle Approach, SP 800-37, Revision 1 

(Gaithersburg, MD.: February 2010).
Page 8 GAO-16-325 Cloud Computing

Joint Authorization Board,13 which may grant a provisional authorization. 

Federal agencies can leverage cloud service provider authorization 

packages for review when granting an agency authority to operate, where 

this reuse is intended to save time and money.

Further, at the direction of OMB, the Chief Information Officers Council 

and the Chief Acquisition Officers Council issued, in February 2012, 

guidance to help agencies acquire cloud services.14 In particular, the 

guidance highlights that SLAs are a key factor for ensuring the success of 

cloud based services and that federal agencies should include an SLA 

when creating a cloud computing contract or as a reference. The 

guidance provides important areas of an SLA to be addressed; for 

example, it states that an SLA should define performance with clear terms 

and definitions, demonstrate how performance is being measured, and 

identify what enforcement mechanisms are in place to ensure the 

conditions are being met.

In addition, NIST, in its role designated by OMB in the Federal Cloud 

Computing Strategy, collaborated with private sector organizations to 

release cloud computing guidance,15 which affirms the importance of 

using an SLA when acquiring cloud computing services.

Moreover, a number of other public and private sector organizations have 

issued research on the incorporation of an SLA in a cloud computing 

 13The Joint Authorization Board is composed of the chief information officers from DOD, 

DHS, and the General Services Administration and establishes the baseline controls for 

FedRAMP and criteria for accrediting third-party independent assessment organizations.

14Chief Information Officer’s Council and Chief Acquisition Officers Council, Creating 

Effective Cloud Computing Contracts for the Federal Government, Best Practices for 

Acquiring IT as a Service.

15See, for example, NIST, US Government Cloud Computing Technology Roadmap 

Volume 1: High-Priority Requirements to Further USG Agency Cloud Computing Adoption, 

NIST SP 500-293, (Gaithersburg, MD.: October 2014); and NIST, Cloud Computing 

Synopsis and Recommendations, NIST SP 800-146, (Gaithersburg, MD.: May 2012).
Page 9 GAO-16-325 Cloud Computing

contract.16 According to these studies, an SLA is important because it 

ensures that services are being performed at the levels specified in the 

cloud computing contract, can significantly contribute to avoiding conflict, 

and can facilitate the resolution of an issue before it escalates into a 

dispute. The studies also highlight that a typical SLA describes levels of 

service using various attributes such as availability, serviceability or 

performance, and specifies thresholds and financial penalties associated 

with a failure to comply with these thresholds.

We have previously reported on federal agencies’ efforts to implement 

cloud computing services and on progress oversight that agencies have 

made to help federal agencies in those efforts. These include

• In May 2010, we reported on the efforts of multiple agencies to ensure 

the security of government-wide cloud computing services.17 We 

noted that, while OMB, the General Services Administration (GSA), 

and NIST had initiated efforts to ensure secure cloud computing 

services, OMB had not yet finished a cloud computing strategy; GSA 

had begun a procurement for expanding cloud computing services for 

its website that served as a central location for federal agencies to 

purchase cloud services, but had not yet developed specific plans for 

establishing a shared information security assessment and 

authorization process; and NIST had not yet issued cloud-specific 

security guidance. We recommended that OMB establish milestones 

to complete a strategy for federal cloud computing and ensure it 

addressed information security challenges. These include having a 

process to assess vendor compliance with government information 

security requirements and division of information security 

responsibilities between the customer and vendor.

 16See, for example, European Commission Directorate General for Communications 

Networks, Content and Technology, Cloud Service Level Agreement Standardization 

Guidelines (Brussels, Belgium: June 2014); MITRE, Cloud SLA Considerations for the 

Government Consumer (September 2012); Cloud Standards Customer Council, Practical 

Guide to Cloud Service Agreements, Version 2.0 (April 2015); and Gartner, Best Practices 

for Service Level Agreements for Software as a Service (Stamford, CT: November 2010).

17GAO, Information Security: Federal Guidance Needed to Address Control Issues with 

Implementing Cloud Computing, GAO-10-513 (Washington, D.C.: May 27, 2010).

Agencies Are Taking Steps 

to Implement Prior GAOIdentified Improvements 

for Cloud-based 

Computing Services
Page 10 GAO-16-325 Cloud Computing

OMB agreed with our recommendations and subsequently published 

a strategy in February 2011 that addressed the importance of 

information security when using cloud computing, but it did not fully 

address several key challenges confronting agencies, such as the 

appropriate use of attestation standards for control assessments of 

cloud computing service providers, and division of information 

security-related responsibilities between customer and provider. We 

also recommended that GSA consider security in its procurement for 

cloud services, including consideration of a shared assessment and 

authorization process. GSA generally agreed with our 

recommendations and has since developed the FedRAMP program. 

Finally, we recommended that NIST issue guidance specific to cloud 

computing security. NIST agreed with our recommendations and has 

since issued multiple publications that address such guidance.

• In April 2012, we reported that more needed to be done to implement 

OMB’s 25-Point Plan and measure its results.18 Among other things, 

we reported that, of the 10 key action items that we reviewed, 3 had 

been completed and 7 had been partially completed by December 

2011. In particular, OMB and agencies’ cloud-related efforts only 

partially addressed requirements. Specifically, agencies’ plans were 

missing key practices, such as a discussion of needed resources, a 

migration schedule, and plans for retiring legacy systems. As a result, 

we recommended, among other things, that the Secretaries of 

Homeland Security and Veterans Affairs, and the Attorney General 

direct their respective CIOs to complete practices missing from the 

agencies’ plans for migrating services to a cloud computing 

environment. Officials from each of the agencies generally agreed 

with our recommendations and have taken steps to implement them.

• In July 2012, we reported on the efforts of seven agencies to 

implement three services by June 2012, including the challenges 

associated with doing so.19 Specifically, we reported that selected 

federal agencies had made progress in implementing OMB’s “Cloud 

First” policy. Seven agencies had implemented 21 cloud computing

solutions and had spent a total of $307 million for cloud computing in 

 18GAO, Information Technology Reform: Progress Made; More Needs to Be Done to 

Complete Actions and Measure Results, GAO-12-461 (Washington, D.C.: Apr. 26, 2012).

19GAO, Information Technology Reform: Progress Made but Future Cloud Computing 

Efforts Should Be Better Planned, GAO-12-756 (Washington, D.C.: July 11, 2012).
Page 11 GAO-16-325 Cloud Computing

fiscal year 2012, about 1 percent of their total IT budgets. While each 

of the seven agencies had submitted plans to OMB for implementing 

their cloud services, a majority of the plans were missing required 

elements. Agencies also identified opportunities for future cloud 

service implementations, such as moving storage and help desk 

services to a cloud environment. Agencies also shared seven 

common challenges that they experienced in moving services to cloud 

computing. We made recommendations to the agencies to develop 

planning information, such as estimated costs and legacy IT systems’

retirement plans, for existing and planned services. The agencies 

generally agreed with our recommendations and have taken actions 

to implement them.

• In September 2014, we reported on the aforementioned seven 

agencies’ efforts to implement additional cloud computing services, 

any reported cost savings as a result of implementing those cloud 

services, and challenges associated with the implementation.20 All of 

the seven federal agencies we reviewed had added more cloud 

computing services; the number of cloud services implemented by 

them had increased from 21 to 101 between fiscal years 2012 and 

2014. In addition, agencies had collectively doubled the percentage of 

their IT budgets from 1 to 2 percent during the fiscal year 2012–14 

period. Further, the agencies reported a collective cost savings of 

about $96 million through fiscal year 2013. We made 

recommendations to the agencies to assess their IT investments that 

had yet to be evaluated for suitability for cloud computing services. 

For the most part, the agencies generally agreed with our 

recommendations and have taken actions to implement them.

 20GAO, Cloud Computing: Additional Opportunities and Savings Need to Be Pursued, 

GAO-14-753 (Washington, D.C.: Sept. 25, 2014).
Page 12 GAO-16-325 Cloud Computing

Based on our analysis of practices recommended by the ten 

organizations with expertise in the area of SLAs and OMB, we compiled 

the following list of ten practices that are key for federal agencies to 

incorporate into a contract to help ensure services are performed 

effectively, efficiently, and securely for cloud computing services. The key 

practices are organized by the following management areas—roles and 

responsibilities, performance measures, security, and consequences. 21

Table 1: Key Practices for a Cloud Computing Service Level Agreement

Roles and responsibilities 

1. Specify roles and responsibilities of all parties with respect to the SLA, and, at a minimum, include agency and cloud providers.

2. Define key terms, such as dates and performance.

Performance measures

3. Define clear measures for performance by the contractor. Include which party is responsible for measuring performance. 

Examples of such measures would include

• Level of service (e.g., service availability—duration the service is to be available to the agency).

• Capacity and capability of cloud service (e.g., maximum number of users that can access the cloud at one time and ability of 

provider to expand services to more users).

• Response time (e.g., how quickly cloud service provider systems process a transaction entered by the customer, response time 

for responding to service outages).

4. Specify how and when the agency has access to its own data and networks. This includes how data and networks are to be 

managed and maintained throughout the duration of the SLA and transitioned back to the agency in case of exit/termination of 

service.

5. Specify the following service management requirements:

• How the cloud service provider will monitor performance and report results to the agency.

• When and how the agency, via an audit, is to confirm performance of the cloud service provider.

6. Provide for disaster recovery and continuity of operations planning and testing, including how and when the cloud service 

provider is to report such failures and outages to the agency. In addition, how the provider will remediate such situations and 

mitigate the risks of such problems from recurring.

 

21This report identifies key practices for cloud computing SLAs, including a key practice 

that specifies consequences for missing SLA performance measures. This key practice is 

defined only in terms of consequences and available remedies that are specifically linked 

to the SLA performance measures, and does not refer to enforcement mechanisms (such 

as termination for default) that are made available in the standard terms of federal 

government contracts generally.

Key Practices for 

Cloud Computing 

Service Level 

Agreements Can 

Help Agencies 

Manage Services 

More Effectively
Page 13 GAO-16-325 Cloud Computing

7. Describe any applicable exception criteria when the cloud provider’s performance measures do not apply (e.g., during scheduled 

maintenance or updates).

Security

8. Specify metrics the cloud provider must meet in order to show it is meeting the agency’s security performance requirements for 

protecting data (e.g., clearly define who has access to the data and the protections in place to protect the agency’s data).

9. Specifies performance requirements and attributes defining how and when the cloud service provider is to notify the agency when 

security requirements are not being met (e.g., when there is a data breach).

Consequences

10. Specify a range of enforceable consequences, such as penalties, for non-compliance with SLA performance measures.

Source: GAO analysis of data from public and private organizations. I GAO-16-325

Roles and responsibilities: (1) Define the roles and responsibilities of 

the major stakeholders involved in the performance of the SLA and cloud 

contract. These definitions would include, for example, the persons 

responsible for oversight of the contract, audit, performance 

management, maintenance, and security. (2) Define key terms, including 

activation date, performance, and identify any ambiguities in the 

definitions of cloud computing terms in order to provide the agency with 

the level of service they can expect from their cloud provider. Without 

clearly defined roles, responsibilities, and terms, the agency may not be 

able to appropriately measure the cloud provider’s performance.

Performance measures: (1) Define the performance measures of the 

cloud service, including who is responsible for measuring performance. 

These measures would include, among other things, the availability of the 

cloud service; the number of users that can access the cloud at any given 

time; and the response time for processing a customer transaction.

Providing performance parameters provides both the agency and service 

provider with a well-defined set of instructions to be followed. (2) Specify 

how and when the agency would have access to its data, including how 

data and networks will be managed and maintained throughout the life 

cycle of the service. Provide any data limitations, such as who may or 

may not have access to the data and if there are any geographic 

limitations. (3) Specify management requirements, for example, how the 

cloud service provider would monitor the performance of the cloud, report 

incidents, and how and when they would plan to resolve them. In addition, 

identify how and when the agency would conduct an audit to monitor the 

performance of the service provider, including access to the provider’s 

performance logs and reports. (4) Provide for disaster recovery and 

continuity of operations planning and testing. This includes, among other 

things, performing a risk management assessment; how the cloud service 

would be managed by the provider in the case of a disaster; how data 

would be recovered; and what remedies would apply during a service 
Page 14 GAO-16-325 Cloud Computing

failure. (5) Describe applicable exception criteria for when the cloud

provider’s service performance measures do not apply, such as during 

scheduled cloud maintenance or when updates occur. Without any type 

of performance measures in place, agencies would not be able to 

determine whether the cloud services under contract are meeting 

expectations.

Security: (1) Specify the security performance requirements that the 

service provider is to meet. This would include describing security 

performance metrics for protecting data, such as data reliability, data 

preservation, and data privacy. Cleary define the access rights of the 

cloud service provider and the agency as well as their respective 

responsibilities for securing the data, applications, and processes to meet 

all federal requirements. (2) Describe what would constitute a breach of 

security and how and when the service provider is to notify the agency 

when the requirements are not being met. Without these safeguards, 

computer systems and networks as well as the critical operations and key 

infrastructures they support may be lost, and information—including 

sensitive personal information—may be compromised, and the agency’s 

operations could be disrupted.

Consequences: Specify a range of enforceable consequences, including 

the terms under which a range of penalties and remedies would apply for 

non-compliance with the SLA performance measures. Identify how such 

enforcement mechanisms would be imposed or exercised by the agency. 

Without penalties and remedies, the agency may lack leverage to enforce 

compliance with contract terms when situations arise.

Guidance issued in February 2012, at the direction of OMB highlighted 

SLAs as being a key factor for ensuring the success of cloud-based 

services and advised that federal agencies should include an SLA or a 

reference within the contract when creating a cloud computing contract. 

The guidance provides areas of an SLA to be addressed; for example, it 

states that an SLA should define performance with clear terms and 

definitions, demonstrate how performance is being measured, and 

identify what enforcement mechanisms are in place to ensure the 

conditions are being met.

However, the guidance addressed only seven of the ten key practices 

listed in table 1 that could help agencies better track performance and 

thus ensure the effectiveness of their cloud services. Specifically, the 

guidance did not specify how and when the agency would have access to 

OMB Guidance Addresses 

Seven of the Ten Key 

Practices
Page 15 GAO-16-325 Cloud Computing

its data, provide for disaster recovery and continuity of operations 

planning, and describe any exception criteria.

OMB staff members said that, although the guidance drafted by the Chief 

Information Officers Council and the Chief Acquisition Officers Council 

was a good start, including all ten key practices should be considered. 

Without complete guidance from OMB, there is limited assurance that 

agencies will apply all the key SLA practices into their cloud computing 

contracts, and therefore may be unable to hold contractors accountable 

when performance falls short of their goals.

Many of the 21 cloud service contracts we reviewed at the five selected 

agencies incorporated a majority of the key practices, but the number of 

practices differed among contracts. Specifically, seven of the cloud 

service contracts reviewed met all 10 of the key practices. This included 

three from DHS, three from Treasury, and one from VA. The following 

figure shows the total cloud service contracts reviewed and the number 

that met the 10 key practices at the five selected agencies.

Selected Agencies 

Incorporated Most of 

the Key Practices, but 

Differed in Addressing 

Them
Page 16 GAO-16-325 Cloud Computing

Figure 2: Number of Cloud Service Contracts That Met All 10 Key Practices at 

Selected Agencies

Of the remaining 14 cloud service contracts, 13 incorporated five or more 

of the key practices, and 1 did not meet any of the key practices. Figure 3 

shows each of the cloud service contracts we reviewed and the extent to 

which the agency had included key practices in its SLA contracts. 

Appendix II includes our analysis of all the cloud services we reviewed, by 

agency.
Page 17 GAO-16-325 Cloud Computing

Figure 3: Number of 10 Key Practices Met on 21 Cloud Service Contracts at Selected Agencies

A primary reason that the agencies did not include all of the practices was 

that they lacked guidance that addresses these SLA practices. Of the five 

agencies, only DOD had developed cloud service contracting guidance 

that addressed some of the practices. More specifically, DOD’s guidance 

only addressed three of the key practices: disaster recovery and 

continuity of operations planning, metrics on security performance 

requirements, and notifying the agency when there is a security breach. 

In addition, the guidance partially addressed the practice on access to 

agency data, specifically, with regard to transitioning data back to the 

agency in case of exit/termination of service.

Agency officials responsible for the cloud services that did not meet or 

only partially met key practices provided the following additional reasons 

for not including all ten practices:

• Officials from DOD’s Office of the Chief Information Officer told us that 

the reason key practices were not always fully addressed is that, 

when the contracts and associated SLAs were developed, they did 

not have the aforementioned DOD guidance on cloud service 

acquisition and use—namely, the agency’s memorandum on 
Page 18 GAO-16-325 Cloud Computing

acquiring cloud services that was released in December 2014,

22 and 

the current Defense Federal Acquisition Regulation Supplement, 

which was finalized in August 2015.23 However, as previously stated, 

this updated guidance addressed three of the ten key practices, and 

part of one other.

• Officials from DHS’s Office of the Chief Information Officer stated that 

the Infrastructure as a Service cloud service addressed the partially 

met and not met key practices but did not provide supporting 

documentation to show that the practices were in place. If key 

practices have not been incorporated, the system may have 

decreased performance and the cloud service may not meet its 

intended goals.

• HHS officials from the National Institutes of Health attributed unmet or 

partially met practices for four cloud services—Remedy Force, 

Medidata, the BioMedical Imaging and BioEngineering website, and 

the Drug Abuse public website—to the fact that they evaluate the 

cloud vendor’s ability to meet defined agency needs, rather than 

negotiate with vendors on SLA requirements. While this may explain 

their shortfalls in not addressing all SLA key practices, the agency 

may be placing their systems at risk of not conducting adequate 

service level measurements, which may result in decreased service 

levels.

HHS officials from the Administration of Children and Families stated 

that the reason key practices were partially addressed or not 

addressed for the Grant Solutions cloud service was that these 

practices were being managed by HHS personnel using other tools 

and plans, rather than via the SLA established for this service. For 

example, according to the officials, they are using a management 

information system to monitor performance of the cloud provider. In 

addition, with respect to disaster management, the officials said that 

they have their own disaster recovery plan. Nonetheless, leading 

studies show that these practices should still be incorporated as part 

of the cloud service contract to ensure agencies have the proper 

control over their cloud services.

 22DOD Memorandum, Updated Guidance on the Acquisition and Use of Commercial 

Cloud Computing Services (December 15, 2014).

2380 Fed. Reg. 51739 (August 26, 2015), adding 48 C.F.R. Subpart 239.76.
Page 19 GAO-16-325 Cloud Computing

• Treasury officials said the reason, among other things, the SLAs for 

Treasury Web Services and IRS Portal Environment only partially met 

certain key practices was because the practices were being provided 

by support contractors hired by the cloud service provider, and were 

not directly subject to the SLAs established between Treasury and the 

cloud service provider. Nonetheless, while having contractors perform 

practices is an acceptable approach, Treasury officials were unable to 

provide supporting documentation to show that support contractors 

were assisting with the practices in question.

• Officials from VA’s Office of Information and Technology said the 

reason the key practice associated with penalties and remedies was 

not included in the Terremark SLA was because penalties were 

addressed within other parts of the contract; however, officials were 

not able to provide documentation identifying such penalties. With 

regard to an SLA for eKidney, officials told us they had not addressed 

any of the key practices due to the fact that an SLA was not 

developed between the agency and cloud service provider. Without 

including an SLA in cloud service contracts, the agency runs the risk 

of not having the mechanisms in place to effectively evaluate or 

control contractor performance.

Until these agencies develop SLA guidance and incorporate all key 

practices into their cloud computing contracts, they may be limited in their 

ability to measure the performance of the services, and, therefore, may 

not receive the services they require.

Although OMB has provided agencies guidance to better manage 

contracts for cloud computing services, this guidance does not include all 

the key practices that we identified as necessary for effective SLAs. 

Similarly, Defense, Homeland Security, Health and Human Services, 

Treasury, and Veterans Affairs have incorporated many of the key 

practices in the cloud service contracts they have entered into. Overall, 

this is a good start towards ensuring that agencies have mechanisms in 

place to manage the contracts governing their cloud services. However, 

given the importance of SLAs to the management of these million-dollar 

service contracts, agencies can better protect their interests by 

incorporating the pertinent key practices into their contracts in order to 

ensure the delivery and effective implementation of services they contract 

for. In addition, agencies can improve management and control over their 

cloud service providers by implementing all recommended and applicable 

SLA key practices.

Conclusions
Page 20 GAO-16-325 Cloud Computing

• To ensure that agencies are provided with more complete guidance 

for contracts for cloud computing services, we recommend that the 

Director of OMB include all ten key practices in future guidance to 

agencies.

• To help ensure continued progress in the implementation of effective 

cloud computing SLAs, we recommend that the Secretary of Defense 

direct the appropriate officials to ensure key practices are fully 

incorporated for cloud services as the contracts and associated SLAs 

expire. These efforts should include updating the DOD memorandum 

on acquiring cloud services and current Defense Acquisition 

Regulations System to more completely include the key practices.

• To help ensure continued progress in the implementation of effective 

cloud computing SLAs, we recommend that the Secretaries of Health 

and Human Services, Homeland Security, Treasury, and Veterans 

Affairs direct appropriate officials to develop SLA guidance and 

ensure key practices are fully incorporated as the contract and 

associated SLAs expire.

In commenting on a draft of this report, four of the agencies—DOD, DHS, 

HHS, and VA—agreed with our recommendations; and OMB and one

agency (Treasury) had no comments. The specific comments from each 

agency are as follows: 

• In an e-mail received on March 25, 2016, OMB staff from the Office of 

E-Government and Information Technology stated that the agency

had no comments at this time. 

• In written comments, the Department of Defense concurred with our 

recommendation and described actions it plans to take to address the 

recommendation. Specifically, DOD stated that it will update its cloud 

computing guidance and contracting guidance as appropriate. The 

Department of Defense’s comments are reprinted in appendix III. 

• In written comments, the Department of Homeland Security concurred 

with our recommendation and described actions it plans to take to 

address the recommendation. Specifically, the department will 

establish common cloud computing service level agreement guidance. 

DHS also provided technical comments, which we have incorporated 

in the report as appropriate. The Department of Homeland Security’s 

comments are provided in appendix IV.

Recommendations for 

Executive Action

Agency Comments

and Our Evaluation
Page 21 GAO-16-325 Cloud Computing

• In written comments, the Department of Health and Human Services 

concurred with our recommendation, but noted that it was not directed

by a federal mandate. We acknowledge that our recommendation is

not directed by a mandate; however, implementing leading practices

for cloud computing can result in significant benefits. The department

also provided technical comments, which we have incorporated in the

report as appropriate. The Department of Health and Human

Service’s comments are provided in appendix V.

• In an e-mail received on March 18, 2016, an audit liaison from the

Department of the Treasury’s Office of the CIO stated that the

department had no comment.

• In written comments, the Department of Veterans Affairs concurred

with our recommendation and described planned actions to address it.

For example, the department will develop service level agreement

guidance to include the 10 key practices. The Department of Veterans

Affairs comments are provided in appendix VI.

We are sending copies of this report to interested congressional 

committees; the Secretaries of Defense, Health and Human Services, 

Homeland Security, the Treasury, and Veterans Affairs; and the Director 

of the Office of Management and Budget, and other interested parties. 

This report will also be available at no charge on our website at 

http://www.gao.gov. 

If you or your staffs have any questions on matters discussed in this 

report, please contact me at (202) 512-9286 or pownerd@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this report. GAO staff who made 

major contributions to this report are listed in appendix VII.

David A. Powner

Director

Information Technology Management Issues
Appendix I: Objectives, Scope, and 

Methodology

Page 22 GAO-16-325 Cloud Computing

Our objectives were to (1) identify key practices used in cloud computing 

service level agreements (SLA) to ensure service is performed at 

specified levels and (2) determine the extent to which federal agencies 

have incorporated such practices into their cloud computing service level 

agreements.

To identify key practices used in cloud computing service level 

agreements, we analyzed SLA research, studies, and guidance 

developed and used by federal agencies and private entities. 

We then performed a comparative analysis of the practices to identify the 

practices that were recommended by at least two sources. Specifically, 

we analyzed information from publications and related documentation 

issued by the following ten public and private organizations to determine 

key SLA practices:

• Federal Chief Information Officer Council

• Chief Acquisitions Officers Council

• National Institute of Standards and Technology

• European Commission Directorate General for Communications 

Networks, Content and Technology

• Office of Management and Budget

• Gartner

• MITRE Corporation

• Cloud Standards Customer Council

• International Organization for Standardization

• International Electrotechnical Commission

Next, we organized these practices into management areas and validated 

our analysis through interviews with experts from these organizations. We 

also had officials from the Office of Management and Budget (OMB) 

review and validate that these practices are the ones the office expects 

federal agencies to follow. In cases where experts disagreed, we 

analyzed their responses, including the reasons they disagreed, and 

made changes as appropriate. These actions resulted in our list of key 

practices for cloud service SLAs.

To determine the extent to which federal agencies have incorporated key 

practices into their cloud computing contracts, we selected five agencies 

to review based, in part, on those with the largest fiscal year 2015 IT 

budgets and planned spending on cloud computing services. The 

agencies selected were the Departments of Defense (DOD), Homeland 

Security (DHS), Health and Human Services (HHS), Treasury, and 

Veterans Affairs (VA).

Appendix I: Objectives, Scope, and 

Methodology
Appendix I: Objectives, Scope, and 

Methodology

Page 23 GAO-16-325 Cloud Computing

We selected these agencies based on the following two factors. First, 

they have the largest planned IT budgets for fiscal year 2015. Their 

budgets, which collectively totaled $57 billion, represent about 72 percent 

of the total federal IT budget ($78 billion). Second, these agencies plan to 

spend relatively large amounts on cloud computing. Specifically, based 

on our analysis of OMB’s fiscal year 2015 budget data, each of the five 

departments were in the top 10 for the largest amount budgeted for cloud 

computing and collectively planned to spend $1.2 billion on cloud 

computing, which represents about 57 percent of the total amount that 

federal agencies plan to invest in cloud computing ($2.1 billion).

To select and review the cloud services used by the agencies, we 

obtained an inventory of cloud services for each of the five agencies, and 

then, for each agency, we listed their cloud services in a random fashion 

and selected the first two cloud services in the list for each of the three 

major cloud service models (infrastructure, platform, and software). In 

certain cases, the agency did not have two cloud services for a service 

model, so the number chosen for that service model was less than two. 

This resulted in a non-generalizable sample of 23 cloud services. 

However, near the end of our engagement, agencies identified 2 of the 

services as being in a pilot stage (one from DHS, and one from HHS), 

and thus not operational. We excluded these services from our analysis, 

as our methodology to only assess operational cloud services. Due to the 

stage of the engagement, we were unable to select additional services for 

review. Further, because no computer-generated data was used we 

determined that there were no data reliability issues.

For each of the selected services, we compared its cloud service contract 

(if one existed) and any associated SLA documentation to our list of key 

practices to determine if there were variances and, if so, their cause and 

impact. To do so, two team analysts independently reviewed the cloud 

service contracts against the key practices using the following criteria:

• Met: all aspects of the key practices were fully addressed.

• Partially met: some key practices were addressed.

• Did not meet: no key practices were addressed.

In cases where analysts differed on the assessments, we discussed what 

the rating should be until we reached a consensus. We also interviewed 

agency officials to corroborate our analysis and identify the causes and 

impacts of any variances.
Appendix I: Objectives, Scope, and 

Methodology

Page 24 GAO-16-325 Cloud Computing

We conducted this performance audit from January 2015 to April 2016 in 

accordance to generally accepted government auditing standards. Those 

standards require that we plan and perform the audit to obtain sufficient, 

appropriate evidence to provide a reasonable basis for our findings and 

conclusions based on our audit objectives. We believe that the evidence 

obtained provides a reasonable basis for our findings and conclusions 

based on our audit objectives.
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 25 GAO-16-325 Cloud Computing

The following tables show each of the five agencies’—DOD, DHS, HHS, 

Treasury, and VA—cloud services we assessed and our analysis of each 

contract for cloud services against the key practices. In cases where the 

SLA partially met a practice, the analysis also includes discussion of the 

rationale for why that assessment was provided.

(Note: M – met, P – partially met, NM – did not meet.)

Table 2: Analysis of DOD’s Cloud Service SLAs against Key Practices

DOD cloud services

Air Force,

Integrated Risk

Information System

Air Force,

Case Tracking

Army, Email

as a Service

Navy, Navy

Web Portal

Key practices M P NM M P NM M P NM M P NM

Stakeholder roles and 

responsibilities X X X X

Terms and dates X X X X

Measurable 

performance objectives X X X X

Access to agency data X X X X

Service management

requirements X X X X

Disaster recovery 

planning X X X X

Exception criteria X X X X

Security performance 

requirements X X X X

Notification of security 

breach X X X X

Consequences X X X X

Total 5 1 4 9 1 7 2 1 5 2 3

Source: GAO analysis of DOD data. I GAO-16-325

With regard to those services that partially met key practices:

Air Force

• The Integrated Risk Information System partially addressed one key 

practice on how and when the agency was to have access to its data 

and networks. It included how the data would be transitioned, but did 

not specify how access to data and networks was to be managed or 

maintained.

Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 26 GAO-16-325 Cloud Computing

• The Case Tracking cloud service partially included the practice on 

specifying metrics for security performance requirements. It specified 

how security needs were to be met but did not give specific metrics for 

doing so.

Army

• Email as a Service partially addressed two key practices. For the 

practice on specifying service management requirements, it specified 

how the cloud service provider was to monitor performance, but did 

not address how the provider was to report performance or how the 

agency was to confirm the performance. For the other practice on 

specifying metrics for security performance requirements, it included 

how security needs were to be met but did not specify the security 

metrics.

Navy

• The Web Portal partially incorporated two key practices. For the 

practice on how and when the agency was to have access to its data 

and networks, it specified how the data was to be transitioned, but not 

how access to data and networks was to be managed or maintained. 

For the other practice on specifying metrics for security performance 

requirements, it included monitoring of the contractor regarding 

security, but did not specify security metrics.
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 27 GAO-16-325 Cloud Computing

Table 3: Analysis of DHS’s Cloud Service SLAs against Key Practices

DHS Cloud Services

Business Intelligence 

as a Service E-mail as a Service

Infrastructure

as a Service

Web Content

Management

as a Service

Key practice M P NM M P NM M P NM M P NM

Stakeholder roles and 

responsibilities X X X X

Terms and dates X X X X

Measurable 

performance objectives X X X X

Access to agency data X X X X

Service management 

requirements X X X X

Disaster recovery 

planning X X X X

Exception criteria X X X X

Security performance 

requirements X X X X

Notification of security 

breach X X X X

Consequences X X X X

Total 10 10 7 2 1 10

Source: GAO analysis of DHS data. I GAO-16-325

Infrastructure as a Service partially incorporated two key practices. For 

the practice on how and when the agency was to have access to its data

and networks, it specified how and when the agency was to have access 

to its data and networks, but did not provide how data and networks was 

to be transitioned back to the agency in case of an exit. For the other 

practice on service management requirements, it described how the cloud 

service is to monitor performance, but did not specify how and when the 

agency was to confirm audits of the service provider’s performance.
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 28 GAO-16-325 Cloud Computing

Table 4: Analysis of HHS’s Cloud Service SLAs against Key Practices

HHS Cloud Service Remedy Force Medidata Rave

National Institute of 

Bio-Medical Imaging 

and Bio-Engineering 

Public Website

National Institute 

on Drug Abuse 

Pubic Website Grant Solutions

Key practice M P NM M P NM M P NM M P NM M P NM

Stakeholder roles 

and responsibilities X X X X X

Terms and dates X X X X X

Measurable 

performance 

objectives

X X X X X

Access to agency 

data X X X X X

Service 

management 

requirements

X X X X X

Disaster recovery 

planning X X X X X

Exception criteria X X X X X

Security 

performance 

requirements

X X X X X

Notification of 

security breach X X X X X

Consequences X X X X X

Total 8 1 1 8 2 9 1 8 2 6 1 3

Source: GAO analysis of HHS data. I GAO-16-325

With regard to those services that partially met key practices,

• National Institute of Health’s Remedy Force partially addressed one 

key practice on defining measurable performance objectives. It 

included various performance objectives, such as levels of service 

and availability of the cloud service, capacity and capability, and 

measures for response time, but it did not include which party was to 

be responsible for measuring performance.

• The National Institute of Health’s Medidata Rave partially incorporated 

two key practices. It defined measurable performance objectives, 

specifically it specified levels of service, capacity and capability of the 

service, and response time, but did not specify the period of time that 

it was to be measured. For the other practice on specifying a range of 
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 29 GAO-16-325 Cloud Computing

enforceable consequences, it specified remedies, but did not identify 

any penalties related to non-compliance with performance measures.

• The National Institute on Drug Abuse Public Website partially 

addressed two key practices. For the practice on specifying how and 

when the agency is to have access to its data and networks, it 

specified how and when the agency was to have access to its data 

and networks, but did not identify how data and networks were to be 

managed throughout duration of the SLA. For the other practice on 

specifying a range of enforceable consequences, it included a number 

of remedies, but did not specify a range of enforceable penalties.

• HHS’s Grant Solutions partially incorporated one key practice on 

specifying service management requirements. It provided for when 

and how the agency was to confirm cloud provider performance, but 

did not specify how the cloud service provider was to monitor 

performance and report results.
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 30 GAO-16-325 Cloud Computing

Table 5: Analysis of Treasury’s Cloud Service SLAs against Key Practices

Treasury

Cloud Services

Enterprise Asset 

Management 

System

Franchise Financial 

and Administrative 

Services

IRS Portal 

Environment

Manufacturing 

Support Suite

Treasury Web 

Solutions

Key Practices M P NM M P NM M P NM M P NM M P NM

Stakeholder roles 

and responsibilities X X X X X

Terms and dates X X X X X

Measurable 

performance 

objectives

X X X X X

Access to agency 

data X X X X X

Service 

management 

requirements

X X X X X

Disaster recovery 

planning X X X X X

Exception criteria X X X X X

Security 

performance 

requirements

X X X X X

Notification of 

security breach X X X X X

Consequences X X X X X

Total 10 10 9 1 10 8 2

Source: GAO analysis of Treasury data. I GAO-16-325

With regard to those services that partially met key practices,

• Treasury’s Internal Revenue Service’s Portal Environment partially 

included one key practice on specifying how and when the agency 

was to have access to its data and networks. It specified how and 

when the agency was to have access to its data and networks, but it 

did not provide on how data and networks were to be transitioned 

back to the agency in case of an exit.

• The Treasury’s Web Solutions partially addressed two key practices. 

For the practice on specifying how and when the agency was to have 

access to its data and networks, it specified how and when the 

agency was to have access to its data and networks, but it did not 

provide how data and networks would be transitioned back to the 

agency in case of an exit. For the other practice on specifying a range 
Appendix II: Analysis of Agencies’ Cloud 

Service SLAs against Key Practices

Page 31 GAO-16-325 Cloud Computing

of enforceable consequences, it did not provide detailed information 

on a range of enforceable penalties and remedies for non-compliance 

with SLA performance measures.

Table 6: Analysis of VA’s Cloud Service SLAs against Key Practices

VA Cloud Services

Customer Relationship 

Management eKidney Terremark

Key Practices M P NM M P NM M P NM

Stakeholder roles and 

responsibilities X X X

Terms and dates X X X

Measurable performance 

objectives X X X

Access to agency data X X X

Service management 

requirements X X X

Disaster recovery planning X X X

Exception criteria X X X

Security performance 

requirements X X X

Notification of security breach X X X

Consequences X X X

Total 10 10 9 1

Source: GAO analysis of VA data. I GAO-16-325
Appendix III: Comments from the Department 

of Defense

Page 32 GAO-16-325 Cloud Computing

Appendix III: Comments from the 

Department of Defense
Appendix III: Comments from the Department

of Defense

Page 33 GAO-16-325 Cloud Computing
Appendix IV: Comments from the Department 

of Homeland Security

Page 34 GAO-16-325 Cloud Computing

Appendix IV: Comments from the 

Department of Homeland Security
Appendix IV: Comments from the Department 

of Homeland Security

Page 35 GAO-16-325 Cloud Computing
Appendix V: Comments from the Department of Health & 

Human Services

Page 36 GAO-16-325 Cloud Computing

Appendix V: Comments from the Department 

of Health & Human Services
Appendix V: Comments from the Department of Health & 

Human Services

Page 37 GAO-16-325 Cloud Computing
Appendix VI: Comments from the Department 

of Veterans Affairs

Page 38 GAO-16-325 Cloud Computing

Appendix VI: Comments from the 

Department of Veterans Affairs
Appendix VI: Comments from the Department 

of Veterans Affairs

Page 39 GAO-16-325 Cloud Computing
Appendix VII: GAO Contact and Staff 

Acknowledgments

Page 40 GAO-16-325 Cloud Computing

David A. Powner, (202) 512-9286 or pownerd@gao.gov

In addition to the contact name above, individuals making contributions to 

this report included Gary Mountjoy (assistant director), Gerard Aflague, 

Scott Borre, Nancy Glover, Lori Martinez, Tarunkant Mithani, Karl Seifert, 

and Andrew Stavisky.

Appendix VII: GAO Contact and Staff 

Acknowledgments

GAO Contact

Staff 

Acknowledgments

(311411)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537.

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube.

Subscribe to our RSS Feeds or E-mail Updates. 

Listen to our Podcasts and read The Watchblog.

Visit GAO on the web at www.gao.gov.

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.