National Aeronautics and Space Administration

Office of Inspector General

Washington, DC 20546-0001

January 30, 2014

TO: Larry N. Sweet

NASA Chief Information Officer

FROM: Paul K. Martin

Inspector General

SUBJECT: Review of NASA’s Agency Consolidated End-User Services Contract 

(IG-14-013)

In December 2010, NASA awarded the Agency Consolidated End-User Services (ACES) 

contract to HP Enterprise Services (HP) to provide desktop computers, laptops, mobile 

devices, printers, and other computing equipment as well as end-user services, such as a 

help desk and data backup, to NASA employees and contractors. The ACES contract is a 

firm fixed price, indefinite-delivery/indefinite-quantity contract with a maximum value of 

$2.5 billion. The 4-year contract runs from November 2011 through October 2015, after 

which NASA may extend the contract under two, 3-year options. With the ACES 

contract, NASA moved from a Center-based end-user services delivery model under 

which the individual Centers had greater control over products and services to a centrally 

managed, Agency-wide end-user services model. By adopting this enterprise model for 

its most common information technology (IT) services, NASA hoped to save money and 

enhance the security of its IT systems through leveraging economies of scale and 

standardizing institutional IT architecture.

However, NASA and HP have encountered significant problems implementing the ACES 

contract, including a failed effort to replace most NASA employees’ computers within the 

first 6 months and low customer satisfaction. Given that NASA is halfway through the 

base contract period, it must soon decide whether to exercise the first 3-year option or end 

the contract after the base period and find another way to obtain these critical IT services.

The ACES contract requires that prior to exercising an option to extend the contract, the 

Agency’s Contracting Officer must determine that doing so is the most advantageous 

method of fulfilling NASA’s IT requirements. To support that determination, the Office 

of the Chief Information Officer (OCIO) and the Contracting Officer must analyze option 
2

prices, HP’s performance, current market conditions, advances in technology, and other 

programmatic factors. This analysis must allow sufficient time for NASA officials to

pursue appropriate alternative approaches with minimal impact to the Agency in terms of 

technical, cost, or schedule risk should they conclude the best path forward is not to 

extend the ACES contract.

The Office of Inspector General (OIG) initiated an audit of the ACES contract in April 

2013 to determine whether the contract is improving employee end-user services, 

realizing cost savings and other efficiencies, and meeting Agency mission requirements. 

However, given that the time is fast approaching for NASA to decide whether to extend 

the ACES contract or seek other options, we are truncating our audit work and issuing 

this memorandum to enable NASA to consider the issues we identified during the course 

of our review. In light of the criticality of the IT services provided under the ACES 

contract, NASA’s decision on how to move forward will directly affect NASA’s more 

than 17,000 employees and thousands of contractors.

Executive Summary

NASA’s lack of adequate preparation prior to deploying the ACES contract together with 

HP’s failure to meet important contract objectives has resulted in the contract falling 

short of Agency expectations. We attribute these shortcomings to several factors,

including a lack of technical and cultural readiness by NASA for an Agency-wide IT 

delivery model, unclear contract requirements, and the failure of HP to deliver on some 

of its promises. In general, these issues fall into two categories: (1) issues related to the 

Agency’s overall IT governance and (2) management and problems specific to the ACES 

contract.

NASA Not Prepared for an Enterprise-wide IT Approach. Moving from a 

Center-managed IT services contract to a centrally managed enterprise-wide end-user 

services delivery model required a cultural transition at NASA. Based on our previous 

work and work for this review, we found the transition has been difficult and remains 

incomplete. As noted in our June 2013 audit report, NASA’s current IT governance 

model is ineffective, overly complex, and not suitable for managing an Agency-wide IT 

environment.1 Implementing an enterprise delivery model within a historically 

decentralized IT environment requires a strong governance structure in which authority is 

exercised from an Agency-wide perspective, “buy-in” is obtained from key stakeholders,

and adequate consideration is given to the state of the organization’s current and future IT 

needs. However, NASA did not adequately address the shortcomings in its IT 

governance practices prior to initiation of the ACES contract. Consequently, the Agency

has been trying to implement an enterprise-wide IT solution across a decentralized and 

disparate IT environment led by a management culture largely resistant to such change.

 

1

 NASA OIG, “NASA’s Information Technology Governance” (IG-13-015, June 5, 2013). 
3

Contract Not Meeting Agency Goals. Poor implementation by HP on important aspects 

of the contract and inconsistent oversight by NASA have contributed to the ACES 

contract failing to meet Agency expectations. Overall, we found that NASA employees 

have a negative perception of the ACES “brand.” We believe HP’s failure to replace or 

“refresh” computers across the Agency at the start of the contract as promised, the 

inability of HP and NASA to maintain a complete and accurate inventory of IT 

equipment, and inaccuracies in billing invoices greatly contribute to this perception. 

NASA’s lack of a complete system for ordering IT equipment and services further 

hinders the success of the ACES contract. Finally, top NASA IT officials expressed the 

view that HP is performing poorly under the contract even after taking into consideration 

the Agency’s failure to establish sound performance metrics.

NASA is fast approaching a critical decision point when it must weigh the benefits of 

exercising the first 3-year option period or ending the ACES contract and seeking

alternatives to meet the Agency’s IT needs. Regardless of its decision, NASA must 

ensure that its choice aligns with the Agency’s overall enterprise architecture and can be 

executed within the current and planned IT environment and within the expected budget. 

We urge Agency officials to consider the issues we highlight in this memorandum when 

determining how best to meet NASA’s future IT needs. We provided management with 

our draft memorandum for review and have incorporated the resulting technical 

comments, as appropriate. 

Background

For more than a decade, Lockheed Martin Corporation provided NASA with IT end-user 

services under the Outsourcing Desktop Initiative for NASA (ODIN) contract. Each 

NASA Center could tailor the ODIN contract to meet its needs using Center-specific 

delivery orders and Centers had the flexibility to purchase varying hardware and services 

and establish security parameters to meet Center-specific needs. Additionally, each 

Center had its own contracting and support staff assigned to the contract. 

The ACES contract is one of four contracts that make up NASA’s Information 

Technology Infrastructure Integration Program (I3P) and as such is part of an Agency

strategy to move from a Center-centric to an enterprise model of providing IT services.

2

 

NASA established I3P in 2007 with the goals of enabling Agency-wide collaboration 

through a seamless IT infrastructure; realizing efficiencies in IT infrastructure operating 

costs; reducing the complexity of managing IT services; and improving IT security. To 

accomplish these objectives, the I3P program sought to identify IT infrastructure services 

common to all NASA Centers, consolidate those services into fewer contracts, and 

manage them from a centralized service office.

 

2

 The other three contracts that make up NASA’s IP3 effort are: the Web Enterprises Services and 

Technology contract, which provides an Agency-wide capability to create, maintain, and manage 

websites; the NASA Integrated Communications Services contract, which consolidates the NASA 

Integrated Services Network wide area and Center local area networks and services; and the Enterprise 

Applications Service Technologies contract, supporting the NASA Enterprise Applications Competency 

Center to deliver enterprise application services.
4

In December 2010, NASA awarded the ACES contract to HP to provide, manage, secure,

and maintain the bulk of NASA’s personal computing hardware, standard software, 

mobile IT services (including smartphones), and associated end-user services. Under the 

contract, HP provides both “computing seats” and base services. Computing seats 

include hardware (e.g., desktop computer, laptop, operating system, monitor, and docking 

station) and services (e.g., software and system administration). Base services include

e-mail and calendaring, user authentication, security patching, encryption, “loaner pool” 

equipment management, instant messaging services, and a “help desk” to respond to 

customer questions and computer problems.

NASA pays HP a fixed price for each computer seat and additional amounts for base 

services based on the estimated number of Agency employees.3 Unlike the ODIN 

contract under which Centers paid for their delivery orders, the ACES contract requires 

Centers to pay into a common working capital fund managed by the I3P Business Office 

located at the NASA Shared Services Center (NSSC).4 From contract inception through 

August 2013, NASA has paid HP $169 million – $93 million for seat services, $51

million for base services, and $25 million for contract adjustments such as infrastructure 

upgrades, IT hardware, and software made via Infrastructure Upgrade Proposals (IUP)

and catalog purchases.

5

 

In an effort to promote high quality service and customer satisfaction, NASA established 

a series of performance metrics in the contract that affect HP’s compensation. 

Specifically, NASA may retain up to 16 percent of the amount HP invoices monthly if 

the company fails to meet agreed-upon metrics in areas such as customer satisfaction, 

incident reporting, adherence to schedule, and subcontracting goals.

6

As part of the ACES base contract, HP planned to replace all existing ODIN equipment 

with HP equipment within the first 6 months of contract execution. However, for a 

variety of reasons, this did not occur and instead HP purchased the existing computer 

equipment from Lockheed Martin. Many of the significant issues and delays surrounding 

the ACES contract relating to inventory, billing, and security are traceable to this 

deviation from the original contract. Moreover, NASA cannot accurately measure the 

true cost of the ACES contract nor determine whether it has resulted in savings compared 

to the ODIN contract because the Agency is incurring additional costs beyond the base 

and seat charges that affect the total cost of IT services. For example, HP has submitted 

 

3 According to the ACES contract, NASA’s N2 database, the system that contains the total estimated 

number of civil service and contractor employees at each Center, is used to determine this figure. 

4

 The NASA Shared Services Center is a partnership between NASA and a contractor that consolidates 

support functions such as financial management, human resources, IT, and procurement.

5

The ACES contract provides NASA with two options for ordering services and supplies: (1) Enterprise 

Service Request System (ESRS) or (2) IUPs. Because ESRS can only be used for individual orders, 

IUPs are used when supplies and services are needed for multiple users or for items not included in the 

ACES contract.

6

 A complete list of the metrics appears on pages 10 and 11 of this memorandum.
5

two claims demanding several million dollars for services provided in excess of contract 

requirements and disputing amounts NASA has retained based on the company’s failure 

to meet performance metrics. These types of issues, along with a high level of turnover 

and staffing shortages at both HP and NASA, have increased tensions, burdened the 

working relationship, and diminished trust between HP and NASA.

The OIG assessed the current state of the contract against the backdrop of these

challenges, concerns we heard from NASA managers, and the critical decision about the 

future of the ACES contract the Agency Chief Information Officer (CIO) will soon need 

to make. To obtain a broad perspective, we interviewed ACES stakeholders across the 

Agency including the Agency CIO, several Center CIOs, representatives from the EndUser Service Office, ACES Subject Matter Experts, HP representatives, a NASA 

Program Support Manager, procurement representatives, ACES Contracting Officers,

Contracting Officer Technical Representatives, and members of the ACES Source 

Evaluation Board.7 We also reviewed the base contract, contract modifications, and other 

documentation relevant to the contract. 

NASA Not Prepared for an Enterprise Approach 

NASA did not establish and continues to lack the necessary governance structure to 

successfully implement an Agency-wide IT solution. NASA’s current IT governance 

structure is overly complex and ineffective and, in our opinion, not well positioned to 

manage an enterprise-wide IT environment encompassing more than 50,000 federal 

employees and contractors. The ACES contract describes NASA as a singular “enterprise” 

when from a technical and practical standpoint it is not. Functionally, NASA does not 

operate as a single enterprise, but rather as 10 different enterprises as reflected in the 

relatively autonomous operating nature of its Centers. With implementation of the ACES 

contract, NASA moved from a Center-centric IT model to an enterprise model without 

fully considering the technical challenges of such a dramatic change. As one example, 

Center IT representatives told us that HP did not coordinate with Agency personnel to 

adjust firewall rules to allow for the shift from Center-operated to enterprise-wide 

deployment and management. Without this critical coordination, implementation of the 

contract was further delayed. In addition, IT representatives noted that requests previously 

addressed by Center-based ODIN personnel are now routed through the centralized ACES 

end-user services office. In their view, this more cumbersome process results in delays.

Further, NASA IT managers failed to commit sufficient resources to execute an 

enterprise-wide approach. Notwithstanding enthusiasm for an enterprise model among 

some NASA IT representatives, most agreed that the funding and staffing applied to the 

transition were not adequate to ensure a smooth transition. 

 

7

 The End-User Service Office (EUSO) is responsible for providing service management and oversight for 

Agency end-user services. EUSO staff manage technical operations with oversight from the Marshall 

Space Flight Center CIO in collaboration with the enterprise level Service Executive. NASA Centers 

have named subject matter experts to monitor day-to-day contractor activity and be familiar with contract 

requirements. Both EUSO staff and subject matter experts are responsible for providing input used to 

evaluate HP’s performance.
6

NASA’s culture also affects its ability to implement an Agency-wide IT services model. 

As noted in our 2013 IT Governance report, the Agency’s history and organizational 

structure hinders the CIO’s ability to implement and enforce sound IT governance 

initiatives. Moreover, the CIO has limited visibility and control over a majority of 

NASA’s approximate $1.5 billion in annual IT investments. For example, each NASA 

Center employs its own CIO and IT staff, and the Agency CIO has delegated to the 

Center CIOs the responsibility, authority, and accountability for Center IT portfolios. 

Given this structure, it is not surprising that a move to an enterprise model encountered 

resistance. Indeed, HP officials told us that after awarding the ACES contract, NASA’s 

former CIO tasked them with “selling” the enterprise model to Center personnel.

Complicating matters further, NASA’s decision-making process under the ACES contract 

is highly bureaucratic and disseminated throughout multiple levels of Agency 

management, including the Contracting Officer, the End-User Services Office, Center 

representatives, the IT Management Board, and the Agency CIO. According to HP 

representatives, this has complicated nearly every aspect of the contract because of the 

length of time it takes to obtain agreement from all parties. Additionally, HP 

representatives stated that at times NASA required top Agency managers to be consulted 

and in agreement on basic decisions concerning the contract, which contributed to

inefficiencies. Moreover, we found that decisions that should be made at the 

Agency-level are instead being made at individual Centers, sometimes with different 

outcomes. For example, decisions regarding whether each Center must provide HP 

employees with workspace and who is responsible for scheduling technical refreshes 

have been left at the Center-level, leading to inconsistent guidance to HP. 

We also found that satisfaction with the ACES contract and HP varied widely from 

Center to Center, with the level of satisfaction often related to the relationships formed

between HP and NASA personnel at each location. This finding is similar to results of 

our 2013 IT Governance report in which we found that NASA IT representatives tend to 

rely on informal relationships rather than formalized business processes when managing 

Agency IT resources.

Further complicating implementation of an Agency-wide IT model is the lack of a fully 

mature enterprise architecture. The purpose of an enterprise architecture is to ensure that 

business strategies and IT investments are aligned with and support an organization’s 

strategic plan. At NASA, the Enterprise Architecture Office within the OCIO is 

responsible for articulating the Agency’s mission supporting technologies and operational 

model to accomplish the Agency’s IT goals. The Enterprise Architecture Office assesses 

the Agency’s current IT architecture and determines an approach to move from the 

“As-Is” state to the “To-Be” state. However, these plans were not in place prior to the 

implementation of the ACES contract, leading to inconsistent deployments across the 

Agency. Several NASA IT representatives told us that going forward the Agency should 

establish strict guidelines to identify requirements that are truly unique to the individual 

Centers. Further, several NASA IT officials suggested that the Agency needs to establish 

a comprehensive IT foundation before an enterprise-based IT solution could be 
7

successful. Many of these IT officials also said that integration of all the I3P contracts, 

including ACES, is weak and incomplete. In fact, an August 2013 review of the I3P 

program by NASA officials recommended that NASA either reaffirm its commitment to 

the program by providing adequate resources to accomplish its objective, de-scope its 

objective to make it an executable program, or abandon the enterprise approach for 

delivery of IT services.8

Source Evaluation Board. NASA managers we spoke with expressed concerns that 

some of the issues encountered with the ACES contract can be traced to weaknesses in 

the Agency’s request for proposal (RFP), the Source Evaluation Board (SEB)

methodology, and the staffing and process used to draft the contract.

9

 The 

SEB – composed of NASA civil servants from a variety of Centers – was charged with 

assisting the Source Selection Authority by providing expert analyses of the proposals 

from prospective contractors. Although the SEB’s operations were based at the NSSC,

not all of the Board members were located there. Several SEB members worked at other 

NASA Centers and had to travel to the NSSC for weeks at a time to perform board 

functions. These members were not specifically dedicated to the contractor selection 

process and would often return to their home Center to perform their normal duties. 

Some NASA officials also questioned whether SEB members from the Centers had the 

ability to view the prospective contract from an Agency-wide perspective as opposed to a

Center-specific viewpoint. Center representatives who were not SEB members expressed 

concern that while they had an opportunity to review parts of the RFP and contract, the 

SEB was not staffed to handle their feedback and many issues were left unaddressed by 

the Board. Additionally, many officials we spoke with said the RFP was not fully 

representative of the state of NASA’s IT environment and that HP did not perform 

sufficient due diligence in preparing its proposal.

At the conclusion of their SEB duties, several Board members transitioned into positions 

responsible for administering the ACES contract. This appears to have contributed to 

disagreement between HP and NASA personnel regarding the terms of the contract and 

further stressed the working relationship between the two groups. For example, HP 

officials told us they believe the Agency is asking them to undertake tasks not required 

by the contract and therefore the company is charging NASA and submitting invoices 

when it performs these services. On the other hand, many NASA officials expressed the 

view that HP should be more flexible regarding contract interpretation and show a greater 

commitment to improving customer service. 

Lessons Learned. During our review, NASA managers, IT representatives, and 

procurement personnel shared their experiences relating to the transition from a 

Center-based to an enterprise-level IT approach, including several “lessons learned”: 

 

8

 I3P Assessment Team, “I3P Assessment Team: Final Report” (August 14, 2013).

9

 The SEB is a group of government civilian personnel representing functional and technical disciplines 

charged with evaluating contractor proposals and developing summary facts and findings. The SEB 

assists the Source Selection Authority by providing expert analyses of the proposals in relation to the 

evaluation factors contained in the solicitation.
8

 Consider as a threshold matter whether NASA’s IT environment is sufficiently 

homogenous and mature to implement an Agency-wide IT solution.

 Ensure adequate staffing and funding are available to support Agency-wide IT 

initiatives.

 Establish standardized end-user IT processes across the Agency and designate the 

appropriate decision making authorities.

 Establish guidelines to help identify requirements that are truly unique to 

individual Centers.

 Determine the Agency’s commitment to the overall I3P initiative.

 Include a thorough and complete representation of NASA’s IT environment in 

any future RFPs.

 Ensure personnel involved in the RFP process and SEB members have the skill 

sets necessary to handle feedback and concerns of Centers and Mission 

Directorates.

 Ensure SEB members are provided the appropriate time to perform their 

evaluation duties.

 Review whether it is appropriate for members of the RFP team or SEB to

administer the resulting contract.

 Establish a single group to perform Agency-wide IT implementation to ensure

consistency and leverage lessons learned during implementation.

Contract Not Meeting Agency Goals

The ACES contract has faced significant challenges from the outset. Not only was the 

enterprise-wide approach a radical change for NASA, but HP was unfamiliar with 

NASA’s IT environment and culture. Additionally, the OCIO had limited experience 

developing and awarding an enterprise-wide IT contract and failed to adequately prepare 

for the difficulties entailed in consolidating the heterogeneous, Center-specific IT 

services previously provided under the ODIN contract into a single, Agency-wide IT 

services program.

10

Technology Refresh. In its proposal, HP promised to replace all existing ODIN laptop 

and desktop computers with new HP equipment – known as a technology refresh – within 

6 months of contract award, and this provision was incorporated into the ACES contract. 

However, HP was unable to deliver on this promise. According to Agency IT officials, 

this occurred because HP did not have a good understanding of NASA’s IT environment 

and did not perform sufficient due diligence to identify the issues the company would 

face during contract implementation. On the other hand, HP representatives attributed 

 

10 Upon award of the contract to HP in December 2010, Lockheed Martin Corporation, the incumbent 

contractor, filed a protest claiming that NASA’s evaluation of proposals and its selection process was 

unreasonable. The Government Accountability Office denied Lockheed Martin’s claim in April 2011,

but the protest delayed the start of the ACES contract for 4 months.
9

the delay in refreshing equipment to inefficient decision making by the OCIO on 

hardware specifications for new computers and NASA’s inability to develop a 

functioning ordering system for new IT equipment.

11 Whatever the exact causes, HP’s

plan changed from a complete refresh of all NASA computers within the first 6 months to 

a phased replacement approach that will not be complete until April 2014. 

This early failure significantly affected the success and acceptance of the ACES contract. 

According to the original contract, “the successful phase-in of ACES seats would 

establish the foundation for IT services management and the achievement of ACES goals 

and objectives as well as setting the tone for end-user service delivery throughout the 

Agency.” The contract further stated that HP would execute a seamless phase-in of 

products and services from ODIN to ACES and provide users with an immediate 

technology refresh of computing services capabilities. According to HP, the refresh was 

the foundation for achieving NASA’s goals for ACES and underpinned HP’s strategy to 

save NASA more than 40 percent compared to the previous contractor, establish a single 

stable IT security environment, and enable the introduction of new technology early in its 

tenure as NASA’s new contractor for end-user services.

Once HP realized it could not accomplish the full technology refresh in the timeframe it 

had promised, it purchased the computers, laptops, and other IT assets from Lockheed 

Martin for approximately $27 million. However, a complete and accurate inventory of 

ODIN assets was not available, and consequently HP had little assurance of exactly what 

equipment it had purchased. In addition, purchasing ODIN equipment left many NASA 

employees and contractors with computers not equipped with features specified in the 

ACES contract. For example, the ACES contract required the refreshed computers be 

equipped with encryption software that most legacy equipment lacked. The subsequent 

theft of a laptop computer from a NASA employee containing sensitive information in 

October 2012 prompted NASA IT officials to devote significant time and money to 

expediting the deployment of encryption software on Agency computers. In the end, 

NASA paid HP an additional $220,538 to undertake the hurried encryption effort – a task 

and expense that would have been unnecessary had HP met its original requirement to 

refresh all of the ODIN equipment with new, encrypted machines within the first 

6 months of the contract. More than 2 years into the ACES contract, HP has yet to 

complete a total hardware refresh; specifically, more than 9,000 of approximately 44,000 

computers have yet to be refreshed as of September 2013.

Contract Modification 48. NASA uses performance metrics to assess HP’s progress in 

meeting contract objectives and goals. Early in the implementation of the ACES 

contract, NASA deemed HP “failing” in the areas of service delivery and incident 

management and characterized customer satisfaction as inconsistent. According to the 

 

11 Once it became clear that HP would be unable to meet its contractual requirement to deliver a complete 

refresh in 6 months, the former Agency CIO needed to decide whether to continue working with HP or 

recommend terminating the ACES contract. According to several Agency IT officials, NASA decided 

against termination because the Agency had neither the funding nor the desire to extend the ODIN 

contract and the former NASA CIO was confident that HP could deliver on its other contract 

requirements.
10

I3P Program Manager at the time, all HP’s metrics were indicating failure because HP 

was not providing the products and services specified in the contract. In response to 

these issues, in April 2012, NASA and HP signed contract modification 48, which 

fundamentally changed many of the requirements of the original contract, including the

phase-in-plan, deployment schedule, and performance metrics. In addition, HP agreed to

provide NASA monthly credits or discounts to its base services and seat service charges

up to $15 million. We summarize the major contract changes resulting from 

modification 48 below:

 Performance Metrics and Retainage Pools. Changed the performance metric 

categories and the calculation approach and added a retainage pool schedule, or 

the amount NASA may withhold from HP based on failure to meet performance 

metrics. NASA increased the amount it can withhold from 12 to 19 percent in 

order to incentivize HP to improve service delivery.

 Phase-In Plan. Adjusted the transition approach to include HP’s purchase of 

existing ODIN assets and support of those assets until HP refreshes the seats. The 

updated phase-in plan describes HP’s plan to deploy new ACES equipment and 

its approach for completing the ACES transition.

 Management Plan. Revised HP’s Management Plan and replaced several key 

members of the ACES management team.

Performance Metrics. NASA assesses HP’s performance via service level agreements 

that outline the level, scope, and quality of a service; the way in which NASA measures 

the service; and the penalty for inadequate performance (retainage). Service level 

agreement categories include such areas as service delivery, customer satisfaction, 

incident management, and adherence to the equipment refresh schedule. Throughout the 

first 2 years of the contract, NASA has struggled to develop sound performance measures 

to evaluate HP’s performance. As noted previously, NASA changed the original metrics 

in April 2012 with contract modification 48 after HP failed to complete the refresh in the 

promised 6 months. A second revision to this criteria occurred in October 2013 when 

NASA agreed to make changes to the performance metrics in response to a claim by HP 

related to performance calculations and retainage amounts. In January 2014, NASA 

rewrote the performance metrics for a third time.

Since contract inception, NASA has retained approximately $6.9 million from HP for 

inadequate performance related to the contract service level agreements. HP’s

performance is measured using four retainage pools and their associated service level 

agreements:

 Metrics Retainage Pool is calculated monthly and is comprised of 8 percent of the 

total monthly costs allocated between seven areas: (1) Service Delivery, (2) 

Service Availability Non-Base Services, (3) Service Availability Base Services,

(4) Customer Satisfaction, (5) Security Management Services, (6) Incident 

Management, and (7) Service Asset Management Effectiveness.
11

 Performance Retainage Pool is assessed quarterly and is comprised of a retainage 

at risk amount of 2 percent per month or 6 percent for each quarterly review 

period.

 Schedule Retainage Pool is calculated monthly and is comprised of 5 percent of 

the total monthly costs allocated between two areas: (1) Legacy Refresh Schedule 

Adherence and (2) Contract Compliance Schedule. Review and evaluation of 

these metrics will continue through the end of the initial ACES computer seat 

deployment or completion of the ACES technical services, whichever is later.

 Small Business Utilization Pool is comprised of 1 percent of the contractor’s net 

monthly invoice and is evaluated against originally proposed Contractor 

Subcontracting Plan goals.

HP is required to submit monthly reports to NASA identifying its performance against 

the established metrics. NASA uses these reports along with the personal observations

and assessments of NASA staff to evaluate HP’s performance. Several Agency IT 

representatives expressed concern that NASA is placing too much reliance on HP’s data 

to measure the company’s performance. Further, many Agency IT representatives told us 

that even though HP is generally meeting most performance metrics, a significant number 

of users remain unsatisfied, leading the representatives to question whether NASA is 

incentivizing the right behavior. While NASA intended to design the contract’s 

performance metrics and retainage pools to promote excellent service delivery and

customer satisfaction, Agency officials did not accurately foresee the resources necessary 

to monitor and evaluate contractor performance in a process many IT representatives 

describe as overly burdensome.

ACES Product Catalog. The ACES contract requires HP to provide a web-based 

catalog of commercial IT products not included as part of base services. Using this 

catalog, employees can place orders, check order status, resolve disputed orders, schedule

delivery and installation, and return equipment. The contract states that HP shall offer 

hardware and software prices at a 30 percent discount below the manufacturer’s

suggested retail price. However, we found that the product catalog is limited and 

equipment and software is often unavailable. According to NASA managers, HP’s

shrinking profit margins are driving product availability and HP has removed items from 

the catalog. According to the contract, NASA can direct HP to add or remove catalog 

items. However, for approximately 4 months in 2013, HP removed all software from the 

catalog without Agency permission, which led users to make IT purchases elsewhere. 

Several IT managers told us that users at their Centers are placing orders using alternate 

procurement vehicles and as a result NASA is not realizing the efficiencies and cost 

savings initially envisioned with the ACES contract.

Lack of a Complete Ordering System. NASA does not have a fully functional system 

to order ACES-provided equipment and services. The ACES contract states that NASA 

will place all orders through either the Enterprise Service Request System (ESRS) or the 
12

Infrastructure Upgrade Ordering Process.

12

 NASA planned for the ESRS to provide 

users with a single automated tool to place orders for end-user services and catalog 

purchases. However, NASA included a statement in the “Ordering of Services and 

Supplies” section of the contract noting that ESRS was still under development:

Note to Offerors: The ESRS is being developed concurrently by the NASA Shared 

Services Center (NSSC) and Computer Sciences Corporation (CSC) under contract 

NNX05AA01C. As development of the ESRS matures, this clause will be revised to 

provide more comprehensive information on the system.

Nearly 2 years later, NASA still has not developed a complete and fully functional

ordering system. While NASA uses ESRS to place single orders, the system cannot 

process bulk orders and no automated integration exists between NASA’s ordering 

system and HP’s order fulfillment and asset tracking system. Consequently, the Agency 

submits bulk orders using spreadsheets, which HP personnel manually enter into their

services and asset management systems.

The absence of a fully functional ordering system has affected HP’s ability to deliver 

equipment and services. For example, the contract states that HP will refresh mobile 

devices such as smartphones every 18 months. However, because the ordering system 

has no mechanism for NASA managers to approve employee mobile device purchases or 

upgrades, these devices are not being replaced on schedule. As a workaround, NASA 

and HP have agreed to upgrades that do not involve additional service charges and are 

strictly technology upgrades for the same service ordered, such as upgrading from an 

iPhone 4 to an iPhone 5. Further, HP proposed adding an additional fee of $104 per 

technical refresh to cover the cost of manually processing bulk orders. NASA has 

rejected this fee proposal and, according to OCIO officials, is focusing on resolving 

issues jointly with HP. 

In addition to the incomplete ESRS ordering system, the Computer Sciences Corporation 

(CSC) is also responsible for developing the Configuration Management Database to be 

used to identify, maintain, track, and report on all ACES-managed equipment. However, 

like the ESRS, the database is not fully functional. The requirements for developing both 

systems were added to an existing service delivery contract between CSC and the NSSC. 

However, because CSC receives its direction from the NSSC Board of Directors rather 

than the OCIO, the OCIO has limited authority over CSC and does not direct its

activities.13 Some NASA IT managers believe that the ordering and tracking systems are 

not complete because of competing priorities and a struggle for funding and resources 

between the NSSC and the OCIO. Assigning the design and development of both the 

ordering system and asset control systems to a service delivery contract outside the 

control of the OCIO has not worked well, and to date neither system is complete or 

functioning as intended.

 

12 ESRS is an integrated management tool designed to be used by all I3P Contractors to coordinate and 

fulfill service requests.

13 CSC is an NSSC support contractor.
13

Incomplete Inventory and Inaccurate Invoices. NASA does not have an accurate 

database to track the services and associated equipment ordered through the ACES

contract. According to the contract, HP is required to maintain configuration control for 

the ACES-managed/provided IT environment and update NASA’s tracking database with 

current information after receiving, installing, refreshing, excessing, or moving items. 

While the database was designed to be NASA’s authoritative record for validating ACES 

services and invoices, NASA managers said the ordering system and database are not 

integrated and the information in the database is not accurate, two factors that

significantly hinder NASA’s ability to validate ACES services and invoices. Currently, a 

cumbersome multi-step process is used to populate the database with IT asset 

information. Specifically, HP extracts the information from its asset tracking system into 

a spreadsheet that NASA personnel load into the database. Agency officials informed us 

that this manual process is error prone and time consuming. Further complicating 

NASA’s ability to maintain an accurate inventory is the presence of unreliable 

information in the database regarding legacy ODIN equipment. When HP purchased the 

ODIN assets, the inventory was outdated and incomplete. However, this admittedly 

inaccurate data was relied upon to determine the refresh schedule and to produce invoices 

for NASA. Although NASA and HP are currently working to correct the data, this task 

will require substantial resources and tremendous effort.

While an interface with the vendor is necessary to gather asset information for deployed 

seats, we question why NASA does not maintain its own inventory of assets in order to 

verify the accuracy of the invoices it receives from HP. During our review, we identified 

several reoccurring issues with ACES invoices, including: (1) IT equipment or mobile 

devices assigned to the wrong employee, the wrong Center, or both; (2) invoices with 

incorrect installation dates; and (3) invoice errors being corrected one month but 

reappearing the next. We also noted that NASA Centers and Programs are expending a 

great deal of internal and contractor resources to verify monthly invoices.

Without an accurate inventory, NASA cannot be sure it is paying the correct amount for 

each ACES seat. To determine the amount of money potentially at risk, we averaged the 

cost of the standard computer seat or “S” available to all NASA employees for a 

Windows and Apple desktop and laptop during the first 3 years of the contract. Because 

the cost per seat can vary, we used the lowest cost for each device in our calculations. 

We then determined the average price NASA will pay for the computer seat based on an 

expected 3-year useful life of the equipment. We calculated that NASA pays between 

$2,300-4,000 to order an “S” seat from HPES for 3 years. Even with the substantial 

resources devoted each month to reconciling HP invoices, almost 2 years into the contract 

NASA has little assurance the amount it pays HP is correct.
14

Table 1. ACES Cost Per Seat

Equipment

Average Cost Per 

Device Per Month

Average 3 Year 

Service Cost 

Microsoft Desktop $65.11 $2,343.96

Microsoft Laptop 78.93 2,841.48

Apple Desktop 101.42 3,651.12

Apple Laptop 111.36 4,008.96

Source: OIG analysis of ACES contract data. 

The cost-per-seat listed in Table 1 is for the standard computing seat. However, the 

contract provides that HP will also provide three other types of seats : the “M” seat with 

pre-defined services and services that can be modified by the end-user; the “B” seat for 

which services are “built” to specific end-user requirements and service options; and the 

“T” seat for which services are rendered through a thin client appliance with predefined 

services and service options.

14

 According to NASA IT officials, “B” seats are currently 

very limited and HP has yet to deliver any “T” seats.

Security and Patch Management. In addition to billing issues, the lack of an accurate 

and complete inventory poses a significant risk to NASA’s IT security. According to the

SANS Institute, a leading research and education organization of IT security 

professionals, the top security control for effective cyber defense is an “inventory of 

authorized and unauthorized devices.” In other words, to secure its network an 

organization must know what equipment is connected to that network. Accordingly, the 

lack of a complete and accurate inventory poses an ongoing security challenge for 

NASA. 

In addition, NASA’s Security Operations Center – the entity that provides centralized, 

continuous monitoring of the Agency’s computer network traffic as well as the

coordination, tracking, and reporting of security incidents – reported in September 2013 

that ACES failed to deploy multiple updates, such as security patches, in a timely 

manner, with some updates several months overdue. Patch management is the practice of

installing software designed to fix problems or update a computer program and its 

supporting data. Because these patches are critical to proactively prevent the exploitation 

of vulnerabilities on IT devices and ensure the security of NASA’s networks, it is crucial 

they be timely installed. However, NASA IT officials told us that patch management of 

ACES computers needed improvement. Many cited the requirement of end-user 

interaction and the delay in installing patches as two of the more crucial IT security 

issues. Because patches are only installed on machines during business hours, many 

users do not interrupt their workflow to install the necessary patch, leaving their 

computers susceptible to vulnerabilities. To address this issue, HP recently adjusted the 

patch schedule to occur at night. However, because computers must be powered on and 

connected to the NASA network to receive the patches, user interaction is still required. 

 

14 “Service Option” is the characteristics and metrics that define a particular type of support to be provided 

by the contractor. A thin client appliance refers to either a software program or an actual computer that 

relies heavily on another computer to do most of its work.
15

Data Management. We are concerned that NASA data stored on ACES computers may 

not be adequately safeguarded at the end of the machines’ useful lives. NASA does not 

own the ACES equipment and returns it to HP when no longer needed. As part of this 

process, the contract requires HP to provide “Wipe and Load” services for ACES seats. 

Wiping is the act of erasing all information on computer hard drives and bringing the seat 

back to the current, fully functional baseline configuration. HP relies on one 

subcontractor to collect devices and another to wipe the hard drives. According to the 

ACES Subject Matter Experts we spoke with, there is little accountability and weak 

internal controls when the computers are collected for Wipe and Load services. For 

example, many users do not receive a receipt for removed equipment and the equipment

often continues to appear as “active” in the ACES inventory. Once the computers are 

collected, the subcontractor loads them into vehicles (sometimes the personal vehicles of 

subcontractor staff) and the computers leave NASA control to travel to another 

subcontractor for hard drive wiping. In many cases, this equipment – potentially 

containing large amounts of NASA data – travels hundreds of miles from the NASA 

Center to the subcontractor’s location before the hard drives are sanitized.

Staffing. Both NASA and HP have experienced extensive turnover of key IT,

procurement, and other staff involved in the daily administration of the ACES contract.

Moreover, NASA relies on a single contracting officer at the NSSC to administer the 

ACES contract and accomplish the tasks multiple contracting officers performed for the 

ODIN contract. During the first 2 years of the ACES contract, the project has had four

contracting officers and five contracting officer technical representatives.

15

 Several 

people involved in administering the ACES contract raised concerns that the contracting 

officer is located at the NSSC while the contracting officer technical representative works 

out of the End-User Service Office at the Marshall Space Flight Center. NASA has also 

experienced turnover in the ACES End-User Services Office and among Center IT 

personnel dedicated to implementing the contract. On the HP side, company officials 

acknowledge they did not appropriately staff the transition from ODIN to ACES and that 

they have experienced high staff turnover at the Centers.

A September 2013 study by the OCIO found that lack of skills and limited knowledge of 

HP Center technicians was leading to inefficient processes, a lack of confidence by 

customers, and an increase in customer dissatisfaction. Additionally, an HP 

representative told us that subcontractors at NASA Headquarters walked off the job over 

a dispute regarding how they were paid for installing new equipment. Specifically, they 

were being paid based on the number of computers installed – installations they could not 

complete if NASA users turned them away. HP representatives estimated that at the time

employees were denying approximately 40 percent of planned refreshes at Goddard

Space Flight Center. NASA IT managers said some of the users refused equipment either 

because they did not order it or because it was wrong equipment. To address this issue, 

NASA and HP are instituting a $100 per seat charge to the party at fault for any 

installation missed four or more times. We are concerned that these staffing issues affect 

the quality and consistency of service delivery.

 

15 A contracting officer’s technical representative assists in the technical monitoring and administration of a 

contract.
16

We are also concerned about the safety of NASA-leased equipment and information if 

subcontractors with access to this property have not completed proper background 

checks. Specifically, we identified several HP subcontractors working at NASA Centers 

who had criminal histories related to theft and child pornography – issues that should 

have been flagged with appropriate background screenings.

Cost Savings. NASA estimates that had the ACES contract been implemented as 

planned, the Agency would have saved approximately $31 million a year compared to the 

ODIN contract. However, because of the many modifications to the contract and the lack 

of a complete ACES inventory, NASA cannot accurately measure the true cost of the 

ACES contract or determine whether it has resulted in any savings to the Agency. In 

addition to the indirect cost related to validating ACES invoices, staff turnover, and the 

civil service resources devoted to the contract activities, NASA is incurring additional 

costs beyond the base and seat charges that may affect the total cost of the contract. For 

example, among the claims submitted by HP for additional payments from NASA are 

charges for $12.9 million in June 2013 for services the company believes it provided in 

excess of contract requirements and $5.4 million in July 2013 disputing performance 

calculations and subsequent payment retainage. NASA settled the latter claim in the fall 

of 2013. Furthermore, as previously mentioned NASA paid HP an additional $220,000

between July 2012 and March 2013 to expedite encryption on laptops computers.

NASA continues to modify the contract to address gaps and request additional services

that result in additional charges. Of the 200-plus contract modifications, 128 were for 

IUPs. NASA uses IUPs to request a variety of services such as a dedicated ACES 

technician for a specific NASA program or a Center paying for an early technical refresh. 

As of November 2013, NASA has negotiated $31.6 million in IUPs with HP. In the 2 

years since its inception, NASA has made over 200 contract modifications to the ACES 

contract. In our opinion, the high number of modifications in such a relatively short 

period indicates that the contract was not specific enough to meet the needs of NASA’s 

decentralized IT environment and that the services provided to date are not meeting

NASA’s expectations.

Lessons Learned. During our review, ACES stakeholders, including HP representatives

and NASA IT and procurement officials, shared with us the issues, risks, and lessons 

learned they have encountered during development and implementation of the ACES 

contract, including:

 Ensure Agency requirements are clearly defined in a complete, concise, and 

realistic contract.

 Consider mechanisms or options available to the Agency to enforce contract 

requirements that are not met.

 Review performance metrics to ensure they promote the intended outcome and 

ensure that NASA has a viable means to measure the contractor’s performance, 

including allocating sufficient staff with adequate time to spend on the project.
17

 Address the barriers hindering development of a fully functional ordering system 

or explore alternative solutions.

 Establish an inventory of services and assets in order to verify the accuracy of 

invoices and ensure proper security and patching of devices.

 Consider reviewing current data management practices.

 Ensure identification of the proper resources, such as staffing, technical skills, and 

management to execute contract requirements.

Management Action

NASA is fast approaching a crossroads related to the ACES contract and soon must 

decide whether to execute the first 3-year option period or begin a lengthy and 

labor-intensive effort to identify alternates to obtain its critical IT services. Prior to 

making this decision, we encourage NASA to consider the information contained in this 

memorandum in addition to its own reviews, feedback from customers and stakeholders, 

and estimates of its projected future funding levels. 

cc: David Radzanowski

Chief of Staff, Office of the Administrator

Bill McNally

Assistant Administrator for Procurement