NASA

Earned Value 

Management 

Implementation 

across Major 

Spaceflight Projects Is 

Uneven

Report to Congressional Requesters

November 2012

GAO-13-22

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-13-22, a report to 

congressional requesters 

November 2012 

NASA

Earned Value Management Implementation across 

Major Spaceflight Projects Is Uneven

Why GAO Did This Study

NASA historically has experienced cost 

growth and schedule slippage in its 

portfolio of major projects and has

taken actions to improve in this area, 

including adopting the use of EVM. 

EVM is a tool developed to help project 

managers monitor risks. GAO was 

asked to examine (1) the extent to 

which NASA is using EVM to manage 

its major space flight acquisitions, (2) 

the challenges that NASA has faced in 

implementing an effective EVM 

system, and (3) NASA’s efforts to 

improve its use of EVM. To address 

these questions, GAO obtained 

contractor and project EVM data and 

used established formulas and tools to 

analyze the data and assess NASA’s 

implementation of EVM on 10 major 

spaceflight projects; interviewed 

relevant NASA headquarters, center 

and mission directorate officials on 

their views on EVM; and reviewed prior

reports on EVM and organizational 

transformations. GAO compared NASA 

policies and guidance on EVM to best 

practices contained in GAO’s cost 

estimating best practices guide. 

What GAO Recommends

GAO recommends that NASA 

establish a time frame for requiring 

new spaceflight projects to implement 

its new EVM system; conduct an EVM 

skills gap assessment; develop a 

change management plan for EVM; 

and strengthen its EVM requirements

by requiring projects to implement

formal EVM surveillance. NASA 

concurred with two recommendations 

and partially concurred with two others 

citing resource constraints. Despite 

NASA's plans to address some issues 

GAO identified, not addressing all key 

issues lessens the usefulness of EVM 

at NASA.

What GAO Found

The National Aeronautics and Space Administration's (NASA) 10 major 

spaceflight projects discussed in this report have not yet fully implemented 

earned value management (EVM). As a result, NASA is not taking full advantage 

of opportunities to use an important tool that could help reduce acquisition risk. 

GAO assessed the 10 projects against three fundamental EVM practices that, 

according to GAO’s best practices cost guide, are necessary for maintaining a 

reliable EVM system. GAO found shortfalls in two of three fundamental practices. 

Specifically, we found that

• More than half of the projects did not use an EVM system that was fully

certified as compliant with the industry EVM standard.

• Only 4 of the 10 projects established formal surveillance reviews, which

ensure that key data produced by the system was reliable. The remaining 6 

projects provided evidence of monthly EVM data reviews; however, the rigor 

of both the formal and informal surveillance reviews is questionable given the 

numerous data anomalies GAO found. 

GAO also found that 3 projects had reliable EVM data while 7 had only partially 

reliable data. For the EVM data to be considered reliable per best practices it 

must be complete and accurate with all data anomalies explained. 

NASA EVM focal points, headquarters officials, project representatives, and 

program executives cited cultural and other challenges as impediments to the 

effective use of EVM at the agency. Traditionally, NASA's culture has focused on 

managing science and engineering challenges and not on monitoring cost and

schedule data, like an effective EVM system produces. As a result, several 

representatives said this information traditionally has not been valued across the 

agency. This sentiment was also echoed in a NASA study of EVM 

implementation. Also cited as a challenge to the effective use of EVM was 

NASA’s insufficient number of staff with the skills to analyze EVM data. Without a 

sufficient number of staff with such skills, NASA’s ability to conduct a sound 

analysis of the EVM data is limited. However, NASA has not conducted an EVM 

skills gap analysis to determine the extent of its workforce needs. 

NASA has undertaken several initiatives aimed at improving the agency's use of 

EVM. For example, NASA strengthened its spaceflight management policy to 

reflect the industry EVM standard and has developed the processes and tools for 

projects to meet these standards through its new EVM system. While these are 

positive steps, the revised policy contains only the minimum requirements for 

earned value management. For example, it lacks a requirement for rigorous 

surveillance of how projects are implementing EVM and also does not require

use of the agency’s newly developed EVM system to help meet the new 

requirements. NASA has attempted to address EVM shortcomings through policy 

changes over the years, but these efforts have failed to adequately address the 

cultural resistance to implementing EVM. 

View GAO-13-22. For more information, 

contact Cristina Chaplain at (202) 512-4841 or 

chaplainc@gao.gov. 
Page i GAO-13-22 NASA

Letter 1 

Background 4

NASA Projects Have Not Consistently Implemented Key EVM 

Practices and Most Did Not Have Access to Reliable EVM Data 8

Cultural, Technical, and Other Challenges Seen as Impediments to 

EVM Implementation 18

NASA Policy Is in Line with Best Practices but Implementation 

Remains the Challenge 22

Conclusions 26

Recommendations for Executive Action 27

Agency Comments and Our Evaluation 28

Appendix I Scope and Methodology 31

Appendix II Importance of Earned Value Management 35

Appendix III Case Studies of Selected Projects’ Implementation of Earned 

Value Management 39

Appendix IV Comments from the National Aeronautics and Space 

Administration 101

Appendix V GAO Contact and Staff Acknowledgments 104

Tables

Table 1: EVM Standards, Regulations, and Requirements Applicable 

to NASA 5

Table 2: Summary of the 10 NASA Spaceflight Projects Use of 

Three Fundamental EVM Practices and Reliability of the 

Data 9

Table 3: Examples of NASA EVM System Compliance with 

ANSI/EIA Guidelines 24

Table 4: NASA 10 Spaceflights Project EVM Data Summary 40

Contents 
Page ii GAO-13-22 NASA

Table 5: Assessment of GPM EVM Practices 45

Table 6: Assessment of JWST EVM Practices 49

Table 7: Assessment of Landsat OLI EVM Practices 55

Table 8: Assessment of LADEE EVM Practices 58

Table 9: Assessment of MMS EVM Practices 62

Table 10: Assessment of MAVEN EVM Practices 69

Table 11: Assessment of OCO-2 EVM Practices 81

Table 12: Assessment of RBSP EVM Practices 87

Table 13: Assessment of SOFIA EVM Practices 91

Table 14: Assessment of TDRS EVM Practices 96

Figures

Figure 1: Example of a Project Work Breakdown Structure 11

Figure 2: GAO Earned Value Analysis of GMI-1 Data 47

Figure 3: GAO Earned Value Analysis of Observatory Data 51

Figure 4: GAO Earned Value Analysis of NIRcam Data 53

Figure 5: GAO Earned Value Analysis of Operational Land Imager 

Data 57

Figure 6: GAO Earned Value Analysis of LADEE Project Data 61

Figure 7: GAO Earned Value Analysis of MMS Project Data 65

Figure 8: GAO Earned Value Analysis of Instrument Suite Data 68

Figure 9: GAO Earned Value Analysis of MAVEN Project Data 72

Figure 10: GAO Earned Value Analysis of Spacecraft Data 75

Figure 11: GAO Earned Value Analysis of Science Operations 

Center Data 78

Figure 12: GAO Earned Value Analysis of Remote Sensing 

Instrument Data 79

Figure 13: GAO Earned Value Analysis of Langmuir Probe and 

Waves Data 80

Figure 14: GAO Earned Value Analysis of OCO-2 Project Data 84

Figure 15: GAO Earned Value Analysis of the Spacecraft Data 86

Figure 16: GAO Earned Value Analysis of Project Data 90

Figure 17: GAO Earned Value Analysis of the SOFIA Project Data 94

Figure 18: GAO Earned Value Analysis of the SOFIA Project Data 95

Figure 19: GAO Earned Value Analysis of the TDRS Spacecraft 

Project Data 98

Figure 20: GAO Earned Value Analysis of the TDRS Sustainment 

Data 100
Page iii GAO-13-22 NASA

Abbreviations

ANSI American National Standards Institute

BAC budget at completion

CPR contract performance report

DCMA Defense Contract Management Agency

EIA Electronic Industries Alliance

EVM earned value management

FAR Federal Acquisition Regulation

GPM Global Precipitation Measurement

GSFC Goddard Space Flight Center

IBR integrated baseline review

ICESat-2 Ice, Cloud, and Land Elevation Satellite2

IMS integrated master schedule

JWST James Webb Space Telescope

LADEE Lunar Atmosphere and Dust Environment Explorer

LDCM Landsat Data Continuity Mission

LOE level of effort

MAVEN Mars Atmosphere and Volatile EvolutioN Mission

MMS Magnetospheric Multiscale

NASA National Aeronautics and Space Administration

NID NASA Interim Directive

NPR NASA Procedural Requirements

OCO-2 Orbiting Carbon Observatory 2

OMB Office of Management and Budget

RBSP Radiation Belt Storm Probes

SOFIA Stratospheric Observatory for Infrared Astronomy

TDRS Tracking and Data Relay Satellite System

VAC variance at completion

WBS work breakdown structure

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-13-22 NASA

United States Government Accountability Office 

Washington, DC 20548

November 19, 2012

The Honorable Kay Bailey Hutchison

Ranking Member

Committee on Commerce, Science, and Transportation

United States Senate

The Honorable Ralph M. Hall

Chairman

The Honorable Eddie Bernice Johnson

Ranking Member

Committee on Science, Space, and Technology

House of Representatives

The Honorable Bill Nelson

Chairman

Subcommittee on Science and Space

Committee on Commerce, Science, and Transportation

United States Senate

The National Aeronautics and Space Administration (NASA) has a 

portfolio of 21 major spaceflight projects that are expected to cost $19.4 

billion to develop.1 Historically, NASA has experienced problems with 

persistent cost growth and schedule slippage in the majority of its major 

projects. As a result, NASA acquisition management has remained on 

GAO’s high risk list since it was first introduced in 1990.2

Over the years, NASA has undertaken a number of actions to improve its 

acquisition management function and has shown some progress in 

improving its performance. One of these actions has been to adopt 

Our work has 

shown that several factors contribute to NASA’s cost and schedule 

performance, including poor cost estimating and underestimating risks 

associated with the development of its major systems.

 

1

GAO, NASA: Assessments of Selected Large-Scale Projects, GAO-12-207SP 

(Washington, D.C.: March 1, 2012). Each of these major spaceflight projects have a 

lifecycle cost of $250 million or more. Since we began our review, five of the 21 projects 

have launched, one was canceled, and two other projects have entered the portfolio.

2

GAO, High-Risk Series: An Update, GAO-11-278 (Washington, D.C.: Feb. 2011). 

 
 

Page 2 GAO-13-22 NASA

earned value management (EVM). EVM is a project management tool 

developed by the Department of Defense in the 1960s to help project 

managers monitor risks. EVM measures the value of work accomplished 

in a given period and compares it with the planned value of work 

scheduled for that period and the actual cost of work accomplished. 

EVM’s intended purpose is to integrate a project’s cost, schedule, and 

technical efforts for management and provide reliable data to decision 

makers.

Starting in 1997, NASA began to require its projects to implement EVM 

on all significant contracts.3 In 2005, NASA broadened its application of 

EVM to encompass significant project efforts implemented by in-house 

civil service and associated support contractor personnel. NASA has had 

uneven success in effectively implementing EVM, according to GAO and 

NASA Inspector General reports conducted since 1999.4

To determine the extent to which NASA’s major spaceflight projects are 

using EVM to manage the acquisition, we reviewed 10 major spaceflight 

projects, each with a life cycle cost estimate of more than $250 million 

that had been approved to proceed into the implementation phase of 

NASA has 

recognized a need for improved EVM implementation, and has an 

ongoing effort focused on doing so. In light of these issues, the House 

Science, Space, and Technology Committee and the Senate Committee 

on Commerce, Science, and Transportation asked GAO to examine the 

use of EVM at NASA. Specifically, we assessed (1) the extent to which

NASA is using EVM to manage its major space flight acquisitions, (2) the 

challenges that NASA faces in implementing an effective EVM system, 

and (3) NASA’s efforts to improve its use of EVM.

 

3

The NASA Inspector General report notes that NASA Policy Directive 9501.3, Earned 

Value Performance Measurement, issued in February 1997, established the basis for 

applying EVM to its contracts. NASA considered a contract significant if it was a

production contract with an estimated value of $250 million or more or an RDT&E contract 

with an estimated value of $60 million or more with a period of performance greater than 1 

year. In March 2006, this policy was canceled and was replaced by NASA Procedural 

Requirements 7120.5. 

4

NASA IG-99-058, Earned Value Management at NASA, September 30, 1999; GAO, 

NASA: Lack of Disciplined Cost Estimating Processes Hinders Effective Program 

Management; GAO-04-642 (Washington, D.C., May 28, 2004); and GAO, Information 

Technology: Agencies Need to Improve the Implementation and Use of Earned Value 

Techniques to Help Manage Major System Acquisitions; GAO-10-2 (Washington, D.C., 

Oct. 8, 2009). 
 

Page 3 GAO-13-22 NASA

development. Collectively, the projects we reviewed will cost over $14 

billion to develop (See table 2 for a list of the projects).5 We collected all 

available EVM data for the 10 projects for the period of August 2010 to 

August 2011 and used established earned value formulas and tools to 

identify cost and schedule variances and trends. We also assessed the 

projects’ implementation of three fundamental EVM practices that, 

according to GAO’s Cost Estimating and Assessment Guide, are 

necessary for maintaining a reliable EVM system—using a certified 

system, conducting integrated baseline reviews, and performing 

surveillance.6

 

5

NASA's life cycle for flight systems is defined by two phases—formulation and 

implementation. The implementation phase, preceded by the formulation phase, is defined 

as the execution of approved plans for the development and operation of the program or 

project, and the use of control systems to ensure performance to approved plans and 

requirements and continued alignment with the agency’s strategic goals. NASA 

Procedural Requirements 7120.5E, NASA Space Flight Program and Project 

Management Requirements, figure 2-4 and paragraph 1.3.1(c) (Aug. 14, 

2012).(Hereinafter cited as NPR 7120.5E (Aug. 14, 2012.))

We analyzed monthly project management review briefings 

to support our analysis. To assess the reliability of the cost data, we 

electronically tested the data for anomalies, and reviewed relevant project 

documentation and interviewed agency and project officials about the 

data. To determine the challenges the agency faces in its efforts to 

implement an effective EVM system, we developed standard sets of 

questions and interviewed NASA headquarters officials, program 

executives, and EVM focal points at each center and the Human 

Exploration and Operations and Science Mission Directorates to discuss 

their roles as well as the extent to which EVM data is used to inform 

decision making. We also reviewed prior GAO and relevant NASA 

Inspector General reports on EVM. Also, we submitted written questions 

to the project offices to obtain their views on implementing EVM on their 

projects. To determine the steps that NASA is taking to improve its use of 

EVM, we compared NASA policies and guidance intended to improve the 

agency’s implementation of EVM to best practices for earned value 

management as documented in GAO’s cost guide. We also interviewed 

agency officials responsible for developing and implementing the NASA’s 

new system to discuss ongoing initiatives. For additional details on our 

scope and methodology, see appendix I.

6

GAO, GAO Cost Estimating and Assessment Guide, GAO-09-3SP (Washington, D.C.: 

Mar. 2009).
 

Page 4 GAO-13-22 NASA

We conducted this performance audit from July 2011 through November 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

EVM is a project management tool that, when properly used, can provide 

accurate assessments of project progress, produce early warning signs of 

impending schedule delays and cost overruns, and provide unbiased 

estimates of anticipated costs at completion. Pulling together essential 

cost, schedule, and technical information in a meaningful, coherent 

fashion is a challenge for most projects. Without such information, 

managers can have a distorted view of a project’s status and risks. EVM 

also allows individuals outside the project to see a standardized metric 

describing the cost and schedule performance of that particular project 

and compare it consistently with other projects.

EVM measures the value of work accomplished in a given period and 

compares it with the planned value of work scheduled for that period and 

with the actual cost of work accomplished. Differences in these values are 

measured in both cost and schedule variances. Positive variances 

indicate that activities are costing less or are completed ahead of 

schedule. Negative variances indicate activities are costing more or are 

falling behind schedule. For example, if a contractor completed $5 million 

worth of work and the work actually cost $6.7 million, there would be a 

$1.7 million negative cost variance. Schedule variances are also 

measured in dollars, but they compare the earned value of the work 

completed with the value of work that was expected to be completed. For 

example, if a contractor completed $5 million worth of work at the end of 

the month but was budgeted to complete $10 million worth of work, there 

would be a $5 million negative schedule variance.

Earned value provides information that is necessary for understanding the 

health of a project and an objective view of project status. Cost and 

schedule variances can also be used in estimating the cost and time 

needed to complete the project. While some data that an EVM system 

produces are retrospective and indicate performance to date, EVM data 

can also be used to predict future performance. For example, estimates 

at completion for a project can be calculated by using efficiency indices 

Background
 

Page 5 GAO-13-22 NASA

which are based on a project’s past cost and schedule performance. See 

appendix II for additional information on the importance of EVM.

A project’s EVM data comes from multiple sources. For example, each 

contractor that supports a project will produce and deliver EVM reports to 

the project for the work they and their subcontractors perform, if the 

contract so requires. In addition, the project will collect EVM information 

for the work that it performs in-house at a NASA center. All of this lower 

level EVM data can then be consolidated at the project level to produce a 

project level EVM report. Pulling together EVM data from multiple levels 

into a project level report gives the project a comprehensive outlook of its 

cost and schedule, and provides the project manager with early warning 

of potential cost and schedule overruns.

EVM has evolved from an industrial engineering tool to a government and 

industry best practice, providing improved information to conduct 

oversight of acquisition programs. As such, it is guided by industry best 

practices and standard, and is required by regulations and requirements 

at the federal government and the agency level at NASA. These 

requirements and standards are summarized in table 1 below.

Table 1: EVM Standards, Regulations, and Requirements Applicable to NASA

American National Standards 

Institute/Electronic Industries 

Alliance (ANSI/EIA)-748

• Regarded as the national standard and an industry best practice for EVM systems.

• Describes 32 guidelines that a certified EVM system must meet in the areas of organization; 

planning, scheduling, and budgeting; accounting; analysis and management reports; and 

revisions and data maintenance.

Office of Management and Budget 

(OMB) Circular A-11 and Capital 

Programming Guide

• EVM is to be applied for parts of a major acquisition with developmental effort and is to be 

used for both in-house government and contractor work.

• EVM system must be in compliance with the guidelines in ANSI/EIA-748.

• When there is both government and contractor work, the data from the two EVM systems 

must be consolidated at the reporting level for total program management and visibility.

Federal Acquisition Regulation 

(FAR), Subpart 34.2

• EVM system required for major acquisitions in development.

• Contractors and subcontractors are required to report EVM on a monthly basis. 

• When an EVM system is required, an integrated baseline review is required to verify technical 

content and realism of related performance budgets, resources, and schedules.

NASA FAR Supplement, Subpart

1834.2

• EVM system is required on acquisitions for development or production work for flight and 

ground support systems and components, prototypes, and institutional investments, such as 

facilities and IT infrastructure.

• Certified ANSI/EIA-compliant EVM system (as determined by cognizant Federal agency) is 

required for cost or fixed-price incentive contracts and subcontracts valued at $50 million or 

more.

• ANSI/EIA-compliant EVM system (as determined by the contracting officer) required for cost 

or fixed-price incentive contracts and subcontracts valued at $20 million or more but less than 

$50 million.
 

Page 6 GAO-13-22 NASA

NASA Procedural Requirements 

(NPR) 7120.5

• NPR 7120.5E strengthens the previous version by requiring projects to perform EVM with a 

system that complies with the guidelines in ANSI/EIA-748. This directive establishes the 

requirements by which NASA will formulate and implement spaceflight programs and projects 

including flowing down EVM system requirements to applicable suppliers.a

• Requires an integrated review of project baselines as part of their preparations for the 

confirmation review to ensure that the project’s work is properly linked with its cost, schedule, 

and risk and that the systems are in place to conduct EVM.

Source: GAO analysis of standards, regulations, and requirements.

a

For some of its projects, international partners contribute elements of a mission. According to NASA, 

the agreements reached with its international partners do not require the international partners to 

share EVM data for their contribution to a project. When NASA collaborates with an international 

partner, it is done on a no exchange of funds basis. Each partner commits to providing certain 

elements of the overall mission, such as an instrument, spacecraft bus, or launch vehicle, and are 

responsible for delivering the element(s) at the appropriate time. Each partner determines how they 

want to manage their development efforts and assumes all costs for their work. Partners are not 

required to share any information on how much their efforts cost. 

Almost two decades of NASA’s past efforts to improve its use of earned 

value management have had uneven success. An EVM Focal Point 

Council was created in 1996 and focal points were designated at each 

NASA center and the Office of Procurement and the Office of the Chief 

Financial Officer to provide an open forum to share experiences and 

develop a network of support within the NASA EVM community. In 1997, 

the agency issued NASA Policy Directive 9501.3 “Earned Value 

Management Performance” which established the basis for applying EVM 

to NASA contracts. Prior to this policy, centers used their own individual 

policies on performance measurement systems. However, in 1999, the 

NASA Inspector General reported that EVM policy was not consolidated

as an overall program management responsibility and that the Agency 

Program Management Council did not receive comprehensive EVM 

information.7

In 2004, GAO reported that only 2 of 10 NASA projects reviewed used 

EVM consistently and appropriately.

As a result, in 2003, NASA shifted responsibility for the EVM 

policy from the Office of the Chief Financial Officer to the Office of the 

Chief Engineer to emphasize EVM as a project management tool, rather 

than a financial management tool.

8

 

7

The Program Management Council has the responsibility for periodically evaluating the 

technical, safety, and programmatic performance (including cost, schedule, and risk) and 

content of a program or project under their purview. These evaluations focus on whether 

the program or project is meeting its commitments to the agency. 

Several barriers to EVM

implementation were identified, such as lack of reliable financial data, 

8

GAO-04-642. 
 

Page 7 GAO-13-22 NASA

trained EVM staff, data analysis tools, and incentives. Among other 

things, GAO recommended that NASA take action to ensure that a true 

EVM system is used as an organizational management tool to bring cost 

to the forefront in NASA’s decision-making process and that acquisition 

and EVM management policies and procedures be enforced. In response 

to our recommendations, NASA stated that it was updating NASA 

Procedural Requirements 7120.5, its program and project management 

processes and requirements policy, to improve its cost estimating and 

ensure that its cost estimate and earned value analyses were effectively 

used, and the updated policy was issued in 2005. Similar to other 

agencies in our 2009 report on governmentwide use of EVM, we reported 

on weaknesses in NASA’s EVM policies and practices and recommended 

that the agency modify its policies governing EVM to ensure that they are 

consistent with best practices.9 In particular, we found problems with the 

EVM training requirements for personnel responsible for investment 

oversight and management responsibilities. In addition, NASA’s policy for 

revising project cost and schedule baselines did not have adequately 

defined criteria on the acceptable reasons for permitting a rebaselining.10

In January 2010, the Agency Program Management Council approved the 

funding for the EVM capability project to develop an EVM system that 

complies with the guidelines in ANSI/EIA-748. Another goal of the project 

was to determine whether it was feasible to implement a single EVM 

system that would integrate the scope, schedule, and budget of EVM data 

for NASA’s in-house managed efforts and contractor data across the 

We also found weaknesses in how the NASA projects we reviewed 

implemented EVM and managed their negative performance trends. 

NASA acknowledged the identified weaknesses and stated that it was 

revising its NASA procedural requirements for programs and projects to 

include expanded and strengthened policies governing EVM application 

and processes, and revised policies for rebaselining projects.

 

9

GAO-10-2. As part of the audit we selected NASA projects that rely on various elements 

of information technology and reviewed both the agency’s spaceflight and information 

technology specific guidance.

10A rebaseline is when cost, resources, and/or schedule commitments are revised. NASA 

projects are rebaselined when their estimated development cost exceeds NASA’s 

baseline commitment development cost by 30 percent or more and Congress has 

reauthorized the project; events external to NASA make a rebaseline appropriate; or a 

NASA Associate Administrator determines that the project’s scope changed from the 

approved project baseline.
 

Page 8 GAO-13-22 NASA

agency. According to agency officials, NASA invested about $2 million 

into the capability project to pilot the EVM system through two projects: 

the Ice, Cloud, and Land Elevation Satellite-2 (ICESat-2) and the 

Constellation Extra-Vehicular Activity project.11 Through the pilots, the 

capability project demonstrated that an agency-wide EVM system was 

feasible. As a result, a finalized set of processes, tools, guidance, and 

training products that compose NASA’s new EVM system was developed. 

This new system was peer reviewed and approved by a panel of EVM 

experts.

Our assessment of 10 major spaceflight projects showed that NASA has 

not yet fully implemented EVM and thus is not taking full advantage of an 

important tool that could help reduce acquisition risk. GAO found that the 

projects had shortfalls in two of the three fundamental practices that we 

assessed. Specifically, we found that half of the projects did not use an 

EVM system that was certified as compliant with the ANSI/EIA-748 

standard. Most of the projects conducted an integrated baseline review 

(IBR), a practice that ensures the performance measurement baseline 

reflects all requirements and that resources are adequate to complete the 

work. Specifically, 9 projects conducted an IBR for their overall efforts; 

however, the Stratospheric Observatory for Infrared Astronomy (SOFIA) 

project office only conducted an IBR of one of its major contractors. In 

addition, we found that only 4 of the 10 projects had established formal 

independent surveillance reviews to ensure that key elements of the EVM 

process were maintained over time so that the data produced by the 

system provided timely indications of actual or potential problems. For the 

6 projects that did not have formal independent surveillance in place, 

each provided evidence that they instituted monthly EVM data reviews, 

which according to project officials, helps them to continually monitor cost 

and schedule performance. However, the rigor of both the formal and 

informal surveillance reviews is questionable given the numerous EVM 

data anomalies we found in the monthly EVM reports. Specifically, we 

found many unexplained anomalies, such as the presence of negative 

numbers or missing data, which caused us to question the reliability of the 

data. Out of the 10 projects we reviewed, we found that just 3 projects 

had reliable EVM data while the remaining 7 had only partially reliable 

data.

 

11The second EVM pilot on the Constellation Extra-Vehicular Activity Project was not 

completed due to the termination of the Constellation program. 

NASA Projects Have 

Not Consistently 

Implemented Key 

EVM Practices and 

Most Did Not Have 

Access to Reliable 

EVM Data
 

Page 9 GAO-13-22 NASA

Overall, the projects are using EVM, but NASA has not consistently 

implemented EVM across these projects. For example, we found that 

several projects were not implementing EVM at the project level, which is 

considered a best practice. Table 2 summarizes the performance of each 

of the 10 projects in meeting the three fundamental EVM practices and 

the reliability of the data. 

Table 2: Summary of the 10 NASA Spaceflight Projects Use of Three Fundamental EVM Practices and Reliability of the Data 

Projects

Used A certified EVM 

system compliant with 

ANSI/EIA standard

Conducted an 

integrated baseline 

review

EVM System

surveillance 

is being performed

Data resulting from 

the EVM system 

are reliable

Global Precipitation Measurement    ◐

James Webb Space Telescope    ◐

Landsat Data Continuity Mission    ◐

Lunar Atmosphere and Dust 

Environment Explorer   ◐ ◐

Magnetospheric Multiscale   ◐ ◐

Mars Atmosphere and Volatile 

Evolution Mission ◐  ◐  

Orbiting Carbon Observatory 2a

  ◐  

Radiation Belt Storm Probes   ◐  

Stratospheric Observatory for 

Infrared Astronomy Project ◐ ◐ ◐ ◐

Tracking and Data Relay Satellite 

Replenishment    ◐

Source: GAO analysis of NASA and contractor data.

Notes: For detailed information on the EVM performance of the projects selected for our case studies, 

see appendix III. 

a

At the time of our review, the Orbital Sciences Corporation did not have a certified EVM system that 

complied with the ANSI/EIA-748 standard; however, the Defense Contract Management Agency 

certified their EVM system in January 2012. 

Key:  The agency met this criterion

◐ The agency partially met this criterion

 The agency did not meet this criterion
 

Page 10 GAO-13-22 NASA

Of the 10 projects we reviewed, 4 projects had a certified EVM system, 3 

did not, and 3 had a mixture in which some contractors and 

subcontractors had certified systems and some did not. When an EVM 

system is certified, the agency has assurance that the implemented 

system was validated for compliance with the ANSI/EIA-748 standard by 

independent and qualified staff and therefore can be considered to 

provide reliable and valid data from which to manage a project. The 

Global Precipitation Measurement (GPM), Tracking and Data Relay 

Satellite System (TDRS), Landsat Data Continuity Mission (LDCM), and 

James Webb Space Telescope (JWST) were the only projects that 

provided evidence that the contract performance reports provided came 

from EVM systems that were certified as compliant with the ANSI/EIA-748 

standard. The Lunar Atmosphere and Dust Environment Explorer 

(LADEE), Magnetospheric Multiscale (MMS) and Radiation Belt Storm 

Probes (RBSP) projects did not have EVM systems that were certified to 

be compliant with the ASNI/EIA-748 standard. Finally, the Jet Propulsion 

Laboratory, a federally funded research and development center that the 

California Institute of Technology manages under a contract with NASA, 

was the only NASA Center with a certified EVM system. The Jet 

Propulsion Laboratory is responsible for managing the Orbiting Carbon 

Observatory 2 (OCO-2) project. The Mars Atmosphere and Volatile 

Evolution Mission (MAVEN) and SOFIA prime contractors also had 

certified systems; however, their project offices did not. NASA does not 

require a certified EVM system for their in-house work.

Using the project’s integrated master schedule and contract performance 

reports, we assessed the EVM data provided by the projects against 

selected fundamental ANSI/EIA guidelines to determine the extent to 

which each project’s EVM system, whether certified or not, was meeting 

them. The guidelines we reviewed included whether the work breakdown 

structure (WBS)—which provides the basis of the project schedule—was 

consistent between the EVM report and the schedule, whether the 

schedule identified significant task interdependencies, and whether the 

project had identified a time-phased budget baseline for tracking cost and 

schedule variances. As shown in figure 1, a work breakdown structure 

breaks down product-oriented elements into a hierarchical structure that 

shows how elements relate to one another as well as to the overall end 

product. By subdividing a project into smaller elements, management can 

more easily plan and schedule the program’s activities and assign 

responsibility for the work.

Few Projects Had EVM 

Systems Compliant with 

the Industry Standard

Criteria

• Project obtained EVM data from 

certified system(s).

• Project shows a consistent work 

breakdown structure between the 

EVM report and the schedule. 

• Project schedule identifies 

significant task 

interdependencies.

• Project uses a time-phased 

budget baseline for tracking cost 

and schedule variances.
 

Page 11 GAO-13-22 NASA

Figure 1: Example of a Project Work Breakdown Structure

We found that even for the projects that had certified systems, there were 

problems with consistency between the WBS and the EVM report and the 

schedule. For example, we found discrepancies in the hierarchical 
 

Page 12 GAO-13-22 NASA

structure and numbering of WBS elements for JWST, an $8.8 billion 

project. Specifically, the project’s WBS dictionary showed mission 

assurance efforts numbered differently than contractor reports for two 

contractors, each of which had mission assurance labeled with different 

WBS numbers.12

Our review of the project schedules also revealed that about half of the 

schedules were missing predecessor and/or successor dependencies 

and had constraints that prevented the schedule from responding 

properly to updates. Since the schedule is the foundation for the EVM 

baseline, it must be properly sequenced. This means knowing how one 

activity (the predecessor) affects another (the successor) and how each 

affects the critical path.

NASA officials explained that neither the spacecraft nor 

the near infrared camera contractor was required to follow the projectlevel WBS structure or numbering scheme. NASA officials explained that 

while it is not a requirement for the project and contractor WBSs to be the 

same, it is recommended that the prime contractor lower-level WBS 

numbering scheme be consistent with the overall project WBS numbering 

format. Doing so allows easier total project integration of cost and EVM 

data for project reporting. Consistency of the WBS element between the 

cost estimate and the schedule facilitates updating the cost estimate with 

actual costs and ensures there is correlation between the cost estimate 

and schedule.

13 When the schedule is not sequenced correctly, 

the reliability of the EVM data is called into question. Our review found 

that the MMS project was missing dependencies for 31 percent of its 

remaining activities for its instrument suite contract and 36 percent of the 

remaining activities for the instrument suite were constrained.14

 

12EVM data are typically provided to NASA in the form of a standard report called the 

contract performance report. This report is the primary source of cost and schedule status 

and provides the information needed for effective program control. The contract 

performance report provides cost and schedule variances, based on actual performance 

against the plan, which can be further examined to understand the causes of any 

differences.

Due to 

13The critical path method is used to derive the critical activities—that is, activities that 

cannot be delayed without delaying the end date of the program. The amount of time an 

activity can slip before the program’s end date is affected is known as “total float.” 

14Constraints are generally used to demonstrate an external event’s effect on the 

schedule. However, because they prevent activities from responding dynamically to 

network logic, including actual progress and availability of resources, they can affect float 

calculations and the identification or continuity of the critical path and can mask actual 

progress or delays in the schedule. Date constraints should be minimized because they 

restrict the movement of activities and can cause false dates in a schedule. 
 

Page 13 GAO-13-22 NASA

the major sequencing issues in the MMS instrument schedule, we 

questioned the reliability of the overall network and the schedule’s ability 

to correctly calculate float values and the critical path. MMS project 

officials said they believed many of the constrained activities we found 

were not valid because they reside in another schedule. In addition, 

officials said some of the constraints found are in the Harness area, and if 

removed, these constraints would have no effect on the overall schedule. 

Furthermore, MMS officials said some of the sequencing issues may be 

caused by manual integration because some instrument provider 

schedules are in Microsoft Project and others are in Primavera, and 

therefore it is not possible to ensure all tasks have been linked properly. 

As part of their schedule health check process, the MMS project 

scheduler tests the schedule for missing dependencies, logic errors, and 

reasonable durations and the results are shared with the project office 

and the contractor so that appropriate action can be taken. However, 

when we removed the 15 level of effort type activities from the missing 

dependencies count, the schedule still showed 28 percent of the 

remaining activities missing dependencies. We also removed the 14 level 

of effort type activities and 4 Harness from the constraint count, the 

schedule still showed 33 percent of the remaining activities were 

constrained. MMS program officials said that they have a process in place 

to manage float and the critical path, however, the schedule we reviewed 

still showed significant sequencing issues. 

Finally, we found that 4 of the 14 the schedules we analyzed were not 

resource loaded. This means that the schedule did not have the required 

labor, materials, equipment, and other resources assigned to the 

appropriate activities. When the schedule is not resource loaded, then 

costs need to be spread over time using some other method that may not 

be as straightforward as having the costs integrated directly within the 

schedule. Having a resource-loaded schedule is a best practice for 

developing the time phased budget baseline. The time-phased budget 

baseline represents the plan that performance is measured against for 

the life of a project. It takes into account that program activities occur in a 

sequenced order, based on finite resources, with budgets representing 

those resources spread over time. Deviations from the baseline identify 

areas where management should focus their attention.
 

Page 14 GAO-13-22 NASA

In keeping with best practices, 9 of the 10 projects conducted integrated 

baseline reviews. An IBR is an evaluation of the performance 

measurement baseline—the foundation for an EVM system—to 

determine whether all project requirements have been addressed, risks 

have been identified, mitigation plans are in place, and available and 

planned resources are sufficient to complete the work. Conducting an IBR 

increases confidence that the performance measurement baseline 

provides reliable cost and schedule data for managing the project and 

that it projects accurate estimated costs at completion. OMB has 

endorsed the IBR as a critical process for risk management on major 

investments and requires agencies to conduct IBRs for all contracts that 

require EVM. Since an IBR’s goal is to verify that the technical baseline’s 

budget and schedule are adequate for performing the work, it offers many 

key benefits such as laying a solid foundation for successfully executing 

the project and enabling better understanding of the risks.

Officials for the SOFIA project did not conduct an integrated baseline 

review at the project level; however, its prime contractor for the 

engineering and modification of the airborne observatory platform did 

conduct an integrated baseline review. According to project officials, the 

lack of a project-level IBR is related to the EVM system being 

implemented “on the fly” late in the development phase for SOFIA, as a 

result of an audit recommendation in 2010.15 However, project officials 

noted that the EVM baseline was established concurrently with an agency 

approved re-plan and Joint Cost and Schedule Confidence Level analysis 

in 2009 and 2010 and was reviewed by a Standing Review Board as part 

of that process.16

 

15 NASA Inspector General, Final Memorandum on Audit of the Stratospheric Observatory 

for Infrared Astronomy (SOFIA) Program Management Effectiveness, IG-09-013, 

(Washington, D.C., Mar. 27, 2009).

16The Joint Cost and Schedule Confidence Level is a cost estimation tool that involves a 

probabilistic analysis of cost, schedule, and risk inputs to arrive at development cost and 

schedule estimates associated with various confidence levels.

Majority of Projects 

Conducted an Integrated 

Baseline Review

Criteria

• Project performed an IBR to 

verify that the performance 

measurement baseline is realistic 

and to ensure that the contractor 

and government mutually 

understand the potential project 

risks. 
 

Page 15 GAO-13-22 NASA

Four of the 10 projects we assessed had a comprehensive EVM 

surveillance system in place. Of the remaining 6 projects, 1 had formal 

surveillance at the project level but its contractor did not, and 2 projects 

did not have a formal surveillance system at the project level, only their 

prime contractors did. The remaining 3 projects did not have any formal 

surveillance, but provided evidence that EVM data, such as cost and 

schedule variances, were being reviewed during their monthly status 

reviews. Beyond reviewing cost and schedule variances and variances at 

completion, formal surveillance reviews ensure that the processes and 

procedures continue to satisfy the ANSI/EIA EVM guidelines. A formal 

surveillance plan involves establishing an independent surveillance 

organization with members who have practical experience using EVM. 

This organization then conducts periodic surveillance reviews to ensure 

the integrity of the contractor’s EVM system and where necessary 

discusses corrective actions to mitigate risks and manage cost and 

schedule performance. Effective surveillance ensures that the key 

elements of the EVM process are maintained over time and on 

subsequent applications.

NASA delegates surveillance of contractor EVM systems to the Defense 

Contract Management Agency (DCMA); however, NASA has no entity to 

perform independent surveillance reviews to ensure that the ANSI/EIA748 standard is being met for EVM efforts performed in-house or by 

nonprofit organizations. Without an independent surveillance function, an 

organization’s ability to use EVM as intended may be hampered since 

surveillance monitors problems with the performance measurement 

baseline and EVM data. If the kinds of problems that formal surveillance 

can identify go undetected, EVM data may be distorted and may not be 

meaningful for decision making.

Only 3 of the 10 projects we reviewed, MAVEN, RBSP, and OCO-2, 

produced fully reliable data for managing the project and reporting status. 

The other projects only partially met the criterion to have an EVM system 

in place that produces reliable data. If done correctly, EVM data can 

provide an objective means for measuring project status and forecasting 

potential project cost overruns and schedule slippages so that timely 

action can be taken to minimize their impact. To do so, however, the 

underlying EVM data must be reliable, meaning that they are complete 

and accurate and all data anomalies are explained. In our analysis, we 

found multiple cases of data anomalies that caused us to question the 

reliability of the EVM data. For example, we found several EVM reports 

where a contractor reported that no work was planned or accomplished, 

Majority of Projects Do 

Not Have a Comprehensive 

Surveillance System in 

Place

Unreliable EVM Data Limit 

NASA’s Ability to Measure 

Project Performance

Criteria

• Project verifies that EVM 

processes and procedures 

continue to satisfy the ANSI/EIA 

EVM guidelines.

• Independent surveillance 

organization conducts reviews to 

ensure the integrity of the 

organization’s EVM system. 

Criteria

• Project EVM system(s) produces 

reports that are complete and 

accurate. 

• All reported data anomalies are 

explained. 

• EVM data can be mapped 

through various levels to 

precisely identify root cause(s) of 

issues.
 

Page 16 GAO-13-22 NASA

but actual costs were incurred without an explanation in the variance 

analysis report to say why this happened. Additionally, we found cases 

where a contractor reported that work was planned and actual costs were 

incurred, but a negative amount of work was performed—work that was 

previously reported as completed was now reported as not completed. 

Further, we also found several instances where a project reported an 

estimate at completion but no budget at completion.17

In some cases, the projects provided no explanations for the anomalies 

we found. In other cases, the projects said that the size of the anomalies 

did not breach the variance reporting threshold and therefore no 

explanation was required.

Finally, we found 

instances of negative values in the EVM reports. When explanations were 

provided in the variance analysis reports, the reasons for these anomalies 

included use of estimated, rather than actual costs, or adjustments from 

prior periods due to mistakes or errors with the underlying EVM systems. 

For example, the SOFIA project said that many of the negative values in 

its EVM reports were due to over-reporting of earlier progress, 

mischarges by employees, delayed cost postings, inappropriate use of 

charge codes. When there are data anomalies such as those we 

identified for SOFIA, the EVM data can become skewed and can distort 

true performance.

18

 

17An estimate at completion is an independent assessment of the cost to complete

authorized work based on the project’s historical EVM performance. Budget at completion 

is the sum of all estimated budgets, representing at the project level the cumulative value 

of the budgeted cost of the work scheduled over the life of the project or at lower levels, 

such as a control account or work breakdown structure element.

However, whether the variance breaches the 

threshold or not, cost and schedule deviations from the baseline plan give 

management information about where corrective action plans are needed 

to bring the project back on track or to update completion dates. If 

programs only address those issues that breach the thresholds, 

management may be losing valuable insight into project risks and causes. 

EVM data should be valid and free from unexplained anomalies (e.g., 

missing or negative values) because without reliable performance data, 

project management’s ability to identify potential problems and intervene 

early to mitigate the problems can be limited. Therefore, anomalies 

18Variance thresholds try to quantify an acceptable range of deviation; those that do not 

exceed a threshold are usually not a concern while those that do are worthy of further 

inspection to determine the best course of action to minimize any negative impacts to the 

cost and schedule objectives.
 

Page 17 GAO-13-22 NASA

should be minimized and the reason for each should be fully explained in 

the monthly EVM variance analysis reports. To do less limits the 

completeness and accuracy of the EVM data and makes the resulting 

variance determinations unreliable. While an industry standard for what 

constitutes an acceptable volume of anomalies does not exist, EVM 

experts in the public and private sectors stated that the occurrence of 

EVM data anomalies should be rare.

For four projects that provided subcontractor EVM data, we tried to map 

EVM data at the project level to lower level EVM data at the subcontractor 

level. However, we were only able to successfully map the data for one of 

the projects. This mapping allows project managers to track cost and 

schedule by defined deliverables to more precisely identify which 

components are causing cost or schedule overruns and to more 

effectively mitigate the root cause of the overruns. When the reports do 

not allow for traceability, project managers are not able to effectively 

measure progress, use the reports to monitor and control costs based on 

the original baseline, and/or to track where and why there were 

differences. For example, when we attempted to map the EVM data in the 

lower level reports for MAVEN’s spacecraft, science operations center, 

remote sensing instrument, and Langmuir Probes and Waves instrument 

efforts to the overall MAVEN project EVM report, we were not able to see 

how the costs tracked from one report to another and therefore could not 

reconcile the costs between the reports. Such issues raise the question of 

which reports contain the true costs for these efforts. However, MAVEN 

officials walked us through the process they use to ensure that lower-level 

reports map to the project level reports. Furthermore, MAVEN officials 

said they do not mandate that their contractors follow a certain reporting 

format, instead any adjustments necessary to ensure that the lower-level

reports map to the project-level reports are made by the project office. 

Though the MAVEN project does not prescribe to a standard reporting 

format, attempting to manually resolve incompatible pieces of data can 

become time-consuming, expensive and can lead to data reliability 

issues. We also had similar difficulty mapping various levels of EVM data 

for the MMS project. The MMS project was able to demonstrate how the 

Southwest Research Institute (SwRI) budget at completion in the lowerlevel report mapped to the SwRI budget at completion in the MMS project 

report, but because of the way the contractor submits their data, project 

officials said that the two reports will never match. Although the project 

was able to explain how the data tracked, again, attempting to manually 

resolve incompatible pieces of data can become time-consuming and can 

lead to data reliability issues. Since our review, MMS officials said the 
 

Page 18 GAO-13-22 NASA

project is working to capture data at lower WBS levels which will allow for 

a closer tie between the cost and schedule data.

All the projects we reviewed were using some EVM to manage their work. 

However, the extent to which EVM is implemented across NASA’s inhouse projects and their contractors varies by project and center. For 

example, 3 of the 10 projects we reviewed did not report project level 

EVM data.19 Implementing EVM at the project level rather than just for the 

contract is considered a best practice. In addition, OMB policy requires 

the use of an EVM system for both in-house and contractor work and 

when there is both government and contractor work, the data from the 

two EVM systems must be consolidated at the reporting level for total 

program management and visibility. Integrating government and 

contractor cost, schedule, and performance status at the project level 

should result in better project execution through more effective 

management. In addition, some of the in-house projects we analyzed 

were only required to meet EVM principles and gather “EVM-like” data. 

Further, contractors such as the nonprofit organization managing the 

MMS instrument suite, do not report its EVM in the standard contract 

performance report format because they are only required to meet the 

intent of the standard. Other contractors for the JWST and MAVEN 

projects are required to provide EVM reports that show cost and schedule 

data by WBS elements.

NASA EVM focal points, headquarters officials, project representatives, 

and program executives cited cultural and technical challenges, as well 

as other challenges, as impediments to the effective use of EVM at the 

agency. NASA’s culture traditionally has focused on solving science and 

engineering challenges and not on monitoring cost and schedule data 

such as data produced by an effective EVM system. Technical challenges 

were also cited as an impediment to effective EVM use, but opinions 

differed within NASA on the extent of their impact. The technical 

challenges cited involved difficulty in gathering sufficiently detailed data

for timely inclusion and analysis in an EVM system. In addition, though 

NASA has not conducted an EVM skills gap analysis, NASA 

 

19GPM collects EVM data at the project level, however, this data was provided to GAO in 

a format we could not analyze. We included GPM in the 7 projects that had project level 

data.

Projects Have Not 

Consistently Applied EVM

Cultural, Technical, 

and Other Challenges 

Seen as Impediments 

to EVM 

Implementation

Criteria

• Project uses an integrated EVM 

system across the entire project 

to measure cost, schedule, and 

performance goals. 

• Project uses the 32 guidelines in 

ANSI/EIA-748.
 

Page 19 GAO-13-22 NASA

representatives said it is a challenge for the agency to implement EVM 

effectively due to a lack of sufficient staff with the skills and experience to 

analyze EVM data.

Almost half of the more than 30 NASA representatives we interviewed, 

including a large number of those charged with implementing EVM 

agencywide—known as EVM focal points—and officials from the Chief 

Engineer’s office, cited NASA’s culture as a challenge to the effective use 

of EVM at the agency. Specifically, several NASA representatives said 

that historically, NASA’s culture has not focused on or valued the kind of 

information that EVM can highlight. For example, a NASA EVM analyst 

said the culture has been focused on science and engineering and that 

accomplishment of that work has been the first priority for managers. 

Discussion of the cost of the work has been a secondary concern. A 

NASA EVM focal point described overcoming a culture that is based on 

applied and basic research rather than discrete tasks with discrete 

deliverables, which are among the requirements for effective EVM 

implementation. Further, a senior official at NASA headquarters told us 

that in project reviews, discussion of cost and schedule information, like 

EVM data and analysis, tends to be pushed to the very end of the review 

meeting and generally is not discussed in detail. Because EVM data is 

not universally valued within NASA, in some cases, the data generated to 

satisfy a project’s EVM requirement may be of limited use. For example, 

one NASA EVM focal point said some managers are just “checking the 

box” with respect to using EVM. The task is performed but the 

requirement to collect EVM data was viewed as a nuisance that ultimately 

did not provide worthwhile information. Nonetheless, several of those we 

interviewed said persistent inquiries about EVM data from senior 

management at headquarters, especially over the last couple of years, 

are having a positive impact on the culture and forcing projects to pay 

more attention to the data.

The NASA representatives we interviewed also cited technical challenges 

as having an impact on the effective use of EVM, although their views 

varied on the extent of these challenges. For example, about half of the 

focal points we interviewed reported that a challenge to using an EVM 

system at NASA was aligning it with the agency’s accounting system, the 

SAP Core Financial System. One of the problems cited was that EVM 

data collection may require more detailed data than a project has 

collected for the agency’s accounting system and this may require the 

use of estimated costs instead of actual costs. Estimating these costs can 

NASA Culture Seen as Not 

Valuing EVM

Technical and Other 

Challenges Cited as 

Making EVM Use More 

Difficult
 

Page 20 GAO-13-22 NASA

create additional work for the project, delay the production of EVM data, 

and limit the reliability of the EVM data that is produced. Nonetheless, a 

NASA manager said, projects could more effectively plan their work to 

better accommodate the accounting system. For example, NASA’s 

accounting system is set up to measure and report on labor in terms of 

full-time equivalents. A project, however, may have set up its earned 

value management system with a different measure for labor, such as 

productive hours. As a result, the accounting system cannot fill in the 

proper numbers for an earned value analysis, potentially causing more 

work for the project. Further, the EVM data could be less accurate due to 

the use of estimates rather the actual figures. However, if the project had 

planned from the outset to have the same measure of labor as the 

accounting system, there would not be a problem having this data fit the 

EVM system.

An Office of the Chief Financial Officer representative did not believe that 

the projects have consistently demonstrated that the accounting system is 

a problem, but nonetheless agreed that potential work-arounds and slight 

changes to processes are potential solutions for these issues. The Office 

of the Chief Financial Officer has started an initiative to address both the 

level of detail of the data and improve the monitoring of contractor cost 

performance at levels that may be lower than levels at which obligations 

are made and costs are reported in the financial system.20

NASA’s use of contracts with one entity to provide goods or services to 

several different NASA projects was also cited as a challenge to use of 

EVM. For example, NASA centers may have a contract with one firm to 

provide engineering support services. Multiple projects may seek services 

using a single task order on this contract. Because of the way the NASA 

accounting system is configured, this approach can create artificial 

variances when looking at EVM data on a month-to-month basis. 

According to a NASA representative, contractual requirements can 

correct these issues and allow for a closer accounting of the funds for 

A report on a 

NASA EVM pilot project noted that the greatest impediment for 

implementing EVM is cultural resistance, not technical challenges. 

Specifically, it noted that “it’s not the EVM Process. It’s not the EVM 

Tools. It’s not the SAP Accounting System. It is the NASA culture.” 

 

20As part of the initiative, the Office of the Chief Financial Officer piloted an approach for 

collecting cost at the level necessary to manage the work performed to support the work 

elements for effective project EVM.
 

Page 21 GAO-13-22 NASA

EVM purposes and in fact, one NASA center has already instituted such 

requirements in a new contract for services. NASA plans to address this 

issue as current service contracts expire, but it will take time for the new 

data requirements that would provide the desired data to be implemented.

Other challenges cited include the difficulty of gathering sound EVM data 

from nonprofit subcontractors, such as universities. One project, for 

example, reported that EVM data from nonprofit subcontractors were 

immature or non-existent. The nonprofits may be doing a significant 

amount of work for a center, but are not equipped to collect earned value 

data at the level of detail needed, Office of the Chief Financial Officer and 

center representatives reported. EVM focal points said the problem of 

collecting EVM data from nonprofits to feed into a larger project-level 

EVM system could be mitigated through contract language that clearly 

specifies what data are required from contractors. According to these 

officials, this kind of language has been included in Jet Propulsion 

Laboratory contracts and has been successfully demonstrated as a result. 

Furthermore, NASA is concerned that if nonprofits and small businesses 

are required to have a fully compliant or certified EVM system, they may 

not be able to bid on the work. However, the Federal Acquisition 

Regulation is clear that no offeror can be eliminated from consideration 

for a contract award because the entity does not have a compliant or 

certified EVM system.21

NASA representatives we interviewed said there was a need for improved 

abilities across the agency to analyze EVM data and implement EVM 

systems. Specifically, several focal points said the challenge for NASA is 

not as much in obtaining EVM data because most of the information 

comes from private contractors responsible for much of NASA’s work, but 

in having a staff that can analyze the data and integrate it at the project 

level. A senior NASA official also noted that the career civil servants, who 

typically are the first level of review for EVM data, do not have 

background or training in EVM and cannot conduct a sound analysis of 

the data. A project representative echoed this comment and noted a 

general awareness of EVM within projects but a shortage of in-depth 

knowledge to understand EVM fundamentals and how to interpret the 

data it produces. For example, some projects seek to reset EVM 

 

21FAR § 34.201(b). 

Understanding of EVM 

Varies Widely Across 

NASA
 

Page 22 GAO-13-22 NASA

baselines to match funding allocations, which thwarts efforts to examine 

cost and schedule trends. One EVM focal point told us it has been difficult 

to convince project managers that EVM can predict what will happen in 

their projects given the highly technical nature of their work. For example, 

a senior manager of a program that experienced significant schedule

delays and cost overruns stated that he is an “EVM skeptic” and that he 

does not see EVM data as helpful in helping him track the performance of 

a project.

Additionally, the employee skill sets available to analyze and implement 

EVM vary widely from center to center, headquarters officials said. In 

recent years, NASA has provided EVM training to a large number of 

employees; however, the agency has not conducted a skills gaps 

analysis, which could help to determine the number and extent of the 

staff’s EVM expertise. NASA centers may have staff skill levels reflective 

of the level of EVM use at the center. Some centers have many projects 

producing EVM data while others may only rarely work on a project that 

uses EVM. Without a sufficient number of trained staff to analyze 

contractor data and implement in-house EVM efforts, NASA will likely 

continue to struggle to effectively use EVM as a valuable project 

management tool.

NASA has undertaken several initiatives aimed at improving the agency’s 

use of EVM. For example, NASA strengthened its spaceflight 

management policy to require projects to comply with the 32 ANSI/EIA748 guidelines and has developed the processes and tools for projects to 

meet this requirement through its new EVM system. While these are 

positive steps, the policy continues to lack a requirement for rigorous 

oversight or surveillance of how projects are implementing EVM and 

NASA does not require projects to use the new EVM system to implement 

the EVM requirement of the revised policy. In addition, the issues that 

have impeded NASA’s ability to effectively implement EVM, such as its 

culture, are longstanding and, as a result, NASA has not had much 

success implementing EVM in the past.

The agency’s recent revision of NASA Procedural Requirements 

7120.5—the policy that governs NASA’s spaceflight projects and contains 

project EVM requirements—strengthened the EVM requirements over 

prior versions of the policy. For example, the revised policy requires all 

NASA Policy Is in 

Line with Best 

Practices but 

Implementation 

Remains the 

Challenge
 

Page 23 GAO-13-22 NASA

spaceflight projects to demonstrate compliance with each of the 32 EVM 

guidelines as set forth in ANSI/EIA-748, whereas the prior policy only 

required projects to comply with seven high-level EVM principles.22

However, the new policy still only contains the minimum requirements for 

earned value management, such as the thresholds for implementing EVM 

and the requirement to comply with ANSI/EIA-748 guidelines.

The 

new requirements took effect through the release of an interim directive 

on September 28, 2011 and have since been made final in NASA’s most 

recent update to 7120.5. As a result, projects meeting EVM reporting 

thresholds that enter implementation after that date are required to 

comply with the new requirement. According to an agency official, the 

Office of the Chief Engineer and the mission directorates will determine 

which projects that began development under the prior policy must 

comply with the new EVM requirements. At major milestones, Office of 

the Chief Engineer representatives will review whether the projects have 

implemented the 32 EVM guidelines.

23

 

22The seven EVM principles as defined by ANSI/EIA-748 entail planning and integrating 

work scope, actual costs, and schedule into a performance management baseline against 

which accomplishments may be measured and assessed and the resulting information 

used to inform management’s decision-making.

The policy 

does not require projects to implement formal independent surveillance of 

their EVM systems. Without effective surveillance, agencies cannot 

ensure they are meeting the ANSI/EIA-748 guidelines because internal 

management systems are not being reviewed to determine if they are 

providing reliable cost, schedule and technical performance data. In 

addition, effective surveillance helps pinpoint problems, and is useful for 

verifying the effectiveness of corrective action plans used to mitigate EVM 

system deficiencies. While projects are not required to implement formal 

independent surveillance, NASA does plan to conduct periodic 

surveillance of project EVM systems. For example, NASA plans to 

conduct EVM assessments at two key decision point life cycle reviews 

23Additional, detailed, guidance is provided in an expanded NASA EVM Implementation 

Handbook. For example, the handbook describes roles and responsibilities of key project 

officials with respect to EVM, direction on how to implement EVM on the agency’s inhouse developed projects, and guidance on how EVM surveillance is to be conducted, as 

well as other important considerations. Projects are not required to follow the guidance in 

the handbook.
 

Page 24 GAO-13-22 NASA

and through the Office of the Chief Engineer compliance surveys.24

The policy also does not require projects to use NASA’s new EVM 

system, although the system was designed to help projects meet the 

ANSI/EIA-748 guidelines. We found that the system meets the intent of 

the ANSI/EIA-748 guidelines. Examples of how NASA’s EVM system will 

satisfy three key ANSI/EIA-748 guidelines are summarized in table 3 

below.

While 

these methods will increase the agency’s surveillance efforts, best 

practices call for project level surveillance to be an ongoing, continuous 

process conducted by an independent surveillance function.

Table 3: Examples of NASA EVM System Compliance with ANSI/EIA Guidelines

ANSI/EIA guideline NASA EVM system description 

1. Define the authorized work elements for the 

program. A work breakdown structure, tailored for 

effective internal management control, is commonly 

used in this process.

• Requires a work breakdown structure and outlines how the work will be 

structured and controlled. 

6. Schedule the authorized work in a manner that 

describes the sequence of work and identifies 

significant task interdependencies required to meet the 

requirements of the program.

• Outlines how work will be scheduled including the methodology used to 

create the schedule.

• Explains how the project will document activity and milestone sequences 

through the assignment of predecessor and successor 

interdependencies within detailed Control Account schedules.

• Establishes a process for updating and making changes.

8. Establish and maintain a time-phased budget 

baseline, at the control account level, against which 

program performance can be measured. Initial budgets 

established for performance measurement will be 

based on either internal management goals or the 

external customer negotiated target cost, including 

estimates for authorized but undefinitized work. Budget 

for far-term efforts may be held in higher level 

accounts until an appropriate time for allocation at the 

control account level. If an over-target baseline is used 

for performance measurement reporting, prior 

notification must be provided to the customer.

• Outlines the Performance Measurement Baseline, which is the timephased budget expressed in dollars with separate identification of cost 

elements. It applies to the entire scope of authorized work and spans the 

project’s period of performance.

• Outlines the process and steps for creating a Performance Measurement 

Baseline that is time-phased via integration with the network schedule at 

the Work Package or Planning Package level. The Performance 

Measurement Baseline consists of the aggregation of individual timephased Control Account budgets, including all burdens, for current 

project plans.

• Includes how the control accounts should reflect the budget for the work 

planned and outlines who is responsible for completing the tasks.

Source: GAO analysis of draft NASA EVM system description.

 

24These surveys review center processes and infrastructure for compliance with Office of 

the Chief Engineer requirements, policy, procedures, processes, statutes, and regulations. 

A sample of projects is selected across NASA’s centers every 2 years for review. The 

results are used to give centers feedback on areas where NASA policy and requirements 

should be modified and to respond to OMB and GAO concerns over the implementation of 

requirements. 
 

Page 25 GAO-13-22 NASA

For the projects required to comply with the new policy, use of the 

agency-developed EVM system would meet the ANSI/EIA guidelines; 

however, some projects will be permitted to continue using their individual 

EVM systems as long as the 32 guidelines are met. According to agency 

officials, while future revisions to the policy may require use of the 

standardized agency-developed EVM system by all projects, at this time, 

the agency does not plan to require projects to use the agency-developed 

system in order to meet the guidelines. Instead, senior managers will 

determine on a case by case basis whether a project will use the 

agency’s new EVM system. Currently, only the Space Launch System 

and ICESat-2 projects have been selected to implement the new EVM 

system. According to an agency senior official, the Agency Program 

Management Council approved a phased rollout of the new system 

because NASA does not have the resources to implement it agency-wide. 

For example, there are not enough NASA subject matter experts to 

provide the support needed by the projects when applying the new EVM 

system and to build the institutional capability at the centers. Their 

approach aims to incrementally build the capacity to do EVM, and seek 

increased acceptance of EVM as the requirement for its use is expanded.

Over the years, NASA has attempted to address its EVM shortcomings 

through a series of policy changes, but these efforts have failed to 

adequately address the cultural resistance to implementing EVM 

highlighted by many of the NASA officials we interviewed. NASA has 

made uneven progress since we reported in 2004 that the agency needed 

to improve its use of EVM as a project management tool. Furthermore, a 

2008 NASA internal study noted that projects needed to be educated on 

the value and approaches for using EVM and to provide support for 

setting up EVM within the projects early. Also, an internal agency briefing 

on EVM stated that a change management initiative would be necessary 

in order to successfully implement EVM at NASA centers. Our work has 

also shown that implementing a large-scale initiative, such as EVM, 

requires more than just policy changes.25

 

25GAO, Results-Oriented Cultures: Implementation Steps to Assist Mergers and 

Organizational Transformations, 

To see real change and, in 

effect, a cultural shift at NASA, top leadership must provide to employees 

a succinct and compelling reason that effective implementation of EVM is 

important. Articulating a compelling reason for implementing EVM 

GAO-03-669 (Washington, D.C.: July 2, 2003).

Strong Leadership Needed 

to Fully Implement EVM
 

Page 26 GAO-13-22 NASA

enables employees and other stakeholders to understand the expected 

outcome of the management initiative and engenders not only their 

cooperation, but also their ownership of the outcome, which our work has 

shown can take at least 5 to 7 years to fully implement. NASA, by having 

a policy that is not comprehensive, allowing projects to opt out of using 

the new EVM system, and not committing resources to adequately train 

staff, continues to limit progress in the cultural change needed to 

implement EVM. Without breaking through the cultural resistance to EVM 

and committing to efforts intended to strengthen the use of EVM, NASA is 

missing an opportunity to make full use of a key tool that could help it to 

manage its projects more effectively.

Implementing an effective earned value management system and using it 

across a large federal agency with well-established processes is without 

doubt a challenging task. However, NASA has had uneven progress to 

date. NASA acknowledges that EVM can be a valuable tool for monitoring 

project development and has initiated an effort to implement an 

agencywide system. Currently, only a few of the 10 major spaceflight 

projects we reviewed were able to produce reliable EVM data, raising 

concern that they cannot produce reliable estimates of cost at completion. 

Moreover, until the data are sufficiently reliable, NASA, as well as 

external stakeholders, lose valuable insights into project performance that 

EVM provides. A sound EVM system is not merely an accounting tool; it 

can alert managers to developing problems so that they can be proactive 

in reducing the project’s cost and schedule overruns. However, NASA is 

not making full use of a key tool that could help it address the cost and 

schedule issues that have kept NASA acquisition management on GAO’s 

high risk list for more than 20 years.

Though NASA’s recent efforts to improve its EVM capability and 

strengthen its policy are steps in the right direction, implementation—fully 

integrating EVM into management processes—has been the biggest 

challenge and remains so today. NASA faces cultural and technical 

challenges that it must overcome to successfully implement an earned 

value system and to use this data on a regular basis to inform decisionmaking. Managing change will be key if NASA’s latest effort to overcome 

these challenges and implement an agencywide EVM plan is to succeed. 

To accomplish effective earned value management, strong leadership is 

required to set an expectation that reliable and credible data are 

necessary to manage a successful project. This should be buttressed with 

a sound EVM policy and system that are required, and a commitment of 

resources to enable staff. Without sustained momentum and commitment, 

Conclusions
 

Page 27 GAO-13-22 NASA

its current efforts could suffer the same consequence as those in the 

past.

To improve NASA management and oversight of its spaceflight projects, 

we recommend that the NASA Administrator direct the appropriate offices 

to take the following four actions:

• Establish a time frame by which all new spaceflight projects will be 

required to implement NASA’s newly developed EVM system, unless 

the project is proposing to use a certified system, to ensure that inhouse efforts are compliant with ANSI/EIA-748. The time frame 

selected should take in to account the need to increase NASA’s 

institutional capability for conducting EVM and analyzing and reporting 

the data.

• Conduct an EVM skills gap analysis to identify areas requiring 

augmented capability across the agency. Based on the results of the 

assessment, develop a workforce training plan to address any 

deficiencies.

• Develop an EVM change management plan to assist managers and 

employees throughout the agency with accepting and embracing 

earned value techniques while reducing the operational impact on the 

agency. The plan should include a strategy for having the agency’s 

senior leadership communicate their commitment to implementation of 

EVM. 

• To improve the reliability of project EVM data, NASA Procedural 

Requirements (NPR) 7120.5 should be modified to require projects to 

implement a formal surveillance program that:

• Ensures anomalies in contractor-delivered and in-house monthly 

earned value management reports are identified and explained, 

and report periodically to the center and mission directorate’s 

leadership on relevant trends in the number of unexplained 

anomalies.

• Ensures consistent use of WBSs for both the EVM report and the 

schedule.

• Ensures that lower level EVM data reconcile to project level EVM 

data using the same WBS structure.

• Improves underlying schedules so that they are properly 

sequenced using predecessor and successor dependencies and 

are free of constraints to the extent practicable so that the EVM 

baseline is reliable.

Recommendations for 

Executive Action
 

Page 28 GAO-13-22 NASA

We provided a draft of this report to NASA for comment. In its written 

comments, reproduced in appendix IV, NASA’s Chief Engineer stated that 

the agency concurred with two recommendations and partially concurred 

with two other recommendations. In particular, the agency agreed with 

our recommendation to perform an EVM skills gap analysis and develop a 

workforce training plan to address any deficiencies identified. To that end, 

NASA plans to conduct a skills gap assessment and to augment its EVM 

training program to address the gaps identified. In addition, the agency 

also concurred with our recommendation to develop an EVM change 

management plan and is planning to develop a strategy targeted at all 

levels of the workforce from project team members to the agency’s 

leadership.

The agency partially concurred with our recommendation that NASA 

establish a time frame by which all new spaceflight projects will be 

required to implement NASA's newly developed EVM system, stating that 

they already require projects to perform EVM with an ANSI/EIA 748 

compliant system. NASA stated that its phased rollout approach for 

implementing the agency’s EVM system is based on available resources, 

budgetary constraints, and institutional and project needs. However, 

NASA's approach does not include a timeframe for when projects will be 

required to use the new system. We recommended that NASA establish a 

timeframe for rolling out the system to all projects because a large 

number of projects are not in compliance with NASA’s requirement, and 

very few are implementing the new EVM system. Using the newly 

developed EVM system could help projects better ensure NASA's 

projects are using a system that is compliant with the ANSI/EIA standard. 

The agency also noted its disagreement with the notion that all projects, 

in particular those that have a skilled EVM workforce and a compliant 

system in place, should be forced to use the agency's new system. 

Accordingly, we acknowledged in our report that there may be situations 

where a project should not be required to use the agency’s EVM system, 

such as when a project already uses a certified system or for current, 

ongoing projects. Furthermore, we reported that NASA lacks the 

appropriate level of surveillance of its projects’ EVM systems to monitor 

project adherence to the EVM standard; in addition, the extent to which 

EVM has been effectively implemented across NASA's projects varies. If 

NASA chooses not to require projects to use its new system it should take 

steps to ensure that it monitors their compliance with the standard. 

Finally, while we appreciate that NASA must balance its resources with its 

needs, the benefits that an effective EVM system can provide, such as 

allowing project managers to identify cost growth and take actions to stem 

Agency Comments 

and Our Evaluation
 

Page 29 GAO-13-22 NASA

further growth, warrants prioritization of resources to ensure earlier 

widespread implementation of NASA's EVM system.

The agency also partially concurred with our recommendation that NPR 

7120.5 be modified to require projects to implement a formal EVM 

surveillance program. Citing resource constraints, NASA commented that 

it does not plan to implement a formal surveillance program, but agreed 

that the reliability and utility of the EVM data needed to be improved. As a 

result, the agency plans to establish a surveillance process, expand the 

workforce’s EVM skills, and provide analytical tools including developing 

an EVM System Acceptance and Surveillance Guide. Furthermore, NASA 

said that it was not appropriate to incorporate the surveillance 

requirement in NPR 7120.5 because of the level of detail associated with 

requirements in that policy. The most important part of our 

recommendation is that EVM surveillance should be required to ensure 

better quality data. We reported that only 4 of the 10 projects we 

assessed had a comprehensive EVM surveillance system in place and 

the others had limited or no surveillance being performed and only 3 of 

the 10 projects had fully reliable data. Without an effective surveillance 

program, NASA cannot ensure its projects are meeting the ANSI/EIA-748 

standard because internal management systems are not being reviewed 

to determine if they are providing reliable cost, schedule and technical 

performance data. 

In its response, NASA also noted that the project data we used in our 

report is over a year old and does not take in to account progress made 

over the past year. We disagree and note that in the report we discuss 

progress the agency has made over the past year, such as strengthening 

the EVM requirements in its policy and developing its new EVM system. 

Furthermore, we did not solely rely on project EVM data to develop our 

findings. For example, interviews with project officials and additional 

documentation they provided further validated our findings. Finally, it is 

important to note that NASA Acquisition Management has been on GAO’s 

High Risk list for many years due to the agency’s cost and schedule 

performance on its major projects. EVM is an important project 

management tool that can assist project managers in managing and 

assessing performance. Not addressing key issues that impact the 

availability of accurate and reliable data could lessen the usefulness of 

this key project management tool.

NASA also provided technical comments, which have been addressed in 

the report, as appropriate.
 

Page 30 GAO-13-22 NASA

We are sending copies of this report to interested congressional 

committees, NASA’s Administrator, and other interested parties. In 

addition, this report will also be available at no charge on the GAO 

website at http://www.gao.gov. 

Should you or your staff have any questions concerning this report, 

please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact 

points for our Offices of Congressional Relations and Public Affairs may 

be found on the last page of this report. GAO staff who made key 

contributions to this report are listed in appendix V.

Cristina T. Chaplain

Director

Acquisition and Sourcing Management
Appendix I: Scope and Methodology

Page 31 GAO-13-22 NASA

To determine to what extent NASA’s major spaceflight projects are using 

earned value management (EVM) to manage acquisitions, we reviewed 

all NASA major spaceflight projects with a life cycle cost of over $250 

million that were in the implementation phase and thus required to report 

EVM. There were 13 projects that met these criteria. Of these, 2 projects 

had recently launched and the launch of a 3rd was imminent. These 3 

were excluded from our assessment because the work on these projects 

was nearly complete. Collectively, the 10 projects we reviewed will cost 

over $14 billion to develop.

Our review looked at EVM data for the period of August 2010 to August 

2011. While the majority of the 10 projects we reviewed had at least 6 

months of EVM data, a few did not because the project had only recently 

entered the implementation phase. Additionally, some projects were 

undergoing a re-plan and, therefore, were not required to provide EVM 

data for certain periods of time. In particular, the Tracking and Data Relay 

Satellite Sustainment contract had 5 months of data, the Mars 

Atmosphere Volatile Evolution Mission project had only 3 months of data, 

and the Orbiting Carbon Observatory 2 Orbital contract had just 4 months 

of data. The James Webb Space Telescope contract had EVM data for 

the whole period; however, the contractor underwent a major replan in 

which all EVM data except for the reporting of actual costs were 

suspended from January 2011 to April 2011. Although the Global 

Participation Measurement (GPM) project provided EVM reports for the 

entire project, we did not conduct an analysis of the project EVM data, 

because the performance reports did not contain the detailed data we 

needed for our analysis. However, we were able to assess the 

performance data for GPM’s Microwave Imager Instrument.

To determine cost and schedule performance for the selected major 

projects based on an evaluation of the earned value data, we analyzed 

project and contractor data and documentation including contract 

performance reports; project work breakdown structures; project 

schedules; integrated baseline review briefings; the extent to which 

surveillance of the EVM system was occurring; and monthly management 

briefings for the 10 major spaceflight projects. Specifically, we compared 

project documentation with EVM and scheduling best practices as 

identified in GAO’s Cost Estimating and Assessment Guide and Schedule 

Appendix I: Scope and Methodology
Appendix I: Scope and Methodology

Page 32 GAO-13-22 NASA

Assessment Guide.1 To the extent practicable, we assessed how each of 

the 10 projects’ EVM data adhered to 3 of the American National 

Standard Institute’s (ANSI) and Electronic Industries Alliance (EIA) 32 

guidelines. In addition, we assessed the projects against 3 fundamental 

EVM practices that we believe are necessary for maintaining a reliable 

EVM system, as identified in our cost guide.2

For the contracts that had more than 6 months of data, we used contract 

performance report data in order to generate our estimated overrun or 

underrun of the contract cost at completion by using formulas accepted 

by the EVM community and printed in the GAO Cost Estimating and 

Assessment Guide. To perform this analysis, we examined contractor 

performance reports over the period for which we had data to show 

trends in cost and schedule performances. We generated multiple 

formulas for the projected contract cost at completion that were based on 

how much of the contract had been completed up to August 2011 or 

earlier for some projects. The ranges in the estimates at completion are 

driven by using different efficiency indices based on the project’s past 

cost and schedule performance to forecast the cost of the remaining work 

and adding that cost to the actual costs to date. The efficiency indices 

capture how the project has performed in the past and can be useful in 

predicting how it will perform in the future.

We also analyzed the 

contract performance reports for each project to determine the level of 

data reliability. Specifically, we identified instances of the following: (1) 

negative planned value, earned value, or actual cost; (2) planned value 

and earned value without actual cost; (3) earned value and actual cost 

without planned value; (4) actual cost without planned value or earned 

value; (5) earned value without planned value and actual cost; (6) 

inconsistencies between the estimated cost at completion and the 

planned cost at completion; (7) actual cost exceeding estimated cost at 

completion; and (8) planned or earned values exceeding planned cost at 

completion.

 

1

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009), and 

GAO Schedule Assessment Guide: Best Practices for Project Schedules—Exposure 

Draft, GAO-12-120G (Washington, D.C.: May 2012).

2

The three fundamental EVM practices we assessed were using a certified EVM system, 

conducting integrated baseline reviews, and performing surveillance of the EVM system. 
Appendix I: Scope and Methodology

Page 33 GAO-13-22 NASA

We also analyzed monthly project management review briefings to 

support our analysis. Finally, we analyzed the earned value data 

contained in EVM performance reports obtained from the projects. To 

perform this analysis, we compared the cost of work completed with 

budgeted costs for scheduled work to show trends in cost and schedule 

performances.

To assess the reliability of the cost data, in addition to electronically 

testing the data for anomalies, we also reviewed relevant project 

documentation and interviewed agency and project officials about the 

data. We then followed up on these anomalies with the project offices that 

manage each of the spaceflight projects by sharing our preliminary 

analysis for each of the 10 projects. When warranted, we updated our 

analyses based on the agency’s response and additional documentation 

provided to us. The data that we used were sufficiently reliable for how 

we portrayed them in our report and we are making recommendations to 

the agency to improve NASA’s data reliability based on the findings 

discussed in our report. We did not test the adequacy of the agency or 

contractor accounting systems.

To support and clarify information in our documentation reviews, we 

interviewed agency officials at NASA headquarters and EVM Focal Point 

Working Group members—the agency officials that are responsible for 

developing an integrated, consistent approach for implementing EVM 

throughout NASA, as well as addressing EVM review and surveillance 

issues and activities—at each center and the Human Exploration and 

Operations and Science mission directorates to discuss their roles as well 

as the extent to which EVM data is used to inform decision making. We 

interviewed officials at NASA headquarters in Washington, D.C.; and 

officials from Ames Research Center in Moffett Field, California; Dryden 

Flight Research Center in Edwards, California; Glenn Research Center in 

Cleveland, Ohio; Goddard Space Flight Center in Greenbelt, Maryland; 

Johnson Space Center in Houston, Texas; the Jet Propulsion Laboratory 

in Pasadena, California; Kennedy Space Center in Florida; Langley 

Research Center in Hampton, Virginia; Marshall Space Flight Center in 

Huntsville, Alabama; and Stennis Space Center in Mississippi. 

Additionally, we received responses to questions regarding the EVM data 

from each of the 10 projects we selected for review. These questions 

addressed how EVM practices are implemented at the project level and 

how the project utilizes EVM data.

To determine the challenges that NASA has faced in implementing an 

effective EVM system, we interviewed NASA headquarters personnel to 
Appendix I: Scope and Methodology

Page 34 GAO-13-22 NASA

discuss the status and plans for implementing the agency-wide EVM 

system. We developed a standard set of questions and interviewed EVM 

Focal Point Working Group members at each center and the Human 

Exploration and Operations and Science mission directorates to assess 

the challenges of implementing EVM at individual centers and across the 

agency. We also interviewed a selection of senior officials and program 

executives at NASA headquarters that represent projects from each 

mission directorate and NASA center included in our engagement to 

obtain their perspective on the challenges of implementing and using 

EVM on their projects. We also reviewed prior GAO and NASA Inspector 

General reports that discuss the agency’s prior efforts to implement EVM. 

We examined GAO and NASA Inspector General reports that discuss the 

importance of effective organizational change. Additionally, we received 

written responses to a standard set of questions regarding the challenges 

associated with implementing EVM from each of the 10 projects we 

selected for review.

To determine the steps that NASA is taking to improve its use of earned 

value management, we examined the results of NASA’s EVM capability 

pilot projects and draft policies and guidance and compared these with 

best practices in EVM as discussed in GAO’s Cost Estimating and 

Assessment Guide,3

We conducted this performance audit from June 2011 to November 2012 

in accordance with generally accepted government auditing standards. 

Those standards require that we plan and perform the audit to obtain 

sufficient, appropriate evidence to provide a reasonable basis for our 

findings and conclusions based on our audit objectives. We believe that 

the evidence obtained provides a reasonable basis for our findings and 

conclusions based on our audit objectives.

the ANSI/EIA-748 standard, and OMB Circular A-11, 

Preparation, Submission, and Execution of the Budget and the Capital 

Programming Guide. In addition, we interviewed NASA headquarters 

personnel and EVM Focal Point Working Group members at each center 

and the Human Exploration and Operations and Science mission 

directorates to discuss and obtain information on ongoing initiatives the 

agency has undertaken.

 

3

GAO-09-03SP. 
Appendix II: Importance of Earned Value 

Management

Page 35 GAO-13-22 NASA

Pulling together essential cost, schedule, and technical information in a 

meaningful, coherent fashion is always a challenge for any project. 

Without this information, management of the project will be fragmented, 

presenting a distorted view of project status. For several decades, the 

Department of Defense (DOD) has utilized a tool called earned value 

management (EVM) to compare the value of work performed to the 

work’s actual cost. Earned value management goes beyond the twodimensional approach of comparing budgeted costs to actual costs. It 

attempts to compare the value of work accomplished during a given 

period with the work scheduled for that period. By using the value of 

completed work as a basis for estimating the cost and time needed to 

complete the project, the earned value concept should alert project and 

senior managers to the potential problems early in the project.

In 1996, DOD adopted 32 criteria for evaluating the quality of earned 

value management systems. These 32 criteria are organized into 5 basic 

categories: organization, planning and budgeting, accounting 

considerations, analysis and management reports, and revisions and data 

maintenance. In general terms, the criteria require contractors to

• define the contractual scope of work using a work breakdown 

structure;

• identify organizational responsibility for the work;

• integrate internal management subsystems;

• schedule and budget authorized work;

• measure the progress of work based on objective indicators;

• collect the cost of labor and materials associated with the work 

performed;

• analyze any variances from planned cost and schedules;

• forecast costs at contract completion; and

• control changes.

The criteria have evolved to become an American National Standards 

Institute (ANSI) and Electronic Industries Alliance (EIA) standard for EVM, 

which has been adopted by major U.S. government agencies, industry, 

and the governments of Canada and Australia. The full application of 

EVM system criteria is appropriate for large-cost reimbursable contracts 

where the government bears the cost risk. For such contracts, the 

management discipline described by the criteria is essential. In addition, 

data from an EVM system have been proven to provide objective reports 

of contract status, allowing numerous indices and performance measures 

to be calculated. These can then be used to develop accurate estimates 

Appendix II: Importance of Earned Value 

Management 
Appendix II: Importance of Earned Value 

Management

Page 36 GAO-13-22 NASA

of anticipated costs at completion, providing early warning of impending 

schedule delays and cost overruns.

The standard format for tracking earned value is through a Contract 

Performance Report (CPR). The CPR is a monthly compilation of cost, 

schedule and technical data which displays the performance 

measurement baseline, any cost and schedule variances from that 

baseline, the amount of management reserve used to date, the portion of 

the contract that is authorized unpriced work, and the contractor’s latest 

revised estimate to complete the project.

As a result, the CPR can be used as an effective management tool 

because it provides the project manager with early warning of potential 

cost and schedule overruns. Using data from the CPR, a project manager 

can assess trends in cost and schedule performance. This information is 

useful because trends can be difficult to reverse. Studies have shown that 

once projects are 15 percent complete, the performance indicators are 

indicative of the final outcome. For example, a CPR showing a negative 

trend for schedule status would indicate that the project is behind 

schedule. By analyzing the CPR, one could determine the cause of the 

schedule problem such as delayed flight tests, changes in requirements, 

or test problems because the CPR contains a section that describes the 

reasons for the negative status. A negative schedule condition is a cause 

for concern, because it can be a predictor of later cost problems since 

additional spending is often necessary to resolve problems. For instance, 

if a project finishes 6 months later than planned, additional costs will be 

expended to cover the salaries of personnel and their overhead beyond 

what was originally expected. CPR data provides the basis for 

independent assessments of a project’s cost and schedule status and can 

be used to project final costs at completion in addition to determining 

when a project should be completed.

Examining a project’s management reserve is another way that a project 

can use a CPR to determine potential issues early on. Management 

reserves, which are funds that may be used as needed, provide flexibility 

to cope with problems or unexpected events. EVM experts agree that 

transfers of management reserve should be tracked and reported 

because they are often problem indicators. An alarming situation arises if 

the CPR shows that the management reserve is being used at a faster 

pace than the project is progressing toward completion. For example, a 

problem would be indicated if a project has used 80 percent of its 

management reserve but only completed 40 percent of its work. A 

project’s management reserve should contain at least 10 percent of the 
Appendix II: Importance of Earned Value 

Management

Page 37 GAO-13-22 NASA

cost to complete a project so that funds will always be available to cover 

future unexpected problems that are more likely to surface as the project 

moves into the testing and evaluation phase.

An Integrated Baseline Review (IBR) is conducted to ensure the reliability 

of the EVM data and that the performance measurement baseline 

accurately captures all the work to be accomplished. Data from the CPR 

can then be used to assess project status—typically, monthly. Cost and 

schedule variances are examined and various estimates at completion 

are developed and compared to available funding. The results are shared 

with management for evaluating contractor performance. Finally, because 

EVM requires detailed planning for near-term work, as time progresses, 

planning packages are converted into detailed work packages. This cycle 

continues until all work has been planned and the project is complete.

An IBR is an evaluation of the performance measurement baseline to 

determine whether all project requirements have been addressed, risks 

identified, and mitigation plans put in place and all available and planned 

resources are sufficient to complete the work. Too often, projects overrun 

because estimates fail to account for the full technical definition, 

unexpected changes, and risks. Using poor estimates to develop the 

performance measurement baseline will result in an unrealistic baseline 

for performance measurement.

After the CPRs start being delivered to the government, it is important to 

oversee the project by conducting surveillance of the EVM system. 

Surveillance is reviewing a contractor’s EVM system as it is applied to 

one or more projects. Its purpose is to focus on how well a contractor is 

using its EVM system to manage cost, schedule, and technical 

performance. For instance, surveillance checks whether the contractor’s 

EVM system summarizes timely and reliable cost, schedule, and 

technical performance information directly from its internal management 

system; complies with the contractor’s implementation of ANSI/EIA-748 

guidelines; provides timely indications of actual or potential problems by 

performing spot checks, sample data traces, and random interviews; 

maintains baseline integrity; gives information that depicts actual 

conditions and trends; and provides comprehensive variance analyses at 

the appropriate levels, including corrections for cost, schedule, technical, 

and other problem areas.

Effective surveillance ensures that the key elements of the EVM process 

are maintained over time and on subsequent applications. EVM system 

surveillance ensures that the contractor is following its own corporate 
Appendix II: Importance of Earned Value 

Management

Page 38 GAO-13-22 NASA

processes and procedures and confirms that the contractor’s processes 

and procedures continue to satisfy the ANSI guidelines.

The surveillance team designated to perform project reviews should 

consist of a few experienced staff who fully understand the contractor’s 

EVM system and the processes being reviewed. The surveillance 

organization should appoint the team leader and ensure that all 

surveillance team members are independent. This means that they 

should not be responsible for any part of the projects they assess. Key 

activities on the surveillance team’s agenda include reviewing documents, 

addressing government project office concerns, and discussing prior 

surveillance findings and any open issues. Sufficient time should be 

allocated to all these activities to complete them. The documents for 

review should give the team an overview of the project’s implementation 

of the EVM process.

Successful surveillance is predicated on access to objective information 

that verifies that the project team is using EVM effectively to manage the 

contract and complies with company EVM procedures. Objective 

information includes project documentation created in the normal conduct 

of business. Besides collecting documentation, the surveillance team 

should interview control account managers and other project staff to see if 

they can describe how they comply with EVM policies, procedures, or 

processes. During interviews, the surveillance team should ask them to 

verify their responses with objective project documentation such as work 

authorizations, cost and schedule status data, variance analysis reports, 

and back-up data for any estimates at completion.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 39 GAO-13-22 NASA

We conducted case studies of 10 major spaceflight system acquisition 

projects. This appendix provides a brief description of each project, 

including an analysis of the project’s earned value data and trends. As 

part of our analysis, we assessed the projects’ implementation of three 

fundamental earned value management (EVM) practices that we believe 

are necessary for maintaining a reliable EVM system—using a certified 

American National Standards Institute (ANSI) and Electronic Industries 

Alliance (EIA) compliant system, performing surveillance, and conducting 

integrated baseline reviews.

We also assessed the projects’ EVM data against three ANSI and EIA 

guidelines. These guidelines state that the authorized work elements for 

the project should be defined typically using a work breakdown structure 

(WBS) that has been tailored to the project and that the WBS is the same 

for the cost estimate, schedule, and EVM. The ANSI/EIA guidelines also 

state that projects should have a schedule that describes the sequence of 

work by listing activities in the order in which they are to be carried out 

and identifying significant task interdependencies required to meet project 

requirements. Finally, the ANSI/EIA guidelines state that a project should 

establish and maintain a time-phased budget baseline to track cost and 

schedule variances in an EVM system.

As mentioned above, this appendix includes an analysis of each project’s 

earned value trends from August 2010 to August 2011. These data and 

trends are often described in terms of cost and schedule variances. Cost 

variances compare the earned value of the completed work with the 

actual cost of the work performed. Schedule variances are also measured 

in dollars, but they compare the earned value of the completed work with 

the value of the work that was expected to be completed. Positive 

variances are good—they indicate that activities are costing less than 

expected or are completed ahead of schedule. Negative variances are 

bad—they indicate activities are costing more than expected or are falling 

behind schedule. Variances are merely measures that indicate that work 

is not being performed according to plan and that it must be assessed 

further to understand why. Although our EVM cost projections may show 

that a project is experiencing negative cost variances and schedule 

slippages, this does not mean that a project has exceeded its agency 

baseline commitment and will require additional funds to complete the 

Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 40 GAO-13-22 NASA

project.1 These estimates use a project’s EVM baseline, which represents 

only a portion of the agency baseline commitment for a project. The EVM 

baseline contains the cost and schedule contained in a project’s 

management agreement minus unallocated future expenses and 

schedule margin held by the project and others above the project.2

As of August 2011, the budget at completion for the 10 projects was 

estimated to be $6.4 billion. To estimate the project variance at 

completion, we examined the trends for the earned value data for the 

entire project, if data was collected at that level, or elements of the 

project.3

Table 4: NASA 10 Spaceflights Project EVM Data Summary 

Table 4 provides a summary of the projects’ implementation of 

each EVM best practice we assessed and projected costs.

Dollars in millions

Projects Subcomponenta

Data

review

period

Used a 

certified 

EVM 

system 

compliant 

with 

ANSI/EIA 

standard

Conducted 

an 

integrated 

baseline 

review

EVM 

system

surveillance 

is being 

performed

Data 

resulting 

from the 

EVM 

system 

are 

reliable BAC

Cumulative 

cost 

variance GAO VACb

 

Global 

Precipitation 

Measurementc

Overall

   ◐

 

1

The agency baseline commitment is the integrated set of requirements, cost, schedule, 

technical content, and an agreed-to joint confidence level that forms the basis for NASA’s 

commitment with OMB and Congress. NPR 7120.5E, appendix A (Aug. 14, 2012).

2

Unallocated future expenses are costs that are expected to be incurred but cannot yet be 

allocated to a specific work breakdown structure subelement of a project’s plan. 

Management control of some unallocated future expenses may be retained above the 

level of the project (i.e., Agency, Mission Directorate, or program). The management 

agreement defines the parameters and authorities over which the program or project 

manager has management control.

3

The Landsat Data Continuity Mission (LDCM) and James Webb Space Telescope 

(JWST) projects do not collect project level data and GPM did not provide project level 

data in a format we could analyze. Therefore, for these projects, we were only able to 

analyze EVM data for segments of the project that reported EVM. Also, these projects had 

project segments, such as instruments, that did not produce any EVM data.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 41 GAO-13-22 NASA

Dollars in millions

Projects Subcomponenta

Data

review

period

Used a 

certified 

EVM 

system 

compliant 

with 

ANSI/EIA 

standard

Conducted 

an 

integrated 

baseline 

review

EVM 

system

surveillance 

is being 

performed

Data 

resulting 

from the 

EVM 

system 

are 

reliable BAC

Cumulative 

cost 

variance GAO VACb

 

GPM Microwave 

Imager

August

2010-July 

2011

   ◐ $159 -$13 -$14 to -$22

James Webb 

Space 

Telescope

Overall

  ◐ ◐

Observatory May 2011-

July 2011    ◐ 2,967 2 NA

Near infrared 

camera 

August 

2010-July 

2011

  ◐ ◐ 205 -33 -34 to -48

Landsat Data 

Continuity 

Mission 

Overall

   ◐

Operational Land 

Imager

August 

2010-

August 

2011

   ◐ 129 -45 -49 to -52

Lunar 

Atmosphere 

and Dust 

Environment 

Explorer

Overall

  ◐ ◐

Project September 

2010-June 

2011

  ◐ ◐ 140 3 -0.1 to -13

Magnetospheric 

Multiscale 

Overall

  ◐ ◐

Project August 

2010-July 

2011

  ◐ ◐ 1,037 -18 -47 to -80

Instrument Suite August 

2010-

August 

2011

  ◐ ◐ 233d -4 -10 to -24

Mars 

Atmosphere 

and Volatile 

Evolution 

Mission 

Overall

◐  ◐  
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 42 GAO-13-22 NASA

Dollars in millions

Projects Subcomponenta

Data

review

period

Used a 

certified 

EVM 

system 

compliant 

with 

ANSI/EIA 

standard

Conducted 

an 

integrated 

baseline 

review

EVM 

system

surveillance 

is being 

performed

Data 

resulting 

from the 

EVM 

system 

are 

reliable BAC

Cumulative 

cost 

variance GAO VACb

 

Project June 

2011-

August 

2011

  ◐  259 5 NA

Spacecraft January 

2011-

August 

2011

    169 6 1 to 14

Science 

Operations 

Centere

November 

2010-

August 

2011

 —  ◐  3 5 0 to 0.4

Remote sensing 

Packagee November 2010-

August 

2011

—  ◐  20 1 1 to 7

Langmuir Probe 

and Waves 

Instrumene

November 

2010-

August 

2011

—  ◐  5 -1 -2 to -3 

Orbiting Carbon 

Observatory 2f Overall   ◐  

Project December 

2010-July 

2011

    129 1 -1 to -26

Spacecraft April 2011-

July 2011   ◐  30 0 NA

Radiation Belt 

Storm Probes

Overall   ◐  

Project August 

2010-

August 

2011g

  ◐  291 -32 -40 to -41

Stratospheric 

Observatory for 

Infrared 

Astronomy 

Project 

Overall

◐ ◐ ◐ ◐
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 43 GAO-13-22 NASA

Dollars in millions

Projects Subcomponenta

Data

review

period

Used a 

certified 

EVM 

system 

compliant 

with 

ANSI/EIA 

standard

Conducted 

an 

integrated 

baseline 

review

EVM 

system

surveillance 

is being 

performed

Data 

resulting 

from the 

EVM 

system 

are 

reliable BAC

Cumulative 

cost 

variance GAO VACb

 

Project August 

2010-

August 

2011

  ◐ ◐ 384 4 -1.4 to -76

Airborne System August 

2010-July 

2011

    38 3 3 to 4

Tracking and 

Data Relay 

Satellite

Overall

   ◐

Spacecraft August 

2010-

August 

2011

    698 -131 -152 to -185

Sustainment August 

2010-

August 

2011

   ◐ 5 0 NA

Source: GAO analysis of NASA and contractor data.

a

Items in italics represent subcomponents that are accounted for in higher level EVM reports. 

b

Forecast range based on Actual cost + Budgeted Cost of Work Remaining / Performance factor 

where Performance factor was either the cumulative cost performance index (CPI) * schedule 

performance index (SPI), the current CPI, or current CPI * SPI. 

c

In October 2011, the Global Precipitation Measurement mission was approved for a replan. 

d

Only reflects Phases B-D. 

e

The contract for the Science Operations Center, Remote sensing Package, Langmuir Probe and 

Waves Instrument does not exceed $50 million. Therefore, the supplier is not required to have a 

certified system for this contract. 

f

Orbiting Carbon Observatory 2 is in the process of being rebaselined due to a change in the launch 

vehicle. 

g

Missing data for October and November 2010. 

NA - Not applicable means there was not enough data to forecast an Estimate at Completion. 

Key:  The agency met this criterion 

 ◐ The agency partially met this criterion 

 The agency did not meet this criterion 

With timely and effective action taken by project and executive 

management, it is possible to reverse negative performance trends so 

that the projected negative cost variances at completion may be reduced. 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 44 GAO-13-22 NASA

To get such results, management needs to obtain reliable EVM data from 

EVM systems that adhere to the ANSI/EIA-748 standard for informed 

decision making. Until project offices undertake a rigorous validation of 

their EVM data, NASA faces an increased risk that managers may not be 

receiving the information they need to effectively manage their projects.

The following information describes the key that we used in tables 5 

through 14 to convey the results of our assessment of the 10 case study 

projects’ implementation of EVM practices.

Key description Key symbol

Fully met this criterion 

Partially met this criterion ◐

Did not meet this criterion 

The Global Precipitation Measurement (GPM) mission, a joint NASA and 

Japan Aerospace Exploration Agency (JAXA) project, seeks to improve 

the scientific understanding of the global water cycle and the accuracy of 

precipitation forecasts. The GPM is composed of a core spacecraft 

carrying two main instruments: a Dual-frequency Precipitation Radar and 

a GPM Microwave Imager (GMI). GPM builds on the work of the Tropical 

Rainfall Measuring Mission, and will provide an opportunity to calibrate 

measurements of global precipitation when it launches in 2014.

This analysis focuses only on the GMI-1 effort. Ball Aerospace and 

Technology Company is the prime contractor for GMI. GMI’s current 

contract value is $217 million, which represents approximately 23 percent 

of the total GPM project budget of $932.8 million. The GMI instrument 

was delivered to Goddard Space Flight Center in February 2012 for 

integration into NASA’s upcoming Earth science spacecraft. All remaining 

effort for GMI-1 is post delivery support, which is all level of effort.4

 

4

NASA was also developing a second Microwave Imager instrument for flight on another 

spacecraft—GMI-2—that was descoped from the project in 2012. However, the science 

requirements for GPM can still be met without flying the GMI-2 instrument.

The 

GPM project provided EVM reports for the entire project, but we did not 

Global Precipitation 

Measurement
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 45 GAO-13-22 NASA

conduct an analysis of the project EVM data because the performance 

reports did not contain the detailed data we needed for our analysis.

Table 5: Assessment of GPM EVM Practices

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI/EIA 

standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion 

Data resulting 

from the EVM 

system are 

reliable

GAO variance at 

completion 

GPM Microwave 

Imager

   $159 ◐ -$14 to -$22

Source: GAO analysis of contractor EVM data as of July 2011

The GMI-1 contractor met the three fundamental ANSI/EIA-748 practices 

necessary for a reliable EVM system. The Defense Contract Management 

Agency (DCMA) certified that the GMI-1 contractor’s EVM system 

complied with the ANSI/EIA standard in April 2008. Though the contractor 

has a certified system, the implementation of that system is questionable 

based on our findings below. We assessed GMI-1’s EVM data against 

three ANSI/EIA guidelines. These guidelines state that the authorized 

work elements for the project should be defined typically using a WBS 

that has been tailored to the project and the WBS is the same for the cost 

estimate, schedule, and EVM. Our review found that the WBS in the GMI1 schedule did not match the WBS used for the EVM data. Project 

officials said that the WBS for the project schedule did not match the 

WBS used in GMI contractor’s EVM reports because GMI-1 is only one 

element of the total project and the project level schedule has a simplified 

summary of the GMI-1 schedule that was used for completeness. The 

ANSI/EIA guidelines also state that projects should have a schedule that 

describes the sequence of work by listing activities in the order in which 

they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. Our review of 

the GMI-1 schedule found some sequencing issues. For example, about 

5 percent of the remaining activities were missing predecessor and 

successor links, which are necessary for properly sequencing work so 

that the schedule will update in response to changes. We also found that 

19 percent of the remaining activities had date constraints, which also 

hinder the schedule’s ability to respond dynamically to status updates 

resulting in an artificial or unrealistic view of the project plan. These 

sequencing issues and constraint dates within the schedule affect the 

reliability of the overall network and the schedule’s ability to correctly 

GPM Microwave 

Imager Contractor 

Uses a Certified EVM 

System Compliant 

with the ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 46 GAO-13-22 NASA

calculate float values and the critical path. Project officials agreed with our 

findings but said that these issues are corrected as they are discovered, 

so there is no impact to the project. Finally, though resource loading the 

schedule is not required to meet the ANSI/EIA guidelines, it is a best 

practice and therefore resources should be accounted for in the schedule 

in order to develop this baseline, according to the GAO cost guide. We 

found that the GPM schedule was not resource loaded, it is GAO’s 

assessment that the project did not show evidence that it had established 

and maintained a time-phased budget baseline.

The project conducted an integrated baseline review (IBR) in January 

2006. From that review, officials believed the results were less than 

satisfactory due to the contractor’s inability to demonstrate the integration 

of contract schedule and cost in accordance with their EVM system 

description. In particular, 64 areas of concern were identified during the 

IBR, including major concerns with data continuity, cost/schedule risk, 

and EVM processes. As a result of these issues and changes to the 

contract another IBR was performed in January 2011.

Joint surveillance reviews of the EVM data are being performed by DCMA 

and the contractor. According to DCMA, although they have found some 

deficiencies in the contractor’s EVM data, the contractor has responded 

with acceptable corrective action plans. 

We reviewed contract performance reports from August 2010 to July 

2011. Our review of the GMI-1 EVM data found various data anomalies 

that call into question the reliability of the data. For example, we found 

negative values for EVM data without any explanation in three monthly 

reports. Project officials responded that the negative values all fell within 

the contract threshold and therefore the GMI-1 contractor did not need to 

provide an explanation in the variance analysis report. In addition, there 

were many instances of costs and performances being recorded when no 

work had been scheduled. In response to this finding, project officials 

explained that this work had been accounted for in previous months, 

which explain the missing values. Anomalous EVM data prevents the 

project from being able to gain meaningful and proactive insight into 

potential cost and schedule performance shortfalls, and take corrective 

action to avoid shortfalls in the future. Figure 2 below illustrates that as of 

July 2011, the project was reporting a negative cumulative cost variance 

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Being 

Performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 47 GAO-13-22 NASA

of $13 million and a negative cumulative schedule variance of 

$3 million.

Figure 2: GAO Earned Value Analysis of GMI-1 Data

GPM project officials said the cumulative negative cost and schedule 

variances were due to slips caused by suppliers not delivering flight 

hardware as planned, which pushed uncompleted work into the future. 

The negative variances were also caused by tasks being worked that 

were not included in the baseline. Officials also said the project 

experienced unfavorable variances in labor costs across all integrated 

project teams, which were further affected by the unfavorable 2010 year 

end indirect rate adjustment.5

 

5

Year-end indirect rate adjustments often happen when financial accounting assumptions 

about forward pricing do not hold true. When a company reconciles its books at the year 

end, it often finds that during the year it forecasted a larger business base than the actual 

business base, and at the end of the year, indirect rate adjustments have to be made, 

which can result in additional cost variances for all contracts.

To address the negative schedule variance, 

officials said the contractor increased staffing and added extra shifts, 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 48 GAO-13-22 NASA

which increased labor costs thereby increasing the contract value. 

Officials noted, however, that the increase in contract value did not 

translate into an increase in the baseline, just an increase in the project 

funding. Due to both the negative cost and schedule variances, we are 

forecasting a negative variance ranging from $14 million to $22 million at 

contract completion.6 According to NASA, the project is not overrunning 

its approved baseline commitment, because the EVM baseline does not 

include unallocated future expenses held at the project and headquarters 

level.

The James Webb Space Telescope (JWST) is a large, infrared-optimized 

space telescope that is designed to find the first galaxies that formed in 

the early universe. Its focus will include searching for first light, assembly 

of galaxies, origins of stars and planetary systems, and origins of the 

elements necessary for life. Scheduled to launch in October 2018, 

JWST’s instruments will be designed to work primarily in the infrared 

range of the electromagnetic spectrum, with some capability in the visible 

range. JWST will have a large primary mirror composed of 18 smaller 

mirrors, measuring 6.5 meters (21.3 feet) in diameter, and a sunshield 

that is the size of a tennis court. A successor to the Hubble Space 

Telescope and the Spitzer Space Telescope, JWST will reside in an orbit 

about 1 million miles from the Earth.

NASA rebaselined JWST in September 2011 and made changes in the 

project’s management in response to cost and schedule performance 

issues and the recommendations of the Independent Comprehensive 

Review Panel report. As part of the rebaseline, NASA took the lead role 

for systems engineering from the prime contractor. The telescope, along 

with a segmented primary mirror, will deliver infrared light to the Fine 

Guidance Sensor and fine pointing updates to the Observatory and four 

scientific instruments including the Near-Infrared Camera (NIRCam), the 

Near-Infrared Spectrograph, the Mid-Infrared Instrument and the Fine 

Guidance Sensor/Near InfraRed Imager and Slitless Spectrograph. For

work being performed by its international partners, such as the NearInfrared Spectrograph, EVM data is not collected.

 

6

Variance at completion is calculated by taking the difference between the estimate at 

completion and the budget at completion.

James Webb Space 

Telescope
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 49 GAO-13-22 NASA

At the time of our review, there was no EVM data for the overall JWST 

project or for the work done in-house on the Integrated Science 

Instrument Module. According to project officials, the Integrated Science 

Instrument Module effort began over a decade ago, and significant parts 

of the project, particularly those undertaken in-house at GSFC were not 

structured to enable EVM to be implemented easily. However, the JWST 

project office is collecting EVM data from Northrop Grumman Aerospace 

Systems and obtains copies of the Lockheed Martin Space SystemsAdvanced Technology EVM data, which is the University of Arizona’s 

prime contractor. Northrop Grumman Aerospace Systems is responsible 

for developing and launching the JWST Observatory, which comprises 

the spacecraft, sunshield, and the optical telescope element, systems 

integration and test observatory verification, observatory commissioning, 

and ground and launch support equipment. The University of Arizona is 

responsible for developing the Near-Infrared Camera science instrument.

Table 6: Assessment of JWST EVM Practicesa

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI/EIA standard

Conducted an 

integrated 

baseline 

review

EVM system

surveillance is 

being performed

Budget at 

completion

Data resulting 

from the EVM 

system are 

reliable

GAO variance at 

completion 

Observatory    $2,967 ◐ NAb

Near infrared camera  ◐ ◐ $205 ◐ -$34 to -$48 

Source: GAO analysis of contractor EVM data as of July 2011.

a

All budget and variance at completion data is before the JWST re-plan. 

b

NA - Not applicable means there was not enough data to forecast an Estimate at Completion. 

The observatory contractor met all three fundamental ANSI/EIA-748 

practices necessary for a reliable EVM system. For the observatory 

portion of the JWST project, Northrop Grumman Aerospace Systems has 

a certified EVM System, which it uses to fulfill the earned value reporting 

requirement. Though the contractor has a certified system, the 

implementation of that system is questionable based on our findings 

below. We assessed the observatory contractor’s EVM data against three 

fundamental ANSI/EIA guidelines. These guidelines state that the 

authorized work elements for the project should be defined typically using 

a WBS that has been tailored to the project and the WBS is the same for 

the cost estimate, schedule, and EVM. We found that while consistent 

naming conventions existed between the WBS and contract performance 

Observatory 

Contractor Has a 

Certified EVM System 

Compliant with 

ANSI/EIA Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 50 GAO-13-22 NASA

reports, there were discrepancies in the hierarchical structure and 

numbering of the WBS elements. For example, the WBS dictionary shows 

the Mission Assurance listed as 3.0, while the contract performance 

report provided by the observatory contractor has Mission Assurance 

listed as 2.0. NASA officials responded that the observatory contractor is 

not required to follow the project level WBS hierarchical structure nor the 

WBS numbering scheme. They further stated that the project and the 

observatory WBS structures are not identical because procurement of the 

observatory is only one element of the overall JWST project. NASA did 

not provide us with a schedule in a format that would allow us to 

determine if the schedule had the proper sequencing in place. As a result, 

we were unable to determine if significant task interdependencies such as 

predecessor and successor links were in place to ensure that the 

schedule would update in response to changes. In addition, because we 

could not view the schedule in its native software, we were unable to 

determine if the schedule was resource loaded, which is a best practice 

for establishing and maintaining a time-phased budget baseline.

An Integrated Baseline Review was conducted in February 2010; 

however, because of the rebaseline in September 2011, according to the 

project an additional IBR was held for the observatory in October 2012.

Surveillance is being performed by DCMA. In addition, EVM data is 

reviewed monthly by multiple individuals on the project office as well as at 

higher levels of NASA headquarters and the Goddard Space Flight 

Center.

We reviewed contract performance reports from August 2010 to July 

2011. Our review of the Observatory EVM data found various data 

anomalies that call into question the reliability of the data. For example, 

we found actual costs recorded without any work being performed, 

inconsistencies between the reported estimate at completion and budget 

at completion, large month to month performance swings, and 

unexplained variances. NASA officials explained that during this time 

period the observatory contractor was engaged in re-planning efforts so 

NASA did not want them to expend resources reporting performance 

management data to an outdated performance measurement baseline 

that did not reflect the new rebaseline assumptions. Further, while 

variances analyses were provided in the variance analysis reports for 

WBS elements that exceeded contractual thresholds, there was no 

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Being 

Performed

Data Resulting from the 

EVM System Was 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 51 GAO-13-22 NASA

explanation for the anomalies we found. A variance analysis report 

provides a detailed, narrative report explaining significant cost and 

schedule variances and other contract problems and topics. Without this 

information, management cannot understand the reasons for the 

variances and the contractor’s plan for fixing them. When information is 

missing in a variance analysis report, the EVM data will not be meaningful 

or useful as a management tool.

As of July 2011, the observatory portion of the JWST Project was 51 

percent complete with a positive cumulative cost variance of $2.4 million. 

For the same period the project was also experiencing a positive 

cumulative schedule variance of $0.9 million as seen in figure 3 below. In 

January 2011, the observatory contractor began replanning the remaining 

effort in order to meet the October 2018 launch readiness date. As a 

result, NASA suspended performance measurement reporting during the 

period of January 2011 through April 2011. However, the observatory 

contractor was still required to submit contract performance reports 

depicting actual cost and estimate at completion data.

Figure 3: GAO Earned Value Analysis of Observatory Data
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 52 GAO-13-22 NASA

Since there were only 3 months of EVM data after the rebaseline, we 

were not able to forecast a variance at completion. Since our 

assessment, JWST project officials said they have made some significant 

improvements in implementation and use of EVM that includes an EVM 

approach for the in-house work that will provide EVM metrics to measure 

progress. In addition, the officials said the project is also doing 

managerial analysis on its contracts and project components and 

producing independent estimate of completion each month based on the 

EVM data.

The Near-Infrared Camera (NIRCam) contractor, Lockheed Martin, met 

one of the three fundamental ANSI/EIA-748 practices necessary for a 

reliable EVM system. The contractor has a certified EVM System that it is 

using to report EVM data for NIRCam. Our review found similar problems 

with the WBS hierarchical structure and numbering of the elements. For 

example, the WBS dictionary shows Mission Assurance listed as 3.0 

while the NIRCam contract performance report shows this effort under 

element 5.5. NASA officials explained that the NIRCam contractor is not 

required to follow the project level WBS hierarchical structure or 

numbering of elements. As stated above, we did not receive a schedule in 

its native software, so we were unable to confirm whether the schedule 

was sequenced using predecessor and successor links or if it was 

resource loaded, necessary for establishing the time-phased budget 

baseline.

As stated above, while the JWST project conducted an IBR in February 

2010, the rebaseline necessitated a new IBR, which occurred in March 

2012.

While formal surveillance is not occurring for the Lockheed Martin EVM 

data, monthly reviews of the EVM data are performed by both the project 

and program offices and by independent groups. As part of these 

reviews, trending metrics are prepared and presented to management as 

part of the internal project reviews and monthly status reviews.

We reviewed contract performance reports from August 2010 through 

July 2011 and found various data anomalies that call into question the 

reliability of the data. For example, we found EVM data with negative 

values, no work scheduled but work performed, and actual costs being 

Near-Infrared Camera 

Contractor Has a 

Certified EVM System 

Compliant with 

ANSI/EIA Standard

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is not 

Being performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 53 GAO-13-22 NASA

incurred without any work being performed. NASA officials stated that 

variances that exceeded contractual thresholds should be reflected in the 

variance analysis reports, however many of these anomalies did not 

breach a variance threshold so the report provided no explanation. In 

addition, NASA officials explained that many of these anomalies occurred 

during the project replan, which was formally approved in September 

2011, when the project was rebaselined. As a result of the replan, NASA 

suspended EVM data reporting, which resulted in many of the anomalies 

we found.

As of July 2011, the NIRCam portion of the JWST project was 98 percent 

complete. In July 2011, there was a negative cumulative cost variance of 

$33 million and a negative cumulative schedule variance of $4.4 million 

as seen in figure 4 below. The reasons for the downward trend reflected 

in the graph below were not explained because variance analysis 

reporting was suspended during the replan period.

Figure 4: GAO Earned Value Analysis of NIRcam Data

Based on the downward trend, we are forecasting a negative variance at 

completion ranging anywhere from $34 million to $48 million. This 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 54 GAO-13-22 NASA

analysis is based on information prior to the project’s 2011 replan. The 

variances projected do not take in to account the establishment of a new 

EVM baseline as a result of the replan.

The Landsat Data Continuity Mission (LDCM) is a joint mission between 

NASA and the United States Geological Survey (USGS) that seeks to 

extend the ability to detect and quantitatively characterize changes on the 

global land surface at a scale where natural and man-made causes of 

change can be detected and differentiated. It is the successor mission to 

Landsat 7. The Landsat data series, begun in 1972, has provided the 

longest continuous record of changes in the Earth’s surface as seen from 

space.

Landsat data is a resource for people who work in agriculture, geology, 

forestry, regional planning, education, mapping, and global change 

research. The LDCM provides remotely sensed, highly calibrated, 

moderate resolution, multispectral imagery affording systematic global 

coverage of the Earth’s land surfaces on a seasonal basis and makes the 

data readily available for large-scale and long-term Earth System Science 

and Land use/land cover change research and management. The project 

plans to launch early in February 2013.

LDCM consists of an Operational Land Imager (OLI) and a Thermal 

Infrared Sensor (TIRS) science instrument, a spacecraft, and a mission 

operations element. LDCM does not collect EVM data at the project level. 

The decision not to perform EVM at the project level was reviewed 

extensively prior to proceeding into the design and development phase, 

according to project officials. Also, there is no EVM data for the 

spacecraft effort because this work is being done under a firm fixed price 

contract and NASA regulations do not require EVM for firm fixed price 

contracts. The TIRS instrument, built in-house at NASA’s Goddard Space 

Flight Center, was added late in the formulation phase with an aggressive 

delivery schedule and delivered in February 2012. The Ground System is 

being built and delivered by the USGS. The developmental part of the OLI 

contract with Ball Aerospace and Technology Corporation was completed 

with delivery of the instrument in early October 2011. Following on-orbit 

checkout, the contract will transfer to USGS for management. The OLI 

instrument is the only part of the project that performed EVM. 

Landsat Data 

Continuity Mission
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 55 GAO-13-22 NASA

Table 7: Assessment of Landsat OLI EVM Practices

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI/EIA standard 

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion

Data resulting from 

the EVM system 

are reliable

GAO variance 

at completion

Operational Land 

Imager    $129 ◐ -$49 to -$52

Source: GAO analysis of contractor EVM data as of August 2011.

The OLI contractor met two of the three fundamental ANSI/EIA-748 

practices necessary for a reliable EVM system. In 2007, the Defense 

Contract Management Agency (DCMA) certified Ball Aerospace and 

Technology Corporation’s EVM system. Though the contractor has a 

certified system, the implementation of that system is questionable based 

on our findings below. We assessed the contractor EVM data against 

three ANSI/EIA guidelines. These guidelines state that the authorized 

work elements for the project should be defined typically using a WBS 

that has been tailored to the project and the WBS is the same for the cost 

estimate, schedule, and EVM. Our review found that the WBS in the OLI 

schedule was consistent with the WBS used for the EVM data. The

ANSI/EIA guidelines also state that projects should have a schedule that 

describes the sequence of work by listing activities in the order in which 

they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. Our analysis 

found 11 percent of the remaining activities were missing dependencies, 

13 percent had lags, and 24 percent had constraints, among other things. 

When schedules are not sequenced properly, float values and the 

calculated critical path will not be valid. Project officials said the schedule 

sequencing is driven by external forces such as facilities availability, 

spacecraft and ground system interfaces, DCMA inspections, and so 

forth. The effects of these forces on schedule sequencing and critical path 

are reviewed extensively and the validity of the critical path is not typically 

an issue for project management. Finally, the ANSI/EIA guidelines state 

that a project should establish and maintain a time-phased budget 

baseline to track cost and schedule variances in an EVM system. Though 

resource loading the schedule is not required to meet the ANSI/EIA 

guideline, it is a best practice and therefore resources should be 

accounted for in the schedule in order to develop this baseline, according

to the GAO cost guide. Our analysis found that the OLI schedule was 

resource loaded.

Operational Land 

Imager Contractor 

Has a Certified EVM 

System Compliant 

with ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 56 GAO-13-22 NASA

The project conducted an IBR in 2007. The IBR identified 80 areas of 

concern and as a result the LDCM Project did not accept the 

Performance Measurement Baseline at the IBR. However since 2007, 

Ball Aerospace and Technology Corporation has addressed the areas of 

concern.

Joint surveillance reviews are being conducted on Ball Aerospace and 

Technology Corporation’s EVM system by DCMA and the Defense 

Contract Audit Agency.

We reviewed contract performance reports from August 2010 to August 

2011. While the EVM data reflected several data anomalies, Ball 

Aerospace provided detailed explanations for each of them. For example, 

negative values were attributed to year end rate savings, labor 

corrections, or material transfers that were greater than the current month 

actual costs which resulted in a negative number.

The EVM data assessed below reflects all work associated with the 

LDCM OLI instrument. Figure 5 below illustrates that as of August 2011, 

the project was a reporting a negative cumulative cost variance of 

approximately $46 million and a negative cumulative schedule variance of 

$1.3 million.

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Being 

performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 57 GAO-13-22 NASA

Figure 5: GAO Earned Value Analysis of Operational Land Imager Data

The negative cumulative cost and schedule variances were due to 

various technical challenges experienced during instrument development, 

including detector fabrication issues, coatings issues that necessitated 

the build of a second calibration subassembly, and instrument integration 

onto the baseplate taking two weeks longer than expected to complete 

fabrication. Due to the negative variances, we are forecasting a negative 

variance at completion ranging from $49 million to $52 million. According 

to a project official, the project is not overrunning its agency baseline 

commitment, because the EVM baseline does not include unallocated 

future expenses held by the project or NASA headquarters. The 

developmental part of the OLI instrument contract with Ball Aerospace 

and Technology Corporation was completed with delivery of the 

instrument in early October 2011.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 58 GAO-13-22 NASA

The Lunar Atmosphere and Dust Environment Explorer (LADEE) will 

determine the global density, composition, and time variability of the lunar 

atmosphere. LADEE’s measurements will determine the size, charge, and 

spatial distribution of electrostatically transported dust grains. Additionally, 

it will carry an optical laser communications demonstrator that will test 

high-bandwidth communication from lunar orbit.7 Finally, it will broaden 

the scientific understanding of other planetary bodies regarding 

exospheres or very thin atmospheres, like the moon.

Table 8: Assessment of LADEE EVM Practices

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI/EIA standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion 

Data resulting 

from the EVM 

system are 

reliable

GAO variance at 

completion

Project   ◐ $140 ◐ -$0.1 to -$13

Source: GAO analysis of NASA data as of June 2011.

 

7

The Lunar Laser Communications Demonstration instrument is being funded by NASA’s 

Human Exploration and Operations Mission Directorate at a cost of approximately $65 

million. This cost is not included in the $262.9 million LADEE cost baseline.

Lunar Atmosphere 

and Dust 

Environment 

Explorer
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 59 GAO-13-22 NASA

LADEE met only one of three key fundamental ANSI/EIA-748 practices 

for reliable EVM system. NASA’s EVM guidance says that projects must 

start reporting EVM data once the project enters the project

implementation phase, if the project’s life-cycle cost is at or greater than 

$20 million. While LADEE may not have a contract that exceeds $20 

million, the overall LADEE project cost is about $262.9 million. 

Nevertheless, project officials said LADEE is responsible only for 

gathering “EVM-like” data at the project level. The “EVM-like” data is 

collected using the “EVM Lite” process, which attempts to meet the 

ANSI/EIA-748 standard where applicable. Project officials said that when 

LADEE was initiated in February 2008, NPR 7120.5D was still in effect 

which required application of EVM principles. This is, in effect, “EVM-Lite” 

or “EVM-Like.” Officials further stated that prior to August 2011, the 

LADEE project evaluated candidate EVM techniques using sample data 

from January to March 2011. Based on that evaluation period, LADEE 

decided to use the “EVM Lite” technique to collect the necessary data to 

manage the project. From April to June 2011, additional evaluations of 

this technique continued. Therefore, standard reporting of the LADEE 

“EVM-like” project level data did not begin until August 1, 2011. We 

reviewed all available EVM data from September 2010 to June 2011, 

which was before the standard EVM reporting period began.

Though LADEE does not have a certified system, we assessed how well 

LADEE project was meeting three ANSI/EIA guidelines. These guidelines 

state that the authorized work elements for the project should be defined 

typically using a WBS tailored to the project and the WBS should be the 

same for the schedule, cost estimate, and EVM. Our analysis found that 

the WBS in the LADEE schedule matched the WBS used for EVM data. 

The ANSI/EIA guidelines also states that projects should have a schedule 

that describes the sequence of work by listing activities in the order in 

which they are to be carried out and identifies significant task 

interdependencies required to meet project requirements. Our review of 

the LADEE schedule found 3 percent of the remaining activities were 

missing predecessor or successor links, which cause the schedule to not 

properly update in response to changes. We also found that about 6 

percent of the remaining activities had date constraints and/or lags, which 

also hinder the schedule from responding dynamically to changes and 

can portray an artificial or unrealistic view of the project plan. While these 

issues may be relatively small, any missing dependencies, constraints, 

and lags may disrupt the reliability of the overall network. Finally, the 

ANSI/EIA guidelines state that a project should establish and maintain a 

time-phased budget baseline to track cost and schedule variances in an 

EVM system. Though resource loading the schedule is not required to 

Project Office Does 

not Have a Certified 

EVM System 

Compliant with the 

ANSI/EIA Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 60 GAO-13-22 NASA

meet the ANSI guidelines, it is a best practice and therefore resources 

should be accounted for in the schedule in order to develop this baseline, 

according to the GAO cost guide. We found that the LADEE schedule 

was not resource-loaded, therefore, it is GAO’s assessment that the 

project did not show evidence that it had established and maintained a 

time-phased budget baseline.

LADEE conducted an IBR in December 2010. From that review, a list of 

concerns about the EVM data and schedule were identified. Specifically, 

the project was not using a consistent approach to collect EVM data, 

calling into question the credibility of the data. In addition, it was unclear if 

an objective assessment of cost and schedule performance could be 

made using data generated by the “EVM Lite” approach. Since that 

review, officials said all issues and actions have been addressed and 

were formally closed by the IBR panel in July 2011.

While formal surveillance is not occurring, EVM data assurance reviews 

are being performed during monthly management reviews with the Lunar 

Quest Program office.8

We reviewed contract performance reports from September 2010 to June 

2011. Our review of LADEE’s EVM data found various data anomalies 

that call into question the reliability of the data. For example, from 

October 2010 to December 2010 there were some instances of negative 

values in the EVM reports that were unexplained. Since a variance 

analysis report provides a detailed narrative explaining significant cost 

and schedule variances, when this information is missing, management 

cannot understand the reasons for variances and the plan for fixing them. 

Also, the EVM data provided by the project office was not presented in a 

standard EVM format. This could be attributed to the fact that LADEE is 

required to provide only “EVM-like” data.

 

8

The Lunar Quest Program consists of flight missions and instruments for lunar missions 

of opportunity, as well as research and analysis efforts.

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Not 

Being Performed

Data Resulting From the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 61 GAO-13-22 NASA

Figure 6 below illustrates that as of June 2011, the LADEE project was 

reporting a positive cumulative cost variance of $3 million and a negative 

cumulative schedule variance of $10 million.

Figure 6: GAO Earned Value Analysis of LADEE Project Data

LADEE project officials provided no information regarding positive cost 

and negative schedule variance drivers. As such, we have no insight into 

what could be causing deviations from the plan. Based on the positive 

cost variance and negative schedule variance thus far, we are forecasting 

a negative variance at completion ranging from $0.1 million to $13 million 

dollars. According to NASA, the project is not overrunning its commitment 

because the EVM baseline does not include unallocated future expenses 

held at the project and NASA headquarters level.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 62 GAO-13-22 NASA

The Magnetospheric Multiscale (MMS) is made up of four identically 

instrumented spacecraft. The mission will use the Earth’s magnetosphere 

as a laboratory to study the microphysics of magnetic reconnection, 

energetic particle acceleration, and turbulence. Magnetic reconnection is 

the primary process by which energy is transferred from solar wind to 

Earth’s magnetosphere and is the physical process determining the size 

of a space weather storm. The four spacecraft will fly in a pyramid 

formation, adjustable over a range of 10 to 400 kilometers. The data from 

MMS will be used as a basis for predictive models of space weather in 

support of exploration.

The MMS spacecraft is being designed, developed, and tested in-house 

at Goddard Spaceflight Center (GSFC) while instrument development 

activities are under contract with Southwest Research Institute (SwRI). 

The Mission Operations Center and the Flight Dynamics Operations Area 

will be developed and operated at GSFC. The Science Operations Center 

for the instruments will be developed and operated at the Laboratory for 

Atmospheric and Space Physics at the University of Colorado and is 

under contract to SwRI. The MMS project office is collecting EVM data 

both at the project level as well as from SwRI, which is responsible for the 

entire instrument suite. Therefore, the SwRI Instrument Suite effort is a 

subset of the overall MMS project level EVM report. 

Table 9: Assessment of MMS EVM Practices

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI/EIA 

standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion 

Data resulting 

from the EVM 

system are 

reliable

GAO estimate at 

completion

Project Level   ◐ $1,037 ◐ -$47 to -$80

Instrument

Suite   ◐ $233a

◐ -$10 to -$24

Source: GAO analysis of NASA EVM data as of July 2011 and contractor EVM data as of August 2011.

a

The SwRI budget at completion is only for Phases B-D. 

Magnetospheric 

Multiscale
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 63 GAO-13-22 NASA

At the project level, MMS met one of three fundamental ANSI/EIA-748 

practices for a reliable EVM system. MMS does not have a certified EVM 

system that complies with the ANSI/EIA-748 standard. NASA project 

officials said in-house projects are required only to be ANSI/EIA 

compliant, and are not required to have a certified system. Although the 

MMS project does not have a certified system, we assessed how well the 

MMS project was meeting three ANSI/EIA guidelines. These guidelines 

state that the authorized work elements for the project should be defined 

typically using a WBS that has been tailored to the project and the WBS 

is the same for the cost estimate, schedule, and EVM. Our review found 

that the WBS in the MMS schedule did not match the WBS used for the 

EVM data, which is not in line with best practices. According to project 

officials, the MMS project was started before the requirements for earned 

value management were developed. As a result, the schedule and WBS 

were created without significant consideration of a one-to-one correlation 

between the two. A project official stated that MMS has retrofitted its EVM 

system to provide as close a correlation as possible without having to 

rebuild the WBS.

The ANSI/EIA guidelines also state that projects should have a schedule 

that describes the sequence of work by listing activities in the order in 

which they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. Our review of 

the MMS schedule found some sequencing issues within the schedule. 

For example, 9 percent of predecessor and successor tasks were not 

linked to one another, which is necessary for properly sequencing work 

so that the schedule will update properly once changes are made. This 

number accounts for removing all external tasks and level of effort (LOE) 

type activities. We also found that 21 percent of the remaining activities 

were constrained. This number also does not include LOE type activities. 

In fact, the majority of these constraints were hard constraints. Hard 

constraints can sometimes be impossible to meet, given the network 

characteristics, and can thereby result in schedules that are logically 

impossible to carry out. The presence of constraints also impacts the 

schedule’s ability to respond dynamically to changes and may portray an 

unrealistic view of the project plan. As a result, these sequencing issues 

and date constraints within the schedule affect the reliability of the overall 

network and the schedule’s ability to correctly calculate float values and 

the critical path. Finally, the ANSI/EIA guidelines state that a project 

should establish and maintain a time-phased budget baseline to track 

cost and schedule variances in an EVM system. Though resource loading 

the schedule is not required to meet the ANSI/EIA guideline, it is a best 

practice and therefore resources should be accounted for in the schedule 

Project Office Does 

Not Have a Certified 

EVM System 

Compliant with the 

ANSI/EIA Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 64 GAO-13-22 NASA

in order to develop this baseline, according to the GAO cost guide. We 

found that that the MMS schedule was resource loaded.

The MMS project office conducted an IBR in June 2010. This resulted in 

positive comments, general observations, and constructive 

recommendations by the review team.

While formal surveillance is not occurring at the project level, EVM data 

assurance reviews are being performed by the Explorers and 

Heliophysics Program Office, GSFC Flight Projects Directorate 

managers, GSFC Chief Financial Officer’s Office, Standing Review 

Board, and NASA headquarters.

We reviewed contract performance reports from August 2010 to July 

2011. Our review of the MMS project level EVM data found some minor 

data reliability issues. The 12 months of project level EVM data reflect all 

of the performed in-house effort at GSFC for the project and the 

instrument suite contractor’s summary level effort. Also, the data provided 

was not reported in the standard contract performance report format. 

Project officials said a defined contract performance report was not 

dictated to the MMS project, but that all reporting has the same 

information as a “standard” contract performance report even though the 

formatting may be different. However, beginning in October 2011, MMS 

began reporting with the standard format 1 contract performance report, 

which provides cost and schedule data for each element in the project’s 

product-oriented WBS.

We tried to map the EVM data in the lower level report for the instrument 

suite to the MMS project level report and we were not able to see how the 

costs tracked from one report to another. For example, the project level 

EVM data showed that the instrument suite contractor’s July 2011 budget 

at completion was $296 million, whereas the lower level EVM data in the 

instrument suite contractor’s report showed the budget at completion to 

be $217 million. The MMS project was able to demonstrate how the SwRI 

budget at completion in the lower-level report mapped to the SwRI budget 

at completion in the MMS project report. However, officials said because 

of the way the contractor submits their data the two reports will never 

match. Though we acknowledge that the project was able to explain how 

the data tracked, attempting to manually resolve incompatible pieces of 

data can become time-consuming, expensive and can lead to data 

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Not 

being Performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 65 GAO-13-22 NASA

reliability issues. Project officials said that MMS is working to capture data 

at lower WBS levels, which will allow for a closer tie between the cost and 

schedule data. In addition, MMS said that in addition to receiving the 

SwRI EVM reports, the project now internally calculates earned value 

metrics on the contractor provided instrument suite EVM data, which 

gives MMS completely internally derived earned value performance 

reports, based on the project team’s assessment without bias from 

contractor data.

Figure 7 below illustrates that as of July 2011, the project was reporting a 

negative cumulative cost variance of $18 million and a negative 

cumulative schedule variance of $25 million.

Figure 7: GAO Earned Value Analysis of MMS Project Data

In October 2010, the project was in the midst of a replan and not all data 

was available at the time of report submission to generate detailed 

variance explanations. The replan was conducted so that the earned 

value baseline was the same as the cost plan required by the agency for 

monthly plan versus actual reporting. The goal of the replan was to 

prevent the project from having to report variances against two different 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 66 GAO-13-22 NASA

plans. Since the replan, however, the project has experienced a 

downward trend in both cost and schedule performance. As a result, we 

are forecasting a negative variance at completion ranging from $47 

million to $80 million dollars. NASA stated that the project is not 

overrunning its approved baseline commitment, because the EVM 

baseline does not include unallocated future expenses held at the project 

and headquarters level.

SwRI, the contractor responsible for the entire instrument suite for MMS, 

met one of the three fundamental ANSI/EIA-748 practices for a reliable 

EVM system. SwRI does not have a certified EVM system that complies 

with the ANSI/EIA-748 standard. According to a project official, the SwRI 

contract does not require SwRI to have a certified system but only to be 

compliant with the ANSI/EIA-748. NASA convened an independent team 

to review the contractor’s readiness for EVM system certification and 

concluded that while the contractor has qualified people to support 

implementation of EVM, a single point of failure exists without a 

documented process. Not documenting the process is a problem because 

if the people who know the process leave, new staff will not know what to 

do. In addition, the team found that even though the right software tools 

are in place to support EVM, more integration is needed to reduce 

manual inputs. Finally, the team reported that compliance with the 

ANSI/EIA-748 standard would not be achievable without management 

support and resources. Despite these findings, the project office believed 

that contractor’s EVM data is useful in examining trends and overall 

performance of the instrument suite effort.

Though SwRI does not have a certified system, we assessed how well 

the contractor was meeting three ANSI/EIA guidelines. These guidelines 

state that the authorized work elements for the project should be defined 

typically using a WBS that has been tailored to the project and the WBS 

is the same for the cost estimate, schedule, and EVM. Our review found 

that the WBS used in the schedule was not consistent with the WBS used 

for the EVM data. The ANSI/EIA guidelines also state that projects should 

have a schedule that describes the sequence of work by listing activities 

in the order in which they are to be carried out and identifying significant 

task interdependencies required to meet project requirements. Our review 

of the schedule found some sequencing issues. For example, the 

Primavera schedule provided showed 31 percent of the remaining 

activities were missing dependencies and 36 percent were constrained. 

Officials said majority of the constrained tasks were due to external 

dependencies. Due to the major sequencing issues in the MMS SwRI 

Instrument Suite 

Contractor Does not 

Have a Certified EVM 

System Compliant 

with ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 67 GAO-13-22 NASA

schedule, we question the reliability of the overall network and the 

schedule’s ability to correctly calculate float values and the critical path. 

MMS project officials said they believed many of the constrained activities 

are not valid because they reside in another schedule. Also, some 

constraints, in the Harness area for example, if removed, have no effect 

on the overall schedule. In addition, officials also said some of the 

sequencing issues may be caused by the manual integration because 

since some instrument provider schedules are in Microsoft Project and 

others are in Primavera, and therefore it is not possible to ensure all tasks 

have been linked properly. However, the MMS project scheduler tests the 

schedule for missing dependencies, logic errors, and reasonable 

durations and the results are shared with the project office and the 

contractor so that appropriate action can be taken. Lastly, officials said 

several of the activities identified in our analysis are not really schedule 

items but level of effort type activities. When we removed the 15 LOE 

type activities from the missing dependencies count, the schedule still 

showed 28 percent of the remaining activities were missing 

dependencies. When we removed the 14 LOE type activities and the 4 

Harness activities from the constraint count, the schedule still showed 33 

percent of remaining activities were constrained. Because the schedule is 

the foundation for the EVM baseline, we question the reliability of the 

Instrument Suite EVM data. Finally, the ANSI/EIA guidelines state that a 

project should establish and maintain a time-phased budget baseline to 

track cost and schedule variances in an EVM system. Though resource 

loading the schedule is not required to meet the ANSI/EIA guidelines, it is 

a best practice and therefore resources should be accounted for in the 

schedule in order to develop this baseline, according to the GAO cost 

guide. We found that the MMS schedule was resource-loaded.

The MMS project conducted an IBR of the MMS instrument suite effort in 

January 2010 and all action items have been closed out.

While the instrument suite contractor does not have a formal surveillance 

program, the MMS project office has an EVM analyst, schedule team, 

resources team, instrument management team, and project management 

team who all review the instrument suite EVM data on a monthly basis. In 

addition, the Solar Terrestrial Probes project office, Science Mission 

Directorate management, and The Aerospace Corporation review the 

instrument suite EVM data monthly.

Instrument Suite 

Contractor Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Not 

Being Performed
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 68 GAO-13-22 NASA

We reviewed contract performance reports from August 2010 to August 

2011. Our review of the instrument suite EVM data found various data 

anomalies that call into question the reliability of the data. For example, 

there were negative numbers reported for EVM data in four of the months 

that we reviewed. For some of the negative numbers, there was no 

explanation for the cause. For others, the negative values were due to 

correcting several months of translation errors brought on by a known 

issue with importing data from the schedule into the EVM system 

software. There were also errors such as inflated EVM data that once 

corrected, resulted in negative values. Figure 8 below illustrates that as 

August 2011, the project was reporting a negative cumulative cost 

variance of 

$4 million and negative cumulative schedule variance of $6 million.

Figure 8: GAO Earned Value Analysis of Instrument Suite Data

The cost variance in August 2011 dramatically improved from the 

downward trend during the previous months due to the project applying 

almost $13 million from its management reserve to the instrument suite 

contract. However, due to the negative cumulative cost and schedule 

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 69 GAO-13-22 NASA

variances, we are forecasting a negative variance at completion ranging 

from $10 million to $24 million.

The Mars Atmosphere and Volatile EvolutioN (MAVEN) mission is part of 

NASA’s Mars Scout program, a robotic orbiter mission that will provide a 

comprehensive picture of the Mars upper atmosphere, ionosphere, solar 

energetic drivers, and atmospheric losses. Set to launch in 2013, MAVEN 

will deliver comprehensive answers to long-standing questions regarding 

the loss of Mars’ atmosphere, climate history, liquid water, and 

habitability. MAVEN will provide the first direct measurements ever taken 

to address key scientific questions about Mars’ evolution.

Lockheed Martin is building the MAVEN spacecraft and will carry out 

mission operations for MAVEN. NASA’s Jet Propulsion Laboratory will 

navigate the spacecraft. The Laboratory for Atmospheric and Space 

Physics (LASP) at the University of Colorado at Boulder will coordinate 

the science team and science operations and lead the education and 

public outreach activities. NASA’s Goddard Spaceflight Center will 

provide management and technical oversight for the mission and will also 

provide mission systems engineering, mission design, and safety and 

mission assurance.

The MAVEN project office is using EVM at the project level as well as 

collecting EVM data from Lockheed Martin, the spacecraft contractor, and 

LASP, which is responsible for the Science Operations Center, Remote 

Sensing, and Langmuir Probe and Waves instrument efforts. Both the 

Lockheed Martin and LASP contracts are subsets of the overall MAVEN 

Project EVM report.

Table 10: Assessment of MAVEN EVM Practices

Dollars in millions

Used a certified EVM 

system compliant 

with ANSI standard

Conducted an 

integrated 

baseline 

review

EVM system

surveillance is 

being 

performed

Budget at 

completion 

Data resulting from 

the EVM system 

are reliable

GAO 

variance at 

completion 

Project level   ◐ $259  NAa

Spacecraft    $169  $1 to $14

Mars Atmosphere and 

Volatile Environment
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 70 GAO-13-22 NASA

Dollars in millions

Used a certified EVM 

system compliant 

with ANSI standard

Conducted an 

integrated 

baseline 

review

EVM system

surveillance is 

being 

performed

Budget at 

completion 

Data resulting from 

the EVM system 

are reliable

GAO 

variance at 

completion 

Science 

Operations 

Centerb

—  ◐ $3  $0 to $0.4

Remote sensing 

Packageb —  ◐ $20  $1 to $7

Langmuir Probe 

and Waves 

Instrumentb

—  ◐ $5  -$2 to -$3

Source: GAO Analysis of NASA EVM data as of July 2011 and contractor EVM data as of August 2011.

a

NA - Not applicable means there was not enough data to complete a forecast. 

b

A certified EVM system is not required due to the contract value. 

MAVEN fully met one of the three key practices for implementing EVM at 

the project level. Specifically, the project did not have a certified EVM 

system and is not required to have a certified system. Nevertheless, we 

assessed how well the MAVEN Project was meeting three ANSI/EIA 

guidelines. As part of our analysis, we assessed MAVEN’s EVM data 

against three ANSI/EIA guidelines. These guidelines state that the 

authorized work elements for the project should be defined typically using 

a WBS that has been tailored to the project and that the WBS should be 

the same for the cost estimate, schedule, and EVM. We found that the 

project’s WBS was consistent between the schedule and EVM data.

The ANSI/EIA guidelines also state that projects should have a schedule 

that describes the sequence of work by listing activities in the order in 

which they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. Our review 

found some sequencing issues in the schedule. For example, 5 percent of 

the activities were missing dependencies, 6 percent had open ended logic 

links, and 20 percent had constraints, among other things. When 

schedules are not sequenced properly, float values and the calculated 

critical path will not be valid. In addition, the project conducted an 

integrated baseline review in July 2011. The MAVEN project office 

provided a May 2012 Schedule Health Check Report that showed the 

number of missing dependencies and constraints had been reduced 

considerably. Though we cannot validate the improvement in the 

schedule without performing our own assessment, we believe that 

MAVEN is working towards producing a more reliable schedule. This 

Project Does Not 

Have a Certified EVM 

System Compliant 

with ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 71 GAO-13-22 NASA

review found that the project schedule and technical design were in good 

shape, but noted concerns that more resources were needed to 

implement and maintain EVM, there were cost and schedule integration 

issues that caused the budgets for some work packages to not be in sync 

with the schedule, and reliable critical path analysis was at risk because 

of missing schedule links and constraints. Finally, a project should 

establish and maintain a time-phased budget baseline at the control 

account level, against which performance can be measured. Though 

resource loading the schedule is not required to meet the ANSI/EIA 

guidelines, it is a best practice and therefore resources should be 

accounted for in the schedule in order to develop this baseline, according 

to the GAO cost guide. We found that the schedule was resource-loaded.

Project conducted an IBR in July 2011 and all nine areas of concern were 

addressed and closed.

While formal surveillance is not occurring at the project level, EVM data 

assurance reviews are being performed by the Mars Program Office 

representatives, MAVEN standing Review Board representatives, and 

Aerospace Corporation representatives at both the project level and for 

LASP efforts.

We reviewed contract performance reports from June 2011 to August 

2011. EVM data prior to spring 2011 was not available because MAVEN 

had not been confirmed into the implementation phase. Our review of the 

MAVEN project EVM data found that there was a mistake causing the 

costs to be overstated by twice their actual amount. Other than this 

mistake, which affected most of the data in the report, no other errors 

were found. The project EVM data reflects all work on this project 

including effort related to the project office, two in-house instruments, and 

the Space Sciences Laboratory, as well as major efforts from Lockheed 

Martin and LASP. Consolidated reporting of all components at a summary 

level began with the May 2011 data, with the first full summary report 

delivered on July 15, 2011. Therefore, we had only 3 months of data to 

review. Figure 9 below illustrates that as of August 2011, the project was 

reporting a positive cumulative cost variance of $5 million while also 

experiencing a negative cumulative schedule variance of $5 million.

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Not 

Being Performed

Data Resulting from the 

EVM System Is Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 72 GAO-13-22 NASA

Figure 9: GAO Earned Value Analysis of MAVEN Project Data

The positive cumulative cost variance was due to a decrease in labor 

charges, delayed material costs, a reduction in re-work, and the 

leveraging of common engineering products from other projects. The 

negative schedule variance was being driven by the Neutral Gas and Ion 

Mass Spectrometer instrument, which experienced technical issues such 

as vendor machines not being manufactured to specifications. Because 

we had only 3 months of data, we did not forecast an estimate at 

completion.

In addition, we tried to map the EVM data in the lower level reports for 

spacecraft, Science Operations Center, Remote Sensing, and Langmuir 

Probe and Waves efforts to the overall MAVEN project EVM report and in 

some cases we were not able to see how the costs tracked from one 

report to another. For example, while we could easily trace the costs for 

the Science Operations Center effort from the lower level EVM report to 

the overall MAVEN project report, we could not clearly map the costs for 

the spacecraft, Remote Sensing or Langmuir Probe and Waves efforts. In 

particular, for Remote Sensing and Langmuir Probe and Waves efforts, 

the lower level EVM report cost elements did not have their costs 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 73 GAO-13-22 NASA

burdened at the WBS level, which could account for some of the 

differences between the lower level report costs and the overall MAVEN 

project costs for those elements. MAVEN project officials walked us 

through their process of how they ensure lower-level reports map to the 

project level reports. In addition, MAVEN project provided supporting 

documentation that validated this assertion. Though MAVEN project 

officials helped explain the mapping, officials said they do not mandate 

that their contractors follow a certain reporting format, instead any 

adjustments necessary to ensure that the lower-level reports map to the 

project-level reports are made manually by the project office. Though 

MAVEN project does not prescribe to a standard reporting format,

attempting to manually resolve incompatible pieces of data can become 

time-consuming, expensive and can lead to data reliability issues. 

Although the agency provided explanations for the mapping issues, the 

ability to reconcile the costs between the reports should be evident, 

without additional explanations.

Lockheed Martin, the spacecraft contractor, met the three fundamental 

ANSI/EIA practices necessary for a reliable EVM system. In August 2008, 

the Defense Contract Management Agency (DCMA) certified that 

Lockheed Martin’s EVM system is compliant with the ANSI/EIA standard. 

However, the implementation of that EVM system is questionable based 

on our findings. We assessed the contractor’s data against the three 

ANSI/EIA guidelines. These guidelines state that the authorized work 

elements for the project should be defined typically using a WBS that has 

been tailored to the project and the WBS is the same for the cost 

estimate, schedule, and EVM. We found that the project’s WBS in the 

schedule was consistent with the WBS used in the EVM data. However, 

we found issues with the schedule. The ANSI/EIA guidelines also state 

that projects should have a schedule that describes the sequence of work 

by listing activities in the order in which they are to be carried out and 

identifying significant task interdependencies required to meet project 

requirements. Our review found some sequencing issues in the schedule. 

For example, 2 percent of the activities remaining were missing 

predecessor and successor links, which are necessary for properly 

sequencing work so that the schedule will update in response to changes. 

We also found that 37 percent of the remaining activities had constraints, 

which also hinder the schedule’s ability to respond dynamically to status 

updates resulting in an artificial or unrealistic view of the project plan. 

These issues with the schedule affected the reliability of the overall 

network and the schedule’s ability to correctly calculate float values and 

the critical path. Project officials further explained that some of the “start 

Lockheed Martin Has 

a Certified EVM 

System Compliant 

with the ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 74 GAO-13-22 NASA

no earlier than” constraints were due to resource availability and “finish no 

later than” constraints were used intentionally to plan task activities to 

occur as late as possible. Finally, the ANSI/EIA guidelines state that a 

project should establish and maintain a time-phased budget baseline to 

track cost and schedule variances in an EVM system. Though resource 

loading the schedule is not required to meet the ANSI/EIA guidelines, it is 

a best practice and therefore resources should be accounted for in the 

schedule in order to develop this baseline, according to the GAO cost 

guide. We found that the contractor schedule was resource loaded.

The project conducted the spacecraft’s IBR in April 2011. During the 

review, several areas of concern regarding the schedule were identified. 

The project office said that during the integrated baseline review, the 

review team identified many of the same observations with the schedule 

as our findings. As a result, the project office directed the contractor to 

eliminate the constraints, lags, and missing logic links in their integrated 

master schedule. Since the November 2011 schedule submittal, the 

contractor has decreased the number of sequencing issues in the 

schedule, according to project officials. The project office also said that 

the contractor continues to conduct schedule health checks to uncover 

any sequencing issues and provide the project office with schedule 

variance reports and critical path analysis, which are discussed during 

monthly management meetings.

Joint surveillance reviews of the EVM data are being performed by 

DCMA, MAVEN project officials, and the contractor.

We reviewed contract management reports from January 2011 to August 

2011. Our review of the EVM data found no major issues with data 

reliability. However, from January to May 2011, there were no variance 

analysis reports produced to explain significant cost and schedule 

variances and other contract problems and topics because they did not 

meet reporting thresholds. Without this information, however, 

management cannot understand the reasons for variances and the 

contractor’s plan for fixing them.

Figure 10 below illustrates that as of August 2011, the project was 

reporting a positive cumulative cost variance of $6 million and a negative 

cumulative schedule variance of $3 million.

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Being 

Performed

Data Resulting from the 

EVM System Is Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 75 GAO-13-22 NASA

Figure 10: GAO Earned Value Analysis of Spacecraft Data

One reason for the positive cost variance was due to the ability to 

leverage a lower subcontractor rate than planned, which resulted in a cost 

savings. The negative schedule variance was also being driven by the 

mechanism subsystem falling behind schedule due to the shop being 

overloaded with work and the mechanism designers supporting other 

NASA efforts, among other things. Due to the positive cost variance, we 

are forecasting a positive variance at completion ranging from $1 million 

to $14 million.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 76 GAO-13-22 NASA

LASP, the contractor responsible for the Science Operations Center, 

Remote Sensing package, and Langmuir Probe and Waves instrument 

efforts met one of three fundamental ANSI/EIA-748 practices necessary 

for a reliable EVM system. To date, LASP does not have a certified EVM 

system. Though the contractor is not required to have a certified system, 

we assessed how well the contractor was meeting three ANSI/EIA 

guidelines. These guidelines state that the authorized work elements for 

the project should be defined typically using a WBS that has been tailored 

for effective internal management control of the project and the WBS is 

the same for the cost estimate, schedule, and EVM. We found some 

slight inconsistencies in the WBS numbering between the EVM report and 

the schedule for the Remote Sensing package, which calls into question 

the reliability of their overall schedule network. Moreover, the Langmuir 

Probe and Waves Instrument had issues with consistency between the 

WBS and the schedule. In particular, there was varying levels of 

information between the two WBSs, making it hard to use the WBS as a 

common thread between the EVM data and the schedule. Since the WBS 

is a critical component of EVM, it should be the same for developing the 

EVM performance measurement baseline and the schedule. Without a 

common link between these two features, project managers cannot fully 

understand project cost and schedule variances.

The ANSI/EIA guidelines also state that projects should have a schedule 

that describes the sequence of work by listing activities in the order in 

which they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. We found some 

sequencing issues with the Langmuir Probe and Waves Instrument 

schedule. For example, 47 percent of the remaining activities had

constraints, which defeated the purpose of a using a dynamic schedule. 

The quality of the schedule was also hampered by the presence of 

schedule lags on 18 percent of the remaining activities. Schedule lags 

must be justified because they cannot be easily monitored or included in 

risk assessments. Finally, the ANSI/EIA guidelines state that a project 

should establish and maintain a time-phased budget baseline to track 

cost and schedule variances in an EVM system. Though resource loading 

the schedule is not required to meet the ANSI/EIA guidelines, it is a best 

practice and therefore resources should be accounted for in the schedule 

in order to develop this baseline, according to the GAO cost guide. We

found that all three components showed evidence of a time-phased 

budget baseline.

Laboratory for 

Atmospheric and 

Space Physics Does 

Not Have a Certified 

EVM System 

Compliant with 

ANSI/EIA Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 77 GAO-13-22 NASA

The project conducted an IBR in March 2011 and several areas of 

concern were noted. MAVEN project officials stated that many of the 

schedule issues we found were also discovered during the IBR and have 

now been corrected.

While formal surveillance is not occurring, EVM data assurance reviews 

are being performed by the, project office, Mars Program Office, and 

MAVEN Standing Review Board representatives. Also, The Aerospace 

Corporation is working as an advisor to Science Mission Directorate’s 

Planetary Systems Division.

Our review of MAVEN’s Science Operations Center, Remote Sensing 

package, and Langmuir Probe and Waves instrument EVM data found no 

major issues with data reliability. However, there was a lack of variance 

analysis reports for these efforts. For example, Science Operations 

Center had no variance analysis reports with the explanations for any of 

the months reported. In addition, the Remote Sensing package, and 

Langmuir Probe and Waves instrument variance analysis reports did not 

provide any explanation for major performance swings from one month to 

another.

Figure 11 below illustrates that as of August 2011, the Science 

Operations Center portion of the MAVEN project was reporting a positive 

cumulative cost variance of $0.01 million and a slightly negative 

cumulative schedule variance of $0.01 million.

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Not 

Being Performed

Data Resulting from the 

EVM System Is Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 78 GAO-13-22 NASA

Figure 11: GAO Earned Value Analysis of Science Operations Center Data

However, since the variance analysis reports provided no information 

regarding what is driving the positive cost and slightly negative schedule 

variances, we have no insight into the causes for deviations from the 

plan. Based on the positive cost variance thus far, we are forecasting a 

positive variance at completion of less than $0.5 million at contract 

completion.

Figure 12 below illustrates that as of August 2011 the Remote Sensing 

package portion of the MAVEN project was reporting a positive 

cumulative cost variance of $0.8 million and a slightly negative cumulative 

schedule variance of $0.6 million.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 79 GAO-13-22 NASA

Figure 12: GAO Earned Value Analysis of Remote Sensing Instrument Data

Factors behind the positive cost variance include an accounting lag on 

the invoicing and payment process associated with the procurements, 

which results in the appearance of cost efficiency. This issue has been 

occurring for several months and is causing the EVM metrics to be 

skewed to reflect false positive cost variances. The variance analysis 

reports do not give any explanation for why there is a negative schedule 

variance situation as of August 2011. As a result of the positive cost 

variance, we are forecasting a positive variance at completion ranging 

from $1 million to $7 million.

Figure 13 below illustrates that as of August 2011 the Langmuir Probe 

and Waves portion of the MAVEN project was reporting a negative 

cumulative cost variance of $0.6 million and negative cumulative 

schedule variance of $0.3 million.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 80 GAO-13-22 NASA

Figure 13: GAO Earned Value Analysis of Langmuir Probe and Waves Data

Reasons for the negative cost and schedule variances are due to costs 

for outside services and materials being more than planned as well as 

additional work required to troubleshoot problems and mitigate risks. Due 

to these problems, we are forecasting a negative variance at completion 

ranging from $2 million and $3 million. According to NASA, the project is 

not overrunning its commitment because the EVM baseline does not 

include unallocated future expenses held at the project and headquarters 

level.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 81 GAO-13-22 NASA

NASA’s Orbiting Carbon Observatory 2 (OCO-2) is designed to enable 

more reliable predictions of climate change and is based on the original 

OCO mission that failed to reach orbit in 2009. It will make precise, timedependent global measurements of atmospheric carbon dioxide. These 

measurements will be combined with data from a ground-based network 

to provide scientists with the information needed to better understand the 

processes that regulate atmospheric carbon dioxide and its role in the 

carbon cycle. NASA expects enhanced understanding of the carbon cycle 

will improve predictions of future atmospheric carbon dioxide increases 

and the potential impact on the climate.

The OCO-2 mission consists of a dedicated spacecraft with a single 

instrument, flying in a near-polar, sun-synchronous orbit. The Jet 

Propulsion Laboratory (JPL) has overall responsibility for project 

management. The OCO-2 spacecraft is being built by Orbital Sciences 

Corporation while the instrument is being built in-house at JPL. Orbital 

Sciences Corporation submits spacecraft effort EVM data monthly and 

the project incorporates that data into the overall project EVM report. The 

project is facing a launch delay because the Taurus XL launch vehicle 

failed on the Glory Mission, and the contract was terminated. The project 

will be rebaselined as a result of NASA having to select a new launch 

vehicle. The current $477.2 million total project cost is a preliminary 

amount pending the outcome of the rebaseline process.

Table 11: Assessment of OCO-2 EVM Practicesa

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being 

performed

Budget at 

completion

Data resulting 

from the EVM 

system are 

reliable

GAO variance at 

completion 

Project    $129  -$1 to -$26

Spacecraftb

  ◐ $30  NAc

Source: GAO analysis of JPL and contractor EVM data as of July 2011.

a

All the budget and variance data reported is prior to the rebaseling resulting from changing the 

launch vehicle. 

b

DCMA certified Orbital Sciences Corporation’s EVM system in January 2012. 

c

NA means that there was not enough EVM data available for us to forecast an estimate at 

completion. 

Orbiting Carbon 

Observatory 2
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 82 GAO-13-22 NASA

The OCO-2 project met all three fundamental ANSI/EIA-748 practices 

necessary for a reliable EVM system. JPL has a certified EVM system 

that complies with the ANSI/EIA standard. However, the implementation 

of its EVM system is questionable based on our findings below. We 

assessed the JPL EVM data against three ANSI/EIA guidelines. These 

guidelines state that the authorized work elements for the project should 

be defined typically using a WBS that has been tailored to the project and 

the WBS is the same for the cost estimate, schedule, and EVM. We found 

that the project had a work breakdown structure that was consistent with 

the WBS used for the EVM data. The ANSI/EIA guidelines also state that 

projects should have a schedule that describes the sequence of work by 

listing activities in the order in which they are to be carried out and 

identifying significant task interdependencies required to meet project 

requirements. Our review found some sequencing issues in the schedule. 

In particular, 14 percent of predecessor and successor tasks were not 

linked to one another, which is necessary for properly sequencing work 

so that it will update in response to changes. We also found 15 percent of 

the remaining activities were constrained, which also hinders the 

schedule from responding dynamically to changes and can portray an 

artificial or unrealistic view of the project plan. These sequencing issues 

and constraint dates within the schedule affect the reliability of the overall 

network and the schedule’s ability to correctly calculate float values and 

the critical path. Project officials said the missing dependencies are 

mainly handoffs produced by level of effort (LOE) activities. Officials said 

these activities are not necessary for valid schedule network logic, and 

under no circumstances do these activities drive the critical path. They 

also said the constrained activities are largely composed of mandated 

delivery dates, which JPL uses as control points to manage subsystem 

schedule performance prior to assembly, test, and launch operations 

delivery, and coordinate major meeting logistics. Finally, the ANSI/EIA 

guideline states that a project should establish and maintain a timephased budget baseline to track cost and schedule variances in an EVM 

system. Though resource loading the schedule is not required to meet the 

ANSI guidelines, it is a best practice and therefore resources should be

accounted for in the schedule in order to develop this baseline, according 

to the GAO cost guide. To develop the baseline, resources must be 

accounted for in the schedule and our review found that the schedule was 

resource loaded.

Project Using a 

Certified EVM System 

Compliant with the 

ANSI/EIA Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 83 GAO-13-22 NASA

The project conducted an integrated baseline review in March 2011.

JPL also has a formal surveillance plan in place for monitoring the EVM 

data. In particular, project officials said each month detailed earned value 

data analysis is performed on each cost account and each work package 

so that conclusive understanding of the performance status can be 

reached and communicated within the project team. In addition, during 

the project monthly management reviews, both cost and schedule 

variances and reasons causing them are presented.

We reviewed contractor management reports from December 2010 to 

July 2011. Though we found no major data reliability issues when we tried 

to map the EVM data in the lower level spacecraft report to the JPL 

project level report, we could not understand how the costs tracked from 

one report to another. In a subsequent interview, officials explained, with 

supporting documentation, how the lower level Orbital Sciences budget at 

completion mapped to the budget at completion found in the JPL Project 

level EVM report. Though we appreciate the explanations provided by 

project officials regarding the differences between the two reports, the 

issue remains that without additional documentation and explanations by 

project officials, GAO or another independent party could not have 

reconciled the data.

Figure 14 below illustrates that as of July 2011, the project was reporting 

a positive cumulative cost variance of $0.6 million and a negative 

schedule variance of $8 million as seen in the graph below. According to 

project officials, the schedule variance was being caused by spectrometer 

slit instability and an incompatible memory chip in the Remote Electronics 

Module.

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Being 

Performed

Data Resulting from the 

EVM System Are Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 84 GAO-13-22 NASA

Figure 14: GAO Earned Value Analysis of OCO-2 Project Data

Due to the positive cost variance and negative schedule variance we are 

forecasting a negative variance at completion ranging from $1 million to 

$26 million. Project officials said the project is not overrunning its 

commitment because the EVM baseline provided to GAO does not 

include unallocated future expenses held at the project and headquarters 

level.

The spacecraft contractor, Orbital Sciences Corporation, met two of the 

three fundamental ANSI/EIA-748 practices necessary for a reliable EVM 

System. At the time of our review, the Orbital Sciences did not have a 

certified EVM system that complied with the ANSI/EIA-748 standard. In 

January 2012, DCMA certified the Orbital Sciences EVM System. Though 

the contractor now has a certified system, the implementation of that 

system is questionable based on our findings below. We assessed how 

well the contractor’s EVM system was meeting three ANSI/EIA 

guidelines. These guidelines state that the authorized work elements for 

the project should be defined typically using a WBS that has been tailored 

to the project and the WBS is the same for the cost estimate, schedule, 

Spacecraft Contractor 

Has a Certified EVM 

System Compliant 

with ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 85 GAO-13-22 NASA

and EVM. We found that the spacecraft’s WBS in the schedule was not 

consistent with the WBS used for the EVM data. The ANSI/EIA guidelines 

also state that projects should have a schedule that describes the 

sequence of work by listing activities in the order in which they are to be 

carried out and identifying significant task interdependencies required to 

meet project requirements. We found some sequencing issues in the 

spacecraft contractor schedule. We found that 27 percent of the 

remaining activities in the spacecraft’s schedule were missing 

predecessor and successor links, which are necessary for properly 

sequencing work so that the schedule will update in response to changes. 

In addition, 24 percent of the remaining activities had date constraints, 

which also hinder the schedule’s ability to respond dynamically to status 

updates resulting in an artificial or unrealistic view of the project plan. 

These sequencing issues and constraint dates within the schedule affect 

the reliability of the overall network and the schedule’s ability to correctly 

calculate float values and the critical path.

Orbital Sciences conducted an integrated baseline review of the 

spacecraft effort in March 2011.

While formal surveillance is not occurring, project officials stated that 

EVM performance data is reviewed during the monthly status reviews.

We reviewed contract performance reports from April 2011 to July 2011. 

Because we had only 4 months of data to review, we were unable to 

forecast estimate variance at completion. As of July 2011, the project was 

reporting a positive cumulative cost variance of $0.4 million and a 

negative cumulative schedule variance of $2 million as seen in the graph 

below. The positive cost variance was being driven by lower than 

expected burden rates and less staff supporting project management, 

flight assurance, and systems engineering efforts.

Spacecraft Contractor 

Conducted an Integrated 

Baseline Review

EVM Surveillance Is Not 

Being Performed

Data Resulting from the 

EVM System Are Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 86 GAO-13-22 NASA

Figure 15: GAO Earned Value Analysis of the Spacecraft Data

The positive cost variance was being driven by lower than expected 

burden rates and less staff supporting project management, flight 

assurance, and systems engineering efforts.
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 87 GAO-13-22 NASA

The Radiation Belt Storm Probes (RBSP) mission will explore the Sun’s 

influence on the Earth and near-Earth space by studying the planet’s 

radiation belts at various scales of space and time. This insight into the 

physical dynamics of the Earth’s radiation belts will provide scientists with 

data to make predictions of changes in this little understood region of 

space. Understanding the radiation belt environment has practical 

applications in the areas of spacecraft system design, mission planning, 

spacecraft operations, and astronaut safety. The RBSP project built two 

spacecraft that will be used to measure the particles, magnetic and 

electric fields, and waves that reside in the Van Allen radiation belts. 

RBSP launched on August 30, 2012 on a two-year prime mission.

The RBSP spacecrafts and ground system are being designed, 

developed, and tested by the John Hopkins University’s Applied Physics 

Laboratory.9 The RBSP project office is collecting EVM data at the project 

level.

Table 12: Assessment of RBSP EVM Practices

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion 

Data resulting 

from the EVM 

system are 

reliable

GAO variance at 

completion 

Project   ◐ $291  -$40 to -$41

Source: GAO analysis of contractor EVM data as of April 2011.

 

9

The Applied Physics Laboratory is organized as a Limited Liability Company, wholly 

owned by Johns Hopkins University and operated as a division of the university.

Radiation Belt Storm 

Probes 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 88 GAO-13-22 NASA

At the project level, the Applied Physics Laboratory only fully met one and 

partially met another of the three fundamental practices necessary for a 

reliable EVM system. RBSP is the first full NASA mission to use EVM at 

the Applied Physics Laboratory. According to the RBSP project manager, 

the RBSP project implemented a limited earned value management 

system in Phase B as a risk mitigation activity for Phase C/D. This early 

implementation was a risk mitigation activity, which allowed the project’s 

control account manager, instrument provider, and project office to better 

understand the reporting process and the use of the EVM system. The 

use of EVM during this phase was also intended to allow for timely, 

accurate, and useful EVM reporting during the formal reporting in later 

phases of the project. Since then Applied Physics Laboratory has made 

good progress and is in the process of meeting the intent of being 

compliant with the 32 ANSI/EIA-748 guidelines.

We assessed how well the Applied Physics Laboratory was meeting the 

three ANSI/EIA guidelines. These guidelines state that the authorized 

work elements for the project should be defined typically using a WBS 

that has been tailored to the project and the WBS is the same for the cost 

estimate, schedule, and EVM. Our review, we found that that the WBS 

used in the schedule was not consistent with the WBS used for the EVM 

data. Project officials said that the project utilizes the Applied Physics 

Laboratory WBS for all earned value management activity, which is then 

mapped to the NASA WBS for reporting to the sponsor. Also, the Applied 

Physics Laboratory WBS is uniformly utilized and consistent across all 

control accounts in the EVM system. This internal WBS ties into both the 

contract performance report and the integrated master schedule utilized 

on the project.

The ANSI/EIA guidelines also state that projects should have a schedule 

that describes the sequence of work by listing activities in the order in 

which they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. Our review 

found some sequencing issues in the schedule. For example, 23 percent 

of the remaining activities were missing predecessor and successor links, 

which are necessary for properly sequencing work so that the schedule 

will update in response to changes. We also found that 29 percent of the 

remaining activities had date constraints, which also hinder the 

schedule’s ability to respond dynamically to status updates resulting in an 

artificial or unrealistic view of the project plan. When schedules are not 

sequenced properly, float values and the calculated critical path will not 

be valid. Project officials explained that sequencing issues were a result 

of the project consciously including constrained instrument deliveries and 

Project Does Not 

Have a Certified EVM 

System Compliant 

with the ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 89 GAO-13-22 NASA

deliverables, level of effort activities and material and subcontractor 

expenditures in the integrated master schedule, and though the RBSP 

integrated master schedule had these issues it was able to monitor the 

critical and near critical paths of all spacecraft systems and subsystems. 

Finally, the ANSI/EIA guidelines state that a project should establish and 

maintain a time-phased budget baseline to track cost and schedule 

variances in an EVM system. Though resource loading the schedule is 

not required to meet the ANSI/EIA guidelines, it is a best practice and 

therefore resources should be accounted for in the schedule in order to 

develop this baseline, according to the GAO cost guide. Our review found 

that the schedule was resource loaded.

NASA conducted an integrated baseline review in August 2009. Of the 

seven IBR objectives identified, two were partially met and five were met. 

In December 2010, the IBR deputy chief notified the Applied Physics 

Laboratory that all areas of concern had been closed. As a result, the 

overall consensus of the government review team was that the integrated 

baseline review was successful.

While formal surveillance is not being performed, the project office 

reviews the EVM data on a monthly basis. Although they do not perform 

formal surveillance, an Applied Physics Laboratory official said that they 

performed additional monthly independent reviews of the RBSP EVM 

system throughout Phase C/D.

We reviewed contract performance reports from August 2010 to April 

2011 and July 2011 to August 2011. We were not provided reports for 

May and June 2011. This was due to a re-plan determined necessary by 

the Living with a Star Program Office and NASA headquarters in May 

2011 due to changes in the launch manifest. Figure 16 below illustrates 

that as of August 2011, the project was reporting a negative cumulative 

cost variance of approximately $32 million and a negative cumulative 

schedule variance of $3 million. 

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Not 

Being Performed

Data Resulting from the 

EVM System Are Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 90 GAO-13-22 NASA

Figure 16: GAO Earned Value Analysis of Project Data

The negative cumulative cost variance was caused by sustained effort on 

the radio frequency communications, as well as by work on the avionics 

equipment, and ground system software launch and post launch 

components being behind schedule. The schedule variance is minimal 

since the project is nearing completion. Due to the negative cost variance 

we are forecasting a negative variance at completion from $40 million to 

$41 million. Project officials noted the forecasted variance at completion 

is below the revised contract value of $351.1 million, although above the 

project’s estimated budget at completion. In addition, officials said this is 

a project-level variance, and does not account for the application of 

unallocated future expenses to fund the movement of the launch date and 

to keep the project on track. When the project launched in August 2012, 

its estimate at completion was below the $351.1 million budget at 

completion. 
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 91 GAO-13-22 NASA

The Stratospheric Observatory for Infrared Astronomy (SOFIA) is a joint 

project between NASA and Deutsche Zentrum für Luft-und Raumfahrt 

(DLR), the German space agency, to install a 2.5 meter telescope, as well 

as other scientific instruments capable of infrared and sub-millimeter 

observations, in a specially modified Boeing 747SP aircraft. This airborne 

observatory is designed to provide routine access to the visual, infrared, 

far-infrared, and sub-millimeter parts of the electromagnetic spectrum. Its 

mission objectives include studying many different kinds of astronomical 

objects and phenomena, including star birth and death; the formation of 

new solar systems; planets, comets, and asteroids in our solar system; 

and black holes at the center of galaxies. Currently, five U.S. and two 

German funded interchangeable instruments for the observatory are 

being developed to allow a range of scientific measurement to be taken 

by SOFIA.

The SOFIA project office is using EVM at the project level as well as 

collecting EVM data from the L-3 Communications Integrated Systems 

L.P. (L-3), which is responsible for the airborne system platform effort. 

The EVM data provided to the project for the airborne observatory 

platform effort is a subset of the overall SOFIA project level EVM report. 

The German component of the SOFIA project does not generate earned 

value data and are not part of the project’s budget baseline. The 

University Space Research Association (USRA) has a support contract to 

help the Ames Research Center manage SOFIA’s science and mission 

operations in cooperation with the Deutsches SOFIA Institut. The USRA 

contract was established before NASA began requiring earned value 

management compliance. As a result, they are not required to generate 

earned value data. However, all of these components are subsets of the 

overall SOFIA project.

Table 13: Assessment of SOFIA EVM Practices

Dollars in millions

Used a certified 

EVM system 

compliant with 

ANSI standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion

Data resulting 

from the EVM 

system are 

reliable

GAO variance at 

completion

Project level   ◐ $384 ◐ -$1.4 to -$76

Airborne system    $38  $3 to $4

Source: GAO analysis of EVM data as of August 2011 for the project and July 2011 for the contractor data.

Stratospheric 

Observatory for 

Infrared Astronomy
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 92 GAO-13-22 NASA

SOFIA project did not meet any of the three fundamental ANSI/EIA-748 

practices necessary for a reliable EVM system. Project officials said inhouse projects are not required to have certified EVM systems. Though 

the SOFIA project does not have a certified system, we assessed how 

well the project was meeting three ANSI/EIA guidelines. These guidelines 

state that the authorized work elements for the project should be defined 

typically using a WBS that has been tailored to the project and the WBS 

is the same for the cost estimate, schedule, and EVM. We found that the 

WBS used in the SOFIA schedule was consistent with the WBS used for 

the EVM data. The ANSI/EIA guidelines also state that projects should 

have a schedule that describes the sequence of work by listing activities 

in the order in which they are to be carried out and identifying significant 

task interdependencies required to meet project requirements. Our review 

found some sequencing issues in the schedule. For example, 16 percent 

of the remaining activities had open ended logic links, and 1 percent had 

date constraints, among other things. SOFIA project officials explained 

that the majority of the missing links were due to using hammock 

activities to resource load the schedule, which resulted in missing 

successors.10 Out of all of the activities missing successors, only 11 were 

not hammocked activities, which accounted for a very small amount. In 

addition, SOFIA project officials said that constraints in the schedule were 

justified since they represented external deliveries or fiscal year funding 

availability. They added that the one hard constraint, Must Start On, was 

used to represent a fixed date for an international visit. While the majority 

of these explanations seem reasonable, when schedules are not 

sequenced properly, the slack values and the calculated critical path will 

not be valid.

An IBR was not conducted at the project level. Project officials said 

SOFIA, an in-house project, did not begin collecting EVM data until very 

late in the development phase, and because of this an IBR was not 

conducted. Officials added although SOFIA did not conduct a projectlevel IBR, the EVM baseline was established concurrently with an Agency 

approved re-plan and joint confidence level analysis in the 2009/2010 

time frame, and was reviewed by a Standing Review Board. Though the 

 

10A hammock activity is a way to represent level of effort activities in the schedule. 

Having no set duration of its own, the hammock activity duration is determined by the 

number of days between the beginning of the first activity and the end of the last activity in 

the group-not the sum of the activities' durations.

Project does not have 

a Certified EVM 

System Compliant 

with the ANSI/EIA 

Standard

Project Did Not Conducted 

an Integrated Baseline 

Review
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 93 GAO-13-22 NASA

Standing Review Board review satisfied some of the objectives of an IBR, 

including confirmation of the schedule and budget baselines (e.g., 

schedule review, risk review, key milestones identified), it did not address 

the full IBR checklist (e.g. work authorizations, control account plans, 

EVM system description).

While formal surveillance is not occurring at the project level, EVM data 

assurance reviews are being performed monthly and quarterly by the 

SOFIA project office representatives.

We reviewed contract performance reports from August 2010 to August 

2011. We found various issues that bring into question the reliability of the 

SOFIA Project EVM data. For example, we found negative values due to 

an “over-reporting” of progress in previous months that was caused by a 

problem with translating percent complete progress from the University 

Space Research Association schedule to the SOFIA integrated master 

schedule as well as corrections/modifications in costs posted for support 

service contracts. In addition, we found other anomalies in the data that 

SOFIA project officials explained were most likely due to mischarges by 

employees, delayed cost postings, or employees continuing to use 

charge codes inappropriately. While the cost impact of these problems 

was not that large for any one WBS element, each of these issues causes 

us to question the reliability of the data. According to project officials, the 

variances that caused these anomalies did not meet the reporting 

threshold. A variance analysis report provides a detailed, narrative report 

explaining significant cost and schedule variances and other contract 

problems and topics. Without this information, management cannot 

understand the reasons for variances and the contractor’s plan for fixing 

them. When variance analysis reports are not produced, the EVM data 

will not be meaningful or useful as a management tool.

Figure 17 below illustrates that as of August 2011, the SOFIA project was 

reporting a positive cumulative cost variance of $3.5 million and a 

negative cumulative schedule variance of $6.6 million.

EVM Surveillance Is Not 

being Performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 94 GAO-13-22 NASA

Figure 17: GAO Earned Value Analysis of the SOFIA Project Data

Due to the cost and schedule variances we are forecasting a negative 

variance at completion ranging from $1.4 million to $76 million.

The contractor met all three fundamental ANSI/EIA-748 practices 

necessary for a reliable EVM system. In 2002, DCMA certified that the 

contractor has an EVM system compliant with the ANSI/EIA standard.

The project conducted an IBR in November 2007.

Airborne System 

Platform Contractor 

Has a Certified EVM 

System Compliant 

with the ANSI/EIA 

Standard

Project Conducted an 

Integrated Baseline 

Review of Airborne System 

Platform Effort
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 95 GAO-13-22 NASA

It is unclear if EVM surveillance is being performed on the L-3 EVM data.

We reviewed contract performance reports from August 2010 to July 

2011. We found various data anomalies that call into question the 

reliability of the contractor’s EVM data for the contract performance 

reports we reviewed. For example, we found negative values, actual cost 

of work performed being reported without work being scheduled and/or 

performed. Project officials stated that these anomalies were caused by 

performance being taken in later months and also because these issues 

did not trip the threshold reporting.

Figure 18 below illustrates that as of July 2011, the airborne system 

platform contractor was reporting a positive cumulative cost variance of 

$3.2 million and a negative schedule variance of $0.6 million.

Figure 18: GAO Earned Value Analysis of the SOFIA Airborne System Data

The favorable cost variance is due to efficiencies in project oversight and 

engineering. The unfavorable cumulative schedule variance is being 

driven by a lack of government furnished equipment supposed to be 

EVM Surveillance Is Being 

Performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 96 GAO-13-22 NASA

received from NASA that is causing a backlog of tasks. Due to the cost 

and schedule variances we are forecasting a positive variance at 

completion ranging from $3 million to $4 million.

The Tracking and Data Relay Satellite (TDRS) System consists of in-orbit 

communication satellites stationed at geosynchronous altitude coupled 

with two ground stations located in New Mexico and Guam. The satellite 

network and ground stations provide mission services for near-Earth user 

satellites and orbiting vehicles. TDRS-K and L are the 11th and 12th 

satellites, respectively, to be built for the TDRS system. They will 

contribute to the existing network by providing continuous high bandwidth 

digital voice, video, and mission payload data, as well as health and 

safety data relay services to Earth-orbiting spacecraft such as the 

International Space Station and the Hubble Space Telescope. NASA is 

planning to launch TDRS-K in December 2012 followed by the TDRS-L 

launch in February 2014.

NASA is collecting EVM data from both the spacecraft and sustainment 

efforts. In December 2007, NASA awarded a fixed price incentive contract 

to design, develop, fabricate, integrate, test, ship, provide launch support, 

conduct on-orbit checkout operations and provide sustaining engineering 

support for two spacecraft, TDRS-K and TDRS-L, to Boeing Satellite 

Systems, Inc (Boeing).

Table 14: Assessment of TDRS EVM Practices

Dollars in millions

Used a 

certified EVM 

system 

compliant 

with ANSI/EIA 

standard

Conducted an 

integrated 

baseline review

EVM system

surveillance is 

being performed

Budget at 

completion

Data resulting from 

the EVM system 

are reliable

GAO Variance at 

completion

Spacecraft    $698  -$152 to -$185

Sustainment    $5 ◐ NAa

Source: GAO analysis of contractor data as of August 2011.

a

NA - Not applicable means there was not enough data to forecast an estimate at completion 

Tracking and Data 

Relay Satellite 

Replenishment
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 97 GAO-13-22 NASA

The spacecraft contractor, Boeing, met the three fundamental ANSI/EIA748 practices necessary for a reliable EVM system. Boeing has a certified 

EVM system that complies with the ANSI/EIA EVM standard. Though the 

contractor has a certified system, the implementation of that system is 

questionable based on our findings below. We also assessed whether the 

spacecraft contractor’s EVM data against three ANSI/EIA guidelines. 

These guidelines state that the authorized work elements for the project 

should be defined typically using a WBS that has been tailored to the 

project and the WBS is the same for the cost estimate, schedule, and 

EVM. We found that the WBS used in the spacecraft integrated master 

schedule was consistent with the WBS used for the EVM data.

The ANSI/EIA guidelines also state that projects should have a schedule 

that describes the sequence of work by listing activities in the order in 

which they are to be carried out and identifying significant task 

interdependencies required to meet project requirements. We found some 

sequencing issues in the contractor’s schedule. For example, 13 percent 

of the remaining activities were constrained. When schedules are not 

sequenced properly, float values and the calculated critical path will not 

be valid. Project officials acknowledged the constraints and said they are 

a result of having to adjust support activities due to spacecraft integration 

and test delays as well as alignments to the current/actual manifest dates, 

which differ from the Boeing contractual and launch readiness dates. In 

addition, critical path metrics are generated and analyzed monthly to track 

the Boeing performance against the critical path activities. Finally, the 

ANSI/EIA guidelines state that a project should establish and maintain a 

time-phased budget baseline to track cost and schedule variances in an 

EVM system. Though resource loading the schedule is not required to 

meet the ANSI/EIA guidelines, it is a best practice and therefore 

resources should be accounted for in the schedule in order to develop 

this baseline, according to the GAO cost guide. We found that the 

schedule was not resource loaded. Project officials said the integrated 

master schedule is resource loaded but not inside the Microsoft Project 

schedule because it does not directly interface with the Boeing 

financial/EVM system. They stated that the integrated master schedule is 

produced using Microsoft Project and imported into a planning software 

tool. After the resource loading effort is performed, the planning data is 

transferred into the EVM system. The tool integrates the Microsoft Project 

schedule with the Boeing financial system and the resource allocations 

for each task. When adjustments are required Boeing again utilizes the 

planning tool.

Spacecraft Contractor 

Has a Certified EVM 

System Compliant 

with the ANSI/EIA 

Standard
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 98 GAO-13-22 NASA

An integrated baseline review was conducted in 2008 where 211 issues 

were raised. All of these issues have since been resolved and closed.

DCMA prepares a monthly EVM analysis report and performs continuous 

surveillance of Boeing’s EVM implementation by sampling various control 

account managers for interviews about the process.

We examined contract performance reports from August 2010 through 

August 2011. Figure 19 illustrates that as of August 2011, the project was 

reporting a negative cumulative cost variance of approximately $131 

million and a negative cumulative schedule variance of $7 million.

Figure 19: GAO Earned Value Analysis of the TDRS Spacecraft Project Data

The negative cumulative cost variance was being driven by higher staffing 

levels to support integration, the Preliminary Design Review and Critical 

Design Review, as well as an incorrect assessment of project 

requirements and the inability to use heritage specifications. Labor costs 

were also higher than expected due to part failures and the late 

Project Conducted an 

Integrated Baseline 

Review

EVM Surveillance Is Being 

Performed

Data Resulting from the 

EVM System Are 

Somewhat Reliable
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 99 GAO-13-22 NASA

completion of component qualifications. Extended test activities also 

contributed to the cumulative negative cost variance. Finally, more than 

expected resources were needed to complete board and slice designs, 

generate drawings, and assemble and test components because of the 

complexity of the design. Due to the negative cost and schedule variance 

we are forecasting a variance at completion ranging from $152 million to 

$185 million. Project officials said GAO’s independent variance at 

completion gives the impression that NASA may request additional 

funding to complete the TDRS K and L. Because this is a fixed price, 

incentive fee contract, NASA officials said the agency is obligated only to 

pay up to the price ceiling of the contract.

Boeing is the contractor for both the spacecraft and sustainment efforts. 

As noted above, Boeing met all three fundamental practices for a reliable

EVM system.

In addition, the 2008 integrated baseline review was conducted for both 

the spacecraft and sustainment effort. As well, the formal surveillance 

performed by Boeing and the Defense Contract Management Agency 

applies to the sustainment effort

We reviewed contract performance reports from April 2011 to August 

2011. Because we had only 5 months of data, we were not able to 

forecast an independent estimate at completion. Figure 20 below 

illustrates that as of August 2011, the project was reporting a positive 

cumulative cost variance of $0.08 million. Because there were no 

variance analysis reports accompanying the sustaining effort, we were 

unable to determine what was causing the positive cost variance. Project 

officials stated that the Performance Measurement Baseline for this effort 

was almost entirely level of effort, so minimal variances would be 

occurring. They added that during the August 2011 time period, the 

contract reflected an April 2012 launch date even though the launch was 

being delayed. Consequently, Boeing’s reports were reflecting work 

scheduled to occur in support of the earlier launch date when in fact very 

little effort was being done. As a result, since minimal costs were 

incurred, this resulted in a positive cumulative cost variance.

TDRS Sustainment 

Effort EVM Summary
Appendix III: Case Studies of Selected 

Projects’ Implementation of Earned Value 

Management

Page 100 GAO-13-22 NASA

Figure 20: GAO Earned Value Analysis of the TDRS Sustainment Data
Appendix IV: Comments from the National 

Aeronautics and Space Administration 

Page 101 GAO-13-22 NASA

Appendix IV: Comments from the National 

Aeronautics and Space Administration
Appendix IV: Comments from the National 

Aeronautics and Space Administration 

Page 102 GAO-13-22 NASA
Appendix IV: Comments from the National 

Aeronautics and Space Administration 

Page 103 GAO-13-22 NASA
Appendix V: GAO Contact and Staff 

Acknowledgments

Page 104 GAO-13-22 NASA

Cristina Chaplain, (202) 512-4841 or chaplainc@gao.gov

In addition to the contact named above, Shelby S. Oakley and Karen 

Richey (Assistant Directors), Greg Campbell; Jennifer K. Echard; Tisha D. 

Derricotte; Laura Greifner; Kristine R. Hassinger; Ben Jaskiewicz; William 

Laing; Richard Lee; Eric Lofgren; Kenneth E. Patton; Jose A. Ramos; 

Carrie W. Rogers; Stacey L. Steele; Roxanna T. Sun; and Umesh 

Thakkar made key contributions to this report.

Appendix V: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments

(121002)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations

Public Affairs

Please Print on Recycled Paper.