JAMES WEBB SPACE 

TELESCOPE

Actions Needed to 

Improve Cost 

Estimate and 

Oversight of Test and 

Integration

Report to Congressional Committees

December 2012

GAO-13-4

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-13-4, a report to 

congressional committees

December 2012

JAMES WEBB SPACE TELESCOPE

Actions Needed to Improve Cost Estimate and 

Oversight of Test and Integration

Why GAO Did This Study

JWST is one of NASA’s most 

expensive and technologically 

advanced science projects, intended to 

advance understanding of the origin of 

the universe. In 2011, JWST was 

rebaselined with a life cycle cost 

estimate of $8.8 billion and a launch 

readiness date in October 2018—

almost nine times the cost and more 

than a decade later than originally 

projected in 1999. Concern about the

magnitude of JWST's cost increase 

and schedule delay and their effects on 

NASA’s progress on other high-priority 

missions led conferees for the 

Consolidated and Further Continuing 

Appropriations Act, 2012, to direct

GAO to report on the project. 

Specifically, GAO assessed (1) the 

extent to which NASA’s revised cost 

and schedule estimates are reliable

based on best practices, (2) the major 

risks and technological challenges 

JWST faces, and (3) the extent to 

which NASA has improved oversight of 

JWST. To do this, GAO compared 

NASA’s revised cost and schedule 

estimates with best practice criteria, 

reviewed relevant contractor and 

NASA documents, and interviewed 

project and contractor officials. 

What GAO Recommends

GAO recommends NASA take six 

actions including, among others, to 

take steps to improve its cost estimate; 

to conduct an additional, earlier 

independent review of test and 

integration activities; and to develop a 

long-term oversight plan that 

anticipates planned travel budget 

reductions. In commenting on a draft of 

this report, NASA fully or partially 

concurred with the recommendations

citing ongoing efforts, but GAO 

believes some do not go far enough.

What GAO Found

The National Aeronautics and Space Administration (NASA) has provided

significantly more time and money to the James Webb Space Telescope (JWST)

than previously planned and expressed high confidence in the project's new 

baselines. Its current cost estimate reflects some features of best practices for 

developing reliable and credible estimates. For example, the estimate

substantially meets one of four cost characteristics—comprehensive—that GAO 

looks for in a reliable cost estimate, in part because all life cycle costs were 

included. The estimate, however, only partially met the other three 

characteristics—well documented, accurate, and credible—which detracts from

its reliability. For example, the estimate's accuracy, and therefore the confidence 

level assigned to the estimate, was lessened by the summary schedule used for 

the joint cost and schedule risk analysis because it did not provide enough detail 

to determine how risks were applied to critical project activities. The estimate's 

credibility was also lessened because officials did not perform a sensitivity

analysis that would have identified key drivers of costs, such as workforce size. 

Program officials believe that it would have been difficult to fully address all best 

practice characteristics. GAO believes there is time to improve the estimate and 

enhance the prospects for delivering the project according to plan. 

Project officials report that the JWST schedule has 14 months of reserve, which 

meets Goddard guidance for schedule reserve; however, only 7 of the 14 months 

are likely to be available for the last three of JWST’s five complex integration and 

test efforts. GAO's prior work shows that the integration and test phases are 

where problems are commonly found and schedules tend to slip. Given that 

JWST has a challenging integration and test schedule, this could particularly be 

likely. The project has made some significant progress in the past year, notably 

successfully completing development of the 18 primary mirror segments—

considered JWST's top technical risk. Nevertheless, ongoing challenges are 

indicative of the kinds of issues that can require significant effort to address. For 

example, instrument challenges have delayed the first integration and test effort. 

In addition, key long-term risks on subsystems with a significant amount of work 

remaining will not be retired until 2016. Currently, NASA’s plan for project 

oversight calls for one independent mission-level system integration review about 

13 months before launch. While this is consistent with what NASA requires for its 

projects, this approach may not be sufficient for a project as complex as JWST.

JWST has taken several steps to improve communications and oversight of the 

project and its contractors—such as taking over responsibility for mission 

systems engineering from the prime contractor; instituting meetings that include

various levels of NASA, contractor, and subcontractor management; and 

implementing a new risk management system to allow for better tracking of risks. 

The enhancements to the oversight of the project are steps in the right direction, 

but it will take time to assess their effectiveness and ensure that the efforts are 

sustained by the project in the future. Reductions in travel budgets, however,

could require the project to adjust the oversight approach that was adopted as a 

result of the replan. Additional reductions in travel budgets are anticipated in 

future years, but officials do not have a plan to address such reductions and their 

potential impact on continuing the current oversight approach. 

View GAO-13-4. For more information, contact 

Cristina Chaplain at (202) 512-4841 or 

chaplainc@gao.gov.
Page i GAO-13-4 James Webb Space Telescope

Letter 1 

Background 3

Revised JWST Cost Estimate Is Not Fully Consistent with Best 

Practices for Reliable and Credible Estimates and the Integrated 

Master Schedule Is Not Finalized 9

Technically Challenging JWST Project Lacks the Schedule Reserve 

Flexibility and Commensurate Oversight of Integration and Test 

Efforts 16

JWST Project Has Taken Steps to Enhance Communications with 

and Oversight of Its Contractors 26

Conclusions 32

Recommendations for Executive Action 33

Agency Comments and Our Evaluation 34

Appendix I Objectives, Scope, and Methodology 40

Appendix II Subsystems of the JWST Observatory 44

Appendix III Our Evaluation of JWST’s Cost Estimate Process 45

Appendix IV Comments from the National Aeronautics and Space 

Administration 47

Appendix V GAO Contact and Staff Acknowledgments 51

Tables

Table 1: Issues with JWST Science Instruments Included in ISIM 21

Table 2: ICRP Recommendations Concerning Oversight and 

Communications and the Actions Taken by NASA 27

Table 3: NASA Office of Evaluation Review Team Concerns 31

Table 4: Summary Assessment of JWST Cost Estimate Compared 

to Best Practices 45

Contents 
Page ii GAO-13-4 James Webb Space Telescope

Figures

Figure 1: Subsystems of the JWST 6

Figure 2: Organizational Chart for the JWST Program 8

Figure 3: Summary of Results of GAO Assessment of JWST Cost 

Estimate Based on Best Practices Criteria 10

Figure 4: Project Integration and Test Time Line 17

Figure 5: Percentage of Budgeted Development Work Remaining 

for Major Subsystems 19

Figure 6: Current Mass Estimate for Spacecraft Subsystem 20

Figure 7: Changes in ISIM Schedule Since the Project Replan 24

Figure 8: Subsystems of the JWST: Interactive Information 44

Abbreviations

CSA Canadian Space Agency

ESA European Space Agency

FGS Fine Guidance Sensor

ICRP Independent Comprehensive Review Panel

ISIM Integrated Science Instrument Module

JCL Joint Cost and Schedule Confidence Level

JWST James Webb Space Telescope

KDP key decision point

MIRI Mid-Infrared Instrument

NASA National Aeronautics and Space Administration

NIRCam Near-Infrared Camera

NIRISS Near-Infrared Imager and Slitless Spectrograph

NIRSpec Near-Infrared Spectrograph

NPR NASA Procedural Requirements

OTE Optical Telescope Element

OTIS Optical Telescope Element and Integrated Science 

 Instrument Module

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-13-4 James Webb Space Telescope

United States Government Accountability Office

Washington, DC 20548

December 3, 2012

Congressional Committees

The James Webb Space Telescope (JWST) is one of the National 

Aeronautics and Space Administration’s (NASA) most expensive and 

technologically advanced science projects. It is intended to revolutionize 

our understanding of star and planet formation and advance the search 

for the origins of our universe. The project has been hampered by 

ineffective management, unrealistic estimates of costs, and insufficient 

resources, and is projected to cost significantly more and to take much 

longer to complete than expected when it was authorized to proceed in 

1999. In 2011, JWST was rebaselined with a lifecycle cost estimate of 

$8.8 billion and a launch date in October 2018, which is almost nine times 

the cost and more than a decade later than originally projected. The 

magnitude of JWST’s cost increase and schedule delay has and will 

continue to have a major effect on NASA’s ability to implement or begin 

high-priority science missions for years to come.

NASA has expressed high confidence in being able to execute the project 

within its new cost and schedule baselines, and officials report that the 

project is being executed to its revised estimates in the year since the 

rebaseline. The project, however, is less than halfway through its 

development with known and unknown technical risks and challenges 

remaining to be addressed. The on-time and on-budget delivery of JWST 

is a high congressional priority. In November 2011, the Joint Explanatory 

Statement of the Committee of Conference for the Consolidated and 

Further Continuing Appropriations Act, 2012, recommended that JWST’s 

formulation and development costs, not including the cost to operate the 

mission after launch, be no more than $8 billion. The conferees also 

directed GAO to assess the program annually and to report to the 

Committees on Appropriations on key issues relating to program and risk 

management, achievement of cost and schedule goals, and program 

technical status. This report responds to that direction. Specifically, we 

assessed (1) the extent to which NASA’s revised cost and schedule 

estimates are reliable based on GAO best practices, (2) the major risks 

and technological challenges the JWST project faces, and (3) the extent 

to which NASA has improved the oversight of the JWST project.

 
 

Page 2 GAO-13-4 James Webb Space Telescope

Our approach included an examination of the revised cost and schedule 

estimates NASA developed during the replanning process using GAO’s 

best practice criteria.1

We conducted this performance audit from February 2012 to December 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

This included review of documentation of the inputs 

that supported the detailed cost and schedule estimates used by 

decision-makers to rebaseline the program and interviews with project 

officials involved in developing the estimates. We also reviewed the 

proposal from the JWST prime contractor used as input to the replan and 

interviewed contractor officials. We did not assess the most current JWST 

integrated master schedule because of ongoing negotiations for contract 

modifications associated with the replan. To identify the major short and

long term risks and technological challenges facing the project, we 

interviewed project officials for the major subsystems as well as 

contractor officials. We also reviewed the project’s risk database, monthly 

status reviews, and other documentation provided by the project officials. 

To understand the steps NASA implemented to improve the oversight 

being performed on the project, we interviewed project and contractor 

officials and reviewed documentation from independent reviews prior to 

the replan and actions taken by NASA in response. We also interviewed 

the Director of NASA’s Office of Evaluation concerning a recent 

independent review of the JWST project performed by his office. We took 

appropriate steps to address data reliability, such as clarifying data 

discrepancies and corroborating NASA-generated data with other sources 

where applicable. NASA provided comments and technical clarifications 

on a draft of this report, which were incorporated as appropriate. 

Appendix I contains detailed information on our scope and methodology.

 

1

GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and 

Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: Mar. 2, 2009); and 

GAO Schedule Assessment Guide: Best Practices for Project Schedules, GAO-12-120G

(Washington, D. C.: May 30, 2012).
 

Page 3 GAO-13-4 James Webb Space Telescope

Initially referred to as the “Next Generation Space Telescope,” JWST is a 

large deployable, infrared-optimized space telescope intended to be the 

successor to the aging Hubble Space Telescope. JWST is designed to be 

a 5-year mission to find the first stars and trace the evolution of galaxies 

from their beginning to their current formation, and is intended to operate 

in an orbit approximately 1.5 million kilometers—or 1 million miles—from 

the Earth. In a 2001 decadal survey, the National Research Council rated 

the JWST as the top-priority new initiative for astronomy and physics. 

With its 6.5-meter primary mirror, JWST will be able to operate at 100 

times the sensitivity of the Hubble Space Telescope. A tennis-court-sized 

sunshield will protect the mirrors and instruments from the sun’s heat to 

allow the JWST to look at very faint infrared sources. The Hubble Space 

Telescope operates primarily in the visible and ultraviolet regions.2

JWST has experienced significant increases to project costs and 

schedule delays. Prior to being approved for development, cost estimates 

of the project ranged from $1 billion to $3.5 billion with expected launch 

dates ranging from 2007 to 2011. In March 2005, NASA increased the 

JWST’s life-cycle cost estimate to $4.5 billion and slipped the launch date 

to 2013. We reported in 2006 that about half of the cost growth was due 

to schedule slippage—a 1-year schedule slip because of a delay in the 

decision to use a European Space Agency-supplied Ariane 5 launch 

vehicle and an additional 10-month slip caused by budget profile 

limitations in fiscal years 2006 and 2007. More than a third of the cost 

increase was caused by requirements and other changes. An increase in 

the program’s contingency funding accounted for the remainder—about 

12 percent—of the growth. NASA Headquarters chartered an 

Independent Review Team to evaluate the project that same year. In April 

2006, the review team’s assessment confirmed that the program’s 

technical content was complete and sound, but expressed concern over 

the project’s contingency reserve funding—funding used to mitigate 

issues that arise but which were previously unknown—reporting that it 

was too low and phased in too late in the development life cycle. The 

team reported that for a project as complex as the JWST, a 25 to 30 

 

2

The electromagnetic spectrum is the wavelengths of all the visible and invisible light. The 

infrared part of the spectrum, also known as radiant heat, has wavelengths that go from 

about 0.75 microns to a few hundred microns. The Hubble is designed to operate primarily 

in the ultraviolet and visible wavelengths of the spectrum from 0.1 to 0.8 microns. Humans 

cannot see in the ultraviolet region.

Background 

History of Cost Growth 

and Schedule Delays
 

Page 4 GAO-13-4 James Webb Space Telescope

percent total contingency was appropriate. At that time, JWST’s total 

contingency was about 19 percent. The team cautioned that this 

contingency compromised the project’s ability to resolve issues, address 

risk areas, and accommodate unknown problems. The team also 

concluded that the 2013 launch date was not viable for the project based 

on its anticipated budget. It recommended that before the project was 

formally approved for development and baselined, NASA should take 

steps to provide the JWST project with adequate time-phased reserve 

funding to secure a stable launch date. Additional reserves were added 

and the project was baselined in April 2009 with a life-cycle cost estimate 

of $4.964 billion and a launch date in June 2014.

Shortly after JWST was approved for development and its cost and 

schedule estimates were baselined, project costs continued to increase. 

In 2010, Senator Barbara Mikulski, chair of the Senate Committee on 

Appropriations, Subcommittee on Commerce, Justice, Science, and 

Related Agencies, asked NASA to initiate another independent review in 

response to the project’s cost increases and reports that the June 2014 

launch date was in jeopardy. The Independent Comprehensive Review 

Panel (ICRP) was commissioned by NASA and began its review in 

August 2010. In October 2010, the ICRP issued its report and cited 

several reasons for the project’s problems including management, 

budgeting, oversight, governance and accountability, and communication 

issues.3

 

3

James Webb Space Telescope (JWST) Independent Comprehensive Review Panel 

(ICRP): Final Report (Oct. 29, 2010).

The panel concluded JWST was executing well from a technical 

standpoint, but that the baseline funding did not reflect the most probable 

cost with adequate reserves in each year of project execution, resulting in 

an unexecutable project. The review panel recommended that additional 

resources be considered along with organizational and management 

restructuring. Following this review, the JWST program underwent a 

replan in 2011. In November 2011, the JWST project was reauthorized, 

but not before it was recommended for termination by the House 

Appropriations Committee. On the basis of the replan, NASA announced 

that the project would be rebaselined at $8.835 billion—a 78 percent 

increase to the project’s life-cycle cost from the confirmed baseline—and 

would launch in October 2018—a delay of 52 months. The revised lifecycle cost estimate included 13 months of funded schedule reserve. In 

the President’s Fiscal Year 2013 budget request, NASA reported a 66 
 

Page 5 GAO-13-4 James Webb Space Telescope

percent joint cost and schedule confidence level associated with these 

estimates. A joint cost and schedule confidence level (JCL) is the process 

NASA uses to assign a percentage to the probable success of meeting 

cost and schedule targets and is part of the project’s estimating process.

The JWST project is divided into three major segments: the launch 

segment, the ground segment, and the observatory segment. The launch 

segment is primarily provided by the European Space Agency (ESA), 

which is contributing the Ariane 5 launch vehicle and launch site 

operations in French Guiana. The ground segment will be responsible for 

collecting the data obtained by JWST in space and making it usable for 

scientists and researchers. This includes the development of software 

that will translate data into usable formats as well as operation of the 

software once the telescope is in space. The Space Telescope Science 

Institute, operated by the Association of Universities for Research in 

Astronomy (AURA) on a contract awarded by NASA, which currently 

performs science operations for the Hubble Space Telescope, is 

developing the science and operations and flight operations center for 

JWST and will conduct the first 6 months of flight and science operations. 

The NASA contract with the Space Telescope Science Institute extends

through the first 6 months of JWST operations. A contract to manage the 

long term operations of JWST is planned to be awarded approximately 2 

years prior to launch. The observatory segment will be launched into 

space and includes five major subsystems.4

 

4

The hardware configuration created when the Optical Telescope Element and the 

Integrated Science Instrument Module are integrated, referred to as OTIS, is not 

considered a subsystem by NASA, but we categorize it as such for ease of discussion.

These subsystems are being 

developed through a mixture of NASA, contractor, and international 

partner efforts. See figure 1.

Current JWST 

Organization
Figure 1: James Webb Space Telescope

Sources: GAO (analysis); NASA (data and images).

Interactive Graphic Rollover the white dots to see description. See appendix II for the non-interactive, printer-friendly version.

Integrated Science

Instrument Module

Optical 

Telescope

Element

Ground systems

Spacecraft

Sunshield

Optical Telescope &

Integrated Science

Instrument Module

Page 6 GAO-13-4 James Webb Space Telescope
 

Page 7 GAO-13-4 James Webb Space Telescope

JWST is a single project program reporting directly to the NASA 

Associate Administrator for programmatic oversight and to the Associate 

Administrator for the Science Mission Directorate for technical and 

analysis support.5

 

5

A project typically reports to a program within a mission directorate, such as the Science 

Mission Directorate. A similar reporting structure was also instituted in the past with both 

the Hubble Space Telescope and the Mars Exploration Program when they began 

experiencing significant cost and schedule issues. For JWST, the change was made in 

response to recommendations from the ICRP.

Goddard Space Flight Center is the NASA center 

responsible for the management of JWST. See figure 2 for the current 

JWST organizational chart.
 

Page 8 GAO-13-4 James Webb Space Telescope

Figure 2: Organizational Chart for the JWST Program
 

Page 9 GAO-13-4 James Webb Space Telescope

Our analysis of JWST’s revised cost estimate showed that it is not fully 

consistent with best practices for developing reliable and credible 

estimates, although project officials took some steps in line with best 

practices in the development of the estimate. For example, as part of its 

cost estimation process, the project conducted a joint cost and schedule 

risk analysis, or joint cost and schedule confidence level (JCL), which 

assigned a 66 percent confidence level to the estimate.6

 

6

The JCL is a quantitative probability analysis that requires the project to combine its cost, 

schedule, and risks into a complete quantitative picture to help assess whether the project 

will be successfully completed within cost and on schedule. NASA introduced the analysis 

in 2009, and it is among the agency’s initiatives to reduce acquisition management risk. 

The move to probabilistic estimating marks a major departure from NASA’s prior practice 

of establishing a point estimate and adding a percentage on top of that point estimate to

provide for contingencies. NASA’s procedural requirements state that Mission Directorates 

should plan and budget programs and projects based on a 70 percent JCL, or at a 

different level as approved by the Decision Authority—which in the case of the JWST 

project would be the Agency Program Management Council—and any JCL approved at 

less than 70 percent must be justified and documented. NASA Procedural Requirements 

(NPR) 7120.5E, NASA’s Space Flight Program and Project Management Requirements, 

paragraph 2.4.4 (Aug. 14, 2012).

In addition, we 

found that the cost estimate included all life cycle costs for the project. 

Although NASA’s methods for developing the JWST cost estimate reflect 

some features of best practices, our review of the estimate showed that 

based on best practice criteria, it did not fully meet the four characteristics 

of a reliable estimate. See figure 3.

Revised JWST Cost 

Estimate Is Not Fully 

Consistent with Best 

Practices for Reliable 

and Credible 

Estimates and the 

Integrated Master 

Schedule Is Not

Finalized 
 

Page 10 GAO-13-4 James Webb Space Telescope

Figure 3: Summary of Results of GAO Assessment of JWST Cost Estimate Based on Best Practices Criteria

Note: Met: NASA provided complete evidence that satisfies the entire criterion; Substantially Met: 

NASA provided evidence that satisfies a large portion of the criterion; Partially Met: NASA provided 

evidence that satisfies about half of the criterion; Minimally Met: NASA provided evidence that 

satisfies a small portion of the criterion; and Not Met: NASA provided no evidence that satisfies any of 

the criterion. See appendix III for our evaluation of JWST’s cost estimate process.

Specifically, the project’s estimate was found to substantially meet the 

best practice criteria for being comprehensive, and the remaining three 

characteristics of being well documented, accurate, and credible were 

found to be only partially met. For example, the accuracy of the cost 

estimate, and therefore the confidence level assigned to the estimate, 

was lessened by the schedule used in the JCL analysis because it 

prevented us from, among other things, identifying the activities that were 

on the critical path—defined as time associated with activities that drive 

the overall schedule. The credibility of the estimate was lessened 
 

Page 11 GAO-13-4 James Webb Space Telescope

because project officials did not perform a sensitivity analysis that would 

have identified key drivers of costs, such as workforce size. Although 

NASA is not required to adhere to these best practices, our prior work has 

shown that not following best practices for cost estimating can make the 

cost estimate less reliable, putting projects at risk of experiencing cost 

overruns, missed deadlines, and performance shortfalls.7

Comprehensive: The JWST cost estimate substantially met the criteria 

for being comprehensive because it included all life-cycle costs, 

documented cost-influencing ground rules and assumptions, and used a 

work breakdown structure, or the structure used to define in detail the 

work necessary to accomplish its objectives, that is detailed and traceable 

to the cost of each work element and the contract statement of work.

The best 

practices stem from practices federal cost estimating organizations and 

industry use to develop and maintain reliable cost estimates, including the 

Department of Defense and NASA. According to program officials, it 

would have been difficult, if not impossible, for the project to have met all 

of the best practice criteria given the complexity of the project and that 

some elements of the project are quite mature in their development. 

Instead, the program manager stated that the project followed a tailored 

process to develop the cost estimate that was appropriate for the project. 

Furthermore, officials report the project is currently meeting a majority of 

its milestones and executing as planned to the revised estimates for the 

JWST.

8

 

7

The 

cost estimate is not considered fully comprehensive, however, because 

we were unable to align the schedule to the cost estimate and could not 

align all of the subcontractor work to the work breakdown structure of the 

cost estimate. Although NASA officials believe that the project has 

visibility into, and can map, subcontractor activities, we were unable to 

confirm that cost elements were neither omitted nor double counted. In 

addition, we were unable to map the project’s earned value management 

data to the work breakdown structure because information between the 

GAO-09-3SP. 

8

A work breakdown structure reflects the requirements and what must be accomplished to 

develop a program, and it provides a basis for identifying resources and tasks for 

developing a program cost estimate. The work breakdown structure should be used to 

define all program activities and tasks to ensure that the schedule encompasses the entire 

work. 
 

Page 12 GAO-13-4 James Webb Space Telescope

two was not compatible.9

Well documented: The JWST cost estimate only partially met the criteria 

for being well documented because it did not include a step-by-step 

description of how the estimate was developed, the raw data used to 

develop the estimate, or the calculations and estimating methodology for 

specific cost elements of the work breakdown structure. Without good 

documentation, a cost analyst unfamiliar with the program will not be able 

to replicate the estimate, because he or she will not understand the logic 

behind it. Good documentation, for example, assists management and 

oversight in assessing the credibility of the estimate, helps to keep a 

history of reasons for cost changes and to record lessons learned, 

defines the scope of the analysis, and answers questions about the 

approach or data used to create the estimate. Project documentation, 

however, does provide evidence that NASA management reviewed and 

accepted the cost estimate because managers were briefed on the 

technical aspects of the estimate and were provided an overview of the 

joint cost and schedule risk analysis that was conducted.

Finally, although the project outlined and 

documented the ground rules and assumptions, we were unable to 

determine whether risks associated with any assumptions were identified 

and traced to specific elements.

Accurate: The JWST cost estimate only partially met the criteria for being 

accurate because the projected costs of schedule reserve did not reflect 

actual data, the summary schedule used to derive the JCL prevented us 

from sufficiently understanding how risks were incorporated, and the 

project did not provide evidence that it regularly updates the estimate or 

plans to conduct another JCL. For example, using historical actual cost 

data from Northrop Grumman, we estimated that 13 months of schedule 

reserve is likely to be $204 million instead of NASA’s estimate of $121 

million—a potential underestimation of 69 percent related to the schedule 

reserve. Project officials, however, believe they have adequate reserves 

available to offset any underestimation. In addition, the summary 

 

9

Earned value management is a project management tool that integrates the technical 

scope of work with schedule and cost elements and compares the value of work 

accomplished in a given period with the value of the work expected in that period. When 

used properly, earned value management can provide objective assessments of project 

progress, produce early warning signs of impending schedule delays and cost overruns, 

and provide unbiased estimates of anticipated costs at completion. As a best practice the 

work breakdown structure should match the schedule, cost estimate, and earned value 

management system at a high level so that it clearly reflects the work to be done. 
 

Page 13 GAO-13-4 James Webb Space Telescope

schedule the project used as an input to the JCL, although deemed 

acceptable by NASA, contained many long-duration activities, some with 

1,000 days or more. Because of these long durations in the summary 

schedule used for the JCL, the lack of detail prevented us from identifying 

the activities that were on the critical path, as well as which risks were 

applied to remaining activities.10

Finally, it was unclear whether the cost estimate was regularly updated to 

reflect material changes in actual costs and in the project itself, such as 

when schedules or other assumptions change, due to a lack of detailed 

documentation for the cost estimate. Project officials stated that in 

keeping with NASA policy they do not plan, nor are they required, to 

conduct another JCL analysis. GAO’s cost estimating best practices call 

for estimates to be continually updated through the life of the project, 

ideally every month as actual costs are reported in earned value 

management reports, and that a risk analysis and risk simulation 

exercise—like the JCL analysis—be conducted periodically through the 

life of the program, as risks can materialize or change throughout the life 

of a project. Unless properly updated on a regular basis, the cost estimate 

cannot provide decision makers with accurate information to assess the 

current status of the project. NASA officials state that the life-cycle cost 

estimate is updated annually for the budgeting process, and that historical 

records such as earned value data were used to develop the estimate. 

They also stated that this information is updated in several different 

documents being provided to management; however, we were unable to 

determine how this information was used in updating the cost estimate on 

a regular basis.

As a result, there is no way to ensure 

that risks were appropriately assigned to activities in the schedule to 

account for the impact of the risks during the JCL analysis.

Credible: The JWST cost estimate only partially met the criteria for being 

credible because project officials did not adequately test and verify the 

reasonableness of the cost estimate and the schedule used in conducting 

the JCL did not have a valid critical path and contained durations that 

were too long to properly account for risks. For example, project officials 

said they did not perform a sensitivity analysis for the cost estimate. A 

sensitivity analysis identifies key elements that drive cost and permits 

 

10A summary schedule is derived from the integrated master schedule and is often used in 

a JCL analysis when, for example, time or resources needed to simulate the full detailed 

schedule are limited. 
 

Page 14 GAO-13-4 James Webb Space Telescope

analysis of different outcomes and is often used to develop cost ranges 

and risk reserves. NASA officials stated that the largest cost driver for the 

JWST project is the size of the workforce, which could have been 

subjected to a sensitivity analysis; yet, the cost model did not include a 

sensitivity analysis that would show how staff increasing or decreasing 

over time affects cost. In addition, NASA officials believe that all risks 

were sufficiently accounted for when conducting the JCL, however, the 

software used to conduct the JCL analysis does not recognize certain 

risks that officials had placed on activities in the project schedule and, 

therefore, some risks were discarded during the simulation. 

The schedule used to conduct the JCL was also summarized at such a 

high level that the durations were too long to effectively model the risks. 

For example, one of the activities that drove the launch date was over 4 

years in duration and should have been broken down further prior to 

conducting the simulation. Moreover, the critical path in the JCL schedule 

consisted of six level of effort activities all with the same duration of 2,238 

days in length.11

In addition, project officials did not commission an independent cost 

estimate, which is considered one of the best and most reliable estimate 

validation methods because it shows whether other estimating 

procedures produce similar results, and it provides an independent view 

of expected program costs that tests the program office’s estimate for 

reasonableness.

Level of effort activities should never be on the critical 

path because support activities should never drive any milestone finish 

date. As a result of the schedule used in the JCL not fully meeting best 

practices, we question the results of the analysis. Furthermore, the risk of 

having to carry the JWST workforce to support the project if delayed was 

not included since a sensitivity analysis was not performed. Project 

officials report that, instead, risk associated with the workforce was 

factored in when establishing cost reserves. 

12

 

11Level of effort activities are those activities that have no measurable output and cannot 

be associated with a physical product or defined deliverable. These activities are typically 

related to management and other oversight that continues until the detailed activities they 

support have been completed.

An estimate that has not been reconciled with an 

independent cost estimate has an increased risk of being underfunded 

12An independent cost estimate is conducted by a group outside the acquiring 

organization that uses the same detailed technical information as the program estimate to 

determine whether other estimating methods produce similar results. 
 

Page 15 GAO-13-4 James Webb Space Telescope

because the independent cost estimate provides an objective and 

unbiased assessment of whether the project estimate can be achieved. 

Notably, however, project officials provided evidence that an independent 

cost assessment was done for the project at the request of the JWST 

Standing Review Board, the independent review team for the project, and 

the assessment was within 2 percent of the project’s estimated cost for 

the rebaseline. Project officials contend that the approach they used in 

developing the life-cycle cost estimate for the project is more accurate 

than the types of approaches often used to develop and independent 

estimate.

We did not conduct a full schedule assessment to determine the reliability 

of the revised schedule based on best practices due to on-going contract 

negotiations.13 The project has an integrated master schedule developed 

as part of the replan; however, it is not finalized because major contract 

modifications have yet to be negotiated and definitized.14

 

13

Specifically, the 

modification to the Northrop Grumman contract, which accounts for 

approximately 40 percent of the total project cost and spans much of the 

work on the spacecraft and OTE, remains undefinitized more than a year 

after the project was rebaselined. Once the project completes 

negotiations for the contract modification and all schedule dates are set, 

the project can then have a measurable integrated master schedule. 

Project officials stated that the negotiation process and updating of 

associated schedules are planned to be complete in January 2013 for the 

Northrop Grumman contract modification—a year after submission of the 

latest update to its proposal for the replan. The project also reported that 

multiple audits of the proposals submitted by Northrop Grumman and its 

subcontractor by the Defense Contract Audit Agency have delayed 

definitization. Negotiations for the modification to NASA’s contract with 

the Space Telescope Science Institute to incorporate the October 2018 

GAO-12-120G. 

14An integrated master schedule constitutes a program schedule that includes the entire 

required scope of effort, including the effort necessary from all government, contractor, 

and other key parties for a program’s successful execution from start to finish. As a key 

focal point for management, the integrated master schedule documents and integrates the 

planned work, the resources necessary to accomplish that work, and the associated 

budget. It also consists of logically related activities whose forecasted dates are 

automatically recalculated when activities change allowing for management to respond to 

the consequences of these changes.

Complete Schedule Not 

Finalized; Full Assessment 

of Schedule Needed
 

Page 16 GAO-13-4 James Webb Space Telescope

launch readiness date are not scheduled to be complete until spring 

2013.

Once all the contracts have been definitized and the project’s integrated 

master schedule is baselined, we plan to conduct a comprehensive best 

practices assessment of the reliability of the project’s schedule estimates.

Project officials report that the JWST schedule has 14 months of reserve, 

which meets Goddard guidance for schedule reserve; however, only 7 of 

the 14 months are likely to be available for the last three of JWST’s five 

complex integration and test efforts. GAO’s prior work shows that it is 

during integration and test where problems are commonly found and 

schedules tend to slip. Given that JWST has a challenging integration and 

test schedule, this could particularly be the case. The project has made 

some significant progress in the past year, notably successfully 

completing development of the 18 primary mirror segments—considered 

JWST’s top technical risk. Nevertheless, ongoing challenges are 

indicative of the kinds of issues that can require a significant amount of 

effort to address. For example, instrument challenges have delayed the 

first integration and test effort. In addition, key long-term risks on 

subsystems with a significant amount of work remaining will not be retired 

until 2016. Currently, NASA’s plan for project oversight calls for one 

independent system integration review about 13 months before launch. 

While this is consistent with what NASA requires for its projects, this 

approach may not be sufficient for a project as complex as JWST. As a 

result, the current plan may be inadequate to ensure key technical and 

management issues are identified early enough to be addressed within 

the current integration and test phase schedule.

JWST has a complex and lengthy integration and test phase, which 

includes five major integration and test efforts—ISIM, OTE, OTIS, 

spacecraft, and observatory. See figure 4 for the project reported dates 

for the major integration and test efforts and the schedule reserve 

allocated for each effort.

Technically 

Challenging JWST 

Project Lacks the 

Schedule Reserve 

Flexibility and 

Commensurate 

Oversight of 

Integration and Test 

Efforts

Test and Integration 

Schedule Lacks Schedule 

Reserve Late in the 

Process
 

Page 17 GAO-13-4 James Webb Space Telescope

Figure 4: Project Integration and Test Time Line

Overall, project officials report that the critical path schedule has 14 

months of reserve with 7 months after the ISIM and OTE integration and 

test efforts. If these efforts are delayed beyond those 7 months, they will 

impinge on the schedule for the remaining three integration and test 

efforts. Project officials stated that the baseline plan is for the OTIS 

integration and test effort to not begin earlier than May 2016. These 

officials reported it is likely that all of the 7 months of schedule reserve 

held by the OTE subsystem will be utilized during its integration and test 

prior to delivery to OTIS and that the OTE effort is on the critical path for 

the project. Therefore, the remaining integration and test efforts—OTIS, 

Spacecraft, and Observatory—will likely have at most 7 months divided 

among them to use if issues are found during integration and test.

In addition to not likely being able to conserve any of the unused first 7 

months of schedule reserve, the project has limited time allocated to the 

final three integration and test efforts, with between 2 to 4 months for 

each. This time could be used easily by the project if an issue were to 

arise during integration and test. An example of this is seen in the OTIS 
 

Page 18 GAO-13-4 James Webb Space Telescope

integration and test schedule, which currently has 3 months of schedule 

reserve. The final event in the OTIS integration and test effort is a lengthy 

cryo-vacuum test—the first time that the optics integrated with the 

instruments will be tested at operational temperatures near absolute zero 

(less than -400 degrees Fahrenheit)—that takes approximately 3 months, 

due to the requirements of the test. If an issue were to arise during this 

test that requires shutting the test down and working on the hardware, the 

chamber would have to be slowly warmed to a temperature safe for 

removal of the hardware from the chamber, work would be performed, 

and the 3-month test process would need to begin again. This could 

easily exhaust the available schedule reserve. Prior GAO work shows 

that it is during integration and test when problems are commonly found, 

and schedules tend to slip. A project official confirmed that this is the case 

because during integration and test the process is more sequential and 

there is less flexibility to move work around if problems are found. A 

NASA Inspector General report on the Mars Science Laboratory, another 

complex and high-cost mission, found that historically the probability that 

schedule-impacting problems will arise is commensurate with the 

complexity of the project.15 JWST is one of NASA’s most technologically 

complex projects to date.

The project has made significant progress overcoming several technical 

challenges over the last year. In December 2011, for example, the project 

completed development of the 18 segments of the primary mirror—the 

project’s primary technology risk—approximately 6 weeks ahead of 

schedule. In addition, project officials stated that during the last year they 

were also able to accelerate other optics-related work, which added one 

month of funded reserve to the schedule, bringing the total to 14 months. 

Finally, the project successfully addressed an increase in the estimated 

amount of heat on the instruments, which otherwise could have pushed 

observatory temperatures close to where the optics would not function 

correctly.

Although technical challenges are being overcome, the project will likely 

continue to experience additional challenges over the remainder of the 

project, given the significant portion and complexity of the work 

 

15NASA, Office of Inspector General, NASA’s Management of the Mars Science 

Laboratory Project. IG-11-019. (Washington, D.C.: June 2011). 

Despite Overcoming 

Several Technical 

Challenges, JWST Has a 

Significant Amount of 

Work with Complex 

Technical Challenges 

Remaining
 

Page 19 GAO-13-4 James Webb Space Telescope

remaining. Four of six major subsystems have nearly 50 percent or more 

of their development work remaining based on its current budget 

information, although the dollar amounts associated with the work vary. 

See figure 5.

Figure 5: Percentage of Budgeted Development Work Remaining for Major 

Subsystems

Currently, the project is experiencing several technical issues that have 

required a significant amount of time and effort to address. For example, 

the spacecraft subsystem, which experienced delays in development prior 

to the replan, is currently estimated to be heavier than its mass limit.16

 

16 Mass is a measurement of how much matter is in an object. It is related to an object’s 

weight and is mathematically equal to mass multiplied by acceleration due to gravity. The 

project uses mass for JWST because when it goes into space, its weight changes with 

gravity, but its mass stays the same. 

Spacecraft development has lagged behind other subsystems because it 

was viewed as a lower risk part of the project and was therefore not 

allocated funding when budgets were limited prior to the replan. In March 

2010, the project passed its mission critical design review, which 

evaluated the project design and its ability to meet mission requirements 

and indicated that the design was ready for fabrication phase; however, 
 

Page 20 GAO-13-4 James Webb Space Telescope

the spacecraft was not included in this review due to its delayed 

development. Under the initial replan, which had constrained funding in 

fiscal years 2011 and 2012, the spacecraft critical design review was 

scheduled for June 2014; however, due to additional funding in the final 

agency-approved replan, the project was able to accelerate work and this 

review is now planned for December 2013.

Project officials have been concerned with the mass of JWST since its 

inception because of the telescope size and the limits of available launch 

vehicles. Accordingly, mass limits have been allocated for each 

subsystem, including the spacecraft. Project officials stated that they 

expected to encounter mass growth on the spacecraft, but that the 

magnitude of the mass growth on the spacecraft was unexpected. As 

shown in figure 6, the current spacecraft projected mass exceeds its 

mass allocation.

Figure 6: Current Mass Estimate for Spacecraft Subsystem

Primary drivers of the mass growth on the spacecraft are increases in the 

estimated weight of the wiring harnesses, which distribute power and 

electric signals between different parts of the observatory, the solar array, 

and other structures that make up the spacecraft. The burden to find 
 

Page 21 GAO-13-4 James Webb Space Telescope

ways to reduce mass has been primarily placed with the spacecraft 

because it was assessed by the project to have the least technical risk 

and because it is the least mature subsystem and can more easily 

accommodate design changes. Over 100 kilograms, or 220 pounds, of 

mass savings options are being evaluated by the project and Northrop 

Grumman, which is developing the spacecraft. Potential mass solutions 

have been identified by Northrop Grumman and the project; however, 

cost and risk vary with each solution and the project is still evaluating the 

trade-offs of the various solutions. Project officials stated that final 

decisions for all tradeoffs will need to occur before spacecraft critical 

design review in December 2013.

The ISIM subsystem is experiencing technology and engineering 

challenges that resulted in the use of 18 of ISIM’s 26 months of schedule 

reserve. The schedule for the instruments needed for ISIM continues to 

slip, which could result in use of more schedule reserve. Based on the 

replan, all four instruments were to be delivered by September 2012; 

however, only two instruments were delivered by that time and those still 

have issues that must be addressed. The remaining two instruments are 

currently scheduled to be delivered at least 11 months late. See table 1 

below for the instrument specific issues.

Table 1: Issues with JWST Science Instruments Included in ISIM

Instrument Instrument specific issues

MIRI MIRI was delivered in May 2012, 1 month later than the replan date, which did not impact the ISIM integration and 

test schedule. However, the instrument team is still working to address an instrument sensitivity issue. The 

instrument team is conducting data analysis to determine a root cause for why the instrument is less sensitive than 

required. A formal report on the root cause and recommended corrective action is expected in November 2012. 

NASA officials report that preliminary findings indicate that hardware changes will not be necessary.

NIRSpec NIRSpec is currently planned for delivery in August 2013, a delay of 11 months from the replan date as a result of 

several technical issues. First, in July 2011, three cracks were found in the part that holds the optics components 

for the instrument. A failure review board was held in January 2012. ESA, which is building the instrument, will 

need to reassemble the instrument using the flight spare part. This part cannot be rebuilt in time to include the 

instrument in the first of three cryo-vacuum tests of ISIM. The project team adjusted the integration and test 

schedule to be able to accept delivery of NIRSpec in February 2013. However, additional issues during testing, 

including failure of the microshutters to close and slower than planned progress on the reintegration by the ESA 

contractor, have further delayed NIRSpec until August 2013. The microshutters were provided to ESA by Goddard 

Space Flight Center and enable the instrument to study 100 targets simultaneously. To mitigate schedule risk, the 

project is studying an option to build new microshutters that will replace the current ones, if necessary. Delivery of 

NIRSpec after August 2013 may threaten ISIM’s critical path schedule. 
 

Page 22 GAO-13-4 James Webb Space Telescope

Instrument Instrument specific issues

FGS/NIRISS FGS/NIRISS was delivered as planned in July 2012; however, it was delivered with a known gear motor issue that 

could result in the early loss of NIRISS in space. In July 2012, the Canadian Space Agency (CSA), which built the 

instrument, began to redesign the gear motors and they are being rebuilt. Around the time of the replan, CSA had 

to redesign its instrument because the original design was not meeting performance requirements. Part of 

instrument design was simplified, which resulted in some loss of expected science performance; however, NIRISS 

science is not required for JWST mission success. 

NIRCam NIRCam is currently planned for delivery in August 2013, a delay of 11 months from the replan date as a result of 

technical and workmanship issues. The project determined the need to electrically ground a mirror on the 

instrument and the team is reviewing several potential options to address this issue. Workmanship issues, such as 

poor soldering quality of the wires in the instrument controls electronic box, resulted in the need for corrective 

actions. The project has decided to run the first ISIM cryo-vacuum test without NIRCam. 

Source: GAO analysis of NASA information.

In addition to the instrument delays, two other technical challenges 

associated with ISIM are: (1) the detectors used by three of the four 

instruments to capture infrared light in space are degrading and may 

need to be replaced, resulting in the addition of another round of cryovacuum testing—in which a test chamber is used to simulate the near 

absolute zero temperatures in space, and (2) issues with the 

development of the cryo-cooler system that removes heat and cools MIRI. 

In December 2010 the project became aware that the detectors in three 

of the instruments were degrading.17

 

17NIRCam, NIRSpec, and FGS/NIRISS use detectors fabricated by the same vendor; 

whereas MIRI uses different detectors that do not need to be replaced. 

As a result, approximately $42 

million and 15 months of schedule reserve to replace the detectors were 

included in the replan. These additions covered the cost of manufacturing 

the detectors; fabrication, assembly, and test of new focal plane 

assemblies; changing the detectors on three instruments, and the 

addition of a third ISIM cryo-vacuum test. The manufacturing process for 

new detectors takes approximately 30 months, which means that they 

cannot be delivered until after the second round of ISIM cryo-vacuum 

testing in 2014. As a result, $2 million of the $42 million in the replan was 

used to add a third round of cryo-vacuum testing for ISIM. The third test 

will validate the performance requirement of the ISIM and is the only time 

the instruments are tested with the flight detectors. Changing the 

detectors requires disassembling the instruments from ISIM, a process 

that will risk damage to the structure and instruments. Project officials 

stated that they will continue to monitor the degradation rate of the current 

detectors because if the degradation rate is low, they may not replace the 

detectors.
 

Page 23 GAO-13-4 James Webb Space Telescope

Development issues with a part of the cryo-cooler needed for MIRI have 

delayed its delivery to ISIM. In 2010, project officials realized that an 

essential valve in the cryo-cooler was leaking at rates that exceeded 

requirements. Following the results of a failure review board, the 

contractor manufactured a newly designed valve, but it also did not meet 

leak rate requirements. Project officials stated that a new valve design will 

not be manufactured in time for use in the first ISIM cryo-vacuum test. 

The project is concurrently developing three alternatives and authorized 

manufacturing for one of the alternatives in October 2012. Project officials 

stated that the MIRI cryo-cooler is particularly complex because it spans 

approximately 10 meters—or approximately 33 feet—through the entire 

JWST observatory. These issues combined required the use of 18 

months of schedule reserve, which reduced ISIM’s schedule reserve from 

the 26 months established in the replan to 8 months before it is needed 

for integration with the OTIS subsystem.

These types of issues are not uncommon among NASA programs as 

technical issues tend to arise when disparate parts are integrated and 

tested together for the first time. Given the complexity and cutting edge 

technology developed and used on JWST, it is expected that these kinds 

of issues will continue to materialize as the program moves through its 

complex integration and test program. Figure 7 shows the delay of 

instrument deliveries as well as changes to the ISIM integration and test 

and final delivery dates over the last year.
 

Page 24 GAO-13-4 James Webb Space Telescope

Figure 7: Changes in ISIM Schedule Since the Project Replan

Until the project is able to overcome the major issues with the instruments 

and other parts of the ISIM, it is likely that the schedule would continue to 

slip and may begin to affect the overall project schedule. ISIM still has 8 

months of schedule reserve before the slipping of its schedule would 

affect the schedule for the remainder of the project. The instrument, 

detector, and cryo-cooler issues have all contributed to the delay in the 

ISIM integration and test schedule and the reduction of objectives that 

can be achieved in the first two rounds of cryo-vacuum testing. The first 

round of testing will not include two instruments, a final design of the cryocooler hardware, or new detectors. As a result project officials will only be 

able to gather risk reduction information on the FGS/NIRISS, MIRI, test 

procedures, and test support equipment from the first cryo-vacuum test.

The project also has several known long term risks and challenges 

remaining. For example, risks related to OTIS, the sunshield, and the 
 

Page 25 GAO-13-4 James Webb Space Telescope

ground system subsystems are not scheduled to be addressed until late 

in project development. As of October 2012, seven of the top 10 project 

risks were related to the long-term risks associated with the OTIS and 

sunshield, most of which will not be resolved until 2016 or later. For 

example, several risks relating to OTE are not scheduled to be closed 

until the OTIS testing in the chamber at Johnson Space Center in 

February 2017. Project officials are adding risk mitigation through early 

and additional testing, where possible, to these subsystems. Prior to the 

replan, the ground system software was at high risk for not being

completed before launch and many tasks were planned for completion 

after launch. Space Telescope Science Institute officials stated that the 

replan allows them to plan for completion of their work before launch on a 

more realistic time schedule, which decreases schedule and operational 

risk. A continuing challenge on the ground system is that some 

development and testing is dependent on the final design of subsystems 

such as the instruments, which continue to slip delivery dates.

The project plans to hold independent and management reviews required 

for all projects during the integration and test phase, but this phase for 

JWST is particularly complex. JWST has five major integration and test 

efforts that span 7 years and only one independent mission-level 

technical review—the system integration review. The system integration 

review evaluates the readiness of the project and associated supporting 

infrastructure to begin system assembly, integration, and test, and 

evaluates whether the remaining project development can be completed 

within available resources. For JWST, this review is scheduled in 

September 2017, only 13 months prior to launch. Projects we reviewed 

that had recently launched, however, held their system integration review 

on average approximately 22 months prior to launch. The project has an 

internal review with participation from standing review board members 

planned before the beginning of OTIS integration and test activities begin, 

and it will be subject to independent lower level reviews conducted by the 

Goddard Systems Review Office of the integration and test process. In 

addition, key decision point D (KDP-D)—when the senior agency decision 

authority would approve the project to proceed into the system integration 

and test phase—is scheduled for December 2017, 3 months after the 

commencement of the final major integration and test activity. According 

to NASA policy, this review should be held prior to the start of the system 

Independent and 

Management Oversight 

during Test and Integration 

May Not Be

Commensurate with JWST 

Test and Integration 

Complexity 
 

Page 26 GAO-13-4 James Webb Space Telescope

integration and test phase of the project.18 Project budget information 

shows that over 90 percent of expected integration and test funding will 

be spent on four major integration and test activities prior to the 

scheduled mission-level system integration review and KDP-D approval 

by NASA senior management. As a result, the current plan may be 

inadequate to ensure that key technical and management issues are 

identified early enough to be addressed within the current integration and 

test phase schedule.

The JWST project has taken steps to improve communications and 

oversight of its contractors as part of the replanning activities. For 

example, based on recommendations from the ICRP, the project has 

instituted meetings at various levels throughout NASA and its contractors 

and subcontractors. In addition, the project has added personnel at 

contractor facilities, which has allowed for more direct interaction and 

quicker resolution of issues. The project also assumed responsibility of 

the mission-level systems engineering function from Northrop Grumman, 

a move that shifts the authority to make trades or decisions to NASA. An 

independent NASA review of the project conducted in May 2012 found, 

however, that agencywide reductions in travel budgets have put the 

effectiveness of the JWST project’s oversight plans in jeopardy. While the 

project received partial relief from travel budget reductions in fiscal year 

2012, project officials are concerned that the current level of oversight will 

not be sustained if similar cuts in travel funding occur in future years as 

anticipated. The project is also taking steps to enhance its oversight of 

project risks by implementing a new risk management system. The new 

project manager found that the previous system lacked rigor and was 

relatively ineffective for managing project risks, especially for a project as 

complex as JWST. The new system should allow for better tracking of 

risks than did the previous system. While these enhancements to the 

oversight of the project are steps in the right direction, it will take time to 

assess their effectiveness.

 

18NPR 7120.5E, table 2-5 and appendix A (Aug. 14, 2012).

JWST Project Has 

Taken Steps to 

Enhance 

Communications with 

and Oversight of Its 

Contractors 
 

Page 27 GAO-13-4 James Webb Space Telescope

Based on recommendations in the ICRP report, NASA has taken action to 

enhance oversight and communications. See table 2 for the ICRP 

recommendations and actions taken by NASA in response.

Table 2: ICRP Recommendations Concerning Oversight and Communications and the Actions Taken by NASA

ICRP recommendation Action taken

Move the JWST management and 

accountability from the Astrophysics 

Division to a new organizational entity at 

NASA headquarters that has responsibility 

only for the management and execution of 

JWST.

Completed: The management of JWST at Headquarters has been reorganized. JWST is 

now a stand-alone program reporting to the NASA Associate Administrator for 

programmatic oversight and to the Associate Administrator for the Science Mission 

Directorate for technical and analysis support. A similar approach was taken in the past 

with both the Hubble Space Telescope and the Mars Exploration Program. 

Assign management and execution 

responsibility for the JWST Project to the 

Goddard Center Director, with 

accountability to the Science Mission 

Directorate Associate Administrator at 

NASA headquarters.

Completed: NASA has reorganized the JWST project at Goddard to report directly to the 

Center Director, who is responsible for the management and execution of the JWST 

Project and accountable to the NASA Administrator. 

Assign at least one senior Goddard 

project person to be resident at Northrop 

Grumman throughout the Project.

In progress: In August 2012, the Deputy Observatory Manager began spending at least 

half time at the Northrop Grumman facility, with a permanent move scheduled in early 

2013. 

Improve communications between the 

JWST Project and both Goddard 

management and NASA headquarters 

Science Mission Directorate.

Completed: The JWST project office provides weekly updates of risks and new issues to 

the Goddard Center Director. In addition, the Project Manager at Goddard has daily 

interaction with the Program Manager at NASA HQ.

Conduct monthly or bi-monthly JWST 

Project Executive meetings, attended by 

the NASA Associate Administrator and the 

President of Northrop Grumman 

Aerospace Systems.

Completed: Meetings were first held in December 2010 between the NASA Associate 

Administrator, the new JWST Program Director, and the Goddard Center Director and 

senior management of the prime contractor. Since that time, senior level management 

reviews have occurred on a quarterly basis and are open to subcontractors as well. 

Topics at these meeting have included issues such as the spacecraft mass problems, 

instrument delivery delays, and the process to find alternative solutions for the MIRI cryocooler valve problem.

Strengthen the role and the independent 

voice of the science team in the Project.

Completed: The project added a Deputy Senior Project Scientist/Technical position to the 

project science team. This individual:

• is responsible for day-to-day interactions with senior project management on all 

aspects of the mission; scientific, technical, budgetary, and schedule.

• meets regularly with other members of the project science team to ensure better 

communication with other functions in the project.

• assists the Senior Project Scientist to facilitate better integration of the science 

activities with the hardware development activities and enable closer coordination 

and understanding of technical drivers to science performance so fully informed 

decisions can be made.

Source: GAO analysis of NASA data.

NASA Responded to 

Recent Independent 

Review Panel Report with 

Increased Communication 

and Oversight
 

Page 28 GAO-13-4 James Webb Space Telescope

NASA has taken steps to increase communication between the project 

and its contractors and subcontractors in an effort to enhance oversight. 

According to project officials, the increased communication has allowed 

them to better identify and manage project risks by having more visibility 

into contractors’ activities. The project reports that a great deal of 

communication existed across the project prior to the ICPR and replan; 

however, improvements have been made. For example, monthly 

meetings between project officials at Goddard and all of the contractors 

have continued on a regular basis and include half-day sessions devoted 

to business discussions. The project reports that these meetings have 

benefits over other forms of communication. For example, it was through 

dialogue with several technical leads at Northrop Grumman during 

detailed reviews of analytical models that the project identified that the 

mass issue on the spacecraft was likely to occur.

In addition, the project has increased its presence at contractor facilities 

as necessary to provide assistance with issues. For example, the project 

has had two engineers working on a recurring basis at Lockheed Martin 

to assist in solving problems with the NIRCam instrument. The ISIM 

manager said that these engineers have insight into Lockheed Martin’s 

work and are having a positive effect as they offer technical help and are 

involved in devising the solutions to issues. He added that that these 

engineers have authority to make decisions on routine issues to allow the 

work flow to continue, but decisions that are more complex or require a

commitment of funds are communicated to project management for 

disposition. The project reports that the Jet Propulsion Laboratory, 

responsible for NASA contribution to the MIRI instrument and its 

associated cryo-cooler, has an in-house representative in the responsible 

Northrop Grumman division to monitor the work being performed on the 

cryo-cooler.

The JWST project also assumed full responsibility for the mission system 

engineering functions from Northrop Grumman in March 2011. NASA and 

Northrop Grumman officials both said that NASA is better suited to 

perform these tasks. Project officials stated the systems engineering 

requires the ability to make trades and decisions across the entire 

observatory, and because Northrop Grumman is only responsible for 

portions of the observatory, it did not have the authority to make trades or 

decisions for areas outside of its control. Although responsibility for the 

overall mission systems engineering function was removed from Northrop 

Grumman, it retains system engineering responsibility for work still under 

its contract, such as development of the spacecraft and sunshield. The 

ICRP noted that a highly capable, experienced systems engineering 
 

Page 29 GAO-13-4 James Webb Space Telescope

group is fundamental to project success and appropriate to ensure 

accountability especially for a project of JWST’s complexity and visibility.

While these enhancements to the oversight of the project are steps in the 

right direction, it will take time to assess their effectiveness. In addition, 

sustainment of these efforts on the part of the project will be important. 

Project and contractor officials we spoke with believe that the increased 

communication has had a positive effect on the relationships between 

them. We will continue to monitor the interaction between the project and 

its contractors and its frequency in future reviews to identify whether the 

changes have had the desired results.

The JWST project reported that its travel budget was reduced by 

approximately $200,000 from the $1.2 million planned in fiscal year 2012 

as a result of NASA’s implementation of an Executive Order to promote 

more efficient spending.19

 

19Executive Order 13589, Promoting Efficient Spending, The White House, November 9, 

2011. NASA provided allocations to the Centers based on this order. In turn, Goddard 

conducted an internal travel prioritization process and center management determined the 

allocation for JWST and other Goddard projects. 

According to project officials, the changes in 

oversight necessitated by a reduction in travel funds represent a major 

shift away from the management paradigm adopted during the replan. 

Proposed reductions in future fiscal years could significantly reduce the 

project’s travel budget. The project reports that the travel requirements for 

fiscal years 2013 through 2015 are $1.6 million, $1.7 million, and $1.8 

million, respectively. Officials reported that while travel is a small 

percentage of the project’s annual budget, the majority of expected 

travel—about 87 percent—is for oversight functions put in place as a 

result of the ICRP recommendations, such as having a permanent on-site 

presence at Northrop Grumman. These oversight functions include 

attending and participating in contractor monthly programmatic and 

technical reviews, technical interface meetings, recurring on-site 

presence at contractor facilities for quality assurance reviews and 

inspection of hardware. JWST project officials are concerned that 

decreased oversight could translate into the project increasing its use of 

cost and schedule reserves as they will not be conducting planned 

oversight to better ensure success. A recent NASA Office of Evaluation 

review concluded that by not having an adequate travel budget, the 

project is at risk of cost/schedule growth and/or technical risk due to the 

Travel Budget Reductions 

May Hamper Planned 

Oversight Activities
 

Page 30 GAO-13-4 James Webb Space Telescope

late identification of issues or timely resolution strategies. The project has 

made adjustments to absorb the reduction in fiscal year 2012 and plans 

to identify instances of increased cost or schedule risk due to late 

identification of issues. However, the project does not have a strategy to 

address anticipated future reductions. Ensuring adequate oversight is 

particularly important as the project begins its complex and lengthy test 

and integration phase, where issues will likely surface.

As part of NASA’s approach to increase oversight of the project at 

headquarters, NASA’s Office of Evaluation recently conducted an 

independent review of the JWST project to assess the progress since the 

September 2011 rebaseline was approved. According to the Director of 

the Office of Evaluation, the goal of the review was not to reproduce the 

replan assessment, but rather to assess progress based on cost, 

schedule, and technical performance of the project and the status of 

oversight functions within NASA headquarters, the JWST Program Office, 

and Goddard Space Flight Center. The intended outcome of the review 

was 1) to obtain a snapshot of performance to determine if the program 

was progressing in accordance with its plan, and 2) to identify leading 

indicators for upper management to use when tracking future 

performance. The review team identified several areas of concern within 

the program, many of which we have highlighted, and recommended a list 

of leading indicators that project management should consider tracking.

The Director of the Office of Evaluation said that the project is generally 

performing the activities and maintaining the schedule set forth in the 

replan; however, the team identified key areas that should be monitored 

as the project moves forward. The review team also recommended a set 

of leading indicators for project management to consider tracking to 

measure and monitor progress. The Director added that these indicators 

are for the project to use and would not be specific criteria for use by 

independent review boards such as the Standing Review Board. These 

indicators are a positive step to ensure that NASA management has the 

information necessary to monitor the progress of the JWST project. See 

table 3 below for the concerns raised by the review team.

NASA Headquarters 

Independent Review of the 

JWST Program Identified 

Concerns and 

Recommends Indicators to 

Measure Progress
 

Page 31 GAO-13-4 James Webb Space Telescope

Table 3: NASA Office of Evaluation Review Team Concerns 

Concern identified NASA Office of Evaluation Review Team finding

Resolving the on-going mass issue.a The technical and programmatic risks involved in possible solutions need to be explicitly 

understood.

Understanding root causes for the 

NIRSpec Optical Bench cracking issue.

There needs to be a better understanding of the root causes and investigate what would 

constitute a flyable structure.

Monitoring of schedule erosion by the 

project and recognition of the dynamics 

of four competing critical paths.a

There are several competing critical paths for the JWST program to include development 

and integration and test of the OTE, ISIM, sunshield, and the spacecraft.

Monitoring the stability of the schedule 

going forward.

The master schedule took 8 months to baseline and stability of the schedule going forward 

is unknown. The project will need an accurate reporting system at the integrated master 

schedule level.

Ensuring that Northrop Grumman will 

have an adequate workforce and that 

skills are available in fiscal year 2013.a

The project needs to work with Northrop Grumman to ensure that it has an adequate 

workforce and skill sets available to accomplish the planned ramp up for fiscal year 2013.

Monitoring the increased risk of 

cost/schedule growth and technical risk.

Without an adequate travel budget to execute the original plan, there is an increased risk of 

cost or schedule growth and technical risks due to late identification of issues or timely 

resolution strategies.

Source: GAO analysis of NASA data.

a

These issues were also identified as leading indicators the review team recommended that project 

management should consider tracking.

The new JWST project manager re-emphasized the importance of the 

project’s risk management system and, in August 2012, a new risk 

management database was implemented to support the system. The 

project manager told us that he evaluated the risk management system 

being utilized by the project when he assumed his position and found it to 

be ineffective and not robust, especially for a project as complex as 

JWST. While the basic risk management methodology remains 

unchanged, the project manager wanted a more regimented system. For 

example, the project utilizes a hierarchy of risk boards that periodically 

reviews and provides disposition of all new and existing risks. These risk 

boards reviewed and assessed new risks and lower level risk board 

actions and met on an ad hoc basis. The project manager instituted a 

more regimented system that re-emphasized and revised the weekly 

project risk board meetings. Lower level risk boards meet a minimum of 

once a month depending on activity.

The project manager also determined that a new risk management 

database needed to be put in place that would bring more rigor to the risk 

management process. The project manager told us that he directed an 

overhaul of the risk management database to provide more complete 

information to management on the purpose and history for each risk. The 

Changes Made to Project 

Risk Management System
 

Page 32 GAO-13-4 James Webb Space Telescope

goal was to improve consistency in how the project determined the 

potential for a risk to occur and its impact, and provide greater detail on 

mitigation and better tracking of the status for each risk. For example, the 

new system puts more emphasis on understanding and capturing the key 

events in the mitigation plan that are intended to result in a change in 

likelihood or consequence of a risk. The new system has a provision 

where the mitigation plan will be entered and updated over time, and the 

capability to store data such as mitigation steps throughout the life of the 

risk. In addition, the new system now archives data automatically to 

provide a traceable history of the risk. The prior data system did not have 

as robust of an archiving function. Furthermore, the project manager 

wanted to improve the linkage between the risk database entries and 

financial records to ensure consistency of the data in the risk database 

with regard to cost and schedule for risk mitigations with project office 

financial records. As the changes to the risk management system and 

database, as well as other changes we identified that were put in place to 

enhance oversight were just recently implemented, we will continue to 

monitor their continued use and assess the impact they may be having on 

the project.

The JWST project is among the most challenging and high-risk projects 

NASA has pursued in recent years. It is also one of the most expensive, 

with a recent major replan resulting in a total cost of $8.8 billion. The 

reasons for cost and schedule growth were largely recognized by an 

independent review team to be rooted in ineffective funding, 

management, communication, and oversight. NASA has invested 

considerable time and resources replanning the project and instituting 

management and oversight improvements in order to ensure that it (1) 

can be executed within its new estimates and (2) has addressed the 

majority of issues raised in the recent independent review. It appears that 

communications with contractors and within NASA have improved, that a 

more robust risk mitigation system is in place, that more is known about 

what it will take to complete the project and how much it will cost, and that 

the project is currently meeting the majority of its milestones. 

Nevertheless, over the course of the next several years, the project will 

be executing a large amount of work with several extremely complex and 

challenging integration and test efforts. Because three major test and 

integration efforts must be completed in the last 2 years of the JWST 

schedule, it is essential that issues are identified and addressed early 

enough to be handled within the project’s current schedule. While the 

JWST oversight plan is consistent with NASA’s requirements for all 

project’s required reviews, a single independent review scheduled just 

Conclusions 
 

Page 33 GAO-13-4 James Webb Space Telescope

over a year before launch may not be sufficient to identify and resolve 

problems early for a project of this magnitude. A key element of 

overseeing project progress is monitoring how the project is executing to 

its cost baseline. To that end, while NASA took some steps that were in 

line with best practices to develop its revised baseline, some of the 

deficiencies we found in its process could impact the reliability of the cost 

estimate and the joint cost and schedule confidence level that was 

provided to headquarters decision-makers. Without higher-fidelity, 

regularly updated information related to costs, as well as an oversight 

regime during later phases of test and integration that is commensurate 

with the complexity of that effort, NASA risks late identification of 

technical and cost issues that could delay the launch of JWST and 

increase project costs beyond established baselines. Also important to 

oversight for the remainder of the project is the ability of officials to 

sustain improvements to communication with and oversight of

contractors. Anticipated travel restrictions, however, could decrease the 

project team’s ability to sustain these actions. Without a plan to address 

such reductions in future years, the project could once again become 

susceptible to communication and oversight problems identified in earlier 

reviews, which could also have a detrimental impact on continued project 

performance.

To ensure that the JWST life-cycle cost estimate conforms to best 

practices, GAO recommends that the NASA Administrator direct JWST 

officials to take the following three actions to provide high-fidelity cost 

information for monitoring project progress:

• improve cost estimate documentation and continually update it to 

reflect earned value management actual costs and record any 

reasons for variances,

• conduct a sensitivity analysis on the number of staff working on the 

program to determine how staff variations affect the cost estimate, 

and

• perform an updated integrated cost/schedule risk analysis, or joint 

cost and schedule confidence level analysis, using a schedule that 

meets best practices and includes enough detail so that risks can be 

appropriately mapped to activities and costs; historical, analogous 

data should be used to support the risk analysis.

To ensure that technical risks and challenges are being effectively 

managed and that sufficient oversight is in place and can be sustained, 

Recommendations for 

Executive Action
 

Page 34 GAO-13-4 James Webb Space Telescope

GAO recommends that the NASA Administrator direct JWST officials to 

take the following three actions:

• conduct a separate independent review prior to the beginning of the 

OTIS and spacecraft integration and test efforts to allow the project’s 

independent standing review board the opportunity to evaluate the 

readiness of the project to move forward, given the lack of schedule 

flexibility once these efforts are under way,

• schedule the management review and approval to proceed to 

integration and test (key decision point D or KDP-D) prior to the start 

of observatory integration and test effort, and

• devise an effective, long-term plan for project office oversight of its 

contractors that takes into consideration the anticipated travel budget 

reductions.

NASA provided written comments on a draft of this report. These 

comments are reprinted in appendix IV. NASA also provided technical 

comments, which were incorporated as appropriate.

In responding to a draft of this report, NASA concurred with three 

recommendations and partially concurred with three other 

recommendations and commented on actions in process or planned in 

response. In some cases, these actions meet the intent and are 

responsive to issues we raise; however, some of the responses do not 

fully address the issues we raised in the report.

NASA partially concurred with our recommendation to improve the cost 

estimate documentation of the JWST project, and to continually update it 

to reflect earned value management actual costs and record any reasons 

for variances between planned and actual costs. In response to this 

recommendation, NASA officials stated that the project currently receives 

earned value data from some of its contractors and performs monthly 

analysis of that data to understand the contractors’ estimates at 

completion, and then compares these numbers to similar figures 

independently assessed by the JWST project. NASA also highlighted its 

efforts to improve the agency’s documentation of the earned value 

variances and to extend the earned value management analysis to areas 

where it is not yet implemented, such as ground systems development at 

the Space Telescope Science Institute. In addition, NASA responded that 

its annual budget process generates a requirements-driven budget plan 

consistent with the rebaseline. NASA stated that this information is 

updated in several different documents that are provided to management 

Agency Comments 

and Our Evaluation
 

Page 35 GAO-13-4 James Webb Space Telescope

and it does not plan to revise its JCL documentation developed during the 

replan. Despite these steps, we could not independently confirm that they 

were leading to an updated cost estimate, which is the basis of our 

recommendation. If the estimate is not updated, it will be difficult to 

analyze changes in project costs and collecting cost and technical data to 

support future estimates will be hindered. Furthermore, if not properly 

updated on a regular basis, the cost estimate cannot provide decision 

makers with accurate information for assessing alternative decisions. 

Without a documented comparison between the current estimate 

(updated with actual costs) and the old estimate, the cost estimator 

cannot determine the level of variance between the two estimates and 

cannot see how the project is changing over time. Therefore, we continue 

to believe NASA will be well served by following best practices and 

updating its cost estimate with current information and documenting 

reasons for any variances. We encourage the project to improve the cost 

estimate documentation and record any reasons for variances between 

planned and actual costs and we intend to review the documentation as a 

part of our ongoing review of the project.

NASA officials partially concurred with our recommendation that the 

project conduct a sensitivity analysis on the number of staff working on 

the project to determine how staff variations affect the cost estimate. In its 

response, the agency stated that it believes it met the intent of this 

recommendation when staffing levels were determined in the 2011 JWST 

rebaseline based on programmatic experience from the accomplishment 

of similar activities. To accommodate the possibility of increased costs 

based on increased staffing hours, NASA reports that funded schedule 

reserve was built into the JWST rebaseline, in addition to unallocated 

future expenses being held at various levels of the organization. NASA 

believes that these reserves will be sufficient to cover increases for the 

duration of specific activities that result in increased staffing cost, and that 

an additional workforce sensitivity analysis is not warranted. NASA added 

that the joint cost and schedule confidence level analysis performed 

provided a de facto workforce sensitivity analysis and does not plan any 

further action. A joint cost and schedule confidence level analysis, 

however, is not the same as a sensitivity analysis wherein the sources of 

the workforce variation should be well documented and traceable. While 

we appreciate the steps NASA took to account for workforce variation, the 

JWST cost model does not show how staff levels increasing or 

decreasing over time affects cost. Furthermore, best practices call for a 

risk analysis to be conducted in conjunction with a sensitivity analysis, not 

to be a substitute for it. As a best practice, a sensitivity analysis should be 

included in all cost estimates because it examines the effects of changing 
 

Page 36 GAO-13-4 James Webb Space Telescope

assumptions and ground rules. Since uncertainty cannot be avoided, it is 

necessary to identify the cost elements that represent the most risk and, if 

possible, cost estimators should quantify the risk. Without performing a 

sensitivity analysis that reveals how the cost estimate is affected by a 

change in a single assumption, such as workforce size, the cost estimator 

will not fully understand which variable most affects the cost estimate. 

Therefore, we continue to believe that NASA should conduct a sensitivity 

analysis for the JWST project, given the large number of staff working on 

the program, to determine how staff variations positively or negatively 

affect the cost estimate rather than relying on schedule reserve and 

unallocated future expenses to offset any shortfall. 

NASA concurred with our recommendation to perform an updated 

integrated cost and schedule risk analysis using a schedule that meets 

best practices and includes enough detail so that risks can be 

appropriately mapped to activities and costs. In response to this 

recommendation, NASA stated that the agency is already using tools and 

a method to conduct programmatic assessments of projects after the 

baseline was established using the JCL methodology. While these may 

be good tools, the key point is the need to address shortcomings of the 

schedule that supports the baseline itself. For example, the lack of detail 

in the summary schedule used for the joint cost and schedule risk 

analysis prevented us from sufficiently understanding how risks were 

incorporated; therefore, we question the results of that analysis. Since the 

JCL was a key input to the decision process of approving the project’s 

new cost and schedule baseline estimates, we maintain that the JWST 

project should perform an updated JCL analysis using a schedule with

sufficient detail to map risks to activities and costs. Doing so could help 

increase the reliability of the cost estimate and the confidence level of the 

JCL. Furthermore, risk management is a continuous process that 

constantly monitors a project’s health. Given that JWST is many years 

from launch and the risks that the project faces are likely to change, a risk 

analysis should be conducted periodically throughout the life of the

project. 

NASA concurred with our recommendation to conduct a separate 

independent review prior to the beginning of the OTIS and spacecraft 

integration and test efforts. In response to this recommendation, NASA 

stated that it will request members of the independent JWST Standing 

Review Board participate in OTIS Pre-Environmental Review scheduled 

prior to the beginning of OTIS environmental testing. A member of the 

Standing Review Board will co-chair this review and report its findings to 

the NASA Associate Administrator, which is the practice of all Standing 
 

Page 37 GAO-13-4 James Webb Space Telescope

Review Board reviews. In addition, NASA plans to direct Northrop 

Grumman, the spacecraft developer, to add members of the Standing 

Review Board, as well as members of the Goddard Independent Review 

Team, to the spacecraft element integration readiness review and report 

their findings to the NASA Associate Administrator. We believe these 

actions meet the intent of our recommendation and will afford an 

independent evaluation of the readiness of the project to move forward 

with its major integration and test efforts. 

NASA partially concurred with our recommendation to schedule the 

management review and approval to proceed to integration and test 

(KDP-D) prior to the start of the observatory integration and test effort. In 

response to this recommendation, NASA stated that it will reduce the 3-

month gap between the scheduled system integration review and the 

KDP-D review, which it believes will provide NASA management and the 

NASA Associate Administrator with the full independent assessment 

earlier than currently planned. While we agree that this change will move 

the review earlier than previously planned, based on its response, NASA 

still plans to hold the review after the observatory integration and test is 

already underway. Holding this review after the observatory integration 

and test effort is already underway does not meet agency policy and will 

lessen the impact of the review as it may be inadequate to ensure key 

technical and management issues are identified early enough to be 

addressed. KDP-D is the point in which management approval is given to 

transition to the test and integration phase. We reiterate our 

recommendation that NASA should hold this important key decision point 

prior to the beginning of this last major integration and test effort, as 

required by agency policy. 

NASA concurred with our recommendation to devise an effective, longterm plan for project office oversight of its contractors that takes into 

consideration the anticipated travel budget reductions. In response to this 

recommendation, NASA stated that it will develop a plan based on fiscal 

year 2013 travel allocations and will take into consideration anticipated 

travel budget reductions. In addition, NASA stated that the plan will 

enable the project to maintain oversight of JWST contractors and their 

ability to meet performance and delivery deadlines and work closely with 

the international partners. We believe such a plan will be critical to 

ensuring adequate oversight, which is particularly important as the project 

enters into the complex integration and test efforts where issues will likely 

surface. In addition, we agree with the concerns of project officials that 

the current efforts to increase communication and oversight may not be 

sustained if reductions to future travel budgets occur as anticipated. We 
 

Page 38 GAO-13-4 James Webb Space Telescope

encourage the project to complete this plan in a timely manner and intend 

to review it as a part of our ongoing assessment of the project’s oversight 

efforts.

We will send copies of the report to NASA’s Administrator and interested 

congressional committees. We will also make copies available to others 

upon request. In addition, the report will be available at no charge on 

GAO’s web-site at http://www.gao.gov. 

Should you or your staff have any questions on matters discussed in this 

report, please contact me at (202) 512-4841 or chaplainc@gao.gov. 

Contact points for our Offices of Congressional Relations and Public 

Affairs may be found on the last page of this report. GAO staff who made 

major contributions to this report are listed in appendix V.

Cristina T. Chaplain

Director

Acquisition and Sourcing Management
 

Page 39 GAO-13-4 James Webb Space Telescope

List of Committees

The Honorable Barbara A. Mikulski

Chairwoman

The Honorable Kay Bailey Hutchison

Ranking Member

Subcommittee on Commerce, Justice, Science, and Related Agencies

Committee on Appropriations

United States Senate

The Honorable Frank R. Wolf

Chairman

The Honorable Chaka Fattah

Ranking Member

Subcommittee on Commerce, Justice, Science, and Related Agencies

Committee on Appropriations

House of Representatives

The Honorable Ralph M. Hall

Chairman

The Honorable Eddie Bernice Johnson

Ranking Member

Committee on Science, Space, and Technology

House of Representatives
Appendix I: Objectives, Scope, and 

Methodology

Page 40 GAO-13-4 James Webb Space Telescope

Our objectives were to assess (1) the extent to which NASA’s revised 

cost and schedule estimates are reliable based on GAO best practices, 

(2) the major risks and technological challenges the James Webb Space 

Telescope (JWST) project faces, and (3) the extent to which the National 

Aeronautics and Space Administration (NASA) has improved the 

oversight of the JWST project. In assessing the project’s cost and 

schedule estimates, we performed various checks to determine that the 

provided data were reliable enough for our purposes. Where we 

discovered discrepancies, we clarified the data accordingly. Where 

applicable, we confirmed the accuracy of NASA-generated data with 

multiple sources within NASA.

To assess the current life cycle cost estimate of the JWST project, we 

used the GAO cost guide to evaluate the estimating methodologies, 

assumptions, and results to determine whether the cost estimate was 

comprehensive, accurate, well-documented, and credible.1 We developed 

standardized data collection instruments to request relevant cost and 

schedule documentation and questionnaires for project officials to 

complete. Through the data collection instruments and questionnaires, we 

gathered basic information pertaining to the project’s cost and schedule 

estimation process. We examined documents from the JWST replan 

efforts pertaining to the revised cost estimate and the joint cost and 

schedule confidence level (JCL) analysis, including detailed spreadsheets 

that contained cost, schedule and project risk information. Project 

documents we reviewed included basis of estimates for the replan from 

contractors and the project office, contractor engineering change 

proposals, the JWST program plan and mission requirements, as well as 

the NASA cost estimating handbook.2

 

1

To assess how management and 

independent review teams were involved in the cost estimate process, we 

also reviewed the project’s monthly status presentations to Goddard 

management, Standing Review Board reports concerning the project’s 

revised baseline estimates, cost estimates conducted by NASA’s 

Independent Program Assessment Office, and the Agency Program 

Management Council’s decision memo for the rebaselined estimate and 

JCL analysis. We supplemented our analysis by interviewing officials from 

GAO-09-3SP. For the cost guide, GAO cost experts assessed measures consistently 

applied by cost-estimating organizations throughout the federal government and industry 

and considered best-practices for the development of reliable cost-estimates.

2

NASA, 2008 NASA Cost Estimating Handbook, (2008). 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 41 GAO-13-4 James Webb Space Telescope

Northrop Grumman, the Space Telescope Science Institute, and the 

JWST program and project offices. After reviewing cost estimate 

documentation submitted by NASA and conducting numerous interviews 

with relevant sources within the project office, we calculated the 

assessment rating of each criteria within the four characteristics by 

assigning each individual assessment rating: Not Met = 1, Minimally Met 

= 2, Partially Met = 3, Substantially Met = 4, and Met = 5. We then took 

the average of the individual assessment ratings for the criteria to 

determine the overall rating for each of the four characteristics. The 

resulting average becomes the “Overall Assessment” as follows: Not Met 

= 1.0 to 1.4, Minimally Met = 1.5 to 2.4, Partially Met = 2.5 to 3.4, 

Substantially Met = 3.5 to 4.4, and Met = 4.5 to 5.0. We discussed the 

results of our assessments with officials within the program office at 

NASA headquarters and the project office at Goddard Space Flight 

Center.

We supplemented the assessment of the revised 2011 cost estimate with 

an assessment of the summary schedule used for the JCL, which was a 

part of the project’s cost estimation process, and followed criteria laid out 

in the GAO schedule guide.3

 

3

These practices address whether the 

schedule (1) captured all activities; (2) sequenced all activities—that is, 

listed in the order in which they are to be carried out; (3) assigned 

resources to all activities; (4) established the duration of all activities; (5) 

integrated schedule activities horizontally and vertically, which identifies 

whether products and outcomes associated with other sequenced 

activities are arranged in the right order, and that varying levels of 

activities and supporting subactivities are also aligned properly; (6) 

established for all activities, the critical path, which is the longest 

continuous sequence of activities that is necessary to examine the effects 

of activities slipping in the schedule; (7) identified between activities float, 

which is the amount of time by which a predecessor activity can slip 

before the delay affects the program’s estimated finish date; (8) identified 

a level of confidence using a schedule risk analysis; and (9) was updated 

using logic and durations to determine dates. We also reviewed the inputs 

to the JCL model, the document outlining the methodology of the analysis 

that accompanied the electronic files, and interviewed cognizant project 

officials to discuss their use of the summary schedule.

GAO-12-120G. 
Appendix I: Objectives, Scope, and 

Methodology

Page 42 GAO-13-4 James Webb Space Telescope

Because the project’s detailed integrated master schedule has not been 

finalized because of ongoing negotiations and contract modifications, we 

did not conduct a complete schedule analysis using the GAO schedule 

assessment guide. We plan to perform this assessment in a subsequent 

review of the JWST project.

To assess the major short- and long-term risks and technological 

challenges facing the project, we reviewed the project’s risk list, monthly 

status reviews, and other documentation provided by projects and 

contractor officials. This information covered the risks, mitigation plans, 

and timelines for addressing risk and technological challenges. We also 

interviewed project officials for each major observatory subsystems to 

clarify information and to obtain additional information on risks and 

technological challenges. Further, we interviewed officials from the Jet 

Propulsion Laboratory, Northrop Grumman Aerospace Systems, 

Lockheed Martin Advanced Technology Company, Teledyne Imaging 

Sensors, the University of Arizona, and the Space Telescope Science 

Institute concerning risks and challenges on the subsystems, instruments, 

or components they were developing. We reviewed GAO’s prior work on 

NASA Large Scale Acquisitions, NASA Office of Inspector General 

reports, and NASA’s Space Flight Program and Project Management 

Requirements and Systems Engineering Processes and Requirements 

policy documents.4

To assess the extent to which NASA is performing enhanced oversight of 

the JWST project, we reviewed documentation from the Independent 

Comprehensive Review Panel and the project to determine actions taken 

by NASA in response to the panel’s recommendations. We interviewed 

project officials to understand the impact of these changes on the 

oversight processes for the project and communication between the 

project and its contractors. We also interviewed officials from the Jet 

Propulsion Laboratory, Northrop Grumman Aerospace Systems, 

Lockheed Martin Advanced Technology Company, Teledyne Imaging 

Sensors, the University of Arizona, and the Space Telescope Science 

We compared NASA’s controls as outlined in these 

agency policies with the project plan to assess the extent to which the 

JWST’s plan followed the intent of the policies with regard to independent 

oversight and management approval processes. 

 

4

NPR 7120.5E (Aug. 14, 2012) and NASA Procedural Requirements 7123.1A, NASA 

Systems Engineering Processes and Requirements with Change 1. (Nov.4, 2009). 
Appendix I: Objectives, Scope, and 

Methodology

Page 43 GAO-13-4 James Webb Space Telescope

Institute concerning project oversight of work they were performing and 

the effectiveness of oversight changes. In addition, we reviewed a 

presidential directive and Office of Management and Budget and project 

documentation and interviewed project officials concerning the reductions 

to travel budgets and their impact on project oversight activities. We 

interviewed the Director of NASA’s Office of Evaluation about a recent 

internal review of the JWST project and reviewed documentation from 

that review. We also reviewed documentation and interviewed project 

officials concerning the changes made to the project’s risk management 

system. 

Our work was performed primarily at NASA Headquarters in Washington, 

D.C., and Goddard Space Flight Center in Greenbelt, Maryland. We also 

visited Johnson Space Center in Houston, Texas, and the Jet Propulsion 

Laboratory in Pasadena, California. In addition, we met with 

representatives from Northrop Grumman Aerospace Systems, Lockheed 

Martin Advanced Technology Company, Teledyne Imaging Sensors, the 

University of Arizona, and the Space Telescope Science Institute.

We conducted this performance audit from February 2012 to December 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Subsystems of the JWST 

Observatory

Page 44 GAO-13-4 James Webb Space Telescope

Figure 8: Subsystems of the JWST: Interactive Information

Note: The hardware configuration created when the Optical Telescope Element and the Integrated 

Science Instrument Module are integrated, referred to as OTIS, is not considered a subsystem by 

NASA, but we categorize it as such for ease of discussion.

Appendix II: Subsystems of the JWST 

Observatory
Appendix III: Our Evaluation of JWST’s Cost 

Estimate Process

Page 45 GAO-13-4 James Webb Space Telescope

In determining that the National Aeronautics and Space Administration’s 

(NASA) processes for developing the James Webb Space Telescope 

(JWST) cost estimate do not fully comply with best practices, we 

evaluated the project’s cost estimation methods against our 2009 Cost 

Estimating and Assessment Guide. (See table 4.) We applied the 

following scale across the four categories of best practices:

• Not met: NASA provided no evidence that satisfies any portion of the 

criterion.

• Minimally met: NASA provided evidence that satisfies less than onehalf of the criterion.

• Partially met: NASA provided evidence that satisfies about one-half of 

the criterion.

• Substantially met: NASA provided evidence that satisfies more than 

one-half of the criterion.

• Met: NASA provided complete evidence that satisfies the entire 

criterion.

Table 4: Summary Assessment of JWST Cost Estimate Compared to Best Practices

Characteristic 

Overall 

assessment Best practice 

Individual 

assessment

Comprehensive Substantially met The cost estimate includes all life-cycle costs. Met

The cost estimate completely defines the program, reflects the 

current schedule, and is technically reasonable.

Substantially met

The cost estimate work breakdown structure is product oriented, 

traceable to the statement of work/objective, and at an appropriate 

level of detail to ensure that cost elements are neither omitted nor 

double counted.

Substantially met

The estimate documents all cost-influencing ground rules and 

assumptions.

Substantially met

Well documented Partially met The documentation should capture the source data used, the 

reliability of the data, and how the data were normalized.

Partially met

The documentation describes in sufficient detail the calculations 

performed and the estimating methodology used to derive each 

element’s cost.

Minimally met

The documentation describes step-by-step how the estimate was 

developed so that a cost analyst unfamiliar with the program could 

understand what was done and replicate it.

Minimally met

The documentation discusses the technical baseline description, and 

the data in the baseline are consistent with the estimate.

Substantially met

The documentation provides evidence that the cost estimate was 

reviewed and accepted by management.

Substantially met

Appendix III: Our Evaluation of JWST’s Cost 

Estimate Process
Appendix III: Our Evaluation of JWST’s Cost 

Estimate Process

Page 46 GAO-13-4 James Webb Space Telescope

Characteristic 

Overall 

assessment Best practice 

Individual 

assessment

Accurate Partially met The cost estimate results are unbiased, not overly conservative or 

optimistic and based on an assessment of most likely costs.

Partially met

The estimate has been adjusted properly for inflation. Partially met

The estimate contains few, if any, minor mistakes. Partially met

The cost estimate is regularly updated to reflect significant changes 

in the program so that it is always reflecting current status.

Partially met

Variances between planned and actual costs are documented, 

explained, and reviewed.

Minimally met

The estimate is based on a historical record of cost estimating and

actual experiences from other comparable programs.

Partially met

Credible Partially met The cost estimate includes a sensitivity analysis that identifies a 

range of possible costs based on varying major assumptions, 

parameters, and data inputs.

Minimally met

A risk and uncertainty analysis was conducted that quantified the 

imperfectly understood risks and identified the effects of changing 

key cost driver assumptions and factors.

Partially met

Major cost elements were cross-checked to see whether results 

were similar.

Minimally met

An independent cost estimate was conducted by a group outside the 

acquiring organization to determine whether other estimating 

methods produce similar results.

Partially met

Source: GAO analysis of NASA data. 
Appendix IV: Comments from the National 

Aeronautics and Space Administration

Page 47 GAO-13-4 James Webb Space Telescope

Appendix IV: Comments from the National 

Aeronautics and Space Administration
Appendix IV: Comments from the National 

Aeronautics and Space Administration

Page 48 GAO-13-4 James Webb Space Telescope
Appendix IV: Comments from the National 

Aeronautics and Space Administration

Page 49 GAO-13-4 James Webb Space Telescope
Appendix IV: Comments from the National 

Aeronautics and Space Administration

Page 50 GAO-13-4 James Webb Space Telescope
Appendix V: GAO Contact and Staff 

Acknowledgments

Page 51 GAO-13-4 James Webb Space Telescope

Cristina Chaplain, (202) 512-4841 or chaplainc@gao.gov

In addition to the contact named above, Shelby S. Oakley, Assistant 

Director; Karen Richey, Assistant Director; Richard A. Cederholm; Laura 

Greifner; Cheryl M. Harris; David Hulett; Jason Lee; Kenneth E. Patton; 

Sylvia Schatz; Stacey Steele; Roxanna T. Sun; Jay Tallon; and Jade A. 

Winfree made key contributions to this report.

Appendix V: GAO Contact and Staff 

Acknowledgments 

GAO Contact

Staff 

Acknowledgments 

(121048)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.