James Webb Space Telescope: Project Facing Increased Schedule Risk with Significant Work Remaining
Published: 2015-03-24T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: JWST is one of the National Aeronautics and Space Administration's (NASA) most complex and expensive projects. At an anticipated cost of $8.8 billion, JWST is intended to revolutionize understanding of star and planet formation, advance the search for the origins of the universe, and further the search for earth-like planets. Since entering development in 1999, JWST has experienced significant schedule delays and increases to project costs and was rebaselined in 2011. With significant integration and testing planned during the approximately 3.5 years until the launch date in October 2018, the JWST project will need to address many challenges before NASA can conduct the science the telescope is intended to produce. GAO has reviewed JWST for the last 3 years as part of an annual mandate and for the last 7 years as part of another annual mandate to review all of NASA's major projects. Prior to this, GAO also issued a report on JWST in 2006.
This testimony is based on GAO's third annual report on JWST ( GAO-15-100 ), issued in December 2014, with limited updated information provided where applicable. That report assessed, among other issues, the extent to which (1) technical challenges were impacting the JWST project's ability to stay on schedule and budget, and (2) budget and cost estimates reflected current information about project risks. To conduct that work, GAO reviewed monthly JWST reports, interviewed NASA and contractor officials, reviewed relevant policies, and conducted independent analysis of NASA and contractor data.
section_title: What GAO Found
paragraphs: James Webb Space Telescope (JWST) project officials report that the effort remains on track toward the schedule and budget established in 2011. However, the project is now in the early stages of its extensive integration and testing period. Maintaining as much schedule reserve as possible during this phase is critical to resolve challenges that will likely surface and negatively impact the schedule. JWST has begun integration and testing for only two of five elements and major subsystems. While the project has been able to reorganize work when necessary to mitigate schedule slips thus far, this flexibility will diminish as work during integration and testing tends to be more serial, as initiating work is often dependent on the successful and timely completion of the prior work.
a The cryocooler chills an infrared light detector on one of JWST's four scientific instruments.
In December 2014, GAO reported that delays had occurred on every element and major subsystem schedule, each was at risk of driving the overall project schedule, and the project's schedule reserve had decreased from 14 to 11 months. As a result, further delays on any element or major subsystem would increase the overall schedule risk for the project. At the time of the report, challenges with manufacturing of the cryocooler had delayed that effort and it was the driver of the overall project schedule. Since the December report, the project's overall schedule reserve decreased to 10 months as a result of several problems that were identified following a test of the Integrated Science Instrument Module (ISIM), which contains the telescope's scientific instruments. ISIM is now driving the overall project schedule. Furthermore, additional schedule impacts associated with challenges on several other elements and major subsystems are still being assessed.
At the time of the December 2014 report, the JWST project and prime contractor's cost risk analyses used to validate the JWST budget were outdated and did not account for many new risks identified since 2011. GAO best practices for cost estimating call for regularly updating cost risk analyses to validate that reserves are sufficient to account for new risks. GAO recommended, among other actions, that officials follow best practices while conducting a cost risk analysis on the prime contract and update the analysis as significant risks emerged. NASA partially concurred, noting that it has a range of tools in place to assess performance and would update the analysis as required by policy. Since then, officials completed the analysis and GAO is currently examining the results.

section_title: Letter
paragraphs: Thank you for inviting me to testify on the current state of the National  Aeronautics and Space Administration’s (NASA) acquisition of the James  Webb Space Telescope (JWST). As you know, JWST is one of NASA’s  most complex and expensive projects, at an anticipated cost of $8.8  billion. JWST is intended to revolutionize understanding of star and planet  formation, advance the search for the origins of the universe, and further  the search for earth-like planets. Projects of this scale, complexity, and  technological sophistication confront a myriad of unforeseen challenges in  their ambitious attempts to expand the limits of scientific accomplishment.  With significant integration and testing planned until the scheduled launch  in October 2018, the success of JWST hinges on the ability of NASA and  its contractors to adjust and respond to these challenges in a timely and  cost-effective manner. With future NASA missions and exploration  depending on the success of JWST, its ultimate outcome will have effects  many years into the future.
We have reviewed JWST for the last 3 years as part of an annual  mandate and for the last 7 years as part of our mandate to assess all of  NASA’s major projects. Prior to this, we also issued a report on JWST in  2006. Information on all of our previous work on JWST is listed in the  back of this testimony. Over this time, we have seen much progress and  the project has addressed many technical challenges. Significant portions  of the telescope are built and being tested or are in manufacturing,  including the 18 primary mirrors, four scientific instruments, and parts of  the sunshield and spacecraft. Today, I will discuss a portion of our recent  work on the acquisition of the JWST. Specifically, I will discuss the extent  to which (1) technical challenges are impacting the JWST project’s ability  to stay on schedule and budget, and (2) budget and cost estimates reflect  current information about project risks.
My testimony today is based on our most recent report released in  December 2014, some limited updated information since the report was  published, and prior reports on JWST for background. For our most  recent report, we examined monthly status reports from the project and  contractors; conducted an analysis of JWST cost reserves; reviewed best  practices for cost risk analyses; and interviewed NASA project, program,  and contracting officials, among other actions. Our December report  includes a detailed description of our scope and methodology.
All work on which this testimony is based was performed in accordance  with generally accepted government auditing standards. Those standards  require that we plan and perform the audit to obtain sufficient, appropriate  evidence to provide a reasonable basis for our findings and conclusions  based on our audit objectives. We believe that the evidence obtained  provides a reasonable basis for our findings and conclusions based on  our audit objectives.
JWST is defined by three elements: the Integrated Science Instrument  Module (ISIM), the Optical Telescope Element (OTE), and the spacecraft  and two of its major subsystems—the cryocooler and sunshield. See  appendix I for a description of all the elements and major subsystems of  JWST. Projects managed out of Goddard Space Flight Center, like  JWST, are required to reserve extra time in their schedules—which is  referred to as schedule reserve—and extra money in their budgets— which is referred to as cost reserve—to address unforeseen technical   For  challenges or absorb additional costs as the project progresses.JWST, schedule reserve is held at the overall project level with some  reserve allocated to each element and major subsystem. The element or  major subsystem with the least amount of schedule reserve determines  the project’s critical path—the earliest completion date or minimum  duration it will take to complete all project activities. Any delay to an  activity that is on the critical path will reduce schedule reserve for the  whole project, and could ultimately impact the overall project schedule.
Since entering development in 1999, JWST has experienced significant  schedule delays and increases to project costs. Prior to being approved  for development, cost estimates of the project originally ranged from $1  billion to $3.5 billion with expected launch dates ranging from 2007 to  2011. In March 2005, NASA increased the JWST’s life-cycle cost  estimate to $4.5 billion and delayed the launch date to 2013. We reported  in 2006 that the cost growth was due to a delay in launch vehicle  selection, budget limitations in fiscal years 2006 and 2007, requirements  changes, and an increase in the project’s cost reserves. In April 2006, an  Independent Review Team confirmed that the project’s technical content  was complete and sound, but expressed concern over the project’s  reserve funding, reporting that it was too low and phased in too late in the  development life cycle. The review team reported that for a project as  complex as JWST, a 25 to 30 percent total reserve funding was  appropriate. The team cautioned that low reserve funding compromised  the project’s ability to resolve issues, address risk areas, and  accommodate unknown problems. The project was baselined in April  2009 with a life-cycle cost estimate of $4.964 billion—including additional  cost reserves—and a launch date in June 2014. Shortly after JWST was  approved for development and its cost and schedule estimates were  baselined, project costs continued to increase and the schedule was  extended.
In response to a request in 2010 from the Chair of the of the Senate  Subcommittee on Commerce, Justice, Science, and Related Agencies to  the NASA Administrator for another independent review of JWST, NASA  commissioned the Independent Comprehensive Review Panel (ICRP). In  October 2010, the ICRP issued its report and cited several reasons for  the project’s problems including management, budgeting, oversight,  governance and accountability, and communication issues. The panel  also identified changes to address root causes to diminish the risk of  future cost increases and delays to the launch date. The panel concluded  that JWST was executing well from a technical standpoint, but that  baseline funding did not reflect the most probable cost with adequate  reserves in each year of project execution, resulting in an unexecutable  project. Following this review, the JWST program underwent a replan in  September 2011. On the basis of the replan, NASA announced that the  project would be rebaselined with a life cycle cost of $8.835 billion—a 78  percent increase—and would launch in October 2018—a delay of an  additional 52 months. The replan included 13 months of funded schedule  reserve and a 66 percent joint cost and schedule confidence level (JCL)  for the revised cost and schedule baselines. A JCL is the process that  NASA uses to assign a percentage to the probable success of meeting  cost and schedule estimates as part of the project’s estimating process.  Currently, JWST makes up a significant portion—approximately 54  percent, or $6.2 billion—of NASA’s 2015 science portfolio’s total  development cost of $11.5 billion for major projects.
Since 2012, we have reported annually that JWST is on track with respect  to its new cost and schedule goals, but have raised questions about the  reliability of the project’s estimates. In 2012, we reported that the lack of  detail in the summary schedule used for JWST’s JCL analysis during the  2011 replan prevented us from sufficiently understanding how risks were  incorporated, calling into question the results of that analysis and,  therefore, the reliability of the replanned cost estimate. We recommended  that the JWST project perform an updated JCL analysis. NASA concurred  but, to date, has not updated the JCL. In our January 2014 report, our  analysis of three subsystem schedules determined that the reliability of  the project’s integrated master schedule—which is dependent on the  reliability of JWST’s subsystem schedules—was questionable. We  recommended that NASA perform schedule risk analyses on the three  specific schedules we examined and any others for which a schedule risk  analysis was not performed. NASA concurred and has performed  schedule risk analyses on two of the three schedules we examined.
subsections: section_title: Project Risk Heightened Due to Widespread Use of Schedule Reserve to Address Technical Issues
paragraphs: The JWST project continues to report that it remains on schedule and  budget with its overall schedule reserve currently above its plan.  However, the project is now entering a difficult phase of development— integration and testing—which is expected to take another 3.5 years to  complete. Maintaining as much schedule reserve as possible is critical  during this phase to resolve known risks and unknown problems that may  be discovered. Being one of the most complex projects in NASA’s history,  significant risks lie ahead for the project, as it is during integration and  testing where problems are likely to be found and as a result, schedules  tend to slip. As seen in figure 1, only two of five elements and major  subsystems—ISIM and OTE—have entered the integration and testing  phase. Integration and testing for the spacecraft and sunshield and for the  ISIM and OTE when they are integrated together begins in 2016 and the  entire observatory will begin this phase in late 2017.
In December 2014, we reported that schedule risk was increasing for the  project because it had lost schedule reserve across all elements and  major subsystems. As a result, all were within weeks of becoming the  critical path of the project and driving the project’s overall schedule.  Figure 2 shows the different amounts of schedule reserve remaining on  all elements and major subsystems, their proximity to the critical path,  and the total schedule reserve for the critical path at the time of our  review.
The proximity of all the elements and major subsystem schedules to the  critical path means that a delay on any of the elements or major  subsystems may reduce the overall project schedule reserve further,  which could put the overall project schedule at risk. As a result, the  project has less flexibility to choose which issues to mitigate. While the  project has been able to reorganize work when necessary to mitigate  schedule slips thus far, with further progression into subsequent  integration and testing periods, flexibility will be diminished because work  during integration and testing tends to be more serial, as the initiation of  work is often dependent on the successful and timely completion of the  prior work. This is particularly the case with JWST given its complexity.
Challenges with the development and manufacturing of the sunshield and  the cryocooler were the most significant causes of the decline in schedule  reserve that we reported on in December 2014. The sunshield  experienced a significant manufacturing problem during the construction  of the large composite panel that forms part of the sunshield’s primary  support structure. The cryocooler compressor assembly—one component  of the cryocooler—delivery is a top issue for the project and its  development has required a disproportionate amount of cost reserves to  fund additional work, caused in part, by issues such as a manufacturing  error and manufacturing process mistake that caused delays to the  schedule. The development of the cryocooler has been a concern for  project officials as far back as 2006. Since that time, the cryocooler has  faced a number of technical challenges, including valve leaks and  cryocooler underperformance, which required two subcontract  modifications and significant cost reserves to fund. The contractor and  subcontractor were focused on addressing valve problems, which limited  their attention to the cooling underperformance issue. This raised  questions about the oversight of the cryocooler and why it did not get  more attention sooner before significant delays occurred. In August 2013,  the cryocooler subcontract was modified to reflect a 69 percent cost  increase and that the workforce dedicated to the cryocooler effort at the  subcontractor increased from 40 staff to approximately 110 staff.
Since we issued our December 2014 report, JWST schedule reserve  continued to decline: project schedule reserve decreased by 1 month,  leaving 10 months of schedule reserve remaining, and the critical path  switched from the cryocooler to the ISIM. The project is facing additional  challenges with the testing of the ISIM and OTE and the manufacturing of  the spacecraft in addition to continuing challenges with the cryocooler  compressor assembly that further demonstrates continued schedule risk  for the project. For example, after the second test for the ISIM—the  element of JWST that contains the telescope’s four different scientific  instruments—electronic, sensor, and heat strap problems were identified  that impact two of the four instruments. Mitigating some of these issues  led to a 1.5-month slip to the ISIM schedule and made ISIM the current  critical path of the project to allow officials time to replace the unusable  and damaged parts. As a result, ISIM’s third and final cryovacuum test  scheduled to begin in August 2015 has slipped until September 2015.
The OTE and spacecraft efforts are also experiencing challenges that  may impact the schedules for those efforts. For example, it was  discovered that over 70 harnesses on the OTE potentially had nicks on  some wires and the majority will need to be repaired or rebuilt. The  effects of these challenges on the project’s schedule are still being  determined. Finally, the cryocooler compressor assembly has yet to be  delivered and will be more than 16 months late if the current delivery date  holds. Since our December 2014 report, the cryocooler compressor  assembly’s delivery slipped almost an additional 2 months due to  manufacturing and build issues and for an investigation of a leak to a joint  with the pulse tube pre-cooler. Currently, the cryocooler compressor  assembly is expected to be delivered in mid-June 2015 and is only 1  week off of the project’s critical path.
Entering fiscal year 2015, the JWST project had limited short-term cost  reserves to address technical challenges and maintain schedule. We  reported the project had committed approximately 40 percent of the fiscal  year 2015 cost reserves before the start of the fiscal year. As a result,  one of the project’s top issues for fiscal year 2015 is its cost reserve  posture, which the project reported is less than desired and will require  close monitoring. At the end of February, project officials had committed  approximately 60 percent of the fiscal year 2015 cost reserves and noted  that maintaining fiscal year 2016 reserves needed close watching.
section_title: Project Lacked Updated Cost Risk Analysis Despite Significant Changes to Risk
paragraphs: The types of technical problems JWST is experiencing are not unusual for  a project that is unique and complex. They are an inherent aspect of  pushing technological, design, and engineering boundaries. What is  important when managing such a project is having a good picture of risks,  which can shift from day to day, and having effective tools for mitigating  risks as they surface. Using up-to-date and thorough data on risks is also  integral to estimating resources needed to complete the project. Given  the cost of JWST, its previous problems with oversight, and the fact that  the program is entering its most difficult phases of development, risk  analysis and risk management have been a key focus of our work.
JWST officials have taken an array of actions following the 2011 replan to  enable the program to have better insight into risks and to mitigate them.  For instance, we reported in 2012 that the project had implemented a new  risk management system after it found the previous system lacked rigor  and was ineffective for managing risks. The project instituted meetings at  various levels throughout NASA and its contractors and subcontractors to  facilitate communication about risks. The project also added personnel at  contractor facilities, which allowed for more direct interaction and quicker  resolution of issues.
However, we reported in December 2014 that neither NASA nor the prime  contractor had updated the cost risk analysis that underpinned the cost  and schedule estimates for the 2011 replan. A cost risk analysis  quantifies the cost impacts of risks and should be used to develop and  update a credible estimate that accounts for all possible risks—technical,  programmatic, and those associated with budget and funding. Moreover,  conditions have changed significantly since the replan. For example, the  delivery of the cryocooler compressor assembly is one of the project’s top  issues and was not an evident risk when the cost risk analysis was  conducted in 2011. On the prime contract, our analysis found that 67  percent of risks tracked by Northrop Grumman in April 2014 at the time of  our analysis were not present in September 2011 at the time of the  replan.
We determined that a current and independent cost risk analysis was  needed to provide Congress with insight into JWST’s remaining work on  the Northrop Grumman prime contract—the largest (most expensive)  portion of work remaining. A key reason for this determination was and  continues to be the significant potential impact that any additional cost  growth on JWST would have on NASA’s broader portfolio of science  projects. To provide updated and current insight in to the project’s cost  status, we took steps to conduct an independent, unbiased analysis.were, however, unable to conduct the analysis because Northrop  Grumman did not allow us to conduct anonymous interviews of technical  experts without a manager present. In order to collect unbiased data,  interviewees must be assured that their opinions on risks and  opportunities remain anonymous. Unbiased data would have allowed us  to provide a credible assessment of risks for Northrop Grumman’s  remaining work.
NASA and the JWST project disagreed that an independent cost risk  analysis conducted by an outside organization at this point in the project  would be useful. Neither believed that an organization external to NASA  could fully comprehend the project’s risks. Further, they noted that any  such analysis would be overly conservative due to the complexities of the  risks and not representative of the real risk posture of the project. GAO’s  best practices call for cost estimates to be compared to independent cost  estimates in addition to being regularly updated. Without an independent  and updated analysis, both the committee members’ and NASA’s  oversight and management of JWST will be constrained since the impact  of newer risks have not been reflected in key tools, including the cost  estimate. Moreover, our methodology would have provided both NASA  and Northrop Grumman with several opportunities to address concerns  with our findings, including concerns about conservatism.
After we were unable to conduct the cost risk analysis, NASA decided to  conduct its own cost risk analysis of the Northrop Grumman remaining  work. However, a NASA project official said that they did not plan to use  data from the cost risk analysis to manage the project. Instead, they  indicated that they planned to use the information to inform committee  members of the project’s cost risk and would continue to rely on other  tools already in place to project the future costs of the project, such as  earned value management (EVM) analysis. To maintain quality cost  estimates over the life of a project, best practices state that cost risk  analyses should be updated regularly to incorporate new risks and be  used in conjunction with EVM analysis to validate cost estimates. While  EVM is a very useful tool for tracking contractor costs and potential  overruns, the analyses are based on past performance that do not reflect  the potential impact of future risks. We reported that if the project did not  follow best practices in conducting its cost risk analysis or use it to inform  project management, the resulting information may be unreliable and may  not substantively provide insight into JWST’s potential cost to allow either  Congress or project officials to take any warranted action. To better  ensure NASA’s efforts would produce a credible cost risk analysis, in  December 2014, we recommended that officials follow best practices  while conducting a cost risk analysis on the prime contract for the work  remaining and update it as significant risks emerged. Doing so would  ensure it provided information to effectively manage the program. NASA  partially concurred with our recommendation, again noting that it has a  range of tools in place to assess all contractors’ performance, the  approach the project has in place is consistent with best practices, and  officials will update the cost risk analysis again when required by NASA  policy. We found that NASA best practices for cost estimating  recommend updating the cost risk analysis while a project is being  designed, developed, and tested, as changes can impact the estimate  and the risk assessment.
Since our report was published, NASA completed its analysis and  provided the results to us. We are currently examining the analysis to  assess its quality and reliability and the extent to which it was done in  accordance with NASA and GAO best practices. Our initial examination  indicates the JWST project took the cost risk analysis seriously and took  into account best practices in the execution of the analysis. The project  has also recently begun conducting a new analysis of EVM data which  they term a secondary estimate at completion analysis for two of its  largest contractors–Northrop Grumman and Exelis—on work to go. This  analysis should provide the project additional insight on the probabilities  of outcomes while incorporating current risks against the cost reserves  that remain. The initial analysis we have reviewed indicates that both  contracts are forecasted to generally cost more at completion than the  information produced using EVM analysis alone, but within the JWST life- cycle cost. However, we still have work to do to understand how NASA is  analyzing the information and what assumptions it is putting into its  analysis.
In conclusion, with approximately 3.5 years until launch, project officials  have made much progress building and testing significant pieces of  hardware and are currently on schedule—achieving important  milestones—and on budget. They have also taken important steps to  increase their insight and oversight into potential problems. What is  important going forward is having good insight into risks and preserving  as much schedule reserve as possible—particularly given the complexity  of the project, the fact it is entering deeper into its integration and testing  cycle, and the fact that it has limited funds available in the short term to  preserve schedule. Any cost growth on JWST may have wider  implications on NASA’s other major programs. While we are concerned  about NASA’s reluctance to accept an independent cost risk assessment,  particularly in light of past problems with oversight, we are also  encouraged that NASA took steps to conduct an updated risk analysis of  Northrop Grumman’s work and that NASA has sustained and enhanced  its use of other tools to monitor and manage risk. As we undertake this  year’s review of JWST, we will continue to focus on risk management, the  use of cost reserves, progress with testing, as well as the extent to which  its cost risk analysis followed best practices. We look forward to  continuing to work with NASA on this important project and reporting to  Congress on the results of our work.
Chairman Palazzo, Ranking Member Edwards, and Members of the  Subcommittee, this completes my prepared statement. I would be  pleased to answer questions related to our work on JWST and acquisition  best practices at this time.
section_title: GAO Contact and Staff Acknowledgments
paragraphs: For questions about this statement, please contact Cristina Chaplain at  (202) 512-4841, or at chaplainc@gao.gov. Contact points for our Offices  of Congressional Relationship and Public Affairs may be found on the last  page of this statement. Individuals making key contributions to this  testimony were Shelby Oakley, Assistant Director; Karen Richey,  Assistant Director; Jason Lee, Assistant Director; Patrick Breiding; Laura  Greifner; Silvia Porres; Carrie Rogers; Ozzy Trevino; and Sylvia Schatz.
section_title: Appendix I: Elements and Major Subsystems of the James Webb Space Telescope (JWST) Observatory
section_title: Related GAO Products
paragraphs: James Webb Space Telescope: Project Facing Increased Schedule Risk  with Significant Work Remaining. GAO-15-100. Washington, D.C.:  December 15, 2014.
James Webb Space Telescope: Project Meeting Commitments but  Current Technical, Cost, and Schedule Challenges Could Affect  Continued Progress. GAO-14-72. Washington, D.C.: January 8, 2014.
James Webb Space Telescope: Actions Needed to Improve Cost  Estimate and Oversight of Test and Integration. GAO-13-4. Washington,  D.C.: December 3, 2012.
NASA’s James Webb Space Telescope: Knowledge-Based Acquisition  Approach Key to Addressing Program Challenges. GAO-06-634.  Washington, D.C.: July 14, 2006.
NASA: Assessments of Selected Large-Scale Projects. GAO-15-320SP.  Washington, D.C.: March 24, 2015.
NASA: Assessments of Selected Large-Scale Projects. GAO-14-338SP.  Washington, D.C.: April 15, 2014.
NASA: Assessments of Selected Large-Scale Projects. GAO-13-276SP.  Washington, D.C.: April 17, 2013.
NASA: Assessments of Selected Large-Scale Projects. GAO-12-207SP.  Washington, D.C.: March 1, 2012.
NASA: Assessments of Selected Large-Scale Projects. GAO-11-239SP.  Washington, D.C.: March 3, 2011.
NASA: Assessments of Selected Large-Scale Projects. GAO-10-227SP.  Washington, D.C.: February 1, 2010.
NASA: Assessments of Selected Large-Scale Projects. GAO-09-306SP.  Washington, D.C.: March 2, 2009.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.