James Webb Space Telescope: Project Facing Increased Schedule Risk with Significant Work Remaining
Released: 2015-03-24T00:00:00

Letter

Project Risk Heightened Due to Widespread Use of Schedule Reserve to Address Technical Issues

The JWST project continues to report that it remains on schedule and budget with its overall schedule reserve currently above its plan. However, the project is now entering a difficult phase of development— integration and testing—which is expected to take another 3.5 years to complete. Maintaining as much schedule reserve as possible is critical during this phase to resolve known risks and unknown problems that may be discovered. Being one of the most complex projects in NASA’s history, significant risks lie ahead for the project, as it is during integration and testing where problems are likely to be found and as a result, schedules tend to slip. As seen in figure 1, only two of five elements and major subsystems—ISIM and OTE—have entered the integration and testing phase. Integration and testing for the spacecraft and sunshield and for the ISIM and OTE when they are integrated together begins in 2016 and the entire observatory will begin this phase in late 2017.

In December 2014, we reported that schedule risk was increasing for the project because it had lost schedule reserve across all elements and major subsystems. As a result, all were within weeks of becoming the critical path of the project and driving the project’s overall schedule. Figure 2 shows the different amounts of schedule reserve remaining on all elements and major subsystems, their proximity to the critical path, and the total schedule reserve for the critical path at the time of our review.

The proximity of all the elements and major subsystem schedules to the critical path means that a delay on any of the elements or major subsystems may reduce the overall project schedule reserve further, which could put the overall project schedule at risk. As a result, the project has less flexibility to choose which issues to mitigate. While the project has been able to reorganize work when necessary to mitigate schedule slips thus far, with further progression into subsequent integration and testing periods, flexibility will be diminished because work during integration and testing tends to be more serial, as the initiation of work is often dependent on the successful and timely completion of the prior work. This is particularly the case with JWST given its complexity.

Challenges with the development and manufacturing of the sunshield and the cryocooler were the most significant causes of the decline in schedule reserve that we reported on in December 2014. The sunshield experienced a significant manufacturing problem during the construction of the large composite panel that forms part of the sunshield’s primary support structure. The cryocooler compressor assembly—one component of the cryocooler—delivery is a top issue for the project and its development has required a disproportionate amount of cost reserves to fund additional work, caused in part, by issues such as a manufacturing error and manufacturing process mistake that caused delays to the schedule. The development of the cryocooler has been a concern for project officials as far back as 2006. Since that time, the cryocooler has faced a number of technical challenges, including valve leaks and cryocooler underperformance, which required two subcontract modifications and significant cost reserves to fund. The contractor and subcontractor were focused on addressing valve problems, which limited their attention to the cooling underperformance issue. This raised questions about the oversight of the cryocooler and why it did not get more attention sooner before significant delays occurred. In August 2013, the cryocooler subcontract was modified to reflect a 69 percent cost increase and that the workforce dedicated to the cryocooler effort at the subcontractor increased from 40 staff to approximately 110 staff.

Since we issued our December 2014 report, JWST schedule reserve continued to decline: project schedule reserve decreased by 1 month, leaving 10 months of schedule reserve remaining, and the critical path switched from the cryocooler to the ISIM. The project is facing additional challenges with the testing of the ISIM and OTE and the manufacturing of the spacecraft in addition to continuing challenges with the cryocooler compressor assembly that further demonstrates continued schedule risk for the project. For example, after the second test for the ISIM—the element of JWST that contains the telescope’s four different scientific instruments—electronic, sensor, and heat strap problems were identified that impact two of the four instruments. Mitigating some of these issues led to a 1.5-month slip to the ISIM schedule and made ISIM the current critical path of the project to allow officials time to replace the unusable and damaged parts. As a result, ISIM’s third and final cryovacuum test scheduled to begin in August 2015 has slipped until September 2015.

The OTE and spacecraft efforts are also experiencing challenges that may impact the schedules for those efforts. For example, it was discovered that over 70 harnesses on the OTE potentially had nicks on some wires and the majority will need to be repaired or rebuilt. The effects of these challenges on the project’s schedule are still being determined. Finally, the cryocooler compressor assembly has yet to be delivered and will be more than 16 months late if the current delivery date holds. Since our December 2014 report, the cryocooler compressor assembly’s delivery slipped almost an additional 2 months due to manufacturing and build issues and for an investigation of a leak to a joint with the pulse tube pre-cooler. Currently, the cryocooler compressor assembly is expected to be delivered in mid-June 2015 and is only 1 week off of the project’s critical path.

Entering fiscal year 2015, the JWST project had limited short-term cost reserves to address technical challenges and maintain schedule. We reported the project had committed approximately 40 percent of the fiscal year 2015 cost reserves before the start of the fiscal year. As a result, one of the project’s top issues for fiscal year 2015 is its cost reserve posture, which the project reported is less than desired and will require close monitoring. At the end of February, project officials had committed approximately 60 percent of the fiscal year 2015 cost reserves and noted that maintaining fiscal year 2016 reserves needed close watching.

Project Lacked Updated Cost Risk Analysis Despite Significant Changes to Risk

The types of technical problems JWST is experiencing are not unusual for a project that is unique and complex. They are an inherent aspect of pushing technological, design, and engineering boundaries. What is important when managing such a project is having a good picture of risks, which can shift from day to day, and having effective tools for mitigating risks as they surface. Using up-to-date and thorough data on risks is also integral to estimating resources needed to complete the project. Given the cost of JWST, its previous problems with oversight, and the fact that the program is entering its most difficult phases of development, risk analysis and risk management have been a key focus of our work.

JWST officials have taken an array of actions following the 2011 replan to enable the program to have better insight into risks and to mitigate them. For instance, we reported in 2012 that the project had implemented a new risk management system after it found the previous system lacked rigor and was ineffective for managing risks. The project instituted meetings at various levels throughout NASA and its contractors and subcontractors to facilitate communication about risks. The project also added personnel at contractor facilities, which allowed for more direct interaction and quicker resolution of issues.

However, we reported in December 2014 that neither NASA nor the prime contractor had updated the cost risk analysis that underpinned the cost and schedule estimates for the 2011 replan. A cost risk analysis quantifies the cost impacts of risks and should be used to develop and update a credible estimate that accounts for all possible risks—technical, programmatic, and those associated with budget and funding. Moreover, conditions have changed significantly since the replan. For example, the delivery of the cryocooler compressor assembly is one of the project’s top issues and was not an evident risk when the cost risk analysis was conducted in 2011. On the prime contract, our analysis found that 67 percent of risks tracked by Northrop Grumman in April 2014 at the time of our analysis were not present in September 2011 at the time of the replan.

We determined that a current and independent cost risk analysis was needed to provide Congress with insight into JWST’s remaining work on the Northrop Grumman prime contract—the largest (most expensive) portion of work remaining. A key reason for this determination was and continues to be the significant potential impact that any additional cost growth on JWST would have on NASA’s broader portfolio of science projects. To provide updated and current insight in to the project’s cost status, we took steps to conduct an independent, unbiased analysis.were, however, unable to conduct the analysis because Northrop Grumman did not allow us to conduct anonymous interviews of technical experts without a manager present. In order to collect unbiased data, interviewees must be assured that their opinions on risks and opportunities remain anonymous. Unbiased data would have allowed us to provide a credible assessment of risks for Northrop Grumman’s remaining work.

NASA and the JWST project disagreed that an independent cost risk analysis conducted by an outside organization at this point in the project would be useful. Neither believed that an organization external to NASA could fully comprehend the project’s risks. Further, they noted that any such analysis would be overly conservative due to the complexities of the risks and not representative of the real risk posture of the project. GAO’s best practices call for cost estimates to be compared to independent cost estimates in addition to being regularly updated. Without an independent and updated analysis, both the committee members’ and NASA’s oversight and management of JWST will be constrained since the impact of newer risks have not been reflected in key tools, including the cost estimate. Moreover, our methodology would have provided both NASA and Northrop Grumman with several opportunities to address concerns with our findings, including concerns about conservatism.

After we were unable to conduct the cost risk analysis, NASA decided to conduct its own cost risk analysis of the Northrop Grumman remaining work. However, a NASA project official said that they did not plan to use data from the cost risk analysis to manage the project. Instead, they indicated that they planned to use the information to inform committee members of the project’s cost risk and would continue to rely on other tools already in place to project the future costs of the project, such as earned value management (EVM) analysis. To maintain quality cost estimates over the life of a project, best practices state that cost risk analyses should be updated regularly to incorporate new risks and be used in conjunction with EVM analysis to validate cost estimates. While EVM is a very useful tool for tracking contractor costs and potential overruns, the analyses are based on past performance that do not reflect the potential impact of future risks. We reported that if the project did not follow best practices in conducting its cost risk analysis or use it to inform project management, the resulting information may be unreliable and may not substantively provide insight into JWST’s potential cost to allow either Congress or project officials to take any warranted action. To better ensure NASA’s efforts would produce a credible cost risk analysis, in December 2014, we recommended that officials follow best practices while conducting a cost risk analysis on the prime contract for the work remaining and update it as significant risks emerged. Doing so would ensure it provided information to effectively manage the program. NASA partially concurred with our recommendation, again noting that it has a range of tools in place to assess all contractors’ performance, the approach the project has in place is consistent with best practices, and officials will update the cost risk analysis again when required by NASA policy. We found that NASA best practices for cost estimating recommend updating the cost risk analysis while a project is being designed, developed, and tested, as changes can impact the estimate and the risk assessment.

Since our report was published, NASA completed its analysis and provided the results to us. We are currently examining the analysis to assess its quality and reliability and the extent to which it was done in accordance with NASA and GAO best practices. Our initial examination indicates the JWST project took the cost risk analysis seriously and took into account best practices in the execution of the analysis. The project has also recently begun conducting a new analysis of EVM data which they term a secondary estimate at completion analysis for two of its largest contractors–Northrop Grumman and Exelis—on work to go. This analysis should provide the project additional insight on the probabilities of outcomes while incorporating current risks against the cost reserves that remain. The initial analysis we have reviewed indicates that both contracts are forecasted to generally cost more at completion than the information produced using EVM analysis alone, but within the JWST life- cycle cost. However, we still have work to do to understand how NASA is analyzing the information and what assumptions it is putting into its analysis.

In conclusion, with approximately 3.5 years until launch, project officials have made much progress building and testing significant pieces of hardware and are currently on schedule—achieving important milestones—and on budget. They have also taken important steps to increase their insight and oversight into potential problems. What is important going forward is having good insight into risks and preserving as much schedule reserve as possible—particularly given the complexity of the project, the fact it is entering deeper into its integration and testing cycle, and the fact that it has limited funds available in the short term to preserve schedule. Any cost growth on JWST may have wider implications on NASA’s other major programs. While we are concerned about NASA’s reluctance to accept an independent cost risk assessment, particularly in light of past problems with oversight, we are also encouraged that NASA took steps to conduct an updated risk analysis of Northrop Grumman’s work and that NASA has sustained and enhanced its use of other tools to monitor and manage risk. As we undertake this year’s review of JWST, we will continue to focus on risk management, the use of cost reserves, progress with testing, as well as the extent to which its cost risk analysis followed best practices. We look forward to continuing to work with NASA on this important project and reporting to Congress on the results of our work.

Chairman Palazzo, Ranking Member Edwards, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to answer questions related to our work on JWST and acquisition best practices at this time.

GAO Contact and Staff Acknowledgments

For questions about this statement, please contact Cristina Chaplain at (202) 512-4841, or at chaplainc@gao.gov. Contact points for our Offices of Congressional Relationship and Public Affairs may be found on the last page of this statement. Individuals making key contributions to this testimony were Shelby Oakley, Assistant Director; Karen Richey, Assistant Director; Jason Lee, Assistant Director; Patrick Breiding; Laura Greifner; Silvia Porres; Carrie Rogers; Ozzy Trevino; and Sylvia Schatz.

Appendix I: Elements and Major Subsystems of the James Webb Space Telescope (JWST) Observatory

Related GAO Products

James Webb Space Telescope: Project Facing Increased Schedule Risk with Significant Work Remaining. GAO-15-100. Washington, D.C.: December 15, 2014.

James Webb Space Telescope: Project Meeting Commitments but Current Technical, Cost, and Schedule Challenges Could Affect Continued Progress. GAO-14-72. Washington, D.C.: January 8, 2014.

James Webb Space Telescope: Actions Needed to Improve Cost Estimate and Oversight of Test and Integration. GAO-13-4. Washington, D.C.: December 3, 2012.

NASA’s James Webb Space Telescope: Knowledge-Based Acquisition Approach Key to Addressing Program Challenges. GAO-06-634. Washington, D.C.: July 14, 2006.

NASA: Assessments of Selected Large-Scale Projects. GAO-15-320SP. Washington, D.C.: March 24, 2015.

NASA: Assessments of Selected Large-Scale Projects. GAO-14-338SP. Washington, D.C.: April 15, 2014.

NASA: Assessments of Selected Large-Scale Projects. GAO-13-276SP. Washington, D.C.: April 17, 2013.

NASA: Assessments of Selected Large-Scale Projects. GAO-12-207SP. Washington, D.C.: March 1, 2012.

NASA: Assessments of Selected Large-Scale Projects. GAO-11-239SP. Washington, D.C.: March 3, 2011.

NASA: Assessments of Selected Large-Scale Projects. GAO-10-227SP. Washington, D.C.: February 1, 2010.

NASA: Assessments of Selected Large-Scale Projects. GAO-09-306SP. Washington, D.C.: March 2, 2009.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.