NASA: Preliminary Observations on the Management of Space Telescopes
Released: 2017-12-06T00:00:00

Letter

Background

NASA’s mission is to drive advances in science, technology, aeronautics, and space exploration, and contribute to education, innovation, our country’s economic vitality, and the stewardship of the Earth. To accomplish this mission, NASA establishes programs and projects that rely on complex instruments and spacecraft. NASA’s portfolio of major projects ranges from space satellites equipped with advanced sensors to study the Earth to a telescope intended to explore the universe to spacecraft to transport humans and cargo to and beyond low-Earth orbit. Some of NASA’s projects are expected to incorporate new and sophisticated technologies that must operate in harsh, distant environments.

The life cycle for NASA space flight projects consists of two phases— formulation, which takes a project from concept to preliminary design, and implementation, which includes building, launching, and operating the system, among other activities. NASA further divides formulation and implementation into phase A through phase F. Major projects must get approval from senior NASA officials at key decision points before they can enter each new phase. Figure 1 depicts NASA’s life cycle for space flight projects.

Formulation culminates in a review at key decision point C, known as project confirmation, where cost and schedule baselines are established and documented in a decision memorandum. To inform those baselines, each project with a life-cycle cost estimated to be greater than $250 million must also develop a joint cost and schedule confidence level (JCL). The JCL initiative, adopted in January 2009, is a point-in-time estimate that, among other things, includes all cost and schedule elements, incorporates and quantifies known risks, assesses the impacts of cost and schedule to date, and addresses available annual resources. NASA policy requires that projects be baselined and budgeted at the 70 percent confidence level.

The agency baseline commitment established at key decision point C includes cost and schedule reserves held at the project—those within the project manager’s control—and NASA headquarters level. Cost reserves are for costs that are expected to be incurred—for instance, to address project risks—but are not yet allocated to a specific part of the project. Schedule reserves are extra time in project schedules that can be allocated to specific activities, elements, and major subsystems to mitigate delays or address unforeseen risks.

Status of NASA’s Major Telescope Projects

NASA’s current portfolio of major space telescopes includes three projects—WFIRST, TESS, and JWST—that vary in cost, complexity, and phase of the acquisition life cycle. WFIRST, a project that entered the concept and technology development phase and established preliminary cost and schedule estimates in February 2016, is in the earliest stages of the acquisition life cycle. With preliminary cost estimates ranging from $3.2 billion to $3.8 billion, this project is an observatory designed to perform wide-field imaging and survey of the sky at near-infrared wavelengths to answer questions about the structure and evolution of the universe and to expand our knowledge of planets beyond our solar system. The current design includes a 2.4 meter telescope that was built and qualified for another federal agency over 10 years ago; the project is evaluating which components to reuse and which to modify, refurbish, or build new. TESS—a smaller project whose latest cost estimate is approximately $337 million—is targeted to launch in March 2018 and will be used to conduct the first extensive survey of the sky from space for transiting exoplanets.

And finally, JWST, with a life-cycle cost estimate of $8.835 billion, is one of NASA’s most complex projects and top priorities. The telescope is designed to help understand the origin and destiny of the universe, the creation and evolution of the first stars and galaxies, and the formation of stars and planetary systems. With a 6.5-meter primary mirror, JWST is expected to operate at about 100 times the sensitivity of the Hubble Space Telescope. JWST’s science instruments are to detect very faint infrared sources and, as such, are required to operate at extremely cold temperatures. To help keep these instruments cold, a multi-layered tennis-court-sized sunshield is being developed to protect the mirrors and instruments from the sun’s heat.

We have reported for several years on the JWST project, which has experienced significant cost increases and schedule delays. Prior to being approved for development, cost estimates for JWST ranged from $1 billion to $3.5 billion, with expected launch dates ranging from 2007 to 2011. Before 2011, early technical and management challenges, contractor performance issues, low levels of cost reserves, and poorly phased funding levels caused JWST to delay work after confirmation, which contributed to significant cost and schedule overruns, including launch delays. The Chair of the Senate Subcommittee on Commerce, Justice, Science, and Related Agencies requested from NASA an independent review of JWST in June 2010. In response, NASA commissioned the Independent Comprehensive Review Panel, which issued its report in October 2010. The panel concluded that JWST was executing well from a technical standpoint, but that the baseline cost estimate did not reflect the most probable cost with adequate reserves in each year of project execution, resulting in an unexecutable project.

Following this review, Congress in November 2011 placed an $8 billion cap on the formulation and development costs for the project and NASA rebaselined JWST with a life-cycle cost estimate of $8.835 billion that included additional money for operations and a planned launch in October 2018. The new baseline represented a 78 percent increase to the project’s life-cycle cost from the original baseline and a launch date in October 2018, a delay of 52 months. The revised life-cycle cost estimate included a total of 13 months of funded schedule reserve.

Our ongoing work indicates that these three projects are each making progress in line with their phase of the acquisition cycle, but also face challenges in execution. Some of these challenges are unique to the projects themselves and some are common among the projects in NASA’s portfolio. For example, when projects enter the integration and test phase, unforeseen challenges can arise and affect the cost and schedule for the project. Table 1 provides more details about the current acquisition phase, cost, and schedule status of NASA’s major space telescope projects based on our ongoing work.

WFIRST. NASA’s preliminary cost and schedule estimates for the WFIRST project are currently under review as the project responds to findings in the WFIRST Independent External Technical/Management/Cost Review. This independent review was conducted to ensure the mission’s scope and required resources are well understood and executable. NASA initiated this review in April 2017 to address the National Academies’ concerns that WFIRST cost growth could endanger the balance of NASA’s astrophysics program and negatively affect other scientific priorities. The review found that the mission scope is understood, but not aligned with the resources provided and concluded that the mission is not executable without adjustments and/or additional resources. For example, the study team found that NASA’s current forecasted funding profile for the WFIRST project would require the project to slow down activities starting in fiscal year 2020, which would result in an increase in development cost and schedule. NASA agreed with the study team’s results and directed the project to reduce the cost and complexity of the design in order to maintain costs within the $3.2 billion cost target.

The project is currently identifying potential ways to reduce the scope of planned activities (called “descopes”), assessing the science impact of those descopes, and then developing recommendations for the Astrophysics Division leadership. An example of a descope that may be considered is the requirement for WFIRST to be “star-shade ready,” which means the design must be compatible with a star-shade device that is positioned between it and the star being observed to block out starlight while allowing the light emitted by the planet through.

TESS. The TESS project is currently holding cost and schedule reserves consistent with NASA center requirements, but there are no longer headquarters-held cost reserves to cover a delay if the project cannot launch as planned in March 2018. According to a project official, the project is holding 16 days of schedule reserve to its target March 2018 launch readiness date, which includes 6 days for the completion of integration and test, and 10 days for launch operations. The project previously used schedule reserves to accommodate the delayed delivery of its Ka-band transmitter, which is essential for TESS as it transmits the mission data back to Earth, due to continued performance and manufacturing issues. The two main risks to the March 2018 launch date are if: 1) SpaceX requires additional time past December 2017 for NASA’s Launch Services Program to certify that TESS can fly on its upgraded launch vehicle—certification is necessary because it will be the first time that NASA will use this version of the vehicle—and 2) any issues are identified during the remainder of environmental testing.

The project is also conducting additional testing on its spare camera at temperatures seen in space to better understand expected camera performance on orbit. TESS will use four identical, wide field-of-view cameras to conduct the first extensive survey of the sky from space for transiting exoplanets. However, during thermal testing, the project found that the substance attaching the lenses to the camera barrel places pressure on the lenses and causes the cameras to be slightly out of focus. In June 2017, NASA directed the project to proceed with integrating the cameras—as they are expected to meet TESS’s top level science requirements even with the anomaly. At its most recent key decision review in August 2017, NASA reallocated $15 million of TESS’s headquarters-held reserves to the WFIRST project. While this had the effect of decreasing life cycle costs for TESS, it also increased risk as the project no longer has any additional headquarters-held cost reserves to cover a launch delay past March 2018.

JWST. The JWST project continues to make progress towards launch, but the program is encountering technical challenges that require both time and money to fix and may lead to additional delays, beyond a delay recently announced. While the project has made much progress on hardware integration and testing over the past several months, it also used all of its remaining schedule reserves to address various technical issues, particularly on the spacecraft element. In September 2017, the JWST project requested from the European Space Agency—who will contribute the Ariane V launch vehicle—a launch window from March to June 2019, or 5 to 8 months later than the planned October 2018 launch readiness date, established in 2011. The project based this request on the results of a schedule risk assessment that incorporated inputs from the contractor on expected durations of ongoing spacecraft element integration work and other challenges that were expected to increase schedule.

With the later launch window to June 2019, the project expected to have up to 4 months of new schedule reserves. However, shortly after requesting the revised launch window, the project learned from its contractor that up to another 3 months of schedule reserve use is likely, due to lessons learned from conducting deployment exercises of the sunshield, such as reach and access limitations on the flight hardware. As a result, and pending further examination of the schedule, the project now has approximately one month of schedule reserve to complete environmental testing of the spacecraft element and the final integration phase. The final integration phase is where the instruments and telescope will be integrated with the spacecraft and sunshield to form the completed observatory. As I previously noted, our work has shown the integration and test is the riskiest phase of development, where problems are most likely to be found and schedules slip. Given the risks associated with the integration and test work ahead, coupled with a level of schedule reserves that is currently well below the level stated in the procedural requirements issued by the NASA center responsible for managing JWST, additional delays to the project’s revised launch readiness date of June 2019 are likely. As a result, the funding available under the Congressional cost cap of $8 billion may be inadequate as the contractor will need to continue to retain higher workforce levels for longer than expected to prepare the mission for a delayed launch.

Lessons Learned from NASA Acquisitions

As Congress, NASA, and the science community consider future telescope efforts, it will be exceedingly important to shape and manage new programs in a manner that minimizes cost overruns and schedule delays. This is particularly important for the largest programs as even small cost increases can have reverberating effects. NASA’s telescope and other science projects will always have inherent technical, design, and integration risks because they are complex, specialized, and often push the state of the art in space technology. But too often, our reports find that management and oversight problems—which can include poor planning, optimistic cost estimating, funding gaps, lax oversight, and poor contractor performance, among other issues—are the real drivers behind cost and schedule growth.

To its credit, NASA has taken significant steps, partly in response to our past recommendations, to reduce acquisition risk from both a technical and management standpoint, including actions to enhance cost and schedule estimating, provide adequate levels of reserves to projects, establish better processes and metrics to monitor projects, and expand the use of earned value management to better monitor contractor performance. For example, in November 2012, we found that NASA employee skill sets available to analyze and implement earned value management vary widely from center to center, and we recommended that NASA conduct an earned value management skills gap analysis to identify areas requiring augmented capability across the agency, and, based on the results of the assessment, develop a workforce training plan to address any deficiencies. NASA concurred with this recommendation and developed an earned value management training plan in 2014 based on the results of an earned value management skills gap analysis that was conducted in 2013. Moreover, in recent years, we have found that many of the projects within the agency’s major project portfolio have improved their cost and schedule performance. Nevertheless, the extent to which NASA has adopted some of the following lessons learned within its portfolio of major projects is mixed, and NASA has an opportunity to strengthen its program management of major acquisitions, including its space telescopes, by doing so.

Manage Cost and Schedule Performance for Large Projects to Limit Implications for Entire Portfolio. In 2013, following JWST’s cost increases and schedule growth, we found that though cost and schedule growth can occur on any project, increases associated with NASA’s most costly and complex missions can have cascading effects on the rest of the portfolio. For example, we found that the JWST cost growth would have reverberating effects on the portfolio for years to come and required the agency to identify $1.4 billion in additional resources over fiscal years 2012 through 2017, according to Science Mission Directorate officials. NASA identified approximately half of this required funding from the four science divisions within the Science Mission Directorate account. The majority of the cuts were related to future high priority missions, missions in the operations and sustainment phase, and research and analysis.

In essence, NASA had to mortgage future high priority missions and research to address JWST’s additional resource needs. Similarly, the National Academy of Sciences has concluded in the past that it is important for NASA to have a clearly articulated and consistently applied method for prioritizing why and how its scarce fiscal resources are apportioned with respect to the science program in general and on a more granular level among component scientific disciplines. The academy noted that failure to do so could result in a loss of capacity, capability, and human resources in a number of scientific disciplines and technological areas that may take a generation or more to reconstitute once eliminated. NASA’s establishment of the WFIRST Independent External Technical/Management/Cost Review that I previously discussed is a step in the right direction to help ensure the Astrophysics Division incorporates this lesson learned.

Establish Adequate Cost and Schedule Reserves to Address Risks. Twice in the history of the JWST program, independent reviewers found that the program’s planned cost reserves were inadequate. First, in April 2006, an Independent Review Team confirmed that the project’s technical content was complete and sound, but expressed concern over the project’s reserve funding, reporting that it was too low and phased in too late in the development lifecycle. The review team reported that for a project as complex as JWST, 25 to 30 percent total reserve funding was appropriate. The team cautioned that low reserve funding compromised the project’s ability to resolve issues, address risk areas, and accommodate unknown problems. As I previously mentioned, following additional cost increases and schedule threats, NASA commissioned the Independent Comprehensive Review Panel. In 2010, the panel again concluded JWST was executing well from a technical standpoint, but that the baseline cost estimate did not reflect the most probable cost with adequate reserves in each year of project execution, resulting in an unexecutable project.

NASA heeded these lessons when it established a new baseline for JWST in 2011. For example, the revised schedule included more reserves than required by the procedural requirements issued by the NASA center responsible for managing JWST. We have found, however, that NASA has not applied this lesson learned to all of its large projects— most notably with its human spaceflight projects, including the Space Launch System, Orion Crew Capsule, and associated ground systems— and similar outcomes to the JWST project have started to emerge with these projects. We previously reported that all three of these programs were operating with limited cost reserves, which limited each program’s ability to address risks and unforeseen technical challenges.

For example, we found in July 2016 that the Orion program planned to maintain very low levels of annual cost reserves until 2018. The lack of available cost reserves in the near term led to the program deferring work to address technical issues to stay within budget, and put the program’s future cost reserves at risk of being overwhelmed by deferred work. In April 2017, we also found that all three programs faced development challenges in completing work, and each had little to no schedule reserve remaining to the launch date—meaning they would have to complete all remaining work with minimal delay during the most challenging stage of development. We found that it was unlikely that the programs would achieve the planned launch readiness date and recommended that NASA reassess the date. NASA agreed with this recommendation and stated that it would establish a new launch readiness date. In November 2017, NASA announced that a review of the possible manufacturing and production schedule risks indicated a launch date of June 2020—a delay of 19 months—but the agency will manage to a December 2019 launch date because, according to NASA, they have put in mitigation strategies for those risks. We will follow-up on those mitigation strategies as part of future work on the human space exploration programs.

Regularly and Consistently Update Project JCLs to Provide Realistic Estimates to Decision Makers. In 2009, NASA began requiring that programs and projects with estimated life-cycle costs greater than $250 million develop a JCL prior to project confirmation. This was a positive step for NASA to help ensure that cost and schedule estimates are realistic and projects are thoroughly planning for anticipated risks. This is because a JCL assigns a confidence level, or likelihood, of a project meeting its cost and schedule estimates. Our cost estimating best practices recommend that cost estimates should be updated to reflect changes to a program or be kept current as a program moves through milestones. As new risks emerge on a project, an updated cost and schedule risk analysis can provide realistic estimates to decision-makers, including the Congress. This is especially true for NASA’s largest projects as updated estimates may require the Congress to consider a variety of actions.

However, there is no requirement for NASA projects to update their JCLs, and our prior work has found that projects—including JWST—do not regularly update cost risk analyses to take into account newly emerged risks. Our ongoing work indicates that of the 16 major projects currently in NASA’s portfolio that have developed JCL estimates, only 2 have reported updating their JCLs (other than required due to a rebaseline). For example, the Interior Exploration using Seismic Investigations, Geodesy, and Heat Transport Project (InSight), a Mars lander, updated its JCL after the project missed its committed launch date. As a result, the project was able to provide additional information to decision makers about the probability that it will meet its revised cost and schedule estimates. As a project reaches the later stages of development, especially integration and testing, the types of risks the project will face may change. An updated project JCL would provide both project and agency management with data on relevant risks that can guide the project decisions. For example, in December 2012, we recommended the JWST project update its JCL. NASA concurred with this recommendation; however, we recently closed the recommendation because NASA had not taken steps to implement it and the amount of time remaining before launch would not have allowed the benefit of implementing the recommendation to be realized. An updated JCL may have portended the current schedule delays, which could have been proactively addressed by the project.

Enhance Oversight of Contractors to Improve Project Outcomes. In December 2012, we found that the JWST project had taken steps to enhance communications with and oversight of its contractors. According to project officials, the increased communication allowed them to better identify and manage project risks by having more visibility into contractors’ activities. The project reported that a great deal of communication existed across the project prior to the Independent Comprehensive Review Panel; however, additional improvements were made. For example, the project increased its presence at contractor facilities as necessary to provide assistance; this included assigning two engineers on a recurring basis at a Lockheed Martin facility to assist in solving problems with an instrument. The JWST project also assumed full responsibility for the mission system engineering functions from Northrop Grumman in March 2011. NASA and Northrop Grumman officials both said that NASA is better suited to perform these tasks.

We continue to see instances in our ongoing work that highlight the importance of implementing this lesson learned from JWST. For example, we found in 2017 that the Space Network Ground Segment Sustainment project—a project that plans to develop and deliver a new ground system for one Space Network site that provides essential communications tracking services to NASA and non-NASA missions—exceeded its original cost baseline by at least $401.7 million and been delayed by 27 months. The project has attributed some of the cost overruns and schedule delays to the contractor’s incomplete understanding of its requirements, which led to poor contractor plans and late design changes. The project also took steps to assign a new NASA project manager, increase physical presence at the contractor facility, and have more staff focused on validation and verification activities.

In summary, NASA continues to make progress developing its space telescopes to help understand the universe and our place in it. But much like other major projects that NASA is developing, there continues to be an opportunity for NASA to learn from JWST and other projects that have suffered from cost overruns and schedule delays. Key project management tools and prior GAO recommendations that I have highlighted here today, could help to better position these large, complex, and technically challenging efforts for a successful outcome. We look forward to continuing to work with NASA and this subcommittee in addressing these issues.

Chairman Babin, Ranking Member Bera, and Members of the Subcommittee, this completes my prepared statement. I would be pleased to respond to any questions that you may have at this time.

GAO Contact and Staff Acknowledgments

If you or your staff have any questions about this testimony, please contact Cristina T. Chaplain, Director, Acquisition and Sourcing Management at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this statement. GAO staff who made key contributions to this statement include Molly Traci, Assistant Director; Richard Cederholm, Assistant Director; Carrie Rogers; Lisa Fisher; Laura Greifner; Erin Kennedy; and Jose Ramos.

This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately.