NASA: Actions Needed to Improve Transparency and Assess Long-Term Affordability of Human Exploration Programs
Released: 2014-05-08T00:00:00

Letter

Background

The National Aeronautics and Space Administration Authorization Act of 2010 directed NASA to develop a Space Launch System as a follow-on to the Space Shuttle and as a key component in expanding human presence beyond low-earth orbit. The Act also directed NASA to continue development of a multi-purpose crew vehicle for use with that system.that end, NASA plans to incrementally develop three progressively-larger SLS launch vehicle capabilities—70-, 105- and 130-metric ton (MT) variants—complemented by the Orion and supporting ground systems.

To Figure 1 below illustrates NASA’s planned capabilities for the SLS, Orion, and some of the related GSDO efforts.

These capabilities follow the agency’s previous attempt to develop a next- generation human spaceflight system, the Constellation program, which was cancelled in 2010 when the program’s budget proved inadequate to resolve technical challenges. The first version of the SLS being developed is a 70-metric ton launch vehicle known as Block I. NASA expects to conduct two test flights of the Block I vehicle—the first in 2017 and the second in 2021. The vehicle is scheduled to fly some 700,000 kilometers beyond the moon during the first test flight, known as Exploration Mission-1 (EM-1), and to fly a second mission, known as Exploration Mission-2 (EM-2), to test additional aspects of its performance. After 2021, NASA intends to build 105- and 130-metric ton launch vehicles, known respectively as Block IA/B and Block II, which it expects to use as the backbone of manned spaceflight for decades. NASA anticipates that these launch vehicles will require the development of new systems to achieve the agency’s goals for carrying greater amounts of cargo and traveling farther into space. The agency has not yet selected specific missions for the increased capabilities to be provided by Block IA/B and Block II but, in keeping with the language contained in the 2010 Authorization Act, anticipates using the vehicles for such deep- space destinations as near-Earth asteroids and Mars.

In concert with SLS, NASA expects to evolve the Orion and ground systems. The agency plans an un-crewed Orion capsule to fly atop the SLS during EM-1 in 2017, a crewed capsule during EM-2 in 2021, and ultimately, at a date to be determined, a crewed capsule with capability for such missions as a Mars landing. NASA is also modifying the existing ground systems so that they can support the SLS Block I variant and eventually accommodate the Block IA/B and Block II launch vehicles as well as enhanced versions of the Orion crew capsule. For example, NASA plans to add moveable floors to the vehicle assembly building at Kennedy Space Center so that the three launch vehicle variants can be more easily prepared for flight as the SLS capability evolves.

NASA established the preliminary cost estimates for the initial capabilities of the SLS, Orion, and associated GSDO as each of these programs entered the preliminary design and technology completion phase of development, known as key decision point B (KDP-B). At KDP-B, programs use a probability-based analysis to develop a range of preliminary cost and schedule estimates which are used to inform the budget planning for the programs. This phase culminates in a review at key decision point C (KDP-C), known as program confirmation, where cost and schedule baselines with point estimates are established and documented in the agency baseline commitment. After this review, programs are considered to be in the implementation phase of development, and program progress is subsequently measured against these baselines. NASA plans to hold the program confirmation review for SLS in spring 2014 and expects to conduct the KDP-C review for GSDO in May 2014 and Orion in December 2014. Because the life cycle costs of these programs are expected to exceed $250 million, NASA is required to report the programs’ baseline estimates to Congress once the programs are approved to move into implementation. The agency provides this information through its annual budget submission. NASA also uses the annual budget submission to inform Congress about the preliminary cost ranges for projects proceeding into formulation.

Limited Scope of Exploration Program Estimates Does Not Capture Life Cycle Costs

NASA’s preliminary cost estimates for the SLS, Orion, and associated GSDO programs do not provide a complete picture of the costs required to develop and operate the programs through the entire course of their respective life cycles. These preliminary estimates include the funding required for the scope of work related to initial capabilities—that is, development and operations through 2017 for the SLS launch vehicle and ground systems and through 2021 for the Orion. NASA also expects to use this same limited scope of work to develop the SLS, Orion, and GSDO baseline cost estimates. Moreover, NASA’s estimates do not capture the cost of the second flight of the 70-metric ton vehicle during EM-2, the costs of development work that will be necessary to fly the increased 105- and 130-metric ton SLS capabilities, and the costs associated with legacy hardware that will be used for the Orion program. In contrast, best practices for cost estimation call for “cradle to grave” life cycle cost estimates in order to help assess a program’s long-term affordability.

Cost for Initial SLS, Orion, and GSDO Capabilities Totals up to $22 Billion

NASA’s preliminary cost estimates for the three programs’ initial capabilities total a low-to-high cost range of approximately $19 to $22 billion. Table 1 below depicts the scope, including content and schedule, of the SLS, Orion, and GSDO initial capabilities’ preliminary cost estimates. As the SLS, Orion, and GSDO programs move from formulation into implementation phases, NASA plans to use the same content and scope for calculating the programs’ respective baseline cost estimates.

NASA’s Estimates Do Not Include Life Cycle Costs

NASA’s preliminary cost estimates for SLS, Orion, and GSDO provide no information about the longer-term, life cycle costs of developing, manufacturing, and operating the launch vehicle, crew capsule, and ground systems:

The SLS estimate does not cover the cost to build the second 70- metric ton vehicle and conduct EM-2 in 2021 with that vehicle. NASA is already incurring costs for EM-2 because it is funding some EM-2 development in concert with EM-1 efforts, such as work on the solid rocket boosters and core stage that are expected to help power the 70-metric ton SLS. NASA officials indicated at one point in our review that they did not expect to begin formally tracking EM-2 costs until after the SLS design’s maturity was assessed at a critical design review scheduled for 2015; however, the agency stated in technical comments to this report that it is tracking those costs for budget purposes and plans to begin formally reporting them once SLS reaches the project confirmation phase. Additionally, the SLS estimate does not address the potential for costs NASA would incur to produce, operate, and sustain flights of the 70-MT Block I capability beyond 2021. NASA officials stated that there are currently no plans to fly that vehicle beyond 2021, but that the agency could reassess its decision if a specific mission arises for the vehicle.

The SLS estimate also does not include costs to design, develop, build, and produce the 105- or 130-metric ton Block IA/B and Block II SLS variants that NASA intends to use well into the future. NASA indicated that these variants will require new systems development efforts—including advanced boosters and a new upper stage to meet the greater performance requirements associated with larger payloads as well as travel to Mars or other deep-space locations. NASA has started funding concept development, trades, and analyses related to these new designs, such as assessing the use of lightweight materials to construct the upper stage and selective laser melting to produce system components. In addition, NASA anticipates a re-start of the production line for the RS-25 engine that it plans to use to power the Block IA/B and Block II vehicles. Currently, the agency has enough residual RS-25 liquid-fuel engines from the Space Shuttle program to launch the SLS for up to 4 flights. NASA expects to need more of the engines beyond that, but it has not yet finalized acquisition plans to manufacture them. According to agency officials, re-starting the production line would entail at least 3 years, whereas development of a new engine would require a minimum of 8 years.

The Orion estimate does not address costs for production, operations, or sustainment of additional crew capsules after 2021 nor does it address prior costs incurred when Orion was being developed as part of the now-defunct Constellation program. NASA initiated the crew capsule’s development in 2006 as part of the Constellation program. During approximately 4 years that the capsule’s development occurred under Constellation, the agency spent about $4.7 billion for the capsule’s design and development. When Constellation was cancelled in 2010 and the work transitioned to the current Orion program, however, NASA excluded the Constellation-related costs from Orion’s current preliminary cost estimate of $8.5 to $10.3 billion through 2021.

The GSDO estimate does not address the costs to develop or operate SLS ground systems infrastructure beyond EM-1 in 2017, although NASA intends to modify ground architecture to accommodate all SLS variants.

NASA officials have indicated that the road ahead involves many decisions about the programs beyond 2021, including how development will proceed, what missions will be performed, when the programs will end, and how each effort will be managed. They noted that the agency is using a capability-based approach to SLS, Orion, and the associated GSDO development, in which system capability grows over time. They indicated that the programs’ preliminary cost estimates are for attainment of capabilities rather than the full cost of the programs, and that it is difficult to define life cycle costs because the programs’ intended long- term uses and life spans have not been fully determined. According to NASA, the agency is developing a tailored definition for life cycle cost estimating that is allowed by NASA requirements. Because the missions drive the number and types of vehicles, crew capsules, and ground systems that would be required, as missions are defined, NASA officials said they would be in a better position to estimate the programs’ life cycle costs. The officials stated that NASA is looking ahead to future costs as much as possible, and NASA indicated in technical comments to this report that the SLS program plans to begin formally reporting costs for the launch vehicle’s EM-2 after the program’s anticipated confirmation in spring 2014.

We recognize that defining life cycle costs can be difficult when uncertainties exist. However, in contrast to NASA’s tailored approach, both widely-accepted best practices for cost estimation and the agency’s own requirements support the need for full life cycle cost estimates. Even when uncertainties exist, best practices maintain that a high-quality cost estimate takes into account those uncertainties while forecasting the minimum and maximum range of all life cycle costs. The best practices, developed by the GAO in concert with the public and private sector cost estimating communities, call for “cradle to grave” life cycle cost estimates and maintain that life cycle cost estimates should provide an exhaustive, structured accounting of all resources and associated cost elements required to develop, produce, deploy, and sustain a particular program. This entails identification of all pertinent cost elements, from initial concept through operations, support, and disposal. Likewise, NASA’s program management requirements direct that programs develop a preliminary full life cycle cost estimate. In accordance with the agency’s guidance regarding life cycle costs, such an estimate would encompass total costs from the formulation through the implementation phase, including design, development, mission operations, support, and disposal activities.

According to best practices, because life cycle estimates encompass all possible costs, they provide a wealth of information about how much programs are expected to cost over time. Life cycle cost estimates, including a range for preliminary costs as directed by NASA requirements for programs in the formulation phase, enhance decision making, especially in early planning and concept formulation of acquisition. High- quality cost estimates, as noted by best practices, can support budgetary decisions, key decision points, milestone reviews, and investment decisions. For example, a preliminary life cycle cost estimate provides the basis of the financial investment that the agency is committing the government to, while a baseline life cycle cost estimate forms the basis for measuring cost growth over time. Because NASA expects to continue with a limited scope for the SLS, Orion, and baseline estimates, however, cost growth over time within the programs will be difficult to identify and could be masked as growth in the SLS capability if the most current cost estimate did not contain the same content as the baseline estimate.

As noted in best practices for cost estimating, the quality of a program’s cost estimate is also key to determining its affordability, that is, the degree to which a program’s funding requirements fit within an agency’s overall portfolio plan. However, NASA’s preliminary cost estimates do not address the affordability of increased capabilities because they exclude the life cycle costs associated with the SLS Block IA/B and Block II launch vehicles that the agency intends to use well into the future. According to agency officials at the time of our review, NASA has not yet decided whether it will manage the Block IA/B and Block II development efforts as individual programs and, if so, what the programs’ scope would be. Best practices for cost estimating look favorably on the incremental development approach NASA has chosen for SLS, and they also state that programs following such an approach should clearly define the characteristics of each increment of capability so that a rigorous life cycle cost estimate can be developed. In addition, we have previously concluded that it is prudent for an agency to manage increasing capabilities of an existing program on par with the investments yet to come and in a way that is beneficial for oversight. For example, we have recommended that agencies developing weapon systems in increments consider establishing each increment of increased capability with its own cost and schedule baseline. According to cost estimating best practices, dividing programs into smaller pieces makes management and testing easier and helps avoid unrealistic cost estimates, resulting in more realistic long-range investment funding and more effective resource allocation. These are important considerations given that NASA is likely to spend billions of dollars beyond its initial investment of up to $22 billion to develop the increased capabilities. Development of human-rated liquid- fueled engines, for example, has been among the most difficult, time- intensive, and costly parts of launch vehicle development. As a case in point, NASA spent about 8 years and $1.5 billion to develop a human- rated engine known as J-2X for use on Ares launch vehicles within the agency’s now-defunct Constellation program.

NASA has faced issues with affordability of its manned space flight investments and other major projects in the past, and those affordability issues have sometimes contributed to a program’s cancellation. For example, NASA originally envisioned that the Space Shuttle would fly up to 100 times per vehicle at a cost of $7.7 million per launch. In reality, the Shuttle flew 135 times in total over a period of 30 years at a cost that was about $3.5 billion per year around the 2008 timeframe. Amid concerns that included the Shuttle’s costs and safety, the program ended. NASA then focused on building human spaceflight alternatives that included Constellation. In 2010, Constellation was canceled because, as noted by NASA’s Administrator, the program could not return astronauts to the moon at an affordable cost and would require far more funding to make the agency’s approach viable. In a recent example noted in the agency’s 2015 presidential budget request, NASA may place in storage the Stratospheric Observatory for Infrared Astronomy, an airborne observatory for studying astronomical objects and phenomena, after spending some 23 years and more than $1 billion to develop the project. The agency cited high operating costs, estimated at some $1.8 billion over the project’s planned life, as a factor in its considerations.

Conclusions

The SLS, Orion, and GSDO programs NASA has established to fulfill its mandate of providing the capability for transporting humans to space are well underway. These programs represent a significant investment for the country—as much as $22 billion for initial capabilities and potentially billions more to field increased capabilities over time as envisioned in the 2010 NASA Authorization Act. Given the goals that have been outlined for NASA as part of the National Space Transportation Policy, the success of these programs is to be measured not only by the capability that is achieved but also by NASA’s ability to achieve them within a reasonable timeframe and cost to the U.S. taxpayer. As such, establishing these programs with both near-term and long-term affordability in mind is key. The limited scope that NASA has chosen to use as the basis for formulating the programs’ cost baselines, however, does not provide the transparency necessary to assess long-term affordability and will hamper oversight by those tasked with assessing whether the agency is progressing in a cost-effective and affordable manner. If the SLS, Orion, and GSDO baseline cost estimates cannot be compared to current costs, the baseline estimates lose their usefulness because they no longer serve as a means to hold NASA accountable for cost growth and program progress. Furthermore, if NASA does not clearly delineate costs for operations and sustainment of the initial capabilities or separate cost and schedule baselines for upcoming capabilities, then it will be difficult to assess program affordability and for the Congress to make informed, long-term budgetary decisions. Estimates that use all available information to establish a potential range of costs for the full scope of these upcoming capabilities can help inform such decisions.

Recommendations for Executive Action

To provide the Congress with the necessary insight into program affordability, ensure its ability to effectively monitor total program costs and execution, and to facilitate investment decisions, we recommend that NASA’s Administrator direct the Human Exploration and Operations Mission Directorate take the following 3 actions:

Establish a separate cost and schedule baseline for work required to support the SLS Block I EM-2 and report this information to the Congress through NASA’s annual budget submission. If NASA decides to fly the SLS Block I beyond EM-2, establish separate life cycle cost and schedule baseline estimates for those efforts, to include funding for operations and sustainment, and report this information annually to Congress via the agency’s budget submission.

Because NASA intends to use the increased capabilities of the SLS, Orion, and GSDO efforts well into the future and has chosen to estimate costs associated with achieving the capabilities, establish separate cost and schedule baselines for each additional capability that encompass all life cycle costs, to include operations and sustainment. When NASA cannot fully specify costs due to lack of well-defined missions or flight manifests, forecast a cost estimate range — including life cycle costs — having minimum and maximum boundaries. These baselines or ranges should be reported to Congress annually via the agency’s budget submission.

Because a significant amount of the original Orion development work occurred under the Constellation program, include those costs in the baseline cost estimate for the Orion program.

Agency Comments and Our Evaluation

NASA provided written comments on a draft of this report. These comments are reprinted in Appendix I.

In responding to a draft of our report, NASA partially concurred with our three recommendations, citing among other reasons that actions already in place at the time of our review such as establishing SLS, Orion, GSDO as separate programs and a block upgrade approach for SLS—and actions it plans to take to track costs—met the intent of our recommendations. In most cases, the actions that NASA plans to take do not fully address the issues we raised in this report. We continue to believe that our recommendations are valid and should be fully addressed as discussed below. NASA also provided technical comments which we incorporated as appropriate.

NASA partially concurred with our first recommendation to establish a separate cost and schedule baseline for work required to support the SLS Block I EM-2, report this information to the Congress through NASA's annual budget submission, and establish separate life cycle cost and schedule baseline estimates for EM-2 if NASA decides to fly Block I beyond EM-2. NASA also partially concurred with our second recommendation to establish separate cost and schedule baselines that encompass life cycle costs, including operations and sustainment, for each additional SLS, Orion, and GSDO capability and to report cost estimates for the capabilities annually via the agency budget submission until key requirements are defined and baselines can be established. In its response, NASA stated that it had established separate programs for SLS, Orion, and GSDO and adopted a block upgrade approach for SLS. This approach, NASA stated, is in concert with best practices and NASA policy. In addition, NASA indicated that it will establish cost and schedule estimates for initial demonstration of the three programs as they enter respective implementation phases and will begin reporting development, operations, and sustainment costs for SLS Block I and subsequent variants starting in fiscal year 2016 via its annual budget submission to Congress. Finally, the agency stated that it intends to conduct design reviews for upgraded SLS elements, including the upper stage and booster, and set up cost commitments similar to what it has done for Block I capability as part of that design review process, but that it does not intend to establish life cycle estimates for SLS through the end of the program because flight rates, mission destinations and other strategic parameters are yet unknown.

As discussed in the report, best practices for cost estimating recognize that NASA’s evolutionary development approach for SLS, Orion, and GSDO helps reduce risk and provide capabilities more quickly. Given NASA’s planned long-term use of the SLS, Orion, and GSDO, its block upgrade approach and intention to conduct design reviews for each of the planned upgrades will provide some understanding of the development work and resources required. For example, such reviews are typically expected to yield information about technical progress against requirements. While NASA's prior establishment of SLS, Orion, and GSDO as separate programs lends some insight into expected costs and schedule at the broader program level, it does not meet the intent of our first two recommendations because cost and schedule identified at that level is unlikely to provide the detail necessary to monitor the progress of each block against a baseline. Furthermore, it is unclear from NASA's response whether the cost commitments the agency plans within the design review process will serve the same purpose as establishing a cost baseline for each respective upgrade.

Additionally, NASA's planned approach for reporting costs associated with EM-2 and subsequent variants of SLS via its annual budget submission only partially meets the intent of our first two recommendations. Providing cost information at an early phase when baseline estimates have yet to be established is helpful to ensure costs associated with EM-1 and EM-2 are not conflated and funding requirements for future flights of the Block I SLS and future variants are somewhat understood. Reporting the costs via the budget process alone, however, will not provide information about potential costs over the long- term because budget requests neither offer all the same information as life cycle cost estimates nor serve the same purpose. Plainly, progress cannot be assessed without a baseline that serves as a means to compare current costs against expected costs. An agency’s budget submission reflects its current annual fiscal needs and anticipated short- term needs up through an additional 4-year period for a particular program, is subject to change based on fiscal negotiation, and is not necessarily linked to an established baseline that indicates how much the agency expects to invest to develop, operate, and sustain a capability over the long-term. Conversely, life cycle cost estimates establish a full accounting of all program costs for planning, procurement, operations and maintenance, and disposal and provide a long-term means to measure progress over a program’s life span. As NASA establishes parameters for the additional flights of the first SLS capability and upgraded capabilities, including flight rates, mission destinations, and other requirements, it will be well-poised to move from reporting costs in budget submissions to establishing baseline cost and schedule estimates for each capability and reporting progress against these respective baselines. Therefore, we continue to believe that NASA should baseline costs for EM-2 and each future variant of SLS and report progress against those established baselines.

NASA makes no specific mention of how it plans to account for future work associated with Orion and GSDO. We believe it is important to treat Orion and GSDO with the same significance as SLS because this trio of programs is expected to work in concert now and in the future to achieve NASA’s goals for human space exploration. Reporting Orion and GSDO development, operations, and sustainment costs in the annual budget request, as NASA plans for SLS, would be a logical first step. Just as with SLS, however, it will be important for NASA to establish and report progress against baseline costs and schedules for each block of Orion and GSDO efforts as flight rates, missions, and other strategic parameters are defined because doing so will help the agency more effectively manage not only each program but its human exploration portfolio as a whole.

NASA partially concurred with our third recommendation to include the costs of Orion development work under the Constellation program as part of the baseline cost estimate for the Orion program. Agency officials stated that they agree those costs should be tracked and disclosed, but that the current Orion program has a new concept of operations, requirements, and budget plan than that under the Constellation effort.

The past costs incurred for Orion’s development are important because they provide visibility into the total cost of developing a crew capsule for human space exploration. Exclusion of these costs from Orion’s current estimate understates how much NASA will invest to put humans into space. Although NASA notes that it has changed Orion’s concept of operations and requirements, the agency nonetheless migrated Orion critical technology development efforts from Constellation to the SLS program. For example, NASA began efforts to develop the coating for Orion’s heat shield as part of Constellation, and the agency continues that development today in preparation for the capsule’s launch atop SLS. Therefore, we continue to believe our recommendation to include Orion development costs under Constellation in the baseline cost estimate for the current Orion program is valid and should be fully implemented.

We are sending this report to NASA’s Administrator and to interested congressional committees. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

Should you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Other key contributors to this report are listed in Appendix II.

Appendix I: Comments from the National Aeronautics and Space Administration

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Staff Acknowledgments

Key contributors to this report were Shelby S. Oakley, Assistant Director; Tana M. Davis; John S. Warren, Jr.; Jennifer Echard; Laura Greifner; Roxanna Sun; and Sylvia Schatz.