Space Launch System: Resources Need to be Matched to Requirements to Decrease Risk and Support Long Term Affordability
Released: 2014-07-23T00:00:00

Letter

Background

The National Aeronautics and Space Administration Authorization Act of 2010 directed NASA to, among another things, develop a Space Launch System as a follow-on to the Space Shuttle and as a key component in expanding human presence beyond low-Earth orbit. To that end, NASA plans to incrementally develop three progressively more capable SLS launch vehicles—70-, 105-, and 130-metric ton (mt) variants. When complete, the 130-mt vehicle is expected to have more launch capability than the Saturn V vehicle, which was used for Apollo missions, and be significantly more capable than any recent or current launch vehicle. The act also directed NASA to prioritize the core elements with the goal of operational capability for the core elements not later than December 2016. NASA subsequently negotiated an extension of that date, to December 2017, based on the agency’s assessment of the tasks associated with developing the new launch vehicle.

In 2011, NASA formally established the SLS program. To fulfill the direction of the 2010 act, the agency plans to develop the three SLS launch vehicle capabilities, complemented by the Orion Multi-Purpose Crew Vehicle (Orion) to transport humans and cargo into space. The first version of the SLS that NASA is developing is a 70-mt launch vehicle known as Block I. NASA expects to conduct two test flights of the Block I vehicle—the first in 2017 and the second in 2021. The vehicle is scheduled to fly an uncrewed Orion some 70,000 kilometers beyond the moon during the first test flight, known as Exploration Mission-1 (EM-1), and to fly a second mission known as Exploration Mission-2 (EM-2) beyond the moon to further test performance with a crewed Orion vehicle. After 2021, NASA intends to build 105- and 130-mt launch vehicles, known respectively as Block IA/B and Block II, which it expects to use as the backbone of manned spaceflight for decades. The agency plans for these vehicles to carry larger cargo and travel farther into space, but it has not yet selected specific missions for the increased capabilities to be provided by Block IA/B and Block II. NASA anticipates using the Block IA/B vehicles for destinations such as near-Earth asteroids and LaGrange points and the Block II vehicles for eventual Mars missions.

NASA’s Acquisition Life Cycle

NASA plans to evolve the 70-mt SLS design following the agency’s life cycle acquisition process for flight systems. That process is defined by two broad phases—formulation and implementation—and several key decision points.discrete pieces with different purposes: pre-phase A through phase F. See figure 1 for a depiction of NASA’s life cycle for flight systems.

Acquisition Best Practices

GAO’s best practices for systems acquisition state that evolutionary In evolutionary acquisition, a acquisition is a commercial best practice.program uses mature technologies and capabilities that fit within existing resources to field initial capabilities sooner and gradually develops progressively more capable increments until it achieves its ultimate requirements. This approach is considered a best practice because it allows commercial companies and the government to develop and produce more sophisticated products faster and less expensively.

GAO’s work on best practices has also shown that success in development efforts such as SLS depends on establishing an executable business case based on matching requirements and resources before committing to a new product development effort. In its simplest form, a business case requires a balance between the concept selected to satisfy customer needs and the resources—technologies, design knowledge, funding, time, and management capacity—needed to transform the concept into a product. At the heart of a business case is a knowledge- based approach that requires that managers demonstrate high levels of knowledge as the program proceeds from technology development to system development and, finally, production. Ideally, with such an approach, key technologies are demonstrated before development begins, the design is stabilized before prototypes are built or production begins, and testing is used to validate product maturity at each level. At each decision point, the balance among time, money, and capacity is confirmed. In essence, knowledge supplants risk over time. Having adequate knowledge about requirements and resources is particularly important for a program like SLS because human spaceflight development projects are complex, difficult, and costly.

We have found that within NASA’s acquisition framework, the PDR and corresponding confirmation review are the point at which development projects should have a sound business case in hand. NASA’s Systems Engineering Policy states that this review demonstrates that the preliminary design meets all system requirements with acceptable risk and within the cost and schedule constraints. After a project is confirmed following PDR, it begins implementation during which time senior NASA officials periodically review the programmatic and technical status of projects and verify the project’s readiness to proceed forward. For example, a critical design review (CDR) before the project’s subsystems are integrated evaluates the integrity of the project design and its ability to meet mission requirements, with appropriate margins and acceptable risk, within defined project constraints, including available resources. In short, the CDR determines if the design is appropriately mature to support proceeding with the final design and fabrication phase. Our past work on product development best practices has found that programs having at least 90 percent of engineering drawings releasable by the critical design review lower their risk of subsequent cost growth and schedule delays, and guidance in NASA’s Systems Engineering Handbook mirrors this metric. NASA also tracks both mass margin and resolution of requests for action, which are technical, safety, and programmatic comments from independent reviewers, as additional measures of design maturity. In addition, experts in the space community have identified other metrics that can be useful to assess the design stability of unique space systems.schedule margin at various points in the development life cycle as well as the percent of verification and validation plans completed at both PDR and CDR.

Elements of SLS

In accordance with direction contained in the NASA Authorization Act of 2010, NASA’s acquisition approach for building the initial variant of the SLS is predicated on the use of legacy systems, designs, and contracts from the Space Shuttle and Constellation programs. Figure 2 provides details about the heritage of each SLS hardware element and its source.

NASA plans to use 16 RS-25 engines remaining from the Space Shuttle program to provide power for up to four flights of the SLS. Similarly, the agency is procuring five-segment boosters that were developed under the Constellation program to provide thrust during the initial minutes of SLS flight. NASA is also procuring a cryogenic rocket stage used on United Launch Alliance’s Delta IV launch vehicle and modifying it to operate as the Interim Cryogenic Propulsion Stage (ICPS) to provide in-space power for SLS. Finally, the design for the new core stage, which functions as the SLS’s fuel tank and structural backbone, is derived from the Shuttle’s external tank and Ares I upper stage.

Program’s Ability to Meet Schedule for 2017 Test Flight at Risk

While the SLS program is satisfying many of NASA’s metrics that measure progress against overall design goals, NASA has not established an executable business case that matches the SLS program’s cost and schedule resources with the requirement to develop the SLS and launch the first flight test in December 2017 at the required confidence level of 70 percent. Matching resources to requirements is considered a best practice for establishing a successful acquisition program. Reduced confidence levels increase the risk that the program will miss its launch date or overrun its cost baseline. The program is on target to achieve goals for design drawing release and mass and expects to meet necessary documentation requirements for its critical design review. However, the development schedule of the core stage, which is driving the overall program schedule, is aggressive and substantial amounts of schedule that the program reserved to resolve unanticipated issues is already threatened. NASA has focused on integration issues, and the program is on plan for closing design trades, environmental studies, and design interface documents that are intended to reduce this risk. However, the modification and integration of heritage elements may be a challenge—one which NASA often underestimates. In addition, according to the program’s risk analysis, the agency’s current funding plan for SLS may be $400 million short of what the program needs to launch by 2017. This funding discrepancy was, in part, responsible for the significant delay in finalizing the contracts for element development, which increased cost and schedule risk to the government as well as reduced visibility into contractor performance.

NASA Has Not Matched Resources and Requirements for 2017 Initial Flight Test

NASA has not established an executable business case based on matching the SLS program’s cost and schedule resources with the requirement to develop the SLS and launch the first flight test in December 2017 at the required confidence level of 70 percent. NASA delayed the SLS key decision point C decision from October 2013 to at least July 2014, as the agency considered future plans for the program. If the agency determines the current funding plan for SLS is insufficient to match requirements to resources for the December 2017 flight test at the 70 percent confidence level, the agency’s options for matching resources to requirements are largely limited to increasing program funding, delaying the schedule or accepting a reduced confidence level for the initial flight test. While the program’s decision authority is allowed in some instances to approve a confidence level of less than 70 percent, doing so increases the likelihood that the program will miss the launch date or overrun the current cost estimate. Should cost growth or schedule delay occur, it could place other programs’ funding at risk if NASA chooses to take planned funding from those programs in order to maintain the SLS schedule. Although cost and schedule growth can occur on any project, increases associated with NASA’s most costly and complex missions— such as SLS, which makes up about 9 percent of NASA’s annual budget—can have dramatic effects on the availability of funding for NASA’s portfolio of major projects.

Funding levels have impaired the SLS program’s ability to match requirements to resources since its inception. As illustrated in figure 3, NASA has requested relatively consistent amounts of funding for SLS each year. According to agency officials, the program has taken steps to live within that flat funding profile, including streamlining program office operations and asking each contractor to identify efficiencies in their production processes. Even so, according to the program’s own analysis, going into the confirmation review, SLS’s top risk was that the current planned budget through 2017 would be insufficient to allow the SLS as designed to meet the EM-1 flight date. The SLS program office calculated the risk associated with insufficient funding through 2017 as having a 90 percent likelihood of occurrence; furthermore, it indicated the insufficient budget could push the planned December 2017 launch date out 6 months and add some $400 million to the overall cost of SLS development. Program officials stated that the potential cost impacts of this risk were considerably higher in the past, but that they were able to reduce the impact due to receiving more funding than requested in fiscal years 2013 and 2014.

In May 2014, the House Committee on Appropriations report—which accompanied the Commerce, Justice, Science, and Related Agencies Appropriations Bill, 2015—“expressed frustration with NASA’s practices of requesting arbitrarily reduced funding levels for SLS and insisting that the program manage to an inefficient flat-line budget profile.” The Committee noted that this practice has “detrimental results” and is likely to lead to a “launch delay for EM-1 and deferral of long-lead work needed for the timely achievement of EM-2 and other future flights.” In light of these and other issues, the Committee has chosen to recommend maintaining SLS vehicle development funding at the fiscal year 2014 enacted level of $1.6 billion.

Best practices for acquisition programs indicate that matching resources to requirements and rationally balancing cost, schedule, and performance is a key step in establishing a successful acquisition program. We have also found that NASA’s previous attempts to develop new transportation systems, such as the Constellation program, have failed in part because they were focused on maturing designs without adequate funding to support those efforts. Our work has also shown that developing a sound business case before committing resources to development—which includes firm requirements, mature technologies, a knowledge-based acquisition strategy, a realistic cost estimate, and sufficient funding and time—would help mitigate the risks inherent in NASA’s programs.

SLS Program Is Meeting Some Design Goals for 2017 Test Flight but Challenges Could Threaten Launch Date

Design Goals

NASA’s metrics indicate that the program is on track to meet many of its design goals for demonstrating the initial capability of SLS. Based on our review of top level design metrics that the program is tracking, the program is currently meeting its plan for design drawing release for the core stage, where most of the new design work is occurring. As of May 2014 the program had released 82 percent of the core stage drawings and expects to have 95 percent released by the core stage CDR in July 2014. The program as a whole expects to release 95 percent of its design drawings by the program level CDR in March 2015, which exceeds the best practice metric for design drawing release at CDR. Because the CDR is the time in a project’s life cycle when the integrity of the project design and its ability to meet mission requirements are assessed, it is important that a project’s design is stable enough to warrant continuation with design and fabrication, which is evidenced by release of 90 percent of design drawings at CDR. A stable design allows projects to “freeze” the design and minimize changes prior to beginning the fabrication of hardware. It also helps to avoid re-engineering and rework efforts due to design changes that can be costly to the project in terms of time and funding.

The program is also meeting or close to meeting several design goals related to SLS mass margin and program documentation. As of May 2014, SLS had sufficient mass margin to meet its design goals for EM-1; however, the program has less mass margin for some EM-2 test flight mission options in 2021. If mass becomes an issue leading into EM-2, the vehicle may require design changes that could lead to cost and schedule growth. While the program is behind on its own schedule for addressing the requests for action from the preliminary design review, the program’s current plan indicates that all requests will be addressed prior to the CDR scheduled for 2015. In addition to these overall design goals, the program’s verification and validation plan, which aids in assessing whether a chosen development solution is capable of meeting its intended purpose and if the system is being developed according to agency requirements, was baselined and approved at PDR with one exception that the program expects to resolve in 2014. With regards to finalizing interface control documents, which define requirements for how systems will connect and interact with one another, the program is 4 percent behind its planned schedule for determining interface requirements by CDR. According to NASA, these documents should help reduce integration issues, such as those between the heritage hardware elements and new core stage, by specifying interaction between the heritage elements and core stage, as well as provide a set of interface definitions for when new elements are being designed.

Aggressive Schedule

Although the program is making progress against its design metrics, the core stage development schedule is aggressive and already threatened, and any delays could impair SLS readiness for first flight in 2017. The core stage development drives the SLS schedule because it represents the critical path of activities that must be completed to maintain the program’s schedule as a whole. Based on our review of NASA documentation and discussions with program officials, it is clear that the core stage’s major program milestones and developmental activities are tightly spaced. For example, the program is allowing 18 months from the point at which it evaluates the design at PDR in January 2013 to the point at which it expects the design to be fully mature at the core stage CDR in July 2014. By comparison, the Ares I project within the Constellation program planned for about 2 years between PDR and CDR. NASA officials indicated that they compressed the core stage development schedule in order to meet the SLS 2017 launch date. The schedule allows for 7 months of schedule reserve after CDR and before the core stage ships to Kennedy Space Center for integration into the SLS, which NASA believes will allow time to address problems it could encounter during development. While allowing 7 months of schedule reserve generally complies with NASA guidance for space flight projects, it may prove insufficient in this instance because the program has already identified a number of threats to the schedule.

The SLS program is tracking threats to the core stage schedule that could take up as much as 70 percent of the 7 months of reserve. Many of these threats are associated with the schedule for acquiring liquid oxygen feeder lines that provide liquid oxygen from the fuel tanks within the core stage to the core stage engine. Agency officials indicated the program is encountering difficulties acquiring feeder lines from available suppliers to meet the core stage development schedule because the feeder lines are larger and will be used in more stressful environments than the lines that suppliers have manufactured in the past. The feeder lines must still be designed, built, and tested for operation in the SLS environment before they can be delivered for integration into the system. While these challenges are not overly complex from a technical viewpoint, resolving such threats to the schedule is critical because the element is in early development phases and still has several significant milestones and developmental activities ahead. For example, the core stage has CDR currently scheduled for July 2014 that will assess the program’s design maturity. Further, qualification testing to determine if system components meet requirements is currently scheduled to begin in January 2015. As we have found in the past, the integration and test phase of development is generally where issues that require unexpected time and resources to address are discovered.

Heritage Hardware

The SLS program could experience additional schedule pressure if unanticipated challenges associated with using heritage hardware occur when integrating it into the launch vehicle’s operational environment and modifying manufacturing process to incorporate new materials. The use of heritage hardware—legacy engine, booster, and propulsion systems— was prescribed in the NASA Authorization Act of 2010, but the hardware was not originally designed for SLS. Therefore, the SLS program must ensure each heritage hardware element meets SLS performance requirements and current design standards prior to the 2017 test flight. Although the heritage hardware challenges have yet to affect the SLS schedule, each heritage hardware element shares the common issue of operating in the SLS environment that is likely to be more stressful than that of its original launch vehicle as well as unique integration issues particular to that element, which must be resolved prior to SLS first flight in 2017. For example, according to agency officials the engines from the Space Shuttle require additional heat shielding because of the increased temperatures they will experience in the SLS environment, and the avionics within the solid rocket boosters from the Constellation program are likely to require additional cushioning to protect them from increased vibrations. Until the core stage is demonstrated, however, the SLS operating environment can be defined only through analytical predictions. Further, eliminating asbestos as a key insulating material within the solid rocket boosters on the SLS has required changes to the booster manufacturing processes to meet safety requirements.

According to agency officials, the challenges they face are typical of those expected when heritage hardware is integrated into a new system. They also noted that addressing these challenges involves less time, money, and overall effort than developing new hardware and that each heritage hardware element has schedule reserves to address anticipated integration challenges as well as unknown issues that might arise. Nevertheless, the engineering effort required to address them is significant and, as found by both GAO’s and the NASA Inspector General’s prior work, the complexity associated with required modifications and problems with availability of components used on projects is often underestimated.

Unclear Scope and Funding Uncertainties Increased Risk by Delaying Contract Definitization

Uncertainty regarding resources increased risk to the program because contracts were not finalized and complete contractor performance data was not provided. According to agency officials, the program was delayed in definitizing contracts, in part, due to resource uncertainty. As a result, the SLS program contractors have worked for extended periods of time without contract definitization—meaning no final agreement on the terms and conditions of their contracts has been negotiated with the government—which put the government at risk of increased costs and limited the program’s ability to monitor contractor progress. Contractors have performed, and are performing, work under undefinitized contract actions (UCA) until their respective contracts are definitized and cost and schedule baselines are set. Contract actions such as these authorize contractors to begin work before reaching a final agreement with the government on contract terms and conditions. The government is thus in a weaker position to control costs.

The NASA supplement to the Federal Acquisition Regulation (NFS) provides that the NASA goal is to definitize UCAs within 180 days of issuance, or approximately 6 months.ability to definitize the element contracts was impacted by the need to use existing contracts, as directed by the NASA Authorization Act of 2010, and to complete work necessary to define requirements and modify those contracts in order to accommodate the SLS effort’s scope and content.

Officials stated that the agency’s The NFS allows for such use of UCAs on an exception basis and also notes that, when UCAs are used, the agency’s liabilities and commitments must be minimized. As figure 4 illustrates, the booster and main engine element contractors performed development work under UCAs for more than a year and the core stage and ICPS contractor continued those elements’ development under UCAs for 2 years or more.

NASA allowed high-value modifications to the SLS contracts to remain undefinitized for extended periods—in one instance a contract remained undefinitized for 30 months. Because lack of agreement on contract terms prolonged NASA’s timeframes for finalizing SLS contracts, the establishment of contractor cost and schedule baselines necessary to monitor performance was delayed. Specifically, in most cases, the SLS program did not receive complete earned value management (EVM) data derived from approved baselines on these SLS contracts. Earned value, or the planned cost of completed work and work in progress, can provide accurate assessments of project progress, produce early warning signs of impending schedule delays and cost overruns, and provide unbiased estimates of anticipated costs at completion. The use of EVM, which integrates the project scope of work with cost, schedule, and performance elements for optimum project planning and control, is advocated by both GAO’s best practices for cost estimating and NASA’s own guidance. Without having a contracted baseline against which to measure the earned value, the program is missing a useful indicator of true program status—in the case of SLS, the program’s performance toward its 2017 launch date.

GAO, Defense Contract Management: DOD’s Lack of Adherence to Key Contracting Principles on Iraq Oil Contract Put Government Interests at Risk, GAO-07-839 (Washington, D.C.: Jul. 31, 2007); Defense Contracting: Use of Undefinitized Contract Actions Understated and Definitization Time Frames Often Not Met, GAO-07-559 (Washington, D.C.: Jun. 19, 2007); and Rebuilding Iraq: Fiscal Year 2003 Contract Award Procedures and Management Challenges, GAO-04-605 (Washington, D.C.: Jun. 1, 2004). from their internal baselines. Until the program and the contractor reach final agreement on contract terms, however, this data may not reflect whether the contractor is accomplishing the work planned within the agency’s allocated cost and schedule. Until baselines are set and an integrated baseline review is held, the cost and schedule data can only be used as indicators of progress. Furthermore, such data can be misleading as it lacks a comparison between work performed versus planned.

SLS Program Has Critical Gaps in Knowledge Needed to Assess Long-Term Affordability, but Opportunities Exist to Promote Affordability

NASA has yet to make mission decisions beyond EM-2 for the SLS program and has not produced a complete life-cycle cost estimate for any of the three planned variants; however, competition opportunities exist for future development work that may promote long-term affordability. Although the agency has identified several possible destinations, it has not decided upon specific missions for the SLS program beyond EM-1 and EM-2, which will directly affect the program’s future development path and flight schedule. Mission selection will likely determine the order of future development efforts, as the program can only afford to develop one upgraded element at a time. Each development effort may be large enough to constitute a separate project, but the full magnitude of these development efforts is unknown as the program has not established cost estimates for any of the three SLS variants, including the 70-mt vehicle, beyond 2017. Although the program costs are as yet unknown, there are opportunities to improve long-term affordability through competition once the element upgrade development path has been determined. While the main engine contractor for future SLS vehicles is likely to remain the same, the program plans to compete the acquisition of the advanced boosters. Additionally, the program could compete the acquisition of several other elements in order to potentially reduce costs.

The SLS Program’s Long-Term Mission Plans and Requirements Are Unknown

NASA has not yet defined specific mission requirements for any variant of the SLS. The two currently scheduled flights, EM-1 and EM-2, are developmental test flights designed to demonstrate and test the capabilities of the 70-mt launch vehicle and the capability of the core stage in particular. Office of Management and Budget guidance indicates that agencies should develop long range objectives, supported by detailed budgets and plans that identify the agency’s performance gaps and the resources needed to close them.beyond the two scheduled test flights, no future mission destinations have been determined. In the absence of specific mission requirements, officials indicated the SLS program is developing the variants based on top-level requirements derived from NASA’s Design Reference Architectures that lay out the technical and scientific framework for conducting missions in line with the agency’s strategic plan. NASA’s 2014 strategic plan, for example, identifies sending humans to Mars as one of the agency’s long-term goals; in turn, the agency’s Mars Design Reference Architecture indicates that multiple missions using a vehicle with a lift capability of about 130 mt will be necessary to support that goal.

According to agency officials, Although NASA is considering long-term Mars missions and asteroid redirect missions, NASA has not yet finalized plans for the next step in evolving the SLS and risks investing limited available resources in systems and designs that are not yet needed. The agency intends to use the same core stage design for all variants of the SLS, but it will be faced with replacing all the other major hardware elements used on the 70-mt vehicle—that is, the upper stage, boosters, and engines—so that the 105- and 130-mt vehicles can perform missions that support the agency’s strategic goals of carrying heavier payloads and traveling deeper into space. To that end, NASA is working on analyses and concept designs for a new upper stage and for advanced boosters. Because mission requirements for the 105-mt variant are not yet defined, however, NASA faces the possibility that it will spend time and money developing systems and designs that may not be needed for the specific mission that will be defined. For example, according to program officials and as illustrated in figure 5, in order to have a 105-mt capability ready to launch soon after 2021, the agency must make a decision not later than 2016 about whether it will first pursue development of the upper stage or advanced boosters as the next step in increasing SLS capability beyond the 70-mt variant.

Program officials stated that the agency does not have resources to conduct more than one development effort at a time and that it is difficult to determine which system to develop first because each improves SLS lift capability in different ways. A new upper stage would provide more capability beyond Earth orbit and better support missions that require more in-space propulsion, such as missions to a near-Earth asteroid or other distant locations. Advanced boosters would provide more capability to low-Earth orbit and better support missions that require the SLS to place a larger payload in orbit around the Earth. No matter which effort NASA pursues, in order for the agency to begin full-scale development after 2016, it is confronted with requesting related funding in the year before it plans to begin work. For example, if NASA intends to initiate full- scale upper stage development in 2017, including the related funding request in its fiscal year 2016 budget submission would be prudent. The sooner NASA makes a decision about which development to pursue first, the sooner it will be able to focus its limited resources on the chosen effort. Conversely, the longer the agency delays the decision, the more likely it will expend those resources on a design that might not yet be needed.

Regardless of when the mission requirements are established and what development NASA first pursues, those future development efforts are likely to be of sufficient cost and scope to constitute programs or projects in and of themselves. According to NASA officials, the advanced booster and new upper stage, for example, will involve distinct development, integration, qualification, and testing efforts. As such, they meet NASA’s own criteria for a new project in that they will represent a specific effort with defined requirements, a life-cycle cost estimate, and a beginning and end, and will result in new or revised products, in the form of improved boosters and stages, to address strategic goals. Further, these efforts are likely to far exceed the $250 million life-cycle cost established in NASA policy as the threshold for requiring the NASA Associate Administrator approval of cost and schedule baseline commitments. These projects are subject to oversight by senior agency officials through periodic programmatic reviews that assess technical and programmatic readiness to move forward at various decision points. At this time, future SLS development efforts are not planned to be managed as separate projects, though NASA has indicated that it plans to hold design reviews for these efforts consistent with its acquisition policies and practices. Best practices for system acquisition and cost estimating advocate an evolutionary approach wherein each capability increment has its own cost, schedule, and performance baselines and methods to report progress against these baselines. In addition, we have previously concluded that it is prudent for an agency to manage increasing capabilities of an existing program on par with the level of investments yet to come and in a way that is beneficial for oversight. For example, we have recommended that agencies developing weapon systems in increments consider establishing each increment of increased capability with its own cost and schedule baseline. Establishing cost, schedule, and performance baselines for each increment that has costs exceeding $250 million could provide decision makers with additional insights into the progress and long-term affordability of each increment.

The Program’s Long-Term Affordability Is Unknown Because SLS Cost Estimate Does Not Capture Life Cycle Costs

The long-term affordability of the SLS program is also unknown, as we found in May 2014, because NASA’s baseline cost estimate for the program will not provide any information about the longer-term, life cycle costs of developing, manufacturing, and operating the launch vehicle. NASA does not plan for that baseline estimate, which will be established when SLS moves into implementation, to cover program costs after EM-1 or costs to design, develop, build, and produce the 105- or 130-mt variants.

Based on the tenets of widely-accepted best practices for cost estimation as well as NASA’s own requirements and guidance regarding life cycle costs, in May 2014 we made recommendations to enhance transparency, assist congressional oversight, and ensure insight into affordability as the agency moves ahead with the SLS, Orion, and related ground support In terms of SLS, we recommended that NASA establish a programs. separate cost and schedule baseline for work required to support SLS Block I EM-2 and report that information via its annual budget submission. As part of that recommendation, we noted that, if NASA decides to fly the SLS Block I 70-mt variant beyond EM-2, the agency should then establish separate life cycle cost and schedule baseline estimates for that variant and report this information via its annual budget submission. Additionally, we recommended that NASA establish life cycle cost and schedule baselines for each upgraded block of the SLS, Orion, and associated ground support. We stated that, if the agency believed it could not do so because missions and flight manifests were unknown, then it should forecast minimum and maximum ranges for the increased capabilities’ life cycle costs in its annual budget submission. NASA partially concurred with our recommendations, citing among other reasons that actions already in place such as establishing SLS, Orion, and related ground support as separate programs and a block upgrade approach for SLS— and actions it plans to take to track costs—met the intent of our recommendations.

GAO-09-3SP; NPR 7120.5E § 2.4(Aug. 14, 2012); 2008 NASA Cost Estimating Handbook; and GAO-14-385.

In our evaluation of NASA’s comments, we acknowledged that these actions were a step in the right direction, but noted that NASA’s actions do not fully address our concerns. Specifically, we pointed out that establishing cost and schedule at the broader program level was unlikely to provide the detail necessary to monitor the progress of each block against a baseline; it was unclear from NASA’s response whether cost commitments the agency plans within the SLS design review process would serve the same purpose as establishing a cost baseline for each respective upgrade; and reporting costs associated with EM-2 and subsequent variants of SLS via the agency’s annual budget submission would not provide information about potential costs over the long term because budget requests neither offer all the same information as life- cycle cost estimates nor are necessarily linked to an established baseline that indicates how much NASA expects to invest to develop, operate, and sustain a capability over the long-term. We stated that, as NASA establishes parameters for the additional flights of the first SLS capability and upgraded capabilities, including flight rates, mission destinations, and other requirements, it will be well-poised to move from reporting costs in budget submissions to establishing baseline cost and schedule estimates for each capability and reporting progress against these respective baselines.

NASA Taking Steps for Long-Term Affordability but Additional Opportunities May Exist

NASA has taken some steps that may promote the long-term affordability of the SLS program as directed in the National Space Transportation Policy. The policy requires that human space flight development efforts such as the SLS program identify and implement measures to enhance affordability. To promote affordability, SLS has funded a series of studies and assessments on advanced boosters by both potential industry partners and universities. According to program officials, these activities—which include improved welding procedures, simulation of material performance, and demonstration of potential propellants—may provide better understanding of the processes, structures, and risk levels of various booster development options, which may in turn reduce the cost or schedule necessary to design and develop an advanced booster. Program officials also indicated that such activities may provide better insight into which contractors may compete for future booster development contracts.

The program, however, could promote longer term affordability by introducing competition in the development and production contracts for other hardware elements. Our body of work on contracting has shown that competition in contracting is a key element for maintaining cost control. We have found that promoting competition increases the potential for acquiring quality goods and services at a lower price and that noncompetitive contracts carry the risk of overspending because, among other reasons, they have been negotiated without the benefit of competition to help establish pricing. NASA has structured other launch programs, such as the Commercial Crew and Launch Services Program, based on the premise that competition will lower costs. In addition, the NASA Authorization Act of 2010 requires that NASA compete sub- elements of the SLS. Furthermore, with certain exceptions, full and open competition in soliciting offers and awarding contracts is generally a requirement of the Competition in Contracting Act of 1984 and the Federal Acquisition Regulation. Given the long-term planned use of SLS, using non-competitive procedures to acquire any portion of the SLS vehicle may lead to higher costs that could be felt for years or decades.

As it replaces hardware elements used on the 70-mt variant with elements necessary to support NASA’s anticipated uses of the 105- and 130-mt variants, the SLS program will likely have several opportunities to promote long-term affordability through competing contracts for the new elements. The program plans to use only the core stage and main engines for all flights beyond EM-2. Program briefings indicate that the SLS program optimized the design of the core stage, which will be used with all vehicle variants, to utilize the RS-25 engines remaining from the Space Shuttle program. According to agency officials, a natural consequence of finalizing the core stage design based on the RS-25 is that the only engine-related efforts open to future competition will be RS- 25 subsystems and components. The program could be in a favorable position, however, to compete contracts for the exploration upper stage, the upper stage engine, and advanced boosters that it expects to use on the 105- and 130-mt variants. According to SLS program officials, it currently plans to compete contracts for the development and production of new advanced boosters that will be more capable than the current five- segment boosters. The program is considering multiple design solutions including metallic or composite casings and liquid or solid fuel. Based on our review of program documentation and discussions with agency officials, however, additional opportunities exist to compete the procurement of other sub-elements needed for the 105- and 130-mt vehicles. In some cases other programs, both within and outside of NASA, use alternative hardware that serves similar purposes.

According to program officials, the program plans to procure a new upper stage through the existing core stage contract to replace the ICPS, which the program is using as an interim solution, for all missions following EM-2. They also stated that the program does not plan to compete the upper stage development because its initial award to Boeing in 2007 under the Constellation program was done competitively. Since that award, however, our work indicates that the marketplace for spacecraft development has shifted considerably as new commercial providers have since developed and have launched, or are currently developing, upper stages. These providers may offer viable competition for the new upper stage. For example, in recent years Orbital Sciences Corporation and SpaceX have successfully flown upper stages for NASA’s Commercial Cargo program and SpaceX and Boeing will need to human-rate upper stages as part of NASA’s Commercial Crew program with flights expected to begin around 2017.

The SLS program must also procure a new upper stage engine.

According to NASA officials, the upper stage engine will be selected based on mission need. Potential competitors include Aerojet Rocketdyne’s J-2X, the Evolved Expandable Launch Vehicle’s RL10, and the BE-3 engine that Blue Origin is developing for the Commercial Crew program, among other options.

Because, except for the RS-25 engines, NASA’s current contracting approach for the SLS program, does not commit the program beyond the hardware needed for EM-2, moving forward the agency will be in the position to take advantage of the evolving launch vehicle market. An updated assessment of the launch vehicle market could better position NASA to sustain competition, control costs, and better inform the Congress about the long-term affordability of the program.

Conclusions

NASA established the SLS program to provide the capability for transporting humans into space, but the agency is unlikely to do so as quickly as it intended. The initial launch date for SLS is just 3 and a half years away. That is a short time given the amount of development, integration, and testing that must occur for the vehicle to fly in 2017. While the technical challenges associated with those efforts appear manageable, the compressed development schedule in conjunction with the agency’s relatively flat funding profile for SLS through 2017 place the program at high risk of missing the planned December 2017 launch date for the EM-1 initial test flight.

Beyond the SLS second flight in 2021, the program’s path is unclear. NASA is waiting for additional policy direction on future missions, but the agency is approaching a crossroads wherein it is confronted with defining a developmental path forward to the more capable variants of SLS. Without identifying a range of mission possibilities and their required funding, the program is at risk of making uninformed decisions and pursuing development paths that may not make the most efficient use of limited resources in the near term and could negatively impact longer term affordability. Furthermore, without this information the agency’s ability to make important decisions about the affordability of the program in the context of the agency’s overall budget and competing priorities is limited. Additionally, it is unclear how the agency plans to manage the upgraded capability efforts. Without carefully structuring the approach for future variants, decision makers will lack transparency into costs and will be limited in their ability to assess long-term affordability and progress. This lack of knowledge about program direction and cost is of concern because the success of these programs will be measured by both the capabilities they achieve and their affordability. NASA has at its disposal one critical tool for addressing SLS affordability concerns—competition— more specifically, competition for procurement associated with the two variants’ elements. While NASA is planning to compete procurement of the advanced boosters, the agency also has opportunities to compete procurement of other upgraded elements, including the upper stage and the upper stage engine to help it achieve long-term affordability.

Recommendations for Executive Action

To provide the Congress with the necessary insight into program planning and affordability, and to decrease the risk of cost and schedule overruns, we recommend that NASA’s Administrator direct the Human Exploration and Operations Mission Directorate to take the following four actions:

NASA should develop an executable business case for SLS based on matching requirements and resources that results in a level of risk commensurate with its policies. For example, NASA could delay the planned first flight test or increase funding to allow the program to establish cost and schedule baselines for demonstration of the initial capability at the 70 percent confidence level.

To provide decision makers with an informed basis for making investment decisions regarding the SLS program, NASA should identify a range of possible missions for each future SLS variant that includes cost and schedule estimates and plans for how those possible missions would fit within NASA’s funding profile.

To allow for a continued assessment of progress and affordability, NASA should structure each future increment of SLS capability with a total cost exceeding the $250 million threshold for designation as a major project as a separate development effort within the SLS program. In doing so, NASA should require each increment to complete both the technical and programmatic reviews required of other major development projects, per the agency’s acquisition and system engineering policies.

To promote affordability, before finalizing acquisition plans for future capability variants, NASA should assess the full range of competition opportunities and provide to the Congress the agency’s assessment of the extent to which development and production of future elements of the SLS could be competitively procured.

Agency Comments

NASA provided written comments on a draft of this report. These comments are reprinted in appendix II.

In written and oral comments on a draft of this report, NASA concurred with our four recommendations. NASA agreed that future increments of SLS should be structured as separate development efforts and indicated that those increments would be subject to the appropriate technical and programmatic reviews, as well as rigorous cost and schedule management. NASA also recognized the need to define missions to guide program planning and indicated it will develop plans for specific SLS missions within NASA’s standard mission selection and review process.

We would expect that this process would also include cost and schedule estimates associated with each option and plans for how these possible missions would fit within NASA’s funding profile. NASA acknowledged the importance of establishing a business case and cost and schedule baselines for the SLS program consistent with NASA policies and recognized that the program’s schedule may have to be revised to accommodate decreased funding levels or to address technical problems. We would expect that an executable business case for the SLS program would be based on cost and schedule baselines that reflect a joint cost and schedule confidence level of 70 percent. NASA also indicated it would assess opportunities for increased competition and follow all applicable federal and NASA acquisition regulations governing competition within its contracting activities. As our recommendation indicates, NASA’s assessment of competitive procurement opportunities should address development and production of all future elements of the SLS, including the upper stage, and result in a report of this current assessment to the Congress. Separately, NASA provided technical comments, which have been incorporated into the report, as appropriate.

We are sending this report to NASA’s Administrator and to interested congressional committees. In addition, the report will be available at no charge on GAO’s website at http://www.gao.gov.

Should you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-4841 or chaplainc@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Other key contributors to this report are listed in appendix III.

Appendix I: Scope and Methodology

In order to assess the National Aeronautics and Space Administration’s (NASA) progress to conduct its first flight in 2017, we interviewed and obtained briefings and relevant documents from NASA and contractor officials. We identified and evaluated technical and programmatic issues associated with each major Space Launch System (SLS) element, including new and heritage hardware, by reviewing associated development plans and discussing relevant schedules and issues with agency officials. We also compared planned and actual progress in maturing system designs at the element and program level and evaluated how delays in element maturation and integration could affect the SLS program as a whole. We assessed NASA’s risk mitigation plans for SLS to gauge the program’s progress in addressing technical issues and to evaluate the potential cost and schedule impact to program milestones, including events such as delivery and flight dates. We also reviewed other technical and programmatic indicators—including progress against mass and performance goals, progress made addressing required actions from programmatic reviews at both the subsystem and vehicle level, and the status of schedule threats to the program’s critical path—and assessed program status against those indicators. We assessed the status of the program’s allocation of top level requirements to the different elements and the preparation of verification and validation plans for those requirements at the preliminary design review. We also evaluated the status of contracting activities for each element, including determining whether respective contracts were definitized and baselined and whether earned value information about contractor progress against planned cost and schedule was available to the program.

To assess long-term affordability, we discussed long-term development plans in support of future missions with agency officials. We also reviewed the design reference architecture establishing the need for the large launch vehicle and compared the program’s approach for satisfying the need to the agency’s strategic plan. For purposes of assessing the cost estimate, we reviewed NASA preliminary cost estimates for the SLS, Orion, and associated ground systems programs and information related to NASA’s plans for the SLS baseline cost estimate in order to determine the scope of the estimates and assessed whether that scope provided transparency into costs and enabled assessment of long-term affordability. We assessed the preliminary estimates’ scope against best practices criteria outlined in GAO’s cost estimating guidebook as well as NASA’s own guidance and procedural requirements. We also discussed the estimate with NASA officials, including the rationale for the estimates’ scope and exclusions to that scope. As NASA had not released the baseline estimate for SLS at the time of our review, we did not asses the reliability of the SLS baseline estimate. Furthermore, we evaluated the program’s development and acquisition plans to compete future variants of the SLS by reviewing contract information, including any follow-on contract options, and discussing supplier availability with agency officials.

We conducted our work at locations where the SLS program is being managed and executed including NASA headquarters in Washington, D.C.; NASA’s Kennedy Space Flight Center in Cape Canaveral, Fla.; NASA’s Marshall Space Flight Center in Huntsville, Ala.; AeroJet Rocketdyne in Canoga Park, Calif.; Alliant Technologies in Promontory, Utah; and the Boeing Company in Huntsville, Ala.

We conducted this performance audit from June 2013 to July 2014 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the National Aeronautics and Space Administration

Appendix III: GAO Contact and Staff Acknowledgments

GAO Contact

Cristina T. Chaplain (202) 512-4841 or chaplainc@gao.gov.

Staff Acknowledgments

In addition to the contact named above, Shelby S. Oakley (Assistant Director), Andrea M. Bivens, Tana M. Davis, Laura Greifner, Sylvia Schatz, Ryan Stott, Roxanna T. Sun, and John S. Warren, Jr. made key contributions to this report.