Space Transportation: Challenges Facing NASA's Space Launch Initiative
Released: 2002-10-02T00:00:00

Letter

Background

NASA’s Space Launch Initiative is an effort to develop and build a second generation of reusable space transportation vehicles. (See fig. 1 and table 1.) NASA’s current transportation vehicle, the space shuttle, has been in use for 20 years and requires a significant portion of NASA’s resources to operate and maintain. The primary goals for SLI are to reduce the risk of crew loss as well as substantially lower the cost of space transportation so that more funds can be made available for scientific research, technology development, and exploration activities. Currently, NASA spends nearly one-third of its budget on space transportation.

SLI is part of a broader program—known as NASA’s Integrated Space Transportation Plan—to address space transportation needs. Under the plan, NASA could operate the space shuttle through 2020 and make software and hardware upgrades to the shuttle in order to extend its use to this point. It envisions the deployment of second-generation cargo vehicles to begin around 2011 and crew vehicles around 2014. As with the shuttle, NASA envisions that the second-generation vehicle will reach orbit in two stages. NASA also anticipates building a third generation of vehicles in 2025 and even a fourth generation in 2040. (See fig. 2.) NASA anticipates that these vehicles would reach orbit in one stage; travel beyond low-earth orbit to far-reaching interstellar missions; and employ revolutionary technologies, such as (1) rocket engines that breath oxygen from the air during the climb to orbit rather than carrying heavy oxidizers onboard, (2) propellantless electromagnetic propulsion, and (3) solar-powered space sails.

Building the second-generation vehicle will be a considerably complex and challenging endeavor for NASA—from both a technical and business standpoint. For example, NASA plans to develop and advance new technologies for the new vehicle. These include (1) new airframe technologies that will include robust, low-cost, low-maintenance structure, tanks, and thermal protection systems, using advanced ceramic and metallic composite materials, and (2) new propulsion technologies, including main propulsion systems, orbital maneuvering systems, main engines, and propellant management. If successfully developed, NASA believes that these new technologies could substantially reduce the costs to maintain and operate the vehicle and increase its reliability.

NASA also plans to develop the new vehicle through partnerships with private industry, academia, and DOD. It is partnering with the private sector so that it can help create business opportunities in space, and it is partnering with DOD because of military needs for a reusable launch vehicle. Such partnerships will require a high level of coordination and communication, since agreements need to be reached on what the basic capabilities of the new vehicle will be, what designs or architecturesshould be pursued, how development costs will be shared, and what individual partner responsibilities will be.

NASA recognizes that there are substantial technical and business risks with SLI, and it is undertaking activities aimed at reducing them. To reduce technical risks, for example, NASA is exploring, developing, and testing technologies to make sure that they can be used on the new vehicle, rather than proceeding with the program with uncertainty about whether technologies will be mature enough when they are needed. To reduce business and costs risks, NASA’s goal is to ensure that there is adequate competition in the development of the architecture and that the architectures that NASA pursues will enable convergences with NASA’s requirements and commercial and military missions, and will not pursue vehicles that cannot meet program goals.

To date, NASA has already explored hundreds of concepts and technologies that could be used for future government and commercial launch systems and space transportation operations. The “leap-ahead” technologies studied include crew survival systems, advanced tanks and airframe structures, long-life rocket engines, and thermal protection systems. In March 2002, NASA selected 15 of the most promising candidates to go forward into more detailed development. In November 2002, NASA plans to narrow the field down to 3 candidates and in September 2003 to at least 2 candidates. From fiscal year 2003 through fiscal 2006, two competing system architectures will be developed, ready for a full-scale development decision in fiscal 2006.

NASA plans to spend $4.8 billion for the SLI program through fiscal year 2006. This investment is to be used for (1) systems engineering and activities concerning the definition of requirements; (2) efforts to compete designs for the reusable launch vehicle as well as efforts to reduce business and technical risks; (3) activities to develop and demonstrate designs, technologies, and system-level integration issues associated with such NASA-unique transportation elements as a crew transport vehicle and cargo carriers; and (4) activities to develop alternative access to the space station as well as to provide contingency backup or relief for the space shuttle. NASA’s efforts to compete designs for the vehicle and reduce technical and business risks consist of approximately $3 billion of the planned initial investment. At the present time, no prototype vehicle integrating all new SLI technologies is planned before NASA’s fiscal year 2006 decision on whether to proceed to full-scale development.

Important Decisions to Be Made Before Requirements Can Be Defined

According to a NASA official, NASA plans to define the basic requirements for its second-generation space transportation—that is, what the crew size will be, what the payload capacity will be, and what designs or architectures are worth pursuing—by November 2002. These decisions will have a significant impact on the cost, size, and design of the new vehicle. For example, as the payload capacity increases, so does the thrust requirement for the propulsion system, and in turn, the cost to develop and build the system, as well as the cost to operate the system.

However, NASA is facing a considerable challenge in reaching the point to where it can finalize SLI requirements. This is primarily because NASA must first make some difficult broader decisions regarding the future of space transportation and other NASA projects, including (1) whether NASA should skip development of a second-generation vehicle in favor of concentrating on the third generation, (2) what the future of the International Space Station will be, and (3) what DOD’s role in developing the new vehicle will be. Moreover, in making these decisions, NASA will need to reach consensus with a wide range of partners who have priorities and concerns different from NASA’s, and it will need to make trade-offs amid uncertainties as to the availability of advanced technologies.

First, NASA must complete its ongoing reassessment of its overall space transportation plan. This evaluation is being done as part of NASA’s development of a budget proposal for fiscal year 2004. The options NASA is considering could have a far-reaching impact on the SLI program. One option, in fact, involves postponing efforts to develop a new generation of vehicles indefinitely until there is a major breakthrough in technology that could vastly improve performance and reduce costs. Other options being examined could drastically change NASA’s timetable and requirements for SLI. For example, NASA is looking at developing a crewed vehicle more quickly than currently planned and launching it atop an expendable rocket until it can field a reusable launch vehicle. NASA is also considering deferring some development efforts so that it can aggressively pursue a goal of building a relaunchable vehicle prototype by 2009. Finalizing its decisions on where to go with the space transportation plan will be difficult for NASA, given the uncertainties about the availability of technologies needed to reduce costs and achieve NASA’s performance and safety objectives.

Second, NASA is currently reassessing the future of the International Space Station. One decision it needs to make that could significantly affect the SLI program is whether the station should support a crew of seven astronauts, as originally planned, or three. Because of cost growth, NASA plans to cut back to a crew of three, but its international partners have not agreed yet to this decision and are concerned that the cutback will severely undermine planned scientific research. This decision could significantly affect the design and cost of the second-generation vehicle. For example, with a bigger crew size, the overall vehicle will have to be larger; a larger crew cabin and additional backup systems will be required; and, as a result, the cost to develop the vehicle will increase.

NASA is also planning to cut back the number of flights to the station from seven to four per year, which could also have an impact on SLI. But again, NASA’s international partners have concerns about this decision, since it would limit the deliveries of resources needed to carry out research activities. At the same time that NASA is looking at cutting back on space station capabilities, it is planning to extend the life of its space shuttle to 2020 with software and hardware upgrades. If NASA can successfully extend the life of the shuttle, it may well find that a second-generation vehicle, which is not expected to begin transporting crews to the space station until at least 2014, would generally be duplicative in capability and therefore unnecessary. NASA plans to make final decisions on the space station by November/December 2002.

Third, while DOD and NASA have explored potential common areas of interest for a new space transportation vehicle, DOD has not yet formally defined its requirements. In fact, it is still uncertain whether SLI will be a joint DOD/NASA program. A study conducted by the Air Force and NASA earlier this year revealed that both NASA and DOD shared similar objectives when it came to technologies needed for the new vehicle as well as cargo lift requirements and launch architecture elements. But there were differences with priorities and certain requirements. The Air Force would like the vehicle to operate in stronger winds, more precipitation, and a wider range of temperatures. It would also like the vehicle to operate from an inland U.S. Air Force base. NASA would like the vehicle to stay in orbit for a longer duration and have more maneuverability because of its mission to service the International Space Station. Additionally, NASA will have a higher weight-delivered-to-orbit requirement for its crew vehicle.

It is apparent that some of DOD’s objectives and priorities may not match up with NASA’s. For example, DOD envisions developing an unpiloted vehicle, while NASA is focusing on a crewed design. DOD also envisions developing a vehicle that could be capable of relaunching within 12 to 48 hours and making as many as 20 flights in a 2-week time frame. At this time, NASA does not share these objectives. According to a DOD official, DOD expects to finalize its requirements before the end of 2002. Since DOD’s decisions will affect the size, design, and capabilities of the vehicle, it is important that these requirements be known before NASA finalizes its own requirements in November 2002. Otherwise, NASA will need to reexamine its requirements after narrowing down potential system architectures to three possibilities. While significant cost and operational benefits may accrue with a vehicle capable of satisfying both DOD’s and NASA’s requirements, NASA will need to guard against making compromises that might negatively affect its goal of substantially lowering launch costs.

Key Management Controls Are Not Yet Implemented

NASA cannot implement key management controls for the SLI program until it defines its basic requirements. These include cost estimates, controls designed to provide early warnings of cost and schedule problems, as well as risk mitigation plans. Moreover, NASA does not have plans to implement some performance measures, including ones that would assess overall progress toward achieving key objectives, reducing risks, and maturing technology. It is essential that these controls be implemented quickly so that NASA can predict what the total costs of the program will be and provide assurance that its investment in SLI is being spent wisely.

Importance of Management Controls for SLI

Undertaking ambitious, technically challenging efforts like SLI—which involve multiple contractors and technologies that have to be developed and proven—requires careful oversight and management. Importantly, accurate and reliable cost estimates need to be developed, technical and program risks need to be anticipated and mitigated, and performance and readiness need to be closely monitored. Such undertakings also require a high level of communication and coordination. Not carefully implementing such project management tools and activities is a recipe for failure. Without realistically estimating costs and risks, and providing budgetary reserves needed to mitigate those risks, management may not be in a position to effectively deal with the technical problems that cutting-edge projects invariably face.

In fact, we found that NASA did not successfully implement and adhere to a number of critical project management tools and activities in its previous efforts to build new space transportation vehicles. As we testified in June 2001, neither of NASA’s X-33 or X-34 programs—which attempted to build a new transportation vehicle that would reach orbit in one stage—-fully assessed the costs associated with developing new, unproven technologies; provided for the financial reserves needed to deal with technical risks and accommodate normal development delays; developed plans to quantify and mitigate risks to NASA; or established performance targets showing a clear path to an operational launch vehicle. Underlying these difficulties were problems with the agreements and contracts that established the relationship between NASA and its industry partners and the eventual erosion of commercial prospects for the development of new reusable launch vehicles. We testified that lax management controls led to numerous problems with both the X-33 and X-34 programs. Technical problems, for example, resulted in significant schedule and cost overruns, which NASA was ill prepared to deal with.

NASA has taken steps to avoid the problems it encountered with the X-33 and X-34 programs. In our testimony last year, for example, we pointed out that NASA planned to increase the level of insight into SLI projects by providing more formal reviews and varying levels of project documentation from contractors depending on the risk involved and the contract value. NASA also required that all proposals submitted in response to its research announcement be accompanied by certifiable cost and pricing data. Finally, NASA discouraged the use of cooperative agreements, since these agreements did not prove to be effective contracting mechanisms for research and development efforts where large investments are required.

Cost Estimates Cannot Be Developed Until Requirements Are Defined

NASA cannot develop cost estimates for the SLI program until it defines the basic requirements for the new transportation vehicle and narrows the field of possible architectures. Moreover, until requirements are defined, NASA cannot determine whether the $4.8 billion already planned for SLI through fiscal year 2006 will achieve the desired results for the program’s formulation. NASA guidance requires that life-cycle costs be estimated, assessed, and controlled throughout a program’s life cycle. The estimates are to be prepared to support major program reviews and the development of budget submissions.

In our recent review of the International Space Station, we reported that NASA is facing additional challenges to developing reliable cost estimates and effectively using them to manage programs. For example, a recent study performed by the Rand Corporation for the Office of Science and Technology Policy found that NASA has “very good” cost and risk modeling capabilities. However, the study also found that NASA’s in-house capabilities were not well integrated into the program’s planning and management and that NASA programs had been reluctant to integrate cost estimate and control practices that were sufficiently robust to yield confidence in budget estimates. In addition, a task force appointed last year to conduct an independent external review and assessment of the space station’s cost, budget, and management found that NASA tends to take a short-term focus on executing programs—that is, rather than managing programs to come within overall cost and scheduling goals, it manages them around annual budgets. The task force cited NASA’s culture of managing the space station program in adherence to its annual budgets as perhaps the single greatest factor in the space station program’s cost growth.

We also reported that NASA’s ability to develop good cost estimates for programs is hampered by NASA’s lack of a modern integrated financial system to track and maintain data needed for estimating and controlling costs. NASA has made it a top priority to develop and implement a new system. However, the first major component of the system—the core financial system—is not expected to be implemented until June 2003.

It will be important for NASA to overcome barriers that relate to estimating costs for SLI and to use estimates to take a long-term perspective in managing the SLI program. Without good cost information, decision makers at NASA and in the Congress will not know whether the $4.8 billion investment is sufficient for the early phases of the program or how much more it will cost to actually develop and deploy the new vehicle. Moreover, NASA managers will lack the information they need to monitor costs, schedule, and performance.

Other Management Controls Cannot Be Implemented Without a Cost Estimate

Other management controls that are integral to successfully managing the SLI program cannot be implemented until NASA has a cost estimate. First, NASA has a system—known as Earned Value Management—intended to help provide program managers and others with early warnings of cost and schedule problems. But this system cannot be effectively implemented without having baseline requirements defined or detailed cost estimates, schedules, and timelines developed for the SLI program.

Earned Value Management goes beyond the two-dimensional approach of comparing budgeted costs with actual costs. It also attempts to compare the value of work accomplished during a given period with the value of work scheduled for that period. By using the value of work done as a basis for estimating the cost and time to complete it, the earned value concept should alert program managers to potential problems sooner than expenditures alone can. The communities that have a vested interest in earned value are the (1) program managers, who are charged with overall management responsibility for acquisition programs; (2) contractors, who are responsible for the contract’s successful execution; and (3) overseers, such as acquisition executives, financial managers, contract surveillance officials, and cost estimators, who are tasked with tracking and estimating program costs.

Second, individual SLI activities have prepared risk mitigation plans. However, a program official told us that these plans cannot be quantified at the overall program cost and schedule level without complete cost estimates. Moreover, without a cost estimate, risks can be measured only on a year-by-year basis and not on a multiyear basis—for example, to 2006. Risk mitigation plans identify, assess, and document the risks associated with the cost, resource, schedule, and technical aspects of a project and determine the procedures that will be used to manage those risks. In doing so, they help ensure that a system will meet performance requirements and be delivered on schedule and within budget. NASA’s guidance requires that these plans be developed during the formulation phase of a project.

Measures to Assess Performance Not Implemented

Agencies are required to prepare annual performance plans that establish performance goals with measurable target levels of performance for each program activity in the agency’s budget and to provide a basis for comparing actual performance with performance goals. Doing so enables agencies to gauge the progress of programs like SLI and, in turn, to take quick action when performance goals are not being met.

Individual SLI activities are using computer simulations to help gauge whether the technologies under development can meet specific performance targets. However, NASA has not yet established broader measures that can show NASA decision makers and the Congress whether the program as a whole is meeting objectives such as reducing the payload cost to approximately $1,000 per pound and reducing the risk of crew loss to approximately 1 in 10,000 missions. Like cost and risk controls, the development of such metrics depends on NASA’s definition of requirements and cost estimation for SLI.

It is important that NASA develop such measures as soon as possible. We testified last year that one problem that hampered the previous X-33 and X-34 efforts was the fact that NASA had not developed performance targets that establish a clear path leading to a reusable launch vehicle.

In addition, while NASA plans to assess the readiness of technology to gauge the maturity of individual technologies, it does not plan to verify and validate the maturity of technology for the SLI program as a whole. To ensure that individual technologies are sufficiently mature by NASA’s planned full-scale development decision for 2006, NASA intends to use technology readiness levels (TRLs). Our prior reports have shown that TRLs, which were pioneered by NASA, are a good way to gauge the maturity of technologies. Readiness levels are measured along a scale of 1 to 9, starting with paper studies of the basic concept, proceeding with laboratory demonstrations, and ending with a technology that has proven itself on the intended product. NASA would like to achieve a TRL of 6 by 2006 for key technologies, such as the propulsion system. At this level, a prototype is tested in a relevant environment, such as a high- fidelity laboratory environment or in a simulated operational environment. Currently, most of the technology areas are at levels 3 or 4. At level 3, analytical studies and laboratory studies are performed to physically validate analytical predictions of separate elements of the technology. At level 4, basic technological components are integrated to establish that the pieces will work together.

While assessing technologies separately should help NASA decide when and where to insert new technologies into the SLI program, it is still important for NASA to look at the readiness of the product as a whole because how well various components being developed will work together is unknown.

Another measure that NASA officials told us they do not plan to implement is one that would assess the extent or percentage of total risks that have been reduced with NASA’s initial $4.8 billion investment as well as the amount of risk remaining. Program officials told us that it would be too resource-intensive to develop this measure. However, in the absence of such information, decision makers at NASA and in the Congress have little assurance that the $4.8 billion investment in SLI can fulfill the goals expected of the program, and they have reduced confidence that what has already been spent has placed NASA on track to meeting its primary goals.

Conclusions

NASA aims to be able to define system requirements for SLI by November 2002. But meeting this goal may not be realistic. NASA must first decide whether developing a second-generation vehicle to be deployed in 2014 to 2015 is still a worthwhile endeavor, given plans to extend the life of the space shuttle and cut back on the space station’s capabilities, and if so, what specific direction the program should take and how it will fit in with DOD efforts. Making such decisions within the short time remaining before the SLI systems requirements review will be difficult because it will require NASA to (1) resolve differences with its space station partners, who have concerns about planned cutbacks to the station; (2) reach consensus with DOD on requirements and priorities, which are now considerably different; and (3) make trade-offs as to what capabilities and technologies it should pursue without really knowing when advancements can be achieved. When NASA is able to resolve these challenges and finalize requirements for SLI, it will be critical for NASA to swiftly implement effective management to oversee the effort. Until it does so, NASA will not be able to assure its managers and the Congress that the initial investment is being spent wisely and that risks are being reduced, and it will not be able to predict what the total costs of the program will be.

Recommendations

We recommend that the NASA Administrator do the following: 1. Reassess the schedule for defining the requirements for the Space Launch Initiative in order to ensure that the agency takes the following actions before making final decisions on basic requirements and selecting three architectures to pursue: (1) complete the reassessment of NASA’s integrated space transportation plan, (2) reach consensus with its international partners on the future of the space station, and (3) reach consensus with the Department of Defense on its role in the SLI effort. 2. If DOD is to jointly develop the second-generation vehicle, reach consensus with DOD on priorities and objectives for SLI and factor DOD’s requirements into NASA’s own. 3. After NASA completes its system requirements review, ensure that a cost estimate is promptly developed for the SLI investment and that this estimate is detailed and reliable enough to be used to complete risk mitigation plans and carry out earned value management activities. 4. Ensure that NASA can demonstrate how the $4.8 billion initial investment supports the requirements that NASA decides to pursue in November 2002. 5. After system requirements are defined, ensure that performance measures are developed to assess the progress of the program, as a whole, toward (1) meeting key performance objectives, including lowering the cost of delivering payloads to low-Earth orbit to less than $1,000 per pound and reducing the risk of crew loss to approximately 1 in 10,000 missions, and (2) achieving an overall technology readiness level of 6 by 2006. 6. Ensure that measures are developed to assess the amount of risk reduced following each year of expenditure and the amount of risk remaining.

Agency Comments

In written comments on a draft of this report, NASA’s Associate Deputy Administrator said the agency concurs with the recommendations, adding that the program completed its first year as planned and within budget.

According to the Associate Deputy Administrator, the agency is reassessing the Integrated Space Transportation Plan and working with DOD to develop their requirements. Furthermore, NASA concurs with the recommendation concerning the development of a cost estimate. However, the agency notes that the RLV industry suffers uncertainties in predicting costs, primarily because of a limited set of models and data to validate those models. NASA also concurred with the need to validate SLI requirements and the need for performance measures to assess the progress of the program towards meeting objectives and achieving technology readiness levels.

Our draft report contained a recommendation that NASA develop measures to assess the percentage of risk reduced following each year of expenditures and the percentage of risk remaining. NASA believes that its process of narrowing potential SLI technology alternatives to the most promising concepts meriting further funding enables the successful candidates to be built and become operational. In that way, the SLI program is reducing the risks inherent in an advanced research and technology program of this magnitude. According to NASA, it will develop a reporting mechanism that communicates the amount of risk reduction achieved and remaining on an annual basis. Our intent is for NASA to provide a quantifiable means of measuring progress. Thus, we modified that recommendation accordingly.

Scope and Methodology

To assess the coordination of requirements with DOD and other NASA programs, we interviewed officials within the Department of Defense and NASA. We evaluated studies conducted by DOD and NASA on efforts to address requirements and coordination.

To address NASA’s management controls for the SLI program, we interviewed NASA officials regarding cost estimates and the process by which cost information is studied and communicated throughout NASA. We reviewed NASA’s and the Office of Management and Budget’s guidance regarding earned value management and discussed the program’s methods for coordinating activities. We also reviewed risk mitigation plans to determine how cost affects these plans and discussed these plans with program officials.

To assess program performance measures, we reviewed NASA’s policies and procedures governing program management and the program’s plans for assessing performance measures. We also interviewed SLI program officials to understand the program’s plans to meet goals and objectives.

To accomplish our work, we interviewed officials from NASA’s headquarters, Washington, D.C.; Marshall Space Flight Center, Alabama; Johnson Space Center, Texas; Kennedy Space Center, Florida; and Independent Program Assessment Office, Virginia; and DOD’s Space Command, Colorado.

We performed our review from October 2001 through August 2002 in accordance with generally accepted government auditing standards.

Unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from its issue date. At that time, we will send copies to the NASA Administrator; Director, Office of Management and Budget; and other interested parties. We will also make copies available to others upon request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Please contact me at (202) 512-4841 if you or your staffs have any questions about this report. Major contributors to this report are listed in appendix II.

Appendix I: Comments from the National Aeronautics and Space Administration

Appendix II: GAO Contact and Staff Acknowledgments

GAO Contact

Acknowledgments

In addition to the person named above, Cristina Chaplain, Ivy Hubler, Danny Owens, and Dana Solomon made key contributions to this report.