NASA: Constellation Program Cost and Schedule Will Remain Uncertain Until a Sound Business Case Is Established
Released: 2009-09-25T00:00:00

Letter

Background

NASA’s Vision for Space Exploration calls for a return of humans to the Moon and eventual human spaceflight to Mars. In September 2005, NASA outlined an initial architecture for implementing the Vision in its Exploration Systems Architecture Study (ESAS). NASA is implementing this architecture under the Constellation program. Among the first major efforts of this program are the developments of new space flight systems—including the Ares I Crew Launch Vehicle and the Orion Crew Exploration Vehicle. Ares I and Orion are currently targeted for operation no later than 2015 (see fig. 1).

As illustrated by figure 1 above, the Constellation program, including the Ares I and Orion projects, is approaching the end of the formulation phase of NASA’s acquisition life-cycle for spaceflight programs and projects. The purpose of the formulation phase is to establish a cost-effective program that is demonstrably capable of meeting the agency’s objectives. The formulation phase concludes with the preliminary design review and a non-advocate review which marks the end of the formulation phase and the beginning of the implementation phase. During the implementation phase, the program will execute plans developed during the formulation phase.

Our Work on Best Practices Support Business Case Establishment Prior to Product Development

Our work on best practices over the past decade has shown that success in large-scale development efforts like Constellation depends on establishing an executable business case before committing resources to a new product development effort. In its simplest form, a business case requires a balance between the concept selected to satisfy customer needs and the resources—technologies, design knowledge, funding, time, and management capacity—needed to transform the concept into a product. At the heart of a business case is a knowledge-based approach that requires that managers demonstrate high levels of knowledge as the program proceeds from technology development to system development and, finally, production. Ideally, in such an approach, key technologies are demonstrated before development begins, the design is stabilized before prototypes are built or production begins, and testing is used to validate product maturity at each level. At each decision point, the balance among time, money, and capacity is confirmed. In essence, knowledge supplants risk over time. Having adequate knowledge about requirements and resources is particularly important for a program like Constellation because human spaceflight development projects are inherently complex, difficult, and costly.

We have reported on several occasions that within NASA’s acquisition framework, the preliminary design/non-advocate review—the hurdle marking transition from program formulation to program implementation—is the point at which development projects should have a sound business case in hand. NASA’s Systems Engineering Policy states that the preliminary design review demonstrates that the preliminary design meets all system requirements with acceptable risk and within the cost and schedule constraints. NASA realized that the Orion project was not ready to complete the preliminary design review process as planned and delayed its initiation from summer 2008 to summer 2009. Furthermore, although NASA officially closed the Ares I preliminary design review process in September 2008, it deferred resolution of the thrust oscillation issue until the Constellation program preliminary design review in March 2010.

The business case is the essential first step in any acquisition program that sets the stage for the remaining stages of a program, namely the business or contracting strategy and actual execution or performance. If the business case is not sound, execution may be subpar. This does not mean that all potential problems can be eliminated and perfection achieved, but rather that sound business cases can help produce better outcomes and better return on investment. If any one element of the business case is weak, problems are more likely in implementation. Thus far in the Constellation program, the failure of NASA to establish a sound business case for both the Ares I and Orion projects early is manifesting itself in schedule delays and cost increases.

The Constellation Program Has Not Yet Developed the Sound Business Case Needed to Justify Entry into the Implementation Phase

The Constellation program has not yet developed all of the elements of a sound business case needed to justify entry into implementation. Progress has been made; however, technical and design challenges are still significant and until they are resolved NASA will not be able to reliably estimate the time and money needed to execute the program. In addition, cost issues and a poorly phased funding plan continue to hamper the program. Consequently, NASA is changing the acquisition strategy for the Orion project as the agency attempts to increase confidence in its ability to meet a March 2015 first crewed launch. However, technical design and other challenges facing the program are not likely to be overcome in time to meet the 2015 date, even with changes to scope and requirements.

Constellation Faces Significant Technical and Design Challenges

Technical and design challenges within the Constellation are proving difficult, costly, and time intensive to resolve. The Constellation program tracks technical challenges in its Integrated Risk Management Application (IRMA). NASA procedures recommend that programs identify and track risks as part of continuous risk management. As of June 9, 2009, IRMA was tracking 464 risks for Ares I and Orion—207 high risks, 206 medium risks, and 51 low risks. We have reported on some of these areas of technical challenge in the past, including thrust oscillation, thermal protection system, common bulkhead, and J-2X nozzle extension. In addition to these challenges, our recent work has highlighted other technical challenges, including Orion mass control, vibroacoustics, lift-off drift, launch abort system, and meeting safety requirements. While NASA has made progress in resolving each of these technical challenges, significant knowledge gaps remain in each of these areas. Descriptions of these technical challenges follow.

Thrust Oscillation

Thrust oscillation, which causes shaking during launch and ascent, occurs in some form on every solid rocket engine. Last year, we reported that computer modeling indicated that there was a possibility that the thrust oscillation frequency and magnitude may be outside the limits of the Ares I design and could potentially cause excessive vibration in the Orion capsule. Agency officials stated that thrust oscillation is well understood and they are pursuing multiple solutions. These include incorporating a passive damping system inside the first stage solid rocket booster aft skirt that will act like a shock absorber during launch; adding a composite structure and springs between the first and second stages to isolate the upper stage and crew vehicle from the first stage; and could possibly use the upper stage propellant fuel tanks to offset thrust oscillation in the first stage. Officials said that NASA will be unable to verify the success of solutions until thrust oscillation occurs during an integrated flight. Officials noted that because thrust oscillation is not expected to occur in every flight, it is difficult to forecast when the solutions will be verified.

Thermal Protection System

The Orion vehicle requires a large-scale ablative heat shield, at the base of the spacecraft, to survive reentry from earth orbit. These heat shields burn up, or ablate, in a controlled fashion, transporting heat away from the crew module during its descent through the atmosphere. NASA is using an ablative material derived from the substance used in the Apollo program. After some difficulties, NASA was successful in recreating the material. Because it uses a framework with many honeycomb-shaped cells, each of which must be individually filled without voids or imperfections, it may be difficult to repeatedly manufacture to consistent standards. According to program officials, during the Apollo program the cells were filled by hand. The contractor plans to automate the process for the Orion Thermal Protection System, but this capability is still being developed.

Common Bulkhead

The common bulkhead separates the hydrogen and oxygen fuel within the Ares I upper stage fuel tank. The initial Ares I design employed a simpler two-tank configuration with lower manufacturing costs but did not meet mass requirements. According to project officials, the common bulkhead represents the critical path in both the development and manufacturing of the upper stage. Lessons learned from the Apollo program indicate that common bulkheads are complex and difficult to manufacture and recommend against their use. According to NASA officials, the difficulty of designing and manufacturing common bulkheads stems from the sheer size of components and the tight tolerances to which they must be manufactured. To accelerate the manufacturing process NASA is exploring using an oven with a vacuum bag instead of an autoclave to bond and cure the metallic and composite materials used in the manufacture of the common bulkhead. If this process proves unsuccessful, the program may encounter schedule delays.

J-2X Nozzle Extension

We have reported in prior years that although the J-2X engine is based on the J-2 and J-2S engines used on the Saturn V and leverages knowledge from subsequent engine development efforts, the number of planned changes is such that, according to NASA review boards, the effort essentially represents a new engine development. A risk within this development is a requirement for a nozzle extension to meet performance requirements. NASA originally planned to pursue a composite nozzle. However, NASA eliminated the composite nozzle extension from the J-2X design because of cost and other considerations, and went with a unique aluminum alloy design, which, according to agency officials, should reduce costs, but has the potential to decrease engine performance and increase mass. Analysis indicates that the alloy nozzle is more likely to be affected by heat than a composite nozzle. In essence, while the alloy nozzle should withstand the heat environment, the composite nozzle allowed for improved performance margins. According to officials, to mitigate the potential problem, NASA is using a proven aluminum alloy with a honeycomb design, similar structurally to the Space Shuttle external tank, which will reduce weight. Contractor officials stated that they will continue to modify the nozzle design as test results are received and analyzed.

Orion Mass Control

Controlling for mass has led to significant design changes to the Orion vehicle. Our previous work has shown that controlling for mass is a key factor in the development of space systems. As the mass of a particular system increases, the power or thrust required to launch that system will also increase. This could result in the need to develop additional power or thrust capability to lift the system, leading to additional costs, or to stripping down the vehicle to accommodate current power or thrust capability. For example, NASA went through the process in 2007 to zero- base the design for the Orion to address mass concerns. In its efforts to reduce the mass of the Orion vehicle, NASA chose to move from land nominal landing to water nominal landing to reduce mass by eliminating air bags and, according to officials, by reducing the number of parachutes.

NASA also incorporated jettisonable, load-bearing fairings into the Orion’s service module design that, according to officials, saved 1,000 pounds. This change, however, increased development risk because the fairing design has no historical precedent and the fairing panels may not deploy properly and could recontact the Orion vehicle or the Ares I rocket after they are jettisoned.

Vibroacoustics

Another issue related to vibration is vibroacoustics—the pressure of the acoustic waves—produced by the firing of the Ares I first stage and the rocket’s acceleration through the atmosphere—which may cause unacceptable structural vibrations throughout Ares I and Orion. According to agency officials, NASA is still determining how these vibrations and acoustic environments may affect the vehicles. NASA is concerned that severe vibroacoustics could force NASA to qualify Ares I and Orion components to higher vibration tolerance thresholds than originally expected. For example, if current concerns are realized, key subsystems within the Upper Stage would be unable to meet requirements, would fail qualification testing, and would have to be redesigned.

Lift-Off Drift

Analysis of the Ares I flight path as it lifts off from the launch pad indicates the rocket may drift during launch and could possibly hit the launch tower or damage the launch facilities with the rocket plume. Factors contributing to lift-off drift include wind speed and direction, misalignment of the rocket’s thrust, and duration of lift-off. NASA plans to establish a clear, safe, and predicted lift-off drift curve by steering the vehicle away from the launch tower and not launching when southerly winds exceed 15 to 20 knots.

Launch Abort System

NASA continues to address challenges designing the launch abort system, which pulls the Orion capsule away from the Ares I launch vehicle in the case of a catastrophic problem during launch. The Orion contractor had trouble finding a subcontractor who could design and build a working attitude control motor that steers the system during an abort. According to agency officials, previous attitude control motors have had 700 pounds of thrust, while the requirement for the attitude control motor is 7,000 pounds of thrust. Developing an attitude control motor with high levels of thrust and long burn durations that is steerable is proving to be a difficult technical challenge. A year after the initial contract was awarded, the first subcontractor did not have a viable design and had to be replaced. The current subcontractor, however, is making progress. For example, although the valves used by the complex steering system failed during high-thrust testing in April 2008, redesigned valves have subsequently passed two high-thrust tests.

Safety Requirements

Orion’s safety requirements are no more than one loss of crew event in 1,700 flights and one loss of mission event for every 250 flights for the ISS mission. According to Orion officials, these requirements are an order of magnitude higher than the Space Shuttle’s safety requirements, were arbitrarily set by ESAS, and may be unattainable. According to the Constellation program manager, NASA has added robustness to current systems as well as redundant systems to increase safety margins. However, these added redundancies and system robustness have added mass to the system.

The technical challenges presented here do not capture all of the risks, technical or programmatic, which the Constellation program faces. As noted earlier, there are over 200 risks categorized as “high” for the Ares I/Orion programs, meaning that if not successfully mitigated, these risks (1) are either nearly certain, highly likely, or may occur, and (2) will have major effects on system cost, schedule, performance, or safety. These risks range in nature from highly complex technical risks, such as those noted above, to straightforward programmatic risks related to areas such as transitioning support work from the Marshall Space Flight Center to Michoud Assembly Facility for long-term vehicle production, compressing the software development cycle for the Orion vehicle, and creating a test program for Orion’s communication and tracking system.

Funding Issues and Cost Increases Continue to Hamper the Program

The Constellation program’s poorly phased funding plan has affected the program’s ability to deal with technical challenges. In our October 2007 report, we noted that NASA initiated the Constellation program recognizing that the agency’s total budget authority would be insufficient to fund all necessary activities in fiscal years 2009 and 2010. NASA’s funding strategy relied on the accumulation of a large rolling budget reserve in fiscal years 2006 and 2007 to fund Constellation activities in fiscal years 2008, 2009, and 2010. Thereafter, NASA anticipated that the retirement of the space shuttle program in 2010 would free funding for the Constellation program. In our October 2007 report, we noted that NASA’s approach to funding was risky and that the approved budget profile at that time was insufficient to meet Constellation’s estimated needs. The Constellation program’s integrated risk management system also identified this strategy as high risk and warned that funding shortfalls could occur in fiscal years 2009 through 2012, resulting in planned work not being completed to support schedules and milestones. According to project officials, these shortfalls limited NASA’s ability to mitigate technical risks early in development and precluded the orderly ramp-up of workforce and developmental activities.

According to the Constellation program manager, these funding shortfalls are reducing his flexibility to resolve technical challenges. The Constellation program tracks unfunded risk mitigation—engineering work identified as potentially needed but not currently funded—as cost threats in IRMA. The Constellation IRMA system currently tracks 192 cost threats for the Ares I and Orion projects totaling about $2.4 billion through fiscal year 2015. Of this $2.4 billion, NASA classifies 35 threats valued at about $730 million as likely to be needed, 54 threats valued at about $670 million as may or may not be needed, and 103 threats valued at about $1 billion as not likely to be needed. Our analysis of the cost threats indicates these cost threats may be understated. For example, of the 157 threats classified as may or may not be needed or not likely to be needed, IRMA likelihood scores indicate that 69 cost threats worth about $789 million are either highly likely or nearly certain to occur. Some examples of cost threats include $4.7 million to develop and mature Orion’s data network technology and $12.5 million for an Upper Stage and First Stage separation test.

The cost of the Constellation program’s developmental contracts have increased as NASA added new effort to resolve technical and design challenges. Constellation program officials and contractor cost reports indicate that the new effort has increased the value of the Constellation program’s developmental contracts from $7.2 billion in 2007 to $10.2 billion in June 2009. Some of these modifications remained undefinitized for extended periods as NASA worked through design issues and matured program requirements in response to technical challenges. Undefinitized contract actions authorize contractors to begin work before reaching a final agreement on contract terms. By allowing undefinitized contract actions to continue for extended periods, NASA loses its ability to monitor contractor performance because the cost reports are not useful for evaluating the contractor’s performance or for projecting the remaining cost of the work under contract. With a current, valid baseline, the reports would indicate when cost or schedule thresholds had been exceeded, and NASA could then require the contractor to explain the reasons for the variances and to identify and take appropriate corrective actions. Yet, NASA allowed high-value modifications to the Constellation contracts to remain undefinitized for extended periods, in one instance, more than 13 months.

Tradeoffs Made to Address Cost Increases and Funding Shortfalls Are Changing Constellation Program Test Strategy

In August 2008, when faced with cost increases and funding shortfalls, the Constellation program responded by reducing program reserves and deferring development effort and test activities. These changes resulted in a minimized flight test program that was so success oriented there was no room for test failures. During the course of our review, NASA test officials expressed multiple concerns about the test approach the program was then pursuing. NASA test officials also expressed concerns about the sufficiency of planned integrated system flight testing. NASA was planning only one integrated system flight test prior to the first crewed flight. Officials stated that while NASA would have been able to address each of the programs’ specific test objectives during the planned flight tests, additional integrated system flight tests could have provided the agency increased confidence that the system performed as planned and allowed the agency the opportunity to design and implement solutions to performance problems without affecting the first crewed flight. According to agency officials, any problems encountered during integrated system flight testing could lead to significant delays in the first crewed flight.

Test officials were also concerned that the highly concurrent test schedule had significant overlap between component qualification and fabrication of flight hardware. This concurrency could have resulted in schedule slips and increased costs if any component failed qualification tests. Our past work indicates that it is unlikely that the program will complete its test program without encountering developmental problems or test failures. The discovery of problems in complex products is a normal part of any development process, and testing is perhaps the most effective tool for discovering such problems. According to the Constellation program manager, the test plan strategy for the Constellation program is currently evolving as the program reshapes its acquisition strategy to defer all work on lunar content beyond the March 2015 first crewed flight. The test strategy is likely to continue to evolve until the Constellation program’s Systems Integration Plan is finalized when the project enters the implementation phase.

Changing Acquisition Strategy

In response to technical challenges and cost and funding issues, NASA is changing the Orion project acquisition strategy. In December 2008, NASA determined that the current Constellation program was high risk and unachievable within the current budget and schedule. To increase its level of confidence in the Constellation program baseline NASA delayed the first crewed flight from September 2014 to March 2015 and according to officials, adopted a two-phased approach to developing the Orion vehicle. NASA’s original strategy for the Orion project was to develop one vehicle capable of supporting both ISS and lunar missions. According to the Constellation program manager, the Constellation program is currently deferring work on Orion lunar content beyond 2015 to focus its efforts on developing a vehicle that can fly the ISS mission. This phased approach, however, could require two qualification programs for the Orion vehicle— one pre-2015 Orion qualification program for ISS mission requirements and a second post-2015 Orion qualification program for lunar mission requirements.

According to the program manager, the knowledge gained from flying the initial Orion to the ISS will inform the design of the lunar vehicle. The Constellation program manager also told us that NASA is unwilling to further trade schedule in order to reduce risk. He asserted that delaying the schedule is an inefficient means of mitigating risk because of the high costs of maintaining fixed assets and contractor staff.

Though these changes to overarching requirements are likely to increase the confidence level associated with the March 2015 first crewed flight, they do not guarantee that the program will conduct a successful first crewed flight in March 2015. For example, in May 2009 the program announced its plan to reduce the number of crew for the ISS mission from six to four. According to project officials, NASA does not plan to finalize the preliminary design of the four-crew ISS configuration until after the Orion preliminary design review. Revising the ISS design for four crew and optimizing the area freed up by removing two crew for the ISS mission will entail additional effort on the part of the Orion design team. Furthermore, as noted above, both the Ares I and Orion projects continue to face technical and design challenges that will require significant time, money, and effort to resolve irrespective of the decision to defer lunar requirements. While deferring the lunar requirement is likely to relieve pressure on Orion’s mass margins allowing increased flexibility to deal with some Orion-specific technical challenges, the lunar requirement has little bearing on many of the Ares I technical challenges discussed above. Furthermore, it is unclear how deferring the lunar requirement will affect the technical challenges faced in the development of the Orion launch abort system and in dealing with vibroacoustics.

Conclusion

NASA’s human spaceflight program is at a crossroads. Efforts to establish a sound business case for Constellation’s Ares I and Orion projects are complicated by (1) an aggressive schedule, (2) significant technical and design challenges, (3) funding issues and cost increases, and (4) an evolving acquisition strategy that continues to change Orion project requirements. Human spaceflight development programs are complex and difficult by nature and NASA’s previous attempts to build new transportation systems have failed in part because they were focused on advancing technologies and designs without resources—primarily time and money—to adequately support those efforts. While the current program, Constellation, was originally structured to rely on heritage systems and thus avoid problems seen in previous programs, the failure to establish a sound business case has placed the program in a poor risk posture to proceed into implementation as planned in 2010. In the past, NASA has recognized these shortfalls and has delayed design reviews for both the Ares I and Orion vehicles in an effort to gain the knowledge needed for a sound business case. NASA’s current approach, however, is based on changing requirements to increase confidence in meeting the schedule. Nevertheless, the need to establish a sound business case, wherein resources match requirements and a knowledge-based acquisition strategy drives development efforts, is paramount to any successful program outcome. Until the Constellation program has a sound business case in hand, it remains doubtful that NASA will be able to reliably estimate cost and schedule to complete the program.

Meanwhile, the new Administration is conducting an independent review of NASA’s human spaceflight activities, with the potential for recommendations of broad changes to the agency’s approach toward future efforts. While the fact that the review is taking place does not guarantee wholesale changes to the current approach, it does implicitly recognize the challenges facing the Constellation program. We believe this review is appropriate as it presents an opportunity to reassess both requirements and resources for Constellation as well as alternative ways for meeting requirements.

Recommendations

Regardless of NASA’s final plans for moving forward, the agency faces daunting challenges developing human rated spacecraft for use after the Space Shuttle is retired, and it is important that the agency lay out an acquisition strategy grounded in knowledge-based principles that is executable with acceptable levels of risk within the program’s available budget. As NASA addresses the findings and recommendations of the Review of U.S. Human Space Flight Plans Committee, we recommend that the new NASA Administrator direct the Constellation program, or its successor, to develop a sound business case—supported by firm requirements, mature technologies, a preliminary design, a realistic cost estimate, and sufficient funding and time—before proceeding into implementation, and, if necessary, delay the preliminary design review until a sound business case demonstrating the program’s readiness to move forward into implementation is in hand.

Agency Comments and Our Evaluation

In written comments on a draft of this report (see app. II), NASA concurred with our recommendation. NASA acknowledged that, while substantial work has been completed, the Constellation program faces knowledge gaps concerning requirements, technologies, funding, schedule, and other resources. NASA stated that it is working to close these gaps before committing to significant, long-term investments in the Constellation program. NASA stated that the Constellation program manager is required to demonstrate at the preliminary design review that the program and its projects meet all system requirements with acceptable risk and within cost and schedule constraints, and that the program has established a sound business case for proceeding into the implementation phase. At this point, the NASA Agency Program Management Council will review the Constellation program and determine the program’s readiness to proceed into the implementation phase and begin detailed design. Separately, NASA provided technical comments, which have been addressed in the report, as appropriate.

As agreed with your offices, unless you announce its contents earlier, we will not distribute this report further until 30 days from its date. At that time, we will send copies to NASA’s Administrator and interested congressional committees. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov.

Should you or your staff have any questions on matters discussed in this report, please contact me at (202) 512-4841 or at ChaplainC@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff that made key contributions to this report are listed in appendix III.

Appendix I: Scope and Methodology

To assess NASA’s progress toward establishing a sound business case for the Ares I and Orion projects and identify key technical challenges NASA faces in developing the Ares I Crew Launch and the Orion Crew Exploration Vehicles, we obtained and analyzed Constellation plans and schedules, risk mitigation information, and contract performance data relative to the standards in our knowledge-based acquisition practices including program and project plans, contracts, schedules, risk assessments, funding profile, budget documentation, earned value reports, and the results of NASA’s assessments of the program. We interviewed and received briefings from officials associated with the Constellation program office, including Exploration Systems Mission Directorate officials at NASA headquarters in Washington, D.C.; Orion project and Constellation program officials at the Johnson Space Center in Houston, Texas; and, Ares I and J-2X officials at the Marshall Space Flight Center in Huntsville, Alabama, regarding the program and projects’ risk areas and test strategy, technical challenges, the status of requirements, acquisition strategy and the status of awarded contracts.

We also conducted interviews and received briefings from NASA contractors on heritage hardware and design changes, and top risks and testing strategy, for the J-2X engine, Ares I First Stage, Ares I Upper Stage, Launch Abort System, and Orion vehicle. We analyzed risk documented through the Constellation program’s Integrated Risk Management Application and followed up with project officials for clarification and updates to these risks. We also attended the Constellation Program’s Quarterly Risk Review at the Johnson Space Center.

In addition, we interviewed Constellation program officials from Johnson Space Center about program risks, requirements, and the impact of budget reductions. We also spoke with NASA headquarters officials from the Exploration Systems Mission Directorate’s Resources Management Office in Washington, D.C., to gain insight into the program’s top risks and the basis for fiscal year 2006 through fiscal year 2010 budget requests as well as the funding strategy employed by the Constellation program. Furthermore, we reviewed NASA’s program and project management directives and systems engineering directives. Our review and analysis of these documents focused on requirements and goals set for spaceflight systems. We compared examples of the centers’ implementation of the directives and specific criteria included in these directives with our best practices work on system acquisition.

We conducted this performance audit from December 2008 through August 2009 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives.

Appendix II: Comments from the National Aeronautics and Space Administration

Appendix III: GAO Contacts and Staff Acknowledgments

GAO Contacts

Acknowledgments

In addition to the contact named above, Jim Morrison, Assistant Director; Jessica M. Berkholtz; Greg Campbell; Jennifer K. Echard; Nathaniel J. Taylor; John S. Warren Jr.; and Alyssa Weir made key contributions to this report.

Related GAO Products

NASA: Assessments of Selected Large-Scale Projects. GAO-09-306SP. Washington, D.C.: March 2, 2009.

NASA: Agency Faces Challenges Defining Scope and Costs of Space Shuttle Transition and Retirement. GAO-08-1096. Washington, D.C.: September 30, 2008.

NASA: Ares I and Orion Project Risks and Key Indicators to Measure Progress. GAO-08-186T. Washington, D.C.: April 3, 2008.

NASA: Agency Has Taken Steps Toward Making Sound Investment Decisions for Ares I but Still Faces Challenging Knowledge Gaps. GAO-08-51. Washington, D.C.: October 31, 2007.

NASA: Issues Surrounding the Transition from the Space Shuttle to the Next Generation of Human Space Flight Systems. GAO-07-595T. Washington, D.C.: March 28, 2007.

NASA: Long-Term Commitment to and Investment in Space Exploration Program Requires More Knowledge. GAO-06-817R. Washington, D.C.: July 17, 2006.

NASA: Implementing a Knowledge-Based Acquisition Framework Could Lead to Better Investment Decisions and Project Outcomes. GAO-06-218. Washington, D.C.: December 21, 2005.

Defense Space Activities: Continuation of Evolved Expendable Launch Vehicle Program’s Progress to Date Subject to Some Uncertainty. GAO-04-778R. Washington, D.C.: June 24, 2004.

Best Practices: Using a Knowledge-Based Approach to Improve Weapon Acquisition. GAO-04-386SP. Washington, D.C.: January 2004.