NASA: Sound Management and Oversight Key to Addressing Crew Exploration Vehicle Project Risks
Published: 2006-09-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The National Aeronautics and Space Administration (NASA) plans to spend nearly $230 billion over the next two decades implementing the President's Vision for Space Exploration (Vision) plans. In July 2006, GAO issued a report that questioned the program's affordability, and particularly, NASA's acquisition approach for one of the program's major projects--the Crew Exploration Vehicle (CEV). This testimony, which is based upon that report and another recent GAO report evaluating NASA's acquisition policies, highlights GAO's continuing concerns with (1) the affordability of the exploration program; (2) the acquisition approach for the CEV, and; (3) NASA's acquisition policies that lack requirements for projects to proceed with adequate knowledge.
section_title: What GAO Found
paragraphs: NASA's proposals for implementing the space exploration Vision raise a number of concerns. NASA cannot develop a firm cost estimate for the exploration program at this time because the program is in its early stages. The changes that have occurred to the program over the past year and the resulting refinement of its cost estimates are indicative of the evolving nature of the program. While changes are appropriate at this stage of the program, they leave the agency unable to firmly identify program requirements and needed resources and, therefore, not in the position to make a long term commitment to the program. NASA will likely be challenged to implement the program, as laid out in its Exploration Systems Architecture study (ESAS), due to the high costs associated with the program in some years and its long-term sustainability relative to anticipated funding. As we reported in July 2006, there are years when NASA, with some yearly shortfalls exceeding $1 billion, does not have sufficient funding to implement the architecture; while in other years the funding available exceeds needed resources. Despite initial surpluses, the long-term sustainability of the program is questionable, given its long-term funding outlook. NASA's preliminary projections show multibillion-dollar shortfalls for its exploration directorate in all fiscal years from 2014 to 2020, with an overall deficit through 2025 in excess of $18 billion. NASA's acquisition strategy for the CEV was not based upon obtaining an adequate level of knowledge when making key resources decisions, placing the program at risk for cost overruns, schedule delays, and performance shortfalls. These risks were evident in NASA's plan to commit to a long-term product development effort before establishing a sound business case for the project that includes well-defined requirements, mature technology, a preliminary design, and firm cost estimates. NASA adjusted its acquisition approach and the agency included the production and sustainment portions of the contract as options--a move that is consistent with the recommendation in our report because it lessens the government's financial obligation at this early stage. However, risks persist with NASA's approach. As we reported in 2005, NASA's acquisition policies lacked major decision reviews beyond the initial project approval gate and lacked a standard set of criteria with which to measure projects at crucial phases in the development life cycle. These decision reviews and development measures are key markers needed to ensure that projects are proceeding with and decisions are being based upon the appropriate level of knowledge and can help to lessen identified project risks. The CEV project would benefit from the application of such markers.

section_title: Letter
paragraphs: I am pleased to be here today to discuss the National Aeronautics and  Space Administration’s (NASA) plans for implementing the President’s  Vision for Space Exploration (Vision). NASA plans to spend nearly $230  billion over the next two decades—more than $31 billion of which will be  spent in the next 5 years—to bring the Vision to reality. In July 2006, we  issued a report that questioned the program’s affordability, and in  particular, NASA’s acquisition approach for one of the program’s major  projects—the Crew Exploration Vehicle (CEV). My statement today,  which is based upon that report and another report evaluating NASA’s  acquisition policies, highlights our continuing concerns with the  affordability of the exploration program and the acquisition approach for  the CEV project, as well as the absence of firm requirements in NASA’s  acquisition policies for projects to proceed with development with the  appropriate level of knowledge. Given the competing demands facing the  federal government and an already troubling funding profile for the  program, it is imperative that NASA implement the various aspects of the  Vision in a fiscally prudent and competent manner. Our work was  performed in accordance with generally accepted government auditing  standards.
subsections: section_title: Background
paragraphs: Despite many successes in the exploration of space, such as landing the  Pathfinder and Exploration Rovers on Mars, NASA has had difficulty  bringing a number of projects to completion, including several efforts to  build a second generation reusable human spaceflight vehicle to replace  the space shuttle. NASA has attempted several costly endeavors, such as  the National Aero-Space Plane, the X-33 and X-34, and the Space Launch  Initiative. While these endeavors have helped to advance scientific and  technical knowledge, none have completed their objective of fielding a  new reusable space vehicle. We estimate that these unsuccessful  development efforts have cost approximately $4.8 billion since the 1980s.  The high cost of these unsuccessful efforts and the potential costs of  implementing the Vision make it important that NASA achieve success in  its new exploration program beginning with the CEV project.
Our past work has shown that developing a sound business case, based on  matching requirements to available and reasonably expected resources  before committing to a new product development effort, reduces risk and  increases the likelihood of success. High levels of knowledge should be  demonstrated before managers make significant program commitments,  specifically: (1) At program start, the customer’s needs should match the  developer’s available resources in terms of availability of mature  technologies, time, human capital, and funding; (2) Midway through  development, the product’s design should be stable and demonstrate that  it is capable of meeting performance requirements; (3) By the time of the  production decision, the product must be shown to be producible within  cost, schedule, and quality targets, and have demonstrated its reliability.  Our work has shown that programs that have not attained the level of  knowledge needed to support a sound business case have been plagued by  cost overruns, schedule delays, decreased capability, and overall poor  performance. With regard to NASA, we have reported that in some cases  the agency’s failure to define requirements adequately and develop  realistic cost estimates—two key elements of a business case—resulted in  projects costing more, taking longer, and achieving less than originally  planned.
section_title: Firm Cost Estimates Cannot Be Developed at This Time
paragraphs: Although NASA is continuing to refine its exploration architecture cost  estimates, the agency cannot at this time provide a firm estimate of what it  will take to implement the architecture. The absence of firm cost estimates  is mainly due to the fact that the program is in the early stages of its life  cycle. NASA conducted a cost risk analysis of its preliminary estimates  through fiscal year 2011. On the basis of this analysis and through the  addition of programmatic reserves (20 percent on all development and 10  percent on all production costs), NASA is 65 percent confident that the  actual cost of the program will either meet or be less than its estimate of  $31.2 billion through fiscal year 2011. For cost estimates beyond 2011,  when most of the cost risk for implementing the architecture will be  realized, NASA has not applied a confidence level distinction. Since NASA  released its preliminary estimates, the agency has continued to make  architecture changes and refine its estimates in an effort to establish a  program that will be sustainable within projected resources. While  changes to the program are appropriate at this stage when concepts are  still being developed, they leave the agency in the position of being unable  to firmly identify program requirements and needed resources. NASA  plans to commit to a firm cost estimate for the Constellation program at  the preliminary design review in 2008, when requirements, design, and  schedule will all be baselined. It is at this point where we advocate  program commitments should be made on the basis of the knowledge  secured.
section_title: Expected Budget and Competing Demands Will Challenge Architecture Implementation
paragraphs: NASA will be challenged to implement the ESAS recommended  architecture within its projected budget, particularly in the longer-term. As  we reported in July 2006, there are years when NASA has projected  insufficient funding to implement the architecture with some yearly  shortfalls exceeding $1 billion; while in other years the funding available  exceeds needed resources. Per NASA’s approach, it plans to use almost $1  billion in appropriated funds from fiscal years 2006 and 2007 in order to  address the short-term funding shortfalls. NASA, using a “go as you can  afford to pay” approach, maintains that in the short-term the architecture  could be implemented within the projected available budgets through  fiscal year 2011 when funding is considered cumulatively. However,  despite initial surpluses, the long-term sustainability of the program is  questionable given the long-term funding outlook for the program. NASA’s  preliminary projections show multibillion-dollar shortfalls for its  Exploration Systems Mission Directorate in all fiscal years from 2014 to  2020, with an overall deficit through 2025 in excess of $18 billion.  According to NASA officials, the agency will have to keep the program  compelling for both Congress and potential international partners, in  terms of the activities that will be conducted as part of the lunar program,  in order for the program to be sustainable over the long run.
NASA is attempting to address funding shortfalls within the Constellation  program by redirecting funds to that program from other Exploration  Systems Mission Directorate activities to provide a significant surplus in  fiscal years 2006 and 2007 to cover projected shortfalls beginning in fiscal  year 2009. Several Research and Technology programs and missions were  discontinued, descoped, or deferred and that funding was shifted to the  Constellation Program to accelerate development of the CEV and CLV. In  addition, the Constellation program has requested more funds than  required for its projects in several early years to cover shortfalls in later  years. NASA officials stated the identified budget phasing problem could  worsen given the changes that were made to the exploration architecture  following issuance of the study. For example, while life cycle costs may be  lower in the long run, acceleration of development for the five segment  Reusable Solid Rocket Booster and J-2x engine will likely add to the near- term development costs, where the funding is already constrained. NASA  has yet to provide cost estimates associated with program changes.
NASA must also contend with competing budgetary demands within the  agency as implementation of the exploration program continues. NASA’s  estimates beyond 2010 are based upon a surplus of well over $1 billion in  fiscal year 2011 due to the retirement of the space shuttle fleet in 2010.  However, NASA officials said the costs for retiring the space shuttle and  transitioning to the new program are not fully understood; thus, the  expected surplus could be less than anticipated. This year, NASA plans to  spend over 39 percent of its annual budget for space shuttle and  International Space Station (ISS) operations—dollars that will continue to  be obligated each year as NASA completes construction of the ISS by the  end of fiscal year 2010. This does not include the resources necessary to  develop ISS crew rotation or logistics servicing support capabilities for the  ISS during the period between when the space shuttle program retires and  the CEV makes its first mission to the ISS. While, generally, the budget for  the space shuttle is scheduled to decrease as the program moves closer to  retirement, a question mark remains concerning the dollars required to  retire the space shuttle fleet as well as transition portions of the  infrastructure and workforce to support implementation of the  exploration architecture. In addition, there is support within Congress and  the scientific community to restore money to the Science Mission  Directorate that was transferred to the space shuttle program to ensure its  viability through its planned retirement in 2010. Such a change could have  an impact on future exploration funding.
section_title: Lack of Sound Business Case Puts CEV Acquisition at Risk
paragraphs: In July 2006, we reported that NASA’s acquisition strategy for the CEV  placed the project at risk of significant cost overruns, schedule delays, and  performance shortfalls because it committed the government to a long- term contract before establishing a sound business case. We found that the  CEV contract, as structured, committed the government to pay for design,  development, production and sustainment upon contract award—with a  period of performance through at least 2014 with the possibility of  extending through 2019.
Our report highlighted that NASA had yet to develop key elements of a  sound business case, including well-defined requirements, mature  technology, a preliminary design, and firm cost estimates that would  support such a long-term commitment. Without such knowledge, NASA  cannot predict with any confidence how much the program will cost, what  technologies will or will not be available to meet performance  expectations, and when the vehicle will be ready for use. NASA has  acknowledged that it will not have these elements in place until the  project’s preliminary design review scheduled for fiscal year 2008. As a  result, we recommended that the NASA Administrator modify the current  CEV acquisition strategy to ensure that the agency does not commit itself,  and in turn the federal government, to a long-term contractual obligation  prior to establishing a sound business case at the project’s preliminary  design review. In response to our recommendation, NASA disagreed and  stated that it had the appropriate level of knowledge to proceed with its  current acquisition strategy. NASA also indicated that knowledge from the  contractor is required in order to develop a validated set of requirements  and, therefore, it was important to get the contractor on to the project as  soon as possible. In addition, according to NASA officials, selection of a  contractor for the CEV would enable the agency to work with the  contractor to attain knowledge about the project’s required resources and,  therefore, be better able to produce firm estimates of project cost. In our  report, we highlighted that this is the type of information that should be  obtained prior to committing to a long-term contract. To our knowledge,  NASA did not explore the possibility of utilizing the contractor, through a  shorter-term contract, to conduct work needed to develop valid  requirements and establish higher-fidelity cost estimates—a far less risky  and costly strategy.
Subsequent to our report, NASA did, however, take steps to address some  of the concerns we raised. Specifically, NASA modified its acquisition  strategy for the CEV and made the production and sustainment schedules  of the contract—known as Schedules B and C—contract options that the  agency will decide whether to exercise after project’s critical design  review in 2009. Therefore, NASA will only be liable for the minimum  quantities under Schedules B and C when and if it chooses to exercise  those options. These changes to the acquisition strategy lessen the  government’s financial obligation at this early stage. Table 1 outlines the  information related to the CEV acquisition strategy found in the request  for proposal and changes that were made to that strategy prior to contract  award. While we view these changes as in line with our recommendation  and as a positive step to address some of the risks we raised in our report,  NASA still has no assurance that the project will have the elements of a  sound business case in place at the preliminary design review. Therefore,  NASA’s commitment to efforts beyond the project’s preliminary design  review—even when this commitment is limited to design, development,  test and evaluation activities (DDT&E)—is a risky approach. It is at this  point that NASA should (a) have the increased knowledge necessary to  develop a sound business case that includes high-fidelity, engineering- based estimates of life cycle cost for the CEV project, (b) be in a better  position to commit the government to a long-term effort, and (c) have  more certainty in advising Congress on required resources.
section_title: Sound Management and Oversight Key to Addressing CEV Project Risks
paragraphs: Sound project management and oversight will be key to addressing risks  that remain for the CEV project as it proceeds with its acquisition  approach. To help mitigate these risks, NASA should have in place the  markers necessary to help decision makers monitor the CEV project and  ensure that is following a knowledge based approach to its development.  However, in our 2005 report that assessed NASA’s acquisition policies, we  found that NASA’s policies lacked major decision reviews beyond the  initial project approval gate and a standard set of criteria with which to  measure projects at crucial phases in the development life cycle—key  markers for monitoring such progress. In our review of the individual  center policies, we found that the Johnson Space Center project  management policy, which is the policy that the CEV project will be  required to follow, also lacked such key criteria. We concluded that  without such requirements in place, decision makers have little knowledge  about the progress of the agency’s projects and, therefore, cannot be  assured that they are making informed decisions about whether continued  investment in a program or project is warranted.
We recommended that NASA incorporate requirements in its new systems  engineering policy to capture specific product knowledge at key junctures  in project development. The demonstration of such knowledge could then  be used as exit criteria for decision making at the following key junctures:    Before projects are approved to transition in to implementation, we  suggested that projects be required to demonstrate that key  technologies have reached a high maturity level.
Before projects are approved to transition from final design to  fabrication, assembly, and test, we suggested that projects be required  to demonstrate that the design is stable.
Before projects are approved to transition to production, we suggested  that projects be required to demonstrate that the design can be  manufactured within cost, schedule, and quality targets.
In addition, we recommended that NASA institute additional major  decision reviews that are tied to these key junctures to allow decision  makers to reassess the project based upon demonstrated knowledge.
While NASA concurred with our recommendations, the agency has yet to  take significant actions to implement them. With regard to our first  recommendation, NASA stated that the agency would establish  requirements for success at the key junctures mentioned above. NASA  planned to include these requirements in the systems engineering policy it  issued in March 2006. Unfortunately, NASA did not include these criteria  as requirements in the new policy, but included them in an appendix to the  policy as recommended best practices criteria. In response to our second  recommendation, NASA stated it would revise its program and project  management policy for flight systems and ground support projects, due to  be completed in fall 2006. In the revised policy, NASA indicated that it  would require the results of the critical design review and, for projects that  enter a large-scale production phase, the results of the production  readiness review to be reported to the appropriate decision authority in a  timely manner so that a decision about whether to proceed with the  project can be made. NASA has yet to issue its revised policy; therefore, it  remains to be seen as to whether the CEV project decision authorities will  have the opportunity to reassess and make decisions about the project  using the markers recommended above after the project has initially been  approved. Briefings that we have recently received indicate that NASA  plans to implement our recommendation in the revised policy.
The risks that NASA has accepted by moving ahead with awarding the  contract for DDT&E for CEV could be mitigated by implementing our  recommendations as it earlier agreed. Doing so would provide both NASA  and Congress with markers of the project’s progress at key points. For  example, at the preliminary design review, decision makers would be able  to assess the status of the project by using the marker of technology  maturity. In addition, at the critical design review, the agency could assess  the status of the project using design stability (i.e., a high percentage of  engineering drawings completed). If NASA has not demonstrated  technology maturity at the preliminary design review or design stability at  the critical design review, decision makers would have an indication that  the project will likely be headed for trouble. Without such knowledge,  NASA cannot be confident that its decisions about continued investments  in projects are based upon the appropriate knowledge. Furthermore,  NASA’s oversight committees could also use the information when  debating the agency’s yearly budget and authorizing funds not only for the  CEV project, but also for making choices among NASA’s many competing  programs. If provided this type of information from NASA about its key  projects, Congress will be in a better position to make informed decisions  about how to invest the nation’s limited discretionary funds.
NASA’s ability to address a number of long-standing financial management  challenges could also impact management of NASA’s key projects. The  lack of reliable, day-to-day information continues to threaten NASA’s  ability to manage its programs, oversee its contractors, and effectively  allocate its budget across numerous projects and programs. To its credit,  NASA has recognized the need to enhance the capabilities and improve the  functioning of its core financial management system, however, progress  has been slow. NASA contract management has been on GAO’s high-risk  list since 1990 because of such concerns.
section_title: Conclusions
paragraphs: In conclusion, implementing the Vision over the coming decades will  require hundreds of billions of dollars and a sustained commitment from  multiple administrations and Congresses. The realistic identification of the  resources needed to achieve the agency’s short-term goals would provide  support for such a sustained commitment over the long term. With a range  of federal commitments binding the fiscal future of the United States,  competition for resources within the federal government will only increase  over the next several decades. Consequently, it is incumbent upon NASA  to ensure that it is wisely investing its existing resources. As NASA  proceeds with its acquisition strategy for the CEV project and other key  projects, it will be essential that the agency ensure that the investment  decisions it is making are sound and based upon high levels of knowledge.  NASA should require that the progress of its projects are evaluated and  reevaluated using knowledge based criteria, thereby improving the quality  of decisions that will be made about which program warrant further  investment. Furthermore, it will be critical that NASA’s financial  management organization delivers the kind of analysis and forward- looking information needed to effectively manage its programs and  projects. Clear, strong executive leadership will be needed to ensure that  these actions are carried out. Given the nation’s fiscal challenges and  those that exist within NASA, the availability of significant additional  resources is unlikely. NASA has the opportunity to establish a firm  foundation for its entire exploration program by ensuring that the level of  knowledge necessary to allow decision makers to make informed  decisions about where continued investment is justified. Doing so will  enhance confidence in the agency’s ability to finally deliver a replacement  vehicle for future human space flight.
Mr. Chairman, this concludes my prepared statement. I would be pleased  to respond to any questions that you or other Members of the Committee  may have.
section_title: GAO Contact and Staff Acknowledgements
paragraphs: For further information regarding this testimony, please contact Allen Li at  (202) 512-4841 or lia@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this testimony. GAO staff who made key contributions to this testimony  include Greg Campbell, Richard Cederholm, Hillary Loeffler,   James L. Morrison, Jeffrey M. Niblack, and Shelby S. Oakley.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. It may be reproduced and distributed in its entirety without further  permission from GAO. However, because this work may contain copyrighted images or  other material, permission from the copyright holder may be necessary if you wish to  reproduce this material separately.