Space Shuttle: Further Improvements Needed in NASA's Modernization Efforts
Published: 2004-01-15T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: The Columbia tragedy has accentuated the need to modernize the 20-year-old space shuttle, the only U.S. launch system that carries people to and from space. The shuttle will now be needed for another two decades. As it ages, the spacecraft's components will also age, and it may become increasingly unreliable. GAO examined the National Aeronautics and Space Administration's (NASA) plans to upgrade the shuttle through 2020, how it will identify and select what upgrades are needed, how much the upgrades may cost, and what factors will influence that cost over the system's lifetime.
section_title: What GAO Found
paragraphs: NASA cannot fully define shuttle upgrade requirements until it resolves questions over the shuttle's operational life and determines requirements for elements of its Integrated Space Transportation Plan (ISTP) such as the International Space Station (ISS). Prior efforts to upgrade the shuttle have been stymied because NASA could not develop a strategic investment plan or systematically define the spacecraft's requirements because of changes in its life expectancy and mission. NASA is trying to improve how it identifies, selects, and prioritizes shuttle upgrades. In March 2003, it institutionalized a Space Shuttle Service Life Extension Program (SLEP) to ensure safe and effective operations, along with a management plan documenting roles and responsibilities and an annual process for selecting upgraded projects and studies. In addition, NASA will try to improve shuttle safety by implementing the recommendations of the Columbia Accident Investigation Board (CAIB). NASA's estimate of the total cost to upgrade the shuttle--$300 million-$500 million a year, or a total of $5 billion-$8 billion through 2020--is reasonably based but could be significantly higher, as it does not include potential projects such as a crew escape system. It will be difficult for NASA to make an accurate estimate until it firmly establishes the basic requirements (such as life expectancy) for the shuttle and the process for selecting shuttle upgrades. A number of potential changes could significantly increase the cost of shuttle upgrades, including responses to the recommendations of the CAIB.

section_title: Letter
paragraphs: The space shuttle is the only U.S. launch system capable of carrying  people to and from space. It has operated for over 20 years and is planned  for use well into the second decade of this century and possibly beyond.  As the shuttle ages, the National Aeronautics and Space Administration  (NASA) is faced with an increased need to modernize the shuttle due to  component obsolescence and/or to enhance safety. The Shuttle Columbia  tragedy has accentuated this need. At your request, we reviewed the  shuttle modernization efforts to determine (1) NASA’s past requirements  and plans to upgrade the shuttle through 2020, (2) how NASA will identify  what upgrades are required in the future and how those upgrades will be  selected and prioritized, and (3) NASA’s estimated life-cycle cost for  shuttle upgrades through 2020 and identify the potential program  uncertainties that may affect cost.
subsections: section_title: Background
paragraphs: The space shuttle is the world’s first reusable space transportation system.  It consists of a reusable orbiter with three main engines, two partially  reusable solid rocket boosters, and an expendable external fuel tank. The  space shuttle is an essential element of NASA’s transportation plan that  includes a framework for maintaining shuttle fleet capability to fly safely  through 2020. The space shuttle is NASA’s largest individual program  accounting for about 25 percent of the agency’s fiscal year 2004 budget  request. Since it is the nation’s only launch system capable of transporting  people, the shuttle’s viability is critical to the space station.
We have reported in the past that extensive delays in the development and  assembly of the ISS and difficulties defining requirements and maturing  technologies for the next generation space transportation systems have  hindered the development and funding of a long-term space transportation  program. We have also testified that NASA faced a number of  programmatic and technical challenges in making shuttle upgrades,  including revitalizing its workforce and defining shuttle technical  requirements. In another report, we reported that NASA continued to rely  on qualitative risk assessments to supplement engineering judgments and  had made only limited progress in the use of quantitative assessment  methods. Recognizing such needs, NASA has taken steps to bring a more  formal approach to identifying, prioritizing, and funding improvements.
In February 1997, NASA established the Space Shuttle Program  Development Office at NASA’s Johnson Space Center to sustain, improve,  and add capability to the space shuttle through an upgrade program. In  December 2002, a new selection and prioritization process for upgrades  was implemented through the Service Life Extension Program. The SLEP  provided a formal process to select, prioritize, and fund upgrades needed  to keep the shuttle flying safely and efficiently and allow upgrades to be  evaluated and approved on a priority basis. Shuttle upgrades are items that  contribute toward the Space Shuttle Program goals to (1) fly safely,  (2) meet the manifest, (3) improve mission supportability, and (4) improve  the system in order to meet NASA’s commitments and goals for human  operations in space. According to NASA, upgrades achieve major  reductions in the operational risks inherent in the current systems by  making changes that eliminate, reduce, or mitigate significant hazards and  critical failure modes and that increase the overall reliability of the current  system with respect to the likelihood of catastrophic failure. Examples of  upgrade projects currently funded to improve safety include Cockpit  Avionics, Vehicle Main Landing Gear Tire and Wheel, External Tank  Friction Stir Weld, and Shuttle Main Engine Advanced Health Management  System.
section_title: Shuttle Requirements Process Lacks Systematic Approach
paragraphs: To keep the shuttle flying safely, NASA needs to fully implement an  upgrade program to modernize various shuttle components. However,  efforts to do so have been stymied by the agency’s inability to develop a  long-term strategic investment plan and a systematic approach for defining  shuttle requirements, because the spacecraft’s life expectancy and mission  have continued to change. Key decisions about the ultimate life and  mission of the basic elements of the integrated transportation plan—the  ISS and the Orbital Space Plane (OSP)—were not made prior to fully  defining shuttle requirements.
Originally, the shuttle was designed for a 10-year/100-flight service— transporting satellites and other cargo for the Department of Defense and  others and placing in orbit and maintaining the Hubble Space Telescope— after which its life was to end. During this time, NASA was reluctant to  make long-term investments due to the shuttle’s perceived short life  expectancy. With the advent of the ISS, the agency’s transportation plan  indicated that the shuttle would be used to operate and support the ISS  until 2012, when a new space launch vehicle was to take over that mission.  Recently, use of the new launch vehicle was de-emphasized by a new  ISTP, which in its place proposed development of an OSP (to transfer the  crew to the ISS) and continued use of the shuttle (to transfer cargo). The  new plan proposes upgrading the shuttle’s software and hardware to  extend its operational life to 2020.
NASA recognizes the need for a systematic approach for defining  requirements to upgrade the shuttle, and it recently institutionalized a new  process to select and prioritize shuttle upgrades. However, NASA has not  yet fully defined the basic elements of the ISTP—which include the ISS,  the OSP, and the Next Generation Launch Technology. NASA has not  precisely determined when the ISS will be completed; its ultimate mission,  its useful life, and even how many astronauts will be on board, for  example. Specifically, NASA has not made explicit decisions on shuttle  requirements-–such as its future mission, lift capability, and life  expectancy. According to NASA officials, these decisions will significantly  affect shuttle upgrades. Similarly, the CAIB found that the shifting date for  shuttle replacement has severely complicated decisions on how to invest  in shuttle upgrades.
section_title: NASA’s Process for Selecting and Prioritizing Upgrades Could Be Further Improved
paragraphs: NASA is making an effort to improve how it identifies, selects, and  prioritizes shuttle upgrades. In December 2002, NASA initiated a SLEP as  the primary framework for ensuring safe and effective operations. By  March 2003, NASA had prepared a formal management plan documenting  roles and responsibilities and defining an annual process for selecting and  prioritizing upgraded projects and studies. Prior to the SLEP, NASA had no  documented systematic selection process, and managers made decisions  on upgrades using their professional insight and judgment and a limited  number of quantitative or analytic tools rather than extensive use of hard  data or rigorous analysis. As a result, projects that were identified, funded,  and implemented flowed from an informal “bottom-up” approach that  relied largely on insight and judgment of selected managers and limited  use of quantitative tools.
subsections: section_title: Earlier Process to Identify and Prioritize Upgrades
paragraphs: According to NASA officials, prior to the new SLEP process, the  identification, selection, and prioritization of shuttle upgrade projects  largely involved an informal bottom-up approach. The upgrades were first  proposed in an open and a continuous call for projects concepts and were  drawn from shuttle element project organization, industry, or other shuttle  program stakeholders. Upgrade projects would then go to the Space  Shuttle Program Manager, the Shuttle Program Development Manager, and  the directors of the affected NASA field centers, who would provide  proposed projects to the Associate Administrator for Space Flight, who  would select and prioritize the projects. This early process was much more  strongly driven by collective management insight or “judgment” rather  than by hard data or rigorous analysis. During this process, there was little  guidance from top management as to how the decisions on shuttle  upgrades integrated with all the other elements of the ISTP.
The identification, selection, and prioritization of the Cockpit Avionics  Upgrade (CAU) is one example of a lack of a documented, structured, and  systematic selection process prior to the SLEP. The CAU is estimated to  cost $442 million and is NASA’s most costly of the currently approved  upgrade projects. The CAU will update the cockpit’s dials and gauges with  a modern instrument panel. By automating complex procedures in the  shuttle cockpit, the upgrade is intended to improve the situational  awareness of the crew and to better equip them to handle potential flight  problems by reducing crew workload. (See fig. 1.)
Managers gave the CAU project the highest priority based on their  professional insight and judgment and a limited number of quantitative or  analytic tools rather than extensive use of hard data or rigorous analysis.  The upgrade was ranked as the highest priority based on the perceived  importance of crew situational awareness. NASA did not have a metric to  show the relationship of the cost of the upgrade to an increase in shuttle  life and/or safety. The ranking was essentially a collaborative voting  process based on their professional knowledge that crew error accounts  for 50 percent of all incidents. As crew awareness depends on a number of  human factors, a quantitative metric, such as NASA’s Quantitative Risk  Assessment System, could not be used since it did not contain key human  attributes needed to evaluate the percentage of safety improvement of the  upgrade project.
section_title: The SLEP Process Currently in Place
paragraphs: In December 2002, NASA initiated a SLEP as the primary framework for  ensuring safe and effective operations, along with a management plan a  few months later, documenting roles and responsibilities and an annual  process for selecting and prioritizing upgraded projects and studies. The  new process, which was first used in March 2003 at the first SLEP Summit,  uses panels of experts from NASA, which are mostly chaired by the  Deputy Center Directors, who meet periodically to develop and assess  project recommendations. The SLEP is structured around eight panels of  senior managers that make greater use of quantitative tools in areas such  as safety and sustainability, including an outside panel of industry experts  and an Integration Panel. The Integration Panel refines the prioritized  recommendations of each panel into final recommendations to a group of  top-level managers known as the Space Flight Leadership Council (the  Council). As a result of the last Summit in March 2003, the Council  approved all project recommendations of the Integration Panel with a  total estimated cost of about $1.7 billion from fiscal years 2004-08.  (See app. II.) In making its recommendations, the Council was not  restricted by fiscal constraints. The Council endorsed 60 SLEP upgrade  projects for fiscal year 2004 costing $416 million. By contrast, NASA’s  fiscal year 2004 budget request, submitted in February 2003, asked for  $379 million. The difference is being deliberated within NASA’s internal  budget process.
One product resulting from the SLEP 2003 Summit was NASA’s selection  and identification of upgrade projects related to safety improvement,  sustainability, and requirements for new capabilities as defined by  “customers” such as the ISS. NASA then placed the projects into one of the  following four categories: (1) “Should Start”—projects strongly  recommended for start in fiscal year 2004 and which would create near  term risk if they did not start, (2) “Existing Commitments”—projects  previously authorized, (3) “Foundational Activities”—projects that add  insight into the current condition of assets, and (4) “Projects and  Studies”—system specific activities at various levels of maturity.  (See table 1.)
NASA also considers development of the infrastructure to sustain shuttle  operations through 2020 equally as important as upgrades to keep the  shuttle flying safely. One example of a sustainability project for fiscal year  2004 is the replacement of the roof of the 39-year-old Vehicle Assembly  Building at Kennedy Space Center, which is in poor condition, as shown  by the bubbles that have developed in its surface. (See fig. 2.) The roof  replacement is estimated to cost $16 million and is part of NASA’s total  spending on infrastructure of $54 million in fiscal year 2004.
section_title: Further Improvements in the SLEP Possible
paragraphs: NASA needs to improve its analytic tools to help it improve the basis for  identifying and selecting shuttle upgrades. NASA uses Probabilistic Risk  Assessment (PRA) methodologies, specifically the Quantitative Risk  Assessment System, to improve safety by assessing the relative risk  reduction of potential upgrade projects to overall shuttle risk. However,  program managers are aware that the PRA is incomplete and does not  contain certain key attributes that would make it more accurate, reliable,  and useful. Early next year, they plan to begin using a revised PRA more  oriented toward the shuttle. In addition, the Manager of the Shuttle  Program Development Office believes it is important to develop a new  Sustainability Health Metric System in order to mitigate the risk that an  asset required to fly may not be available. The metric would score a  proposed sustainability project after an evaluation of a set of common  sustainability factors for all elements of shuttle flight and ground systems  and subsystems. Similarly, the CAIB could not find adequate application of  a metric that took an integrated systematic view of the entire space shuttle  system. NASA is considering development of a sustainability metric, and  the Manager of the Shuttle Program Development Office believes that if  approved, it could be ready for use during the SLEP Summit in February  2004. NASA expects that the nomination of projects at that meeting will  come from a more comprehensive evaluation through extensive use of  hard data and rigorous analysis.
Although creation of the SLEP may improve the identification and  selection process, further improvements are possible. According to SLEP  program officials responsible for identifying, selecting, and prioritizing  shuttle upgrades, they need clear guidance from top management as to  how those decisions integrate with the other elements of the ISTP, such  as the ISS and the OSP. In addition, SLEP program officials said the  identification and selection of upgrades for the shuttle program lack a  clear measurable metric showing the relationship of an upgrade  investment to an increase in shuttle operational life. They believe such a  metric would be useful to decision makers in identifying, selecting, and  prioritizing shuttle upgrades. Finally, according to NASA Headquarters  officials, recommendations of the CAIB are under study and will likely  change the selection and prioritization of shuttle upgrades for both the  near term and the long term.
section_title: Shuttle Upgrades Could Potentially Cost Billions More Than Currently Estimated
paragraphs: Until NASA finalizes the basic requirements for the shuttle and further  improves its process for identifying and selecting upgrades, it will be  difficult to accurately and reliably estimate the total cost of upgrades  through 2020. NASA’s current estimate for the cost of upgrading the  shuttle is itself highly uncertain. Accurate and reliable cost estimates to  upgrade the shuttle to continue operations are needed by decision makers.  We found that the agency has not yet attempted to prepare a detailed  life-cycle cost estimate for all upgrades through 2020. NASA did prepare a  rough order of magnitude estimate based on an analysis of current project  estimates through 2020. The total cost of shuttle upgrades, however, could  potentially be significantly greater as the estimate did not include potential  projects such as a crew escape system. In addition, a number of potential  changes could significantly increase the estimated cost, such as changes in  program requirements, schedule slippages caused by delays in software  and hardware integration, and implementation of recommendations of  the CAIB.
subsections: section_title: Current Estimate Is Rough Order of Magnitude
paragraphs: A NASA official stated that it is difficult to develop accurate and reliable  long-term estimates of shuttle upgrades through 2020, particularly in  light of uncertainty of the shuttle’s basic requirements such as its life  expectancy. However, developing life-cycle cost estimates for agency  programs is not a new issue in the federal government. The Office of  Management and Budget maintains guidelines for preparing a  cost-effectiveness analysis, including life-cycle cost estimates applicable to  all federal agencies within the executive branch. Cost estimates should  include all costs consistent with agency policy guidance. NASA performs a  cost and systems analysis to produce feasible concepts and explore a wide  range of implementation options to meet its program objectives. To do  this, NASA must develop the life cycle of the program to include the direct,  indirect, recurring, nonrecurring, and other related costs for the design,  development, production, operation, maintenance, support, and retirement  of the program. Comprehensive life-cycle cost estimates include both the  project cost estimate and the operations cost through the end of shuttle  operations.
NASA has not prepared a detailed total life-cycle cost estimate for  upgrades through 2020 due to the uncertainty of the shuttle’s basic  requirements, as well as the difficulty of preparing estimates of out-year  funding to 2020. However, in June 2003, the agency estimated the shuttle  upgrade cost through that year by using a rough order of magnitude  estimate of $300 million-$500 million a year, or a total of $5 billion- $8 billion. The $300 million-$500 million per year estimate projected for  out-year funding was modeled using a simulation tool and developed by  an independent consulting firm. According to a NASA official, they will  rerun this estimate by the next SLEP Summit in February 2004, using as a  basis whatever the recommended upgrade projects are at the time.
We performed an analysis of the rough order of magnitude estimate  completed by NASA for all upgrades through 2020. Based on the data, we  found that the $300 million-$500 million range of estimated costs per year,  and the methodology used to estimate the costs, appears to be reasonable.  According to a NASA official, NASA’s cost estimates are focused on the  annual budget process, rather than long term through 2020, because any  individual project takes a while to mature and near-year estimates, such as  those from the current year and through 2008, would be more accurate  than those from 2009 and beyond, which are more likely to change.  NASA’s estimate is based on known projects for fiscal years 2004 and 2005  whose costs taper off in later years and the assessment of an additional 20  projects through 2020, where cost estimates and implementation plans are  not certain.
Although the rough order of magnitude estimate, as well as the  methodology used to derive it, appears to be reasonable, the total cost  could be billions more since potential upgrade projects such as a crew  escape system are not included. Initially, Boeing released a list of safety  and supportability options that included crew/cockpit escape concepts for  the shuttle. Figure 3 illustrates the primary types of crew escape presently  under consideration. The approximate costs involved for the eight present  concepts range between $1 billion and $3.9 billion, depending on the one  selected. There are three other ejection concepts under development,  none of which have received a full assessment. These other concepts will  be assessed in a more in-depth manner, as well as previous metrics and  costs, at the next SLEP Summit in February 2004. (Appendix III contains  information on all 11 concepts.)
section_title: Potential Program Changes Could Increase Total Upgrade Cost
paragraphs: A number of potential program changes could significantly increase the  estimated cost of shuttle upgrades through 2020. For example, rough order  of magnitude estimates do not account for possible slippages in the shuttle  schedule. According to a NASA official, if NASA and/or Congress deem a  crew escape option a major priority, more highly developed costs and  schedules would be created. Also, slippage due to delays in hardware or  software integration can affect projects where the final vehicle  modifications are planned for the major maintenance periods.
NASA has not yet made explicit decisions about the end state of the  International Space Station. For example, if the useful life of the ISS  were extended and/or an OSP were put into service to support the station  as an alternative to the shuttle, the life-cycle costs of the shuttle may be  affected. Until all requirements about the ISS have been fully defined,  it will be difficult to determine a detailed cost of shuttle upgrades  through 2020.
Other potential program changes that would increase costs include a  requirements change, such as additional lift capability that would require a  new rocket booster. Any redesign option, if selected, would add billions to  the total upgrade cost. For example, redesign and development of new  liquid-fueled rocket boosters is estimated at a rough order of magnitude  cost of $5 billion. Redesign and development of a five-segment solid  booster would be a cheaper but less flexible option, at an estimated rough  order of magnitude of $2 billion.
Another major driver of increased costs would be implementing the  recommendations of the CAIB. Its numerous recommendations, such as  major changes to the shuttle’s thermal protection system, could potentially  increase costs. NASA officials have said the agency intends to implement  all the recommendations the CAIB issued in its report, but precise costs  have yet to be determined.
section_title: Conclusions
paragraphs: NASA is at a critical juncture in the life of the space shuttle. NASA had  planned to upgrade the shuttle in the future. Now, after the Columbia  tragedy, NASA has an increased emphasis to fly the shuttle safely through  2020. NASA officials acknowledge that the loss of the Columbia will be a  key influence on the selection and prioritization of shuttle upgrades as  NASA officials assess both the short-and long-term implications of the  CAIB recommendations. Although creation of the Space Shuttle Service  Life Extension Program institutionalizes the process for identifying,  selecting, and prioritizing upgrades, additional changes are needed to  further strengthen that process such as increased use of analytic tools and  metrics to complement professional judgment. NASA management has  also not yet made explicit decisions about the basic requirements for key  elements in its Integrated Space Transportation Plan—the ISS, the OSP,  and the space shuttle. The agency’s lack of a long-term plan, caused by  frequent changes in the life of the shuttle, has made it hard to fully define,  select, and prioritize shuttle upgrade requirements, which form a basis for  identifying needed upgrades. Such a long-term plan needs to be developed  now in conjunction with activities to return the shuttle to fly safely. In  addition, accurate and reliable life-cycle cost estimates are important for  determining resources needed for the selection and priority of upgrades  and to determine annual budget requests. Even though an estimate of the  total life-cycle cost has not been made, it is evident that the cost of  upgrades through 2020 could be billions more than NASA’s current rough  order of magnitude estimate if potential projects, such as a crew escape  system and new projects resulting from the CAIB recommendations, are  included. Unless improvements are made in NASA’s shuttle modernization  efforts, NASA will not be able to ensure upgrades are being made to  address the most necessary needs or to articulate the extent of safety that  has been enhanced, and determine the total cost of the program.
section_title: Recommendations for Executive Action
paragraphs: To strengthen the agency’s efforts to modernize the space shuttle, we  recommend that the NASA Administrator take the following four actions:    Fully define the requirements for all elements of the ISTP so that those  responsible for identifying, selecting, and prioritizing shuttle upgrades will  have the guidance and a sound basis to ensure their decisions on upgrade  projects are completely integrated with all other elements of the  transportation plan. In particular, the Administrator should determine, in  conjunction with its international partners, the ultimate life and mission of  the ISS in order to provide a sound basis for fully defining shuttle  requirements.
Develop and consistently apply a clear measurable metric to show the  relationship of upgrade investments to an increase in shuttle operational  life and/or safety for the entire space shuttle system. NASA’s Quantitative  Risk Assessment System could be a basis for such a metric since it is  intended to measure the safety improvement of a single upgrade project.
Continue to pursue development of analytic tools and metrics to help  assure that SLEP program officials have accurate, reliable, and timely  quantifiable information to complement their professional judgment.
Develop a total cost estimate for all upgrades through 2020 by updating  the current rough order of magnitude estimate to include new projects  resulting from the CAIB recommendations, estimates of project life-cycle  costs, and estimates of major potential projects, such as a crew escape  system, so that the resources needed to fund shuttle upgrades can  be ascertained.
section_title: Agency Comments
paragraphs: In written comments on a draft of this report, NASA’s Deputy  Administrator stated that the agency concurred with the first three  recommendations. Furthermore, NASA concurred with the intent of the  fourth recommendation concerning development of a cost estimate for all  shuttle upgrades through 2020. However, the Deputy Administrator  commented that there were major uncertainties that severely limit the  agency’s ability to foresee budget requirements beyond 3 to 5 years, such  as unanticipated technical problems and the required time to accurately  assess upgrade projects. Consequently, NASA believes that it is better to  size the long-term (5 to 15 years) anticipated budget run-out based on  broad estimates rather than on specific lists of projects.
We recognize that there can be many uncertainties in developing long-term  budget estimates. However, NASA’s proposal of an anticipated budget run- out based on broad estimates is not a substitute for identifying the  financial implications of identified needs. Specifically, in order for NASA  to develop a credible Integrated Space Transportation Plan, the agency  needs a more accurate and reliable long-term total cost estimate. As we  stated in our recommendation, establishing such an estimate could be  facilitated by (1) using life-cycle cost estimating techniques on its list of  potential projects that NASA used to develop its cost estimate through  2020, (2) updating its list of potential upgrade projects to include  recommended projects of the CAIB, and (3) including major potential  upgrade projects currently under consideration, such as a crew escape  system. The comprehensive nature of this cost estimate will enable  (1) NASA to formulate a more definitive picture of how it will ensure that  the shuttle fleet flies safely in the future and (2) decision makers to  understand associated costs. Therefore, our recommendation remains  unchanged.
section_title: Scope and Methodology
paragraphs: To assess NASA’s requirements and plans to upgrade the shuttle for  continuous service through 2020, we obtained and reviewed internal  documents and independent studies and discussed the requirements and  plans with responsible NASA officials.
To assess how NASA determined what upgrades were needed and how  they were identified, selected, and prioritized, we obtained and analyzed  schedules and documents from program officials and obtained an  understanding of the process for identifying, selecting, and prioritizing  shuttle upgrades. We also reviewed documents regarding analytic tools  used to select and prioritize shuttle upgrades.
To assess the estimated life-cycle cost of shuttle upgrades, we reviewed  and discussed NASA’s guidance regarding preparation of life-cycle cost  estimates with program officials. To assess the rough order of magnitude  estimate for out-year funding completed by NASA for all upgrades through  2020, we obtained data and analyzed the estimate using a Monte Carlo  simulation tool called @Risk—an Excel-based spreadsheet. Monte Carlo  simulation helps to assess the risks and uncertainties associated with  Microsoft Excel spreadsheet models by randomly generating values for  uncertain variables over and over to simulate a model. We assessed this  technique to determine the level of confidence around the estimates and  verified our assessment with responsible program officials.
To accomplish our work, we interviewed officials and analyzed documents  at NASA Headquarters, Washington, D.C.; Johnson Space Center, Houston,  Texas; and Kennedy Space Center, Florida.
We also reviewed reports and interviewed representatives of NASA’s  Office of the Inspector General, Washington, D.C., and NASA’s  Independent Program Assessment Office, Langley Research Center,  Hampton, Virginia.
We conducted our work from April to October 2003 in accordance with  generally accepted government auditing standards.
Unless you publicly announce the contents of this report earlier, we plan  no further distribution of it until 30 days from the date of this letter. We  will then send copies to others who are interested and make copies  available to others who request them. In addition, the report will be  available on the GAO Web site at http://www.gao.gov.
Please contact me at (202) 512-4841 if you or your staffs have any  questions about this report. Major contributors to this report are listed in  appendix IV.
section_title: Appendix I: Comments from the National Aeronautics and Space Administration
section_title: Appendix II: Recommended Upgrade Projects Resulting from the Service Life Extension Program Summit
subsections: section_title: (Customer Driven Capabilities Related)
paragraphs: Performance Trade Studies (Lift, Power,  Stay Time)
Projects and Studies (Sustainability Related)
section_title: Appendix III: Comparison of Crew Escape Concepts Under Consideration
paragraphs: Capsule 10k –  210k   Seat 10k – 70k  6,024 lb added  2,700 ballast  4 y after ATP  12 m OMM  xxxx lb added  xxxx ballast  x.x y after ATP  xx m OMM  xxxx lb added  xxxx ballast  x.x y after ATP  xxx m OMM  xxxx lb added  xxxx ballast  x.x y after ATP  xx m OMM  52%  d/ePRA and Ascent coverage based on 42A ejection seat assessment.
Assessment due by February 2004 for Service Life Extension Program Summit.
section_title: Appendix IV: Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: Individuals making key contributions to this report included Jerry Herley,  Thomas Hopp, T. J. Thomson, and Karen Richey.