NASA's Space Vision: Business Case for Prometheus 1 Needed to Ensure Requirements Match Available Resources
Published: 2005-02-28T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: In 2003, the National Aeronautics and Space Administration (NASA) initiated the Prometheus 1 project to explore the outer reaches of the Solar System. The Prometheus 1 spacecraft is being designed to harness nuclear energy that will increase available electrical power from about 1,000 watts to over 100,000 watts and enable the use of electric propulsion thrusters. Historically, NASA has had difficulty implementing some initiatives. NASA's failure to adequately define requirements and quantify the resources needed to meet those requirements has resulted in some projects costing more, taking longer, and achieving less than originally planned. Prometheus 1 will need to compete for NASA resources with other space missions--including efforts to return the shuttle safely to flight and complete the International Space Station. GAO was asked to determine (1) whether NASA is establishing initial justification for its investment in the Prometheus 1 project and (2) how the agency plans to ensure that critical technologies will be sufficiently mature at key milestones.
section_title: What GAO Found
paragraphs: NASA is in the process of establishing initial justification for its investment in the Prometheus 1 project but faces challenges establishing preliminary requirements and developing accurate cost estimates. Decision makers will not get their first comprehensive picture of the project's requirements and the resources needed to meet those requirements until the preliminary mission and systems review, scheduled for summer 2005. Defining the project's requirements and developing life-cycle cost estimates by then could be challenging, given the short time frames. The fidelity of this information should improve by the preliminary design review scheduled for 2008. At that time, NASA has the opportunity to use these more refined requirements and cost estimates to establish a sound business case for its investment in the Prometheus 1 project. According to Prometheus 1 project management, a flat funding profile is inadequate to ramp up for the planned 2015 launch of Prometheus 1, the project's first spacecraft to its original destination of Jupiter's Icy Moons. By matching requirements to resources a sound business case would allow NASA to determine whether trade-offs in the design of the spacecraft or the agency's expectations are needed to avoid outstripping available resources. Significant program cost and schedule increases in past programs can be traced to not matching requirements with resources at preliminary design review. While development of the Prometheus 1 technologies is under way, each will require extensive advancement before they are mature enough to support reliable cost estimates. NASA is preparing technology development plans that include measurable criteria to ensure the Prometheus 1 technologies are on track for meeting NASA's maturity requirements through the end of the preliminary design phase. GAO's best practices work has shown, however, that establishing a formal business case based on a knowledge-based approach that includes matching requirements and available resources--which include technical and engineering knowledge, time, and funding--and controls to ensure that sufficient knowledge has been attained at critical junctures within the product development process is an essential part of any product development justification. NASA's current policy does not require projects to develop knowledge-based business cases that match requirements to available resources and include controls to ensure that sufficient knowledge has been attained. Therefore, the agency had not planned to develop such a business case for Prometheus 1. Since GAO provided our draft report to NASA for comment, the agency released its fiscal year 2006 budget request that includes changes to Prometheus 1. If properly implemented, these changes could be positive steps in addressing the findings and recommendations in this report.

section_title: Letter
paragraphs: In 2003, the National Aeronautics and Space Administration (NASA)  initiated the Prometheus 1 project to explore the outer reaches of the  Solar System in the hopes of finding answers to some of humankind’s most  profound questions about life and its origins. The Prometheus 1 spacecraft  is being designed to harness nuclear energy that will efficiently increase  available electrical power from about 1,000 watts to over 100,000 watts  and enable the use of electric propulsion thrusters. The availability of  nuclear power at this magnitude will fundamentally change NASA’s  capability to explore deep space.
While NASA has been successful in missions such as Mars Pathfinder and  Exploration rovers, the agency has had difficulty implementing a number  of other costly initiatives because it was overly optimistic in what could be  achieved within available resources. NASA’s failure to adequately define  requirements and quantify the resources needed to meet those  requirements has resulted in some projects costing more, taking longer,  and achieving less than originally planned. Prometheus 1 will compete for  NASA resources with other space missions—including, in the near term,  efforts to return the shuttle safely to flight and completing the  International Space Station.
Cognizant of the outlook for a constrained federal budget for the  foreseeable future and NASA’s difficulty in implementing some major  programs within projected resources, you asked us to review the  Prometheus 1 project to determine (1) whether NASA is establishing initial  justification for its investment in the Prometheus 1 project and (2) how the  agency plans to ensure that critical technologies will be sufficiently mature  at key milestones.
To address our objectives, we obtained and reviewed Prometheus 1 plans,  schedules, risk assessments, budget documentation, technology maturity  assessments, and technology development plans. We conducted further  qualitative and quantitative analyses of these documents and compared  them to criteria established in NASA and Jet Propulsion Laboratory (JPL)  policies governing development programs and in GAO’s best practices  body of work (see GAO Related Products). Our work was conducted  between April 2004 and January 2005 in accordance with generally  accepted government auditing standards.
Since our draft report was provided to NASA for comment, a significant  change has been made to the Prometheus 1 project. NASA’s fiscal year  2006 budget request includes changes to the Prometheus 1 project that  directly address the findings and recommendations of this report. These  changes are briefly outlined in the Agency Comments section of this  report.
subsections: section_title: Background
paragraphs: The Prometheus 1 project is part of NASA’s Prometheus Nuclear Systems  and Technology program to develop nuclear power technologies capable  of providing power and propulsion for a new generation of missions. The  Prometheus 1 spacecraft is being designed to use nuclear power and  electric propulsion technologies to explore the outer reaches of the solar  system. The Jupiter Icy Moons Orbiter (JIMO) mission—a 4 to 6-year study  of three of Jupiter’s moons: Callisto, Europa, and Ganymede—was the  original destination identified by NASA. The JIMO mission’s overarching  science objectives were to (1) investigate the origin and evolution of the  three moons; (2) scout their potential for sustaining life; and (3) determine  the current rate of movement of surface ice and the rates at which the  moons are weathered. With an unprecedented level of power,  Prometheus 1, the first in a potential series of spacecraft, is expected to  support the use of high capability science instruments and high power  communications systems to provide scientists with an a unprecedented  amount of scientific information. Figure 1 depicts the notional  Prometheus 1 spacecraft.
NASA contracted with the Jet Propulsion Laboratory (JPL) to manage the  Prometheus 1 project and to manage development of the science mission  payload. In turn, JPL awarded a $400-million contract for the initial  development of the Prometheus 1 spacecraft to Northrop Grumman Space  Technology in September 2004. NASA is collaborating with the  Department of Energy’s Office of Naval Reactors to develop and handle all  issues related to the spacecraft’s nuclear reactor.
The Prometheus 1 project will have to compete for funding with other  NASA programs. In January 2004, the President charged NASA with  implementing a new strategy for space exploration—which includes the  Prometheus 1 project—while simultaneously returning the shuttle to flight  status and completing the International Space Station. NASA laid out its  plan for implementing the strategy in its fiscal year 2005 budget request. In  essence, NASA’s implementation plan holds aeronautics, science, and  other activities at near constant levels and transitions funding levels  currently dedicated to the Space Station and shuttle programs to the space  exploration strategy as the Space Station and shuttle programs phase out.  This plan was predicated upon NASA’s annual funding level receiving  increases to about $18 billion a year by fiscal year 2008 and then remaining  near that level, except for inflation, through at least 2020.
subsections: section_title: Best Practices Reveal Elements of a Sound Business Case for Product Development
paragraphs: In the last several years, we have undertaken a best practices body of  work on how leading developers in industry and government use a  knowledge-based approach to develop products that reduces risks and  increases the likelihood of successful outcomes. Development of a sound  business case based on this best practices model enables decision makers  to be reasonably certain about their products at critical junctures during  development and helps them make informed investment decisions.
Our best practice work has shown that developing a sound business case  based on matching requirements to resources is essential to implementing  a knowledge-based approach. A sound business case includes the  following elements    well-defined requirements,    preliminary design,   realistic cost estimates, and    mature technology.
A knowledge-based business case also involves the use of controls or exit  criteria to ensure that the required knowledge has been attained at each  critical juncture. It ensures that managers will (1) conduct activities to  capture relevant product development knowledge, (2) provide evidence  that knowledge was captured, and (3) hold decision reviews to determine  that appropriate knowledge was captured to allow a move to the next  phase. If the knowledge attained at each juncture does not confirm the  business case on which the effort was originally justified, the program  does not go forward.
Use of this approach has enabled leading organizations to deliver high  quality products on time and within budget. Product development efforts  that have not followed a knowledge-based business case approach can be  frequently characterized by poor cost, schedule, and performance  outcomes.
Although NASA does not require projects to develop a formal business  case based on matching requirements to resources, JPL project  implementation policy, which establishes JPL’s institutional structure for  implementation and management of JPL flight projects in accordance with  NASA policies, does require projects to develop documentation that  includes elements essential to a sound business case. For example, before  entering the preliminary design phase, JPL projects are required to  develop preliminary requirements, a conceptual design, realistic cost  estimates, and technology development plans. JPL projects are required to  update and improve the fidelity of information in these documents by  PDR. The information in these documents could provide NASA decision  makers with the information necessary to support sound business case  decisions based on matching requirements to resources at preliminary  mission and systems review (PMSR) and PDR.
In September 2004, the Congressional Budget Office reported that if  NASA’s costs for implementing the strategy were similar to prior  analogous NASA programs—such as Apollo, Viking, and Mars Exploration  Rover—NASA’s funding needs could increase by 15 to 23 percent—or  $40 billion to $61 billion—over the 16-year estimate. The Congressional  Budget Office concluded that if funding were held constant, NASA would  likely have to either eliminate mission content or delay schedules.
section_title: Initial Justification for Prometheus 1 Project Could Form the Basis of a Sound Business Case
paragraphs: NASA is still in the process of preparing initial justification for the  Prometheus 1 project to enter the preliminary design phase. Consequently,  at this time the level of funding NASA needs to execute the project is not  fully defined. According to project officials, however, funding levels would  need to be increased to support the planned launch of Prometheus 1 to  Jupiter’s Icy Moons. While NASA plans to have defined preliminary system  requirements and an initial estimate of the life-cycle cost for Prometheus 1  by summer 2005—when the project enters the preliminary design phase— the agency faces significant challenges in doing so.
subsections: section_title: Project Management Believes Current Funding Is Insufficient to Support Launch of Prometheus 1
paragraphs: According to Prometheus 1 project management, current funding is  inadequate to support a 2015 launch of Prometheus 1 as initially planned.  Following small funding increases from fiscal years 2005 through 2007, the  budget profile becomes relatively flat through fiscal year 2009 (see fig. 2).  Project officials believe that the current profile would need to be  increased beginning in fiscal year 2007 to reflect project needs of a Jupiter  Icy Moons mission. Decision makers will not get their first comprehensive  picture of the project’s requirements and the resources needed to meet  those requirements—the first basis for funding decisions—until PMSR  scheduled for summer 2005. While the fiscal year 2006 request includes an  updated Prometheus 1 funding profile, a funding profile based on life-cycle  cost estimates—which NASA plans to have when it enters the preliminary  design phase—will not be included until NASA’s fiscal year 2007 request.
section_title: Short Time Frames and History Foretell Difficulty in Defining Requirements and Developing Life-Cycle Cost Estimates
paragraphs: The Prometheus 1 project office is required to develop preliminary  requirements by PMSR. Defining the project’s requirements and  developing life-cycle cost estimates by then could be challenging, given the  short time frames and NASA’s past difficulties developing requirements  and estimates. While it is not unusual for a project at this stage in  acquisition to still be defining requirements, several factors could make it  difficult for NASA to develop preliminary requirements by PMSR. The  contractor, Northrop Grumman, was only recently selected, and according  to project officials, input from both the contractor and Office of Naval  Reactors is needed to finalize the preliminary ground, space, and launch  systems requirements mandatory for PMSR. In addition, NASA continues  to refine its requirements. For example, Prometheus 1 project  management increased requirements for reactor lifetime, reactor power,  and propellant tank capacity to ensure that the Prometheus 1 spacecraft  and reactor designs could be used to support follow-on missions.  Currently, project managers are working with broad NASA requirements  for deep space exploration and more refined project requirements specific  to the Prometheus 1 ground, space, and launch systems.
NASA is also required to have an initial life-cycle cost estimate for  Prometheus 1 at PMSR. However, because the estimate is based on a  conceptual design, preliminary system requirements, and detailed  technology development plans that are not yet complete, it will be difficult  for NASA to develop an estimate in the short time available by PMSR.
The project office is working with Northrop Grumman to merge and  finalize the conceptual design. Once the conceptual design is finalized, the  project office will update the work breakdown structure and develop a  “grass roots” estimate of the spacecraft cost. However, project officials do  not expect to receive cost estimates from the Office of Naval Reactors and  Northrop Grumman, which are also needed to develop the estimate, until  the end of February 2005. The JPL Costing Office will prepare a separate  cost estimate based on its experiences with prior programs, and both JPL  and NASA will contract for additional independent cost estimates.
Adding to these complexities, NASA has historically had difficulty  establishing life-cycle cost estimates. In May 2004, we reported that  NASA’s basic cost-estimating processes—an important tool for managing  programs—lack the discipline needed to ensure that program estimates  are reasonable. Specifically, we found that 10 NASA programs that we  reviewed in detail did not meet all of our cost-estimating criteria—based  on criteria developed by Carnegie Mellon University’s Software  Engineering Institute. Moreover, none of the 10 programs fully met certain  key criteria—including clearly defining the program’s life cycle to  establish program commitment and manage program costs, as required by  NASA. In addition, only three programs provided a breakdown of the work  to be performed. Without this knowledge, we reported that the programs’  estimated costs may be understated and thereby subject to underfunding  and cost overruns, putting programs at risk of being reduced in scope or  requiring additional funding to meet their objectives. In this report we  recommended that NASA take a number of actions to improve its cost - estimating practices. NASA concurred noting that our recommendations  validated and reinforced the importance of activities underway at NASA.
section_title: Business Case Allows Match of Needs to Resources and Facilitates Informed Decision Making
paragraphs: By PDR—which occurs at end of the preliminary design phase and is  scheduled for 2008—the fidelity of the information is expected to improve  and could allow NASA to develop a business case that would match  requirements with resources and provide decision makers with the  information needed to determine whether continued investment in the  project is warranted. However, in the past NASA has had difficulties  developing the realistic requirements and cost estimates needed to  develop a sound business case.
To help ensure program requirements do not outstrip resources, leading  commercial firms obtain the right knowledge about a new product’s  technology, design, and production at the right time. We have issued a  series of reports on the success these firms have had in estimating the  time and money to develop new and more sophisticated products—the  kinds of results that NASA seeks. Our best practice work has shown that  developing business cases based on matching requirements to resources  before program start leads to more predictable program outcomes—that  is, programs are more likely to be successfully completed within cost and  schedule estimates and deliver anticipated system performance.
A sound business case includes the following elements—well-defined  requirements, a preliminary design, realistic cost estimates, and mature  technology. While NASA does not require projects to develop a formal  business case based on matching requirements to resources, JPL policy,  which implements NASA policy, does require projects to develop  documentation that could support formulation of a sound business case.  Before a JPL project enters the preliminary design phase, JPL project  implementation policy requires that the project develop preliminary  requirements, a conceptual design, realistic cost estimates, and technology  development plans. This policy also requires that the fidelity of  information in these documents improve by PDR.
The requirements and resource estimates NASA is developing for PMSR  could form the basis for an initial business case based on matching  Prometheus 1 requirements to available resources. However, Prometheus  1 project management plans to continue directing requirements changes to  accommodate follow-on missions. While our work shows that the  preliminary design phase is the appropriate place to conduct systems  engineering to support requirement/cost trade-off decisions, NASA needs  to remain cognizant that adding requirements could increase cost and risk.  In addition, NASA has had past difficulty developing the realistic  requirements and cost estimates needed to develop a sound business case.  These difficulties have resulted in the termination of several major efforts  after significant investment of resources. For example, in 2002 NASA  terminated the Space Launch Initiative (SLI) program—a $4.8 billion,  5-year program to build a new generation of space vehicles to replace its  aging space shuttle. SLI was a complex and challenging endeavor for  NASA, both technically and from a business standpoint. The SLI program  faced some of the same challenges that Prometheus 1 is struggling with  today, such as the need to develop and advance new airframe and  propulsion technologies. SLI did not achieve its goals, in part, because  NASA did not develop realistic requirements and cost estimates.
section_title: NASA’s Plans to Ensure Mature Technologies Rely on Prototype Demonstrations
paragraphs: Leading firms make an important distinction between technology  development and product development. Technologies that are not mature  continue to be developed in the technology base—they are not included in  a product development. Our best practices work has also shown that there  is a direct relationship between the maturity of technologies and the  accuracy of cost and schedule estimates. NASA’s Prometheus 1  technologies are currently immature. The Prometheus 1 project office is  preparing technology development plans to guide the development of each  key technology during the preliminary design phase.
subsections: section_title: Mature Technologies Are Key to Minimizing Risk
paragraphs: Maturing technologies during the preliminary design phase is a key  element of matching needs to resources before entering the product  development phase. Our best practices work has shown that technology  readiness levels (TRL)—a concept developed by NASA—can be used to  gauge the maturity of individual technologies (see fig. 4). (See app. I for  detailed definition of TRLs.) Specifically, TRL 6—demonstrating a  technology as a fully integrated prototype in a realistic environment—is  the level of maturity needed to minimize risks for space systems entering  product development.
While development of Prometheus 1 critical technologies is under way, the  technologies will require extensive advancement before they are mature  enough to provide the revolutionary capabilities of the Prometheus 1  spacecraft. The overall technology objective for Prometheus 1 is to safely  develop and operate a spacecraft with a nuclear-reactor-powered electric  propulsion system. To achieve this objective, the spacecraft will require  advancement in several technology areas, including, nuclear electric  power, power conversion and heat rejection systems, nuclear electric  propulsion, high power communications, radiation-hardened electronics,  and AR&D. (See app. II for a more detailed explanation of these  technologies.) NASA’s fiscal year 2005 budget request indicates that these  technologies are either at TRL 3 (individual technologies have been  demonstrated in a laboratory environment) or TRL 4 (system components  have been demonstrated in a laboratory environment). Before NASA  conducts the PDR in 2008, it will need to mature the technologies—each of  which comes with a unique set of engineering challenges.
To gauge the maturation of the Prometheus 1 technologies, the  Prometheus 1 project office is preparing technology development plans,  which rely on the use of maturity criteria tables (MCT), a concept similar  to TRLs. The specific maturation criteria for each technology vary  greatly, but all technologies are to be matured by PDR to the point that    developmental models are complete,    all major risks to each technology are retired,    all major manufacturing issues are resolved, and    plans for obtaining life data that will provide confidence that the  hardware will meet the mission lifetime requirements are in hand.
Prometheus 1 project officials believe these criteria roughly correspond to  a TRL 5 (component and/or breadboard validation in a relevant  environment) or a TRL 6 (system/subsystem model or prototype  demonstration in a relevant environment). The program office’s position is  that using MCTs that are equivalent to TRL 5 and TRL 6 at PDR is  appropriate because the program office is both the technology developer  and product developer and, as such, has a thorough understanding of how  mature the technologies need to be at certain points in time as the  program progresses. Nevertheless, the dual role of project office as both  technology and product developer is not unique, and our best practices  body of work shows that a TRL 6 is the level of maturity needed to  minimize risks for space systems entering product development.
section_title: Conclusions
paragraphs: NASA is quickly approaching one of the most critical phases in its  acquisition of Prometheus 1—the preliminary design phase. While the  impetus for the changes made to the program—subsequent to our  providing a draft of this report to NASA for comment—recognize the  technical, schedule, and operational risk of this program, there is still  much work to be done. Based on the information presented at PMSR, now  scheduled for summer 2005, NASA will need to decide at what level to  fund the project. However, NASA will be challenged to develop the  information required at PMSR, given the compressed time frames.  Although PDR is still several years out, NASA will face significant  challenges in meeting this milestone, given the immaturity of the  revolutionary technologies that NASA anticipates will be needed to  successfully launch Prometheus 1. While NASA is developing well-defined  criteria tables for maturing Prometheus 1 technologies, the many inherent  unknowns in developing technologies frequently results in unanticipated  difficulties and delays. NASA’s current policy does not require projects to  develop knowledge-based business cases that match requirements to  available resources and include controls to ensure that sufficient  knowledge has been attained and therefore the agency had not planned to  develop such a business case for Prometheus 1. We have found, however,  that establishing a formal business case based on a knowledge-based  approach that includes matching requirements and available resources— which include technical and engineering knowledge, time and funding— and controls to ensure that sufficient knowledge has been attained at  critical junctures within the product development process is an essential  part of any product development justification. The risk associated with  failing to meet these challenges is considerable. If NASA decides to move  forward without adequate information at PMSR—that matches  requirements and available resources and provides NASA decision makers  with a clear understanding of Prometheus 1’s potential return on  investment—Prometheus 1 may be unable to compete for funding within  NASA. Ultimately, NASA could find, as it has in the past, that the program  must be cancelled after having invested millions of dollars.
section_title: Recommendations for Executive Action
paragraphs: We recommend that the NASA Administrator take the following two  actions:  identify at PMSR the level of resources the agency is committing to  the project and direct project officials to develop project  requirements based on this resource constraint and ensure that prior to proceeding beyond PDR (currently planned for  2008) a sound business case is established which includes  confirmation that (1) critical technologies have been successfully  demonstrated as mature, (2) systems engineering has been  conducted to support requirements/cost trade-off decisions,  (3) requirements and resource estimates have been updated based  on the results of the preliminary design phase, (4) knowledge  based criteria are established at each critical juncture to ensure  that relevant product development knowledge is captured, and  (5) decision reviews are held to determine that appropriate  knowledge was captured to allow a move to the next phase.
section_title: Agency Comments
paragraphs: In written comments on a draft of this report, NASA’s Deputy  Administrator stated that the agency concurs with the recommendations,  adding that the recommendations validate and reinforce the importance of  activities underway at NASA to improve NASA’s management of complex  technical programs.
Subsequent to our draft report being provided to NASA for comment,  significant changes were made to the Prometheus 1 project. NASA’s fiscal  year 2006 budget request includes changes to the Prometheus 1 project  that directly address the recommendations in this report. According to  NASA’s budget justification, the agency is planning a less complex mission  than the original JIMO mission. According to program officials who we  consulted with following the release of the budget, eliminating the long  reactor lifetime, stringent radiation hardening, multiple launches, and  AR&D required for the JIMO mission will allow NASA “to walk before it  runs” and significantly reduce cost and technical risks. As a result, NASA  has delayed PMSR until summer 2005 and is conducting an analysis of  alternatives to identify a relevant mission with reduced technical,  schedule, and operational risk. The fiscal year 2006 budget request also  reshapes the Prometheus 1 funding profile to provide an orderly increase  in developmental activities.
Notwithstanding agreement with our recommendations, the Deputy  Administrator stated that NPR 7120.5B requires projects to develop a  business case. As we noted in this draft report, we recognize that NASA  policy requires the development of elements that could support the  formulation of a knowledge-based business case. However, we found no  explicit requirement within NPR 7120.5B for NASA projects to develop a  business case of any kind. More importantly, while NPR 7120.5B does  require that projects establish controls to monitor performance against  cost, schedule, and performance baselines and to conduct reviews  throughout the project’s lifecycle, it does not establish specific knowledge- based controls to ensure that the knowledge necessary to match resources  to requirements is in hand before moving forward. For example, whereas  NPR 7120.5B requires projects to conduct a preliminary design review  before entering NASA’s implementation phase, i.e., product development,  it does not establish knowledge-based criteria to ensure that technologies  needed to meet essential product requirements have been demonstrated to  work in a realistic environment. Likewise, NASA policy requires a critical  design review during a project’s implementation phase but does not  include knowledge-based criteria to ensure the design is stable. We have  found that such knowledge-based criteria, when tied to major events on a  program’s schedule, can disclose whether gaps or shortfalls exist in  demonstrated knowledge, which can presage future cost, schedule and  performance problems.
In his comments, the Deputy Administrator also noted that the  Exploration Systems Mission Directorate is in the process of initiating a  number of reforms to its project management policies and specified  formulation dates in the coming months. He outlined these reforms and  explained how they will allow NASA to address the recommendations in  our report. We are encouraged by these planned changes. If properly  implemented, they could be positive steps toward implementing a  knowledge-based approach to project management.
The Deputy Administrator also requested that the relationship between  JPL and NASA project management requirements be explicitly stated in  the report. We moved the information from a footnote into the body of the  report to clarify that relationship. We also addressed NASA’s technical  comments as appropriate throughout the report.
section_title: Scope and Methodology
paragraphs: To determine whether NASA is establishing justification for the project  and ensuring critical technologies are mature, we conducted interviews  with NASA Exploration Systems Mission Directorate and Prometheus 1  project officials at NASA Headquarters, Washington, D.C.; Marshall Space  Flight Center, Huntsville, Ala.; and the Jet Propulsion Laboratory,  Pasadena, Calif. We obtained and reviewed pertinent documents from the  agency. We conducted quantitative and qualitative analyses of project  schedules, risk assessments, budget documentation, technology maturity  assessments and technology development plans. We compared these  documents to criteria established in JPL and NASA policies governing  developmental programs and to criteria for a knowledge based approach  to acquisition described in GAO’s best practices body of work. We  discussed key project challenges with Prometheus 1 project officials, and  conducted GAO team meetings to discuss analyses and developing issues.  Our audit work was completed between April 2004 and January 2005.
As agreed with your office, unless you announce its contents earlier, we  will not distribute this report further until 30 days from its issuance date.  At that time, we will send copies to the NASA Administrator and interested  congressional committees. We will make copies available to others upon  request. In addition, the report will be available at no charge on the GAO  web site at http://www.gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-4841 or lia@gao.gov. Key contributors to this  report are acknowledged in appendix IV.
section_title: Appendix I: Technology Readiness Levels Definitions
subsections: section_title: Lowest level of technology readiness. Scientific research begins to be translated into applied research and development. Examples might include paper studies of a technology’s basic properties.
paragraphs: None  (Paper studies and  analysis)
Invention begins. Once basic principles are  observed, practical applications can be  invented. The application is speculative  and there is no proof or detailed analysis to  support the assumption. Examples are still  limited to paper studies.
None  (Paper studies and  analysis)
Active research and development is  initiated. This includes analytical studies  and laboratory studies to physically  validate analytical predictions of separate  elements of the technology. Examples  include components that are not yet  integrated or representative.
Analytical studies and  demonstration of nonscale  individual components  (pieces of subsystem).
Basic technological components are  integrated to establish that the pieces will  work together. This is relatively “low fidelity”  compared to the eventual system.  Examples include integration of “ad hoc”  hardware in a laboratory.
Low fidelity breadboard.  Integration of nonscale  components to show  pieces will work together.  Not fully functional or form  or fit but representative of  technically feasible  approach suitable for flight  articles.
Fidelity of breadboard technology  increases significantly. The basic  technological components are integrated  with reasonably realistic supporting  elements so that the technology can be  tested in a simulated environment.  Examples include “high fidelity” laboratory  integration of components.
High fidelity breadboard.  Functionally equivalent  but not necessarily form  and/or fit (size weight,  materials, etc.). Should be  approaching appropriate  scale. May include  integration of several  components with  reasonably realistic  support elements/  subsystems to  demonstrate functionality.
Lab demonstrating  functionality but not form  and fit. May include flight  demonstrating breadboard  in surrogate aircraft.  Technology ready for  detailed design studies.
Representative model or prototype system,  which is well beyond the breadboard tested  for TRL 5, is tested in a relevant  environment. Represents a major step up  in a technology’s demonstrated readiness.  Examples include testing a prototype in a  high fidelity laboratory environment or in  simulated operational environment.
Prototype—Should be  very close to form, fit and  function. Probably  includes the integration of  many new components  and realistic supporting  elements/subsystems if  needed to demonstrate  full functionality of the  subsystem.
High-fidelity lab  demonstration or  limited/restricted flight  demonstration for a  relevant environment.  Integration of technology  is well defined.
Prototype near or at planned operational  system. Represents a major step up from  TRL 6, requiring the demonstration of an  actual system prototype in an operational  environment, such as in an aircraft, vehicle  or space. Examples include testing the  prototype in a test bed aircraft.
Prototype.  Should be form, fit and  function integrated with  other key supporting  elements/subsystems to  demonstrate full  functionality of subsystem.
Flight demonstration in  representative operational  environment such as  flying test bed or  demonstrator aircraft.  Technology is well  substantiated with test  data.
Technology has been proven to work in its  final form and under expected conditions.  In almost all cases, this TRL represents the  end of true system development. Examples  include developmental test and evaluation  of the system in its intended weapon  system to determine if it meets design  specifications.
Actual application of the technology in its  final form and under mission conditions,  such as those encountered in operational  test and evaluation. In almost all cases,  this is the end of the last “bug fixing”  aspects of true system development.  Examples include using the system under  operational mission conditions.
section_title: Appendix II: Prometheus 1 Critical Technologies
subsections: section_title: Nuclear Reactor
paragraphs: The nuclear reactor is the key element of the Prometheus 1 spacecraft.  Without the power levels supplied by the reactor, the proposed  propulsion, science, and communication systems are not feasible.  Designing, constructing, and utilizing highly reliable, safe, portable nuclear  reactors is not new—nuclear reactors have been used in submarines for  almost 50 years. However, the United States has very little experience  operating nuclear reactors in a space environment and tackling space  unique nuclear application issues. The Office of Naval Reactors, the  organizational unit in the Department of Energy responsible for  developing nuclear reactors for the Navy, will be responsible for all  portions of the Prometheus 1 reactor development effort.
The space environment places significant weight constraints on the  reactor design and requires semi-autonomous control. Unlike submarines  and aircraft carriers, all spacecraft have serious weight constraints driven  by the cost of launching payloads into orbit. Consequently, spacecraft  designers put great effort into eliminating weight. Further, where  conventional reactors have hands on operators, the Prometheus 1 reactor  must be remotely controlled. NASA estimates that control  communications will take about 40 minutes to travel one way between  Earth and the Jovian system.
section_title: Power Conversion
paragraphs: A power conversion system accepts the thermal energy from the reactor  and converts it to useful electrical power for the spacecraft. Power  conversion is an integral part of any power generation system taking the  form of steam turbine generators in terrestrial utility plants and nuclear  submarines. NASA is considering two types of power conversion  systems—dynamic and static. According to NASA, the dynamic systems  under consideration offer the benefits of increased efficiency, reduced  weight and mass, and decreased nuclear fuel requirements. The static  systems, however, have a technology heritage in prior spacecraft and  could offer increased reliability because they have no moving parts.
section_title: Heat Rejection
paragraphs: Since the conversion process in a fission reactor is never 100 percent  efficient, heat rejection is required to dissipate waste energy. This is  usually accomplished with large pumped-water cooling systems on earth.  Space based power conversion would require a large radiator system to  dissipate the waste heat in the vacuum of space. The requirement to fold  the large radiator system into the launch vehicle fairing and deploy it after  launch complicates the radiator system design. (See fig. 1.)
section_title: Nuclear Electric Propulsion
paragraphs: Operating electric propulsion systems in space applications, including  deep space, is not new. There is extensive experience with electric  propulsion systems on satellites. In addition, NASA’s Deep Space 1  spacecraft was propelled using an electric propulsion ion thruster, similar  in nature to the concept being developed for Prometheus 1. The thruster  power levels required by Prometheus 1 have been demonstrated in a  laboratory environment. The lifetime required by Prometheus 1, however,  has not been demonstrated. Furthermore, lifetime testing of existing ion  thrusters has demonstrated that these thrusters were approaching “wear  out failure” after 30,352 hours. The Prometheus 1 thrusters will need to be  qualified for operational durations approaching 120,000 hours. NASA  recognizes that they will have to develop models and accelerated aging  techniques to demonstrate the lifetime requirement.
section_title: Radiation Hardening
section_title: High Power Communications
paragraphs: The nuclear reactor will provide increased electrical power for  communications. This translates to increased bandwidth and data rates.  The high power communications system onboard the Prometheus 1  spacecraft, will provide tens of compact disks full of data back to earth.  Analogous missions such as Cassini provide only a couple of floppy disks  full of data. (A floppy disk typically holds about 1.44 MB of data.  A compact disk typically holds about 700 MB of data.) According to  project officials, the higher power communications system on the  Prometheus 1 spacecraft will require upgrades to the Deep Space  Network, which are out of the purview of the Prometheus 1 project.
section_title: Autonomous Rendezvous and Docking
paragraphs: There is no launch vehicle in the present or proposed U.S. inventory  capable of launching the Prometheus 1 spacecraft, conceived for a mission  to Jupiter’s Icy Moons, into orbit in one piece. The conceptual design  currently shows the Prometheus 1 spacecraft to weigh between 29 and 36  metric tons and be about 58 meters in length. The current concept is to use  multiple launches, 2 to 5, to place the spacecraft components in orbit and  to use AR&D technology to assemble the spacecraft in orbit. Prometheus 1  is relying on NASA’s Demonstration Autonomous Rendezvous Technology  and Hubble Robotic Servicing Mission, and the Defense Advanced  Research Projects Agency’s Orbital Express programs for AR&D  technology. These programs use different sensors and approaches to  AR&D thereby providing Prometheus 1 with various options for  consideration.
section_title: Appendix III: Comments from the National Aeronautics and Space Administration
section_title: Appendix IV: GAO Contact and Staff Acknowledgments
subsections: section_title: GAO Contact
section_title: Acknowledgments
paragraphs: In addition to the contact named above, James Morrison, Jerry Herley,  John Warren, Tom Gordon, Ruthie Williamson, Karen Sloan and   Sylvia Schatz made key contributions to this report.
section_title: GAO Related Products
paragraphs: Best Practices: Using a Knowledge-Based Approach to Improve Weapon  Acquisition, GAO-04-386SP. Washington, D.C.: January 2004.
Best Practices: Setting Requirements Differently Could Reduce Weapon  Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.:  February 11, 2003.
Defense Acquisitions: DOD’s Revised Policy Emphasizes Best Practices,  but More Controls Are Needed. GAO-04-53. Washington, D.C.:  November 10, 2003.
Best Practices: Capturing Design and Manufacturing Knowledge  Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.:  July 15, 2002.
Defense Acquisitions: DOD Faces Challenges in Implementing Best  Practices. GAO-02-469T. Washington, D.C.: February 27, 2002.
Best Practices: Better Matching of Needs and Resources Will Lead to  Better Weapon System Outcomes. GAO-01-288. Washington, D.C.:  March 8, 2001.
Best Practices: A More Constructive Test Approach Is Key to Better  Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.:  July 31, 2000.
Defense Acquisition: Employing Best Practices Can Shape Better  Weapon System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.:  April 26, 2000.
Best Practices: DOD Training Can Do More to Help Weapon  System Program Implement Best Practices. GAO/NSIAD-99-206.  Washington, D.C.: August 16, 1999.
Best Practices: Better Management of Technology Development  Can Improve Weapon System Outcomes. GAO/NSIAD-99-162.  Washington, D.C.: July 30, 1999.
Defense Acquisitions: Best Commercial Practices Can Improve Program  Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999.
Defense Acquisition: Improved Program Outcomes Are Possible.   GAO/T-NSIAD-98-123. Washington, D.C.: March 18, 1998.
Best Practices: DOD Can Help Suppliers Contribute More to Weapon  System Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998.
Best Practices: Successful Application to Weapon Acquisition Requires  Changes in DOD’s Environment. GAO/NSIAD-98-56. Washington, D.C.:  February 24, 1998.
Major Acquisitions: Significant Changes Underway in DOD’s Earned  Value Management Process. GAO/NSIAD-97-108. Washington, D.C.:  May 5, 1997.
Best Practices: Commercial Quality Assurance Practices Offer  Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.:  August 26, 1996.