Space Station: Inadequate Planning and Design Led to Propulsion Module Project Failure
Published: 2001-06-20T00:00:00

Fast Facts:

Highlights:
section_title: Why GAO Did This Study
paragraphs: This report discusses the National Aeronautics and Space Administration's (NASA) contract with Boeing Reusable Space Systems to build the now-canceled follow-on propulsion module for the International Space Station.
section_title: What GAO Found
paragraphs: GAO found that the initial propulsion module project did not meet performance, cost, and schedule goals largely because NASA proceeded with Boeing's proposal without following fundamental processes involving project planning and execution. Once it was determined that Boeing's proposal was inadequate, NASA began to assess alternatives to the Boeing-proposed propulsion module. The assessment team defined mission success criteria, identified key design assumptions, and performed comparative analysis on competing designs. On the basis of its analyses, the team recommended a follow-on design. NASA acknowledged that its initial approach to developing a propulsion module was inadequate and contributed to the project's unsuccessful conclusion. NASA officials sought to learn lessons from the project in order to avoid similar problems in managing future programs.

section_title: Letter
paragraphs: The National Aeronautics and Space Administration (NASA) faces many challenges in developing and supporting the International Space Station. These challenges, such as Russian difficulty in completing its components on schedule, led NASA to pursue development of a U.S. propulsion capability for the space station to serve as an alternative to the planned Russian capability. In 1998, NASA accepted a proposal from Boeing Reusable Space Systems for a U.S. propulsion module. NASA’s initial effort to develop this module was not successful in meeting the program’s performance, cost, and schedule goals. The effort failed to produce a design that met mission requirements, increased its estimated cost by $265 million (from $479 to $744 million), and slipped its schedule by about 2 years. NASA eventually canceled the program and initiated a follow-on effort.
Although the follow-on propulsion module has since been terminated due to projected cost increases in the space station as a whole, you asked that we analyze the initial propulsion module project to help NASA avoid similar problems in the future. Specifically, we assessed NASA’s processes for program planning and ensuring that the proposed design met technical requirements. In addition, this report describes NASA’s evaluation of alternative approaches for the follow-on propulsion module effort and the results of NASA’s lessons learned studies from the failed initial program.
We have reported separately on NASA’s decision to develop the initial propulsion module under the Boeing space station prime contract.
subsections: section_title: Background
paragraphs: NASA and its international partners (Japan, Canada, the European Space Agency, and Russia) are building the space station as a permanently orbiting laboratory to conduct materials and life sciences research and earth observation and provide for commercial utilization and related uses under nearly weightless conditions. Each partner is providing station hardware and crew members and each is expected to share operating costs and use of the station.
Russia became a partner in 1993. As a partner, Russia agreed to provide hardware, such as the Service Module to provide station propulsion, supply vehicles, and related launch services throughout the station’s life. However, Russia’s funding problems delayed the launch of the Service Module by more than 2 years and raised questions about Russia’s ability to support the station during and after assembly.
Shortly after Russia came into the program, NASA began studying ways to provide the required propulsion using existing designs and hardware. Later, in response to continuing problems in the Russian space program, such as declining launch rates and funding shortages, NASA initiated additional studies at the Marshall Space Flight Center in Alabama. In the spring of 1995, it focused on satisfying the space station’s command and control and propulsion requirements. In 1996, Marshall proposed building a propulsion module in-house, and in 1997, NASA considered using existing Russian hardware to provide the needed propulsion.
In 1997, in response to continuing concerns over Russia’s ability to fulfill its station commitments, NASA developed a contingency plan, which included a strategy to mitigate the risk of both further delay on the Service Module propulsion capabilities and Russia not being able to meet station propulsion needs. Key elements of the plan involved developing an interim control module for near-term needs and a propulsion module to provide a permanent U.S. capability.
In 1998, Boeing Reusable Space Systems proposed a propulsion module concept that was to rely heavily on existing shuttle hardware, provide for on-orbit refueling, and cost about $330 million. This proposal coincided with renewed concern that Russia would not be able to fulfil its commitment to provide station propulsion capability. NASA decided to move ahead with the proposal based on a strategy that included refining the design during subsequent requirements and design reviews. It adopted this strategy based on its assumption that a propulsion module would be required by early 2002 if Russia failed to launch the Service Module. In July 2000, Russia successfully launched the Service Module, thus mitigating NASA’s immediate concern that the space station would not have adequate propulsion capability. However, the agency still proceeded with the propulsion module project because of long-term concerns with Russia’s ability to fulfil its commitments.
section_title: Basic Project Management and Requirements Principles Not Followed
paragraphs: NASA proceeded with Boeing’s proposal without following fundamental processes involving project planning and execution. Specifically, project management never finalized its project plan or operational concept and did not receive timely approval for its risk management plan. The design ultimately required substantial changes.
subsections: section_title: Propulsion Module Project Planning Was Inadequate
paragraphs: To prudently manage the project, NASA should have prepared and completed a number of planning documents and established baseline goals. Specifically, NASA did not do the following:   Complete a project plan. Documented project plans help to define realistic time frames for system acquisition and development and identify responsibilities for key tasks, resources, and performance measures. Without them, there is no yardstick by which to measure the progress of the developmental effort.
Fully develop a concept of operations document. This document describes the range of operational scenarios under which project hardware will have to function. This document is necessary to define requirements, operational plans, and associated training. The project began with a rudimentary concept that was continually refined during the course of the project.
Complete an approved risk management plan in a timely manner. A formal risk management plan helps management identify, assess, and document risk associated with the cost, resource, schedule, and technical aspects of the project. Without such a plan, organizations do not have a disciplined means to predict and mitigate risks.
Develop realistic cost and schedule estimates for the life of the project.
NASA guidance states that life-cycle cost estimates shall be developed as a part of the project’s planning activity.
Because of its concerns that Russia would be unable to provide space station propulsion capability, NASA approached the effort with a sense of urgency. Its analysis indicated that the U.S. propulsion capability would be needed by early 2002 if Russia did not meet its commitment. Given the initial estimate of the time that would be needed to develop and launch the propulsion module, NASA believed it was necessary to expedite the project. As a result, NASA chose to simultaneously plan and execute the project, thereby inhibiting use of fundamental planning documents during project formulation.
According to NASA officials, the absence of approved planning documentation and the urgency NASA perceived in executing the project made it difficult to effectively guide the project or measure its progress. For example, roles and responsibilities continued to change, impeding the flow of information. In addition, the absence of accurate technical, cost, and schedule estimates early in the project made it difficult for NASA to track cost variances. As a result, NASA officials told us that the estimated $265 million cost increase announced just before the program was suspended came as a surprise. They also stated that, had more analytical rigor been applied, they would have determined earlier in the program that the Boeing proposal would not meet project goals.
This procurement strategy also caused NASA to purchase long-lead items before the project’s requirements, concept of operations, and costs were fully understood. According to NASA, prior to the decision to cancel the project, it had obligated about $40 million for the purchase of various long- lead items. Some of these items could be used on the space station or other NASA projects. However, other items were unique to the propulsion module project.
Our findings on lapses in project planning are consistent with results from a NASA independent assessment team, which reviewed the propulsion module project between September 1999 and March 2000. This team concluded that the project was at high risk due, in part, to the fact that these critical project management processes were not followed. Specifically, the team concluded that (1) the project would not be ready to proceed through the design reviews until the project plan was fully developed and approved, (2) a well-integrated risk management program was not in place, and (3) the project could not be completed within the budget or achieve its planned delivery date.
section_title: NASA Reviews Identified Deficiencies in the Propulsion Module Design
paragraphs: NASA proceeded to implement Boeing’s proposal before it determined whether the design would fully meet the project’s technical requirements. The following top-level requirements were established at the beginning of the program:   Provide reboost and attitude control capability.   Provide up to 50 percent of total on-orbit space station propulsion needs.  Provide 12-year on-orbit life.   Maintain orbiter compatibility and transfer capability (pressurized transfer tunnel for crew and supplies).
Provide capability to be launched and returned by the shuttle.   Conform to NASA safety provisions.
Even though the top-level requirements provided a framework to guide propulsion module development, NASA’s reviews of the module’s detailed technical requirements identified major concerns. For example, an April 1999 systems requirements review found that NASA did not have detailed analyses to quantify the amount of propulsion capability that would be required. NASA space station program personnel later determined the definition of what propulsive capability was required; however, this definition was not available until a few months before the initial design review and could not, therefore, be used to judge the design’s suitability. Subsequent reviews found deficiencies with the design elements of the module itself.
Although technical requirements were never finalized and continued to change, NASA accepted and began to implement Boeing’s design. Typically, technical requirements are determined prior to selecting a design to ensure that it can satisfy established technical and safety needs. NASA accepted Boeing’s proposed design and began to implement it because the agency believed (1) the design was stable and mature because some of the proposed hardware had been used on the space shuttle and (2) costs were essentially fixed because the required development activities were understood and would not change.
As NASA implemented the design—establishing the organization for and responsibilities of the project office, purchasing long-lead items, etc.—it discovered a number of unexpected technical complexities and other obstacles in the design. These problems put into question the ability of the design to meet the technical requirements, as indicated in the following examples.
A central requirement was for the propulsion module to be refueled while in orbit. NASA began to question Boeing’s ability to incorporate on-orbit fuel transfer into the design, citing significant cost, safety, operational, and system design issues. Ultimately, NASA eliminated this requirement and reduced the module’s on-orbit life expectancy from 12 to 6 years. Eliminating these requirements meant that the propulsion module had to return to earth for refueling; the concept of returnability had not been fully analyzed; thus, a new design team had to be established to assess these impacts.
The design also proposed a tunnel diameter that proved too small to accommodate crew operations and did not meet space station minimum diameter requirements. In addition to crew passage, the tunnel served as the primary path for equipment/supply movement from the shuttle to the station. The tunnel size was later increased based on NASA’s concerns.
The design made extensive use of existing shuttle flight hardware that had not been designed for a 12-year application, and Boeing assumed NASA would accept the hardware based on prior shuttle experience. However, NASA assessed the hardware and found that much of it could not meet a 12-year life requirement. In addition, the development specification did not fully address testing requirements because Boeing assumed a simplified level of testing. However, testing requirements were later expanded.
The propulsion module project failed to successfully complete its preliminary design review in December 1999, despite the fact that it had been considered a mature design. The review concluded, in part, that the initial propulsion system design did not meet the space station and space shuttle safety requirements and that another review of propulsion related issues was needed. In March 2000, NASA’s independent assessment team concluded that the design was not mature, requirements were not adequately defined, and major design impacts were likely.
The process by which NASA and Boeing attempted to execute the project resulted in design changes, added effort, schedule slippage, and purchase of long-lead items before the design was fully understood. As a result, the project’s total cost estimate increased significantly. In February 1999, Boeing estimated that total program cost would be $479 million and maintained that estimate until April 2000. At that time, Boeing increased its estimate by $265 million—from $479 million to $744 million. Over this period, the scheduled launch date slipped by almost 2 years—from August 2002 to July 2004. Based on Boeing’s revised estimates, NASA began to question the project’s viability, and in July 2000 it informed Boeing that it would not authorize any additional work on the project.
section_title: Follow-on Propulsion Module Effort Included Comparative Analyses of Alternative Designs
paragraphs: Prior to abandoning its original propulsion module design, NASA established an Alternative Propulsion Module Assessment Team in May 2000 to review design concepts for their potential to meet the space station program’s propulsion requirements. According to NASA officials, this effort brought early analytical rigor to requirements definition, which NASA had failed to do in the initial project. During the preliminary phase of the assessment, team members considered many diverse options. These options varied in design factors such as module location, number of propulsion elements, and propellant systems. Each option also had to meet the basic top-level requirements. Specifically, the alternative propulsion module had to provide space station attitude and orbit maneuver control, be located on the U.S. segment of the station, leave two ports available for other vehicles to dock to the station, meet space station safety requirements, and initially provide 50 percent of the space station’s propulsion needs. In addition, the design had to be adaptable to eventually provide 100 percent of the station’s propulsion needs.
The assessment team identified five potential concepts, including two modified versions of Boeing’s baseline propulsion module design; the Z1 truss option, which attached to the station’s truss system; the split element option with separate propulsion and avionics elements; and the Node X option that had the propulsion elements attached to the Node 1 structural test article. The team designated a subteam to refine each option’s design. In addition, the subteams consulted with a cost assessment group to develop cost estimates for each option. The cost assessment group considered both initial capability costs, such as development and integration, and additional life-cycle cost elements, such as shuttle launches, labor, and spare parts.
Using the subteams’ analyses, the assessment team ranked the propulsion module alternatives in three categories—programmatic (composed of schedule, cost, and risk), technical (including safety, design, and performance), and integration issues (such as International Space Station and shuttle impacts and logistics issues). The team weighted programmatic issues the highest at 60 percent and technical and integration issues at 20 percent each. NASA officials told us that these weightings, typical for this kind of analysis, were approved by the space station program management.
The assessment team concluded that the Z1 truss option was the best choice. This option did not require the construction of a pressurized element and was estimated to cost $515 million to develop. The next best alternative was the Node X option, with an estimated cost of $700 million to develop. According to NASA, this option was already well understood because Boeing had already integrated a similar structure, Node 1, into the space station.
Although the assessment team found the Z1 truss option superior, it recommended a follow-on study because issues associated with this option’s integration into the space station were not well understood. Consequently, in July 2000, a joint NASA and Boeing integration evaluation team examined integration risks and identified possible design improvements for the Z1 truss and Node X options. NASA believed Boeing’s involvement was important because as the prime contractor, it would be responsible for integrating the alternative propulsion module into the space station.
The methodology that the integration evaluation team used was similar to that used by NASA’s assessment team in reviewing the propulsion module options. The integration team designated individual teams to evaluate the ZI truss and Node X options from various functional perspectives, such as power; structures and mechanisms; guidance, navigation, and control; and contamination. The functional teams developed criteria for their particular discipline and evaluated the two options accordingly. For example, the structures and mechanisms team evaluated the two options for peak loads and structural fatigue, and the power team for average and peak power consumption. Based on its evaluation, each team recommended a preferred option. The integration team’s project manager then led an effort to compile and analyze the functional teams’ recommendations.
Based on this analysis, the team selected the Node X option, which had the highest overall mission suitability and lowest integration risk. In contrast, the Z1 truss option created structural stress, station controllability, and propellant inefficiency issues. The integration team then concluded that Node X was the preferred choice as a follow-on effort to the initial propulsion module project. Figure 1 depicts the Node X propulsion module configuration.
The cost assessment group incorporated the results of the integration evaluation team into a new cost analysis for the ZI truss and Node X options. According to the new cost analysis, the Z1 truss option’s integration issues increased its estimated cost to $729 million. In contrast, the cost estimate for the Node X option decreased to $675 million, primarily because the outfitting costs for the structural test article were lower than expected.
NASA accepted the integration team’s findings and issued a request for proposal on the Node X option in January 2001. However, 2 months later, NASA canceled the follow-on effort because of cost increases in the space station program as a whole. In addition, NASA believed that the risk of Russian nonperformance was reduced because of the Service Module’s deployment.
section_title: NASA Recently Identified Lessons Learned to Apply to Future Programs
paragraphs: NASA acknowledged that problems with the management of the initial propulsion module project contributed to its unsuccessful conclusion, and it is undertaking lessons learned efforts to help avoid similar problems in managing future programs. These assessments include top down and systems engineering reviews at the Marshall Space Flight Center and an assessment by an engineer at the Johnson Space Center in Texas related specifically to the on-orbit fuel transfer component of the module.
According to NASA officials, drafts of the Marshall assessments identified the lack of early systems analysis and good teamwork as contributing to project failure. For example, key components of the design—use of existing hardware, on-orbit fuel transfer, and tunnel size—were never tested for feasibility, partly because Boeing believed that NASA had fully accepted the assumptions inherent in its design. Later, when these assumptions became invalidated or retracted, it became apparent that the original concept was no longer feasible.
The top down assessment also cited a lack of cooperation between NASA and Boeing as inhibiting the timely completion of required design and risk management analyses. In many cases, when NASA and Boeing teams tried to work together, they became confrontational and nonproductive. It concluded, in part, that, in the future, NASA should ensure that (1) early planning documents define what roles the various project teams have and how they interact, (2) government and contractor counterparts are established to encourage collaboration, and (3) NASA and contractor management monitor the interaction of the overall project team and intercede if conflicts interfere with the project’s success.
Another lessons learned assessment was performed on the on-orbit fuel transfer component of the Boeing proposal, a basic requirement of the proposed design. This assessment concluded that, while some project communication was good, internal contractor communication on this part was less than desirable. For example, center-to-center communication was aided by daily conversations between the on-orbit fuel transfer and propulsion module project managers. However, contractor participation in working group activities was not supported. The assessment also cited NASA’s lack of systems analysis early in the program and its difficulty in establishing requirements, estimating cost and schedule, and providing human capital resources as contributing to the on-orbit fuel transfer project failure.
The themes cited in NASA’s propulsion module project lessons learned studies are consistent with those cited in previous program failure assessments. In December 2000, NASA issued a report synthesizing the findings and recommendations from earlier reports, arriving at five themes it considered necessary for sound project management. The five themes were developing and supporting exceptional people and teams, delivering advanced technology, understanding and controlling risk, ensuring formulation rigor and implementation discipline, and improving communication.
section_title: Agency Comments
paragraphs: In commenting on a draft of this report, NASA stated that while it was in agreement with the findings of the report, the project’s urgency necessitated its management approach. However, NASA acknowledges that its project execution could have been improved and that it will now strive to apply lessons learned from the propulsion module project experience.
Even though NASA perceived a schedule urgency in starting and completing the project, it should have followed sound management practices. The early analytical rigor NASA was applying to the follow-on propulsion module effort would have served the agency well in its execution of the initial project.
section_title: Scope and Methodology
paragraphs: To assess the adequacy of project planning, we reviewed, analyzed, and compared internal NASA guidance and project planning documents. We also discussed planning requirements with cognizant project officials and independent assessment team officials to obtain their views.
To assess the extent to which NASA had defined the technical requirements for the propulsion module, we reviewed the results of requirements meetings and approved requirements lists to gain an understanding of the evolution of requirements determinations. We also held discussions with NASA and Boeing officials to obtain their perspectives on the validity of the technical requirements, as well as reasons for requirements changes over the course of the program.
To describe NASA’s process for reviewing alternative designs, we reviewed NASA briefing materials and other products related to the establishment of the Alternative Propulsion Module Assessment Team and others. We also discussed the teams’ charter, methodology, and results with team members and other cognizant officials.
To describe lessons learned by NASA from the initial program, we reviewed the results of NASA’s efforts and discussed their significance with cognizant officials.
We conducted our review from July 2000 to April 2001 in accordance with generally accepted government auditing standards.
Unless you publicly announce its contents earlier, we plan no further distribution of this report until 10 days from its issue date. At that time, we will send copies to the NASA Administrator; the Director, Office of Management and Budget; and other interested parties. We will also make copies available to others on request.
Please contact me at (202) 512-4841 if you or your staff have any questions about this report. Other key contributors to this report are acknowledged in appendix II.
section_title: Appendix I: Comments From the National Aeronautics and Space Administration
section_title: Appendix II: Staff Acknowledgments
subsections: section_title: Acknowledgments
paragraphs: Jerry Herley, John Gilchrist, James Beard, Fred Felder, Belinda LaValle, Vijay Barnabas, Diane Berry, Rick Eiserman, Susan Woodward, and Cristina Chaplain made key contributions to this report.