Report to the Subcommittee on Space

and Aeronautics, Committee on

Science, House of Representatives

United States General Accounting Office

GAO

September 2002 SPACE

TRANSPORTATION

Challenges Facing

NASA’s Space Launch

Initiative

GAO-02-1020
Page i GAO-02-1020 Space Transportation

Letter 1

Results in Brief 2

Background 3

Important Decisions to Be Made Before Requirements Can Be

Defined 7

Key Management Controls Are Not Yet Implemented 9

Conclusions 14

Recommendations 15

Agency Comments 16

Scope and Methodology 17

Appendix I Comments from the National Aeronautics and Space

Administration 19

Appendix II GAO Contact and Staff Acknowledgments 22

Table

Table 1: Activities Related to the Space Launch Initiative 7

Figures

Figure 1: NASA Illustration of the 2nd Generation Reusable Launch

Vehicle 4

Figure 2: Illustration of NASA’s Overall Space Transportation Plans 5

Abbreviations

DOD Department of Defense

EVM Earned Value Management

NASA National Aeronautics and Space Administration

RFP Request for Proposal

SLI Space Launch Initiative

TRL technology readiness level

Contents
Page 1 GAO-02-1020 Space Transportation

September 17, 2002

The Honorable Dana Rohrabacher

Chairman

The Honorable Bart Gordon

Ranking Member

Subcommittee on Space and Aeronautics

Committee on Science

House of Representatives

In 2001, the National Aeronautics and Space Administration (NASA) began

undertaking a new effort—the Space Launch Initiative (SLI)—to develop a

new generation of space transportation vehicles. SLI is expected to result

in the development of the second generation of reusable launch vehicles,

the space shuttle being the first generation. NASA plans to spend

$4.8 billion on the program through fiscal year 2006. SLI is part of a

broader program—known as NASA’s Integrated Space Transportation

Plan—to address future space transportation needs. Under the plan, NASA

envisions making upgrades to extend the life of the space shuttle and

undertaking longer-term research and development of future

transportation technologies and transportation systems, including space

vehicles that can reach orbit in just one-stage.

SLI is a highly ambitious program. It will require NASA to develop and

advance new technologies, such as propulsion and airframe systems,

which in turn can potentially be used by U.S. industry to create new

business opportunities in space. The undertaking will also require a high

level of communication and coordination between a range of partners,

including private-sector contractors, academia, and the Department of

Defense (DOD). Moreover, it will require effective controls and oversight

to reduce cost, scheduling, and technical risks. NASA’s previous attempts

to develop a new generation of space vehicles were unsuccessful largely

because NASA did not successfully implement and adhere to critical

project management controls and activities.

You requested that we assess NASA’s progress with the Space Launch

Initiative, particularly with respect to defining requirements and

implementing management controls.

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-02-1020 Space Transportation

NASA plans to define basic requirements for its second-generation

reusable launch vehicle—that is, what the crew size will be, what the

payload capacity will be, and what designs or architectures are worth

pursuing—by November 2002. But considerable challenges must be

addressed before NASA can accomplish this.

First, NASA has to complete a reassessment of its overall space

transportation plans. In doing so, it must decide whether it should

continue pursuing the development of second-generation vehicles as

planned, pursue alternative ways to develop the second generation in

order to more quickly replace the space shuttle, or postpone these efforts

altogether indefinitely until there is a major breakthrough in technology

that could vastly improve performance and reduce costs. This decision

will be difficult, given the uncertainties about the availability of

technologies needed to reduce costs and enhance performance for future

space flight.

Second, NASA is currently reassessing the future of the International

Space Station. The decisions it will make as part of this evaluation, such as

how many crew will operate the station, will have a dramatic impact on

NASA’s requirements for a second-generation vehicle. But they will be

difficult to reach, since they require NASA to come to agreement with

international partners who are concerned about planned cutbacks to the

station’s capabilities.

Third, NASA needs to decide whether the SLI program will be developed

jointly with DOD and, if so, how it can accommodate DOD’s requirements

for a reusable launch vehicle. So far, indications are that NASA and DOD

will share many of the same objectives for the vehicle, but there are

significant differences in priorities and requirements.

Until NASA finalizes its basic requirements for SLI, it cannot implement

management controls that are essential to predicting what the total costs

of the program will be and to minimizing risks with NASA’s planned initial

investment of $4.8 billion. These include cost estimates, controls designed

to provide early warnings of cost and schedule overruns, and risk

mitigation plans. Moreover, there are potential impediments to NASA’s

development and effective use of a detailed cost estimate, including the

lack of a modern integrated financial management system. Lastly, NASA

does not plan to develop several measures that are important to assessing

how the program as a whole is making progress toward achieving its key

objectives, reducing risks, and maturing technology.

Results in Brief
Page 3 GAO-02-1020 Space Transportation

It is important for NASA to implement management controls for SLI as

soon as possible, so that it can provide its managers and the Congress with

the information needed to ensure that the program is on track and able to

meet expectations. We are making recommendations to NASA that focus

on the need to make decisions with regard to the future of NASA’s overall

space transportation plan, the future of the space station, and DOD’s

participation in the SLI program before setting requirements for SLI. We

are also making recommendations aimed at implementing management

controls for the SLI program.

In its comments to a draft of this report, NASA stated that it concurs with

the recommendations. NASA believes that much of the SLI program’s

success is directly related to the implementation of project management

controls and appropriate levels of insight. NASA’s response is included as

appendix 1.

NASA’s Space Launch Initiative is an effort to develop and build a second

generation of reusable space transportation vehicles. (See fig. 1 and table

1.) NASA’s current transportation vehicle, the space shuttle, has been in

use for 20 years and requires a significant portion of NASA’s resources to

operate and maintain. The primary goals for SLI are to reduce the risk of

crew loss as well as substantially lower the cost of space transportation so

that more funds can be made available for scientific research, technology

development, and exploration activities. Currently, NASA spends nearly

one-third of its budget on space transportation.

Background
Page 4 GAO-02-1020 Space Transportation

Figure 1: Illustration of the 2nd Generation Reusable Launch Vehicle

Source: NASA.

SLI is part of a broader program—known as NASA’s Integrated Space

Transportation Plan—to address space transportation needs. Under the

plan, NASA could operate the space shuttle through 2020 and make

software and hardware upgrades to the shuttle in order to extend its use to

this point. It envisions the deployment of second-generation cargo vehicles

to begin around 2011 and crew vehicles around 2014. As with the shuttle,

NASA envisions that the second-generation vehicle will reach orbit in two

stages. NASA also anticipates building a third generation of vehicles in

2025 and even a fourth generation in 2040. (See fig. 2.) NASA anticipates

that these vehicles would reach orbit in one stage; travel beyond low-earth

orbit to far-reaching interstellar missions; and employ revolutionary

technologies, such as (1) rocket engines that breath oxygen from the air

during the climb to orbit rather than carrying heavy oxidizers onboard,

(2) propellantless electromagnetic propulsion, and (3) solar-powered

space sails.
Page 5 GAO-02-1020 Space Transportation

Figure 2: Illustration of NASA’s Overall Space Transportation Plans

Source: NASA.

Building the second-generation vehicle will be a considerably complex and

challenging endeavor for NASA—from both a technical and business

standpoint. For example, NASA plans to develop and advance new

technologies for the new vehicle. These include (1) new airframe

technologies that will include robust, low-cost, low-maintenance structure,

tanks, and thermal protection systems, using advanced ceramic and

metallic composite materials, and (2) new propulsion technologies,

including main propulsion systems, orbital maneuvering systems, main

engines, and propellant management. If successfully developed, NASA

believes that these new technologies could substantially reduce the costs

to maintain and operate the vehicle and increase its reliability.

NASA also plans to develop the new vehicle through partnerships with

private industry, academia, and DOD. It is partnering with the private

sector so that it can help create business opportunities in space, and it is

partnering with DOD because of military needs for a reusable launch

vehicle. Such partnerships will require a high level of coordination and

communication, since agreements need to be reached on what the basic

capabilities of the new vehicle will be, what designs or architectures1

should be pursued, how development costs will be shared, and what

individual partner responsibilities will be.

NASA recognizes that there are substantial technical and business risks

with SLI, and it is undertaking activities aimed at reducing them. To

reduce technical risks, for example, NASA is exploring, developing, and

testing technologies to make sure that they can be used on the new

 1 “Architecture” broadly includes an Earth-to-orbit reusable launch vehicle; on-orbit

transfer vehicles and upper stages; mission planning; ground and flight operations; and

support infrastructure, both on the ground and in orbit.
Page 6 GAO-02-1020 Space Transportation

vehicle, rather than proceeding with the program with uncertainty about

whether technologies will be mature enough when they are needed. To

reduce business and costs risks, NASA’s goal is to ensure that there is

adequate competition in the development of the architecture and that the

architectures that NASA pursues will enable convergences with NASA’s

requirements and commercial and military missions, and will not pursue

vehicles that cannot meet program goals.

To date, NASA has already explored hundreds of concepts and

technologies that could be used for future government and commercial

launch systems and space transportation operations. The “leap-ahead”

technologies studied include crew survival systems, advanced tanks and

airframe structures, long-life rocket engines, and thermal protection

systems. In March 2002, NASA selected 15 of the most promising

candidates to go forward into more detailed development. In November

2002, NASA plans to narrow the field down to 3 candidates and in

September 2003 to at least 2 candidates. From fiscal year 2003 through

fiscal 2006, two competing system architectures will be developed, ready

for a full-scale development decision in fiscal 2006.

NASA plans to spend $4.8 billion for the SLI program through fiscal year

2006. This investment is to be used for (1) systems engineering and

activities concerning the definition of requirements; (2) efforts to compete

designs for the reusable launch vehicle as well as efforts to reduce

business and technical risks; (3) activities to develop and demonstrate

designs, technologies, and system-level integration issues associated with

such NASA-unique transportation elements as a crew transport vehicle

and cargo carriers; and (4) activities to develop alternative access to the

space station as well as to provide contingency backup or relief for the

space shuttle. NASA’s efforts to compete designs for the vehicle and

reduce technical and business risks consist of approximately $3 billion of

the planned initial investment. At the present time, no prototype vehicle

integrating all new SLI technologies is planned before NASA’s fiscal year

2006 decision on whether to proceed to full-scale development.
Page 7 GAO-02-1020 Space Transportation

Table 1: Activities Related to the Space Launch Initiative

Year Month Event Narrative

2001 Feb. • SLI program approved NASA plans to budget $4.8 billion for the program through fiscal

year 2006.

2001 May • Initial contracts awarded to

22 contractors

The contracts, valued at almost $800 million, were not intended to

provide a specific vehicle design, but rather to explore concepts

and technologies that could be used for future government and

commercial launch systems and space transportation operations.

The leap-ahead technologies explored included crew survival

systems, advanced tanks and airframe structures, long-life rocket

engines, and thermal protection systems.

2002 Mar. • Interim Architecture and Technology

Review

Design concepts for the second-generation vehicle narrowed down

from hundreds to 15.

2002 Nov. • Systems Requirements Review This review is to focus attention on fewer space transportation

architectures and technology areas, to select three architectures

that can be pursued, and reach agreement on the development of

system requirements.

2003 Feb. • Request for Proposals (RFP) to

select SLI designs

RFPs are to focus on selecting the most promising architectures to

proceed toward a detailed preliminary design.

2003 Sept. • Contract awards for selection of

two designs

Second phase of formulation program to further develop concepts

and risk-reduction activities.

2006 • SLI full-scale development decision Decision for selecting architecture to enter full-scale development.

2009 • Potential prototype vehicle available Reusable unmanned vehicle with limited capabilities.

2012 • Cargo vehicle available Reusable unmanned vehicle with advanced engines and tanks.

2014 • Crew vehicle available Reusable launch vehicle with crew capabilities

Source: Discussions with and documentation from NASA.

According to a NASA official, NASA plans to define the basic requirements

for its second-generation space transportation—that is, what the crew size

will be, what the payload capacity will be, and what designs or

architectures are worth pursuing—by November 2002. These decisions

will have a significant impact on the cost, size, and design of the new

vehicle. For example, as the payload capacity increases, so does the thrust

requirement for the propulsion system, and in turn, the cost to develop

and build the system, as well as the cost to operate the system.

However, NASA is facing a considerable challenge in reaching the point to

where it can finalize SLI requirements. This is primarily because NASA

must first make some difficult broader decisions regarding the future of

space transportation and other NASA projects, including (1) whether

NASA should skip development of a second-generation vehicle in favor of

concentrating on the third generation, (2) what the future of the

International Space Station will be, and (3) what DOD’s role in developing

the new vehicle will be. Moreover, in making these decisions, NASA will

need to reach consensus with a wide range of partners who have priorities

Important Decisions

to Be Made Before

Requirements Can Be

Defined
Page 8 GAO-02-1020 Space Transportation

and concerns different from NASA’s, and it will need to make trade-offs

amid uncertainties as to the availability of advanced technologies.

First, NASA must complete its ongoing reassessment of its overall space

transportation plan. This evaluation is being done as part of NASA’s

development of a budget proposal for fiscal year 2004. The options NASA

is considering could have a far-reaching impact on the SLI program. One

option, in fact, involves postponing efforts to develop a new generation of

vehicles indefinitely until there is a major breakthrough in technology that

could vastly improve performance and reduce costs. Other options being

examined could drastically change NASA’s timetable and requirements for

SLI. For example, NASA is looking at developing a crewed vehicle more

quickly than currently planned and launching it atop an expendable rocket

until it can field a reusable launch vehicle. NASA is also considering

deferring some development efforts so that it can aggressively pursue a

goal of building a relaunchable vehicle prototype by 2009. Finalizing its

decisions on where to go with the space transportation plan will be

difficult for NASA, given the uncertainties about the availability of

technologies needed to reduce costs and achieve NASA’s performance and

safety objectives.

Second, NASA is currently reassessing the future of the International

Space Station. One decision it needs to make that could significantly affect

the SLI program is whether the station should support a crew of seven

astronauts, as originally planned, or three. Because of cost growth, NASA

plans to cut back to a crew of three, but its international partners have not

agreed yet to this decision and are concerned that the cutback will

severely undermine planned scientific research. This decision could

significantly affect the design and cost of the second-generation vehicle.

For example, with a bigger crew size, the overall vehicle will have to be

larger; a larger crew cabin and additional backup systems will be required;

and, as a result, the cost to develop the vehicle will increase.

NASA is also planning to cut back the number of flights to the station from

seven to four per year, which could also have an impact on SLI. But again,

NASA’s international partners have concerns about this decision, since it

would limit the deliveries of resources needed to carry out research

activities. At the same time that NASA is looking at cutting back on space

station capabilities, it is planning to extend the life of its space shuttle to

2020 with software and hardware upgrades. If NASA can successfully

extend the life of the shuttle, it may well find that a second-generation

vehicle, which is not expected to begin transporting crews to the space

station until at least 2014, would generally be duplicative in capability and
Page 9 GAO-02-1020 Space Transportation

therefore unnecessary. NASA plans to make final decisions on the space

station by November/December 2002.

Third, while DOD and NASA have explored potential common areas of

interest for a new space transportation vehicle, DOD has not yet formally

defined its requirements. In fact, it is still uncertain whether SLI will be a

joint DOD/NASA program. A study conducted by the Air Force and NASA

earlier this year revealed that both NASA and DOD shared similar

objectives when it came to technologies needed for the new vehicle as

well as cargo lift requirements and launch architecture elements. But there

were differences with priorities and certain requirements. The Air Force

would like the vehicle to operate in stronger winds, more precipitation,

and a wider range of temperatures. It would also like the vehicle to

operate from an inland U.S. Air Force base. NASA would like the vehicle

to stay in orbit for a longer duration and have more maneuverability

because of its mission to service the International Space Station.

Additionally, NASA will have a higher weight-delivered-to-orbit

requirement for its crew vehicle.

It is apparent that some of DOD’s objectives and priorities may not match

up with NASA’s. For example, DOD envisions developing an unpiloted

vehicle, while NASA is focusing on a crewed design. DOD also envisions

developing a vehicle that could be capable of relaunching within

12 to 48 hours and making as many as 20 flights in a 2-week time frame. At

this time, NASA does not share these objectives. According to a DOD

official, DOD expects to finalize its requirements before the end of 2002.

Since DOD’s decisions will affect the size, design, and capabilities of the

vehicle, it is important that these requirements be known before NASA

finalizes its own requirements in November 2002. Otherwise, NASA will

need to reexamine its requirements after narrowing down potential system

architectures to three possibilities. While significant cost and operational

benefits may accrue with a vehicle capable of satisfying both DOD’s and

NASA’s requirements, NASA will need to guard against making

compromises that might negatively affect its goal of substantially lowering

launch costs.

NASA cannot implement key management controls for the SLI program

until it defines its basic requirements. These include cost estimates,

controls designed to provide early warnings of cost and schedule

problems, as well as risk mitigation plans. Moreover, NASA does not have

plans to implement some performance measures, including ones that

would assess overall progress toward achieving key objectives, reducing

Key Management

Controls Are Not Yet

Implemented
Page 10 GAO-02-1020 Space Transportation

risks, and maturing technology. It is essential that these controls be

implemented quickly so that NASA can predict what the total costs of the

program will be and provide assurance that its investment in SLI is being

spent wisely.

Undertaking ambitious, technically challenging efforts like SLI—which

involve multiple contractors and technologies that have to be developed

and proven—requires careful oversight and management. Importantly,

accurate and reliable cost estimates need to be developed, technical and

program risks need to be anticipated and mitigated, and performance and

readiness need to be closely monitored. Such undertakings also require a

high level of communication and coordination. Not carefully implementing

such project management tools and activities is a recipe for failure.

Without realistically estimating costs and risks, and providing budgetary

reserves needed to mitigate those risks, management may not be in a

position to effectively deal with the technical problems that cutting-edge

projects invariably face.

In fact, we found that NASA did not successfully implement and adhere to

a number of critical project management tools and activities in its previous

efforts to build new space transportation vehicles. As we testified in June

2001, neither of NASA’s X-33 or X-34 programs—which attempted to build

a new transportation vehicle that would reach orbit in one stage—-fully

assessed the costs associated with developing new, unproven

technologies; provided for the financial reserves needed to deal with

technical risks and accommodate normal development delays; developed

plans to quantify and mitigate risks to NASA; or established performance

targets showing a clear path to an operational launch vehicle. 2 Underlying

these difficulties were problems with the agreements and contracts that

established the relationship between NASA and its industry partners and

the eventual erosion of commercial prospects for the development of new

reusable launch vehicles. We testified that lax management controls led to

numerous problems with both the X-33 and X-34 programs. Technical

problems, for example, resulted in significant schedule and cost overruns,

which NASA was ill prepared to deal with.

 2

 See U.S. General Accounting Office, Space Transportation: Critical Areas NASA Needs

to Address in Managing Its Reusable Launch Vehicle Program, GAO-02-826T

(Washington, D.C.: June 20, 2001).

Importance of

Management Controls for

SLI
Page 11 GAO-02-1020 Space Transportation

NASA has taken steps to avoid the problems it encountered with the X-33

and X-34 programs. In our testimony last year, for example, we pointed out

that NASA planned to increase the level of insight into SLI projects by

providing more formal reviews and varying levels of project

documentation from contractors depending on the risk involved and the

contract value. NASA also required that all proposals submitted in

response to its research announcement be accompanied by certifiable cost

and pricing data. Finally, NASA discouraged the use of cooperative

agreements, since these agreements did not prove to be effective

contracting mechanisms for research and development efforts where large

investments are required.

NASA cannot develop cost estimates for the SLI program until it defines

the basic requirements for the new transportation vehicle and narrows the

field of possible architectures. Moreover, until requirements are defined,

NASA cannot determine whether the $4.8 billion already planned for SLI

through fiscal year 2006 will achieve the desired results for the program’s

formulation. NASA guidance requires that life-cycle costs be estimated,

assessed, and controlled throughout a program’s life cycle.

3 The estimates

are to be prepared to support major program reviews and the development

of budget submissions.

In our recent review of the International Space Station, we reported that

NASA is facing additional challenges to developing reliable cost estimates

and effectively using them to manage programs.

4

 For example, a recent

study performed by the Rand Corporation for the Office of Science and

Technology Policy found that NASA has “very good” cost and risk

modeling capabilities. However, the study also found that NASA’s in-house

capabilities were not well integrated into the program’s planning and

management and that NASA programs had been reluctant to integrate cost

estimate and control practices that were sufficiently robust to yield

confidence in budget estimates. In addition, a task force appointed last

year to conduct an independent external review and assessment of the

space station’s cost, budget, and management found that NASA tends to

take a short-term focus on executing programs—that is, rather than

 3

 See NASA Policy Directive 7120.4A, Program/Project Management and NASA Procedures

and Guidelines 7120.5A, Program and Project Management Processes and Requirements.

4

 See U.S. General Accounting Office, Space Station: Actions Under Way to Manage Cost,

but Significant Challenges Remain, GAO-02-735 (Washington, D.C.: July 17, 2002).

Cost Estimates Cannot Be

Developed Until

Requirements Are Defined
Page 12 GAO-02-1020 Space Transportation

managing programs to come within overall cost and scheduling goals, it

manages them around annual budgets. The task force cited NASA’s culture

of managing the space station program in adherence to its annual budgets

as perhaps the single greatest factor in the space station program’s cost

growth.

We also reported that NASA’s ability to develop good cost estimates for

programs is hampered by NASA’s lack of a modern integrated financial

system to track and maintain data needed for estimating and controlling

costs. NASA has made it a top priority to develop and implement a new

system. However, the first major component of the system—the core

financial system—is not expected to be implemented until June 2003.

It will be important for NASA to overcome barriers that relate to

estimating costs for SLI and to use estimates to take a long-term

perspective in managing the SLI program. Without good cost information,

decision makers at NASA and in the Congress will not know whether the

$4.8 billion investment is sufficient for the early phases of the program or

how much more it will cost to actually develop and deploy the new

vehicle. Moreover, NASA managers will lack the information they need to

monitor costs, schedule, and performance.

Other management controls that are integral to successfully managing the

SLI program cannot be implemented until NASA has a cost estimate. First,

NASA has a system—known as Earned Value Management—intended to

help provide program managers and others with early warnings of cost

and schedule problems. But this system cannot be effectively implemented

without having baseline requirements defined or detailed cost estimates,

schedules, and timelines developed for the SLI program.

Earned Value Management goes beyond the two-dimensional approach of

comparing budgeted costs with actual costs. It also attempts to compare

the value of work accomplished during a given period with the value of

work scheduled for that period. By using the value of work done as a basis

for estimating the cost and time to complete it, the earned value concept

should alert program managers to potential problems sooner than

expenditures alone can. The communities that have a vested interest in

earned value are the (1) program managers, who are charged with overall

management responsibility for acquisition programs; (2) contractors, who

are responsible for the contract’s successful execution; and (3) overseers,

such as acquisition executives, financial managers, contract surveillance

Other Management

Controls Cannot Be

Implemented Without a

Cost Estimate
Page 13 GAO-02-1020 Space Transportation

officials, and cost estimators, who are tasked with tracking and estimating

program costs.

Second, individual SLI activities have prepared risk mitigation plans.

However, a program official told us that these plans cannot be quantified

at the overall program cost and schedule level without complete cost

estimates. Moreover, without a cost estimate, risks can be measured only

on a year-by-year basis and not on a multiyear basis—for example, to 2006.

Risk mitigation plans identify, assess, and document the risks associated

with the cost, resource, schedule, and technical aspects of a project and

determine the procedures that will be used to manage those risks. In doing

so, they help ensure that a system will meet performance requirements

and be delivered on schedule and within budget. NASA’s guidance requires

that these plans be developed during the formulation phase of a project.

Agencies are required to prepare annual performance plans that establish

performance goals with measurable target levels of performance for each

program activity in the agency’s budget and to provide a basis for

comparing actual performance with performance goals.5

 Doing so enables

agencies to gauge the progress of programs like SLI and, in turn, to take

quick action when performance goals are not being met.

Individual SLI activities are using computer simulations to help gauge

whether the technologies under development can meet specific

performance targets. However, NASA has not yet established broader

measures that can show NASA decision makers and the Congress whether

the program as a whole is meeting objectives such as reducing the payload

cost to approximately $1,000 per pound and reducing the risk of crew loss

to approximately 1 in 10,000 missions. Like cost and risk controls, the

development of such metrics depends on NASA’s definition of

requirements and cost estimation for SLI.

It is important that NASA develop such measures as soon as possible. We

testified last year that one problem that hampered the previous X-33 and

X-34 efforts was the fact that NASA had not developed performance

targets that establish a clear path leading to a reusable launch vehicle.

 5

 31 U.S.C. §1115.

Measures to Assess

Performance Not

Implemented
Page 14 GAO-02-1020 Space Transportation

In addition, while NASA plans to assess the readiness of technology to

gauge the maturity of individual technologies, it does not plan to verify and

validate the maturity of technology for the SLI program as a whole. To

ensure that individual technologies are sufficiently mature by NASA’s

planned full-scale development decision for 2006, NASA intends to use

technology readiness levels (TRLs). Our prior reports have shown that

TRLs, which were pioneered by NASA, are a good way to gauge the

maturity of technologies. Readiness levels are measured along a scale of

1 to 9, starting with paper studies of the basic concept, proceeding with

laboratory demonstrations, and ending with a technology that has proven

itself on the intended product. NASA would like to achieve a TRL of

6 by 2006 for key technologies, such as the propulsion system. At this

level, a prototype is tested in a relevant environment, such as a highfidelity laboratory environment or in a simulated operational environment.

Currently, most of the technology areas are at levels 3 or 4. At level 3,

analytical studies and laboratory studies are performed to physically

validate analytical predictions of separate elements of the technology. At

level 4, basic technological components are integrated to establish that the

pieces will work together.

While assessing technologies separately should help NASA decide when

and where to insert new technologies into the SLI program, it is still

important for NASA to look at the readiness of the product as a whole

because how well various components being developed will work together

is unknown.

Another measure that NASA officials told us they do not plan to

implement is one that would assess the extent or percentage of total risks

that have been reduced with NASA’s initial $4.8 billion investment as well

as the amount of risk remaining. Program officials told us that it would be

too resource-intensive to develop this measure. However, in the absence

of such information, decision makers at NASA and in the Congress have

little assurance that the $4.8 billion investment in SLI can fulfill the goals

expected of the program, and they have reduced confidence that what has

already been spent has placed NASA on track to meeting its primary goals.

NASA aims to be able to define system requirements for SLI by November

2002. But meeting this goal may not be realistic. NASA must first decide

whether developing a second-generation vehicle to be deployed in 2014 to

2015 is still a worthwhile endeavor, given plans to extend the life of the

space shuttle and cut back on the space station’s capabilities, and if so,

what specific direction the program should take and how it will fit in with

Conclusions
Page 15 GAO-02-1020 Space Transportation

DOD efforts. Making such decisions within the short time remaining

before the SLI systems requirements review will be difficult because it will

require NASA to (1) resolve differences with its space station partners,

who have concerns about planned cutbacks to the station; (2) reach

consensus with DOD on requirements and priorities, which are now

considerably different; and (3) make trade-offs as to what capabilities and

technologies it should pursue without really knowing when advancements

can be achieved. When NASA is able to resolve these challenges and

finalize requirements for SLI, it will be critical for NASA to swiftly

implement effective management to oversee the effort. Until it does so,

NASA will not be able to assure its managers and the Congress that the

initial investment is being spent wisely and that risks are being reduced,

and it will not be able to predict what the total costs of the program will

be.

We recommend that the NASA Administrator do the following:

1. Reassess the schedule for defining the requirements for the Space

Launch Initiative in order to ensure that the agency takes the following

actions before making final decisions on basic requirements and

selecting three architectures to pursue: (1) complete the reassessment

of NASA’s integrated space transportation plan, (2) reach consensus

with its international partners on the future of the space station, and

(3) reach consensus with the Department of Defense on its role in the

SLI effort.

2. If DOD is to jointly develop the second-generation vehicle, reach

consensus with DOD on priorities and objectives for SLI and factor

DOD’s requirements into NASA’s own.

3. After NASA completes its system requirements review, ensure that a

cost estimate is promptly developed for the SLI investment and that

this estimate is detailed and reliable enough to be used to complete

risk mitigation plans and carry out earned value management

activities.

4. Ensure that NASA can demonstrate how the $4.8 billion initial

investment supports the requirements that NASA decides to pursue in

November 2002.

5. After system requirements are defined, ensure that performance

measures are developed to assess the progress of the program, as a

Recommendations
Page 16 GAO-02-1020 Space Transportation

whole, toward (1) meeting key performance objectives, including

lowering the cost of delivering payloads to low-Earth orbit to less than

$1,000 per pound and reducing the risk of crew loss to approximately

1 in 10,000 missions, and (2) achieving an overall technology readiness

level of 6 by 2006.

6. Ensure that measures are developed to assess the amount of risk

reduced following each year of expenditure and the amount of risk

remaining.

In written comments on a draft of this report, NASA’s Associate Deputy

Administrator said the agency concurs with the recommendations, adding

that the program completed its first year as planned and within budget.

According to the Associate Deputy Administrator, the agency is

reassessing the Integrated Space Transportation Plan and working with

DOD to develop their requirements. Furthermore, NASA concurs with the

recommendation concerning the development of a cost estimate.

However, the agency notes that the RLV industry suffers uncertainties in

predicting costs, primarily because of a limited set of models and data to

validate those models. NASA also concurred with the need to validate SLI

requirements and the need for performance measures to assess the

progress of the program towards meeting objectives and achieving

technology readiness levels.

Our draft report contained a recommendation that NASA develop

measures to assess the percentage of risk reduced following each year of

expenditures and the percentage of risk remaining. NASA believes that its

process of narrowing potential SLI technology alternatives to the most

promising concepts meriting further funding enables the successful

candidates to be built and become operational. In that way, the SLI

program is reducing the risks inherent in an advanced research and

technology program of this magnitude. According to NASA, it will develop

a reporting mechanism that communicates the amount of risk reduction

achieved and remaining on an annual basis. Our intent is for NASA to

provide a quantifiable means of measuring progress. Thus, we modified

that recommendation accordingly.

Agency Comments
Page 17 GAO-02-1020 Space Transportation

To assess the coordination of requirements with DOD and other NASA

programs, we interviewed officials within the Department of Defense and

NASA. We evaluated studies conducted by DOD and NASA on efforts to

address requirements and coordination.

To address NASA’s management controls for the SLI program, we

interviewed NASA officials regarding cost estimates and the process by

which cost information is studied and communicated throughout NASA.

We reviewed NASA’s and the Office of Management and Budget’s guidance

regarding earned value management and discussed the program’s methods

for coordinating activities. We also reviewed risk mitigation plans to

determine how cost affects these plans and discussed these plans with

program officials.

To assess program performance measures, we reviewed NASA’s policies

and procedures governing program management and the program’s plans

for assessing performance measures. We also interviewed SLI program

officials to understand the program’s plans to meet goals and objectives.

To accomplish our work, we interviewed officials from NASA’s

headquarters, Washington, D.C.; Marshall Space Flight Center, Alabama;

Johnson Space Center, Texas; Kennedy Space Center, Florida; and

Independent Program Assessment Office, Virginia; and DOD’s Space

Command, Colorado.

We performed our review from October 2001 through August 2002 in

accordance with generally accepted government auditing standards.

Unless you publicly announce its contents earlier, we plan no further

distribution of this report until 30 days from its issue date. At that time,

we will send copies to the NASA Administrator; Director, Office of

Management and Budget; and other interested parties. We will also make

copies available to others upon request. In addition, the report will be

available at no charge on the GAO Web site at http://www.gao.gov.

Scope and

Methodology
Page 18 GAO-02-1020 Space Transportation

Please contact me at (202) 512-4841 if you or your staffs have any

questions about this report. Major contributors to this report are listed in

appendix II.

Allen Li

Director

Acquisition and Sourcing Management
Appendix I: Comments from the National

Aeronautics and Space Administration

Page 19 GAO-02-1020 Space Transportation

Appendix I: Comments from the National

Aeronautics and Space Administration
Appendix I: Comments from the National

Aeronautics and Space Administration

Page 20 GAO-02-1020 Space Transportation
Appendix I: Comments from the National

Aeronautics and Space Administration

Page 21 GAO-02-1020 Space Transportation
Appendix II: GAO Contact and

Staff Acknowledgments

Page 22 GAO-02-1020 Space Transportation

Jerry Herley (202) 512-7609

In addition to the person named above, Cristina Chaplain, Ivy Hubler,

Danny Owens, and Dana Solomon made key contributions to this report.

Appendix II: GAO Contact and Staff

Acknowledgments

GAO Contact

Acknowledgments

(120101)
The General Accounting Office, the investigative arm of Congress, exists to

support Congress in meeting its constitutional responsibilities and to help

improve the performance and accountability of the federal government for the

American people. GAO examines the use of public funds; evaluates federal

programs and policies; and provides analyses, recommendations, and other

assistance to help Congress make informed oversight, policy, and funding

decisions. GAO’s commitment to good government is reflected in its core values

of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no cost is

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older

products. The Web site features a search engine to help you locate documents

using key words and phrases. You can print these documents in their entirety,

including charts and other graphics.

Each day, GAO issues a list of newly released reports, testimony, and

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site

daily. The list contains links to the full-text document files. To have GAO e-mail

this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily

E-mail alert for newly released products” under the GAO Reports heading.

The first copy of each printed report is free. Additional copies are $2 each. A

check or money order should be made out to the Superintendent of Documents.

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a

single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

441 G Street NW, Room LM

Washington, D.C. 20548

To order by Phone: Voice: (202) 512-6000

TDD: (202) 512-2537

Fax: (202) 512-6061

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G Street NW, Room 7149

Washington, D.C. 20548

GAO’s Mission

Obtaining Copies of

GAO Reports and

Testimony

Order by Mail or Phone

To Report Fraud,

Waste, and Abuse in

Federal Programs

Public Affairs