Testimony

Before the Committee on Science, Subcommittee on

Space and Aeronautics, House of Representatives

United States General Accounting Office

GAO

For Release on Delivery

Expected at 2:00 p.m. EDT

Wednesday

June 20, 2001

SPACE

TRANSPORTATION

Critical Areas NASA

Needs to Address in

Managing Its Reusable

Launch Vehicle

Program

Statement of Allen Li, Director, Acquisition and Sourcing

Management

GAO-01-826T
Page 1 GAO-01-826T

Mr. Chairman and Members of the Subcommittee:

I am pleased to be here today to discuss our work, requested by this

Subcommittee, on the National Aeronautics and Space Administration’s

(NASA) X-33 and X-34 programs. As you know, the purpose of these

efforts was to significantly reduce the cost of access to space by

partnering with private industry to develop and demonstrate technologies

needed for future reusable launch vehicles reaching orbit in one stage

(single-stage-to-orbit). In essence, these are vehicles whose components—

either all or in part—can be utilized on subsequent flights. Both programs

were recently terminated because of significant cost increases caused by

problems developing the necessary technologies and flight demonstration

vehicles. NASA is now focusing instead on its new Space Launch Initiative.

This is a broader effort to develop the next generation of reusable launch

vehicles, referred to as the Second Generation Reusable Launch Vehicle

Program (2nd Generation Program). Today, I will discuss the primary

factors that contributed to the difficulties experienced by the X-33 and X34 programs and the steps needed to avoid repeating those problems

within the 2nd Generation Program.

In brief, the X-33 and X-34 programs experienced difficulties achieving

their goals primarily because NASA did not develop realistic cost

estimates, timely acquisition and risk management plans, and adequate

and realistic performance goals. In particular, neither program fully

assessed the costs associated with developing new, unproven

technologies; provided for the financial reserves needed to deal with

technical risks and accommodate normal development delays; developed

plans to quantify and mitigate the risks to NASA; or established

performance targets showing a clear path leading to an operational

reusable launch vehicle. Underlying these difficulties were problems with

the agreements and contracts that established the relationship between

NASA and its industry partners and eventual erosion of commercial

prospects for the development of new reusable launch vehicles.

Currently, NASA is in the process of taking steps in the 2nd Generation

Program to help avoid problems like those encountered in the X-33 and X34 programs. While it is too early to tell if these measures will be

sufficient, our review of the two programs has shown that three critical

areas need to be addressed. First, the technical complexity involved

requires that realistic cost estimates and risk mitigation plans are

developed and the projects are funded accordingly. Second, because the

2nd Generation Program will involve numerous interrelated, complex

efforts to develop technology, NASA needs to ensure that all these efforts
Page 2 GAO-01-826T

move forward in a coordinated manner and that open communication is

fostered at all levels. Third, performance measures need to be

implemented and periodically validated to ensure that the rationale for

developing specific technology applications merits continued support.

The X-33 and X-34 programs were part of an effort that began in 1994—

known as the Reusable Launch Vehicle Technology/Demonstrator

Program (Reusable Launch Vehicle Program)—to pave the way to fullscale, commercially-developed, reusable launch vehicles reaching orbit in

one stage. In embarking on the Reusable Launch Vehicle Program, NASA

sought to significantly reduce the cost of developing, producing and

operating launch vehicles. NASA’s goal was to reduce payload launch

costs from $10,000 per pound on the space shuttle to $1,000 per pound. It

planned to do so, in part, by finding “new ways of doing business” such as

using innovative design methods, streamlined acquisition procedures, and

creating industry-led partnerships with cost sharing to manage the

development of advanced technology demonstration vehicles. The

vehicles were seen as the “stepping stones” in what NASA described as an

incremental flight demonstration program. The strategy was to force

technologies from the laboratory into the operating environment.

The X-34 Project started in 1995 as a cooperative agreement between

NASA and Orbital Sciences Corporation1

 (Orbital). The project was to

demonstrate streamlined management and procurement, industry cost

sharing and lead management, and the economics of reusability. However,

the industry team withdrew from the agreement in less than 1 year, for a

number of reasons including changes in the projected profitability of the

venture. NASA subsequently started a new X-34 program with a smaller

vehicle design. It was intended only as a flight demonstration vehicle to

test some of the key features of reusable launch vehicle operations, such

as quick turn-around times between launches.

Under the new program, NASA again selected Orbital as its contractor in

August 1996, awarding it a fixed price, $49.5 million contract. Under the

new contract, Orbital was given lead responsibility for vehicle design,

fabrication, integration, and initial flight testing for powered flight of the

X-34 test vehicle. The contract also provided for two options, which were

 1

Rockwell International was an industry partner with Orbital Science Corporation in the

development of the commercial reusable vehicle.

Background
Page 3 GAO-01-826T

later exercised, totaling about $17 million for 25 additional experimental

flights and, according to a project official, other tasks, including defining

how the flight tests would be undertaken. Under the new effort, NASA’s

Marshall Space Flight Center was to develop the engine for the X-34 as

part of its Low Cost Booster Technology Project. The initial budget for this

development was about $18.9 million.

In July 1996, NASA and Lockheed Martin Corporation and its industry

partners2

 entered into a cooperative agreement for the design,

development, and flight-testing of the X-33. 3

 The X-33 was to be an

unmanned technology demonstrator. It would take off vertically like a

rocket, reaching an altitude of up to 60 miles and speeds to about Mach 13

(13 times the speed of sound), and land horizontally like an airplane. The

X-33 would flight test a range of technologies needed for future launch

vehicles, such as thermal protection systems, advanced engine design and

lightweight fuel tanks made of composite materials. The vehicle would not

actually achieve orbit, but based on the results of demonstrating the new

technologies, NASA envisioned being in a better position to make a

decision on the feasibility and affordability of building a full-scale system.

Under the initial terms of the cooperative agreement, NASA’s contribution

was fixed at $912.4 million and its industry partners’ initial contribution

was $211.6 million. In view of the potential commercial viability of the

launch vehicle and its technologies, the industry partners also agreed to

finance any additional costs.4

 During a test in November 1999, one of the

fuel tanks failed due to separation of the composite surface. Following the

investigation, NASA and Lockheed Martin agreed to replace the composite

tanks with aluminum tanks.

In February 2001, NASA announced it would not provide any additional

funding for the X-33 or X-34 programs under its new Space Launch

Initiative. The Space Launch Initiative is intended to be a more

comprehensive, long-range plan to reduce high payload launch costs.

NASA’s goal is still to reduce payload launch cost to $1,000 per pound to

low Earth orbit but it is not limited to single-stage-to-orbit concepts.

 2

Lockheed Martin made agreements with Allied Signal Aerospace, B.F. Goodrich

Aerospace, Boeing-Rocketdyne Division, and Sverdrup Corporation for the X-33 Program.

3

NASA’s use of a cooperative agreement allowed the industry partners, and NASA, to

withdraw from the agreement without penalty at any time.

4

We reported on X-33 costs in Space Transportation: Status of the X-33 Reusable Launch

Vehicle Program (GAO/NSIAD-99-176, Aug. 11, 1999).
Page 4 GAO-01-826T

Specifically, the 2nd Generation Program’s objective is to substantially

reduce the technical, programmatic, and business risks associated with

developing reusable space transportation systems that are safe, reliable

and affordable.

NASA has budgeted about $900 million for the SLI initial effort and, in May

2001, it awarded initial contracts to 22 large and small companies for

space transportation system design requirements, technology risk

reduction, and flight demonstration. In subsequent procurements in midfiscal year 2003, NASA plans to select at least two competing reusable

launch system designs. The following 2.5 to 3.5 years (through fiscal years

2005 or 2006) will be spent finalizing the preliminary designs of the

selected space transportation systems, and maturing the specific

technologies associated with those high-risk, high-priority items needed to

develop the selected launch systems.

Undertaking ambitious, technically challenging efforts like the X-33 and X34 programs—which involve multiple contractors and technologies that

have yet to be developed and proven—requires careful oversight and

management. Importantly, accurate and reliable cost estimates need to be

developed, technical and program risks need to be anticipated and

mitigated, sound configuration controls need to be in place, and

performance needs to be closely monitored. Such undertakings also

require a high level of communication and coordination. Not carefully

implementing such project management tools and activities is a recipe for

failure. Without realistically estimating costs and risks, and providing the

reserves needed to mitigate those risks, management may not be in a

position to effectively deal with the technical problems that cutting-edge

projects invariably face.

In fact, we found that NASA did not successfully implement and adhere to

a number of critical project management tools and activities. Specifically:

• NASA did not develop realistic cost estimates in the early stages of the X33 program. From its inception, NASA officials considered the program to

be high risk, with a success-oriented schedule that did not allow for major

delays. Nevertheless, in September 1999, NASA’s Office of the Inspector

General (OIG) reported5

 that NASA’s cost estimate did not include a risk

 5

Audit Report: X-33 Cost Estimating Processes, NASA Office of Inspector General, IG-99-

052,

Factors Contributing

to X-33 and X-34

Program Difficulties
Page 5 GAO-01-826T

analysis to quantify technical and schedule uncertainties. Instead, the cost

estimate assumed that needed technology would be available on schedule

and as planned. According to the OIG, a risk analysis would have alerted

NASA decision-makers to the probability of cost overruns in the program.

Since NASA’s contribution to the program was fixed—with Lockheed

Martin and its industry partners responsible for costs exceeding the initial

$1.1 billion—X-33 program management concluded that there was no risk

of additional government financial contributions due to cost overruns.

They also believed that the projected growth in the launch market and the

advantages of a commercial reusable launch vehicle would provide the

necessary incentive to sustain industry contributions.

• NASA did not prepare risk management plans for both the X-33 and X-34

programs until several years after the projects were implemented. Risk

management plans identify, assess, and document risks associated with

cost, resource, schedule, and technical aspects of a project and determine

the procedures that will be used to manage those risks. In doing so, they

help ensure that a system will meet performance requirements and be

delivered on schedule and within budget. A risk management plan for the

X-34 was not developed until the program was restructured in June 2000.

Although Lockheed Martin developed a plan to manage technical risks as

part of its 1996 cooperative agreement for the X-33, NASA did not develop

its own risk management plan for unique NASA risks until February 2000.

The NASA Administrator and the NASA Advisory Council have both

commented on the need for risk plans when NASA users partnering

arrangements such as a cooperative agreement. Furthermore, we found

that NASA’s risk mitigation plan for the X-33 program provided no

mechanisms for ensuring the completion of the program if significant cost

growth occurred and/or the business case motivating industry

participation weakened substantially.

• Against its own policy, NASA did not prepare program commitment

agreements or program plans at the onset for either program. The

commitment agreement lays out the program’s technical, schedule, and

cost commitments, and overall acquisition strategy. The program plan

addresses these issues as well but also defines the effort’s management

structure as well as program resources, data management, risk

management, test and verification, and planned program reviews. Such

plans would help NASA to define realistic time frames, identify

 Sept. 24, 1999.
Page 6 GAO-01-826T

responsibility for key tasks and deliverables and provide a yardstick by

which to measure the progress of the effort.

• According to the OIG, NASA did not complete a configuration

management plan for the X-33 until May 1998—about 2 years after NASA

awarded the cooperative agreement and Lockheed Martin began the

design and development of a flight demonstration vehicle. Configuration

management plans define the process to be used for defining the

functional and physical characteristics of a product and systematically

controlling changes in the design. As such, they enable organizations to

establish and maintain the integrity of a product throughout its lifecycle

and prevent the production and use of inconsistent product versions. By

the time the plan was implemented, hardware for the demonstration

vehicle was already being fabricated.

• Communications and coordination were not effectively facilitated. In a

report following the failure of the X-33’s composite fuel tank, the

investigation team reported that the design of the tank required high levels

of communication, and that such communication did not occur in this

case.6

 A NASA official told us that some NASA and Lockheed personnel,

who had experience with composite materials and the phenomena

identified as one of the probable causes for the tank’s failure, expressed

concerns about the tank design. However, because of the industry-led

nature of the cooperative agreement, Lockheed Martin was not required to

react to such concerns and did not request additional assistance from

NASA.

• The Government Performance and Results Act of 1993 requires federal

agencies to prepare annual performance plans to establish measurable

objectives and performance targets for major programs. Doing so enables

agencies to gauge the progress of programs like the X-33 and X-34 and in

turn to take quick action when performance goals are not being met. For

example, we reported in August 1999 that NASA’s Fiscal Year 2000

Performance Plan did not include performance targets that established a

clear path leading to a reusable launch vehicle and recommended such

targets be established.

 6

Final Report of the X-33 Liquid Hydrogen Tank Test Investigation Team. National

Aeronautics and Space Administration, George C. Marshall Space Flight Center, Huntsville,

Alabama 35812, May

2000.
Page 7 GAO-01-826T

Without relying on these important project management tools up front,

NASA encountered numerous problems on both the X-33 and X-34

programs. Compounding these difficulties was a decrease in the projected

commercial launch market, which in turn lessened the incentive of NASA’s

X-33 industry partners to continue their investments.

In particular, technical problems in developing the X-33’s composite fuel

tanks, aerospike engines, heat shield, and avionics system resulted in

significant schedule delays and cost overruns. After two program reviews

in 1998 and 1999, the industry partners added a total of $145.6 million to

the cooperative agreement to pay for cost overruns and establish a reserve

to deal with future technical problems and schedule delays. However,

NASA officials stated that they did not independently develop their own

cost estimates for these program events to determine whether the

additional funds provided by industry would be sufficient to complete the

program. Also, these technical problems resulted in the planned first flight

being delayed until October 2003, about 4.5 years after the original March

1999 first flight date.

After the composite fuel tank failed during testing in November 1999,

according to NASA officials, Lockheed Martin opted not to go forward

with the X-33 Program without additional NASA financial support.

Lockheed Martin initially proposed adding $95 million of its own funds to

develop a new aluminum tank for the hydrogen fuel, but also requested

about $200 million from NASA to help complete the program. Such

contributions would have increased the value of the cooperative

agreement to about $1.6 billion or about 45 percent (about $500 million)

more than the $1.1 billion initial cooperative agreement funding. NASA did

not have the reserves available to cover such an increase. The agency did,

however, allow Lockheed Martin to compete, in its 2nd Generation Program

solicitation for the additional funds Lockheed Martin believed it needed to

complete the program.

Similarly, NASA started the X-34 Project, and the related NASA engine

development project, with limited government funding, an accelerated

development schedule, and insufficient reserves to reduce development

risks and ensure a successful test program. Based on a NASA X-34

restructure plan in June 2000, we estimate that NASA’s total funding

requirements for the X-34 would have increased to about $348 million—a

307-percent ($263 million) increase from the estimated $86 million

budgeted for the vehicle and engine development projects in 1996. Also,

since 1996, the projected first powered flight had slipped about 4 years
Page 8 GAO-01-826T

from September 1998 to October 2002 due to the cumulative effect of

added risk mitigation tasks, vehicle and engine development problems,

and testing delays.

Most of the cost increase (about $213 million) was for NASA-directed risk

mitigation tasks initiated after both projects started. For example, in

response to several project technical reviews and internal assessments of

other NASA programs,7 the agency developed a restructure plan for the X34 project in June 2000. This plan included consolidating the vehicle and

engine projects under one NASA manager. The project would be managed

with the NASA project manager having the final decision-making

authority; Orbital would be relegated to a more traditional subordinate

contractor role. Under the plan, the contract with Orbital would also be

rescoped to include only unpowered flights; Orbital would have to

compete for 2nd Generation Program funding for all the powered flight

tests. The plan’s additional risk mitigation activities would have increased

the X-34 project’s funding requirements by an additional $139 million,

which included about $45 million for additional engine testing and

hardware; $33 million for an avionics redesign; $42 million for additional

project management support and personnel; and $18 million to create a

contingency reserve for future risk mitigation efforts.

NASA is revising its acquisition and management approach for the 2nd

Generation Program. Projects funded under the program will be NASA-led

rather than industry-led. NASA also plans to increase the level of insight

into the program’s projects, for example, by providing more formal

reviews and varying levels of project documentation from contractors

depending on the risk involved and the contract value. NASA also required

that all proposals submitted in response to its research announcement be

accompanied by certifiable cost and pricing data. Finally, NASA

discouraged the use of cooperative agreements since these agreements did

not prove to be effective contract vehicles for research and development

efforts where large investments are required.

While it is too early to tell if the agency measures aimed at avoiding the

problems experienced in the X-33 and X-34 programs will be sufficient,

 7

These assessments include NASA internal reports on failures in the Mars Program, Shuttle

wiring problems and an assessment of NASA’s approach to executing “Faster, Better,

Cheaper.”

Incorporating Lessons

Learned in 2nd

Generation Program
Page 9 GAO-01-826T

these experiences show that three critical areas need to be addressed.

These relate to (1) adequate project funding and cost risk provisions, (2)

the effective and efficient coordination and communication required by

many individual but related efforts, and (3) periodically revalidating

underlying assumptions by measuring progress toward achieving a new

safe, affordable space transportation system that meets NASA’s

requirements.

First, the technical complexity of the 2nd Generation Program requires that

NASA develop realistic cost estimates and risk mitigation plans, and

accordingly set aside enough funds to cover the program’s many projects.

NASA plans to invest substantially more funds in the 2nd Generation

Program than it did in the previous Reusable Launch Vehicle Program, and

plans to provide reserves for mitigating program risk. For example, the

agency plans to spend about $3.3 billion over 6 years to define system

requirements for competing space transportation systems and related risk

reduction activities. Most of this amount, about $3.1 billion, is for riskreduction activities, such as the development of new lightweight

composite structures, durable thermal protection systems, and new high

performance engine components.

NASA officials told us that an important way they plan to mitigate risk is

by ensuring adequate management reserves in the 15- to 20-percent range,

or higher if needed. They also acknowledged the need for adequate

program cost estimates on which to base reserve requirements. However,

we are still concerned about the timely preparation of cost estimates. The

2nd Generation deputy program manager stated that, based on the scope of

the first contracts awarded, the program office planned to update their

cost estimate this summer before NASA conducted a separate,

independent technical review and cost estimate in September 2001. Thus,

neither of these important analyses were completed prior to the first

contract awards. We believe that until the program office completes it own

updated cost estimate and NASA conducts an independent cost and

technical review, a credible estimate of total program costs and the

adequacy of planned reserves will not be available. Also, NASA is still in

the process of developing the documentation required for the program,

including a risk mitigation plan. NASA policy requires that key program

documentation be finalized and approved prior to implementing a

program.

Second, NASA will face coordination and communication challenges in

managing the 2nd Generation Program. As noted earlier, NASA recently

awarded initial contracts for systems engineering and various risk
Page 10 GAO-01-826T

reduction efforts to 22 different contractors. Yet to successfully carry out

the program NASA must, early on, have coordinated insight into all of the

space transportation architectures8 being proposed by these contractors

and their related risk reduction activities. Clearly, this will be a significant

challenge. The contractors proposing overall architecture designs must be

aware of all the related risk reduction development activities affecting

their respective designs. It may also prove difficult for contractors

proposing space transportation system designs to coordinate work with

other contractors without a prime contractor to subcontractor

relationship. NASA’s own Aerospace Technology Advisory Committee,

made up of outside experts, has also expressed serious concerns about the

difficulty of integrating these efforts effectively.

The need for improvement in coordination and communications in all

NASA programs has been noted in the past and is not unique to the X-33

and X-34 programs. We and other NASA investigative teams have found

and noted similar problems with other NASA programs such as the

Propulsion Module for the International Space Station, and several other

projects including the two failed Mars missions. NASA’s Space Launch

Initiative Program would benefit from lessons learned from past mishaps.

At the request of the House Science Committee, we are undertaking a

review of NASA’s lessons learned process and procedures. The principal

objectives of this review are to determine (1) how NASA captures and

disseminates lessons learned and (2) if NASA is effectively applying

lessons learned toward current programs and projects. We will report the

results of our evaluation in December of this year.

The third challenge is establishing performance measures that can

accurately gauge the progress being made by NASA and its contractors.

NASA officials told us that they plan to periodically reassess the

assumptions underlying key program objectives to ensure that the

rationale for developing specific technology applications merits continued

support. They also told us that they were in the process of establishing

such metrics to measure performance. Ensuring that the results from the

2nd Generation Program will support a future decision to develop reusable

launch vehicles also deserves attention in NASA’s annual Performance

Plan. The plan would be strengthened by recognizing the importance of

 8

A space transportation architecture is defined as an Earth-to-orbit launch vehicle, on-orbit

transfer vehicles and upper stages, mission planning, ground and flight operations, and

support infrastructure.
Page 11 GAO-01-826T

clearly defined indicators which demonstrate that NASA is (1) on a path

leading to an operational reusable launch vehicle and (2) making progress

toward its objective of significantly reducing launch costs, and increasing

safety and reliability compared to existing systems. Affected NASA

Enterprise and Center performance plans would also be strengthened with

the development of related metrics.

Mr. Chairman, this concludes my statement. I would be happy to answer

any questions you or other Members of the Subcommittee may have.

We interviewed officials at NASA headquarters in Washington D.C.,

NASA’s Marshall Space Flight Center, Huntsville, Alabama, and at the

NASA X-33 program office at Palmdale, California to (1) determine the

primary program management factors that contributed to the difficulties

experienced in the X-33 and X-34 programs, and (2) to identify steps that

need to be taken to avoid repeating those problems within the Space

Launch Initiative framework. We also talked to representatives of NASA’s

Independent Program Assessment Office located at the Langley Research

Center, Hampton, Virginia and the OIG located at NASA headquarters and

Marshall Space Flight Center. At these various locations we obtained and

analyzed key program, contractual and procurement documentation for

the X-33, X-34 and 2nd Generation programs. Further, we reviewed reports

issued by the NASA’s OIG and Independent Program Assessment Office

pertaining to the management and execution of the X-33 and X-34

programs, and NASA Advisory Council minutes regarding NASA’s efforts

to develop reusable launch vehicles. In addition, we reviewed other NASA

internal reports documenting management issues associated with program

formulation and implementation of other NASA programs. We also

reviewed applicable NASA policy regarding how NASA expects its

programs and projects to be implemented and managed.

We conducted our review from August 2000 to June 2001 in accordance

with generally accepted government auditing standards.

Objectives, Scope,

And Methodology
Page 12 GAO-01-826T

For information about this testimony, please contact Allen Li at (202) 512-

4841. Contributors to this testimony included Jerry Herley, Noel Lance,

and Carlos Garcia.

(707549)