Report to the Chairman, Subcommittee

on National Security, Veterans' Affairs,

and International Relations, Committee

on Government Reform, House of

Representatives

United States General Accounting Office

GAO

October 2001 JOINT STRIKE

FIGHTER

ACQUISITION

Mature Critical

Technologies Needed

to Reduce Risks

GAO-02-39
Page i GAO-02-39 Joint Strike Fighter Acquisition

Letter 1

Results in Brief 2

Background 4

Technology Readiness Assessments Provide Opportunities to

Improve Outcomes 6

Critical Technologies Are Not Projected to Be at Low Risk 8

Conclusions 11

Recommendations for Executive Action 11

Agency Comments and Our Evaluation 12

Scope and Methodology 13

Appendix I Technology Readiness Levels and Their Descriptions 15

Appendix II Critical Technologies and Their Descriptions 16

Appendix III Comments From the Department of Defense 17

Table

Table 1: Military Services’ Planned Use for the Joint Strike Fighter 4

Figures

Figure 1: Boeing and Lockheed Martin Joint Strike Fighter Aircraft

Design Concepts 6

Figure 2: Actual and Projected Joint Strike Fighter Critical

Technology Readiness Levels 9

Abbreviations

DOD Department of Defense

TRL technology readiness level

Contents
Page 1 GAO-02-39 Joint Strike Fighter Acquisition

October 19, 2001

The Honorable Christopher Shays

Chairman, Subcommittee on National Security,

 Veterans’ Affairs, and International Relations

Committee on Government Reform

House of Representatives

Dear Mr. Chairman:

The Joint Strike Fighter Program is the most expensive aircraft program in

the Department of Defense (DOD). It is intended to produce affordable,

next-generation aircraft to replace aging aircraft in the military services’

inventories. As currently planned, the program will cost about $200 billion

to develop and procure about 3,000 aircraft and related support

equipment. Two contractor teams, led by the Boeing Company and by the

Lockheed Martin Aeronautics Company, are competing for the engineering

and manufacturing development phase. This phase of the program is

projected to last about 8 years and cost about $20 billion and typically

involves large, fixed investments in human capital, facilities, and materials.

Last year, we testified and reported that a key objective of the program’s

acquisition strategy is affordability and that a part of that strategy—

entering into engineering and manufacturing development with low

technical risk—would not be achieved because technologies critical to

meeting the program’s cost and requirement objectives were projected to

be at low levels of technical maturity in April 2001, the date then

scheduled for awarding the engineering and manufacturing development

contract.1

 We stated that the program’s approach was not consistent with

best practices in which technologies are more fully developed before

proceeding into product development. Organizations that use best

practices recognize that delaying the resolution of technology problems

until product development—analogous to the engineering and

manufacturing development phase—can result in at least a ten-fold cost

 1

Joint Strike Fighter Acquisition: Development Schedule Should Be Changed to Reduce

Risks (GAO/T-NSIAD-00-132, Mar. 16, 2000); Joint Strike Fighter Acquisition:

Development Schedule Should Be Changed to Reduce Risks (GAO/NSIAD-00-74, May 9,

2000); and Defense Acquisitions: Decisions on the Joint Strike Fighter Will Be Critical

for Acquisition Reform (GAO/T-NSIAD-00-173, May 10, 2000).

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-02-39 Joint Strike Fighter Acquisition

increase; delaying the resolution until after the start of production could

increase costs by a hundred-fold.

Because of concerns about the adequacy of the Joint Strike Fighter’s short

take-off and vertical landing flight test program, the maturity of its critical

technologies, and other factors, the Fiscal Year 2001 National Defense

Authorization Act directed that the contract for the aircraft’s engineering

and manufacturing development not be awarded until certain criteria were

met.2

 For example, the act required that the program’s short take-off and

vertical landing demonstration aircraft accumulate at least 20 hours of

flight test time before the program could enter the engineering and

manufacturing development phase. The engineering and manufacturing

development contract award is now scheduled for October 2001.

At your request, we assessed whether the Joint Strike Fighter’s critical

technologies will have matured to a low technical risk at the start of the

upcoming engineering and manufacturing development phase.3

 We

assessed technical maturity based on technology readiness levels—

measures pioneered by the National Aeronautics and Space

Administration and adopted by the Air Force Research Laboratory to

determine the readiness of technologies to be incorporated into a weapon

or other type of system.4

 The research laboratory considers a technology

to be low risk for the engineering and manufacturing development stage

when a prototype of that technology has been developed that includes all

of its critical components in approximately the same size, weight, and

configuration of the end product and that prototype has been

demonstrated to work in an environment similar to the planned

operational system.

Although the Joint Strike Fighter program has made good progress in

some technology areas, the program is at risk of not meeting its

affordability objective because critical technologies are not projected to

 2

Section 212, P.L. 106-398, Oct. 30, 2000.

3

A technology is considered to be mature when it has been developed to a point that it can

be readily integrated into a new product and counted on to meet product requirements.

4

The Air Force Research Laboratory is a science and technology organization that matures

advanced technologies to the point that they can be included in weapon system programs

and be expected to perform as required.

Results in Brief
Page 3 GAO-02-39 Joint Strike Fighter Acquisition

be matured to levels that we believe would indicate a low risk program at

the planned start of engineering and manufacturing development in

October 2001. In fact, the critical technologies are projected to be at the

same level of maturity in October 2001 as they were projected to be in

April 2001, the previous scheduled date for the engineering and

manufacturing development contract. Several of these technologies under

development are critical to making the long-term ownership costs of the

program affordable. These technologies remain at higher risk levels for

engineering and manufacturing development because (1) they have not

been developed to approximately the same size, weight, and configuration

of the end product and/or (2) they have not been demonstrated to work in

an environment similar to the planned operational system. For example,

the contractors have demonstrated certain key component technologies in

flight environments, but the tested hardware was not always the same size

and weight required for the Joint Strike Fighter aircraft. In other instances,

components built to the required size and weight were only demonstrated

in ground-tests.5

Based on our work on best practices in product development, and the

importance of the Joint Strike Fighter to DOD and the industrial base, we

believe that DOD needs to ensure that the program’s critical technologies

are at demonstrated levels of maturity before making engineering and

manufacturing investments in the program. Failure to do so could result in

increases in both the production and long-term ownership costs, schedule

delays, and compromised performance as problems arise in product

development. Moreover, the impact of failing to mature one critical

technology could ripple throughout the program. Therefore, we are

recommending that DOD ensure that critical technologies are mature

before proceeding into engineering and manufacturing to improve the

likelihood of meeting program expectations or to take additional actions if

it chooses to accept the risk of immature technologies.

In commenting on our report, DOD said that it assessed the technology

maturity of the Joint Strike Fighter to be sufficient for the next phase of

the program. DOD also said that the Joint Strike Fighter Program Office

has implemented a rigorous risk management program that will continue

to monitor and address technology risks, as well as other risks, throughout

the program’s life. We disagree with DOD’s assertion that technology is

 5

Due to the current Joint Strike Fighter competition, the technologies are not specified so

as to not associate them with either Boeing or Lockheed Martin, respectively.
Page 4 GAO-02-39 Joint Strike Fighter Acquisition

mature enough to move forward. The technology readiness level

assessment conducted as part of our review of the Joint Strike Fighter

showed that critical technologies are not projected to be matured to levels

that would stem risks at the start of engineering and manufacturing

development. Our previous work has shown that when programs proceed

in this fashion, they experience delays, rework, and substantial cost

increases that could force the Department to divert much-needed funds

from other important weapon system programs.

The Joint Strike Fighter Program is structured to use a common

production line to produce three versions of a single aircraft. These

aircraft will be tailored to meet conventional flight requirements for the

U.S. Air Force, short take-off and vertical landing characteristics for the

U.S. Marine Corps, and carrier operation suitability needs for the U.S.

Navy. The program will also provide aircraft to the British Royal Navy and

Air Force. Table 1 shows the services’ planned use for the Joint Strike

Fighter.

Table 1: Military Services’ Planned Use for the Joint Strike Fighter

Service Quantity Planned use

U.S. Air Force 1,763 Replacement for F-16 and A-10; complement the F-22

U.S. Marine Corps 609 Replacement for the AV-8B and F/A-18 C/D

U.S. Navy 480 Complement the F/A-18 E/F

Great Britain 150 Replacement for the Sea Harrier and GR.7

Source: Joint Strike Fighter program office.

A key objective of the Joint Strike Fighter acquisition strategy is

affordability—reducing the development, production, and ownership costs

of the program relative to prior fighter aircraft programs. To achieve its

affordability objective, the Joint Strike Fighter program has incorporated

various acquisition initiatives into the program’s acquisition strategy and

various technological advances into the fighter. Among the acquisition

initiatives planned was to develop critical technologies to a level where

they represent low technical risk before the engineering and

manufacturing contract is awarded. The expectation was that

incorporating these initiatives into the acquisition strategy would avoid

cost growth, schedule slippage, and performance shortfalls that have been

experienced in other weapon acquisition programs.

Background
Page 5 GAO-02-39 Joint Strike Fighter Acquisition

To date, the Joint Strike Fighter Program has awarded contracts totaling

over $2 billion to Boeing and Lockheed Martin for the current concept

demonstration phase. During this phase, DOD required each contractor to

design and build two aircraft to demonstrate the following:

• commonality/modularity to validate the contractors’ ability to produce

three aircraft versions on the same production line;

• the aircraft’s ability to do a short take-off and vertical landing, hover, and

transition to forward flight; and

• satisfactory low airspeed, carrier approach flying and handling qualities.

Each contractor was required to submit a Preferred Weapon System

Concept, which outlines its final design concept for developing a Joint

Strike Fighter aircraft that is affordable and meets performance

requirements. The Preferred Weapon System Concept includes results

from the flight and ground demonstrations and is being used by DOD to

select the winning aircraft design and to award the engineering and

manufacturing development contract. During engineering and

manufacturing development, the Joint Strike Fighter will be fully

developed, engineered, designed, fabricated, tested, and evaluated to

demonstrate that the production aircraft will meet stated requirements.

Critical junctures in engineering and manufacturing development are the

preliminary and critical design reviews and commitments; testing of

aircraft; and commitments to production hardware, including the purchase

of long lead production items. It is at the critical design review that

decisions are made toward finalizing the aircraft design and begin building

test aircraft. About two-thirds of engineering and manufacturing

development funding will be spent after this review. Figure 1 shows

planned Joint Strike Fighter aircraft designs by contractor.
Page 6 GAO-02-39 Joint Strike Fighter Acquisition

Figure 1: Boeing and Lockheed Martin Joint Strike Fighter Aircraft Design Concepts

Source: Boeing and Lockheed Martin.

In our previous work on best business practices, commercial firms have

told us that a key part of product development is getting the technology

into the right size, weight, and configuration needed for the intended

product—in this case, the final Joint Strike Fighter design. Once this has

been demonstrated, the technology is at an acceptable level for product

development. Technology readiness levels (TRL) can be used to assess the

maturity of technology and can reveal whether a gap exists between a

technology’s maturity and the maturity demanded for successful inclusion

in the intended product. Defining this gap for the Joint Strike Fighter

technologies is important for determining whether they can be expected to

demonstrate required capabilities before being integrated into the aircraft

design. Readiness levels are measured along a scale of one to nine, starting

with paper studies of the basic concept, proceeding with laboratory

demonstrations, and ending with a technology that has proven itself on the

intended product. (See app. I for a detailed description of TRLs.) The Air

Force Research Laboratory considers TRL 7 an acceptable risk for starting

the engineering and manufacturing development phase. The readiness

Technology Readiness

Assessments Provide

Opportunities to

Improve Outcomes
Page 7 GAO-02-39 Joint Strike Fighter Acquisition

level definitions state that for a technology to be rated at TRL 7, it must be

demonstrated using prototype hardware (such as a complete radar

subsystem) that is the same size, weight, and configuration as that called

for in the final aircraft design and that prototype has to be demonstrated

to work in an environment similar to the planned operational system.

We have previously reviewed the impact of incorporating technologies

into new product and weapon system designs.6 The results showed that

programs met product objectives when the technologies were matured to

higher levels and conversely showed that cost and schedule problems

arose when programs started when technologies were at low readiness

levels. For example, the Joint Direct Attack Munition (JDAM) used

modified variants of proven components for guidance and global

positioning. It also used mature, existing components from other proven

manufacturing processes for its own system for controlling tail fin

movements. The munition was touted for its performance in Kosovo and

was purchased for less than half of its expected unit cost. However, the

Comanche helicopter program began with critical technologies such as the

engine, rotor, and integrated avionics at TRL levels of 5 or below. That

program has seen 101 percent cost growth and 120-percent schedule

slippage as a result of these low maturity levels and other factors.

In commenting on our report concerning better management of

technology development, DOD agreed that TRLs are important and

necessary in assisting decision makers in deciding on when and where to

insert new technologies into weapons system programs and that it is

desirable to mature technologies to TRL 7 prior to entering the engineering

and manufacturing development phase of a weapon system program.7

Since that time, DOD has adopted the technology readiness levels as a

means of assessing the technological maturity of new major programs. In a

July 5, 2001, memorandum, the Deputy Under Secretary of Defense

(Science and Technology) stated that new DOD regulations require that

the military services’ science and technology executives conduct a

technology readiness level assessment for critical technologies identified

in major weapon systems programs prior to the start of engineering and

 6

Best Practices: Successful Application to Weapon Acquisitions Requires Changes in

DOD’s Environment (GAO/NSIAD-98-56, Feb. 24, 1998) and Best Practices: Better

Management of Technology Development Can Improve Weapon System Outcomes

(GAO/NSIAD-99-162, July 30, 1999).

7

GAO/NSIAD-99-162, July 30, 1999.
Page 8 GAO-02-39 Joint Strike Fighter Acquisition

manufacturing development and production. The memorandum notes that

technology readiness levels are the preferred approach for all new major

programs unless the Deputy Under Secretary approves an equivalent

assessment method.

The Joint Strike Fighter Program, like many other DOD programs, has

used risk management plans and engineering judgment as a way of

assessing technological maturity. The Principal Deputy Under Secretary of

Defense (Acquisition and Technology) has determined that these means

will continue to be used by DOD and the Joint Strike Fighter contractors

to assess the program’s technological risk. Risk management plans and

judgment are necessary to managing any major development effort like the

Joint Strike Fighter. However, without an underpinning such as

technology readiness levels that allow transparency into program

decisions, these methods allow significant technical unknowns to be

judged acceptable risks because a plan exists for resolving the unknowns

in the future. Experience on previous programs has shown that such

methods have rarely assessed technical unknowns as a high or

unacceptable risk; consequently, they failed to guide programs to meet

promised outcomes. Technology readiness levels are based on actual

demonstrations of how well technologies actually perform. Their strength

lies in the fact that they characterize knowledge that exists rather than

plans to gain knowledge in the future; they are, thus, less susceptible to

optimism.

In May 2000 we reported that all of the eight technologies identified by the

Joint Strike Fighter program office as critical to the program were

expected to be at maturity levels below that considered acceptable for low

risk when entering engineering and manufacturing development (TRL 7).

The eight critical technologies are: prognostics and health management,

integrated flight propulsion control, subsystems, integrated support

system, integrated core processor, radar, manufacturing, and mission

systems integration. (See app. II for a description of these technologies.)

During our review last year, we worked with the two competing

contractors and the program office to arrive at the applicable TRLs for the

critical technologies. Specifically, on separate visits to the contractors,

with program office personnel present, we asked the contractors’ relevant

technology managers to score the technologies they considered critical to

enable their Joint Strike Fighter design to meet DOD requirements for the

aircraft. At that time, we also asked them to describe their plans to mature

the technologies to the planned start of the engineering and manufacturing

development phase, then scheduled for April 2001. Upon reviewing these

Critical Technologies

Are Not Projected to

Be at Low Risk
Page 9 GAO-02-39 Joint Strike Fighter Acquisition

scores with the program office and in order to gain an overall Joint Strike

Fighter Program perspective on technical maturity, the Joint Strike Fighter

office agreed to provide us with TRL scores for the eight technologies they

considered critical for meeting program cost and performance

requirements. Figure 2 reflects the program office scores at the time of our

last review. Due to the current Joint Strike Fighter competition, the

specific technologies mentioned previously are not linked to scores so as

not to divulge competition sensitive information.

Figure 2: Actual and Projected Joint Strike Fighter Critical Technology Readiness Levels

Source: Joint Strike Fighter program office.

As the figure shows, all eight technologies were projected to be below the

level of maturity (TRL 7) considered acceptable for low risk when entering

the engineering and manufacturing development phase and six of the
Page 10 GAO-02-39 Joint Strike Fighter Acquisition

technologies were projected to be below the level of maturity (TRL 6) that

is considered low risk for entering the demonstration phase, which the

Joint Strike Fighter Program began in 1996.

During our current review, we again visited the two competing contractors

to discuss the status of the eight technologies. We learned that they have

essentially accomplished, or plan to accomplish by October 2001, the

technology development and demonstrations that they planned to

accomplish as of April 2001. Thus figure 2 represents the current

assessment of technical maturity. While two of these areas are very close

to appropriate maturity levels, the Joint Strike Fighter’s critical

technologies are not projected to be matured to levels that we believe

would indicate a low risk program at the planned start of the engineering

and manufacturing development phase. Key component technologies

remain at higher risk levels for engineering and manufacturing

development because (1) they have not been developed to approximately

the same size, weight, and configuration called for in the final aircraft

design and/or (2) they have not been demonstrated to work in an

environment similar to the planned operational system.

The Joint Strike Fighter Program has made good progress in some

technology areas. For example, contractor and program officials told us

that because of concerns about propulsion technology, both contractors

focused considerable attention on that area. Both contractors flew aircraft

that demonstrated the capability for short take-off and vertical landing and

accumulated at least 20 hours of flight time on those aircraft, which should

satisfy the requirement in the Fiscal Year 2001 National Defense

Authorization Act. In some other areas, the technology maturation has not

been uniform across all critical components of a technology. For example,

the radar has a number of critical components that must work together as

a system. Both contractors have made considerable progress on one or

more of those components, but the other critical components have not

been matured to an acceptable level of risk. In order for this technology to

achieve a TRL level of 7, all components had to be (1) demonstrated in the

size and weight required to meet aircraft capabilities, (2) integrated

together as they would be in the final aircraft design, and (3) flown in an

environment similar to what the Joint Strike Fighter will be subjected. To

demonstrate some critical technologies, both contractors flew key

electronic and other components in flying avionics test beds (commercial

aircraft reconfigured as flying laboratories). While these tests occurred in

a relevant environment (e.g., in flight), the tested hardware was not always

the same size and weight required for the Joint Strike Fighter aircraft.

Conversely, some components were built to the required size and weight,
Page 11 GAO-02-39 Joint Strike Fighter Acquisition

but were demonstrated only in ground-testing environments. By not

having matured all critical technology areas to appropriate maturity levels,

the program remains at risk for achieving cost and performance goals

upon entering product development.

Moving into engineering and manufacturing development creates an

expectation that the Joint Strike Fighter can be delivered for a stated time

and dollar investment and with a given set of capabilities. The decisions

the Department of Defense makes now and over the next 2 years will

largely determine whether those expectations can be met.

A key component of the Joint Strike Fighter Program’s acquisition strategy

is to enter the engineering and manufacturing development phase with low

technical risk. The program will not have achieved that point by October

2001 because technologies, which the Joint Strike Fighter Program Office

identified as critical to meeting the program’s cost and requirements

objectives, will not have been matured to an acceptable risk level. By

entering the engineering and manufacturing development phase with

immature critical technologies, the program will need to continue to

develop those technologies at the same time it will be concentrating on

production issues and the integration of subsystems into a Joint Strike

Fighter. This approach would not be consistent with best practices. In fact,

it would more closely follow DOD’s traditional practices in weapon system

programs that have often resulted in cost increases, schedule delays, and

compromised performance.

To eliminate one of the major sources of cost and schedule risk, we

recommend that the Secretary of Defense delay the start of engineering

and manufacturing development until critical technologies are matured to

acceptable levels.

Alternatively, if the Secretary of Defense decides to accept these risks and

move the program into engineering and manufacturing development as

scheduled, we recommend that the Secretary dedicate the resources to

ensuring that maturity of the critical technologies is demonstrated by the

critical design review or defer the inclusion of immature technologies

from the approved design.

Conclusions

Recommendations for

Executive Action
Page 12 GAO-02-39 Joint Strike Fighter Acquisition

In written comments on a draft of this report, the Director of Strategic and

Tactical Systems, within the Office of the Under Secretary of Defense for

Acquisition, Technology, and Logistics, partially concurred with our

recommendation. DOD contended that an independent technology

readiness assessment it carried out on the program showed that

technology has extensively matured and the program is now ready to enter

into systems development and demonstration. DOD also stated that the

Joint Strike Fighter Program Office has implemented a risk management

program that will continue to monitor and address technology risks, as

well as other risks, throughout the program’s life. The full text of DOD’s

comments is included in appendix III.

We disagree with the Department’s assessment of technological maturity.

The TRL assessment conducted as part of our review showed that

technologies critical to the Joint Strike Fighter Program are not projected

to be matured to levels that we believe would indicate a low risk program

at the planned start of the engineering and manufacturing development

phase. Many of the technologies have not been demonstrated in their

appropriate size and weight, nor have they been demonstated to function

in an environment in which they will be used. For example, many of the

technologies are still in the laboratory and will require considerable

maturation before they can be incorporated into the final design. By

entering the engineering and manufacturing development phase with

immature critical technologies, the program will need to continue to

develop those technologies at the same time it will be concentrating on

engineering, designing, and fabricating the product. As it has with many

other DOD programs, this approach increases the likelihood of schedule

delays and program cost increases. This is primarily why DOD’s new

acquisition regulations emphasize separating technology development

from product development. In fact, experience has shown that resolving

technology problems in product development can result in at least a tenfold cost increase.

Moreover, DOD incorrectly states that the tools it used to assess its

technology and the TRLs used for our review are equivalent

methodologies for assessing technological maturity. The Willoughby

Templates used by DOD are a risk management tool. They can be an

excellent way to manage program risks, but in practice they have not been

used to identify risk. Identifying risk is the first step to managing it. By

contrast, by focusing specifically on assessing technology maturity against

objective standards, TRLs have proven successful at identifying risks. A

more appropriate approach for DOD to take is to use technology readiness

levels in conjunction with a management tool such as the Willoughby

Agency Comments

and Our Evaluation
Page 13 GAO-02-39 Joint Strike Fighter Acquisition

templates since this can result in more informed decision making and

fewer unanticipated problems in an acquisition program. In fact, the Joint

Strike Fighter program provides DOD with an excellent opportunity to

apply these concepts in tandem.

To assess whether the Joint Strike Fighter’s critical technologies are

projected to mature to low technical risk at the start of the engineering

and manufacturing development phase, we used the technology readiness

level tool and information provided by Joint Strike Fighter program

officials and contractor officials at the Boeing Company, Seattle,

Washington; Lockheed Martin Aeronautics Company, Fort Worth, Texas;

and Pratt & Whitney, East Hartford, Connecticut. During our previous

review, we had obtained detailed briefings from Boeing and Lockheed

Martin officials on their plans to mature critical technologies prior to the

date for awarding the engineering and manufacturing development

contract, then scheduled for April 2001. We had also obtained program

office and contractor assessments of the expected technology readiness

levels for the critical technologies at April 2001. During our current review,

we obtained detailed briefings from program office personnel on the

status of critical technologies. We also obtained detailed briefings from

Boeing, Lockheed Martin, and Pratt & Whitney officials on the contractors’

progress in maturing critical technologies and any further maturation

plans through October 2001. We compared the latest information from the

program office and the contractors to the information obtained during our

prior review to determine if the critical technologies had been matured to

higher technology readiness levels and the levels achieved.

We conducted our review from April through September 2001 in

accordance with generally accepted government auditing standards.

We are sending copies of this report to the congressional defense

committees; the Honorable Donald H. Rumsfeld, Secretary of Defense; the

Honorable James G. Roche, Secretary of the Air Force; the Honorable

Gordon R. England, Secretary of the Navy; General James L. Jones,

Commandant of the Marine Corps; and the Honorable Mitchell E. Daniels,

Jr., Director, Office of Management and Budget. We will also make copies

available to other interested parties on request.

Scope and

Methodology
Page 14 GAO-02-39 Joint Strike Fighter Acquisition

Please contact me at (202) 512-4841 if you or your staff have any questions

concerning this report. Key contributors to this report were Robert

Pelletier and Brian Mullins.

Sincerely yours,

Katherine V. Schinasi

Director, Acquisition and

 Sourcing Management
Appendix I: Technology Readiness Levels and

Their Descriptions

Page 15 GAO-02-39 Joint Strike Fighter Acquisition

Technology readiness level Description

1. Basic principles observed and

reported.

Lowest level of technology readiness. Scientific research begins to be translated into

applied research and development. Examples might include paper studies of a

technology’s basic properties.

2. Technology concept and/or application

formulated.

Invention begins. Once basic principles are observed, practical applications can be

invented. The application is speculative and there is no proof or detailed analysis to

support the assumption. Examples are still limited to paper studies.

3. Analytical and experimental critical

function and/or characteristic proof of

concept.

Active research and development is initiated. This includes analytical studies and

laboratory studies to physically validate analytical predictions of separate elements of the

technology. Examples include components that are not yet integrated or representative.

4. Component and/or breadboard

validation in laboratory environment.

Basic technological components are integrated to establish that the pieces will work

together. This is relatively “low fidelity” compared to the eventual system. Examples

include integration of “ad hoc” hardware in a laboratory.

5. Component and/or breadboard

validation in relevant environment.

Fidelity of breadboard technology increases significantly. The basic technological

components are integrated with reasonably realistic supporting elements so that the

technology can be tested in a simulated environment. Examples include “high fidelity”

laboratory integration of components.

6. System/subsystem model or prototype

demonstration in a relevant environment.

Representative model or prototype system, which is well beyond the breadboard tested

for TRL 5, is tested in a relevant environment. Represents a major step up in a

technology’s demonstrated readiness. Examples include testing a prototype in a high

fidelity laboratory environment or in simulated operational environment.

7. System prototype demonstration in an

operational environment.

Prototype near or at planned operational system. Represents a major step up from TRL 6,

requiring the demonstration of an actual system prototype in an operational environment,

such as in an aircraft, vehicle, or space. Examples include testing the prototype in a test

bed aircraft.

8. Actual system completed and “flight

qualified” through test and

demonstration.

Technology has been proven to work in its final form and under expected conditions. In

almost all cases, this TRL represents the end of true system development. Examples

include developmental test and evaluation of the system in its intended weapon system to

determine if it meets design specifications.

9. Actual system “flight proven” through

successful mission operations.

Actual application of the technology in its final form and under mission conditions, such as

those encountered in operational test and evaluation. In almost all cases, this is the end

of the last “bug fixing” aspects of true system development. Examples include using the

system under operational mission conditions.

Source: Best Practices: Better Management of Technology Development Can Improve Weapon

System Outcomes (GAO/NSIAD-99-162, July 30, 1999).

Appendix I: Technology Readiness Levels and

Their Descriptions
Appendix II: Critical Technologies and Their

Descriptions

Page 16 GAO-02-39 Joint Strike Fighter Acquisition

Critical technologies Description

Prognostics and health management Involves the ability to detect and isolate the cause of aircraft problems and then predict

when maintenance activity will have to occur on systems with pending failures. Life-cycle

cost savings are dependent on prognostics and health management through improved

sortie generation rate, reduced logistics and manpower requirements, and more efficient

inventory control.

Integrated flight propulsion control Includes integration of propulsion, vehicle management system, and other subsystems

as they affect aircraft stability, control, and flying qualities (especially short take-off and

vertical landing). Aircraft improvements are to reduce pilot workload and increase flight

safety.

Subsystems Includes areas of electrical power, electrical wiring, environmental control systems, fire

protection, fuel systems, hydraulics, landing gear systems, mechanisms, and secondary

power. Important for reducing aircraft weight, decreasing maintenance cost, and

improving reliability.

Integrated support systems Involves designing an integrated support concept that includes an aircraft with

supportable stealth characteristics and improved logistics and maintenance functions.

Life-cycle cost savings are expected from improved low observable maintenance

techniques and streamlined logistics and inventory systems.

Integrated core processor Includes the ability to use commercial-based processors in an open architecture design

to provide processing capability for radar, information management, communications,

etc. Use of commercial processors reduces development and production costs and an

open architecture design reduces future development and upgrade costs.

Radar Includes advanced integration of communication, navigation, and identification functions

and electronic warfare functions through improved apertures, antennas, modules,

radomes, etc. Important for reducing avionics cost and weight, and decreasing

maintenance cost through improved reliability.

Manufacturing Involves lean, automated, highly efficient aircraft fabrication and assembly techniques.

Manufacturing costs should be less through improved flow time, lower manpower

requirements, and reduced tooling cost.

Mission systems integration Involves decreasing pilot workload by providing information for targeting, situational

awareness, and survivability through fusion of radar, electronic warfare, and

communication, navigation, and identification data. Improvements are achieved through

highly integrated concept of shared and managed resources, which reduces production

costs, aircraft weight, and volume requirements, in addition to improved reliability.

Source: Joint Strike Fighter program office.

Appendix II: Critical Technologies and Their

Descriptions
Appendix III: Comments From the

Department of Defense

Page 17 GAO-02-39 Joint Strike Fighter Acquisition

Appendix III: Comments From the

Department of Defense
Appendix III: Comments From the

Department of Defense

Page 18 GAO-02-39 Joint Strike Fighter Acquisition
Appendix III: Comments From the

Department of Defense

Page 19 GAO-02-39 Joint Strike Fighter Acquisition (120050)

The General Accounting Office, the investigative arm of Congress, exists to

support Congress in meeting its constitutional responsibilities and to help

improve the performance and accountability of the federal government for the

American people. GAO examines the use of public funds; evaluates federal

programs and policies; and provides analyses, recommendations, and other

assistance to help Congress make informed oversight, policy, and funding

decisions. GAO’s commitment to good government is reflected in its core values

of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents is through the

Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-text files of

current reports and testimony and an expanding archive of older products. The

Web site features a search engine to help you locate documents using key words

and phrases. You can print these documents in their entirety, including charts and

other graphics.

Each day, GAO issues a list of newly released reports, testimony, and

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site

daily. The list contains links to the full-text document files. To have GAO E-mail

this list to you every afternoon, go to our home page and complete the easy-to-use

electronic order form found under “To Order GAO Products.”

The first copy of each printed report is free. Additional copies are $2 each. A

check or money order should be made out to the Superintendent of Documents.

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a

single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

P.O. Box 37050

Washington, D.C. 20013

To order by phone: Voice: (202) 512-6000

TDD: (301) 413-0006

Fax: (202) 258-4066

GAO Building

Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)

Washington, D.C. 20013

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm,

E-mail: fraudnet@gao.gov, or

1-800-424-5454 (automated answering system).

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G. Street NW, Room 7149,

Washington, D.C. 20548

GAO’s Mission

Obtaining Copies of

GAO Reports and

Testimony

Order by Mail or Phone

Visit GAO’s Document

Distribution Center

To Report Fraud,

Waste, and Abuse in

Federal Programs

Public Affairs