GAO

United States General Accounting Office

Report to the Chairman and Ranking 

Member, Subcommittee on Readiness 

and Management Support, Committee on 

Armed Services, U.S. Senate

March 2001 BEST PRACTICES

Better Matching of 

Needs and Resources 

Will Lead to Better 

Weapon System 

Outcomes

GAO-01-288

Page 1 GAO-01-288 Best Practices

Contents

Letter 5

Executive Summary 6

Chapter 1

Introduction

14

How Product Requirements Are Set Is Key to Program Outcomes 16

Objectives, Scope, and Methodology 20

Chapter 2

Timely Matching of 

Requirements and 

Resources Is Critical to 

Product Development 

Outcomes

25

Early Matching of Customer Expectations and Developer’s

Resources Is Critical to Program Success 25

Trade-offs Are Critical to Matching Customer Expectations 

With Developer Resources Before Starting Product Development 28

When Matching Did Not Occur Before Program Launch,

Developers Were Forced to Add Unplanned Resources 31

Chapter 3

Several Factors Enable 

Customer Wants and 

Developer Resources 

to Be Matched Before 

Program Launch

34

Systems Engineering Tools Are Critical for Identifying Gaps Between 

Developer’s Resources and Customer’s Expectations 35

Flexibility in Setting Requirements Is Key to Closing Gaps Between 

Customer Expectations and Developer Resources 44

Balance in the Roles of the Customer and Product Developer

Makes for Effective Trade-offs 50

Chapter 4

Characteristics of 

DOD’s Acquisition 

Process Make It Hard 

to Match Expectations 

and Resources Before 

Program Launch

55

Current Process Puts Requirements Setting and Systems

Engineering on Opposite Sides of the Launch Decision 56

Incentives of Current Process Create Pressure on Product

Requirements 58

More Successful Weapon System Programs Have Departed

From the Normal Process 62

Constructive Changes in DOD’s Policy Not Enough to Match 

Customer Expectations With Developers’ Resources 63
Contents

Page 2 GAO-01-288 Best Practices

Chapter 5

Conclusions and 

Recommendations for 

Executive Action

66

Conclusions 66

Recommendations for Executive Action 68

Agency Comments and Our Evaluation 69

Appendixes Appendix I: Comments From the Department of Defense 70

Appendix II: GAO Staff Acknowledgments 74

Related GAO Products 75

Tables Table 1: Customer and Product Developer Interests in a Product’s 

Development 17

Table 2: Matching of Expectations to Resources and Product

Development Outcomes 26

Figures Figure 1: Knowledge-based Process for Applying Best Practices 

to the Development of New Products 15

Figure 2: The Requirements Process 18

Figure 3: Commercial and DOD Organizations Involved in

Requirements Setting Processes 19

Figure 4: Timing of Match Between Customer Expectations and 

Resources 27

Figure 5: Caterpillar Mining Truck 29

Figure 6: Timeline for Development of the 797 Mining Truck 29

Figure 7: Effect of FUSE Trade-offs on Matching Customer

Expectations with Developer’s Resources 31

Figure 8: Navy’s Development of the Radio Frequency 

Countermeasures System 32

Figure 9: Basic Steps in Systems Engineering Process 35

Figure 10: Systems Engineering Used for Successful and Problematic

Cases 36

Figure 11: Tactical Unmanned Aerial Vehicle 40

Figure 12: Crusader Artillery Vehicle 43

Figure 13: Global Hawk Unmanned Aerial Vehicle 48

Figure 14: Timeline for Developing Crusader’s Requirements 57

Figure 15: Revised DOD Acquisition Process 64
Contents

Page 3 GAO-01-288 Best Practices

Abbreviations

BAT Brilliant Anti-armor Submunition

DOD Department of Defense

FUSE Far Ultraviolet Spectroscopic Explorer

NASA National Aeronautics and Space Administration
Page 4 GAO-01-288 Best Practices
United States General Accounting Office

Washington, D.C. 20548

Page 5 GAO-01-288 Best Practices

Letr

March 8, 2001

The Honorable James Inhofe

Chairman

The Honorable Daniel Akaka

Ranking Member

Subcommittee on Readiness and Management Support

Committee on Armed Services

United States Senate

As you requested, this report examines how best practices offer improvements to the way the 

Department of Defense defines and matches weapon system requirements to available resources such 

as cost, schedule, and mature technologies. It also examines the importance of the timing of this 

match and identifies practices that can improve the prospects for making the match before starting an 

acquisition program. We make recommendations to the Secretary of Defense on how to better align 

the requirements setting and program approval processes to infuse more knowledge earlier into each 

process.

We are sending copies of this report to the Honorable Donald H. Rumsfeld, Secretary of Defense; the 

Honorable Gregory R. Dahlberg, Acting Secretary of the Army; the Honorable Robert B. Pirie, Jr., 

Acting Secretary of the Navy; the Honorable Lawrence Delaney, the Acting Secretary of the Air Force; 

the Honorable Mitchell E. Daniels, Jr., Director, Office of Management and Budget; and to interested 

congressional committees. We will also make copies available to others upon request.

If you have any questions regarding this report, please call me at (202) 512-4841. Other key contacts 

are listed in appendix II.

Katherine V. Schinasi

Director

Acquisition and Sourcing Management
Page 6 GAO-01-288 Best Practices

Executive Summary

Purpose Although the Department of Defense’s (DOD) annual weapon system 

investment has been increased from about $90 billion 3 years ago to almost 

$100 billion for fiscal year 2001, DOD’s buying power will be weakened if 

weapons continue to cost significantly more and take much longer to 

develop than planned. DOD would like to get the most out of this 

investment and has set goals to develop new weapons in half the traditional 

time and within budget. It has a long way to go; long-standing problems that 

work against delivering new weapons within estimates have proven 

resistant to reform. When one program encounters such problems and 

needs more money than planned, it comes at the expense of delaying or 

canceling other programs. This means less overall modernization and a 

lower, unpredictable return on investment. The ability to execute a 

program more predictably within cost and schedule estimates would lessen 

the need to offset cost increases by disrupting other programs. 

GAO has issued a series of reports on the success leading commercial firms 

have had in significantly reducing the time and money to develop new and 

more sophisticated products—the kinds of results that DOD seeks. The 

best practices of these firms center on obtaining knowledge about the 

technology, design, and production of a new product at key points in time. 

The most critical juncture is the decision to start development of a new 

product, for which firms insist on having a match between what the 

customer wants in a new product and what resources they have to develop 

that product. Significant cost and schedule increases in weapon system 

programs can be traced to not having achieved this match at program start. 

This report addresses how the process of setting requirements for new 

products can be managed in a way that does not exceed the developer’s 

resources yet provides a product the customer wants. In response to a 

request from the Chairman and the Ranking Member, Subcommittee on 

Readiness and Management Support, Senate Committee on Armed 

Services, GAO assessed (1) the effect the timing of the match between the 

customer’s needs and the developer’s resources has on a product’s cost and 

schedule; (2) the best practices for obtaining this match during the 

requirements setting process, compared with more traditional DOD 

practices; and (3) the progress made and challenges DOD faces in adopting 

best practices for setting requirements on individual weapon systems. 
Executive Summary

Page 7 GAO-01-288 Best Practices

Background The decisions made in translating the ideas for a new product into actual 

features and characteristics dictate the amount of resources that will be 

necessary to bring the product to market. Thus, they may be the most 

highly leveraged of all product development decisions. In the past, it has 

not been unusual for weapon system requirements to be set so high that the 

initial estimate of the resources necessary to develop a responsive weapon 

falls considerably short of the mark. For both commercial and DOD 

products, a natural amount of tension precedes the setting of requirements, 

because it is common for a customer’s initial expectations to exceed what 

the developer can do within known or available resources. The resources a 

product developer can consider available include (1) knowledge—the 

technology and capabilities the developer has to engineer and manufacture 

the product and (2) the time and money the developer has to design, test, 

and deliver the product. A process of negotiation and trade-offs is usually 

necessary to match customer expectations and developer resources before 

a product can be successfully developed and produced.

Among the key sources of information GAO relied on in this review were 

experiences from nine major product development programs from DOD, 

the National Aeronautics and Space Administration (NASA), and two 

commercial firms recognized for their success in setting product 

requirements. GAO reviewed the requirements setting process for all nine 

programs. For each program, GAO interviewed key managers and obtained 

documentation to determine (1) the process that was used to achieve the 

match between customer expectations and resources to form the eventual 

product’s requirements, (2) the timing of this match and the tools used to 

achieve it, and (3) the extent to which the requirements setting process 

affected the program’s subsequent progress or success. While the 

commercial products differed significantly, much commonality existed 

among the firms’ practices for managing the requirements for a new 

product. The commercial examples in the report were chosen for their 

excellence; as such, they do not necessarily represent the standard 

practices of commercial industry as a whole. 

Results in Brief A match between a developer’s resources and a customer’s expectations is 

eventually met on just about every product or weapon system 

development. A key distinction between successful products—those that 

perform as expected and are developed within estimated resources—and 

problematic products is when this match is achieved. When a customer’s 

needs and a developer’s resources were matched before a product 
Executive Summary

Page 8 GAO-01-288 Best Practices

development started, the more likely the development was to meet cost 

and schedule objectives. When this match took place later, after the 

product development was underway, problems occurred that took 

significantly higher investments—sometimes double—of time and money.

GAO identified three factors that were key to matching needs and 

resources before product development began. First, developers employed 

the technique of systems engineering to identify gaps between resources 

and customer needs before committing to a new product development. 

Second, customers and developers were flexible. Leeway existed to reduce 

or defer customer needs to future programs or for the developer to make 

an investment to increase knowledge about a technology or design feature 

before beginning product development. Third, the roles and responsibilities 

of the customer and the product developer were matched, with the product 

developer being able to determine or significantly influence product 

requirements. In cases where these factors were not present at program 

launch, product development began without a match between 

requirements and resources. Invariably, this imbalance favored meeting 

customer needs by adding resources, which resulted in increased costs and 

later deliveries. 

DOD has recently revised its acquisition policy to encourage a more 

evolutionary approach for setting requirements and developing new 

weapons. This policy merits support; if effectively implemented, it could 

facilitate a better match between customer needs and developer resources 

before program launch. DOD, however, faces two significant hurdles in 

implementing the policy. First, the mechanics of obtaining funds to start 

programs are unchanged. Specifically, requirements must still be set before 

a program can be approved and a program must be approved before money 

can be paid to the product developer to conduct systems engineering. Such 

mechanics deny the knowledge needed to match customer expectations 

with developer resources before starting a program. Second, many of the 

same incentives still exist—such as the competition for program funding—

to push requirements up, making them more difficult to meet and less 

flexible to negotiate. 

GAO makes recommendations to the Secretary of Defense on ways to 

realign the mechanics and incentives of the requirements setting and 

program approval process with the need to match customer expectations 

and developer resources before weapon system programs are started. 
Executive Summary

Page 9 GAO-01-288 Best Practices

Principal Findings

Timely Match of 

Requirements and 

Resources Is Critical to 

Product Development 

Outcomes

For the nine development programs GAO examined, there was a 

relationship between when customer expectations were matched with 

available resources and the programs’ ability to meet cost and schedule 

predictions. In cases where needs and resources were matched before 

program start, like Caterpillar’s 797 mining truck and NASA’s Far 

Ultraviolet Spectroscopic Explorer, cost growth was 20 percent or less and 

product development schedules were met. In cases where programs were 

started with requirements that exceeded resources, like the Crusader 

artillery vehicle and the Radio Frequency Countermeasures system, cost 

growth ranged from 55 percent to nearly 200 percent and schedule delays 

were about 25 percent. Key to the successful cases was the ability to make 

early trade-offs either in the design of the product or in the customer’s 

expectations to avoid outstripping the resources available for product 

development. The less successful cases missed opportunities to make 

trade-offs before product development started. By the time the gaps 

between requirements and resources were recognized and confronted, it 

was too late to materially change the requirements. Consequently, the only 

route left was for the developer to invest more resources than originally 

planned to meet the requirements. 

Several Factors Enable 

Customer’s Needs and 

Developer’s Resources to Be 

Matched Before Program 

Launch

GAO found significant differences between the successful cases and those 

that experienced problems regarding (1) how systems engineering was 

employed, (2) how flexible customers and developers were, and (3) how 

balanced the roles of the developer and the customer were. Systems 

engineering is a disciplined process that translates customer needs into the 

specific capabilities that are needed to create the product, such as 

individual technologies and manufacturing processes. It is critical for 

identifying and resolving gaps between needs and resources and lays the 

factual foundation for pragmatic negotiations. When the product developer 

employed systems engineering before a new program started, the resultant 

gaps could be addressed through investments, alternate designs, and

trade-offs. For example, Bombardier’s systems engineering analysis for a 

new regional-sized jet showed that if the customer would accept a 

3.7-percent reduction in cruise speed, existing propulsion technology could 

be used, greatly reducing risk. The customer agreed to the reduction. 

Flexibility was encouraged when the developer set a limit on how long it 

would allow product development to take but could credibly assure 
Executive Summary

Page 10 GAO-01-288 Best Practices

customers that future versions of a product would meet many expectations 

deferred from the initial product. Finally, the developer and the customer 

were equal partners in setting product requirements; a new product was 

not begun unless both parties agreed to the requirements.

In the cases where the developers did not deliver the products as promised, 

little systems engineering had been done by the time requirements were set 

and the programs were launched. The bulk of systems engineering—

including the identification of gaps between resources and requirements—

was not done until well into product development. For example, it was 

2 years after the Crusader’s launch—after requirements were set and 

resource estimates made— that the developer concluded that a key 

propellant technology could not be developed within resources. Also, in 

these cases, the customers were relatively inflexible regarding 

requirements before the programs were launched. Even when the attempt 

was made to deliberately limit the length of the product development 

cycle—as in the case of the countermeasures program—the customer was 

unwilling to make compensating trade-offs in performance. Finally, the 

customer played the dominant role in setting requirements—a process that 

took over 4 years in one case—with comparatively little input from the 

product developer. In some cases, without having done systems 

engineering, the product developer was in a weak position to disagree with 

the requirements. In other cases, the product developer was forced to 

accept the requirements despite pointing out that resources were 

insufficient. 

Characteristics of DOD’s 

Acquisition Process Make It 

Hard to Match Needs and 

Resources Before Program 

Launch

In DOD, the mechanics of obtaining funding and getting approval to start 

an acquisition program dictate that events proceed in the following 

sequence: set requirements, obtain funding, launch program, perform 

systems engineering. This sequence places the knowledge that is needed to 

identify resource gaps and shape requirements after the program has been 

launched and resource estimates have been made. DOD does not typically 

sign contracts with product developers that conduct systems engineering 

until acquisition programs are started and funding is received. In turn, 

programs cannot be approved unless requirements have been set. By the 

time the systems engineering is started, customers’ needs—as well as those 

of organizations within DOD and the Congress that approve and fund 

programs—have been set, making it difficult to change requirements. 

Adding resources—manifested by cost and schedule increases—then 

becomes the primary option for closing gaps. 
Executive Summary

Page 11 GAO-01-288 Best Practices

DOD’s process for setting requirements and justifying programs creates 

incentives for setting requirements that exceed available resources. For 

example, the intense competition to get programs approved and funded 

encourages requirements that will make the desired weapon system stand 

out from others. Also, DOD requirements setters are often motivated—not 

without reason—to aim for the most capability possible, given that it may 

be several years before they get another opportunity for a new weapon 

system of the same type. Further, they do not necessarily have confidence 

that DOD will be able to fund future, more capable versions of a weapon if 

they accept minimum capabilities on the initial version. Finally, the DOD 

customer is more willing to accept cost increases and schedule delays after 

program launch than a commercial customer. When these additional 

resources are provided after a program is launched, the incentive to let 

requirements drive resources up is reinforced.

DOD has recently adopted policies that could make it possible for a better 

matching of customer expectations and developer resources before 

program launch. These policies discourage programs from accepting 

unreasonable technical risks and identify ways such risks can be reduced 

before program launch. A primary way is the policies’ expressed 

preference for an evolutionary approach to weapons development that 

calls for setting a reasonable—but not ultimate—requirement for an initial 

version of a weapon, with improvements to follow. By themselves, 

however, the policy changes do not materially alter the mechanics or the 

incentives that shape the process for setting requirements and justifying 

programs. 

Recommendations for 

Executive Action

GAO recommends that the Secretary of Defense (1) require that systems 

engineering that is needed to evaluate the sufficiency of available resources 

be conducted before weapon system requirements are formalized, 

(2) reduce the pressures that encourage setting high and inflexible 

requirements to win the competition for program approval, and (3) require, 

as a condition for starting a new weapon system program, that sufficient 

evidence exists to show there is a match between a weapon system’s 

requirements and the resources the program manager has to develop that 

weapon. These recommendations appear in full in chapter 5. 
Executive Summary

Page 12 GAO-01-288 Best Practices

Agency Comments DOD generally agreed with the report and its recommendations. A detailed 

discussion of DOD’s comments appear in appendix I.
Page 13 GAO-01-288 Best Practices
Page 14 GAO-01-288 Best Practices

Chapter 1

Introduction Chapte1r

For several years, the Department of Defense (DOD) has expressed an 

urgent need to acquire new weapon systems to replace its force that it 

believes is becoming outdated and too costly to operate. DOD’s annual 

weapon system investment has increased from about $90 billion 3 years 

ago to almost $100 billion for fiscal year 2001; over the next 5 years, DOD 

plans to spend about $516 billion developing and acquiring weapon 

systems. DOD would like to get the most out of this investment, and it has 

set goals to develop new weapons in half the traditional time and within 

budget. Historically, DOD has not received a predictable return on its 

investment in major weapon systems as they have cost significantly more 

and taken much longer to complete than originally estimated. When one 

program runs into problems and needs more money than planned, it comes 

at the expense of delaying or canceling other programs, which reduces 

buying power and means less overall modernization. The ability to execute 

a program more predictably within cost and schedule estimates would 

lessen the need to offset cost increases by disrupting other programs. DOD 

recognizes that changes are necessary to its acquisition practices to 

achieve its modernization goals. Thus, it has advocated adopting the 

practices of leading commercial firms.

Our reviews over the past 20 years have likewise pointed to a need to adopt 

new practices. We have seen many of the same problems recur in weapon 

system programs—cost increases, schedule delays, and performance 

problems. On many occasions, we found that programs required more 

resources—time and money—than were estimated for demonstrating 

technologies, designing solutions, and providing more production 

capabilities in order to meet customer expectations. Because customer 

expectations for the system’s performance were set when the decision was 

made to invest in the system, adding resources became the primary option 

for solving problems when they arose. Despite good intentions and some 

progress, our ongoing reviews of DOD’s weapon system acquisitions show 

that these persistent problems remain. As a result, we undertook a body of 

work that examines weapon system acquisitions issues from a different, 

more cross-cutting perspective—one that draws lessons learned from the 

best commercial product development efforts to see if they can be applied 

to DOD weapon system developments. In past years, leading commercial 

firms have developed increasingly sophisticated products in significantly 

less time and at lower costs. 

Our past work has shown that leading commercial firms expect their 

program managers to deliver high quality products on time and within 

budget. Thus, the firms have created an environment and adopted practices 
Chapter 1

Introduction

Page 15 GAO-01-288 Best Practices

that put their program managers in a good position to succeed in meeting 

these expectations. Collectively, these practices comprise a development 

process that is anchored in knowledge. The firms demand—and receive—

specific knowledge about a new product at key junctures in the process, 

(see fig. 1). 

Figure 1: Knowledge-based Process for Applying Best Practices to the Development of New Products

There is a synergy in this process, as the attainment of each successive 

knowledge point builds on the preceding one. Such a knowledge-based 

process is essential to commercial firms getting better—and predictable—

cost, schedule, and performance outcomes. It enables decisionmakers to 

be reasonably certain about critical facets of the product under 

development when they need it. We have found that when DOD programs 

have employed similar practices, they also experience good outcomes. This 

knowledge can be broken down into three knowledge points: 

• when a match is made between the customer’s needs and the available 

technology; 

• when the product’s design meets performance requirements, and 

• when the product can be produced within cost, schedule, and quality 

targets. 

The most important knowledge point occurs at launch—the point at which 

the product developer makes a decision to commit (or invest) the 

resources necessary to develop a new product that will meet customer 

needs. This knowledge point makes it easier to reach the remaining two 

knowledge points at the right time. Successful programs are launched only 

when a product developer is confident that it has the resources—

technology, engineering, and production knowledge, along with sufficient 

time, and money—to develop a product the customer wants. Significant 

Technology

Development Product Development Production

Program Launch

Technologies

match

requirements

Design

performs as

expected

Production can meet

cost, schedule, and

quality targets
Chapter 1

Introduction

Page 16 GAO-01-288 Best Practices

problems have occurred during development when programs were 

launched without this match. 

We have reported on how a key resource of a developer—advanced 

technology—can and must be readied to meet product requirements at the 

time a product’s development program is launched.1 In this report, we 

address both sides of the match: how customer needs and product 

developer resources can be managed so that a product developer can 

predictably deliver a product the customer wants. 

How Product 

Requirements Are Set 

Is Key to Program 

Outcomes 

The decisions that are made in translating the ideas for a new product into 

actual features and characteristics dictate the amount of resources—

knowledge, time, money, and capacity—that will be necessary to bring the 

product to market. Thus, they may be the most highly leveraged of all 

product development decisions. A product’s requirements are based on 

customers’ expectations and justify the developer’s investment of 

resources to provide the desired capability. Requirements drive the amount 

of capital, time, expertise, and technologies the developer must invest. In 

the past, it has not been unusual for weapon system requirements to be set 

at such a high level that the initial estimate of the resources necessary to 

develop a responsive product proves insufficient, evidenced by cost growth 

and schedule slippage. The case to justify the requirements is often so 

stridently made that decisionmakers are in a relatively weak position to do 

anything other than find more resources. 

For commercial firms and DOD, the basic process for formulating a 

product’s requirements is the same. Each begins with understanding the 

customers’ expectations. These expectations are then translated into 

product requirements that include the job the product is to perform, the 

functions or characteristics it is to possess, the practicality it must have, 

and its reliability. Typically, the first understanding of customer 

expectations exceeds what the developer can do within available 

resources, because the developer has a limited amount of resources at its 

disposal for product development. On one hand is knowledge—the 

technology and capabilities the developer has to engineer and manufacture 

the product. On the other hand is the time and money the developer has to 

develop additional knowledge, if need be, and to design, build, test, and 

1

Best Practices: Better Management of Technology Development Can Improve Weapon 

System Outcomes (GAO/NSIAD-99-162, July 30, 1999).
Chapter 1

Introduction

Page 17 GAO-01-288 Best Practices

deliver the product. It is not unusual for a customer to want a 

high-performing product that does not cost much or take too long to 

develop. Such an expectation may exceed the developer’s technology or 

engineering expertise or may be more costly and time-consuming to create 

than the customer is willing to accept. The developer must stay within its 

means if the venture is to remain mutually beneficial. Table 1 characterizes 

the divergent interests of the customer and the product developer. 

Table 1: Customer and Product Developer Interests in a Product’s Development

Given these different interests, a customer’s wants and a product 

developer’s available resources must be matched to form an achievable set 

of product requirements. On one hand, the product developer must develop 

and produce the product within the time frames the customer needs or the 

customer may find an alternative product or source. On the other hand, the 

customer must not demand a product that requires so much money or time 

to develop that it cannot be afforded or delivered when needed. There is a 

delicate balance that must be achieved between these two divergent 

interests before a product can be successfully developed and produced.

On all product developments, there is an attempt to match expectations 

with available resources to define the new product. A customer’s 

expectations and a product developer’s resources are more closely 

scrutinized during the matching process that attempts to bring the two 

together. The outcome is a set of product requirements that represent an

Customer wants Product developer’s resources

Performance: what the product should do. 

For example, what an aircraft’s speed, 

range, fuel economy, reliability, and other 

features should be.

Technology: the technology that is needed 

to make the product function to a level 

necessary to meet the customer’s wants.

Timing: when the customer wants the 

product. 

Schedule: the amount of time that is 

needed to develop, design, test, and 

manufacture the product.

Pricing: what the product will cost. The 

customer must be able to afford the product.

Investment funds: the capital that is needed 

to pay for development, test, and 

manufacture of the product until revenue 

from sales begins.

Expertise: the capabilities of the product 

developer in terms of engineering expertise, 

manufacturing capabilities, and production.
Chapter 1

Introduction

Page 18 GAO-01-288 Best Practices

agreement that the product will meet the customer’s wants and that the 

developer can deliver the product within acceptable cost and schedule 

estimates. The requirements then guide the development program. This 

basic requirements setting process is illustrated in figure 2.

Figure 2: The Requirements Process

The process of translating general customer expectations into a specific set 

of product requirements involves information gathering, analysis, 

negotiation, and agreement. In the commercial process, the customer and 

the product developer negotiate requirements, matching expectations and 

available resources into a documented set of product requirements prior to 

committing resources to product development. During this negotiation, the 

customer’s relatively unconstrained wants are often reduced to a set of 

performance characteristics that are achievable with available resources, 

yet still meet the customer’s needs. The commercial process is a two-way 

communication between the customer and the product developer. For 

example, an airline company may want a certain speed to maximize 

revenue per passenger mile from a new aircraft. However, the product 

developer may determine that the resources to develop an aircraft with that 

speed are not available or must be increased dramatically. Both parties 

then work through an iterative process of trades and negotiation to settle 

on an aircraft with mutually acceptable performance and resource 

requirements. 

The DOD process is somewhat more complex and involves 

communications among at least four major players. On one end is the 

customer, which is normally a military organization that belongs to a major 

fighting force. On the other end is the product developer, usually a defense 

firm that serves as the prime contractor for developing and producing the 

Process to match

expectations with resources Product requirements

Customer expectations

Product developer resources
Chapter 1

Introduction

Page 19 GAO-01-288 Best Practices

weapon system. In between are two other players that actually negotiate 

needs and resources to arrive at product requirements. One is referred to 

as the user representative, which is an organization separate from the 

customer but represents the customer and negotiates on its behalf. The 

other player is the DOD program manager, a separate organization that, in 

essence, represents the product developer. Figure 3 illustrates how these 

different players interact in commercial and DOD requirement-setting 

processes.

Figure 3: Commercial and DOD Organizations Involved in Requirements Setting Processes

Both commercial and defense organizations are concerned about how 

much a product or weapon system is going to cost, how long it is going to 

take to build, what resources will be needed to build and maintain it, and 

whether it works properly. All of these concerns are translated into the 

product’s requirements. Unlike the commercial process, the DOD product 

developer does not directly influence the product requirement prior to 

launching product development. Once requirements are formalized in what 

Commercial example: requirement for a new airliner

DOD example: requirement for a fighter aircraft

An airline company that has a market

for a passenger aircraft that can carry

a specific number of passengers over

a specific distance efficiently and

reliably.

Customer Product developer

An aircraft manufacturing firm that is responsible both for developing requirements and the product

itself. The firm assigns one organization, such as a marketing team, to work with the customer to

understand and define needs. The firm then has this team work with a design team - which

understands the firms knowledge, capabilities, and resources - to translate customer wants into a set

of achievable product requirements that can be designed and produced within resources.

Customer

Members of the

warfighting community,

like fighter wings, that use

weapons to perform

combat missions. They

understand what it takes to

perform a given mission.

Members of the

requirements community

that analyze current

deficiencies, decide what

capabilities fighter wings

will need in the future,

and translate them

into requirements.

Customer

representative

Members of the acquisition

community that work with

the customer representative

to translate requirements

into an initial fighter

aircraft design used to

estimate resources needed

for product development.

Government

program manager

Defense firm that signs

a contract with the

program manager to

design and build

the product.

Product

developer
Chapter 1

Introduction

Page 20 GAO-01-288 Best Practices

DOD refers to as the Operational Requirements Document, they are turned 

over to the prime contractor, who actually begins product development. 

Objectives, Scope, and 

Methodology

The Chairman and the Ranking Member, Subcommittee on Readiness and 

Management Support, Senate Committee on Armed Services, requested 

that we examine various aspects of the acquisition process to determine 

whether the application of best practices can improve program outcomes. 

To date, we have issued reports on advanced quality concepts, earned value 

management, management of a product from development to production, 

management of key suppliers, management of technology insertion, 

training, and management of test and evaluation (see related GAO 

products). 

This report covers the beginning of the acquisition process: the 

management of product requirements. Our overall objective was to 

determine whether best practices offer methods to improve the way DOD 

sets product requirements within the framework of a knowledge-based 

product development process. Specifically, we assessed (1) the effect of 

the timing of the match between the customer’s needs and the developer’s 

resources on a product’s cost and schedule; (2) the best practices for 

obtaining this match during the requirements setting process, compared 

with more traditional DOD practices; and (3) the progress made and 

challenges DOD faces in adopting best practices for setting requirements 

on individual weapon systems.

We follow a similar overall methodology for conducting best practices 

reviews in the area of weapon system development. We start by identifying 

individual aspects of weapon system development—in this report, the 

setting of requirements—that have been shown to be a significant and 

recurrent cause of problems. Our sources for such information include our 

many reviews of individual weapon systems; studies from other sources, 

such as the Defense Science Board; and discussions with defense experts, 

including past and current DOD officials, defense industry representatives, 

and analysts from private organizations that study defense issues. Before 

beginning a review of a particular topic, we confirm with DOD officials that 

the topic is one in which the potential for improvement is significant. Once 

we have identified the topic, we use a case study approach because case 

studies provide the in-depth knowledge needed to understand individual 

practices, how they affect program outcomes, and why they are adopted. In 

selecting case studies, we look for examples of excellent practices from 

leading commercial firms, examples of typical or traditional practices from 
Chapter 1

Introduction

Page 21 GAO-01-288 Best Practices

DOD, and where possible, DOD examples that exhibit excellent practices. 

In making our selections, we are careful to make sure that there is a link 

between the practices themselves and the outcomes of the programs.

To identify best practices for setting requirements on commercial products, 

we reviewed literature and spoke with industry and academic experts to 

find companies recognized for managing requirements to help deliver new 

products that were both quicker to market and more advanced than their 

predecessors. We identified three companies 

• Caterpillar Construction and Mining Division, Decatur, Illinois;

• Bombardier Aerospace, Toronto, Canada; and

• Bethlehem Steel, Bethlehem, Pennsylvania

We visited each company, discussed the process used for setting 

requirements, and obtained an understanding of the overall process used 

with emphasis on those practices each felt were critical for success. We 

also met with individual program managers and discussed specific product 

development examples that further illustrated the process. During our 

discussions with the firms, we compared and contrasted the best practices 

with DOD’s practices. 

We developed nine case studies in total. These included two commercial 

case studies, one National Aeronautics and Space Administration (NASA) 

program that had received excellent results by disciplining its 

requirements-setting process, and six DOD weapon system programs that 

represented a mixture of traditional and best practices. We reviewed the 

requirements-setting process for all nine programs. For each program, we 

interviewed key managers and obtained documentation to determine 

(1) the process that was used to achieve the match between customer 

wants and resources to form the product’s requirements, (2) the timing of 

this match and the tools used to achieve it, and (3) the extent to which the 

requirements setting process affected the program’s product development 

outcome. Descriptions of the nine programs we reviewed follow.

Commercial

• Bombardier’s BRJ-X, a commercial jet in development, designed to carry 

between 88 to 110 passengers. It bridges the gap between the current 

fleet of regional jets, 20 to 70 passenger capacity and the larger 111 to 

170 passenger commercial airlines. The BRJ-X program was launched in 

the second quarter of 2000. The first aircraft is scheduled to be delivered 
Chapter 1

Introduction

Page 22 GAO-01-288 Best Practices

to airlines in late 2003. Bombardier estimates customers for 2,500 

aircraft over the next 20 years.

• The Caterpillar 797 mining truck, the largest mining truck ever built. It 

can carry over 360 tons of ore, and features many patented innovations. 

Developed in response to mining companies’ desire to reduce cost per 

ton of hauling ore in large-scale mining operations, Caterpillar launched 

the 797 program in 1997. 

DOD

• The Air Force’s Global Hawk, an unmanned aircraft that is intended to 

fly at altitudes as high as 65,000 feet and for as long as 40 hours to 

provide the Air Force with an intelligence, reconnaissance, and 

surveillance capability. The Air Force built five prototype technology 

demonstrators that were used to demonstrate the aircraft, and it plans 

to launch the product development program in 2001. 

• The Army’s Crusader artillery vehicle program, a self-propelled

155-millimeter howitzer and resupply vehicle. It is expected to be the 

first fully automated, computerized, and tracked artillery system. The 

Crusader development program began in 1994, and the howitzer is 

expected to start production in 2008. The development program is 

estimated to cost $4.3 billion.

• The Army’s Tactical Unmanned Aerial Vehicle, a short-range unmanned 

aircraft that is expected to provide the Army with day or night 

reconnaissance, surveillance, and target acquisition capability. The 

Army began development in March 1999. It plans to buy 44 systems 

starting in 2001. Each system includes three unmanned aircraft; a 

vehicle to carry the aircraft; two ground control stations mounted on 

vehicles; and launch, recovery, and support equipment pulled on trailers 

behind the vehicles. The cost to buy the 44 systems is estimated at 

$430 million through 2004.

• The Navy’s Radio Frequency Countermeasures system, an electronics 

warfare system that uses a jamming device called a techniques 

generator. It is carried onboard an aircraft to produce jamming signals 

that are transmitted by fiber optic cable to a towed device that acts as a 

decoy for the aircraft. The system is used to protect the aircraft from 

radar-controlled weapons like missiles and antiaircraft artillery. It is a 

critical component of the Integrated Defensive Electronics 

Countermeasure System being developed for some Navy and Air Force 

aircraft. System development began in 1995 and is expected to cost over 

$200 million. It is expected to enter production in 2002.
Chapter 1

Introduction

Page 23 GAO-01-288 Best Practices

• The Army’s Brilliant Anti-Armor Submunition program, referred to as 

BAT, an acoustic and infrared terminally guided submunition that 

searches for, detects, tracks and engages moving tanks and armored 

combat vehicles. Its mission is to provide deep attack against motorized 

rifle and tank divisions. The carrier for the submunition is the Army 

Tactical Missile System that is launched from the Multiple Launch 

Rocket System. The Army plans to buy 15,707 submunitions at a cost of 

$2.5 billion. 

• The Army’s Comanche helicopter, a lightweight, twin engine, stealthy 

helicopter that is intended to replace the Army’s OH-58 and AH-1 

helicopters. The primary mission of the aircraft will be armed 

reconnaissance and attack. It is the Army’s largest aviation acquisition 

program, with a projected total development and production cost of

$48 billion for 1,213 helicopters. The development program was 

launched in 1988, and production is expected to start in 2006. 

NASA

• The Far Ultraviolet Spectroscopic Explorer (FUSE), a scientific 

telescope that is used for studying the origin and evolution of stars, 

galaxies, and planetary systems. The telescope is 18 feet tall and weighs 

3,000 pounds. It was developed for NASA by the John Hopkins 

University. The FUSE program began in June 1996 and was completed in 

June 1999 at a cost of $120 million.

We used information from our prior best practices work, including most of 

the information on the BAT and Comanche programs. Similarly, we 

gathered knowledge about many aspects of the product development 

processes, including the setting of requirements, from leading commercial 

firms in addition to the firms included in this report. During the past

4 years, we have gathered information on product development practices 

from 3M, Boeing Airplane Company, Chrysler Corporation, Ford Motor 

Company, Hughes Space and Communications, and Motorola. This 

information enabled us to develop an overall model to describe the general 

approach leading commercial firms take to developing new products. 

We also met with experts in the area of setting product requirements from 

academia and participated in conferences and workshops with recognized 

leaders in the acquisition field to obtain information on how organizations 

were improving their acquisition processes. To obtain a general 

understanding of DOD’s requirements setting process and improvement 

initiatives, we met with officials from the Office of the Secretary of 
Chapter 1

Introduction

Page 24 GAO-01-288 Best Practices

Defense; Army, Navy, and Air Force Headquarters; and the Joint Staff for 

the Joint Chiefs of Staff. We also had discussions with former DOD officials 

and industry experts about DOD acquisition policies and practices. With 

these officials we discussed the current process, initiatives and the 

applicability of best practices to DOD operations. In addition, we visited 

NASA to obtain information on their processes and practices for setting 

requirements for new product development programs. 

We conducted our review from November 1999 through December 2000 in 

accordance with generally accepted government auditing standards.
Page 25 GAO-01-288 Best Practices

Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product Development 

Outcomes Chapte2r

The point in time that a developer for a new product becomes justifiably 

confident that it has the resources—knowledge, capacity, time, and 

money—to develop a product that a customer wants is critical to the 

success of the development effort. Barring program cancellation, the match 

between resources and wants is eventually met on just about every product 

or weapon system development. A key distinction between successful 

products—those that perform as expected and are developed within 

estimated resources—and problematic products is when this match is 

achieved. Simply put, we found that when wants and resources were 

matched before a product development was started, the more likely the 

development was able to meet performance, cost, and schedule objectives. 

When this match took place later, programs encountered problems such as 

increased costs, schedule delays, and performance shortfalls. 

For successful product or weapon system development program cases, 

trade-offs were made either in the design of the product or in the 

customer’s expectations to avoid immature technologies or exotic 

components that threatened to outstrip the developer’s resources. In the 

less successful cases, the opportunity to make such trade-offs before 

starting product development was missed either because gaps between 

expectations and resources were not identified or because customers were 

unwilling to reduce their expectations. When the divergence between 

customer expectations—which by then had become firm product 

requirements—and developer resources was recognized and confronted, 

decisionmakers were reluctant to materially change the requirements. 

Consequently, the developer had to invest more resources than originally 

planned to meet the requirements. 

Early Matching of 

Customer 

Expectations and 

Developer’s Resources 

Is Critical to Program 

Success

We found that the timing of the matching process—when the customer’s 

expectations were successfully matched with the product developer’s 

resources—significantly influenced the likely success of a product’s 

development. After reviewing the process for defining product 

requirements for nine development programs, we found a relationship 

between when expectations were matched with available resources and 

when the cost and schedule predictions for the programs were achieved. 

This relationship is shown in table 2. 
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 26 GAO-01-288 Best Practices

Table 2: Matching of Expectations to Resources and Product Development Outcomes

a

 Specific cost and schedule data for the Tactical Unmanned Aerial Vehicle, Global Hawk Unmanned 

Aerial Vehicle, and Bombardier BRJ-X Regional jet were not included in the table because they had 

not been in the product development phase long enough to report actual cost and schedule variances. 

However, these programs had already avoided some of the problems experienced by the programs 

that did not match expectations and resources before launch. These programs were on target for 

meeting their objectives.

The more successful programs had matches before the commitment to 

launch the programs was made. In each case, the product developer had 

done the initial design of the product and ensured that only proven 

technologies, design features, and production processes would be used. 

This was accomplished by either making additional investments to 

demonstrate uncertainties such as new technology or by reducing the 

product’s initial performance requirements. These steps maximized the 

knowledge content of the product and enabled the program manager to set 

cost and schedule estimates it could reasonably expect to meet. In 

contrast, the programs that did not have matches before launch did so 

during product development by the unplanned addition of resources. This 

contributed significantly to cost and schedule problems. Figure 4 illustrates 

the timing of the match between customer’s expectations and a product 

developer’s resources.

Programs 

Expectations and resources 

adequately matched before 

launch

Product development 

cost growth

Product development 

schedule delays

Caterpillar 797 mining truck Yes 5 percent 0 percent

NASA FUSE Yes 20 percent 0 percent

Radio Frequency Countermeasures 

system

No 197 percent 23 percent

Crusader artillery vehicle No 55 percent 26 percent

Comanche helicopter No 127 percent 119 percent

Brilliant Anti-armor Submunition No 99 percent 46 percent

Bombardier BRJ-Xa Yes On target On target

Tactical Unmanned Aerial Vehiclea Yes On target On target

Global Hawk Unmanned Vehiclea Yes On target On target
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 27 GAO-01-288 Best Practices

Figure 4: Timing of Match Between Customer Expectations and Resources

The programs in which expectations and resources were matched before 

product development started were in a good position to commit to cost and 

schedule estimates that were attainable. In those cases where expectations 

and resources were not matched before launch, cost and schedule 

estimates had to be made at the time of launch. Such estimates were 

necessarily made at levels consistent with the resources the product 

developer had available, under the assumption that either (1) no gaps 

existed between expectations and resources or (2) any gaps could be 

closed within projected resources. Because customer expectations 

regarding the performance of the product tended to become set when 

product development began, adding resources emerged as the primary 

option available to match expectations and resources. These resources 

(time and money) were typically needed for maturing technologies, 

developing design solutions, and providing more production capabilities. 

Perhaps more importantly, they were not estimated or planned for and 

often necessitated sacrifices in other needs, such as reducing the resources 

of other development programs.

Launch point

Successful cases had

matched customer

expectations with

developer’s resources

Customer

expectations

Product developer

resources

Problematic cases matched

later in product

development by investing

additional resources
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 28 GAO-01-288 Best Practices

Trade-offs Are Critical 

to Matching Customer 

Expectations With 

Developer Resources 

Before Starting 

Product Development 

Caterpillar and Bombardier both matched customer expectations with 

available resources prior to setting product requirements and launching the 

products’ development programs. In each case, there were differences 

between expectations and resources that necessitated trade-offs before 

requirements could be set and the product’s development could be 

launched. For these cases, expectations and resources were negotiated so 

that the product’s requirements could be achieved within available 

resources while still meeting the customers’ critical needs. This allowed 

the firms to develop and deliver their products quickly and within 

acceptable cost limits, thereby maintaining their competitive advantage in 

their respective markets.

Caterpillar’s 797 Mining 

Truck

To maintain the competitive advantage in its market, Caterpillar’s board of 

directors believed it had to develop by the end of 1998 a new product that 

was sized to work efficiently with large loading shovels used in mining 

operations. This meant that Caterpillar’s product development team would 

have about 18 months from the time the product development program was 

approved to develop and field the 797, a newly designed truck that could 

efficiently haul at least 360 tons of payload. According to Caterpillar, they 

met this date because they made trade-offs between the customers’ 

expectations and the resources available before the product development 

program began. Figure 5 shows the 797 mining truck developed and

figure 6 shows the history of the truck development.
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 29 GAO-01-288 Best Practices

Figure 5: Caterpillar Mining Truck

Caterpillar developed the 797, a 360-ton capacity mining truck, in 18 months. 

Source: Caterpillar, Inc.

Figure 6: Timeline for Development of the 797 Mining Truck

1996

Caterpillar determined

that there is a customer

need for ultra class

truck.

Start of product

development delayed

because requirements did

not match resources.

Trade-offs needed to

achieve18-month schedule.

Program launched after

requirements that could

not be met with available

technologies were

deferred to future

versions of the 797.

Caterpillar met the

18-month schedule and

was within 5 percent

of its cost objective.

February

1997

April

1997

December

1998
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 30 GAO-01-288 Best Practices

Examples of key trade-offs Caterpillar made to close gaps between 

customer expectations and its own resources were: 

• deferring to the next product line, new prognostic technologies wanted 

by the customer that could assist in forecasting wear and tear to the 

truck, but were immature at the time product development started;

• selecting a twin-engine propulsion design to power the 797 rather than a 

single engine—despite its potential for lower operating costs—because 

it had not yet been developed and was therefore too much of a risk; and

• redesigning the wheel and transmission to avoid the need to develop 

new gears for the differential unit in the drivetrain, which avoided a 

costly and risky development effort that could have impacted the truck’s 

development progress. 

The Bombardier BRJ-X Jet During the BRJ-X jet’s concept definition phase, which preceded product 

development, Bombardier identified some customer expectations that its 

designers believed would put program cost and schedule at risk. The 

expectations were analyzed and trade-offs were made to make the design 

achievable within resources. For example, both airline customers and 

Bombardier wanted to use fly-by-wire flight control technology, which 

would replace heavy hydromechanical flight controls with lighter weight 

electronic controls, on the BRJ-X to reduce weight and lower fuel costs. 

However, Bombardier engineers had some concerns about the technology 

since it had never been integrated into a regional jet configuration before. 

Consequently, Bombardier decided to invest the time and money to 

demonstrate the technology on a surrogate business jet before making it a 

program requirement. Another trade-off was made after Bombardier 

determined that the desired speed would require design effort and features 

not fully proven out. The customers agreed that a slight reduction in speed 

would eliminate the design concerns yet still meet their expectations.

NASA’s FUSE Program FUSE is a telescope used to study the origin and evolution of certain 

elements in space to help determine the age and evolution of stars, 

galaxies, and planetary systems. The customer initially expressed 

expectations in terms of gathering precise images of light from these 

elements in space 24 hours per day. When NASA threatened to cancel the 

program because requirements were not achievable given available 

resources, FUSE program managers negotiated with customers and found 

they could still meet the basic expectations with reduced requirements for 
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 31 GAO-01-288 Best Practices

a high-resolution mirror, bandwidth detection, and time on orbit; switching 

from a highly elliptical orbit to a low earth orbit (see fig. 7). 

Figure 7: Effect of FUSE Trade-offs on Matching Customer Expectations with Developer’s Resources

These reductions in requirements allowed the use of existing technologies 

such as a new grating technique to spread radiation into different 

wavelengths. Matching requirements to resources allowed FUSE to not 

only meet schedule targets and be within 20 percent of its cost objective 

but also a critical NASA need.

When Matching Did 

Not Occur Before 

Program Launch, 

Developers Were 

Forced to Add 

Unplanned Resources 

In cases where the customers’ expectations were not matched with the 

developers’ resources before product development began, the matches 

took place after opportunities to make trade-offs had passed. Because new 

programs had been approved with customer wants formally documented as 

requirements, the main avenue available to close gaps between 

requirements and resources was for the developers to invest more effort, 

time, and money to gain the knowledge and capacity needed to meet the 

requirements. This additional investment—manifested by cost increases 

and schedule delays—was not planned, which forced trade-offs or cuts to 

be made in other programs to free up the additional resources. In some 

cases, the developers may have had indications that there was a mismatch 

at the launch decision, but they were pressured to go forward anyway. In 

other cases, while the developers may not have enough knowledge to be 

confident that there was a match before launch, they were at a 

disadvantage to argue that there was not. DOD’s Radio Frequency 

Countermeasures system and Comanche helicopter programs did not have 

matches before their product development programs were launched. In 

Customer’s expectations Lower mirror resolution

Reduced bandwidth

Less time on orbit

Developer’s resources

Match achieved Program start
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 32 GAO-01-288 Best Practices

fact, despite being several years into both programs, it is still uncertain 

whether this match has been reached. 

Navy’s Radio Frequency 

Countermeasures System 

Customer expectations and developer resources were not matched on the 

Radio Frequency Countermeasures program when product development 

began in 1995. Essentially, the performance wanted by the customers 

exceeded the time, money, and technologies available to the product 

developer to develop an acceptable system. For example, a critical 

component is the fiber-optic towed decoy—a component, towed behind the 

aircraft in flight, which transmits electronic countermeasures. Before 

program launch, the requirement for the power to be transmitted from the 

towed decoy was nearly tripled because of a last minute addition to satisfy 

the Air Force’s expectations for the F-15 fighter and the B-1 bomber. This 

was a demanding requirement that, according to the former program 

manager, required some technological invention. According to the manager, 

this was complicated by the customer’s refusal to either reduce 

performance requirements or accept increases in time and cost to develop 

a product that would meet these requirements. The Navy chose to launch 

the program with this mismatch, recognizing that it added risk to the 

program. Figure 8 shows the timeline for the program.

Figure 8: Navy’s Development of the Radio Frequency Countermeasures System

As the program proceeded, problems associated with the risk 

accompanying the mismatch turned into development problems, and the 

additional resources found to be unacceptable to the customer before 

launch were accepted as a necessity after launch. Consequently, the costs 

Developer proposes

additional resources

F-15 and B-1

requirements added

Customer’s

expectations

Developer’s

resources

Proposal denied

by customer -

program

launched without

match

Program start

Developer reports

197% cost increase and

15-month schedule delay
Chapter 2

Timely Matching of Requirements and 

Resources Is Critical to Product 

Development Outcomes

Page 33 GAO-01-288 Best Practices

rose from $74 million to $221 million and the development schedule was 

extended 15 months.

Comanche Helicopter Customer expectations that became key requirements for the Comanche 

helicopter demanded several technologies that were still very immature 

when the Army decided to launch the program in 1988. For example, the 

integrated avionics and an advanced infrared night vision and targeting 

sensor were included on the program when they were still conceptual in 

nature. These advanced avionics systems were needed to meet the 

customer’s “must-have” requirements for a very lightweight, stealthy, highly 

maneuverable, all-weather reconnaissance and attack helicopter. These 

technologies were also critical to meet cost and weight goals for the 

program. The Army launched the program despite the low readiness of the 

technologies, with the developer having limited design alternatives but 

believing that the needed technological invention could be accomplished 

within projected resources. At the time of launch, the Army estimated that 

product development would cost $3.6 billion and last about 8 years. Due to 

problems that developed with these technologies and budgeting and other 

changes in the program, development is now estimated to take $8.3 billion 

and 18 years—over 100-percent increases. The Army kept the customer’s 

requirements essentially unchanged, electing to double the resources 

needed to meet them.
Page 34 GAO-01-288 Best Practices

Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch Chapte3r

The ability to match a customer’s wants with resources before launching a 

development program is key to putting program managers in a better 

position to succeed. We found three factors that comprise this ability. First, 

developers employed systems engineering to identify gaps between 

resources and customer wants before committing to a new product 

development. Second, customers and product developers were flexible 

before launch. Leeway existed to reduce expectations, defer them to future 

programs, or to invest more resources up front to eliminate gaps between 

resources and expectations. Third, the roles and responsibilities of the 

customer and the product developer were balanced, with the product 

developer given the responsibility to determine or significantly influence 

product requirements. In cases where these factors were not present at 

program launch, product development began with imbalanced product 

requirements. Invariably, this imbalance favored meeting expectations at 

the expense of resources, putting the developers at a disadvantage to 

deliver the products within cost and schedule estimates. 

In the most successful cases, the effective interplay of these factors 

allowed the customers and product developers to arrive at a set of product 

requirements that could be developed within cost and schedule targets. 

Systems engineering provided knowledge necessary to translate customer 

wants into specific capabilities, enabling the developers to identify and 

resolve gaps before product development began. With systems engineering 

knowledge in hand, flexible requirements were essential to lowering risk 

through negotiations because knowledge alone did not produce trade-offs. 

Absent such flexibility, resources and wants could still be matched before 

product development began, but the options to resolving any gaps were 

limited to additional investments on the developers’ part. Finally, with 

knowledge gained from systems engineering and flexible requirements as 

preconditions, successful product development programs benefited from 

an environment in which product developers and customers had balanced 

roles and shared responsibilities for setting product requirements. 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 35 GAO-01-288 Best Practices

Systems Engineering 

Tools Are Critical for 

Identifying Gaps 

Between Developer’s 

Resources and 

Customer’s 

Expectations

When product developers employed systems engineering before 

committing to product development, they were able to identify areas in 

which the customers’ wants exceeded their resources. For those cases in 

which developers did not conduct sufficient systems engineering before 

committing to the new product development, they were weakened in their 

ability to identify gaps between their resources and expectations. These 

gaps were later revealed as unexpected problems that required invention, 

time, and money to resolve. 

Systems engineering is a process that not only translates customer wants 

into specific capabilities, such as individual technologies and 

manufacturing processes, but also provides knowledge that enables a 

developer to identify and resolve gaps before product development begins. 

It is defined as a logical sequence of activities that transforms a customer 

want into specific product characteristics and functions and ultimately into 

a preferred design (see fig. 9). It is not necessarily the use of systems 

engineering in the development of a new product or weapon system, but 

when it is used that distinguishes it as a best practice. 

Figure 9: Basic Steps in Systems Engineering Process

The systems engineering discipline enables the product developer to 

translate customer wants into specific product features for which requisite 

technological, software, engineering, and production capabilities can be 

identified. Once these capabilities are identified, a developer can assess its 

own capabilities to determine if gaps exist. It is critical for a developer to 

involve the right people—those with the affected areas of expertise—in 

this assessment. Gaps identified between what the customer’s wants are 

and what the developer possesses then become the focus of analysis. Some 

gaps can be resolved by investments the developer makes, while others can 

Definition of customer

wants including

planned use, operating

environment, and

performance

characteristics.

Requirements

Analysis

Decomposition

of the requirements

into a set of specific

functions that the

system must perform.

Functional Analysis

and Allocation

Identification of the

technical and design

solutions needed to

meet the required

functions.

Design Synthesis

Product

Design
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 36 GAO-01-288 Best Practices

be closed by finding technical or design alternatives. Remaining gaps—

those that represent capabilities the developer does not have or cannot get 

without increasing the price and timing of the product beyond what the 

customer will accept—must be resolved through trade-offs and 

negotiation.

During systems engineering, a product design progresses through at least 

three iterations. The first is a notional design—a general concept of what 

the product will look like and what it might be capable of that is 

unconstrained by resources. The second iteration is the first detailed 

design that enables a developer to compare its capabilities with the 

demands of the product. The third iteration is the final design, which 

captures improvements to the design generated by testing, analysis, and 

other forms of learning. These iterations can be seen in figure 10, which 

provides a general comparison of the amount of systems engineering 

accomplished in the successful and problematic cases prior to launching a 

product development program. 

Figure 10: Systems Engineering Used for Successful and Problematic Cases

Successful Programs Used 

Systems Engineering Before 

Product Development 

Began 

In each successful case, the product developer worked closely with the 

customer to understand its wants, which were often articulated in a 

bottom-line metric, such as cost per passenger mile for a commercial 

airplane. By the time the developer had committed to a product 

development program, it was well into its systems engineering process and 

had developed a preliminary design of the product. This process identified 

the gaps between resources and expectations, which could be then 

addressed through investments, alternate designs, and, ultimately, 

trade-offs. The knowledge produced by the process put the developer in a 

Program Launch

Problematic

Cases

Successful

Cases

Preliminary

design

Notional design Detailed systems engineering

Detailed systems engineering

Final design

Preliminary design

Final design
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 37 GAO-01-288 Best Practices

good position to negotiate with the customer because consequences could 

be associated with attempts to meet those wants that exceeded the 

developer’s capabilities. The process also involved the customer through 

periodic reviews and acceptance of the product’s final design. According to 

some commercial representatives, systems engineering is a good 

investment to reduce risk, usually comprising a small percentage to the 

overall development cost of a new product. We found two commercial 

firms—Caterpillar and Bombardier Aerospace—and two DOD programs—

the Army’s Tactical Unmanned Aerial Vehicle and the Air Force’s Global 

Hawk unmanned aerial vehicle—that employed fairly extensive systems 

engineering before they committed to product development. 

Caterpillar’s 797 Mining Truck Caterpillar’s New Product Introduction process calls for completing a 

significant amount of the systems engineering process during the concept 

phase for any new product before product development. It applied this 

process to its development of the 797 mining truck, a new product design. 

It spent significant time and effort prior to beginning product development 

establishing the customers’ wants and closing the gap between them and 

available resources. Before committing to its development, Caterpillar 

gathered information from the mining companies about operating 

conditions and the cost per ton of ore hauled desired of the new truck that 

they would be willing to accept during the truck’s lifetime. Caterpillar then 

made a preliminary determination of what the truck’s performance 

requirements would have to be to meet these conditions and costs. For 

example, it determined that the truck’s payload, or bed, would have to haul 

at least 360 tons, travel at certain speeds, and climb certain grades. This 

information served as the starting point for systems engineering to 

determine if the performance requirements for the 797 were achievable 

given Caterpillar’s resources. Once these requirements were established, 

Caterpillar’s engineers began an iterative systems engineering process to 

design a product that would meet the customers’ wants and could be 

developed within funding, schedule, and resource targets. This process 

culminated in a specific product solution that matched requirements to 

resources before Caterpillar committed to product development. 

The systems engineering process for the 797 forced key trade-offs between 

performance requirements and design solutions prior to commitment to 

product development. For example, to transmit the power from the engine 

to the rear wheels, the original design called for using very large differential 

gears. Upon reviewing that design, an experienced Caterpillar production 

engineer noted that no gear manufacturer made a gear that large and that to 

create such a production capability would be risky. Consequently, the 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 38 GAO-01-288 Best Practices

design engineers found an alternative that called for making incremental 

changes in the transmission and wheel designs, which enabled existing 

differential gears to be used. These design changes and the processes 

required to make the changes were demonstrated successfully prior to 

product development. Systems engineering also revealed risk on some of 

the new hydraulic technology for the 797 that would allow easier and more 

reliable unloading. Because this new technology had not been used in the 

field before, Caterpillar engineers demanded demonstrations and field tests 

of the technology before allowing it onto the 797 truck.

Bombardier’s New Regional Jet Bombardier set requirements for a new, larger family of regional jets that 

will carry up to 115 passengers—the BRJ-X series—using its Bombardier 

Engineering System. Beginning with an overall cost-per-passenger-mile 

target from the customer, Bombardier employed an iterative systems 

engineering process to make trade-offs between performance 

requirements—which were based on extensive market analysis—and 

available resources to arrive at requirements, design, and cost and schedule 

targets before committing to product development. The company plans to 

achieve nearly full knowledge of the product’s design before it commits to 

product development. At launch, the product’s performance requirements 

and its configuration will be frozen. 

During Bombardier’s trade study process, customer wants are thoroughly 

challenged by the firm’s assessment of technology capability and readiness, 

as well as by manufacturing, producibility, logistics, and other implications 

of the product’s features. During this process, engineers are cognizant that 

product success and profitability hinge on not only product features but 

also product price and quantity that market research has determined. This 

fact tempers decisions about the viability of key performance or design 

parameters. In other words, if Bombardier cannot produce and develop a 

product within a customer’s price range, then the customer is not likely to 

buy it. Once this process is complete, Bombardier performs a detailed 

aircraft review, which results in a trade-off analysis. If this analysis 

suggests that key performance requirements—speed or range, for 

example—cannot be achieved within the established price and quantity, the 

requirements are changed. If product performance is reduced below the 

customer’s minimum threshold, profit calculations are upset and the 

development program is not initiated. 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 39 GAO-01-288 Best Practices

Army’s Tactical Unmanned 

Aerial Vehicle

The Army’s Tactical Unmanned Aerial Vehicle is a DOD program that used 

systems engineering to gather knowledge prior to starting development. 

The vehicle must meet the brigade commanders’ need for a day or night, 

adverse weather, multisensor data collection system with improved 

communication with joint forces that provides real-time battle information 

and cannot be observed by the enemy. The program grew out of an 

Advanced Concept Technology Demonstration1

 program that was started in 

1996 and completed in 1998. While the air vehicle that was flown and 

evaluated during this demonstration met most of the Army’s close-range 

reconnaissance requirements, it did not meet all requirements and was 

canceled. Figure 11 shows the Army’s Tactical Unmanned Aerial Vehicle.

1

 Advanced Concept Technology Demonstrations attempt to bring mature technologies 

together to meet an existing military need. They are managed by science and technology 

organizations and must demonstrate military utility before they can become acquisition 

programs.
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 40 GAO-01-288 Best Practices

Figure 11: Tactical Unmanned Aerial Vehicle

The Army used systems engineering to transform customer expectations into a set of achievable 

product requirements prior to program launch.

The technology demonstration provided the Army with knowledge about 

the achievability of the customers’ requirements. The product developer 

built a prototype system that was used to identify gaps between what the 

customer wanted and what was achievable with resources. Through 

systems engineering, the developer transformed the wants into a set of 

performance requirements that led to the prototype’s design. The Army 

used the results of the technology demonstration to define the customer’s 

requirements that were geared toward obtaining a system that required 

minimal development, based largely on what was demonstrated. Because 

of the knowledge gained from systems engineering, the Army was able to 

stage a fly-off of four prototype designs within 9 months after the product 

development program began. The fly-off results provided additional 

knowledge to help match customer wants with the product developer’s 

resources through further trades. The Tactical Unmanned Aerial Vehicle 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 41 GAO-01-288 Best Practices

that is now in development closely resembles the design that won that 

competition. Another program, the Air Force’s Global Hawk, followed a 

similar path by conducting systems engineering and building prototype 

systems prior to starting its program. 

Problems Arose When Only 

Limited Systems 

Engineering Was Done Prior 

to Launch

In the problematic cases, product developers had not progressed as far into 

the systems engineering process at the time the program was launched. In 

most of these cases, the product developers had only notional designs at 

that point—not thorough enough to translate expectations into specific 

functions against which resources could be compared. It was not until 

product development was underway that systems engineering was fully 

employed to create a preliminary product design. In each case, this 

occurred several years after the acquisition program was started. The lack 

of knowledge at the start of each program made well-informed trades 

between customer wants and developer resources difficult to see or make. 

Nonetheless, cost and schedule targets for each program were set based on 

available information. Problems that were discovered during product 

development and during the systems engineering process often resulted in 

the need for more time or money than had been estimated at program 

launch. 

Two DOD programs we reviewed—the Radio Frequency Countermeasures 

system and the Crusader program—initiated a systems engineering process 

before product development; however, it was not extensive and performed 

by the eventual product developer hired to design the product and only 

resulted in a notional product design prior to the decision to commit to 

product development. The product developers did not gain significant 

knowledge about what was possible, given the availability of resources, 

until after the decision was made to commit the resources toward 

developing the product. 

Navy’s Radio Frequency 

Countermeasures System 

The majority of the systems engineering by the Radio Frequency 

Countermeasures product developer—the defense firm that was awarded 

the development contract—was not done until after the Navy had launched 

the program. The knowledge used to match requirements with funding, 

schedule, and other resources was limited before the program was 

launched. The Navy did enough analysis—about 25 percent of systems 

engineering, according to the former program manager—to form a notional 

design of the system. The notional design was based on a top-level analysis 

of the functions, which was done primarily by the Navy’s program office. 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 42 GAO-01-288 Best Practices

The Navy nonetheless considered this amount of knowledge sufficient to 

commit to product development. 

The Navy entered product development assuming that a large number of 

the needed parts would be nondevelopmental items—items already used 

on products. The Navy estimated that 90 percent of the parts for one of the 

most critical subsystems, the techniques generator,2

 would be mature, 

readily available items. According to the program manager, it was not until 

the contractor hired to develop the system began the detailed systems 

engineering process that it discovered that only about 50 percent of those 

parts were still available. Many of the required parts had become obsolete 

and had to be replaced with redesigned or newly developed parts—

revealing a gap between what was thought to be within the developer’s 

capabilities and what the requirements were. As a result, problems with the 

assumptions made about the notional design by the Navy were discovered 

in product development, after resources had been determined and the 

fielding date had been established. This resulted in significant disruptions 

to planned cost and schedule; eventually affecting the product’s fielding. 

Army’s Crusader Artillery Vehicle The Crusader artillery vehicle is another case in which systems engineering 

was not performed by the product developer, to a large extent, until after 

the acquisition program had been launched. The development program was 

launched based on a notional design done by the Army’s program office, 

not extensive systems engineering done by the contractor that would 

design and build the product. Key to this notional design was the use of a 

liquid propellant—new technology—for firing weapon projectiles. Program 

officials stated that the optimal range that the Crusader is required to fire a 

weapon is still based on the use of this liquid propellant. The Army 

assessed various aspects of the risk of developing the liquid propellant 

technology and integrating it into the weapon system between low and 

moderately high. Nevertheless, on the basis of the notional design, the 

Army committed to launching product development. 

After the program was launched in 1994, the product developer was 

awarded a contract to develop the Crusader. According to a program 

official, it took 2 years of systems engineering to determine if the 

requirements were feasible given established cost and schedule targets. In 

1996, the product developer determined that the liquid propellant 

2

 A jamming device carried onboard the aircraft produces jamming signals that are 

transmitted by fiber optic cable to the towed decoy for transmission.
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 43 GAO-01-288 Best Practices

technology was high risk in all aspects and that it would cost an additional 

$500 million to develop. This was much more than originally estimated at 

the start of the program and represented a major resource gap. As a result, 

the Army elected to use a more traditional, less capable back-up—a solid 

propellant—to achieve a match between available resources and 

requirements. This decision may impact system performance since, 

according to program officials, the Crusader will probably not achieve its 

optimal firing range. The firing range and the liquid propellant were part of 

the reason for launching the program in the first place. Figure 12 shows the 

Crusader artillery vehicle. 

Figure 12: Crusader Artillery Vehicle

It was not until 2 years after the Crusader development program was launched that the Army 

recognized a key technology could not be developed within available resources. 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 44 GAO-01-288 Best Practices

Flexibility in Setting 

Requirements Is Key to 

Closing Gaps Between 

Customer 

Expectations and 

Developer Resources 

While knowledge is essential to identifying gaps between expectations and 

resources, it takes flexibility on the part of both the customer and the 

product developer to close the gaps. Flexibility represents the customer’s 

ability and willingness to lower product expectations, coupled with the 

product developer’s willingness and ability to invest more resources to 

reduce technical risks and other gaps before program start. Flexibility, 

when informed by systems engineering knowledge prior to program 

launch, was essential to lowering risk and reducing the cost and length of 

product development because knowledge alone does not produce

trade-offs. Absent such flexibility, resources and wants can still be matched 

before starting product development, but the options to closing any gaps 

exposed by systems engineering are limited to additional investments on 

the developer’s part. The scenario with the most potential for costly 

problems is one in which neither the requirements are flexible nor 

sufficient systems engineering has been done to reveal resource gaps 

before launch. Several of the DOD programs we reviewed that had 

significant problems in meeting product development objectives followed 

this scenario. 

While there are instances in which flexibility on the part of the customer is 

inherent, such as for a unique product that has no predecessor, in most 

cases, potential customers for a new product already have expectations. In 

these cases, flexibility has to be fostered. We found two factors that 

fostered such flexibility before product development started. First, product 

development cycle times were limited. This forced product developers and 

customers to agree on requirements that were achievable within 

established time limits. Second, the developer enlisted the customer’s trust 

through an evolutionary approach to product development, which relieved 

the customer of the pressure to want all needs met in a single product 

iteration. These factors made customers more willing to defer 

requirements that demanded more time or unproven technologies for 

succeeding versions of the product. In contrast, when these factors were 

not present and trade-offs were not made, there was an implicit decision to 

accept the risk of not having the product or the capability when needed.

Requirements Were Flexible 

Until Program Launch on 

Successful Cases

In successful cases, requirements were flexible until the decision was made 

to commit to product development because both customers and developers 

wanted to limit cycle time. This made it acceptable to reduce, eliminate, or 

defer some customer wants so that the product’s requirements could be 

matched with the resources available to deliver the product within the 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 45 GAO-01-288 Best Practices

desired cycle time. The customer had incentives to trade off some wants, 

owing to the desire to get the product within cost and time predictions and 

to confidence that future versions of the product would meet many of those 

wants. The two commercial companies we visited exhibited these 

characteristics. In DOD, we found two programs that had exhibited this 

flexibility before launch.

Caterpillar’s 797 Mining Truck Because of mutual interest on the part of the product developer and the 

customer, Caterpillar’s board of director’s mandate for the new mining 

truck to be developed before the end of 1998 became a catalyst for 

flexibility in requirements and design decisions. For example, Caterpillar 

wanted to introduce new prognostic technology on the 797 that could 

significantly lower the cost of operating the truck—an expressed want of 

the customer. At the heart of this technology were monitoring sensors that 

could assist in forecasting wear and tear on the truck—such as on the 

powertrain, brakes, and tires—thereby allowing better maintenance 

management that would reduce how often components had to be replaced 

and increase the truck’s life. However, Caterpillar engineers concluded that 

there was not enough time to make this technology mature enough to be 

included in the initial design if the 18-month delivery schedule was to be 

met. Despite the desire to have the sensors on the 797, the customer 

understood the need to make a trade-off and was willing to do without the 

sensor on the initial version if it meant getting the truck on time. As a 

result, Caterpillar deferred the technology, despite the increased 

performance it offered, because it would put other program resources—

most importantly, schedule—at risk. The customer accepted this trade-off. 

The trade-off was made possible by Caterpillar’s recognition of the risk, the 

customer’s trust in the next generation of the truck, and the mutual desire 

to deliver the basic product on time and within cost. 

Bombardier’s New Regional Jet Bombardier’s goal is to limit cycle time on all product developments to

36 months. At the start of a new product’s development process, 

decisionmakers ask, “What can we accomplish within 36 months?” 

Bombardier representatives explained that this forces them to have 

significant knowledge about a new product’s cost and performance early in 

the process and is consistent with an evolutionary approach to product 

development. If a product’s requirements force them to consider longer 

development time frames, they must consider a future generation of the 

product or a new and different product. This time frame played a role in the 

decision to reduce the BRJ-X customer’s requirement for speed. The initial 

speed requirement, based on market research, was originally .81 mach. 

After subjecting the requirement to systems engineering, Bombardier’s 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 46 GAO-01-288 Best Practices

engineers deduced that, while speeds of up to .78 mach could be achieved 

with current off-the-shelf propulsion technology, new and undemonstrated 

propulsion technology would be required to achieve .81 mach. After gaining 

additional knowledge through wind tunnel testing, the engineers concluded 

that to design and develop the needed propulsion and aerodynamics would 

create significant risk for the 36-month product development time frame.

Before launching the BRJ-X development program, Bombardier’s 

marketing staff went back to the customers—over 30 airlines—and 

presented the information showing that the additional speed would require 

more development time and thus be more costly. On the strength of the 

analysis, the customers agreed that .78 mach would still meet their needs. 

In fact, they discovered that most European air traffic controllers would 

not allow the higher speed. Based on the agreement, Bombardier reduced 

the requirement to .78 mach, allowing the use of a low-risk propulsion 

system. The 36-month goal forced the developer and the customers to 

negotiate between the product’s requirements and resources, while 

systems engineering provided the knowledge so that informed trade-offs 

could be made. 

Air Force’s Global Hawk and 

Army’s Tactical Unmanned Aerial 

Vehicles

In two DOD cases that have met product development objectives so far —

the Air Force’s Global Hawk program and the Army’s Tactical Unmanned 

Aerial Vehicle program—senior acquisition executives placed resource 

constraints on the programs and advocated evolutionary acquisitions, 

which provided the customers and product developers the incentives and 

the opportunity to cooperate and make mutually beneficial trade-offs. The 

Tactical Unmanned Aerial Vehicle had a schedule for first delivery of a 

system within 1-1⁄2 years after launch and the Army established an 

evolutionary acquisition strategy for the product to meet this schedule. 

This mandate was driven by a mutual agreement between senior executives 

from the Army’s acquisition and requirement setting communities that the 

priority is to field it quickly. This agreement fostered a good relationship 

between the two communities that allowed a “no bells and whistles” 

approach to development. 

The product manager told us that using the guidance as a foundation, the 

customer agreed to collaborate on a basic list of key performance 

parameters as the only “must-have” requirements for the initial Tactical 

Unmanned Aerial Vehicle and define the remaining requirements as 

tradable against resources, such as cost and schedule. The requirements 

that were not key to the initial vehicle were grouped into three categories, 

each more important than the next, but none important enough to be added 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 47 GAO-01-288 Best Practices

to the initial vehicle if not achievable within the resources available to the 

developer. Product developers stated that several requirements would have 

been difficult to achieve without additional time and money to develop 

solutions. When they explained this to the customer and pointed out the 

associated high risks for achieving the requirements within the time 

frames, the customer was willing to defer these requirements to future 

versions of the system in order to meet the planned development schedule. 

The requirement for the Tactical Unmanned Aerial Vehicle’s imagery 

capabilities—that is, the electro-optic and infrared sensor systems that are 

used for observing targets—is a good example of this flexibility. The 

customer originally wanted the imagery to have at least a 90-percent 

probability of recognizing and identifying targets from an altitude of 6,000 

to 8,000 feet. On the basis of a systems engineering analysis, the product 

developer concluded that this was not achievable with the technology 

available within the 1-1⁄2 year time frame and that more time and money 

would be required to develop technology that would meet the capability. As 

a result, the customer reduced the requirement to 70 percent, which was 

both technically feasible and still useful to the customer, and made 

90 percent a goal that might be achieved in future generations.

A similar combination—a cost limitation and urgent customer need—led to 

flexible requirements on the Global Hawk unmanned aerial vehicle. A 

development cost limit set by DOD executives forced the customer and the 

program office to prioritize requirements into what could be achieved 

within the limit. Additionally, the customer, the United States Joint Forces 

Command, assessed the usefulness of the system and stated the basic 

capability tested during the Advanced Concept Technology Demonstration 

could provide utility today. This assessment increased the importance of 

quick delivery and forced the requirement setters and the product 

developer to look at an evolutionary approach to meeting their needs, 

planning for a basic capability in the first product line and more advanced 

capabilities added over time. Together, these constraints gave the program 

manager leverage in persuading the user to reduce or defer requirements. 

For example, the requirements manager wanted a more advanced imagery 

system in the first product line. However, the program manager pointed out 

to the requirements manager that the cost and time needed to include this 

capability would exceed the cost limit. In response, the requirements 

manager agreed to defer this capability to a later version of the product. 

Figure 13 shows the Global Hawk unmanned aerial vehicle.
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 48 GAO-01-288 Best Practices

Figure 13: Global Hawk Unmanned Aerial Vehicle

The Air Force has adopted an evolutionary acquisition strategy to match customer expectations with 

the developer’s resources, enabling quicker delivery of the Global Hawk.

Less Successful Cases 

Maintained Inflexible 

Requirements Prior to 

Launch

On the weapon system programs that did not meet development objectives, 

the products’ performance requirements were generally not flexible prior 

to launch—a condition compounded by the fact that systems engineering 

had not been done in time to reveal gaps between requirements and 

resources. The requirements demanded advanced technologies that were 

not yet proven or available. The matching of requirements with available 

resources was not done until well into the product development program. 

However, the requirements were not reduced. Instead, additional 

resources, including technologies, time, and money, had to be added. We 

have reported on the negative consequences of basing cost and schedule 

estimates on the hoped-for success of such technologies.3

 Without the 

countervailing presence of systems engineering knowledge from the 

3

Best Practices: Successful Application to Weapon Acquisitions Requires Changes in DOD’s 

Environment (GAO/NSIAD-98-56, Feb. 24, 1998). Best Practices: Better Management of 

Technology Development Can Improve Weapon System Outcomes (GAO/NSIAD-99-162, 

July 30, 1999).
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 49 GAO-01-288 Best Practices

product developer or the customer’s acceptance of an evolutionary product 

development approach, flexibility was difficult to engender. 

Army’s Comanche Helicopter In the Comanche program, initial requirements for attacking targets and 

performing other tasks, when combined with the desire for small size and 

light weight, called for a unique capability compared with existing 

helicopters. Meeting the size and weight requirements depended on new 

technologies such as advanced forward looking infrared and integrated 

avionics. Both size and weight were critical elements of a mission 

equipment package that was supposed to reduce the pilot’s workload while 

improving aircraft performance in the areas of communications, targeting 

range, and pilot capabilities. The program office stated that at program 

launch the technologies were still conceptual in nature, needing an 

investment in time and funding before they would be mature enough to 

satisfy requirements. As a result, Comanche was expected to provide a 

quantum leap in product performance using new technologies that were 

not fully understood at the time the program began.

Not only did the customer consider the requirements that made these 

technologies necessary as not tradable, they were also confined by weight 

and cost restrictions placed on the program. The cumulative effect of 

several competing requirements—low cost, reduced weight, long range, 

and increased lethality—resulted in solutions that relied on advanced 

technologies. For example, a more mature target sensing technology that 

might have met performance requirements was rejected because it weighed 

too much. The Army decided to launch the program despite the significant 

lack of knowledge about the needed technologies, leaving a mismatch 

between requirements and available resources, and chose to develop the 

new technologies during the product development program. This action 

placed the burden of maturing technologies onto the program manager 

during product development. Difficulties encountered maturing these 

technologies to meet unyielding user requirements contributed, in part, to 

the program’s significant cost and schedule increases. 

Army’s Crusader Artillery Vehicle The Army’s Crusader program also had competing requirements that—

when taken together—resulted in an inflexible system solution. 

Requirements for firing range, accuracy, rate of fire, lethality, and resupply 

remained inflexible prior to program launch. Similar to the Comanche 

program, this inflexibility limited the technical solutions that could meet 

the requirements and forced the program to be launched with immature 

technologies. Many of these technologies—such as the automated 

ammunition loading and handling system and the actively cooled cannon 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 50 GAO-01-288 Best Practices

barrel—were considered technological firsts for U.S. artillery systems. As a 

result, the Crusader program is expected to take over 14 years and cost 

over $4 billion to develop. 

Army’s Brilliant Anti-armor 

Submunition 

The BAT program also had inflexible requirements that exceeded available 

resources at the start of the program. The BAT operational concept is based 

on a need to locate targets from great distances using advanced sensor 

technologies, which are then required to guide the weapon to closer ranges, 

where the submunition would engage the target. The acoustic target 

technology was the most important technology needed to meet the 

weapon’s performance requirements, and the contractor proposed a 

weapon concept based on it. The Army accepted the concept and drafted 

performance requirements based on the acoustic technology. In this case, 

there was limited flexibility to negotiate trade-offs between requirements 

and resources given the small margin of error for a munition to hit the 

target. Consequently, the key to matching requirements with resources 

depended on committing the necessary resources to close gaps after the 

program was launched.

At the start of product development, the cost and schedule targets for 

BAT’s development were set without knowledge about the feasibility of the 

performance requirements. While the requirements were based on a 

specific technology, it was not mature enough for inclusion onto a product. 

In fact, the technology was still being defined in paper studies. The Army 

did not prototype this technology until almost 6 years after the program 

was launched. During its product development program, the BAT 

experienced significant development cost and schedule increases. Program 

officials attribute these increases, at least in part, to unknowns about the 

new technologies. Because requirements remained inflexible, the product 

developer was forced to add resources to achieve a match between the 

two. 

Balance in the Roles of 

the Customer and 

Product Developer 

Makes for Effective 

Trade-offs 

With knowledge gained from systems engineering and flexible 

requirements as preconditions, successful product development programs 

benefited from an environment in which product developers and customers 

had balanced roles and shared responsibilities for setting product 

requirements. In two successful cases, product managers were responsible 

for both (1) translating customer wants into product requirements and

(2) developing and delivering the products within available resources. This 

dual responsibility allowed them to make reasonable trade-offs between 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 51 GAO-01-288 Best Practices

performance requirements and resources prior to committing to product 

development. In two other successful cases, requirement setters and 

product developers had equal roles in establishing and changing 

requirements, using knowledge from systems engineering as a guide. In 

unsuccessful cases, there was no parity in decision-making between the 

requirement setter and the product developer; the requirement setter held 

the upper hand by controlling the requirements without being bound by 

cost and schedule considerations or the developer’s resources.

Successful Programs 

Benefited From Parity in 

Decision-making Between 

Customer and Developer

In the successful commercial cases we examined, the product developer 

had ultimate responsibility for defining the product’s requirements but 

worked intimately with the customers to understand their needs. The firms 

established an environment that teamed customer representatives with 

product development engineers in setting mutually agreeable product 

requirements. Representatives from one commercial firm told us that they 

feel responsible for understanding a customer’s operations better than the 

customer does. They achieve this through their marketing people. Once the 

marketing staff understands a customer’s wants, they can work with the 

design and production engineers to develop a product solution that will 

meet the customer’s needs and can be met with company resources. The 

product manager makes final decisions on the product’s requirements, but 

is mindful of the fact that a dissatisfied customer will not buy the product.

Caterpillar’s 797 Mining Truck An example of how this parity in decision-making worked is the Caterpillar 

797 mining truck engine. To meet the customers’ needs, the engine had to 

produce at least 3,400 horsepower. For maintenance and serviceability 

reasons, some mining companies wanted a single engine to produce this 

power, which Caterpillar’s marketing staff translated into a requirement. 

However, Caterpillar did not have such an engine and its systems 

engineering analysis showed that a new single-engine design could not be 

matured and demonstrated within the 18-month cycle time limit. Thus, 

product developers proposed a twin-engine design that met the 

horsepower requirement but used an existing engine. Caterpillar’s 

customer representatives presented this proposal to the customers, as well 

as their position that the firm could either deliver the truck powered by 

twin engines in 18 months or take longer to deliver a single engine truck—

but not both. Most customers accepted the proposal and the position.

Bombardier’s New Regional Jet Similar parity in decision-making was evident concerning Bombardier’s 

fly-by-wire technology. The airlines wanted fly-by-wire on the BRJ-X 

aircraft because of the benefits. The benefits, which Bombardier’s systems 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 52 GAO-01-288 Best Practices

engineering confirmed included 96 percent fewer parts and 1,250 fewer 

pounds than hydromechanical controls. These benefits would reduce 

engine power requirements, saving about 3 percent of fuel capacity and 

reducing the need for maintenance significantly. All of these savings 

responded directly to the airlines’ performance needs. However, because 

this technology had never been integrated onto a regional jet before, 

Bombardier’s product development engineers believed it to be risky. Also, 

since the workforce of regional jet pilots had never used fly-by-wire, they 

would have to be trained on how to use it, and system engineers who had 

the authority to accept or reject the technology would not commit to 

including the technology unless it was demonstrated first. Bombardier 

closed the technology gap by demonstrating it on an existing aircraft. At 

that point, Bombardier was confident in including fly-by-wire in the 

product requirements for the BRJ-X. 

Less Successful Cases Did 

Not Have Parity Between 

the Requirement Setter and 

the Product Developer

In two less successful cases, the customer set the product requirements 

with comparatively little input from the product developer. These programs 

were initiated with cost and schedule estimates that were, for the most 

part, based on the customer’s requirements and a notional design. The 

product developer was not an equal partner in setting product 

requirements and had to launch programs without requisite resources. 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 53 GAO-01-288 Best Practices

Navy’s Radio Frequency 

Countermeasures System

Before product development began, the program manager for the Radio 

Frequency Countermeasures system had been working with the F/A-18 E/F 

customers to match its needs with available resources. The program 

manager told us that both parties believed they had a good match because 

the technology was mature. Also, both parties agreed that an 

approximately 5-1⁄2 year development cycle would deliver the desired 

product. However, prior to program launch, as part of the normal process 

used for approving product requirements, the proposed Navy requirements 

were distributed to the other services. The Air Force became interested in 

the program and requested that its needs for the F-15 and B-1 be combined 

with the Navy’s to develop a common system. The Air Force’s requirements 

were much more demanding than the Navy’s because more power was 

needed to generate the electronic countermeasures4

 for the F-15, a less 

stealthy aircraft than the F/A-18 E/F. These additional needs created a 

technology gap that required either more time or trade-offs. 

The product developer analyzed the technology gaps and determined that, 

given projected time and money, the effort required to meet them would 

take close to 7 years. The Navy customer would not agree to the additional 

time and insisted that the product developer deliver a product to meet the 

expanded needs within the original 5-1⁄2 year time frame. The customers, in 

sole control of requirements setting, thus firmed the product requirements. 

The program manager informed us that he had little decision-making 

authority in the matter and was compelled to accept the customers’ 

schedule. The result was an imbalance between product requirements and 

resources, which also resulted in cost and schedule increases. Because of 

the delays in developing the system, the Navy plans to incorporate a less 

effective substitute onto the F/A-18 until the product is ready. These 

aircraft will have to be retrofitted with the system when it becomes 

available—a costly and inefficient process. Ultimately, the trade-offs that 

were unpalatable before launch became unavoidable after launch.

Army’s Comanche Helicopter Parity between the requirement setter and the product developer did not 

exist on the Comanche program. As discussed previously, the Comanche 

product development program included immature avionics technologies 

4

 Electronic countermeasures involve the jamming and deceiving of the enemy’s electronic 

devices, such as a radar or communications system, by sending an electronic signal that 

reduces or prevents their usefulness. Because of the more traditional design of the F-15, it 

required the Radio Frequency Countermeasures system to produce as much as 2 to 3 times 

more amplified power than that needed for the F/A-18. 
Chapter 3

Several Factors Enable Customer Wants and 

Developer Resources to Be Matched Before 

Program Launch

Page 54 GAO-01-288 Best Practices

that were needed to meet demanding and inflexible performance 

requirements. These technologies represented significant gaps between 

what the customer wanted and what the program office could confidently 

deliver given the resources available. Prompted by concerns about the time 

needed to advance the necessary technologies, the program manager 

proposed trading off some requirements. However, requirements managers 

informed us they were unwilling to accept trade-offs and that they believed 

the program manager was too risk averse. Consequently, the program was 

launched with the requirements intact, despite the program manager’s 

concerns, and, cost and schedule estimates for its development have 

doubled.
Page 55 GAO-01-288 Best Practices

Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch Chapte4r

Within DOD’s traditional acquisition process, it is difficult to employ 

systems engineering in making trade-offs, maintain flexibility in customer 

expectations, and put the product developers in a balanced position 

relative to the customers. As a result, customer needs—however 

legitimate—are translated into product requirements that make 

unreasonable demands on available resources. This pattern is perpetuated 

because resources are generally added later during product development to 

close any gaps between requirements and resources. The mechanics of 

obtaining funding and getting approval to start an acquisition program 

dictate that events proceed in the following sequence: set requirements, 

obtain funding, launch program, and perform systems engineering. This 

sequence places the knowledge that is needed to identify resource gaps—

and the power that it gives to the product developer—at a disadvantage to 

shape requirements and improve program outcomes. Other aspects of the 

process reward behaviors that can put pressure on requirements, making 

them less flexible and more difficult to meet. For example, the services 

must clearly differentiate a new weapon’s performance characteristics 

from alternatives to successfully compete for funding, encouraging 

detailed cost and schedule estimates to be made with little knowledge from 

systems engineering. The weapon system programs that did match 

requirements and resources before launch—the Tactical Unmanned Aerial 

Vehicle and the Global Hawk—represented departures from this process 

that benefited greatly from the intervention and protection of top-level 

individuals.

DOD has recently changed the requirements setting and acquisition 

processes to better reflect best commercial practices. Specifically, DOD 

has revised its policy for approving new development programs by 

providing more latitude for technology maturation and other

knowledge-generating activities to take place before a program is 

launched. The acquisition policy also supports the evolutionary 

development approach as the preferred method of product development. In 

addition, DOD has revised its policy for developing requirements, calling 

for a staged or stepped approach that will dovetail with evolutionary 

acquisitions. By itself, however, the policy can do little to foster the early 

matching of customer wants with developer resources because the 

mechanics of obtaining funding and the pressures on requirements setters 

are unchanged. The incentives of the process, unlike policy, are more likely 

to be seen in decisions made on the funding and approval of individual 

programs.
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 56 GAO-01-288 Best Practices

Current Process Puts 

Requirements Setting 

and Systems 

Engineering on 

Opposite Sides of the 

Launch Decision

DOD’s acquisition process makes it difficult to know what resources will be 

needed to meet requirements before launching a program. Within this 

process, systems engineering does not take place until a program has been 

launched and cost and schedule targets have been set. One reason for this 

is that DOD does not typically sign contracts with product developers, who 

are responsible for systems engineering, until after the requirements have 

been written. Once an acquisition program is approved and product 

development is funded, a product developer can begin systems engineering 

and gaps between requirements and resources can be identified. However, 

this process does not yield knowledge about the requirements’ 

achievability until well into product development. Even when requirements 

are in draft at the time of launch, they are hard to change because user 

representatives have already invested significant effort in preparing them. 

This sequence of events—setting requirements, launching an acquisition 

program, contracting with a product developer, and conducting systems 

engineering—narrows the options for closing gaps primarily to increasing 

cost and schedule estimates.

DOD’s process for setting requirements can begin many years before a 

program is launched, and it can be several years after the program is 

launched before systems engineering knowledge is obtained. The time 

period from the start of the definition of the user’s needs to the arrival of 

systems engineering knowledge that often uncovers gaps can approach

10 years in some instances. More than 30 organizations within the 

requirements community may have a hand in determining a weapon 

system’s performance requirements before a contractor with systems 

engineering expertise can identify the gaps between requirements and 

available resources. This process means the “doability” of the requirements 

is often not known with certainty until well into product development or 

until a significant percentage of funds planned to develop the system has 

been invested. By this point in time, customers’ expectations have been set, 

making it difficult to change requirements if gaps between requirements 

and available resources are found. 

The process used to set requirements and begin development on the Army’s 

Crusader program provides an example that illustrates the mechanics of 

the process (see fig. 14).
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 57 GAO-01-288 Best Practices

Figure 14: Timeline for Developing Crusader’s Requirements

The user’s representative for the Crusader (the Army’s Training and 

Doctrine Command artillery school at Fort Sill, Oklahoma) began drafting 

the performance requirements for the Crusader in 1990. The process to 

define the requirements took approximately 4 years. During this time, the 

user representative framed the needed features of the Crusader and 

conducted exhaustive analyses and trade studies to identify the optimal 

point to set a specific requirement. Also, as required by DOD acquisition 

policy, the user representative analyzed a notional Crusader that met these 

requirements against other alternatives, such as improved versions of 

existing artillery systems. The result of these analyses was a set of 

requirements from the user representative and notional design prepared by 

the Army program office. In 1994, the requirements for the Crusader system 

were approved, the acquisition program was launched and funded, and 

program cost and schedule targets were baselined. 

By this time, key features of the Crusader were defined. For example, the 

Crusader would have to use liquid propellant—a revolutionary 

technology—to propel the projectiles far enough to meet the optimal range 

requirements. Also, because the Crusader was required to stay on the firing 

line during rearming and refueling, it would have to have an armored and 

automated resupply vehicle to keep the resupply crews protected.

After the program was launched, the Army entered into a contract with 

United Defense Limited Partnership to develop the Crusader. Over the next 

1990

User representative

began to draft initial

Crusader

requirements.

User representative, with

help from program office,

analyzed and compared

key functions with

alternatives. Notional

design reflected key

technologies and 

design features.

Requirements 

approved. Program was

launched and funding

approved based on notional

design. Product developer

was hired and began

 systems engineering

process.

1994

Systems engineering

determined that liquid

propellant technology was

too risky and costly. The

Army switched to a

conventional solid

propellant and the system

was redesigned.

1996

Preliminary design

completed. Later,

Army decided to

reduce Crusader’s

weight, availing trades

ruled out by initial

requirements. Another

redesign began.

1998

Program launch

Requirements setting and notional design Systems engineering
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 58 GAO-01-288 Best Practices

4 years, the firm applied systems engineering practices to the requirements 

in order to develop a preliminary design. During this process, the firm had 

to make trade-offs in order to develop a design that could stay within 

resources. By 1996—2 years into product development and 6 years since 

the need for the Crusader was identified and determined—United Defense 

Limited Partnership concluded that resources needed to develop a liquid 

propellant technology were not available. The firm’s only option was to 

accept a less capable technology—a conventional solid propellant, 

increasing the risk that the Crusader’s firing range requirement may not be 

achieved. According to Crusader officials, United Defense Limited 

Partnership did not develop a good preliminary design of the system until 

1998 or about 8 years after the user representative began the requirements 

setting process. 

Incentives of Current 

Process Create 

Pressure on Product 

Requirements 

Several factors in the DOD environment create incentives to set 

requirements high and to resist trade-offs. The competitive nature of the 

process to justify a new program puts great pressure for a potential 

weapon’s requirements to stand out. Unlike the commercial environment, 

the user representatives are separate from both the customer and the 

product developer, and thus play a unique role that has different interests. 

Once user representatives get a set of requirements through the very 

difficult and lengthy process of approval before a program can be 

launched, they are understandably reluctant to change them. Finally, the 

DOD customer—unlike its commercial counterpart—is more tolerant of 

schedule delays and cost increases.

Pressures on the 

Requirements-setting 

Process

The competition within DOD to win funding and get approval to start a new 

program is intense; establishing the basic need and writing requirements is 

perhaps the most important step in the process. This creates strong 

incentives for requirements setters to write performance requirements that 

will make their particular weapon system stand out from existing or 

alternative systems. Much of the requirement-setting activity prior to 

initiating a program is devoted to proving the superior cost-effectiveness of 

the preferred system over others, with less consideration given to the 

resources that will be needed to develop the system. If user representatives 

do not write requirements that will withstand scrutiny and prevail over 

alternatives, the program could be killed and no capability is gained for the 

customer. Costly, labor-intensive studies, such as analyses of alternatives, 

are done to determine the best possible solution to meet the user’s needs 

and to beat the competition. At the same time, overall DOD funding 
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 59 GAO-01-288 Best Practices

constraints put a high priority on affordability, making it important for 

program sponsors to provide cost estimates that will fit within the funding 

constraints. Instead of forcing trade-offs, challenging performance 

requirements, when coupled with other constraints, such as cost or light 

weight, can drive product developers to pursue exotic solutions and 

technologies that, in theory, could do it all. 

To simplify requirements and foster trade-offs, DOD has adopted the 

practice of identifying only a limited number of key performance 

parameters—“must-haves” for the customer—for each weapon system. 

This practice may help to prioritize requirements. However, if the 

parameters are not informed by systems engineering, given the pressures 

inherent in the process, they still result in unrealistic demands made of 

existing resources. For example, the Army’s Crusader program had only 

five key performance parameters, including characteristics such as 

rate-of-fire, range, and speed, but they were dependent on about 500 other 

subordinate performance requirements that were defined before systems 

engineering was done by the product developer. 

The unique role of the requirement setter in DOD’s acquisition process can 

also put pressure on requirements. DOD’s requirement setters are outside 

of the acquisition community and often represent an operational function 

such as air combat, artillery, or armor. They are not the actual user like an 

operating air wing or Army brigade, but serve as a representative of the 

user. Unlike the commercial world, where a customer’s wants are 

represented by someone within the product developer’s organization, 

DOD’s requirement setters are, for the most part, separate and independent 

of the customer, program manager, and the product developer. A user 

representative can work directly with DOD science and technology 

organizations and defense firms to obtain information about new 

technology, and thus may be less willing to defer to a program manager’s 

advice on these issues. Also, while a customer is focused on current 

problems and near-term performance, a user representative tries to look at 

longer term needs. In fact, the fear that DOD may not procure another such 

system for several years creates incentives for user representatives to 

reach for the most capability possible because they do not know when or if 

they will get another opportunity to develop and acquire the next weapon 

system. 

The Comanche helicopter program is a good example of these pressures at 

work. The Comanche was initiated in the early 1980s as a family of 

lightweight, multipurpose helicopters whose operation and support costs 
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 60 GAO-01-288 Best Practices

would be 50 percent less than the Vietnam-era fleet. The program was 

originally expected to offer as good a technical performance as possible 

within clearly stated—and low—unit cost goals. The requirements were 

simple at the start. However, once the requirement setting process began, 

the program emerged as it is today—a threat-based program to yield the 

next-generation, high-performance helicopter—at a cost significantly 

higher than the existing Apache. It was justified as being faster, stealthier 

and smaller than the Apache helicopter. Resource gaps were identified 

after the program was launched, when the product developer began to 

obtain knowledge through systems engineering and technology 

development about what it would take to meet the requirements. 

Leadership in the Army has called for a transformation in Army forces and 

equipment to a lighter, more mobile force that can be deployed more 

quickly and easily. The impact this change is having on Crusader’s 

requirements further illustrates how DOD incentives can drive 

requirements and ultimately impact program outcomes. The preliminary 

Crusader design —which met all key requirements—was considered too 

heavy for the new, lighter force. In fact, the Army Chief of Staff said he 

would like to have the Crusader’s weight reduced to the point that twice as 

many vehicles could be moved with the same amount of transport aircraft. 

With the program’s future at stake, the user representative, program 

manager, and product developer are working together to reduce the overall 

design weight of the system from 55 to 38 tons. Because of this new 

priority, previously untraded requirements—such as degree of crew 

protection, time on the firing line, and the need for tracked vehicles—are 

now being examined to see if they can be reduced to save weight. 

A specific Crusader requirement is illustrative. In developing the original 

set of requirements, the user representatives determined that the 

Crusader’s effectiveness would be maximized if the Crusader could avoid 

having to leave its battle station to reload and refuel. To meet this 

requirement, the resupply vehicle would have to reload and refuel the 

howitzer at the battle station. This requirement necessitated that the 

resupply vehicle be armored, tracked, and fully automated so that the crew 

would not be exposed to enemy fire. This not only made for a more 

effective howitzer but also distinguished the Crusader from other 

candidates. In looking for weight reductions, the Army is considering 

pulling at least some of the howitzers off station to reload and refuel. If it 

relaxes that requirement, the resupply vehicle and crew can do its work 

under safer conditions. In turn, the vehicle would not have to be as heavily 

armored, automated, or tracked. In fact, the Army is considering using 
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 61 GAO-01-288 Best Practices

existing trucks, which are far lighter and less expensive than a new 

resupply vehicle. In our view, the changed circumstances of the Crusader 

program—past the approval gate, a product developer deep into systems 

engineering, and a top level mandate to reduce weight—created incentives 

to make requirements trade-offs that were not acceptable before. 

Calcification and Customer 

Acceptance Do Not 

Encourage Trade-offs

Once established, system requirements undergo incredible scrutiny and 

review by a myriad of interests within DOD’s process. For example, the 

Crusader draft requirements were circulated to approximately

30 organizations throughout the Army, other services, operational 

commands, and the development community for review. This process 

yielded 943 comments. To incorporate or otherwise dispose of each 

comment, a joint working group with representatives from all of the 

reviewing organizations was established. The group incorporated 702 of 

the 943 comments into the requirements, the cumulative effect of which 

was to add, rather than to trade, requirements. As one official stated “it is 

generally not the practice to reduce or eliminate requirements but to add 

more to appease a particular party.” This lengthy and cumbersome review 

process tends to calcify weapon system requirements before product 

development begins and knowledge from systems engineering can be 

obtained. 

The practice of breaching cost and schedule objectives to meet difficult 

requirements would not persist without a customer’s cooperation. Unlike 

commercial customers, DOD customers tend to be tolerant of cost 

overruns and delays in order to get a high-performance weapon system. 

Traditionally, customers have been willing to wait long periods of time for a 

highly desirable system that they feel will provide them the longest lasting 

capability. They would rather wait for the most desirable system to be 

developed than accept a less capable system, thinking that they may not get 

the opportunity to acquire a new or modified system in the future. Again, 

the Comanche program provides insight. At the time the program was 

started, the Army expected to receive the first operational helicopter in 

1996. The development program has encountered delays; some related to 

weight, cost, and performance requirements that demanded immature 

technologies. However, the Army has chosen to keep the requirements 

generally intact and to do without the helicopter instead of fielding a less 

capable system more quickly. The Army now expects to have an initial 

operational capability in 2006, 10 years after the initial target date.
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 62 GAO-01-288 Best Practices

More Successful 

Weapon System 

Programs Have 

Departed From the 

Normal Process

The two DOD programs we examined—Tactical Unmanned Aerial Vehicle 

and Global Hawk—that set requirements in a way that approximates best 

commercial practices were departures from DOD’s normal process. 

Specifically, (1) their origins as Advanced Concept Technology 

Demonstrations enabled them to hire a product developer much earlier 

than is traditionally the case and (2) they benefited greatly from the 

personal intervention of service and DOD executives who created and 

enforced conditions that were conducive to making trade-offs between 

wants and resources. Other new programs, such as the Joint Strike Fighter 

program, showed that traditional incentives to speed a program along still 

existed and could increase risk in product development.

The Advanced Concept Technology Demonstrations, for the Tactical 

Unmanned Aerial Vehicle and the Global Hawk, showed the feasibility of 

developing a system that could do what the customer wanted before a 

commitment to product development was made. The sequence of events 

leading to program launch on each of these programs was similar to those 

of successful commercial firms. The demonstrations involved the 

customer, user representative, program manager, and product developer in 

a more integrated fashion than is normally the case. Moreover, by building 

prototypes for the demonstrations, the product developers had to conduct 

a significant amount of systems engineering before the programs were 

launched. Consequently, the demonstrations provided valuable knowledge 

that allowed the customer, user representative, and product manager to 

define a set of product requirements that could be met within resources. 

Most importantly, the demonstrations were conducted before program 

launch, before cost and schedule targets were set. 

Both programs had unusual intervention by top-level individuals that set 

resource constraints and encouraged evolutionary acquisition strategies. 

On the Tactical Unmanned Aerial Vehicle program, the top military 

acquisition executive personally met with the head of the user 

representative’s organization and struck an agreement that the product was 

to be fielded in stages, with the first stage being a very basic system. This 

agreement had the effect of establishing mutual interest in the same 

program outcome and shielded the program from criticism by either 

community. The personal involvement of the Under Secretary of Defense 

for Acquisition, Technology, and Logistics helped set the stage for Global 

Hawk’s evolutionary approach to meeting requirements. The Under 

Secretary insisted on an initial capability that could be developed within a 

fixed budget while providing the flexibility to defer other requirements to 
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 63 GAO-01-288 Best Practices

succeeding versions. In both cases, this top-level intervention allowed 

requirements to be flexible and gave the product developers parity with the 

requirements setters in influencing requirements. Equally important, we 

believe the intervention signaled support for the programs, which eased 

some of the pressures that normally accompany efforts to get programs 

approved. In each case, while there was a potential for success, it came not 

so much from a well-established process but from exceptional behavior 

from senior-level leaders. If, for some reason, leadership or priorities 

change, the process may not ensure success. 

Constructive Changes 

in DOD’s Policy Not 

Enough to Match 

Customer 

Expectations With 

Developers’ Resources 

While DOD has taken steps that could help it more efficiently acquire its 

weapons, these steps have not substantively changed the mechanics or 

incentives of the requirements setting process. While the revised 

acquisition policy provides opportunities for improvement, it is not specific 

about when to match wants and resources. In fact, the sequence of 

events—setting requirements, obtaining funding, launching an acquisition 

program, and conducting systems engineering—remains essentially the 

same as before the revision. Recent experiences with the Global Hawk and 

the Joint Strike Fighter programs indicate that traditional pressures on the 

requirements process are still strong. 

Recent DOD Policy 

Revisions Are Supportive of 

Best Practices

DOD has recently revised its policies for operation of the defense 

acquisition and requirements setting processes. The acquisition policy 

reflects best commercial practices and emphasizes better use of 

evolutionary acquisitions, more reliance on mature technology, and 

reduced cycle times and costs. It recognizes shorter acquisition cycle times 

as critical to making the best use of advanced technologies and 

evolutionary requirements as a way to reduce cycle times. The Under 

Secretary of Defense for Acquisition, Technology, and Logistics issued 

related guidance stating that DOD’s objective will be to achieve acquisition 

cycle times—from program start to initial fielding—no longer than 5 to 7 

years. The revised acquisition policy also states a clear preference for 

evolutionary acquisitions over “single step to full capability” acquisitions. 

DOD also revised its acquisition process, which is now four phases: 

(1) concept and technology development, (2) system development and 

demonstration, (3) production and deployment, and (4) operations and 

support (see fig. 15). In addition, guidance that directs the requirements 

setting process was revised to include an emphasis on the use of 

time-phased performance requirements in support of evolutionary 

acquisitions. 
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 64 GAO-01-288 Best Practices

Figure 15: Revised DOD Acquisition Process

The decision to launch a program would normally take place between the 

first and second phases. The revised policy states that the decision is 

dependent on three factors: technology maturity, validated requirements, 

and funding. Assuming that technology is found to be mature, at the start of 

the system demonstration and development phase, the policy requires the 

weapon’s requirements to be formally approved and full funding to be 

provided for the remainder of the program. This is the point at which a 

program manager would commit to performance, cost, and schedule 

estimates and DOD would commit to provide that level of resources. DOD 

later would award contracts to product developers to conduct the majority 

of systems engineering necessary for designing and building the product. In 

that regard, little has changed under this revised approach. It is still 

difficult to know what resources will be needed to meet a set of 

requirements before making the decision to launch the program.

Pressures on Requirements 

Are Still Powerful

While DOD policy encourages evolutionary acquisition approaches, it will 

not be successful if traditional incentives for setting high and inflexible 

requirements persist. Recent developments on the Global Hawk program 

indicate the continuing pressure programs face to increase requirements. 

The initial Global Hawk system provides a capability for high altitude 

reconnaissance that the customer has stated is acceptable for meeting its 

short-term needs. Future generations would add capabilities to match the 

U-2—a manned reconnaissance airplane—as technology, money, and time 

become available. The Air Force had embarked on an evolutionary 

acquisition that defines the first generation of the Global Hawk based on 

the system already demonstrated with performance, supportability, and 

producibility enhancements. However, the Air Combat Command, the user 

representative, indicated that it would accept nothing short of the same 

Concept and

Technology

Development

Program

launched

Pre-program Acquisition program

Production

and Development

Systems Development Operations and Support

and

Demonstration

Initiation of

concept studies

Production
Chapter 4

Characteristics of DOD’s Acquisition Process 

Make It Hard to Match Expectations and 

Resources Before Program Launch

Page 65 GAO-01-288 Best Practices

capability found in the current U-2, despite the customer’s views. The 

Command did not have confidence that the evolutionary approach would 

receive the support and resources to attain the U-2’s capabilities in future 

generations. As a result of pressure from the user representative, the Air 

Force proposed a revised acquisition strategy that while still evolutionary 

in nature, accelerated the inclusion of advanced technologies onto the 

aircraft to provide U-2 capabilities sooner. In December 2000, the Deputy 

Secretary of Defense elected not to approve the revised acquisition strategy 

at this time. While we did not review the revised approach in detail, it did 

appear risk had been added to the program as requirements increased and 

resources changed. 

The Joint Strike Fighter program is another example in which traditional 

incentives can result in decisions contrary to best practices. While DOD 

has designated the Joint Strike Fighter as a flagship program for acquisition 

reform, it is now 4 years into its acquisition program and has not yet 

achieved a match between requirements and resources—particularly in the 

form of the technologies needed. By best practice standards, none of the 

fighter’s critical technology areas that the program office has identified are 

expected to be at readiness levels acceptable for entry into the engineering 

and manufacturing development phase, which is scheduled for 2001. 

Reminiscent of the Comanche, numerous conflicting demands that the 

fighter achieve high performance and low costs have resulted in 

requirements that must be satisfied with several technical advances. The 

requirements have also proven to be relatively inflexible. As we reported1

in May 2000, delaying this phase of the program until technologies are 

mature would improve the chances that the Joint Strike Fighter will be 

fielded as planned. However, despite not having the requisite knowledge 

for the technologies, DOD has deemed the risks manageable and proposes 

to proceed with the program as planned. Such a decision reinforces 

traditional incentives and increases the likelihood for future problems. 

1 Joint Strike Fighter Acquisition: Development Schedule Should Be Changed to Reduce 

Risks (GAO/NSIAD-00-74, May 9, 2000).
Page 66 GAO-01-288 Best Practices

Chapter 5

Conclusions and Recommendations for 

Executive Action Chapte5r

Conclusions Managing the setting of a product’s requirements in a way that matches the 

customer’s needs with the developer’s resources is critical for a successful 

product development. This kind of success—delivering weapons that meet 

needs within predicted costs and time frames—is essential if DOD is to get 

what it wants from its huge modernization investment. The best practices 

for balancing needs and resources before committing to a new product or 

weapon system development are to (1) conduct systems engineering to 

illuminate what has to be done to match the wants with the resources; 

(2) establish fixed cycle times for an initial product within an overall 

evolutionary approach to foster flexibility needed to make key trade-offs; 

and (3) maintain parity between requirement setters and product 

developers when translating customer needs into product requirements. 

These practices provide for identifying gaps between wants and resources 

as well as solutions—whether by trading off needs or investing more 

resources—before setting requirements and starting development.

The environment for commercial products provides incentives that 

encourage these practices. Like DOD, commercial customers have an 

initial advantage in that they have high expectations of a new product, they 

do not have unlimited time and money, and ultimately, their needs have to 

be met. While firms are motivated to offer a customer a product that meets 

these needs, they are keenly aware that promising more than they can 

deliver will ultimately disappoint the customer. Consequently, leading firms 

consciously attempt to manage customer expectations and put themselves 

in a good position to negotiate a reasonable set of product requirements. 

They do this in several ways. First, because they are investing their own 

resources to develop the product, they are at liberty to conduct systems 

engineering early—an investment they consider small relative to the overall 

cost of developing a product. Second, they have a proven track record for 

delivering products on time and making promised improvements to 

succeeding versions of the product. Third, they play a prominent role in 

setting and agreeing to product requirements. Together, these factors help 

create the credibility, confidence, and trust that are essential to maintaining 

the flexibility needed to match their resources with the customer’s needs 

before launching a new product development. 

Within DOD’s traditional acquisition process it is difficult to gain 

knowledge and maintain flexibility in requirements prior to committing 

resources to an acquisition program. As a result, customer needs are 

translated into product requirements that often make unreasonable 
Chapter 5

Conclusions and Recommendations for 

Executive Action

Page 67 GAO-01-288 Best Practices

demands on available resources. There are several reasons for setting 

weapon system requirements this way. Some are the following: 

• The mechanics of the program approval and funding process force a 

sequence of events that keeps a product developer relatively uninvolved 

in shaping requirements, compared with best practices, causing product 

development to begin with cost and schedule targets untempered by 

systems engineering knowledge. 

• Unique or demanding requirements can help gain approval to fund a 

new program because aiming too low can result in losing out to other 

programs, thus denying any new capabilities. 

• There is some mistrust in an evolutionary approach, specifically 

regarding whether future improvements will be approved; consequently, 

attempting to reach for the full capability in a single step can be seen as 

a safer course of action. 

• The challenging and long process to get requirements approved hardens 

requirements setters against trade-offs—in fact, the process of getting a 

requirement approved can actually encourage additions to garner 

support.

• Unlike the commercial world, DOD customers have proven to be 

tolerant of cost and schedule increases once a program is underway. 

Seen in this light, while the pattern of weapon system requirements 

outstripping planned resources is inefficient, it is not irrational; within the 

DOD environment, this approach is successful in starting programs that 

eventually provide superior capabilities. This approach also has negative 

consequences. First, the additional time and money that must be invested 

after launch still yield the same capability called for by the original 

requirements, effectively lowering the buying power of the investment. 

Second, the unplanned nature of the additional investment generally 

requires weapon system quantities to be lowered or money to be taken 

from other programs. Third, when a requirements setter is unwilling to 

make trade-offs before launch but later elects to accept delivery delays to 

meet those requirements, the implicit trade-off is to have the customer do 

without any of the new capability for a longer period of time. The challenge 

in adopting best practices, therefore, is to make them rational—that is, 

critical for success—in the DOD environment. Meeting this challenge will 

take changes in both the mechanics and the incentives of the requirements 

setting and program approval processes. 

DOD’s recently revised policies could make it possible for a better 

matching of customer needs and developer resources before program 
Chapter 5

Conclusions and Recommendations for 

Executive Action

Page 68 GAO-01-288 Best Practices

launch. While these policies are sound and merit support, they retain the 

mechanics of the old policies. Namely, requirements must be set before a 

program can be approved and a program must be approved before the 

money can be obtained to pay the product developer to conduct systems 

engineering. Such mechanics deny the knowledge needed to match wants 

with resources before starting a program. Similarly, the new policies must 

overcome the still extant pressures brought to bear on requirements setters 

to aim high and become inflexible. It took unique circumstances—starting 

as Advanced Concept Technology Demonstrations and enjoying the 

personal intervention of top DOD executives—to create the incentives on 

the Global Hawk and Tactical Unmanned Aerial Vehicle programs for 

making the trade-offs needed to match needs with resources. Even so, the 

Global Hawk is struggling with pressures to reverse some of the trade-offs 

and raise requirements. Other programs, such as the Joint Strike Fighter, 

have requirements that outstrip resources, despite efforts to keep 

requirements flexible and to treat the price of the aircraft as a requirement.

Recommendations for 

Executive Action

To realign the mechanics of the requirements setting and program approval 

processes to bring more knowledge into the process of setting 

requirements, we recommend that the Secretary of Defense require that the 

systems engineering needed to evaluate the sufficiency of available 

resources—knowledge, time, money, and capacity—be conducted in time 

to help identify and make the critical trade-offs that precede the 

formalization of requirements. One option is to allow the award of

well-defined systems engineering contracts to prospective product 

developers—contractors—before the system development and 

demonstration phase.

To realign the incentives of the requirements setting and program approval 

processes with the need to match available resources, we recommend that 

the Secretary of Defense:

• Reduce the pressures put on user representatives to set requirements 

high to win the competition for program approval. One way to reduce 

these pressures, drawing on the experiences of the Tactical Unmanned 

Aerial Vehicle and Global Hawk, is to have higher level officials in the 

services and DOD decide on the type of weapon system that is needed to 

meet a valid need before the requirements setters begin detailed work 

on framing a specific solution. Making such a decision earlier in the 

process would ease the pressure on to set overly demanding and 
Chapter 5

Conclusions and Recommendations for 

Executive Action

Page 69 GAO-01-288 Best Practices

inflexible requirements that will crush alternatives and win program 

approval. 

• Require, as a condition for starting the system development and 

demonstration phase for a weapon system—program launch—that 

sufficient evidence exists to show there is a match between a weapon 

system’s requirements and the resources the program manager has to 

develop that weapon. Based on our current and past work on the best 

practices of leading commercial firms, there is a key tool the Secretary 

can use to define what resources DOD is willing to apply—establishing 

limits on the time it takes to complete system development, such as not 

to exceed 5 years. Further, having a formal agreement among the 

requirements setters, program managers, and resource providers on 

development and delivery of the required product would emulate the 

best practice of establishing accountability for subsequent actions that 

stray from the agreement.

Agency Comments and 

Our Evaluation

DOD concurred with a draft of this report and its recommendations and 

agreed that the requirements process needs to be better informed by 

systems engineering in order to allow for the timely leveling of user needs 

and developer solutions. It noted that the recently revised acquisition 

policy takes the first steps in this new direction with guidance for 

evolutionary acquisitions and the identification of knowledge points in the 

acquisition process, but agreed that more progress needs to be made. 

Specifically, DOD recognized that more knowledge in the setting of 

requirements and the potential ability of the producer to meet those 

requirements would provide a greater understanding of the time, cost, and 

potential success of a program. DOD also recognized that the systems 

development and demonstration phase can begin with a higher level 

definition of requirements (that is, a higher level than the Operational 

Requirements Document), which can be used by the requirements-setter 

and the program manager, after some systems engineering work is done, to 

facilitate trade-offs earlier and relieve some of the pressure on Operational 

Requirements Document. Finally, DOD stated that the Acquisition Program 

Baseline should require signatures from both the user and the resource 

sponsor prior to program initiation signifying agreement that there is 

sufficient evidence—such as determined by systems engineering, 

demonstration of technology maturity, and adequacy of funding—that a 

match between requirements and resources has been made.
Page 70 GAO-01-288 Best Practices

Appendix I

Comments From the Department of Defense ApApenednxidxeiIs
Appendix I

Comments From the Department of Defense

Page 71 GAO-01-288 Best Practices

Now on pp. 11 and 67.
Appendix I

Comments From the Department of Defense

Page 72 GAO-01-288 Best Practices

Now on pp. 11 and 68.

Now on pp. 11 and 67.
Appendix I

Comments From the Department of Defense

Page 73 GAO-01-288 Best Practices
Page 74 GAO-01-288 Best Practices

Appendix II

GAO Staff Acknowledgments ApendxiI

Acknowledgments Paul Francis, Matthew Lea, Michael Sullivan, and Katrina Taylor made key 

contributions to this report.
Page 75 GAO-01-288 Best Practices

Related GAO Products

Best Practices: A More Constructive Test Approach Is Key to Better 

Weapon System Outcomes (GAO-NSIAD-00-199, July 31, 2000).

Defense Acquisition: Employing Best Practices Can Shape Better Weapon 

System Decisions (GAO/T-NSIAD-00-137, Apr. 26, 2000).

Best Practices: DOD Training Can Do More to Help Weapon System 

Program Implement Best Practices (GAO/NSIAD-99-206, Aug. 16, 1999).

Best Practices: Better Management of Technology Development Can 

Improve Weapon System Outcomes (GAO/NSIAD-99-162, July 30, 1999).

Defense Acquisitions: Best Commercial Practices Can Improve Program 

Outcomes (GAO/T-NSIAD-99-116, Mar. 17, 1999).

Defense Acquisition: Improved Program Outcomes Are Possible

(GAO/T-NSIAD-98-123, Mar. 17, 1998).

Best Practices: DOD Can Help Suppliers Contribute More to Weapon 

System Programs (GAO/NSIAD-98-87, Mar. 17, 1998).

Best Practices: Successful Application to Weapon Acquisition Requires 

Changes in DOD’s Environment (GAO/NSIAD-98-56, Feb. 24, 1998).

Major Acquisitions: Significant Changes Underway in DOD’s Earned Value 

Management Process (GAO/NSIAD-97-108, May 5, 1997).

Best Practices: Commercial Quality Assurance Practices Offer 

Improvements for DOD (GAO/NSIAD-96-162, Aug. 26, 1996).

(707454) Letr
Ordering Information The first copy of each GAO report is free. Additional copies of 

reports are $2 each. A check or money order should be made out to 

the Superintendent of Documents. VISA and MasterCard credit 

cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

Orders by visiting:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders by phone:

(202) 512-6000

fax: (202) 512-6061

TDD (202) 512-2537

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

Orders by Internet:

For information on how to access GAO reports on the Internet, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web home page at: 

http://www.gao.gov

To Report Fraud, 

Waste, or Abuse in 

Federal Programs

Contact one:

• Web site: http://www.gao.gov/fraudnet/fraudnet.htm

• e-mail: fraudnet@gao.gov

• 1-800-424-5454 (automated answering system)

United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Presorted Standard

Postage & Fees Paid

GAO

Permit No. GI00