Report to Congressional Committees

United States Government Accountability Office

GAO 

April 2006 DEFENSE 

ACQUISITIONS 

Major Weapon 

Systems Continue to 

Experience Cost and 

Schedule Problems 

under DOD’s Revised 

Policy 

GAO-06-368 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

www.gao.gov/cgi-bin/getrpt?GAO-06-368.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Michael J. 

Sullivan at (202) 512-4841 or 

sullivanm@gao.gov. 

Highlights of GAO-06-368, a report to 

congressional committees 

April 2006

DEFENSE ACQUISITIONS 

Major Weapon Systems Continue to 

Experience Cost and Schedule Problems 

under DOD's Revised Policy 

Changes made in DOD’s acquisition policy over the past 5 years have not 

eliminated cost and schedule problems for major weapons development 

programs. Of the 23 major programs we assessed, 10 are already expecting 

development cost overruns greater than 30 percent or have delayed the 

delivery of initial operational capability to the warfighter by at least 1 year. 

The overall impact of these costly conditions is a reduction in the value of 

DOD’s defense dollars and a lower return on investment. The following table 

illustrates the problem. 

Cost and Schedule Outcomes Sorted by Percent of Product Development Remaining 

Programs 

Percent cost 

growtha 

Schedule growth, 

in months 

Percent of 

development 

remaining 

Aerial Common Sensor 45% 24 85% 

Future Combat System 48% 48 78% 

Joint Strike Fighter 30% 23 60% 

Expeditionary Fighting Vehicle 61% 48 49% 

C-130 Avionics Modernization 

Program 122% Delays anticipated Undetermined 

Global Hawk (RQ-4B) 166% Delays anticipated Undetermined 

Sources: DOD (data); GAO (analysis and presentation). 

a

Cost growth is expressed as the percent change in program development cost estimates in 

2005 base year dollars. 

Poor execution of the revised acquisition policy is a major cause of DOD’s 

continued problems. DOD frequently bypasses key steps of the knowledgebased process outlined in the policy, falls short of attaining key knowledge, 

and continues to pursue revolutionary—rather than evolutionary or 

incremental—advances in capability. Nearly 80 percent of the programs 

GAO reviewed did not fully follow the knowledge-based process to develop a 

sound business case before committing to system development. Most of the 

programs we reviewed started system development with immature 

technologies, and half of the programs that have held design reviews did so 

before achieving a high level of design maturity. These practices increase the 

likelihood that problems will be discovered late in development when they 

are more costly to address. Furthermore, DOD’s continued pursuit of 

revolutionary leaps in capability also runs counter to the policy’s guidance. 

DOD has not closed all of the gaps in the policy that GAO identified nearly 

3 years ago, particularly with regard to adding controls and criteria. Effective 

controls require decision makers to measure progress against specific 

criteria and ensure that managers capture key knowledge before moving to 

the next acquisition phase. However, DOD’s policy continues to allow 

managers to approach major investment decisions with many unknowns. 

Without effective controls that require program officials to satisfy specific 

criteria, it is difficult to hold decision makers or program managers 

accountable to cost and schedule targets. In this environment, decisionmaking transparency is crucial, but DOD is lacking in this area as well. 

The Department of Defense (DOD) 

is planning to invest $1.3 trillion 

between 2005 and 2009 in 

researching, developing, and 

procuring major weapon systems. 

How DOD manages this investment 

has been a matter of congressional 

concern for years. Numerous 

programs have been marked by 

cost overruns, schedule delays, and 

reduced performance. Over the 

past 3 decades, DOD’s acquisition 

environment has undergone many 

changes aimed at curbing cost, 

schedule, and other problems. In 

order to determine if the policy 

DOD put in place is achieving its 

intended goals, we assessed the 

outcomes of major weapons 

development programs initiated 

under the revised policy. 

Additionally, we assessed whether 

the policy’s knowledge-based, 

evolutionary principles are being 

effectively implemented, and 

whether effective controls and 

specific criteria are in place and 

being used to make sound 

investment decisions. 

What GAO Recommends

GAO recommends that DOD insert 

specific criteria into the policy at 

key investment points and require 

programs satisfy those criteria 

before allowing them to move 

forward. In order to insure 

transparency and accountability, 

GAO also recommends that DOD 

require decision makers to include 

the rationale for their decisions in 

decision documentation. DOD 

partially concurred with our 

recommendations. 
Page i GAO-06-368 DOD Acquisition Policy 

Letter 1

Results in Brief 2 

Background 4 

DOD’s Revised Policy Has Not Improved Development Program 

Outcomes 8 

DOD Is Not Effectively Implementing the Policy’s KnowledgeBased, Evolutionary Approach 12 

Specific Criteria Are Needed to Ensure Disciplined and 

Transparent Investment Decisions 18 

Conclusions 22 

Recommendations for Executive Action 23 

Agency Comments and Our Evaluation 24 

Appendix I Scope and Methodology 27 

Appendix II Comments from the Department of Defense 28 

Appendix III Program Data for 23 Programs Initiated under 

DOD’s Revised Acquisition Policy 

(as of December 2005) 31 

Related GAO Products 33 

Tables 

Table 1: Cost and Schedule Outcomes for 23 Programs Initiated 

under the Revised Policy (as of December 2005) 9 

Table 2: Cost and Schedule Outcomes for 6 of the 10 Largest 

Development Programs Sorted by Percent of System 

Development Remaining 10 

Table 3: Assessment of Program Design Maturity 17 

Table 4: Assessment of Program Acquisition Strategies for GAO’s 

Nine Case Studies 18 

Table 5: Types of Controls Considered Best Practices for 

Successful Product Development 20 

Contents 
Page ii GAO-06-368 DOD Acquisition Policy 

Figures 

Figure 1: Development Cost Overruns by Decade (in Fiscal Year 

2005 Dollars) and Key Reform Efforts 5 

Figure 2: Comparison of DOD’s Revised Policy and Commercial 

Best Practices Model 7 

Figure 3: Cost and Schedule Growth under DOD’s Revised Policy 11 

Figure 4: Comparison of Programs with Mature versus Immature 

Technologies at Start of System Development 15 

Abbreviations 

DOD Department of Defense 

GAO Government Accountability Office 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-06-368 DOD Acquisition Policy 

April 13, 2006 

The Honorable John Ensign 

Chairman 

The Honorable Daniel K. Akaka 

Ranking Minority Member 

Subcommittee on Readiness and Management Support 

Committee on Armed Services 

United States Senate 

The Honorable Duncan L. Hunter 

Chairman 

Committee on Armed Services 

House of Representatives 

DOD’s planned investment in research, development, and procurement of 

major weapon systems will total approximately $1.3 trillion between 2005 

and 2009, with over $800 billion of that investment yet to be made. DOD is 

facing a significant number of problems in managing its acquisitions. 

Military operations in Afghanistan and Iraq are consuming a large share of 

DOD resources and causing the department to invest more money sooner 

than expected to replace or fix existing weapons. Meanwhile, DOD is 

intent on transforming military operations while pursuing multiple 

megasystems that are expected to be the most expensive and complex 

ever. These costly conditions coupled with increases in spending for other 

national priorities, such as health care and social security, make it 

essential that DOD effectively leverage its investments, particularly in 

weapon system acquisitions. If DOD manages its current portfolio of 

weapons within traditional margins of error, the financial consequences 

could be dire. 

DOD’s strategy for acquiring major weapon systems has traditionally been 

to plan programs that would achieve a big leap forward in capability 

within a single development program, a strategy that often results in major 

cost and schedule problems. We have assessed weapon acquisitions as a 

high-risk area for 15 years, and although U.S. weapons are among the best 

in the world, the programs to acquire them have continued to produce 

poor cost and schedule outcomes. However, the current defense 

acquisition environment continues to be characterized by cost and 

schedule growth, a lack of confidence by congressional and DOD leaders, 

and no appreciable improvement in the defense acquisition system. DOD 

United States Government Accountability Office

Washington, DC 20548 
Page 2 GAO-06-368 DOD Acquisition Policy 

knows what to do to achieve better outcomes. It has written into policy an 

approach that advocates that adequate knowledge be attained at critical 

junctures before DOD managers agree to invest more money in the next 

phase of weapon system development. The policy also emphasizes 

evolutionary principles for acquiring weapons rather than trying to achieve 

a big leap forward in capability within a single development program. We 

have reported in the past that DOD’s revised policy does not incorporate 

adequate controls to ensure the effective implementation of a knowledgebased, evolutionary acquisition process. However, DOD believes that the 

policy includes the necessary controls to achieve effective outcomes. 

You requested that we evaluate DOD’s compliance with and 

implementation of its revised acquisition policy intended to produce better 

cost, schedule, and performance outcomes for major acquisition 

programs. In order to obtain an early assessment of the cost and schedule 

impact of the revised policy, and to assess DOD’s effectiveness in 

implementing a knowledge-based, evolutionary acquisition approach we 

assessed (1) the cost and schedule status of major weapons development 

programs initiated under the revised policy, (2) whether the policy’s 

knowledge-based, evolutionary acquisition principles are being effectively 

implemented, and (3) whether effective controls and specific criteria are 

in place and being used to make sound investment decisions. 

In conducting our evaluation, we reviewed pertinent acquisition statutes, 

policies, and guidance; analyzed development cost and schedule data for 

23 major acquisition programs approved to start system development 

under DOD’s revised acquisition policy between October 2000 and 

December 2004; conducted case study reviews of nine of those 

23 programs; and interviewed officials from the Office of Secretary of 

Defense and each of the military services. We conducted our review from 

May 2005 to February 2006 in accordance with generally accepted 

government auditing standards. Additional information about our 

methodology is contained in appendix I. 

DOD’s revised acquisition policy has not led to improved acquisition 

program outcomes. Programs initiated under the revised policy are 

already beginning to experience cost and schedule problems similar to 

programs managed under prior versions of the policy. Although the 

programs we reviewed have been in development for only a short period 

of time, nearly half are already estimating development cost growth 

greater than 30 percent or are expecting to delay initial delivery to the 

warfighter by at least 1 year. Program officials are facing the familiar 

Results in Brief 
Page 3 GAO-06-368 DOD Acquisition Policy 

predicament of having to add unplanned money or time or to reduce 

system capabilities and quantities after initial business cases have been 

approved and system development has begun. As a result, DOD is reducing 

its buying power and is not achieving the return on investment it expected 

when the programs began. 

Acquisition officials are not effectively implementing the revised 

acquisition policy’s knowledge-based process. They regularly bypass key 

phases of the early acquisition process, approach key decision points with 

limited knowledge about critical technologies and system design, and do 

not employ evolutionary acquisition principles. Nearly 80 percent of the 

programs we assessed were permitted to bypass the policy’s initial major 

decision review and the associated systems-engineering process that are 

intended to ensure that a system’s requirements match available resources 

and that a sound business case is developed prior to starting system 

development. By not consistently following key processes and strategies, 

acquisition officials are not ensuring that a solid foundation of knowledge 

about cost, schedule, and performance is established before allowing 

programs to start system development, thus resulting in unexecutable 

business cases. Although the policy explicitly states that programs shall 

increase program knowledge by maturing technologies before beginning 

system development, we found that almost three-fourths of the programs 

started since the policy was revised began development with immature 

critical technologies. Our analysis also indicates that decision makers are 

continuing to commit programs to system demonstration and initial 

manufacturing before officials have demonstrated high levels of design 

knowledge, as emphasized in the policy. In addition, programs like the 

Joint Strike Fighter and Future Combat System are still structured to 

achieve major leaps in capability within a single development program, a 

strategy that has historically proven to be problematic in terms of cost and 

schedule outcomes. 

Effective implementation of the revised policy is limited by the absence of 

effective controls that require compliance and specific criteria for clearly 

demonstrating that acceptable levels of knowledge about technology, 

design, and manufacturing have been attained at critical junctures during 

system development before making further investments in a program. 

Without effective controls, the policy cannot prevent DOD decision 

makers from starting system development even when they face significant 

unknowns about technology, design, and production. Without specific 

criteria—or standards against which a judgment or decision is quantifiably 

based—decision makers are permitted to make decisions on the basis of 

subjective judgment. We reported this condition in 2003, yet DOD has not 
Page 4 GAO-06-368 DOD Acquisition Policy 

closed gaps in the policy. In the absence of such controls and criteria, 

DOD faces the added problems of transparency and accountability 

because it often does not sufficiently document the rationale for its 

decisions to allow acquisition programs to advance with low levels of 

technology, design, and manufacturing knowledge. 

This report contains recommendations that DOD require programs to meet 

specific knowledge-based criteria at each key decision points in the 

acquisition process and require decision makers to provide clear and 

specific rationale for their decisions. In addition, we recommend that 

before programs enter system development they should be required to 

complete disciplined concept and technology development phases that 

include specific activities dedicated to capturing knowledge critical to 

developing an executable business. DOD partially concurred with our 

recommendations. DOD agrees that knowledge-based decision making is 

consistent with sound business practice and stated that it would continue 

to develop policy that reflects a knowledge-based approach and improves 

acquisition outcomes. DOD also agrees that acquisition decisions should 

be documented, decision makers should be held accountable, and that 

they should provide the rationale for their decisions. 

Historically, DOD’s programs for acquiring major weapon systems have 

taken longer, cost more, and often delivered fewer quantities and other 

capabilities than planned. GAO has documented these problems for 

decades. In 1970, GAO reported that considerable cost growth had been 

and was continuing to occur on many current development programs. 

Since that report was issued, numerous changes have been made to DOD’s 

acquisition process and environment to try to improve acquisition 

outcomes. Those changes include numerous executive branch initiatives 

and legislative actions as well as roughly 11 revisions to DOD’s acquisition 

policy between 1971 and 2005. Despite these efforts, defense acquisition 

programs in the past 3 decades continued to routinely experience cost 

overruns, schedule slips, and performance shortfalls. 

Figure 1 illustrates the continued problem of development cost overruns. 

The figure depicts the combined cost overruns for large development 

programs (programs totaling more than $1 billion for research, 

development, testing and evaluation in fiscal year 2005 dollars) in each of 

the past 3 decades. The figure also identifies some of the major studies and 

improvement efforts initiated during this time frame. As the figure 

illustrates, efforts to improve acquisition outcomes have not been 

successful in curbing acquisition cost problems. Programs initiated in the 

Background 
Page 5 GAO-06-368 DOD Acquisition Policy 

1970s exceeded DOD’s initial investment estimate by 30 percent, or 

$13 billion (in fiscal year 2005 dollars), and similar outcomes continued 

during the subsequent decades despite numerous reform efforts and policy 

revisions. 

Figure 1: Development Cost Overruns by Decade (in Fiscal Year 2005 Dollars) and Key Reform Efforts 

Since the mid-1990s, we have studied the best practices of leading 

commercial companies. Taking into account the differences between 

commercial product development and weapons acquisitions, we 

articulated a best practices product development model that relies on 

increasing knowledge when developing new products, separating 

technology development from product development, and following an 

evolutionary or incremental product development approach. This 

knowledge-based approach requires developers to make investment 

decisions on the basis of specific, measurable levels of knowledge at 

critical junctures before investing more money and before advancing to 

the next phase of acquisition. An evolutionary product development 

process defines the individual increments on the basis of mature 

technologies and a feasible design that are matched with firm 

requirements. Each increment should be managed as a separate and 

1981 Carlucci Initiatives

1982 Grace Commission

1986 Packard Commission

Development 

cost overrun:

$12 billion

(39%)

Development 

cost overrun:

$15 billion

(40%)

Key Studies and Initiatives Impacting the Defense Acquisition Process

1970 - 1979 1980 - 1989 1990 - 1999

1970 Fitzhugh Commission

1972 Commission on Government 

Procurement

Development 

cost overrun:

$13 billion

(30%)

DOD Acquisition Policy Changes

1994 Federal Acquisition Streamlining Act

1996 Clinger-Cohen Act

1971 DOD 5000 policy established

1975 Policy revised

1977 Policy revised 

1980 Policy revised

1982 Policy revised

1985 Policy revised

1986 Policy revised

1987 Policy revised

1991 Policy revised

1996 Policy revised

Source: DOD (data); GAO (analysis and presentation).
Page 6 GAO-06-368 DOD Acquisition Policy 

distinct acquisition effort with its own cost, schedule and performance 

baseline. An increment that excludes one of these key elements puts an 

extra burden on decision makers and provides a weak foundation for 

making development cost and schedule estimates. The knowledge-based, 

evolutionary approach in our model is intended to help reduce 

development risks and to achieve better program outcomes on a more 

consistent basis. 

Hoping to improve acquisition outcomes, DOD leaders initiated significant 

revisions to the department’s acquisition policy again in October 2000, by 

adopting the knowledge-based, evolutionary system development 

approach.1

 We reported in November 2003, that much of the revised policy 

agrees with GAO’s extensive body of work and that of successful 

commercial firms. DOD’s revised policy emphasizes the importance of and 

provides a good framework for capturing knowledge about critical 

technologies, product design, and manufacturing processes. If properly 

implemented and enforced this approach could help DOD’s decision 

makers gain the confidence they need to make significant and sound 

investment decisions for major weapon systems. Furthermore, the policy’s 

emphasis on evolutionary system development sets up a more manageable 

environment for achieving knowledge. We also noted that DOD’s policy 

strongly suggests the separation of technology development from system 

development, a best practice that helps reduce technological risk at the 

start of a program and makes cost and delivery estimates much more 

predictable.2

 Figure 2 depicts in general how DOD’s revised policy adopts 

key aspects of the best practices model. 

 1

In addition to the acquisition policy, the process used by DOD to establish program 

funding, known as the Planning, Programming, Budgeting and Execution System (PPBES), 

and the process used to determine system requirements, now called the Joint Capabilities 

Integration and Development System (JCIDS), also impact program outcomes. Both 

processes are currently being studied to determine if any changes could be made to 

improve program performance as it relates to funding and requirements. 

2

GAO. Defense Acquisitions: DOD’s Revised Policy Emphasizes Best Practices, but More 

Controls Are Needed, GAO-04-53 (Washington, D.C.: Nov. 10, 2003). 
Page 7 GAO-06-368 DOD Acquisition Policy 

Figure 2: Comparison of DOD’s Revised Policy and Commercial Best Practices Model 

Although DOD took significant steps in the right direction, its policy does 

not include controls that require program officials to meet the key criteria 

that we believe are necessary for ensuring that acceptable levels of 

knowledge are actually captured before making additional significant 

investments. We previously recommended that DOD design and 

implement necessary controls to ensure that appropriate knowledge is 

captured and used to make decisions about moving a program forward 

and investing more money at critical junctures. DOD officials 

acknowledged the advantages of using knowledge-based controls, but 

stated that they believed the policy already included enough controls to 

achieve effective program results. The officials agreed to monitor the 

acquisition process to assess the effectiveness of those controls and to 

determine whether additional ones are necessary. 

Program start

Production

commitment

Knowledge point 1

Technologies, time, funding, and

other resources match customer needs

Knowledge point 3

Production meets cost, schedule,

and quality targets

Concept

refinement

Concept & technology

development Integration Demonstration

Integration Demonstration

Technology

development

System development & demonstration Production &

deployment

Initial Full-rate

Operations & 

support

Design readiness

review

Full-rate production

decision review

Knowledge point 2

Design performs as

expected

Best practices model

DOD’s 5000 policy

Product development Production

Source: DOD (data); GAO (analysis and presentation).

A B C
Page 8 GAO-06-368 DOD Acquisition Policy 

The cost and schedule outcomes being achieved by development programs 

initiated since DOD first issued its revised policy have not improved over 

those achieved by programs managed under prior versions of the policy. 

Of the 23 major programs we assessed, 10 have already reported estimated 

development cost growth greater than 30 percent or expected delays of at 

least 1 year in delivery of an initial operational capability to the warfighter. 

These programs combined represent a cost increase of $23 billion (in 

fiscal year 2005 dollars) and an average delay in delivery of initial 

capability of around 2 years. Most of the other programs were still in the 

early stages as of December 2005 with over half of system development 

remaining and had not yet reported an adequate amount of cost or 

schedule data to effectively analyze their progress. Table 1 contains the 

cost and schedule increases for the 23 programs we assessed, expressed 

as a percentage of each program’s development estimate. 

DOD’s Revised Policy 

Has Not Improved 

Development 

Program Outcomes 
Page 9 GAO-06-368 DOD Acquisition Policy 

Table 1: Cost and Schedule Outcomes for 23 Programs Initiated under the Revised Policy (as of December 2005) 

Program 

Percent growth in estimated 

development costa

Percent growth in estimated 

development schedule

Expeditionary Fighting Vehicle 61% 70%

Active Electronically Scanned Array radar (upgrade 

for F/A-18 E/F fighter/attack aircraft) 14% 1%

Global Hawk unmanned aerial vehicle 166% Undetermined

Joint Strike Fighter 30% 23%

UH-60M helicopter upgrade 151% 25%

C-130 Avionics Modernization Program 122% Undetermined 

C-5 Reliability Enhancement and Re-engining 

Program 0% 25%

Joint Tactical Radio System Cluster 1 31% 44%

Joint Tactical Radio System Waveform 44% Undetermined

Advanced Anti-radiation Guided Missile 7% 0%

Multi-Platform Radar Technology Insertion Program 0% Undetermined

Future Combat System 48% 53%

E-2 Advanced Hawkeye 5% 0%

Warfighter Information Network-Tactical 0% 0%

Small Diameter Bomb 0% 0%

EA-18G 7% 0%

Joint Tactical Radio System Cluster 5 0% 2%

Multi-Mission Maritime Aircraft 0% 0%

Standard Missile-6 Extended Range Active Missile 

Block 1 0% 0%

Aerial Common Sensor 45% 36%

B-2 Radar Modernization Program 0% 0%

Patriot/Medium Extended Air Defense System 

Combined Aggregate Program (fire unit) 0% 0%

Mission Planning System 0% 0%

Sources: DOD (data); GAO (analysis and presentation). 

a

Cost growth is expressed as the percent change in program development cost estimates in fiscal 

year 2005 dollars. 

The Army’s Future Combat System is a case in point. Less than 3 years 

after program initiation and with $4.6 billion invested, the Army has 

already increased its development cost estimate $8.9 billion or 48 percent 

and delayed delivery of initial capability by 4 years over the original 

business case. Similarly, just over 1 year after initiating development of the 

Aerial Common Sensor aircraft, the Army has reported that severe weight 
Page 10 GAO-06-368 DOD Acquisition Policy 

and design problems discovered during development have stopped work 

on the program. As a result, program officials are anticipating at least a 

45 percent cost increase and a delay of 2 years in delivering an initial 

capability to the warfighter. These two Army programs are not the only 

ones experiencing problems. Table 2 contains cost and schedule data for 

6 of the 10 largest development programs initiated under the revised 

policy, including the Future Combat System and Aerial Common Sensor. 

As the table illustrates there are several programs experiencing large cost 

increases and schedule delays. 

Table 2: Cost and Schedule Outcomes for 6 of the 10 Largest Development Programs Sorted by Percent of System 

Development Remaining 

Programs 

Percent development cost 

growth 

Delay in delivery of initial 

capability in months 

Percent of development 

remaining 

Aerial Common Sensor 45% 24 85% 

Future Combat System 48% 48 78% 

Joint Strike Fighter 30% 23 60% 

Expeditionary Fighting Vehicle 61% 48 49% 

C-130 Avionics Modernization 

Program 

122% Delays anticipated due to 

program restructure 

Undetermined due to program 

restructure 

Global Hawk (RQ-4B) 166% Delays anticipated due to 

program restructure 

Undetermined due to program 

restructure 

Sources: DOD (data); GAO (analysis and presentation). 

A good measure of acquisition performance is return on investment as 

expressed in acquisition program unit cost because unit cost represents 

the value DOD is getting for its acquisition dollars invested in a certain 

program. The programs listed in table 2 will not achieve the return on 

investment that DOD anticipated when they began development. In the 

case of Joint Strike Fighter, for example, DOD initially intended to 

purchase 2,866 aircraft at an acquisition program unit cost of about 

$66 million. The Navy has reduced the number of Joint Strike Fighter 

aircraft it plans to buy; technology and design problems encountered 

during development have led to the significant cost growth. As a result, the 

acquisition program unit cost is now about $84 million, an increase of 

27 percent. We recently reported that the risk of even greater increases is 

likely because flight testing has not yet started and the acquisition strategy 

involves substantial overlap of development and production. Similar 

problems have led to increases in the Future Combat System program. At 

program initiation, the Army anticipated that each of 15 units would cost 

about $5.5 billion to develop and deliver. Since that time, instability in the 

program’s technologies and requirements have led to significant cost 
Page 11 GAO-06-368 DOD Acquisition Policy 

increases, leading to a 54 percent increase in acquisition program unit 

cost, now estimated to be $8.5 billion. 

Regarding all 23 development programs, DOD leaders originally planned to 

invest a total of about $83 billion (fiscal year 2005 dollars) for system 

development and anticipated delivering an initial operational capability to 

the warfighter in 77 months on average. However, development costs have 

grown and delivery schedules have been delayed significantly. DOD now 

expects to invest over $106 billion in those same programs, an increase of 

over $23 billion or 28 percent. The delivery of initial capability to the 

warfighter is expected to take an average of 88 months or nearly 1 year 

longer than originally planned. Figure 3 shows changes in these business 

case elements for these programs in the short time since their initiation. 

Figure 3: Cost and Schedule Growth under DOD’s Revised Policy 
Page 12 GAO-06-368 DOD Acquisition Policy 

DOD is not effectively implementing the knowledge-based process and 

evolutionary approach emphasized in its acquisition policy. While the 

policy outlines a specific knowledge-based process of concept refinement 

and technology development3

 to help ensure a sound business case is 

developed before committing to a new development program, almost 

80 percent of the programs we reviewed were permitted to bypass this 

process. Furthermore, the policy emphasizes the need to mature all critical 

technologies before starting system development and to demonstrate that 

the product’s design is mature before beginning system demonstration. 

However, nearly three-fourths of the programs reported having immature 

critical technologies when they received approval to start development, 

and at least half of the programs had not achieved design maturity before 

holding their design review and gaining approval to enter the system 

demonstration phase of development. The policy also emphasizes the use 

of an evolutionary product development approach, yet program officials 

continue to structure major acquisition programs to achieve large 

advances in capability within a single step development program. This 

strategy has historically resulted in poor cost and schedule outcomes. 

DOD decision makers continue to approve programs for system 

development that have not followed key elements of the policy’s suggested 

knowledge-based process. The policy requires program managers to 

provide senior decision makers with knowledge about key aspects of a 

system at critical investment points in the acquisition process. Our prior 

reviews have identified those critical points as the start of system 

development or program start (referred to as Milestone B in the DOD 

acquisition policy), design readiness review separating system integration 

and system demonstration, and production commitment (Milestone C in 

the DOD acquisition policy). The most important point occurs at program 

start, when system development begins. DOD acquisition guidance 

emphasizes the importance of the acquisition phases preceding program 

start, noting that the decisions made during those phases—concept 

 3

According to DOD Instruction 5000.2, the concept refinement phase is intended to refine 

the initial concept and develop a technology development strategy. Concept refinement 

ends when the decision authority approves a preferred solution resulting from the analysis 

of alternatives and approves the associated technology development strategy. After 

concept refinement, a project enters technology development at Milestone A, when the 

decision maker has approved the technology development strategy. The purpose of this 

phase is to reduce technology risk and to determine the appropriate set of technologies to 

be integrated into a full system. 

DOD Is Not 

Effectively 

Implementing the 

Policy’s KnowledgeBased, Evolutionary 

Approach 

Knowledge-Based Process 

Not Enforced 
Page 13 GAO-06-368 DOD Acquisition Policy 

refinement and technology development—generally define the nature of 

an entire acquisition program. 

Acquisition officials continue to begin system development without 

following early processes for developing executable business cases. A 

business case should provide demonstrated evidence that (1) the 

warfighter’s needs are real and necessary and that they can best be met 

with the chosen concept and (2) the chosen concept can be developed and 

produced within existing resources—including technologies, design 

knowledge, funding, and the time to deliver the product when it is needed. 

Establishing a business case calls for a realistic assessment of risks and 

costs; doing otherwise undermines the intent of the business case and 

invites failure. This process requires the user and developer to negotiate 

whatever trade-offs are needed to achieve a match between the user’s 

requirements and the developer’s resources before system development 

begins. 

The revised policy and associated guidance emphasize the importance of 

following a sound process of systems engineering4

 and decision making 

prior to initiating a system development program. The process established 

in the policy consists of two phases, concept refinement and technology 

development, and a major decision review called Milestone A, which if 

rigorously followed, would provide acquisition officials with an 

opportunity to assess whether program officials had the knowledge 

needed to develop an executable business case. However, almost 

80 percent of the programs we reviewed began system development 

without holding any prior decision review. Senior officials with the Office 

of the Secretary of Defense confirmed that this is a common practice 

among defense acquisition programs. This practice eliminates a key 

opportunity for decision makers to assess early product knowledge 

needed to establish a business case that is based on realistic cost, 

schedule, and performance expectations. 

Although program officials conduct analysis before starting a development 

program, they do not consistently follow a process to capture the critical 

knowledge needed to produce executable business cases, as evidenced by 

 4

Systems engineering is a technical management tool that provides the knowledge 

necessary to translate requirements into specific, achievable capabilities. By using the tools 

of systems engineering during these early phases of concept refinement and technology 

development acquisition decision makers and developers can work together to close gaps 

between requirements and available resources—well before system development starts. 

Acquisition Officials Are Not 

Effectively Using Early 

Processes to Develop 

Executable Business Cases 
Page 14 GAO-06-368 DOD Acquisition Policy 

the poor outcomes current programs are experiencing. Officials with the 

Office of the Secretary of Defense recognized this lack of rigor and 

discipline in acquisition process, and in February 2004, the Under 

Secretary of Defense (Acquisition, Technology and Logistics) issued a 

department-wide policy memorandum directing acquisition officials to 

place greater emphasis on systems engineering when planning and 

managing acquisition programs. The policy requires programs to develop a 

systems engineering plan that describes the programs’ overall technical 

approach, including processes, resources, metrics, and applicable 

performance incentives. Although DOD’s systems engineering initiative 

has the potential to improve program performance, officials have found 

that the preliminary results are mixed. Early analysis shows that 

implementation is inconsistent while program officials learn to develop 

and implement systems-engineering plans. 

DOD decision makers continue to permit programs to enter system 

development before critical technologies are mature. Our review of 

technology readiness assessments and acquisition decision memorandums 

for our nine case study programs found that seven of the nine programs 

were approved to begin development even though program officials 

reported levels of knowledge below the criteria suggested in the policy 

and associated guidance, specifically in the area of technology maturity.5

Those seven programs are not isolated cases. As illustrated in Figure 4, 

13 of the programs (nearly three-fourths) that received approval to enter 

system development under the revised policy did so with less than 

100 percent of their critical technologies mature to the level specified by 

DOD. Only 2 of those programs had more than 75 percent of their 

technologies mature when they began (see appendix III for technology 

maturity data for each program). 

 5

DOD’s revised policy emphasizes the importance of reducing technology risk and 

demonstrating technologies in a relevant environment (technology readiness level 6) prior 

to program start. A technology readiness level of 6 means the technology should be very 

close to the planned form, fit, and function of its physical configuration and that it has been 

tested or proven to work in a relevant environment such as a laboratory. GAO recommends 

a higher level of maturity in its best practice model based on best commercial practices. 

This would require a demonstration of the technology in the environment it is expected to 

be used. 

Programs Continue to Enter 

System Development with 

Immature Technologies 
Page 15 GAO-06-368 DOD Acquisition Policy 

Figure 4: Comparison of Programs with Mature versus Immature Technologies at 

Start of System Development 

Note: This figure depicts technology maturity status for 18 of the 23 programs in our review. 

Technology maturity data was not available for the other 5 programs. 

Even though acquisition policy states that technologies shall be mature 

before beginning system development, the practice of accepting high 

levels of technology risk at program start continues to be the norm and not 

the exception. An official with Office of the Secretary of Defense 

responsible for reviewing and validating program assessments of 

technology maturity informed us that the office generally views immature 

critical technologies at the beginning of development as an acceptable risk 

as long as program officials can show that they have a plan to mature the 

technologies by the time the program reaches its design readiness review, 

which requires additional investments to move a program from system 

integration into system demonstration. Therefore, risk management plans 

are consistently viewed as acceptable substitutes for demonstrated 

knowledge. 

In addition to emphasizing the importance of capturing technology 

knowledge before starting system development, DOD’s policy also 

highlights the importance of demonstrating design maturity before moving 

from the integration phase of system development into system 

demonstration and initial manufacturing. The policy establishes a design 

readiness review between the two phases to determine whether a 

product’s design is mature and stable and whether the product is ready to 

move ahead. While DOD’s policy does not require programs to 

Programs Continue Past Design 

Reviews before Design 

Maturity is Demonstrated 
Page 16 GAO-06-368 DOD Acquisition Policy 

demonstrate any specific level of design maturity, our past work has found 

that a key indicator of design maturity is the completion of 90 percent of 

the system’s engineering drawings. We found that defense programs that 

moved forward with lower levels of design maturity, as indicated by 

drawing completion, encountered costly design changes and parts 

shortages that, in turn, caused labor inefficiencies, schedule delays, and 

quality problems. Consequently, those programs required significant 

increases in resources—time and money—over what was estimated at the 

point each program entered the system demonstration phase. 

We analyzed engineering drawing completion data for 8 programs initiated 

under the revised policy that have held a design review,6

 and found that 

more than half of those programs had not completed 90 percent of their 

design drawings before they received approval to enter the system 

demonstration phase of development. We also analyzed drawing-release 

data for three programs that have not yet held their design review but have 

projected the number of drawings officials anticipate will be completed 

when their reviews are held. Based on projections provided by program 

officials, 2 of those 3 programs are expected to have less than 55 percent 

of their drawings complete before they seek approval to begin system 

demonstration and initial manufacturing. 

 6

Some programs did not report having a design readiness review but did report having a 

critical design review. Where this was the case, we assessed those programs’ drawing data 

at their critical design review. 
Page 17 GAO-06-368 DOD Acquisition Policy 

Table 3: Assessment of Program Design Maturity 

Program 

Percentage of design drawings 

complete at design review

Joint Tactical Radio System Cluster 1 28%

Global Hawk unmanned aerial vehicle 33%

Active Electronically Scanned Array radar 

(upgrade for F/A-18 E/F fighter/attack aircraft) 59%

Expeditionary Fighting Vehicle 81%

B-2 Radar Modernization Program 84%

E-2 Advanced Hawkeye 90%

EA-18G 97%

C-5 Reliability Enhancement and Re-engining 

Program 98%

Multi-Platform Radar Technology Insertion 

Program 100%a

Joint Strike Fighter 52%a

Aerial Common Sensor 39%a

Sources: DOD (data); GAO (analysis and presentation). 

a

Program office projections. 

Despite the revised policy’s guidance that capabilities should be developed 

and delivered in individually defined and separately managed increments, 

a majority of major weapon acquisition programs we assessed continue to 

be structured to achieve revolutionary increases in capability within one 

development program. According to the policy, the objective of an 

evolutionary approach is to balance needs and available capability with 

resources and put capability into the hands of the user quickly. The policy 

states that the success of the strategy depends on consistent and 

continuous definition of requirements and the maturation of technologies 

that lead to disciplined development and production of systems that 

provide increasing capability. In this approach, requirements that cannot 

be satisfied within these limits as well as available financial resources 

must wait for future generations of the product and be managed as 

separate system development programs with separate milestones, costs, 

and schedules. In our case studies of nine acquisition programs initiated 

under the revised policy, we found only one program—the Small Diameter 

Bomb—that satisfied all of the criteria of an evolutionary approach. In five 

case studies, we found that program officials had claimed that their 

Evolutionary Acquisition Is 

Not Being Used 
Page 18 GAO-06-368 DOD Acquisition Policy 

programs were evolutionary, yet our evidence shows they were not 

evolutionary in practice;7

 and in three cases, program officials chose not to 

use evolutionary acquisition from the outset. Table 3 summarizes our 

assessment of the nine case studies. 

Table 4: Assessment of Program Acquisition Strategies for GAO’s Nine Case Studies 

Programs in GAO’s case study Claim to be evolutionary? 

Meet evolutionary 

criteria? 

Greater than 30% cost 

growth or more than 

1-year schedule slip 

Future Combat System Yes No Yes 

Global Hawk (RQ-4B) Yes No Yes 

Joint Strike Fighter Yes No Yes 

Aerial Common Sensor Yes No Yes 

Multi-Mission Maritime Aircraft Yes No No 

Small Diameter Bomb Yes Yes No 

E-2 Advanced Hawkeye No No No 

Expeditionary Fighting Vehicle No No Yes 

Multiplatform Radar Technology Insertion 

Program 

No No No 

Sources: DOD (data); GAO (analysis and presentation). 

The revised acquisition policy does not contain effective controls that 

require the demonstration of product knowledge measured against 

specific criteria to ensure that acquisition officials make disciplined, 

transparent, and knowledge-based investment decisions. The lack of 

specific required criteria creates an environment in which unknowns 

about technology, design, and manufacturing processes are acceptable. 

Decision makers and program officials are left with no objective measures 

against which to gauge a program’s level of knowledge, making 

accountability difficult. In the absence of criteria, transparency in 

acquisition decisions is essential to ensuring accountability, but key 

decision documents do not provide sufficient information about major 

decisions. DOD believes that acquisition decision memorandums, used to 

document program decisions, provide adequate transparency. However, 

 7

GAO, TACTICAL AIRCRAFT: Opportunity to Reduce Risks in the Joint Strike Fighter 

Program with Different Acquisition Strategy, GAO-05-271 (Washington, D.C.: March 15, 

2005) and GAO, UNMANNED AERIAL VEHICLES: Changes in Global Hawk’s Acquisition 

Strategy Are Needed to Reduce Program Risks, GAO-05-6 (Washington, D.C.: November 5, 

2004). 

Specific Criteria Are 

Needed to Ensure 

Disciplined and 

Transparent 

Investment Decisions 
Page 19 GAO-06-368 DOD Acquisition Policy 

the decision memorandums we reviewed did not contain an explanation of 

the decision maker’s rationale and rarely identify remaining risks, 

especially as they relate to the key knowledge standards emphasized in the 

policy. Further, the timeliness, accessibility, and depth, of the data 

contained in the Selected Acquisition Reports, DOD’s primary means of 

providing Congress with a status report of program performance, inhibits 

the reports’ usefulness as a management and oversight tool. 

In November 2003, we reported that the revised acquisition policy lacked 

many of the controls that leading commercial companies rely on to attain 

an acceptable level of knowledge before making additional significant 

investments.8

 Controls are considered effective if they are backed by 

specific criteria and if decision makers are required to consider the 

resulting data before deciding to advance a program to the next level. 

Controls used by leading companies help decision makers gauge progress 

in meeting cost, schedule, and performance goals and hold program 

managers accountable for capturing relevant product knowledge to inform 

key investment decisions. The controls we have articulated as best 

practices used by successful commercial product developers are listed 

below in table 5. 

 8

GAO, DEFENSE ACQUISITIONS: DOD’s Revised Policy Emphasizes Best Practices, but 

More Controls Are Needed, GAO-04-53 (November 10, 2003). 
Page 20 GAO-06-368 DOD Acquisition Policy 

Table 5: Types of Controls Considered Best Practices for Successful Product 

Development 

Program start (Milestone B): Start of product development 

Demonstrate technologies to high readiness levels 

Ensure that requirements for the product are informed by the systems engineering 

process 

Establish cost and schedule estimates for product on the basis of knowledge from 

preliminary design using system engineering tools 

Conduct decision review for program start 

Design readiness review: Beginning of system demonstration 

Complete 90 percent of design drawings 

Complete subsystem and system design reviews 

Demonstrate with prototype that design meets requirements 

Obtain stakeholders’ concurrence that drawings are complete and producible 

Complete the failure modes and effects analysis 

Identify key system characteristics 

Identify critical manufacturing processes 

Establish reliability targets and growth plan on the basis of demonstrated reliability rates 

of components and subsystems 

Conduct decision review to enter system demonstration 

Production commitment (Milestone C): Initiation of low-rate production 

Demonstrate manufacturing processes 

Build production-representative prototypes 

Test production-representative prototypes to achieve reliability goal 

Test production-representative prototypes to demonstrate product in operational 

environment 

Collect statistical process control data 

Demonstrate that critical processes are capable and in statistical control 

Conduct decision review to begin production 

Sources: GAO (analysis and presentation). 

Some senior officials with the Office of the Secretary of Defense believe 

that the effective use of controls in DOD’s policy and the establishment of 

more specific criteria for decision making would improve program 

outcomes. They note that specific criteria need to be established and that 

programs need to be held accountable to those criteria before being 

permitted to proceed into the next phase. They also note that the criteria 

for moving an acquisition effort from one phase of the process to the next, 

primarily documented in acquisition decision memorandums as exit 
Page 21 GAO-06-368 DOD Acquisition Policy 

criteria, are not typically specific and often do not relate to the key 

knowledge-based criteria suggested in the policy. 

We found this to be true for our nine case study programs. We reviewed 

acquisition decision memorandums in our case studies and determined 

that they were not useful in explaining the decision maker’s rationale and 

in almost all of the cases they did not address the key knowledge criteria 

suggested in the acquisition policy. In most instances, the decision maker 

simply noted that the program being assessed was ready to proceed into 

system development, but did not provide an explanation of the rationale 

for the decision. Senior officials with the Office of the Secretary of 

Defense told us that they agree that a better explanation of the decision 

maker’s rationale, specifically in instances where the knowledge criteria 

are not fully met, would provide transparency and ultimately allow for a 

more accountable decision-making process. The following two examples 

illustrate how decision documentation is lacking: 

• The Future Combat System program received approval to enter system 

development and demonstration in 2003, with 19 percent of its critical 

technologies mature, well below the policy’s standard. The acquisition 

decision memorandum supporting this decision did not provide the 

rationale for approving the system with such a large number of immature 

critical technologies. The memo did direct an updated review of the 

decision 18 months later and that the program “remain flexible and open 

to accommodate trades in the system architecture and in the individual 

systems’ designs.” 

• The Joint Strike Fighter program was approved to enter system 

development in 2001. The acquisition decision memorandum did not 

address the fact that 75 percent of the program’s critical technologies were 

not mature to the policy’s standard. The memorandum did acknowledge 

that the program’s requirements could be changed or modified, noting that 

further refinements in the requirements should be explored as a potential 

way to reduce program costs. However, the memorandum did not explain 

why the decision maker determined that the program should enter 

development without achieving the technology and requirements 

knowledge emphasized in the policy. 

The acquisition decision memorandums for most of the other programs we 

reviewed did not specifically address critical gaps in knowledge, nor did 

they effectively explain the decision makers’ rationale for deeming those 

programs ready to begin system development. In memos where we found a 

reference to key knowledge principles, such as technology maturity, the 
Page 22 GAO-06-368 DOD Acquisition Policy 

decision makers acknowledged that more effort was needed to meet the 

policy’s suggested criteria but considered the risk acceptable to begin 

development. These memos did not explain why risks were considered 

acceptable. For example, the Navy’s Multi-Mission Maritime Aircraft 

program had none of its critical technologies mature at program initiation. 

The decision maker acknowledged the need to further mature the critical 

technologies but approved the program to enter development. Instead of 

holding the program to the policy’s criteria for entering development, the 

decision maker simply directed the Navy to work with the Office of the 

Secretary of Defense to implement risk mitigation and technology 

maturation plans during the integration phase of system development. 

In addition to the lack of transparency provided through acquisition 

decision memoranda, we also found that the data presented to Congress in 

DOD’s Selected Acquisition Reports (SARs) provided only limited 

usefulness as an oversight tool. Since 1969, SARs have been the primary 

means by which DOD reports the status of major weapon system 

acquisitions to Congress. SARs are reports that are expected to contain 

information on the cost, schedule, and performance of major weapon 

systems in comparison with baseline values established at program start, 

full-scale development, and production decision points. Our analysis, as 

well as a previous GAO review,9

 of current and historical SAR data found 

that the timeliness, accessibility, and depth of the data contained in the 

reports limits their usefulness as an oversight tool. Our prior review noted 

that a number of opportunities exist for DOD to give Congress more 

complete information on the performance of major defense acquisition 

programs. DOD agreed that SAR data could be improved to make it more 

useful to Congress. 

Failing to consistently implement the knowledge-based process and 

evolutionary principles emphasized in the revised acquisition policy—

coupled with a lack of specific criteria for making key investment 

decisions—are keeping DOD on its historical path of poor cost and 

schedule outcomes. Most programs are incurring the same scope of cost 

overruns and schedule delays as programs managed under prior DOD 

policies. More consistent use of the early acquisition processes would 

improve the quality and viability of program business cases by ensuring 

 9

GAO, DEFENSE ACQUISITIONS: Information for Congress on Performance of Major 

Programs Can Be More Complete, Timely, and Accessible, GAO-05-182 (March 28, 2005). 

Conclusions 
Page 23 GAO-06-368 DOD Acquisition Policy 

they are founded on knowledge obtained from rigorous and disciplined 

analysis. The initiative by Office of the Secretary of Defense to reinstitute 

the use of systems engineering is a step in the right direction. However, in 

order for this initiative to be effective DOD must establish and enforce 

specific criteria at key decision points. Our past work has identified and 

recommended criteria and controls that should be consistently applied at 

major decision points. The enforcement of these criteria is critical to 

ensuring that programs have the knowledge necessary to successfully 

move forward through the acquisition process. DOD officials have 

acknowledged the advantages of using knowledge-based criteria and 

controls, but believe the policy already includes enough controls to 

achieve effective program results. However, without enforceable criteria, 

defense officials are challenged to determine whether adequate knowledge 

has been obtained for investing taxpayer dollars. The lack of enforceable 

criteria also makes it difficult to hold defense officials accountable for 

their decisions. 

DOD must ensure that appropriate knowledge is captured and used at 

critical junctures to make decisions about moving a program forward and 

investing more money. We recommend that the Secretary of Defense 

require program officials to demonstrate that they have captured 

appropriate knowledge at three key points—program start, design review 

for transitioning from system integration to system demonstration, and 

production commitment—as a condition for investing resources. At a 

minimum those controls should require program officials to demonstrate 

that they have achieved a level of knowledge that meets or exceeds the 

following criteria at each respective decision point: 

• Program start (Milestone B): Start of product development 

• Demonstrate technologies to high readiness levels 

• Ensure that requirements for the product are informed by the systemsengineering process 

• Establish cost and schedule estimates for product on the basis of 

knowledge from preliminary design using system engineering tools 

• Conduct decision review for program start 

Recommendations 

for Executive Action 
Page 24 GAO-06-368 DOD Acquisition Policy 

• Design readiness review: Beginning of system demonstration 

• Complete 90 percent of design drawings 

• Complete subsystem and system design reviews 

• Demonstrate with prototype that design meets requirements 

• Obtain stakeholders’ concurrence that drawings are complete and 

producible 

• Complete the failure modes and effects analysis 

• Identify key system characteristics 

• Identify critical manufacturing processes 

• Establish reliability targets and growth plan on the basis of 

demonstrated reliability rates of components and subsystems 

• Conduct decision review to enter system demonstration 

• Production commitment (Milestone C): Initiation of low-rate 

production 

• Demonstrate manufacturing processes 

• Build production-representative prototypes 

• Test production-representative prototypes to achieve reliability goal 

• Test production-representative prototypes to demonstrate product in 

operational environment 

• Collect statistical process control data 

• Demonstrate that critical processes are capable and in statistical 

control 

• Conduct decision review to begin production 

Furthermore, to ensure that major decisions are transparent and that program 

officials and decision makers are held accountable, we recommend that the 

Secretary of Defense require decision makers to include written rationale for 

each major decision in acquisition decision documentation. The rationale 

should address the key knowledge-based criteria appropriate for milestone 

decisions, explain why a program’s level of knowledge in each area was 

deemed acceptable if criteria have not been met and provide a plan for 

achieving the knowledge necessary to meet criteria within a given time frame. 

DOD provided written comments on a draft of this report. The comments 

appear in appendix II. 

DOD partially concurred with our recommendation that the Secretary of 

Defense should establish specific controls to insure that program officials 

demonstrate that they have captured a level of knowledge that meets or 

exceeds specific criteria at three key points in the acquisition process: 

program start, design readiness review, and production commitment. DOD 

agreed that knowledge-based decision making is consistent with sound 

business practice and stated that it would continue to develop policy that 

Agency Comments 

and Our Evaluation 
Page 25 GAO-06-368 DOD Acquisition Policy 

reflects a knowledge-based approach and improves acquisition outcomes. 

DOD noted that it would consider our recommendations as it reassesses 

the DOD acquisition business model and the knowledge required at each 

decision point. We believe that DOD’s plan to reassess its business model 

provides a good opportunity to establish the controls and specific criteria 

recommended in this report. Therefore, we are retaining our 

recommendation that the Secretary of Defense should establish controls 

to insure that program officials demonstrate that they have captured a 

level of knowledge that meets or exceeds specific criteria at three key 

points in the acquisition process. 

DOD also partially concurred with our recommendation that the Secretary 

of Defense require decision makers to provide written rationale in 

acquisition decision documentation for each major decision. DOD agreed 

that acquisition decisions should be documented, decision makers should 

be held accountable, and that they should provide the rationale for their 

decisions. DOD believes that the implementation of Section 801 of the 

National Defense Authorization Act for FY 2006 reinforces these 

processes. The act calls for the decision maker to certify that the program 

meets certain requirements, such as technology maturity, prior to starting 

a new development program at Milestone B. However, the act is focused 

on the decision to start a development program and does not identify 

specific criteria for programs to be measured against at design readiness 

review or production commitment. We believe our recommendation adds 

transparency and accountability to the process because it requires the 

decision maker to provide the rationale for a decision to allow a program 

to move forward, not only at Milestone B but at other key decision points 

as well. Therefore, we are retaining our recommendation that the 

Secretary of Defense require decision makers to provide written rationale 

for each major decision in acquisition decision documentation. 

We are sending copies of this report to the Secretary of Defense; the 

Secretaries of the Air Force, Army, and Navy; and the Director of the 

Office of Management and Budget. We will provide copies to others on 

request. This report will also be available at no charge on GAO’s Web site 

at http://www.gao.gov. 
Page 26 GAO-06-368 DOD Acquisition Policy 

If you have any questions about this report or need additional information, 

please call me at (202) 512-4841 (sullivanm@gao.gov). Contact points for 

the offices of Congressional Relations and Public Affairs are located on 

the last page of this report. Key contributors to this report were Michael 

Hazard, Assistant Director; Lily Chin; Ryan Consaul; Christopher DePerro; 

Travis Masters; and Adam Vodraska. 

Michael J. Sullivan 

Director, Acquisition 

 and Sourcing Management 
Appendix I: Scope and Methodology 

Page 27 GAO-06-368 DOD Acquisition Policy 

To assess the impact of DOD’s revised acquisition policy, we analyzed cost 

and schedule data for 23 major defense acquisition programs that were 

approved to begin system development under the revised policy. We did 

not assess space, missile defense, or ship programs. We collected our data 

from Selected Acquisition Reports, presidential budget documents, 

ongoing GAO work, and pertinent program officials. We utilized previous 

GAO reports related to defense acquisition policies and worked with 

knowledgeable GAO staff to ensure the use of current, accurate data. We 

also analyzed more than 150 annual Selected Acquisition Reports covering 

a 36-year period from 1969 to 2005, to determine historical trends related 

to outcomes of acquisition policy implementation. 

We assessed whether the revised policy’s knowledge-based, evolutionary 

acquisition principles were being effectively implemented by conducting 

9 case study reviews and analyzing design maturity data for 11 programs 

that have made engineering-drawing data available to GAO. Our case study 

programs were the Aerial Common Sensor, Multi-Platform Radar 

Technology Insertion Program, Global Hawk Unmanned Aerial Vehicle, 

Small Diameter Bomb, Future Combat System, Joint Strike Fighter, 

Expeditionary Fighting Vehicle, Multi-Mission Maritime Aircraft, and the 

E-2 Advanced Hawkeye. We interacted directly with numerous program 

officials to seek input on current developments with their programs. We 

studied program documents to assess how well programs understand and 

are implementing the revised acquisition policy. We also analyzed drawing 

release data for those programs that have either passed their design 

review or have provided GAO with estimated drawing release data for a 

future design review to assess design maturity. In several cases, we asked 

that program offices verify information in these various documents. 

We also reviewed Department of Defense (DOD) Directive 5000.1, DOD 

Instruction 5000.2, and the Defense Acquisition Guidebook. In addition we 

examined each of the military services’ policy directives and guidance, 

DOD memorandums to include policy intent and DOD expectations 

regarding policy implementation as well as Joint Capabilities Integration 

and Development System documents. We interviewed relevant officials in 

Washington, D.C., from the Office of the Director, Defense Research and 

Engineering, the Joint Staff, the Office of the Secretary of Defense, and 

Army, Navy, and Air Force acquisition policy staff in order to better 

understand the content of these documents and the intent of DOD’s policy. 

We conducted our review from May 2005 to February 2006 in accordance 

with generally accepted government auditing standards. 

Appendix I: Scope and Methodology 
Appendix II: Comments from the Department 

of Defense 

Page 28 GAO-06-368 DOD Acquisition Policy 

Appendix II: Comments from the Department 

of Defense 
Appendix II: Comments from the Department 

of Defense 

Page 29 GAO-06-368 DOD Acquisition Policy 
Appendix II: Comments from the Department 

of Defense 

Page 30 GAO-06-368 DOD Acquisition Policy 
Appendix III: Program Data for 23 Programs 

Initiated under DOD’s Revised Acquisition 

Policy (as of December 2005) 

Page 31 GAO-06-368 DOD Acquisition Policy 

Program 

Program 

start 

Formal 

Milestone Ia

 or 

Milestone A 

decision 

review? 

Percent 

technology 

mature (TRL 

6) at program 

start 

Percent 

design 

drawings 

complete at 

design review 

Percent 

growth in 

estimated 

development 

costc

Percent 

growth in 

estimated 

development 

schedule 

Expeditionary Fighting Vehicle 12/2000 Yes 80% 81% 61% 70% 

Active Electronically Scanned 

Array radar (upgrade for F/A-18 

E/F fighter/attack aircraft) 

12/2000 No 0% 59% 14% 1% 

Global Hawk unmanned aerial 

vehicle 

2/2001 No 0% 33% 166% Undetermined 

UH-60M helicopter upgrade 4/2001 No Not available Not available 151% 25% 

C-130 Avionics Modernization 

Program 

8/2001 No 100% Not available 122% Undetermined 

Joint Strike Fighter 10/2001 Yes 25% 52%b 30% 23% 

C-5 Reliability Enhancement and 

Re-engining Program 

11/2001 Yes 100% 98% 0% 25% 

Joint Tactical Radio System 

Cluster 1 

6/2002 No 0% 28% 31% 44% 

Joint Tactical Radio System 

Waveform 

6/2002 No Not available Not available 44% Undetermined 

Advanced Anti-radiation Guided 

Missile 

4/2003 No Not available Not available 7% 0% 

Multi-Platform Radar Technology 

Insertion Program 

4/2003 No 100% 100% b 0% Undetermined 

Future Combat System 5/2003 No 19% Not available 48% 53% 

E-2 Advanced Hawkeye 6/2003 No 50% 90% 5% 0% 

Warfighter Information NetworkTactical 7/2003 No 25% Not available 0% 0% 

Small Diameter Bomb 10/2003 Yes 100% Not available 0% 0% 

EA-18G 11/2003 No 60% 97% 7% 0% 

Joint Tactical Radio System 

Cluster 5 

4/2004 No 50% Not available 0% 2% 

Multi-Mission Maritime Aircraft 5/2004 No 0% Not available 0% 0% 

Standard Missile-6 Extended 

Range Active Missile Block 1 

6/2004 No Not available Not available 0% 0% 

Aerial Common Sensor 7/2004 Yes 50% 39% b 45% 36% 

B-2 Radar Modernization 

Program 

7/2004 No 100% 84% 0% 0% 

Patriot/Medium Extended Air 

Defense System Combined 

Aggregate Program (fire unit) 

8/2004 No 83% Not available 0% 0% 

Mission Planning System 12/2004 No Not available Not available 0% 0% 

Appendix III: Program Data for 23 Programs 

Initiated under DOD’s Revised Acquisition 

Policy (as of December 2005) 
Appendix III: Program Data for 23 Programs 

Initiated under DOD’s Revised Acquisition 

Policy (as of December 2005) 

Page 32 GAO-06-368 DOD Acquisition Policy 

Sources: DOD (data); GAO (analysis and presentation). 

Note: In this table the term “not available” means that GAO had not received sufficient data to make 

an assessment of the given program’s design and/or technology maturity. 

a

Milestone I was a forerunner to Milestone A, the decision review that currently precedes the start of 

technology development. 

b

Program office projections. 

c

Cost growth is expressed as the percent change in program development cost estimates in fiscal 

year 2005 dollars. 
Related GAO Products 

Page 33 GAO-06-368 DOD Acquisition Policy 

DOD Acquisition Outcomes: A Case for Change. GAO-06-257T. 

Washington, D.C.: November 15, 2005. 

Defense Acquisitions: Stronger Management Practices Are Needed to 

Improve DOD’s Software-Intensive Weapon Acquisitions. GAO-04-393. 

Washington, D.C.: March 1, 2004. 

Best Practices: Setting Requirements Differently Could Reduce Weapon 

Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.: February 

11, 2003 

Best Practices: Capturing Design and Manufacturing Knowledge Early 

Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: July 15, 

2002. 

Defense Acquisitions: DOD Faces Challenges in Implementing Best 

Practices. GAO-02-469T. Washington, D.C.: February 27, 2002. 

Best Practices: Better Matching of Needs and Resources Will Lead to 

Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: March 8, 

2001. 

Best Practices: A More Constructive Test Approach Is Key to Better 

Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: July 31, 

2000. 

Defense Acquisition: Employing Best Practices Can Shape Better 

Weapon System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: April 

26, 2000. 

Best Practices: DOD Training Can Do More to Help Weapon System 

Programs Implement Best Practices. GAO/NSIAD-99-206. Washington, 

D.C.: August16, 1999. 

Best Practices: Better Management of Technology Development Can 

Improve Weapon System Outcomes. GAO/NSIAD-99-162. Washington, 

D.C.: July 30, 1999. 

Defense Acquisitions: Best Commercial Practices Can Improve Program 

Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999. 

Defense Acquisition: Improved Program Outcomes Are Possible. GAO/TNSIAD-98-123. Washington, D.C.: March 17, 1998. 

Related GAO Products 
Related GAO Products 

Page 34 GAO-06-368 DOD Acquisition Policy 

Best Practices: DOD Can Help Suppliers Contribute More to Weapon 

System Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998. 

Best Practices: Successful Application to Weapon Acquisition Requires 

Changes in DOD’s Environment. GAO/NSIAD-98-56. Washington, D.C.: 

February 24, 1998. 

Best Practices: Commercial Quality Assurance Practices Offer 

Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: August 26, 

1996. 

(120448) 
The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help 

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost 

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To 

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800 

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER