Report to the Senate and House 

Committees on Armed Services 

United States General Accounting Office 

GAO 

November 2003 DEFENSE 

ACQUISITIONS 

DOD’s Revised Policy 

Emphasizes Best 

Practices, but More 

Controls Are Needed 

GAO-04-53 
www.gao.gov/cgi-bin/getrpt?GAO-04-53.

To view the full product, including the scope 

and methodology, click on the link above. 

For more information, contact Katherine V. 

Schinasi at (202) 512-4841 or 

schinasik@gao.gov. 

Highlights of GAO-04-53, a report to the 

Senate and House Committees on Armed 

Services 

November 2003 

DEFENSE ACQUISITIONS 

DOD’s Revised Policy Emphasizes Best 

Practices, but More Controls Are Needed 

DOD’s new policy supports knowledge-based, evolutionary acquisitions by 

adopting lessons learned from successful commercial companies. One of 

those lessons is a knowledge-based approach, which requires program 

managers to attain the right knowledge at critical junctures—also known 

as knowledge points—so they can make informed investment decisions 

throughout the acquisition process. The policy also embraces an 

evolutionary or phased development approach, which sets up a more 

manageable environment for attaining knowledge. The customer may not 

get the ultimate capability right away, but the product is available sooner 

and at a lower cost. Leading firms have used these approaches—which 

form the backbone of what GAO calls the best practices model—to 

determine whether a project can be accomplished with the time and money 

available and to reduce risks before moving a product to the next stage 

of development. 

By adopting best practices in the acquisition policy, DOD’s leadership has 

taken a significant step forward. The next step is to provide the necessary 

controls to ensure a knowledge-based, evolutionary approach. Implementing 

the necessary controls at all three knowledge points along the acquisition 

process helps decision makers ensure a knowledge-based approach is 

followed. Without controls in the form of measurable criteria that decision 

makers must consider, DOD runs the risk of making decisions based on 

overly optimistic assumptions. Each successive knowledge point builds on 

the preceding one, and having clearly established controls helps decision 

makers gauge progress in meeting goals and ensuring successful outcomes. 

DOD Policy Incorporates Best Practices but Does Not Have Sufficient Controls 

Knowledge point 

Best practices’ intent in 

DOD policy? 

Sufficient controls in 

DOD policy? 

Knowledge point 1 (at program 

launch): Technologies, time, 

funding, and other resources 

match customer needs 

Yes No 

Knowledge point 2 

(between integration and 

demonstration): Design 

performs as expected 

Yes No 

Knowledge point 3 

(at production commitment):

Production meets cost, schedule, 

and quality targets 

Yes No 

Sources: DOD (data), GAO (analysis). 

DOD was responsive to the requirements in the Bob Stump National Defense 

Authorization Act for Fiscal Year 2003. DOD’s responses reflected the 

committee’s specific concerns about the application of certain statutory and 

regulatory requirements to the new evolutionary acquisition process, for 

more guidance for implementing spiral development, and about technology 

readiness at program initiations. 

The Department of Defense’s 

(DOD) investment in new weapon 

systems is expected to exceed 

$1 trillion from fiscal years 2003 to 

2009. To reduce the risk of cost and 

schedule overruns, DOD revamped 

its acquisition policy in May 2003. 

The policy provides detailed 

guidance on how weapon systems 

acquisitions should be managed. 

The Senate report accompanying 

the National Defense Authorization 

Act for Fiscal Year 2004 required 

GAO to determine whether DOD’s 

policy supports knowledge-based, 

evolutionary acquisitions and 

whether the policy provides the 

necessary controls for DOD to 

ensure successful outcomes, such 

as meeting cost and schedule goals. 

The report also required GAO to 

assess whether the policy is 

responsive to certain requirements 

in the Bob Stump National Defense 

Authorization Act for Fiscal Year 

2003 concerning DOD’s 

management of the acquisition 

process. 

GAO recommends that the 

Secretary of Defense strengthen 

DOD’s acquisition policy by 

requiring additional controls to 

ensure decision makers will follow 

a knowledge-based, evolutionary 

approach. DOD partially concurred 

with GAO’s recommendations. 
Page i GAO-04-53 DOD’s Acquisition Policy 

Letter 1

Results in Brief 2

Background 3

DOD’s Revised Policy Provides a Framework for 

Knowledge-Based, Evolutionary Acquisitions 5

Controls Are Needed to Ensure Knowledge-Based Approach 

Is Followed 8

DOD Is Responsive to Congressional Requirements about New 

Acquisition Process 10

Conclusions 14

Recommendations for Executive Action 14

Agency Comments and Our Evaluation 15

Scope and Methodology 16

Appendix I Section 802, Evolutionary Acquisition 19 

Appendix II Section 803, Spiral Development 24 

Appendix III Section 804, Technology Maturity, and Section 822, 

Independent Technology Readiness Assessments 26 

Appendix IV Comments from the Department of Defense 28 

Related GAO Products 31 

Tables 

Table 1: Excerpts of Best Practices Contained in DOD’s New Policy 6

Table 2: Comparison of Controls Used in Best Practices Model and 

DOD Policy 8

Table 3: How DOD Policy Addresses Section 802’s Statutory and 

Regulatory Requirements 19

Contents 
Page ii GAO-04-53 DOD’s Acquisition Policy 

Table 4: How DOD Responded to Section 802’s Requirements 

Regarding Specific Matters for Each Increment of an 

Evolutionary Acquisition Process 21

Table 5: How DOD Policy Addresses Section 803’s Requirements 24

Table 6: How DOD Responded to Section 804 and Section 822 

Requirements 26

Figures 

Figure 1: Distribution of Development Funds for Fiscal Year 2003 5

Figure 2: Comparison of DOD’s Framework and Commercial Best 

Practices Model 7

Abbreviations 

DOD Department of Defense 

DODD DOD Directive 

DODI DOD Instruction 

MDA Milestone Decision Authority 

TEMP Test and Evaluation Master Plan 

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 
Page 1 GAO-04-53 DOD’s Acquisition Policy 

November 10, 2003 

The Honorable John W. Warner 

Chairman 

The Honorable Carl Levin 

Ranking Minority Member 

Committee on Armed Services 

United States Senate 

The Honorable Duncan Hunter 

Chairman 

The Honorable Ike Skelton 

Ranking Minority Member 

Committee on Armed Services 

House of Representatives 

The Department of Defense (DOD) plans to spend more than $1 trillion 

from fiscal years 2003 to 2009 for developing and procuring weapon 

systems. To get the most out of its investment, DOD has set goals to 

develop and procure weapons faster and at less cost. Guiding this effort is 

DOD’s newly revised acquisition policy. Issued in May 2003, the policy 

seeks to foster greater efficiency while building flexibility in the 

acquisition process. The policy embraces a knowledge-based, evolutionary 

framework that emphasizes shorter development times. 

DOD intends to use its policy to improve its record for meeting cost and 

schedule estimates and for delivering capabilities as promised. Congress 

has expressed support for DOD’s efforts to revamp its acquisition process 

and has established requirements that DOD must take to ensure a 

disciplined approach. Recognizing that a consistent and disciplined 

application of policies will be key to achieving desired outcomes, the 

Senate Armed Services Committee directed us to assess DOD’s 

current acquisition policies to determine whether they (1) support 

knowledge-based, evolutionary acquisitions, (2) provide the necessary 

controls to ensure DOD’s policy intent is followed, and (3) respond to 

specific requirements in the Bob Stump National Defense Authorization 

United States General Accounting Office

Washington, DC 20548 
Page 2 GAO-04-53 DOD’s Acquisition Policy 

Act for Fiscal Year 2003 (Public Law 107-314) regarding DOD plans to 

manage its new acquisition process.1

DOD has made major improvements to its acquisition policy by adopting 

knowledge-based, evolutionary practices used by successful commercial 

companies. If properly applied, these best practices can put DOD’s 

decision makers in a better position to deliver high-quality products on 

time and within budget. The policy requires decision makers to have the 

knowledge they need before moving to the next phase of development. 

To ensure that the acquisition environment is conducive to a knowledgebased approach, the policy embraces evolutionary development, which 

allows managers to develop a product in increments rather than trying to 

incorporate all the desired capabilities in the first version that comes off 

the production line. Leading companies who use the knowledge-based 

evolutionary approach have shown that these best practices help reduce 

risks at critical junctures during a product’s development and help ensure 

that decision makers get the most out of their investments. 

DOD’s recent policy changes are a significant step forward. The next step 

is for DOD to provide the necessary controls to ensure a knowledge-based, 

evolutionary approach is followed. For example, the policy does not 

establish measures to gauge design and manufacturing knowledge at 

critical junctures in the product development process. Without specific 

requirements to demonstrate knowledge at key points, the policy allows 

significant unknowns to be judged as acceptable risks, leaving an opening 

for decision makers to make uninformed decisions about continuing 

product development. 

DOD was responsive to the requirements in the Defense Authorization Act 

for Fiscal Year 2003. DOD’s responses reflected the committee’s specific 

concerns about the application of certain statutory and regulatory 

requirements2

 to the new evolutionary acquisition process, for more 

guidance for implementing spiral development, and about technology 

readiness at program initiations. 

 1

 S. Rept. No. 108-46, Report of the Senate Armed Services Committee accompanying S. 

1050, National Defense Authorization Act for Fiscal Year 2004, at page 346 (May 13, 2003). 

2

 As used here, the term “regulatory requirements” refers to policies governing DOD’s 

acquisition system. 

Results in Brief 
Page 3 GAO-04-53 DOD’s Acquisition Policy 

This report makes recommendations that the Secretary of Defense 

strengthen DOD’s acquisition policy by requiring additional controls to 

ensure decision makers will follow a knowledge-based, evolutionary 

approach. DOD partially concurred with our recommendations. DOD 

believes the current acquisition framework includes the controls 

necessary to achieve effective results, but department officials will 

continue to monitor the process to determine whether other controls are 

needed to achieve the best possible outcomes. DOD agreed it should 

record and justify program decisions for moving from one stage of 

development to next but did not agree with the need to issue a report 

outside of the department. 

Traditionally, DOD’s strategy for acquiring major weapon systems has 

been to plan programs that would achieve a big leap forward in capability. 

However, because the needed technologies often are not yet mature, 

programs stay in development for years until the technologies are 

demonstrated. As a result, weapon systems have frequently been 

characterized by poor cost, schedule, and performance outcomes. This has 

slowed modernization efforts, reduced the buying power of the defense 

dollar, delayed capabilities for the warfighter, and forced unplanned—and 

possibly unnecessary—trade-offs among programs. 

Our extensive body of work shows that leading companies use a product 

development model that helps reduce risks and increase knowledge when 

developing new products. This best practices model enables decision 

makers to be reasonably certain about their products at critical junctures 

during development and helps them make informed investment decisions. 

This knowledge-based process can be broken down into three cumulative 

knowledge points. 

• Knowledge point 1: A match must be made between the customer’s 

needs and the developer’s available resources—technology, engineering 

knowledge, time, and funding—before a program starts. 

• Knowledge point 2: The product’s design must be stable and must 

meet performance requirements before initial manufacturing begins. 

• Knowledge point 3: The product must be producible within cost, 

schedule, and quality targets and demonstrated to be reliable before 

production begins. 

To bolster the knowledge-based process, leading companies use 

evolutionary product development, an incremental approach that enables 

developers to rely more on available resources rather than making 

Background 
Page 4 GAO-04-53 DOD’s Acquisition Policy 

promises about unproven technologies. While the user may not initially 

receive the ultimate capability under this approach, the initial product is 

available sooner and at a lower, more predictable cost. Also, leading 

companies know that invention cannot be scheduled and its cost is 

difficult to estimate. They do not bring technology into new product 

development unless that technology has been demonstrated to meet the 

user’s requirements. Allowing technology development to spill over into 

product development puts an extra burden on decision makers and 

provides a weak foundation for making product development estimates. 

DOD understands that it must improve acquisition process outcomes if 

it is to modernize its forces within currently projected resources. To 

help achieve this goal, DOD has revised its acquisition policy, called the 

5000 series, to reflect best practices from successful commercial and DOD 

programs.3

 The policy covers most—but not all—major acquisitions. The 

Secretary of Defense has delegated authority to the Missile Defense 

Agency and to the National Security Space Team to develop separate 

guidance for missile defense and space systems, respectively.4

Approximately 35 percent of DOD’s development funds in 2003 went to 

these systems. (Figure 1 shows how $43.1 billion in development funds 

were distributed across space, missile defense, and systems covered by 

the 5000 series.) This report addresses policy for the defense programs 

covered exclusively under the 5000 series. 

 3

 DOD Directive 5000.1, The Defense Acquisition System, describes the management 

principles for DOD’s acquisition programs. DOD Instruction 5000.2, The Operation of the 

Defense Acquisition System, outlines a framework for managing acquisition programs. 

Collectively, these are known as the 5000 series. 

4

 Program elements of the ballistic missile defense program enter the formal DOD 

acquisition cycle at milestone C (production commitment) and are subject to the 5000 

series from that point on. To use the streamlined process of the National Security Space 

Acquisition Policy, a Space System Program Director/Program Manager must request that 

the DOD Space Milestone Decision Authority (the Under Secretary of the Air Force) grant a 

waiver and an exemption to the processes and procedures described in DOD Instruction 

5000.2 (the waiver authority does not include DOD Directive 5000.1). Notwithstanding 

these policy exceptions, statutory requirements for major defense acquisition programs 

continue to apply to missile defense and space programs. 
Page 5 GAO-04-53 DOD’s Acquisition Policy 

Figure 1: Distribution of Development Funds for Fiscal Year 2003 

Note: Includes research, development, test, and evaluation budget activities for advanced component 

development and prototypes; system development and demonstration; and operational systems 

development. 

DOD’s leaders have made significant improvements to DOD’s acquisition 

policy by adopting the knowledge-based, evolutionary approach used by 

leading commercial companies. The revised policy has the potential to 

transform DOD’s acquisition process by reducing risks and increasing the 

chances for successful outcomes. The policy provides a framework for 

developers to ask themselves at key decision points whether they have 

the knowledge they need to move to the next phase of acquisition. If 

rigorously applied, this knowledge-based framework can help managers 

gain the confidence they need to make significant and sound investment 

decisions for major weapon systems. In placing greater emphasis on 

evolutionary product development, the policy sets up a more manageable 

environment for achieving knowledge. Another best practice reflected in 

the policy’s framework is separating technology development from 

product development, which reduces technological risk at the start of 

a program. 

DOD’s Revised 

Policy Provides a 

Framework for 

Knowledge-Based, 

Evolutionary 

Acquisitions 
Page 6 GAO-04-53 DOD’s Acquisition Policy 

As shown in table 1, DOD’s policy emphasizes best practices used by 

leading companies. 

Table 1: Excerpts of Best Practices Contained in DOD’s New Policy 

Best practices DOD policy 

Knowledge-based acquisition DOD Directive 5000.1 (enclosure 1, p. 5): Program managers “shall provide knowledge 

about key aspects of a system at key points in the acquisition process. ... shall reduce 

technology risk, demonstrate technologies in a relevant environment ... prior to program 

initiation. They shall reduce integration risk and demonstrate product design prior to the 

design readiness review. They shall reduce manufacturing risk and demonstrate 

producibility prior to full-rate production.” 

Evolutionary acquisition DOD Instruction 5000.2 (p. 3): “Evolutionary acquisition is the preferred DOD strategy for 

rapid acquisition of mature technology for the user. ... The objective is to balance needs 

and available capability with resources, and to put capability into the hands of the user 

quickly. The success of the strategy depends on consistent and continuous definition of 

requirements, and the maturation of technologies ... that provide increasing capability 

towards a materiel concept.” 

Separating technology development from 

product development 

DOD Instruction 5000.2 (p. 6): “The project shall exit Technology Development when an 

affordable increment of militarily-useful capability has been identified, the technology for 

that increment has been demonstrated in a relevant environment, and a system can be 

developed for production within a short timeframe (normally less than five years).” 

Source: DOD. 

Similar to the best practices model, DOD’s policy divides its acquisition 

process into phases, as shown in figure 2. Key decisions are aligned with 

the three critical junctures of a product’s development, or knowledge 

points. 
Page 7 GAO-04-53 DOD’s Acquisition Policy 

Figure 2: Comparison of DOD’s Framework and Commercial Best Practices Model 

In other similarities, DOD’s framework pinpoints program start at 

milestone B, about the same point as program start on the best practices 

model. At the midway point on both approaches, a stable product design 

should be demonstrated. With DOD’s framework, managers are required to 

know—by the time full-rate production decision review occurs—whether 

the product can be produced within cost, schedule, and quality targets. 

This requirement occurs earlier in the best practices model, before 

production begins, or at knowledge point 3. Leading companies have 

used this approach to reduce risks and to make costs and delivery dates 

more predictable. 
Page 8 GAO-04-53 DOD’s Acquisition Policy 

While DOD has strengthened its acquisition policy with a knowledgebased, evolutionary framework, the policy does not include many of the 

same controls that leading companies rely on to attain a high level of 

knowledge before making additional significant investments. Controls are 

considered effective if they are backed by measurable criteria and if 

decision makers are required to consider them before deciding to advance 

a program to the next level. Controls used by leading companies help 

decision makers gauge progress in meeting cost, schedule, and 

performance goals and ensure that managers will (1) conduct activities to 

capture relevant product development knowledge, (2) provide evidence 

that knowledge was captured, and (3) hold decision reviews to determine 

that appropriate knowledge was captured to move to the next phase. 

To determine if DOD has the necessary controls, we compared controls in 

DOD’s policy with those used in the best practices model at three critical 

junctures. Table 2 shows the presence or absence of controls for various 

versions of DOD policy since 1996, including the May 2003 revision. 

Table 2: Comparison of Controls Used in Best Practices Model and DOD Policy 

Controls used in DOD policy 

Controls used in commercial best practices model 

March 1996 

policy 

October 2000 

policy 

October 2002 

interim policy 

May 2003 

policy 

Knowledge point 1: Occurs at program launch. Match exists between requirements and resources. Technologies needed to meet 

essential product requirements have been demonstrated to work in their intended environment and the producer has completed a 

preliminary design of the product. 

Demonstrate technologies to high readiness levels X X X 

Ensure that requirements for the product are informed by 

the systems engineering process X 

Establish cost and schedule estimates for product based 

on knowledge from preliminary design using systems 

engineering tools 

Conduct decision review for program launch X X X X 

Knowledge point 2: Occurs between integration and demonstration. Design is stable and has been demonstrated through prototype 

testing. Ninety percent of engineering drawings are releasable to manufacturing organizations. 

Complete 90 percent of design drawings 

Complete subsystem and system design reviews 

Demonstrate with prototype that design meets 

requirements X X X 

Obtain stakeholders concurrence that drawings are 

complete and producible 

Complete failure modes and effects analysis 

Identify key system characteristics 

Controls Are 

Needed to Ensure 

Knowledge-Based 

Approach Is Followed 
Page 9 GAO-04-53 DOD’s Acquisition Policy 

Controls used in DOD policy 

Controls used in commercial best practices model 

March 1996 

policy 

October 2000 

policy 

October 2002 

interim policy 

May 2003 

policy 

Identify critical manufacturing processes 

Establish reliability targets and growth plan based on 

demonstrated reliability rates of components and 

subsystems 

Conduct decision review to enter system demonstration X X 

Knowledge point 3: Occurs at production commitment. Product is ready to be manufactured within cost, schedule, and quality 

targets. All key manufacturing processes have come under statistical control and product reliability has been demonstrated. 

Demonstrate manufacturing processes 

Build production representative prototypes 

Test production representative prototypes to achieve 

reliability goal 

Test production representative prototypes to demonstrate 

product in operational environment 

Collect statistical process control data 

Demonstrate critical processes capable and in 

statistical control 

Conduct decision review to begin production X X X X 

Sources: GAO and DOD. 

At all three knowledge points, DOD’s policy does not provide all the 

necessary controls used by commercial companies. For example, at 

program launch (milestone B) or when knowledge point 1 should be 

reached, the policy requires decision makers to identify and validate a 

weapon system’s key performance requirements and to have a technical 

solution for the system before program start. This information is then used 

to form cost and schedule estimates for the product’s development. 

However, the policy does not emphasize the use of a disciplined systems 

engineering process for balancing a customer’s needs with resources to 

deliver a preliminary design. The lack of effective controls at knowledge 

point 1 could result in gaps between requirements and resources being 

discovered later in development. 

At the design readiness review or when knowledge point 2 should be 

reached, DOD’s policy does not require specific controls to document that 

a product is ready for initial manufacturing and demonstration. DOD’s 

policy suggests appropriate criteria, such as number of subsystem and 

system design reviews completed, percentage of drawings completed, 

planned corrective actions to hardware and software deficiencies, 

adequate development testing, completed failure modes and effects 
Page 10 GAO-04-53 DOD’s Acquisition Policy 

analysis, identification of key system characteristics and critical 

manufacturing processes, and availability of reliability targets and growth 

plans. However, these criteria are not required. For example, we found 

that a key indicator of a product’s design stability is the completion of 

90 percent of the engineering drawings supported by design reviews. 

DOD’s policy does not require that a certain percentage of drawings or 

design reviews be completed to ensure the design is mature enough to 

enter the system demonstration phase. As a result, a decision maker has 

no benchmark to consider when deciding to advance a program to the 

next level of development. 

Finally, at production commitment or when knowledge point 3 should be 

reached, DOD’s policy does not require specific controls to document that 

a product can be manufactured to meet cost, schedule, and quality targets 

before moving into production. For example, the policy states there 

should be “no significant manufacturing risks” at the start of low-rate 

production but does not define what this means or how it is to be 

measured. DOD’s policy does not require the demonstrated control of 

manufacturing processes and the collection of statistical process control 

data until full-rate production begins but even then fails to specify a 

measurable control. Given that low-rate production can last several years, 

a significant number of products can be manufactured before processes 

are brought under control, creating a higher probability of poor cost and 

schedule outcomes. 

While supporting efforts to build more flexibility into the DOD acquisition 

process and to develop weapon systems using an evolutionary approach, 

Congress asked DOD to be more disciplined in its approach. The Defense 

Authorization Act for Fiscal Year 2003 required DOD to address (1) the 

way it plans to meet certain statutory and regulatory requirements for 

managing its major acquisition programs, (2) needed guidance for 

implementing spiral developments, and (3) technology readiness (at 

acquisition program initiation). DOD was responsive to all three 

requirements. With regard to the second requirement, a description of the 

process that would be used to independently validate that measurable exit 

DOD Is Responsive 

to Congressional 

Requirements about 

New Acquisition 

Process 
Page 11 GAO-04-53 DOD’s Acquisition Policy 

criteria for applying a spiral development process5 have been met was 

unclear. DOD stated that the milestone decision authority6

 provides that 

independent validation as part of DOD’s milestone approval process. 

DOD’s responses to the relevant sections of the act are summarized below. 

More detailed comparisons are provided in appendixes I, II, and III. 

Requirements: This section directed DOD to report on its plan to meet 

certain statutory and regulatory requirements for managing its major 

acquisition programs applying an evolutionary acquisition process. These 

include establishing and approving operational requirements and cost and 

schedule goals for each increment, meeting requirements for operational 

and live fire testing for each increment, and optimizing total system 

performance and minimizing total ownership costs. 

DOD response: In April 2003, DOD submitted its report reflecting how 

these requirements are addressed in its acquisition policy. According to 

the report, the policy addresses the statutory and other requirements 

applicable to all major defense acquisition programs, including each 

increment of evolutionary acquisition programs. For example, the policy 

requires that each program or increment of an evolutionary acquisition 

have a milestone B decision to approve program initiation and to permit 

entry into system development and demonstration. The policy specifies 

the statutory and regulatory information necessary to support the 

decision. 

Requirements: This section authorizes DOD to conduct a research and 

development program for a major defense acquisition program using spiral 

development only if approved by the Secretary of Defense or authorized 

 5

 A spiral development program is defined in section 803 of the Defense Authorization Act 

for Fiscal Year 2003 as a research and development program conducted in discrete phases 

or blocks, each of which will result in the development of fieldable prototypes and will not 

proceed into acquisition until specific performance parameters, including measurable exit 

criteria, have been met. 

6

 The milestone decision authority is typically the DOD component acquisition executive or 

designee, or for certain large programs, the head of the component or the Under Secretary 

of Defense for Acquisition, Technology, and Logistics. 

Section 802: Evolutionary 

Acquisition 

Section 803: Spiral 

Development 
Page 12 GAO-04-53 DOD’s Acquisition Policy 

high-level designee.7 A program cannot be conducted as a spiral 

development unless the Secretary of Defense or designee approves a plan 

that describes such things as the program strategy, test plans, performance 

parameters, and measurable exit criteria. The section also requires the 

Secretary of Defense to issue guidance addressing the appropriate 

processes for an independent validation that exit criteria have been met, 

the operational assessment of fieldable prototypes, and the management 

of these types of programs. It further requires the Secretary to report to 

Congress on the status of each program applying spiral development by 

September 30 of each year from 2003 to 2008. 

DOD response: DOD established a technology development strategy in 

the new policy to address this requirement. The strategy must be 

completed before a program can enter the technology development phase. 

The strategy also documents the cost and schedule goals, the test plans, 

the number of prototypes, and a program strategy for the total research 

and development program. The strategy requires a test plan to ensure the 

goals and exit criteria for the first technology spiral demonstration are 

met, and the policy requires an independent operational assessment for 

the release of each product increment to the user. What is unclear in 

DOD’s guidance is the process that will be used for independently 

validating whether measurable cost, schedule, and performance exit 

criteria have been met. However, DOD stated that the milestone decision 

authority provides independent validation that exit criteria have been met 

as part of DOD’s milestone approval process. As of October 23, 2003, 

DOD’s report on the status of each program applying spiral development 

was still in draft and not yet submitted. DOD’s current draft report states 

that there are no research and development programs that have been 

approved as spiral development programs as of September 30, 2003. 

Section 803 requirements were implemented in DOD Instruction 5000.2, 

which was effective in May 2003. DOD anticipates that there will be 

approved spiral development programs to report in 2004. 

 7

 Section 803 authorized the Secretary of Defense to delegate authority to approve a spiral 

development plan to the Under Secretary of Defense for Acquisition, Technology, and 

Logistics or to the senior acquisition executive of the military department or defense 

agency concerned. The authority may not be delegated further. 
Page 13 GAO-04-53 DOD’s Acquisition Policy 

Requirements: This section added a requirement to section 804 

of the National Defense Authorization Act for Fiscal Year 2002 

(Public Law 107-107) that directed DOD to report by March of each year 

between 2003 and 2006 on the maturity of technology at the initiation of 

major defense acquisition programs. Each report is required to (1) identify 

any major acquisition program that entered system development and 

demonstration during the preceding calendar year with immature key 

technology that was not demonstrated in, at minimum, a relevant 

environment, as required by the new policy; (2) justify the incorporation 

of any key technology on an acquisition program that does not meet that 

requirement; (3) identify any instances that the Deputy Under Secretary 

of Defense for Science and Technology did not concur with the technology 

assessment and explain how the issue has been or will be resolved; 

(4) identify each case in which a decision was made not to conduct an 

independent technology readiness assessment for a critical technology on 

a major defense acquisition; and (5) explain the reasons for the decision 

each year through 2006. 

DOD response: In March 2003, DOD reported that two programs entered 

system development and demonstration in 2002 with critical technologies 

that did not meet demonstration requirements and provided justification 

for them.8

 DOD did not identify or report any cases where an independent 

technology readiness assessment was not conducted or where the 

Under Secretary disagreed with assessment findings. 

 8

 The two programs were the Composite Health Care System II and the Joint Tactical Radio 

System, Cluster 1. 

Section 822: Independent 

Technology Readiness 

Assessments 
Page 14 GAO-04-53 DOD’s Acquisition Policy 

DOD can maximize its $1 trillion investment in new weapons over the next 

6 years by ensuring effective implementation of the new acquisition policy. 

DOD’s leaders have taken noteworthy steps by incorporating into the 

policy a framework that supports a knowledge-based, evolutionary 

acquisition process, similar to one used by leading commercial companies 

to get successful outcomes. A framework is an important and significant 

step. DOD must now turn its attention to establishing controls. As leading 

companies have found, having clearly established controls to capture and 

use appropriate knowledge to make decisions at critical junctures is 

crucial for delivering affordable products as planned. DOD’s policy 

addresses specific congressional requirements and includes some controls 

that leading companies use to capture knowledge at the start of a program. 

However, additional controls are needed to ensure that decisions made 

throughout product development are informed by demonstrated 

knowledge. 

DOD must design and implement necessary controls to ensure that 

appropriate knowledge is captured and used at critical junctures to make 

decisions about moving a program forward and investing more money. 

We recommend that the Secretary of Defense require additional controls 

for capturing knowledge at three key points—program launch, design 

readiness review for transitioning from system integration to system 

demonstration, and production commitment. The additional controls for 

program launch (milestone B) should ensure the capture of knowledge 

about the following: 

• Cost and schedule estimates based on knowledge from a preliminary 

design using systems engineering tools. 

The additional controls for transitioning from system integration to system 

demonstration (design readiness review) should ensure the capture of 

knowledge about the following: 

• Completion of 90 percent of engineering drawings. 

• Completion of subsystem and system design reviews. 

• Agreement from all stakeholders that drawings are complete and the 

design is producible. 

• Completion of failure modes and effects analysis. 

• Identification of key system characteristics. 

• Identification of critical manufacturing processes. 

• Reliability targets and a reliability growth plan based on demonstrated 

reliability rates of components and subsystems. 

Conclusions 

Recommendations for 

Executive Action 
Page 15 GAO-04-53 DOD’s Acquisition Policy 

The additional controls for the production commitment (milestone C) 

should ensure the capture of knowledge about the following: 

• Completion of production representative prototypes. 

• Availability of production representative prototypes to achieve reliability 

goal and demonstrate the product in an operational environment. 

• Collection of statistical process control data. 

• Demonstration that critical manufacturing processes are capable and in 

statistical control. 

Because knowledge about technology, design, and manufacturing at 

critical junctures can lower DOD’s investment risk, decisions that do not 

satisfy knowledge-based criteria should be visible and justified. Therefore, 

we also recommend that the Secretary of Defense document the rationale 

for any decision to move a program to the next stage of development 

without meeting the knowledge-based criteria, including those listed in the 

first recommendation. The responsible milestone decision authority 

should justify the decision in the program’s acquisition decision 

memorandum and in a report to Congress. 

DOD provided us with written comments on a draft of this report. The 

comments appear in appendix IV. 

DOD partially concurred with our recommendation that the Secretary 

require additional controls for capturing knowledge at three key points: 

program launch, design readiness review for transitioning from system 

integration to system demonstration, and production. DOD stated that it 

agrees in principle with the advantages of using knowledge-based controls 

at key points in the acquisition process to assess risk and ensure readiness 

to proceed into the next phase of the acquisition process. DOD believes 

the current acquisition framework includes the controls necessary to 

achieve effective results, but it will continue to monitor the process to 

determine whether others are necessary to achieve the best possible 

outcomes. 

While we believe DOD’s effort to establish a solid framework for 

evolutionary acquisitions is a giant step forward, our work has shown that 

a disciplined application of controls in the process is needed to implement 

the framework if better acquisition outcomes are to be achieved. DOD’s 

policy does not include all the necessary controls to ensure a high level of 

product knowledge is attained and used for making decisions to move a 

Agency Comments 

and Our Evaluation 
Page 16 GAO-04-53 DOD’s Acquisition Policy 

program forward in the product development process. Leading product 

developers use additional controls, as listed in our first recommendation, 

to achieve the knowledge necessary to reduce risk to reasonable levels at 

critical junctures before making additional significant investments in 

product development. Simply monitoring the process may not be enough 

for DOD to achieve the best outcomes. Therefore, we are retaining our 

recommendation that the Secretary require additional controls at three 

critical points in the acquisition process. 

DOD also partially concurred with our recommendation that the Secretary 

document in each program’s acquisition decision memorandum and in a 

report to Congress the rationale for any decision to move a program to the 

next stage of development without meeting the knowledge-based criteria, 

including those described in the first recommendation. DOD agreed that it 

should record and be accountable for program decisions. Decision makers 

will continue to use the acquisition decision memorandum to document 

program decisions and the rationale for them. DOD did not concur with 

the need for a report outside the department. Because we believe strongly 

that knowledge-based criteria used to gauge a product’s development 

progress at critical junctures can lower DOD’s investment risks, we think 

it is important that decisions made without satisfying knowledge-based 

criteria be justified in a visible and transparent way to hold managers 

accountable for moving a program forward absent this knowledge. 

Therefore, we are retaining our recommendation for reporting the basis 

for decisions to move forward in a report to Congress. 

We reviewed DOD’s revised and past acquisition policies, DOD Directive 

5000.1, DOD Instruction 5000.2, and DOD 5000.2-R,9

 which provide 

management principles and mandatory policies and procedures for 

managing acquisitions programs. We contacted an official in the Office of 

the Under Secretary of Defense for Acquisition, Technology, and Logistics 

who is responsible for the development of the policy to better understand 

its content. We also reviewed information from the Defense Acquisition 

University that provided educational material on the policies. 

We reviewed the relevant sections of the Bob Stump National Defense 

Authorization Act for Fiscal Year 2003 and the accompanying Senate 

 9

 DOD 5000.2-R was canceled and replaced with the Interim Defense Acquisition 

Guidebook. 

Scope and 

Methodology 
Page 17 GAO-04-53 DOD’s Acquisition Policy 

Armed Services Committee report to identify the requirements applicable 

to DOD’s acquisition policy. We compared these requirements with DOD’s 

responses to determine whether they have been addressed. 

Finally, we used information from more than 10 GAO products that 

examine how commercial best practices can improve outcomes for 

various DOD programs. During the past 6 years, we have gathered 

information based on discussions and visits with the following companies: 

• 3M • Boeing Commercial Airplane Group 

• Chrysler • Bombardier Aerospace 

• Ford Motor • Hughes Space and Communication 

• Motorola • Xerox 

• Hewlett-Packard • Caterpillar 

• Cummins • General Electric Aircraft Engines 

• Toyota • Harris Semiconductor 

• Honda • Texas Instruments 

• John Deere • Varian Oncology Systems 

• Ethicon-Endo Surgery 

(division of Johnson & 

Johnson) 

Although the approaches varied, these companies consistently applied the 

basic processes and standards in use. We compared this information with 

the acquisition framework and controls established by DOD’s policy. We 

concentrated on whether the policy provides a framework for a 

knowledge-based, evolutionary process and the controls necessary to 

carry out this intent. 

We conducted our review from April 2003 to September 2003 in 

accordance with generally accepted government auditing standards. 

We are sending copies of this report to the Secretary of Defense; the 

Secretaries of the Air Force, Army, and Navy; and the Director of the 

Office of Management and Budget. We will also provide copies to others 

on request. In addition, the report will be available at no charge on the 

GAO Web site at http://www.gao.gov. 
Page 18 GAO-04-53 DOD’s Acquisition Policy 

Please contact me at (202) 512-4841 if you have any questions 

concerning this report. Other key contributors to this report were 

Lily Chin, Chris DePerro, Matt Lea, Mike Sullivan, and Adam Vodraska. 

Katherine V. Schinasi 

Director, Acquisition and Sourcing Management 
Appendix I: Section 802, Evolutionary 

Acquisition 

Page 19 GAO-04-53 DOD’s Acquisition Policy 

Section 802 of the Defense Authorization Act for Fiscal Year 2003 required 

the Secretary of Defense to submit a report to Congress explaining how 

the Department of Defense (DOD) plans to meet certain statutory and 

regulatory requirements for acquisition programs following an 

evolutionary approach. In April 2003, the Secretary reported how these 

requirements were addressed in DOD’s policy (such as in tables of 

statutory and regulatory information requirements contained in enclosure 

3 of Instruction 5000.2). According to the report, DOD’s policy requires 

that each program—including an increment of an evolutionary 

acquisition—have a milestone B decision to approve program initiation 

and to permit entry into systems development and demonstration. DOD’s 

policy specifies the statutory and regulatory information necessary to 

support the decision. We examined the policy to ensure the statutes and 

regulations identified in section 802 were addressed. Table 3 provides a 

list of the statutory and regulatory requirements identified in section 802, a 

corresponding document and page number where the requirement appears 

in DOD’s policy, and a description of the requirement from the policy. 

Table 3: How DOD Policy Addresses Section 802’s Statutory and Regulatory Requirements 

Section 802 requirements DOD policy reference examples Brief description from policy 

Requirements of chapter 144 of title 10, 

United States Code 

 

Sec. 2430, major defense acquisition 

program defined 

Department of Defense Instruction 

(DODI) 5000.2, enclosure 2, p. 16 

Table E.2.T1 provides a description (criteria) and the 

decision authority by acquisition program category. 

Sec. 2431, weapons development 

and procurement schedules 

Not referenceda 

Sec. 2432, Selected Acquisition 

Reports 

DODI 5000.2, enclosure 3, p.19 Required at milestone B or program initiation and 

annually thereafter; end of quarter following 

milestone C decision; full-rate production decision. 

Also required when there is a baseline breach. 

Sec. 2433, unit cost reports DODI 5000.2, enclosure 3, p. 19 Required on quarterly basis. 

Sec. 2434, independent cost 

estimates; operational manpower 

requirements 

DODI 5000.2, enclosure 6, p. 30 

DODI 5000.2, enclosure 3, p. 19 

Required at program initiation for ships (cost 

assessment only). Required at milestones B and C 

and full-rate production decision. 

Sec. 2435, baseline description DODI 5000.2, enclosure 3, p. 19 

DODI 5000.2, p. 8 

Required at program initiation for ships. Required at 

milestones B and C and full-rate production decision. 

Program deviation report required immediately upon 

program deviation. 

Sec 2440, technology and industrial 

base plans 

DODI 5000.2, enclosure 3, p. 19 Required at milestones B and C (part of acquisition 

strategy). 

Appendix I: Section 802, Evolutionary 

Acquisition 
Appendix I: Section 802, Evolutionary 

Acquisition 

Page 20 GAO-04-53 DOD’s Acquisition Policy 

Section 802 requirements DOD policy reference examples Brief description from policy 

Sec. 139, Director of Operational 

Test and Evaluation 

Department of Defense Directive 

(DODD) 5000.1, p. 3 

DODI 5000.2, enclosure 3 

DODI 5000.2, enclosure 5, pp. 27-29

Cites Director of Operational Test and Evaluation as 

key official of defense acquisition system. Director 

responsible for assessing adequacy of operational 

tests and live fire tests and evaluating operational 

effectiveness, suitability, and survivability of systems. 

Sec. 181, Joint Requirements 

Oversight Council 

DODD 5000.1, p. 3 

DODI 5000.2, p. 4 

Chairman of the Joint Chiefs of Staff, with assistance 

of Joint Requirements Oversight Council, responsible 

for assessing and providing advice regarding 

capability needs for defense acquisition programs. 

Chairman also responsible for validating and 

approving capabilities documents. 

Sec. 2366, major systems and 

munitions programs: survivability 

testing and lethality testing required 

before full-scale production 

DODI 5000.2, enclosure 3, p. 19 

DODI 5000.2, enclosure 5, p. 29 

DODI 5000.2, p. 10 

Live fire testing and reporting required for all covered 

systems. Strategy or live fire waiver and alternate 

plan required at milestone B. Live fire test and 

evaluation report required for full-rate production 

decision. 

Sec. 2399, operational test and 

evaluation of defense acquisition 

programs 

DODI 5000.2, enclosure 3, p. 18 

DODI 5000.2, pp. 9-10 

DODI 5000.2, enclosure 5, pp. 27-28

Director of Operational Test and Evaluation shall 

determine operational effectiveness and suitability of 

system under realistic conditions. Beyond Low Rate 

Initial Production Report required at full-rate 

production decision. DOD may not conduct 

operational testing until Director of Operational Test 

and Evaluation approves test plan. 

Sec. 2400, low-rate initial production 

of new systems 

DODI 5000.2, enclosure 3, p. 19 

DODI 5000.2, pp. 9-10 

Low-rate initial production quantities will be 

determined by milestone B. 

DODD 5000.1 DODD 5000.1, pp. 1, 2 Policies in directive apply to all acquisition programs. 

Evolutionary acquisition strategies are preferred 

approach to satisfying operational needs. 

DODI 5000.2 DODI 5000.2, p. 1 Instruction applies to all defense technology projects 

and acquisition projects. 

Chairman of the Joint Chiefs of Staff 

Instruction 3170.01Bb

DODI 5000.2, p. 4 Chairman of Joint Chiefs of Staff provides advice 

through Instruction 3170.01. Capability documents 

required at concept decision and at 

milestones A, B, and C. 

Other provisions of law and regulations 

(including successor documents) that 

are applicable to such programs 

DODI 5000.2, enclosure 3, pp. 18-22 Several other statutory, regulatory, and contract 

requirements are addressed in the policy. 

Sources: Defense Authorization Act for Fiscal Year 2003, DOD, and GAO. 

a

Although the statutory requirement of 10 U.S.C., section 2431 is not specifically cited in the policy, 

the same summarized information is submitted to Congress in the Selected Acquisition Reports for 

the first quarter of a fiscal year as required by 10 U.S.C., section 2432. The requirement for Selected 

Acquisition Reports is addressed by DOD’s acquisition policy. The statute concerning weapons 

development and procurement schedules requires the Secretary of Defense to submit budget 

justification documents regarding this information for each weapon system for which fund 

authorization is required (and for which procurement funds are requested in the budget) not later than 

45 days after the President submits the budget to Congress. 

b

Chairman of the Joint Chiefs of Staff Instruction 3170.01B was revised and reissued June 24, 2003, 

as Joint Chiefs of Staff Instruction 3170.01C with an accompanying manual. 
Appendix I: Section 802, Evolutionary 

Acquisition 

Page 21 GAO-04-53 DOD’s Acquisition Policy 

Section 802 also required DOD to report on its plans for addressing certain 

acquisition process issues regarding each increment of an evolutionary 

process. DOD reported on how it plans to establish and approve 

operational requirements and cost and schedule goals; meet requirements 

for operational and live fire testing; monitor cost and schedule 

performance; achieve interoperability; and consider total system 

performance and total ownership costs. We compared DOD’s response 

with section 802’s reporting requirements. As shown in table 4, DOD was 

responsive to the 802 requirements. 

Table 4: How DOD Responded to Section 802’s Requirements Regarding Specific Matters for Each Increment of an 

Evolutionary Acquisition Process 

Section 802 requirements DOD response Policy reference 

“The manner in which the Secretary plans 

to establish and approve, for each 

increment of an evolutionary acquisition 

process–” 

 

“operational requirements; and” “Operational Requirements: Each program 

is required to have documented, approved 

operational requirements in accordance 

with authorized Joint Staff procedures. For 

evolutionary acquisition programs, the 

requirements documents are typically 

time-phased and specify the capability 

expected of each increment.” 

Capability development document, 

including key performance parameters, 

required at milestone B or program 

initiation (DODI 5000.2, p. 7, and enclosure 

3, p. 20). 

“cost and schedule goals.” “Cost and Schedule Goals: At program 

initiation, each program and program 

increment is required to have an 

Acquisition Program Baseline approved by 

the Milestone Decision Authority (MDA). 

The Acquisition Program Baseline includes 

cost and schedule goals.” 

Acquisition program baseline required at 

milestone B or program initiation (DODI 

5000.2, p. 8, and enclosure 3, p. 19). 
Appendix I: Section 802, Evolutionary 

Acquisition 

Page 22 GAO-04-53 DOD’s Acquisition Policy 

Section 802 requirements DOD response Policy reference 

“The manner in which the Secretary plans, 

for each increment of an evolutionary 

acquisition process–” 

 

“to meet requirements for operational 

testing and live fire testing;” 

“At program initiation, each program or 

program increment must have a Test and 

Evaluation Master Plan (TEMP) approved 

by the Director, Operational Test and 

Evaluation. The TEMP includes 

requirements for operational and live fire 

testing.” 

Test and Evaluation Master Plan required 

at milestone B or program initiation (DODI 

5000.2, enclosure 3, p. 21, and enclosure 

5, p. 25). 

“to monitor cost and schedule 

performance; and” 

“Progress against cost and schedule goals 

[for each increment] is addressed via 

automated reporting systems at both the 

Office of Secretary of Defense staff level 

and at the Service staff level, and by 

reviews conducted in the context of the 

acquisition oversight model.” 

DOD’s policy contains an enclosure on 

resource estimation (DODI 5000.2, 

enclosure 6, pp. 30-31). Also, several 

reporting requirements address monitoring 

cost and schedule performance such as 

acquisition program baselines and selected 

acquisition reports (DODI 5000.2, 

enclosure 3, pp. 18-21). 

“to comply with laws requiring reports to 

Congress on results testing and on cost 

and schedule performance.” 

“DoD acquisition policy requires 

independent assessment by the operational 

test authority prior to release of each 

successive increment to the user. All 

reports by statute will be submitted.” 

The service shall perform an independent 

operational assessment prior to release of 

each successive increment to the user 

(DODI 5000.2, enclosure 5, p. 29). Also, 

several statutory reports are listed in the 

policy addressing testing and cost and 

schedule performance (DODI 5000.2, 

enclosure 3, pp. 18-21). 
Appendix I: Section 802, Evolutionary 

Acquisition 

Page 23 GAO-04-53 DOD’s Acquisition Policy 

Section 802 requirements DOD response Policy reference 

“The manner in which the Secretary plans 

to ensure that each increment of an 

evolutionary acquisition process is 

designed–” 

 

“to achieve interoperability within and 

among United States forces and 

United States coalition partners; and” 

“Each increment of an evolutionary 

acquisition program is required to have an 

Acquisition Program Baseline, approved by 

the MDA, which includes an Interoperability 

key performance parameter.” 

Acquisition program baseline required at 

milestone B or program initiation (DODI 

5000.2, p. 8, and enclosure 3, p. 19). 

“to optimize total system performance 

and minimize total ownership costs by 

giving appropriate consideration to–” 

• “logistics planning;” 

• “manpower, personnel, and training;” 

• “human, environmental, safety, 

occupational health, accessibility, 

survivability, operational continuity 

and security factors;” 

• “protection of critical program 

information; and” 

• “spectrum management” 

“Each increment of an evolutionary 

acquisition program is required to have an 

acquisition strategy, approved by the MDA, 

that addresses [where applicable]: logistics 

planning; manpower, personnel and 

training; human, environmental, safety, 

occupational health; accessibility (human), 

survivability, operational continuity (as 

required by requirements document), 

security factors; critical program 

information; and spectrum management. 

These factors are taken into account as the 

Department of Defense considers total 

system performance and total ownership 

costs.” 

Acquisition strategy required at milestone B 

or program initiation (DODI 5000.2, p. 7, 

and enclosure 3, p. 20). 

Sources: Defense Authorization Act for Fiscal Year 2003, DOD, and GAO. 
Appendix II: Section 803, Spiral Development 

Page 24 GAO-04-53 DOD’s Acquisition Policy 

Section 803 of the Defense Authorization Act for Fiscal Year 2003 

authorized the Secretary of Defense to conduct major defense acquisition 

programs as spiral development programs. However, the section placed a 

limitation on these programs. It stated that a research and development 

program for a major acquisition may not be conducted as a spiral 

development program unless the Secretary of Defense or authorized 

high-level designee gives approval. The section requires the Secretary of 

Defense to issue guidance for the implementation of such programs to 

address appropriate processes for ensuring the independent validation of 

exit criteria being met, the operational assessment of fieldable prototypes, 

and the management of these types of programs. 

DOD responded to these requirements principally by incorporating into 

the acquisition policy the requirement for a technology development 

strategy. This strategy is a prerequisite for a project to enter the 

technology development phase of the acquisition process, or milestone A. 

Table 5 compares the spiral development plan requirements in the act with 

the technology development strategy requirements in DOD’s May 2003 

acquisition policy. 

Table 5: How DOD Policy Addresses Section 803’s Requirements 

Section 803 requirements DOD policy 

“A spiral development plan for research and development 

program for a major defense acquisition program shall, at a 

minimum, include the following matters:” 

Section 3.5.4 of DOD Instruction 5000.2 states that the technology 

development strategy document the following: 

“A rationale for dividing the research and development program 

into separate spirals, together with a preliminary identification of 

the spirals to be included.” 

The rationale for adopting an evolutionary strategy. For an 

evolutionary acquisition, either spiral or incremental, the 

strategy shall include a preliminary description of how the 

program will be divided into technology spirals and 

development increments. 

“A program strategy, including overall cost, schedule and 

performance goals for the total research and development.” 

A program strategy, including overall cost, schedule, and 

performance goals for the total research and development 

program. 

“Specific cost, schedule, and performance parameters, 

including measurable exit criteria, for the first spiral to be 

conducted.” 

Specific cost, schedule, and performance goals, including exit 

criteria, for the first technology spiral demonstration. 

“A testing plan to ensure that performance goals, parameters, 

and exit criteria are met.” 

A test plan to ensure that the goals and exit criteria for the first 

technology spiral demonstration are met. 

“An appropriate limitation on the number of prototype units that 

may be produced under the research and development 

program.” 

An appropriate limitation on the number of prototypes units that 

may be produced and deployed during technology 

development. 

Appendix II: Section 803, Spiral Development 
Appendix II: Section 803, Spiral Development 

Page 25 GAO-04-53 DOD’s Acquisition Policy 

Section 803 requirements DOD policy 

“Specific performance parameters, including measurable exit 

criteria, that must be met before the major defense acquisition 

program proceeds into production units in excess of the 

limitation on the number of prototype units.” 

Specific performance goals and exit criteria that must be met 

before exceeding the number of prototypes that may be 

produced under the research and development program. 

Sources: Defense Authorization Act for Fiscal Year 2003, DOD, and GAO. 

As shown in the table, DOD’s policy generally responded to the 

requirements in the act concerning guidance for implementation of spiral 

development programs. While the policy includes a technology 

development strategy that requires a test plan to ensure the goals and exit 

criteria for the first technology spiral demonstration are met and an 

independent operational assessment for the release of each product 

increment to the user, it is unclear what the process is for independently 

validating that cost, schedule, and performance exit criteria have been. 

However, DOD stated that the milestone decision authority provides 

independent validation that exit criteria have been met as part of DOD’s 

milestone approval process. Section 803 also requires that a spiral 

development plan include “[s]pecific cost, schedule, and performance 

parameters, including measurable exit criteria, for the first spiral to be 

conducted.” DOD’s policy substituted “parameters” for “goals” and did not 

use the term “measurable” in describing the required exit criteria. 

Finally, section 803 requires the Secretary of Defense to submit to 

Congress by September 30 yearly from 2003 through 2008 a status report 

on each spiral development program. The report is to include information 

on unit costs for the projected prototypes. As of October 23, 2003, DOD’s 

report on the status of each program applying spiral development was still 

in draft and not yet submitted. DOD’s current draft report states that there 

are no research and development programs that have been approved as 

spiral development programs as of September 30, 2003. Section 803 

requirements were implemented in DOD Instruction 5000.2, which was 

effective in May 2003. DOD anticipates that there will be approved spiral 

development programs to report in 2004. 
Appendix III: Section 804, Technology 

Maturity, and Section 822, Independent 

Technology Readiness Assessments 

Page 26 GAO-04-53 DOD’s Acquisition Policy 

Section 804 of the Defense Authorization Act for Fiscal Year 2002 required 

DOD to report on the maturity of technology at the initiation of major 

defense acquisition programs. The act directed DOD to report by March 1 

of each year between 2003 and 2006 on a requirement in DOD’s policy that 

technology must have been demonstrated in a relevant environment (or, 

preferably, in an operational environment) to be considered mature 

enough to use for product development in systems integration. Each 

report is required to (1) identify any major acquisition program that 

entered system development and demonstration during the preceding 

calendar year with immature key technology that was not demonstrated 

in, at minimum, a relevant environment, as required by the new policy; 

(2) justify the incorporation of any key technology on an acquisition 

program that does not meet that requirement; (3) and identify any 

instances that the Deputy Under Secretary of Defense for Science and 

Technology did not concur and explain how the issue has been or will be 

resolved, including information on the use of independent readiness 

assessments. Section 822 of the Defense Authorization Act for Fiscal Year 

2003 amended section 804 by adding a requirement that the Secretary of 

Defense identify each case in which an authoritative decision has been 

made within DOD not to conduct an independent technology readiness 

assessment for a critical technology on a major defense acquisition 

program and explain the reasons for the decision. On March 18, 2003, DOD 

submitted its first report. Table 6 shows the specific requirements for the 

report and DOD’s response. 

Table 6: How DOD Responded to Section 804 and Section 822 Requirements 

Reporting requirements DOD’s response 

Section 804: “identify each case in which a major defense 

acquisition program entered system development and 

demonstration during the preceding calendar year and into which 

key technology has been incorporated that does not meet the 

technological maturity requirement ... and provide a justification 

for why such key technology was incorporated.”a

DOD reported two programs (Joint Tactical Radio System Cluster 

1 and Composite Health Care System II) that entered system 

development and demonstration with key technologies that did not 

meet the technological maturity requirement. While DOD did not 

specifically identify all the technologies for these programs that did 

not meet the requirements, it did provide its justification for why 

the technologies were incorporated. 

Section 804: “identify any determination of technological maturity 

with which the Deputy Under Secretary of Defense for Science 

and Technology did not concur and explain how the issue has 

been or will be resolved.” 

DOD reported that, in all cases, the Deputy Under Secretary of 

Defense for Science and Technology concurred with the 

technology readiness assessment of the program manager and 

the milestone decision authority. 

Appendix III: Section 804, Technology 

Maturity, and Section 822, Independent 

Technology Readiness Assessments 
Appendix III: Section 804, Technology 

Maturity, and Section 822, Independent 

Technology Readiness Assessments 

Page 27 GAO-04-53 DOD’s Acquisition Policy 

Reporting requirements DOD’s response 

Section 822: “identify each case in which an authoritative decision 

has been made within the Department of Defense not to conduct 

an independent technology readiness assessment for a critical 

technology on a major defense acquisition program and explain 

the reasons for the decision.” 

DOD did not report on or identify any cases. 

Sources: National Defense Authorization Acts for Fiscal Years 2002 and 2003, DOD, and GAO. 

a

Technology must have been demonstrated in a relevant environment (or, preferably in an operational 

environment) to be considered mature enough to use for product development in systems integration. 
Appendix IV: Comments from the Department of Defense 

Page 28 GAO-04-53 DOD’s Acquisition Policy 

Appendix IV: Comments from the 

Department of Defense 
Appendix IV: Comments from the Department of Defense 

Page 29 GAO-04-53 DOD’s Acquisition Policy 

Now on pp. 14 and 15. 
Appendix IV: Comments from the Department of Defense 

Page 30 GAO-04-53 DOD’s Acquisition Policy 

Now on p. 15. 
Related GAO Products 

Page 31 GAO-04-53 DOD’s Acquisition Policy 

Best Practices: Setting Requirements Differently Could Reduce 

Weapon Systems’ Total Ownership Costs. GAO-03-57. Washington, D.C.: 

February 11, 2003. 

Best Practices: Capturing Design and Manufacturing Knowledge 

Early Improves Acquisition Outcomes. GAO-02-701. Washington, D.C.: 

July 15, 2002. 

Defense Acquisitions: DOD Faces Challenges in Implementing Best 

Practices. GAO-02-469T. Washington, D.C.: February 27, 2002. 

Best Practices: Better Matching of Needs and Resources Will Lead 

to Better Weapon System Outcomes. GAO-01-288. Washington, D.C.: 

March 8, 2001. 

Best Practices: A More Constructive Test Approach Is Key to 

Better Weapon System Outcomes. GAO/NSIAD-00-199. Washington, D.C.: 

July 31, 2000. 

Defense Acquisition: Employing Best Practices Can Shape Better 

Weapon System Decisions. GAO/T-NSIAD-00-137. Washington, D.C.: 

April 26, 2000. 

Best Practices: DOD Training Can Do More to Help Weapon 

System Program Implement Best Practices. GAO/NSIAD-99-206. 

Washington, D.C.: August 16, 1999. 

Best Practices: Better Management of Technology Development 

Can Improve Weapon System Outcomes. GAO/NSIAD-99-162. 

Washington, D.C.: July 30, 1999. 

Defense Acquisitions: Best Commercial Practices Can Improve Program 

Outcomes. GAO/T-NSIAD-99-116. Washington, D.C.: March 17, 1999. 

Defense Acquisition: Improved Program Outcomes Are Possible. 

GAO/T-NSIAD-98-123. Washington, D.C.: March 17, 1998. 

Best Practices: DOD Can Help Suppliers Contribute More to Weapon 

System Programs. GAO/NSIAD-98-87. Washington, D.C.: March 17, 1998. 

Related GAO Products 
Related GAO Products 

Page 32 GAO-04-53 DOD’s Acquisition Policy 

Best Practices: Successful Application to Weapon Acquisition Requires 

Changes in DOD’s Environment. GAO/NSIAD-98-56. Washington, D.C.: 

February 24, 1998. 

Best Practices: Commercial Quality Assurance Practices Offer 

Improvements for DOD. GAO/NSIAD-96-162. Washington, D.C.: 

August 26, 1996. 

(120214) 
The General Accounting Office, the audit, evaluation and investigative arm of 

Congress, exists to support Congress in meeting its constitutional responsibilities 

and to help improve the performance and accountability of the federal 

government for the American people. GAO examines the use of public funds; 

evaluates federal programs and policies; and provides analyses, 

recommendations, and other assistance to help Congress make informed 

oversight, policy, and funding decisions. GAO’s commitment to good government 

is reflected in its core values of accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost is 

through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and fulltext files of current reports and testimony and an expanding archive of older 

products. The Web site features a search engine to help you locate documents 

using key words and phrases. You can print these documents in their entirety, 

including charts and other graphics. 

Each day, GAO issues a list of newly released reports, testimony, and 

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site 

daily. The list contains links to the full-text document files. To have GAO e-mail 

this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail 

alerts” under the “Order GAO Products” heading. 

The first copy of each printed report is free. Additional copies are $2 each. A 

check or money order should be made out to the Superintendent of Documents. 

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a 

single address are discounted 25 percent. Orders should be sent to: 

U.S. General Accounting Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm 

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 

U.S. General Accounting Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

GAO’s Mission 

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Mail or Phone 

To Report Fraud, 

Waste, and Abuse in 

Federal Programs 

Public Affairs