WEAPONS 

ACQUISITION 

REFORM

Reform Act Is Helping 

DOD Acquisition 

Programs Reduce 

Risk, but 

Implementation 

Challenges Remain

Report to the Committee on Armed 

Services, U.S. Senate

December 2012

GAO-13-103

United States Government Accountability Office

GAO
United States Government Accountability Office

Highlights of GAO-13-103, a report to 

Committee on Armed Services, U.S. Senate

December 2012

WEAPONS ACQUISITION REFORM

Reform Act Is Helping DOD Acquisition Programs 

Reduce Risk, but Implementation Challenges Remain

Why GAO Did This Study

For the past 3 years, DOD has been 

implementing the Reform Act 

requirements which are aimed at 

helping weapon acquisition programs 

establish a solid foundation from the 

start. This helps to prevent cost 

growth, thus helping the Defense dollar 

go further. This is the third in a series 

of GAO reports on the Reform Act.

GAO was asked to determine 

(1) DOD’s progress in implementing 

Reform Act provisions; (2) the impact 

the Reform Act has had on specific 

acquisition programs; and 

(3) challenges remaining. To do this, 

GAO analyzed documents and 

interviewed officials from the four OSD 

offices created as a result of the 

Reform Act, other DOD offices, the 

military services, and 11 weapon 

acquisition programs we chose as 

case studies. Case study programs 

were selected based on their 

development status and interaction 

with the four OSD offices. Results 

cannot be generalized to all DOD 

weapon acquisition programs.

What GAO Recommends

GAO recommends DOD develop 

additional cost estimating and Reform 

Act implementation guidance; make 

lessons learned available to the 

acquisition community; and assess the 

adequacy of the military services’ 

systems engineering and 

developmental testing workforce. DOD 

generally concurred with the 

recommendations. GAO clarified one 

recommendation to make it clear that 

DOD needs to designate an office(s) 

within the Acquisition, Technology and 

Logistics organization to provide 

practical Reform Act implementation 

guidance to program offices.

What GAO Found

The Department of Defense (DOD) has taken steps to implement fundamental 

Weapon Systems Acquisition Reform Act of 2009 (Reform Act) provisions, 

including those for approving acquisition strategies and better monitoring 

weapon acquisition programs. DOD is also continuing to take additional steps to 

strengthen policies and capabilities. Some provisions, such as issuing guidance 

for estimating operating and support costs, are being implemented. 

GAO’s analysis of 11 weapon acquisition programs showed the Reform Act has 

reinforced early attention to requirements, cost and schedule estimates, testing, 

and reliability. For example, prior to starting development, an independent review 

team raised concerns about the Ground Combat Vehicle program’s many 

requirements and the risks associated with its 7-year schedule. Subsequently, 

the Army reduced the number of requirements by about 25 percent and 

prioritized them, giving contractors more flexibility in designing solutions. In 

addition, the developmental test and evaluation office—resulting from the Reform 

Act—used test results to help the Joint Light Tactical Vehicle program develop a 

more realistic reliability goal and a better approach to reach it. Shown below are 

areas where the Reform Act influenced several programs in GAO’s review.

Reform Act Influence on Select Case Study Programs

Program Requirements Cost and schedule Testing Reliability

Ohio Class Replacement    

Ground Combat Vehicle    

Joint Light Tactical Vehicle    

Ship to Shore Connector  

KC-46 Tanker  

Source: GAO analysis of DOD data.

While DOD has taken steps to implement most of the fundamental Reform Act 

provisions, some key efforts to date have been primarily focused on DOD’s 

largest weapon acquisition programs. DOD faces five challenges—organizational 

capability constraints, the need for additional guidance on cost estimating and 

Reform Act implementation, the uncertainty about the sufficiency of systems 

engineering and developmental testing resources, limited dissemination of 

lessons learned, and cultural barriers between the Office of the Secretary of 

Defense (OSD) and the military services—that limit its ability to broaden the 

Reform Act’s influence to more programs. Service officials believe additional 

guidance is needed to improve their cost estimates and other implementation 

efforts. They also believe that lessons learned from programs that experience 

significant cost and schedule increases should be shared more broadly within the 

acquisition community. These challenges seem straightforward to address, but 

they may require more resources, which have been difficult to obtain. Ensuring 

the services have key leaders and staff dedicated to systems engineering and 

developmental testing activities, such as chief engineers at the service level and 

technical leads on programs, as well as breaking down cultural barriers are more 

difficult to address. They will require continued monitoring and attention by the 

Under Secretary for Acquisition, Technology and Logistics, service acquisition 

executives, and offices established as a result of the Reform Act to address.

View GAO-13-103. For more information, 

contact Michael J. Sullivan at (202) 512-4841

or sullivanm@gao.gov.
Page i GAO-13-103 Weapons Acquisition Reform

Letter 1 

Background 4

DOD Has Implemented Most of the Fundamental Reform Act 

Provisions and Is Strengthening Acquisition Activities 8

The Reform Act Is Helping Programs Identify and Mitigate Risks 

Earlier in the Acquisition Process 14

Challenges Exist That Could Limit the Reform Act’s Ability to 

Influence Systemic Change 22

Conclusions 28

Recommendations for Executive Action 30

Agency Comments and Our Evaluation 30

Appendix I Objectives, Scope, and Methodology 33

Appendix II Progress of Reform Act Offices in Implementing Weapon 

System Acquisition Reform Act Provisions 36

Appendix III Comments from the Department of Defense 38

Tables 

Table 1: Programs Selected for Case Study Review 3

Table 2: Key Responsibilities of Offices Established as a Result of 

the Reform Act 5

Table 3: Reform Act Influence on Case Study Programs 15

Table 4: List of Programs Selected for Case Study Review 34

Table 5: Implementation of Select Reform Act Provisions- Systems 

Engineering 36

Table 6: Implementation of Select Reform Act ProvisionsDevelopmental Test and Evaluation 36

Table 7: Implementation of Select Reform Act Provisions- Cost 

Assessment and Program Evaluation 37

Table 8: Implementation of Select Reform Act ProvisionsPerformance Assessments and Root Cause Analyses 37

Contents 
Page ii GAO-13-103 Weapons Acquisition Reform

Figures

Figure 1: DOD’s Progress in Implementing Four Fundamental 

Reform Act Provisions 10

Figure 2: Revised Review Process for Major Defense Acquisition 

Programs 12

Abbreviations

CAPE Cost Assessment and Program Evaluation

DOD Department of Defense

DT&E Developmental Test and Evaluation

HMS Handheld, Manpack, and Small Form Fit

OSD Office of the Secretary of Defense

PARCA Performance Assessments and Root Cause Analyses

SE Systems Engineering

This is a work of the U.S. government and is not subject to copyright protection in the 

United States. The published product may be reproduced and distributed in its entirety 

without further permission from GAO. However, because this work may contain 

copyrighted images or other material, permission from the copyright holder may be 

necessary if you wish to reproduce this material separately.
Page 1 GAO-13-103 Weapons Acquisition Reform

United States Government Accountability Office

Washington, DC 20548

December 14, 2012

The Honorable Carl Levin

Chairman

The Honorable John McCain

Ranking Member

Committee on Armed Services

United States Senate

For several decades, GAO has reported on poor outcomes 

encompassing cost and schedule growth on the Department of Defense’s 

(DOD) major weapon acquisition programs. Many problems can be traced 

to a culture where the military services begin programs with inflexible 

requirements, immature technologies, and overly optimistic cost and 

schedule estimates. Given pressures to reduce spending across the 

government, including DOD, finding ways to prevent or mitigate cost 

growth is crucial to U.S. national security. A solid program foundation 

using good developmental testing and systems engineering, and reliable 

cost estimates is needed in order to help avoid cost overruns and 

promote better acquisition outcomes. There have been numerous 

attempts in the past to improve DOD acquisition outcomes, including the 

Packard Commission and the Goldwater-Nichols Act in the 1980s and 

Federal Acquisition Streamlining Act of 1994.1 More recently, Congress 

passed the Weapon Systems Acquisition Reform Act of 2009 (Reform 

Act)2 to improve the way weapon systems are acquired and avoid cost 

and schedule overruns.

In 2009, the Senate Armed Services Committee asked us to begin 

reporting on DOD’s implementation of Reform Act provisions and the 

impact the Reform Act has had on weapon acquisition programs. This is 

our third report addressing these topics. The first report focused on 

DOD’s initial efforts to implement Reform Act provisions for systems 

engineering and developmental testing, including the placement of new 

 

1 Final Report to the President by the President’s Blue Ribbon Commission on Defense 

Management (June 1986); Goldwater-Nichols Department of Defense Reorganization Act 

of 1986, Pub. L. No. 99-433; and Pub. L. No. 103-355 (FASA).

2 Pub. L. No. 111-23, as amended by the Ike Skelton National Defense Authorization Act 

for Fiscal Year 2011, Pub. L. No. 111-383 §§ 813 and 1075, and the National Defense 

Authorization Act for Fiscal Year 2012, Pub. L. No. 112-81 §§ 819 and 837.

 
 

Page 2 GAO-13-103 Weapons Acquisition Reform

offices for these activities within the Office of the Secretary of Defense 

(OSD).3 Our second report examined the challenges the services face as 

they try to strengthen systems engineering and developmental testing 

activities on weapon acquisition programs.4 This report examines 

(1) DOD’s progress in implementing Reform Act provisions; (2) the impact 

the Reform Act has had on specific acquisition programs; and 

(3) challenges remaining in improving the weapons acquisition process.

To determine DOD’s progress in implementing Reform Act provisions we 

interviewed officials and analyzed documentation from the four offices 

whose leadership was established within OSD as a result of the Reform 

Act—Systems Engineering (SE), Developmental Test and Evaluation

(DT&E), Cost Assessment and Program Evaluation (CAPE), and 

Performance Assessments and Root Cause Analyses (PARCA).5 We 

focused our review on the offices’ implementation of four fundamental 

Reform Act provisions: developing policy and guidance; approving 

acquisition documents; monitoring programs and conducting program 

assessments; and developing performance measures. We also analyzed 

documentation we collected on selected weapon acquisitions. To 

determine the Reform Act’s impact on specific defense acquisition 

programs, we chose as case studies, 11 weapon systems at various 

points in the development process based on recommendations by OSD 

officials and GAO’s previous evaluations of these programs. At the time of 

our case study selection, all of the weapon systems in our review had 

been given Milestone A approval. This approval allows the programs to 

start the technology development phase of the acquisition process. Four 

of the 11 weapon systems had not yet received Milestone B approval. 

This approval signifies the start of engineering and manufacturing 

development activities. We believe these four programs offer the best 

insight into how the OSD offices and Reform Act policies are influencing 

 

3 GAO, Defense Acquisitions: DOD Needs to Develop Performance Criteria to Gauge 

Impact of Reform Act Changes and Address Workforce Issues, GAO-10-774 (Washington, 

D.C.: July 29, 2010).

4 GAO, Weapons Acquisition Reform: Actions Needed to Address Systems Engineering 

and Developmental Testing Challenges, GAO-11-806 (Washington, D.C.: Sept. 19, 2011).

5 The Director of Systems Engineering and the Director of Developmental Test and 

Evaluation offices were originally established as a result of the Reform Act. These offices 

have since been renamed as the offices of the Deputy Assistant Secretary of Defense for 

Systems Engineering and the Deputy Assistant Secretary of Defense for Developmental 

Test and Evaluation.
 

Page 3 GAO-13-103 Weapons Acquisition Reform

acquisition strategies. The other seven programs were conducting 

engineering and manufacturing development activities or had been given 

Milestone C approval to begin production and deployment phase 

activities. Each of these programs provided different insights in their 

interactions with the OSD offices, but some Reform Act provisions, such

as conducting preliminary design reviews prior to Milestone B, may not 

apply to these programs since they were beyond development start. A 

complete list of programs we reviewed is provided below.

Table 1: Programs Selected for Case Study Review

Programs that had not started development at the time of our case study selection System development start date

Ground Combat Vehicle December 2013 (estimated)

Joint Light Tactical Vehiclea August 2012

Ohio Class Replacement submarine August 2016 (estimated)

Ship to Shore Connectora July 2012

Programs that were in development at the time of our case study selection 

Global Hawk unmanned aerial vehicle March 2001

Gray Eagle unmanned aerial vehicle April 2005

Joint Strike Fighter October 2001

KC-46 Tanker aircraft February 2011

Littoral Combat Ship Seaframe February 2011

Remote Minehunting System December 1999

Small Diameter Bomb Increment II July 2010

Source: GAO analysis.

a

During the course of our review, the Joint Light Tactical Vehicle and Ship to Shore Connector 

programs held a Milestone B review and received approval to start development.

To determine challenges remaining for implementing the Reform Act, we 

conducted our own assessment and interviewed the leaders of the OSD 

offices established as a result of the Reform Act, military service 

acquisition chiefs, and program managers. These officials are responsible 

for advising and overseeing weapon acquisition program development, 

funding and developing new weapon acquisition programs, and executing 

the day-to-day development plans, respectively. We also solicited the 

opinions of the Under Secretary of Defense for Acquisition, Technology 

and Logistics and other senior OSD leaders. Additional information about 

our scope and methodology can be found in appendix I.

We conducted this performance audit from January 2012 to December 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 
 

Page 4 GAO-13-103 Weapons Acquisition Reform

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.

In May 2009, Congress passed the Weapon Systems Acquisition Reform 

Act of 2009 (Reform Act) in an effort to improve the way weapon systems 

are acquired and avoid further cost overruns on such programs. When 

signing the Reform Act into law, the President stated that its purpose was 

to limit weapon system cost overruns and that it would strengthen 

oversight and accountability by appointing officials who will closely 

monitor the weapons systems acquisition process to ensure that costs 

are controlled.

Four offices were established as a result of the Reform Act: SE, DT&E, 

CAPE and PARCA. The SE and CAPE offices existed under other 

organizational titles prior to the Reform Act. Staffing levels, following the 

Reform Act, remained relatively stable for both of these offices, but some 

reorganization was necessary to reflect new Reform Act responsibilities. 

The DT&E and PARCA offices were newly established. The key roles and 

responsibilities of these four offices as outlined in the Reform Act are 

explained below:

Background 
 

Page 5 GAO-13-103 Weapons Acquisition Reform

Table 2: Key Responsibilities of Offices Established as a Result of the Reform Act

Office Primary responsibilities

Systems Engineering • serves as principal advisor to the Secretary of Defense and the Under Secretary of Defense 

for Acquisition, Technology and Logistics on systems engineering activities in the department

• develops systems engineering and development planning guidance for DOD

• reviews and approves major defense acquisition program systems engineering plans

• monitors major defense acquisition program systems engineering activities

• reports to Congress annually on systems engineering organization, capabilities, and activities 

Developmental Test and 

Evaluation

• serves as principal advisor to the Secretary of Defense and the Under Secretary of Defense 

for Acquisition, Technology and Logistics on developmental test and evaluation activities

• develops developmental test and evaluation guidance for DOD

• reviews and approves major defense acquisition program developmental test and evaluation 

plans

• monitors and reviews acquisition program developmental test and evaluation activities of 

major defense acquisition programs

• reports to Congress annually on developmental test and evaluation organization, capabilities, 

and activities

Cost Assessment and Program 

Evaluation 

• serves as principal advisor to the Secretary of Defense and other senior officials on matters 

related to cost analysis and the planning and programming phases of the planning, 

programming, budgeting, and execution system

• develops independent cost estimates for major defense acquisition programs prior to major 

milestone decisions and updates independent cost estimates when programs have exceeded 

critical cost thresholds, known as Nunn McCurdy breaches

• reviews existing systems and methods for tracking and assessing operation and support 

costs on major defense acquisition programs

• develops analysis of alternative study guidance for major defense acquisition programs

• approves the analysis of alternatives study plan for each major defense acquisition program

Performance Assessments and 

Root Cause Analyses

• assesses major acquisition program performance through independent analyses and through 

the Defense Acquisition Executive Summary process

• identifies the root causes of cost growth and other problems on programs that experience a 

critical Nunn McCurdy cost breach 

Source: GAO Analysis of Weapon Systems Acquisition Reform Act of 2009.

Each of the offices has varying levels of interaction with defense 

acquisition programs. For example, the SE and DT&E offices have 

ongoing interaction with acquisition programs throughout development in 

the form of program reviews, working group activities, and other program 

meetings. They also coordinate program documents in preparation for 

major milestone reviews. CAPE issues guidance to programs on how to 

conduct an analysis of alternatives at the beginning of the acquisition 

process. The office approves the analysis of alternative study plan that is 
 

Page 6 GAO-13-103 Weapons Acquisition Reform

developed based on its guidance. It then develops independent cost 

estimates for major milestone reviews and in the event that an acquisition 

program experiences a Nunn-McCurdy breach.6

 According to PARCA, it 

assesses all major defense acquisition programs at least once per quarter 

or when requested by the Under Secretary of Defense for Acquisition, 

Technology and Logistics, and disseminates this information to senior 

leaders. The office also interacts with specific programs if they experience 

a Nunn-McCurdy breach. In these cases, the office assesses program 

performance not less than semi-annually until 1 year after it receives a 

new milestone approval.

In addition to the new organizational requirements, the Reform Act 

requires DOD to ensure that the acquisition strategy for major defense 

acquisition programs includes measures to ensure competition or the 

option of competition throughout the program life cycle. This could include 

strategies such as maintaining two sources for a system (dual-sourcing) 

and breaking requirements for supplies or services previously provided or 

performed under a single contract into separate smaller contracts 

(unbundling of contracts).7 Major defense acquisition programs are also 

required to provide for competitive prototyping—where two or more 

competing teams produce prototypes before a design is selected for 

further development—prior to Milestone B unless a waiver is properly 

 

6 10 U.S.C. § 2433 establishes the requirement for the DOD to prepare unit cost reports 

on major defense acquisition programs or designated subprograms. If a program exceeds 

cost growth thresholds specified in the law, this is commonly referred to as a NunnMcCurdy breach, which DOD is required to report to Congress and, if applicable, submit a 

certification to Congress in order to continue the program, in accordance with 10 U.S.C. 

§ 2433a. 

7 Pub. L. No. 111-23 § 202.
 

Page 7 GAO-13-103 Weapons Acquisition Reform

granted by the milestone decision authority,8 and to meet the following 

Milestone B certification requirements, including:9

• Appropriate trade-offs among cost, schedule, and performance 

objectives have been made to ensure the program is affordable;

• A preliminary design review and formal post-preliminary design review 

assessment have been conducted and on the basis of such 

assessment the program demonstrates a high likelihood of 

accomplishing its intended mission;

• Technology has been demonstrated in a relevant environment on the 

basis of an independent review and assessment by the Assistant 

Secretary of Defense for Research and Engineering;

• Reasonable cost and schedule estimates have been developed to 

execute, with concurrence of the Director of CAPE, the program’s

product development and production plan;

• Funding is available to execute the program’s product development 

and production plan;

• DOD has completed an analysis of alternatives with respect to the 

program; and

• The Joint Requirements Oversight Council10 has approved program 

requirements, including an analysis of the operational requirements.

 

8

 Pub. L. No. 111-23 § 203(a). Specifically, the Reform Act required DOD to modify its 

guidance relating to the operation of its acquisition system to incorporate these 

competitive prototyping provisions. DOD did so through Directive-Type Memorandum 

(DTM) 09-027, “Implementation of Weapon System Acquisition Reform Act of 2009 (Dec. 

4, 2009, incorporating Change 3, Dec. 9, 2011). Major defense acquisition programs are 

those estimated by DOD to require an eventual total expenditure for research, 

development, test, and evaluation, including all planned increments, of more than $365 

million, or for procurement, including all planned increments, of more than $2.19 billion in 

fiscal year 2000 constant dollars or designated by the Under Secretary of Defense for 

Acquisition, Technology, and Logistics. The Milestone Decision Authority for major 

defense acquisition programs is the Under Secretary of Defense for Acquisition, 

Technology and Logistics, the head of a DOD component, or if delegated the component 

acquisition executive. 

9 Pub. L. No. 111-23; various sections, codified at 10 U.S.C. § 2366b. The Reform Act 

revised the Milestone B certification requirements for trade-offs, preliminary design, 

technology demonstration, and reasonable cost and schedule estimates. The remaining 

Milestone B certification requirements bulleted here were unrevised by the Reform Act. 

10 The Joint Requirements Oversight Council is an advisory council to the Chairman of the 

Joint Chiefs of Staff with the responsibility to: (1) identify, assess, and approve joint 

military requirements; (2) assist acquisition officials in identifying alternatives to acquisition 

programs; and (3) assist the Chairman of the Joint Chiefs of Staff in assigning priority for 

joint military requirements. 
 

Page 8 GAO-13-103 Weapons Acquisition Reform

The Reform Act also requires the Joint Requirements Oversight Council 

to ensure trade-offs among cost, schedule, and performance objectives 

are considered for joint military requirements.11 GAO previously reported 

that the Council considered trade-offs made by the military services 

before validating requirements, but the military services did not 

consistently provide high-quality resource estimates to the Council for

proposed programs in fiscal year 2010. We also found that the Council 

did not prioritize requirements, consider redundancies across proposed 

programs, or prioritize and analyze capability gaps in a consistent 

manner.12

DOD has implemented most of the fundamental Reform Act provisions as 

required and is taking additional steps to strengthen acquisition reviews, 

policies, and capabilities. Offices established as a result of the Reform 

Act are continuing to issue policies, review and approve relevant 

acquisition documents, monitor weapon acquisition program activities, 

and develop performance measures. In addition, all four of the major 

defense acquisition programs we reviewed that had not started 

development when we selected our case studies plan to implement 

Reform Act provisions regarding preliminary design reviews, competitive 

prototyping, and competition. Also, some provisions, such as issuance of 

guidance on estimating operating and support costs, by the CAPE, are 

still in the process of being completed. Finally, we found that the Under 

Secretary of Defense for Acquisition, Technology and Logistics has 

revised the defense acquisition review process to consider additional 

knowledge collected on programs earlier and efforts are being made to 

strengthen acquisition policies and capabilities.

The offices established as a result of the Reform Act—SE, DT&E, CAPE, 

and PARCA—are continuing to make progress in implementing four 

fundamental Reform Act provisions aimed at strengthening acquisition 

outcomes and oversight of weapon acquisition programs. Specifically, the 

offices are (1) developing policy and guidance to the military services for 

conducting work in their respective areas, (2) approving acquisition 

 

11 Pub. L. No. 111-23 § 201.

12 GAO, DOD Weapon Systems: Missed Trade-off Opportunities During Requirements 

Reviews, GAO-11-502, (Washington, D.C.: June 16, 2011). 

DOD Has 

Implemented Most of 

the Fundamental 

Reform Act 

Provisions and Is 

Strengthening 

Acquisition Activities

DOD Is Continuing to 

Implement Reform Act 

Provisions
 

Page 9 GAO-13-103 Weapons Acquisition Reform

documents prior to milestone reviews, (3) monitoring and assessing 

weapon acquisition program activities on a consistent basis, and 

(4) developing performance measures to assess acquisition program 

activities. Figure 1 provides the status of DOD efforts to implement the 

four fundamental provisions. Some offices are still in the process of 

completing a few of these provisions. For example, CAPE and PARCA 

are in the process of developing policies and guidance for their respective 

areas and DT&E is in the process of establishing performance measures 

that can be used to assess weapon acquisition program activities. The 

office piloted the performance measures on two major defense acquisition 

programs and reported that they are currently applying them to over 40 

programs. Note that some activities related to approving documents and 

monitoring or assessing programs require on-going efforts on the part of 

some of the offices.
Figure 1: DOD’s Progress in Implementing Four Fundamental Reform Act Provisions

Source: GAO analysis of DOD data.

a The Reform Act does not specify a date of completion for developing policy and guidance.

b DOD Directive-Type Memorandum (DTM) 11-003, “Reliability Analysis, Planning, Tracking,

 and Reporting,” (Mar. 21, 2011); DOD DTM 10-017, “Development Planning to Inform Materiel

 Development Decision Reviews and Support of Analysis of Alternatives,” (Sept. 13, 2010,

 Incorporating Change 2, Dec. 9, 2011); DOD Instruction 5134.16, “Deputy Assistant Secretary

 of Defense for Systems Engineering (DASD(SE)),” (Aug. 19, 2011).

Office

Developmental

Test and 

Evaluation

Systems

Engineering

Cost Assessment

and Program

Evaluation

Performance

Assessment 

and Root Cause 

Analyses

Develop 

policy and

guidancea

Monitor 

programs/

conduct 

assessments

Develop

performance

measures

Reform Act provisions

status

Approve

documents

Interactive Graphic Click on status to show Reform Act provisions. Click on the “” to clear. See appendix II for the non-interactive, printer-friendly version.

In process

Completed

Completed

In process

Not

applicable

Completing

on annual

basis

Completing

on annual

basis

Not

applicable

Completing

on annual

basis

Completing

on annual

basis

Completing

on annual

basis

Completing

on annual

basis

Not

applicable

Completed

In process

In process

Page 10 GAO-13-103 Weapons Acquisition Reform
 

Page 11 GAO-13-103 Weapons Acquisition Reform

We also found evidence that major defense acquisition programs are 

integrating Reform Act provisions in their acquisition strategies. The four 

weapon acquisition programs we reviewed that had not started 

development activities when we began our review plan to implement 

Reform Act provisions related to preliminary design reviews, competitive 

prototyping, and competition. For example, the Ground Combat Vehicle 

has two contractors developing competitive prototypes of two key 

subsystems to support technology development. The program intends to 

conduct preliminary design reviews on both contractors’ designs prior to 

Milestone B and to conduct full and open competition through Milestone 

C. Similarly, according to program officials, the Joint Light Tactical 

Vehicle program had three contractors develop full-system prototypes 

during the technology development phase and held preliminary design 

reviews on each contractor’s design prior to Milestone B. The program 

plans to continue competition throughout engineering and manufacturing 

development. None of the four programs in our review received a waiver 

from Reform Act provisions. 

OSD is taking additional steps to strengthen the department’s oversight of 

weapon acquisition programs and guidance for developing the programs. 

In June 2011, for example, the Under Secretary for Acquisition, 

Technology and Logistics revised the weapons acquisition review process 

to consider acquired knowledge on weapon acquisition programs earlier 

than before. The revised review process includes two new review points. 

The first new review—the pre-engineering and manufacturing 

development review—occurs before the release of a final request for 

proposal for the engineering and manufacturing development phase. The 

purpose of this new review is to assess each program’s acquisition 

strategy, request for proposal, and key related planning documents earlier 

in the process, and to determine whether program plans are affordable, 

executable, and reflect sound business arrangements. The second new 

review—the acquisition strategy and request for proposals review and 

approval—occurs prior to Milestone C, the production decision. The 

review provides the milestone decision authority an opportunity to review 

the acquisition strategy and request for proposals for the production and 

deployment phase prior to Milestone C. Figure 2 illustrates the revised 

review process.

Additional Emphasis Is 

Being Placed on Improving 

Milestone Review Process, 

Policies, and Service 

Capabilities
 

Page 12 GAO-13-103 Weapons Acquisition Reform

Figure 2: Revised Review Process for Major Defense Acquisition Programs

According to the Under Secretary of Defense for Acquisition, Technology 

and Logistics, who is the authority for making milestone decisions for 

most major weapon acquisitions, the prior review process did not provide 

an adequate opportunity for review of program plans prior to release of 

the final request for proposals—the point at which DOD’s requirements, 

schedule, planned program content, and available funding should be firm 

and available for review. Further, the Under Secretary stated that making 

changes to acquisition strategies and program plans after all bidding 

activities, proposal evaluation, and source selection are complete is 

difficult and highly disruptive.13

DOD is also rewriting the DOD Instruction 5000.0214 to include an 

extensive restructure of acquisition policies according to the Under 

Secretary of Defense for Acquisition, Technology and Logistics. This 

update will implement Section 832 of the National Defense Authorization 

Act for Fiscal Year 2012,15 which requires DOD to issue guidance on 

actions to be taken to improve its processes for estimating, managing, 

and reducing operation and support cost, as well as ensure competition in 

maintenance and sustainment of subsystems of major weapon systems, 

among other things. In addition to current policies implementing the 

Reform Act, officials stated that key provisions from the Reform Act will 

also be included in the updated instruction.

 

13 DOD Principal Deputy Under Secretary of Defense for Acquisition, Technology and 

Logistics, “Improving Milestone Process Effectiveness,” (June 23, 2011).

14 DOD Instruction 5000.02, Operation of the Defense Acquisition System, Dec. 2, 2008.

15 Pub. L. No. 112-81 § 832.
 

Page 13 GAO-13-103 Weapons Acquisition Reform

In addition to implementing the provisions of the Reform Act, DOD offices 

have taken other steps to strengthen acquisition capabilities throughout 

the department. For example:

• The SE office, according to DOD officials, led efforts to establish 

working groups to help the services address systemic reliability issues 

across the unmanned aircraft and rotary wing portfolios earlier in the 

process. The office also led several workforce development initiatives 

to attract and retain a qualified engineering workforce and supported 

the implementation of legislation requiring each acquisition program 

office to name a key technical advisor who is responsible for all 

engineering activities.

• The DT&E office, according to DOD officials, championed updates to 

DOD Instructions that will require weapon acquisition programs to 

consider using DOD test capabilities before paying contractors to 

develop similar capabilities. In addition, the office supported 

legislation requiring major defense acquisition program offices to have 

a government test agency serving as the lead developmental test and 

evaluation organization for the program and a chief developmental 

tester. The chief developmental tester position, as included in the 

National Defense Authorization Act for Fiscal Year 2012, is tasked 

with coordinating the planning, management, and oversight of all 

developmental testing activities, among other things.16

• The CAPE office, according to DOD officials, established an operating 

and support cost directorate to build its expertise and place more 

emphasis on developing better operating and support cost estimates 

throughout the acquisition life cycle. This directorate will coordinate 

the development of an operating and support cost estimating 

guidebook.

• The PARCA office, according to DOD officials, is providing additional 

insights to the Under Secretary of Defense for Acquisition, 

Technology and Logistics on systemic acquisition problems. 

Specifically, the office is examining a wide range of acquisition-related 

information from the past 40 years, such as contract type, stability of 

key performance parameters, and program manager tenure to 

 

16 Pub. L. No. 112-81 § 835.
 

Page 14 GAO-13-103 Weapons Acquisition Reform

determine if there is any statistical correlation between these factors 

and good or poor acquisition outcomes.

We identified four key areas where the Reform Act had a significant 

influence on programs in the 3 years since it was enacted: 

(1) requirements, (2) cost and schedule, (3) testing, and (4) reliability. 

These four areas have been common sources of problems in the past. 

For example, the services typically started new weapon acquisition 

programs with requirements that were both demanding and inflexible and 

planned to use relatively unproven technologies to meet the 

requirements—all of which increased program risks. In addition, cost and 

schedule estimates were frequently too optimistic based on the proposed 

requirements and technologies. Design problems stemming from rigid 

requirements and the use of immature technologies to meet them were 

often discovered during testing and fixed late in the development cycle 

and resulted in cost increases, performance shortfalls, and schedule 

delays. Finally, DOD’s inattention to reliability has resulted in a dramatic

increase in the number of systems that have not met suitability 

requirements during operational testing. Deficiencies—such as high 

failure rates and disappointing improvements in the reliability, availability, 

and maintainability of weapon systems—have limited program 

performance and increased operation and support costs.17 We examined 

11 programs at various stages of the acquisition process to determine 

how the offices and policies established as a result of the Reform Act 

impacted their acquisition strategies and decision-making process. Four 

programs had not yet passed Milestone B, development start, at the time 

we began our review. Of the remaining seven programs, three had 

breached Nunn-McCurdy cost thresholds since the act was passed and 

have had to satisfy the Reform Act’s new requirements with regards to 

certification. The other programs had significant interaction with one or 

more of the OSD offices established by the Reform Act. Table 3 indicates

in which of the four areas each program has been affected by the Reform 

Act.

 

17 DOD Report of the Defense Science Board Task Force on Developmental Test and 

Evaluation (May 1, 2008).

The Reform Act Is 

Helping Programs 

Identify and Mitigate 

Risks Earlier in the 

Acquisition Process
 

Page 15 GAO-13-103 Weapons Acquisition Reform

Table 3: Reform Act Influence on Case Study Programs

Program Requirements Cost and schedule Testing Reliability 

Before Milestone B

Ground Combat Vehicle     

Joint Light Tactical Vehiclea     

Ohio Class Replacement     

Ship to Shore Connectora   

After Milestone B 

Joint Strike Fighter  

Global Hawk    

Gray Eagle     

KC-46 Tanker   

Littoral Combat Ship Seaframe  

Remote Minehunting System    

Small Diameter Bomb II    

Source: GAO analysis of DOD data.

a

During the course of our review, the Joint Light Tactical Vehicle and Ship to Shore Connector 

programs held a Milestone B review.

In some cases, programs have made changes based on input from OSD 

offices like systems engineering or developmental test and evaluation; in 

other cases, programs have integrated Reform Act policies, such as 

preliminary design reviews and competitive prototyping, into their 

acquisition planning. Programs that were already in development or 

production when the Reform Act was passed were less likely to have 

interactions with the OSD offices on requirements trades because these 

discussions typically occur prior to Milestone B. A discussion of how 

individual programs have been affected in the areas of requirements, cost 

and schedule realism, testing, and reliability follows.

The Reform Act places significant emphasis on early problem solving and 

requires programs to put much more effort toward considering trade-offs 

among cost, schedule, and performance requirements prior to Milestone 

B. As part of this effort, it requires the Secretary of Defense to ensure that 

acquisition, budget, and cost estimating officials have the opportunity to 

raise cost and schedule matters before performance objectives are 

established. The Reform Act also charges the Joint Requirements 

Oversight Council with the responsibility to ensure that cost, schedule, 

and performance trade-offs for joint military requirements are considered, 

Greater Emphasis Is Being 

Placed on Requirements, 

but Challenges Remain
 

Page 16 GAO-13-103 Weapons Acquisition Reform

and to include combatant commanders in the process to ensure the 

user’s needs are adequately satisfied. The offices established as a result 

of the Reform Act have helped programs, such as the Joint Light Tactical 

Vehicle and Ground Combat Vehicle, make trade-offs among cost, 

schedule, and technical performance requirements. As a result, these 

programs have developed a more realistic acquisition strategy from a 

cost, schedule, and technical standpoint.

• Joint Light Tactical Vehicle: The program held several reviews prior 

to Milestone B to identify, modify, or eliminate requirements that were 

unachievable or unaffordable, thus establishing a more technically 

realistic program. Officials from DT&E and SE participated in these 

reviews. By involving both the requirements and acquisition 

communities in the reviews, the Army was able to reduce the required 

capability to cut costs while ensuring that trade-off decisions would 

not impair the system’s ability to meet the warfighter’s operational 

needs. Examples of requirements changes that helped to cut costs as 

well as reduce risk include:

• allowing the active suspension system, crew displays, and 

integrated starter-generator to be tradable design features. These 

changes resulted in a 30 percent reduction in the average unit 

manufacturing cost from the initial target of $475,000 to $331,000, 

while at the same time reducing technical and weight risk.

According to program officials, this makes the $250,000 unit 

manufacturing goal more achievable. 

• reducing the reliability requirement and changing the Army 

helicopter lift requirement based on the results of technology 

development prototype testing. This mitigated technical risks 

going into development.

The program recently moved into engineering and manufacturing 

development, but not all requirements issues have been resolved and 

future trade-offs may be necessary. For example, early testing 

showed that none of the three prototype variants met the program’s 

soft soil or sand slope requirement. This requirement has not been 

changed. Program and OSD officials are monitoring this issue closely 

and plan to actively manage it during engineering and manufacturing 

development. 

• Ground Combat Vehicle: The Ground Combat Vehicle program 

exhibited some of the same problems experienced by previous DOD 

programs prior to Milestone B—demanding and inflexible 

requirements. The SE office and the Under Secretary of Defense for 
 

Page 17 GAO-13-103 Weapons Acquisition Reform

Acquisition, Technology and Logistics are helping the program set 

achievable requirements. Following its materiel development decision 

in February 2010, the program issued a request for proposals that 

contained nearly 1,000 requirements and a challenging 7-year 

schedule for the delivery of the first production vehicle. At the request 

of the Under Secretary for Acquisition, Technology and Logistics, the 

Army established an independent review team to assess the risks 

associated with the program’s schedule. The team, which included an 

SE official, raised concerns about the program’s high number of 

mandatory requirements and the risks associated with the 7-year 

schedule. To mitigate program risks, the Army reduced the number of 

performance requirements by about 25 percent and prioritized the 

others, giving competing contractors flexibility in addressing some 

performance requirements. The Army issued a revised request for 

proposal in November 2010. In August 2011, the Under Secretary of 

Acquisition, Technology and Logistics approved the program’s entry 

into technology development, but expressed concern about the cost 

and schedule risks associated with delivering a production vehicle in 7 

years. Because of these concerns, the Under Secretary directed the 

Army to consider other alternatives, such as existing vehicles, that 

could meet warfighter needs. The analysis is currently planned to be 

completed in March 2013 to inform the Milestone B decision.

By establishing a new cost assessment and program evaluation office 

and requiring this office to scrutinize program cost and schedule 

estimates beginning at Milestone A, CAPE officials believe that the 

Reform Act has helped infuse more realism in cost estimates and 

promote earlier discussions about affordability. CAPE officials also 

believe that because their independent cost and schedule estimates have 

become more visible within DOD and Congress, the military services are 

developing more realistic estimates. We saw evidence of these benefits in 

the programs we reviewed, including the Ohio Class Replacement, 

Littoral Combat Ship Seaframe, and F-35 Joint Strike Fighter programs.

• Ohio Class Replacement: The CAPE office was involved in the 

decision-making process to ensure program affordability. The office 

prepared an independent cost estimate and reviewed the program’s 

affordability goals prior to Milestone A. The service and independent 

estimates were within 2 percent of each other. However, the Under 

Secretary of Defense for Acquisition, Technology and Logistics 

directed the Navy to do a rigorous cost comparison of a 16 missile 

tube design versus a 20 missile tube design. The Navy determined 

that a 16 missile tube configuration would meet warfighter 

Program Cost and 

Schedule Estimates Are 

Becoming More Realistic
 

Page 18 GAO-13-103 Weapons Acquisition Reform

requirements and users’ needs while reducing program costs by 

about $200 million per submarine, or approximately $3 billion for the 

total program. It would also simplify the ships’ design and integration 

effort. The CAPE office validated the savings associated with the 16 

missile tube design. As a result, the Navy incorporated the 16 missile 

configuration as the program baseline.

• Littoral Combat Ship Seaframe: The CAPE office helped make 

program costs more visible. Prior to the program’s Milestone B 

decision, CAPE completed an independent cost estimate of the 

seaframe program and found that the resources in the future years’ 

defense plan budget were lower than the projected program costs for 

the same time period. Navy officials attributed this problem to the 

overlap between the timing of the milestone decision and the 

president’s budget submission. The office further noted that the 

resources in the Navy’s budget did not include the additional 

development activities required to support two full ships. Without this 

information, decision makers would not have had visibility into the 

expected costs of the seaframe program or be able to make more fully 

informed decisions. As a result, the Navy re-phased its funding in the 

budget, adding approximately $397 million to fully fund the 

development program.

Joint Strike Fighter: The SE and CAPE offices helped the program 

develop more realistic cost and schedule estimates. CAPE officials 

have been involved in reviews of the F-35 Joint Strike Fighter program 

even prior to the passage of the Reform Act and have continued to be 

heavily involved in subsequent programs reviews. For example, the 

cost analysis improvement group, which was the predecessor to 

CAPE, led a multi-functional joint estimating team review of the 

program in 2008. This review found problems with the program’s 

funding and schedule. In 2010, the Joint Strike Fighter program 

notified Congress that its estimated unit costs had increased by more 

than 80 percent since the original Milestone B baseline in 2001. This 

increase triggered a Nunn-McCurdy unit cost breach and later 

prompted the program executive officer to commission a technical 

baseline review of the program to help determine the resources 

needed to complete development. Officials from the SE office 

participated in this technical review. The CAPE office also did an 

independent cost estimate of the program as part of the NunnMcCurdy certification process. Based on information from these 

efforts, DOD developed a more realistic program plan by adding $4.6 

billion to the development program, reducing near-term procurement 

quantities by 125 aircraft, and extending the development test period 
 

Page 19 GAO-13-103 Weapons Acquisition Reform

by 4 years to accommodate developmental testing, address the 

increased program scope, and fix software issues.

The Reform Act significantly strengthened the role of developmental 

testing in the department. In the 2 decades prior to the Reform Act, the 

prominence of developmental testing had declined within OSD. In the 

early 1990’s, developmental testing was part of an all-encompassing test 

organization that reported directly to the Under Secretary of Defense for 

Acquisition. According to a former senior developmental testing official, by 

2004, two people worked on developmental testing activities within the 

systems engineering organization. In establishing a separate office for 

DT&E, the Reform Act reinforced the need for robust developmental 

testing early in the acquisition process. The Reform Act gave the Deputy 

Assistant Secretary for Developmental Test and Evaluation formal 

approval authority for the test and evaluation master plans of major 

defense acquisition programs. This authority enables the Deputy 

Assistant Secretary to help ensure that programs have robust test and 

evaluation plans. Our case study reviews illustrate the efforts that the 

DT&E office has made to help programs such as the Small Diameter 

Bomb II and KC-46 Tanker obtain more design and performance 

knowledge early in a program’s acquisition life cycle.

• Small Diameter Bomb II: Air Force program officials acknowledged 

that developmental and operational testing officials worked closely 

with them as they prepared the test and evaluation master plan for the 

Milestone B decision. After reviewing the plan, developmental and 

operational test officials concluded that the program would benefit 

from adding a 28-shot test program prior to entering operational 

testing. According to the program office, the purpose of the additional 

testing is to further establish the performance of the weapon in 

realistic scenarios and to increase the likelihood of completing 

operational testing without a failure. DT&E officials stated this testing 

would provide more complete knowledge about the bomb’s 

functionality and help reduce risk of a major redesign moving forward. 

Program officials stated that they allocated an additional $41 million to 

its developmental test program to conduct the 28 additional flight tests 

prior to operational testing.

• KC-46 Tanker: The program office acknowledged that the DT&E 

office, as part of an integrated test team comprised of government 

and industry officials, helped identify options that could add time to the 

test plan for important testing if unexpected delays are encountered. 

In its fiscal year 2011 annual report, the operational test and 

Programs Are Adding More 

Tests to Reduce 

Development Risks 
 

Page 20 GAO-13-103 Weapons Acquisition Reform

evaluation office reported that the KC-46’s planned flight test program 

was not executable, determining that more time would be needed for 

military flight-testing. It based this conclusion on the historical flight 

test experience of similar programs. Program officials stated that they 

were initially reluctant to change the test plan because they had 

awarded a fixed price contract and any changes could result in 

reopening the contract, leading to potential cost increases. However, 

the integrated test team identified a recovery period that may be 

applied to the KC-46 aerial refueling certification if delays are 

encountered. The contractor now has a plan that could allocate an 

additional 1.5 months for two test aircraft to complete this testing, if 

deemed necessary. This testing would provide more knowledge about 

the program’s aerial refueling performance prior to operational testing. 

DT&E officials stated that they plan to continue working with the 

program to address overall flight test challenges. While testing 

remains one of the program’s risk areas, this change may lessen that 

risk. 

The Reform Act emphasizes the need for designing more reliable weapon 

systems. It charges the Deputy Assistant Secretary for Systems 

Engineering with the responsibility to ensure the systems engineering 

approach used by major acquisition programs includes a robust plan for 

improving reliability. The DT&E office reviews programs’ reliability growth 

test plans. This testing provides visibility over how reliability is improving 

and uncovers design problems so fixes can be incorporated before 

production begins. A reliability growth curve is used to track projected and 

actual improvements in reliability over time. The Reform Act further 

requires that the Deputy Assistant Secretary for Systems Engineering 

develop policies and guidance for the inclusion of provisions relating to 

systems engineering and reliability growth in requests for proposals. We 

observed evidence of this increased emphasis in the Joint Light Tactical 

Vehicle, Remote Minehunting System, Gray Eagle, and Global Hawk 

programs.

• Joint Light Tactical Vehicle: The DT&E office helped this program 

develop a more realistic reliability growth plan prior to Milestone B. 

Based on the performance of prototype vehicles, developmental test 

officials determined that the program’s reliability growth curve was 

unrealistic. For example, officials reported that the program’s initial 

reliability growth plan assumed a starting reliability that was almost 60 

percent higher than what had actually been demonstrated during 

technology development. It also assumed commonality between the 

two vehicle variants, a large reliability increase in a short test time, 

Greater Emphasis Is Being 

Placed on Weapon System 

Reliability
 

Page 21 GAO-13-103 Weapons Acquisition Reform

and two corrective action periods. The DT&E office recommended 

that the program eliminate the vehicle commonality assumption, add 

more test miles, and add another corrective action period to its test 

plan. It also recommended that the program consider lowering the 

vehicle’s reliability requirement. Based on this input, the Army revised 

its plan by adding two vehicles and 40,000 more test miles to ensure 

reliability is adequately addressed for both variants. With approval of 

the user, it also reduced the reliability requirement from 3,600 to 2,400 

miles mean time between operational mission failures.

• Remote Minehunting System: The SE office worked with program 

officials to improve reliability growth planning, which was found to be 

one of the key factors leading to the program’s Nunn McCurdy unit 

cost breach in 2009. Before the breach, program officials had not 

funded a reliability growth program or established a design for 

reliability process. The program had a reliability goal of 150 hours 

mean time between failures, but program officials stated that testing 

demonstrated a reliability of only about 45 hours. Since the breach, 

the program has worked closely with the SE office to establish a 

reliability program plan and a growth curve to track reliability 

improvements. During the Nunn-McCurdy review, the program 

developed a three phase reliability growth program to improve the 

program’s subsystems, components, and manufacturing processes 

that contributed to poor reliability. According to program officials, 

phase one of the reliability growth program was completed in 2011, 

and reliability has improved by 40 percent, going from 45 hours mean 

time between operational mission failures to 63 hours. Although this 

improvement is still below the minimum requirement of 75 hours,

program officials stated that phase two of the reliability growth 

program is scheduled to be completed in April 2013 and is projected 

to achieve the program’s 75 hour minimum requirement. 

• Gray Eagle and Global Hawk: The SE office has worked to improve 

reliability across the unmanned aircraft portfolio, including the Gray 

Eagle and Global Hawk. Prior to the Gray Eagle’s second low rate 

initial production decision in 2011,18 SE officials raised concerns about 

the system’s poor reliability. As a result, the Army was directed to 

undertake a reliability improvement program. The Under Secretary of 

 

18 Low-rate initial production is production of the system in the minimum quantity 

necessary to provide articles for operational tests, establish an initial production base, and 

permit an orderly increase in the production rate for the system.
 

Page 22 GAO-13-103 Weapons Acquisition Reform

Defense for Acquisition, Technology and Logistics approved the 

program for low rate initial production, but stressed the need to 

improve the operational reliability as quickly as possible. SE officials 

worked with program officials to establish a reliability working group, 

develop reliability growth curves, and develop a reliability 

enhancement management plan. According to SE officials, the Gray 

Eagle initially improved the reliability of the aircraft by 15 percent and 

the ground control station by 30 percent. According to program 

officials, the initial reliability goals were overstated and not needed to 

meet the program’s overall operational availability requirement. Based 

on initial operational test results in August 2012, the program office is 

working with the user to redefine the reliability goals without impacting 

the system’s ability to meet its overall operational availability 

requirement. According to the PARCA office, these efforts have been 

informed by a detailed reliability model that they built in consultation 

with the Army. This model showed the relationship between the 

aircraft’s reliability and its availability to perform operational missions. 

SE officials also found similar reliability problems on the Global Hawk 

program and worked with program officials to establish a reliability 

growth and improvement plan and reliability growth curves. According 

to SE officials, the time between unscheduled maintenance on the 

Global Hawk has improved on the order of 50 to 80 percent.

While DOD has taken steps to implement most of the fundamental 

Reform Act provisions, some key efforts to date have been primarily 

focused on DOD’s largest major defense acquisition programs. 

Expanding the reach of the Reform Act to bring about systemic change to 

DOD’s weapons acquisition process so that it influences all programs, 

however, still has challenges. Although senior leaders were receptive to 

the Reform Act principles, they identified several challenges that currently 

limit DOD’s ability to broaden the Reform Act’s influence. We grouped 

these challenges into five general categories: (1) organizational capability 

constraints; (2) need for additional guidance on cost estimating and 

Reform Act implementation; (3) uncertainty about the sufficiency of 

service level systems engineering and developmental testing resources; 

(4) limited dissemination of lessons learned; and (5) cultural barriers. 

Leaders of two of the offices established as a result of the Reform Act told 

us that even though they have implemented most of the fundamental 

Reform Act provisions, they have had to limit their activities to a portion of 

acquisition programs in their portfolios due to resource constraints. Thus,

it is doubtful that they could expand the scope of their activities to include 

Challenges Exist That 

Could Limit the 

Reform Act’s Ability 

to Influence Systemic 

Change 

Organizational Capability 

Constraints Make It 

Difficult to Expand Reform 

Act’s Impact
 

Page 23 GAO-13-103 Weapons Acquisition Reform

more weapon acquisition programs at current staffing levels. For 

example, the DT&E office has had to be selective in its level of oversight 

of acquisition programs because the current staff of around 70 

government and contractor personnel cannot adequately cover a portfolio 

of over 200 acquisition programs, according to its Deputy Assistant 

Secretary. The office has dropped virtually all but the largest programs 

from its oversight list and eliminated oversight of some major automated 

information systems. CAPE officials estimated that its cost assessment 

division would need to double in size in order to meet the Reform Act’s 

requirements. However, soon after the Reform Act was enacted, 

budgetary constraints limited the expansion of the cost estimating 

workforce to about 25 percent of the necessary growth. According to 

CAPE officials, its current cost analysis staffing is not adequate to meet 

its mission of improving the analytical skills of the defense cost estimating 

workforce, issuing policy, and providing sound and unbiased cost and 

schedule estimates. The office has delegated its independent cost 

estimating responsibility for most major automated information systems to 

the military services and some guidance has yet to be issued.

The SE and PARCA offices are also struggling in some regards. For 

example, according to its Deputy Assistant Secretary, the SE office is 

continuously challenged to maintain the high caliber, qualified personnel 

required to provide assistance to and oversight of its portfolio of over 200 

acquisition programs. Further, PARCA officials stated that the availability 

of government positions, particularly at the senior executive service level, 

continues to be a critical issue for the office. The two divisions within the 

PARCA office, the performance assessments division and the root cause 

division, do not currently have permanent government personnel at the 

senior executive level. Officials also stated current proprietary information 

rules limit the ability of PARCA contractor personnel to handle and 

maintain some weapon system information, severely impeding 

operations.

Offices within OSD have not yet issued more detailed guidance that could 

help institutionalize better cost estimating practices and steer program 

decisions related to competitive prototyping and preliminary design 

reviews. The CAPE office has not issued guidance for operating and 

support costs estimates, such as fuel and maintenance costs, that have 

been estimated to account for two-thirds or more of a system’s total life 

Services Would Like More 

Cost Estimating and 

Reform Act 

Implementation Guidance
 

Page 24 GAO-13-103 Weapons Acquisition Reform

cycle cost.19 In addition, although not specifically required by the Reform 

Act, the CAPE office has not issued guidance for the services to use 

when developing Milestone A program cost estimates. As a result, senior 

leaders may not have access to realistic cost estimates prior to Milestone 

B for decision making purposes. Military service officials told us they are 

particularly interested in getting guidance on data that should be included 

in the cost analysis requirements description, which forms the basis for 

their cost estimates at Milestones B and C. CAPE officials recognize that, 

while some progress has been made, they need to complete the 

guidance, but have not been able to dedicate resources to do so. 

Some officials also told us that they find the competitive prototyping and 

preliminary design review requirements confusing and would like 

guidance on how to implement these requirements. DOD policy requires 

the technology development strategy for major defense acquisition 

programs to provide for prototypes of the system or, if a system prototype 

is not feasible, for prototypes of critical subsystems before Milestone B 

approval.20 However, officials from the Ground Combat Vehicle program 

were unclear as to when and what type of prototype to use. From a 

broader perspective, other military officials questioned the value of 

competitive prototyping as a blanket requirement for all programs, 

especially for programs that are using mature technologies, given the 

cost. For example, senior acquisition officials questioned the necessity of 

spending $400 million on competitive prototyping for the Small Diameter 

Bomb II program since the program was aware of problems with one 

contractor’s design. However, program officials indicated that competitive 

prototyping enabled them to identify design issues early in development 

and realize a savings of $1 billion. Officials from the Ground Combat 

Vehicle program we spoke with also indicated that they struggled with the 

timing of when to hold the program’s preliminary design review and what 

type of knowledge was required, since better guidance is needed. The 

program plans to hold multiple design reviews prior to Milestone B to 

consider contractor and government designs of the weapon system and 

then hold another review after Milestone B in order to resolve differences 

between the government’s and selected contractor’s preliminary designs. 

 

19 The Reform Act required CAPE to issue guidance relating to full consideration of life 

cycle management and sustainability costs in major defense acquisition programs.

20 DOD Directive-Type Memorandum 09-027, Implementation of the Weapon Systems 

Acquisition Reform Act of 2009 (Dec. 4, 2009, incorporating Change 3, Dec. 9, 2011).
 

Page 25 GAO-13-103 Weapons Acquisition Reform

We spoke with OSD officials to determine which office should be 

providing guidance or assistance to program managers on competitive 

prototyping and preliminary design review issues. None of the offices 

have official responsibility for these efforts. OSD officials stated that these 

are program decisions and should be discussed with their respective 

military service level acquisition officials.

OSD officials believe that the services may lack resources in key 

positions that could help strengthen systems engineering and 

developmental testing activities on weapon acquisition programs. For 

example, according to the Deputy Assistant Secretary of Defense for 

Systems Engineering, the Navy and Air Force have reassigned the duties 

and responsibilities of their service-level chief engineers, thereby deemphasizing the importance of systems engineering. The Deputy 

Assistant Secretary believes maintaining strong systems engineering 

leadership at the service level is essential for tying the systems 

engineering community together and promoting good systems 

engineering practices throughout each respective service. According to 

the DT&E and SE office’s March 2012 joint annual report to the 

Congress, the Navy abolished its chief engineer position and while the Air 

Force recently began to take steps to relocate the systems engineering 

function to the headquarters level, the impact of a recent reorganization 

on systems engineering activities is not yet known. In addition, the Deputy 

Assistant Secretary for Developmental Test and Evaluation expressed 

concern that the military services may not be implementing new 

legislation that requires each major defense acquisition program be 

supported by a chief developmental tester that oversees developmental 

test and evaluation activities. He stated that in some cases one person is 

serving as the chief developmental tester across multiple programs 

instead of having one person dedicated specifically to each program.21

The Deputy Assistant Secretary is trying to determine the extent to which 

this practice is occurring and then plans to work with the services to get 

more focused leadership for each program.

 

21 The requirement contained in the National Defense Authorization Act for Fiscal Year 

2012 required “each major defense acquisition program to be supported by a chief 

developmental tester.” However, it did not specifically prohibit one person serving as the 

chief developmental tester across multiple programs. 

Services May Not Have 

Sufficient Resources to 

Oversee and Conduct 

Systems Engineering and 

Developmental Testing 

Activities
 

Page 26 GAO-13-103 Weapons Acquisition Reform

It is also unclear whether the services have a sufficient number of 

qualified personnel to conduct systems engineering and test and 

evaluation activities. The services planned to grow these workforces by 

over a combined 5,000 people between fiscal years 2009 and 2015 and 

had made progress in growing each of these workforces through fiscal 

year 2010. However, budget cuts have resulted in DOD canceling some 

of its weapon acquisition programs and reassessing its decision to 

increase the acquisition workforce. Last year, we recommended that the 

Secretary of Defense report the impact budget cuts were having on the 

military service workforce and their ability to meet weapon acquisition 

program needs in the areas of developmental testing and systems 

engineering.22 In the DT&E and SE offices’ March 2012 joint annual report 

to the Congress, the Deputy Assistant Secretary for Systems Engineering 

reported that the Army has reduced its systems engineering workforce 

growth plan as compared to the plan reported in March 2011 joint annual 

report, and that contractor-to-civilian conversions have been suspended. 

In addition, the Deputy Assistant Secretary believes a prolonged hiring 

freeze in the Air Force could potentially create new experience gaps in 

the workforce. The Deputy Assistant Secretary for Developmental Test 

and Evaluation did not discuss the impact of budget cuts on the services’ 

test and evaluation workforce growth plan in the March 2012 joint annual 

report to the Congress and neither office reported on whether the 

services had an adequate workforce to meet the needs of the current 

portfolio of weapon acquisition programs.

DOD has not taken full advantage of sharing lessons learned obtained 

through root cause analyses of programs that experience Nunn-McCurdy 

cost and schedule breaches with the acquisition workforce, particularly 

program managers. According to the Defense Acquisition Guidebook, 

which provides best practices the acquisition workforce can use on 

programs, lessons learned are a tool that the program manager may use 

to help identify potential areas of risk associated with a weapon 

acquisition system by reviewing the experiences encountered in past 

programs. Lessons learned databases document what worked and what 

did not work in past programs, in the hopes that future programs can 

avoid the same pitfalls. Further, if the right best practices are applied, 

they help to avoid common problems and improve quality, cost, or both. 

 

22 GAO-11-806. 

Expanded Use of Lessons 

Learned Would Help 

Expand Impact across the 

Acquisition Portfolio
 

Page 27 GAO-13-103 Weapons Acquisition Reform

The PARCA office has made some effort to educate program managers 

on how to avoid acquisition problems through classes taught at the 

Defense Acquisition University. However, these courses are geared 

towards educating new program managers and may not be reaching a 

wide range of program officials. Nevertheless, some officials indicated 

that this information would be helpful for program officials to understand 

and avoid problems that have affected weapon acquisition programs in 

the past. Other officials also stated that it would be helpful if root cause 

analysis assessments contained more detailed information so acquisition 

officials could better understand problems and apply lessons learned. For 

example, when cost estimating was determined to be a root cause of a 

problem, officials stated they would have found it more beneficial to know 

if immature technologies or unrealistic requirements were the basis for 

the poor cost estimate.

Perhaps the most difficult challenge the department faces in making 

systemic changes to the acquisition process is changing the cultural 

relationship between the military services, which fund and develop new 

weapon acquisition programs, and OSD offices, which provide advice to 

and oversee the programs. Senior military service officials have told us 

they believe they understand and can manage the risks of specific 

weapon acquisition programs without much assistance from OSD. On the 

other hand, OSD officials believe more assistance is needed, as 

evidenced by the high number of programs that have experienced NunnMcCurdy breaches and poor operational testing results. For example, 

since it was established in 2009, the DT&E office has assessed whether 

15 programs were ready to begin operational testing. The office 

recommended that 5 of the programs—Global Hawk Block 20/30, 

Standard Missile 6, Joint Tactical Radio System Handheld Manpack 

Small Form (HMS) Rifleman Radio, Joint Tactical Radio System HMS 

Manpack, and MQ-1C Gray Eagle—not proceed into operational testing. 

However, military service acquisition chiefs decided to allow all 5 of these 

programs to proceed anyway. Four of the programs—Global Hawk Block 

20/30, Standard Missile 6, Joint Tactical Radio System HMS Rifleman 

Radio and Joint Tactical Radio System HMS Manpack—demonstrated 

poor performance in operational testing, in areas such as reliability, 

effectiveness, or suitability. Operational testing results for the MQ-1C 

Gray Eagle have not yet been reported.

On the other hand, a few service officials we met with were reluctant to 

accept some recommendations made by OSD offices because they 

believed the recommendations were overly burdensome and could 

Systemic Changes Will Be 

Difficult Until Cultural 

Barriers Are Addressed
 

Page 28 GAO-13-103 Weapons Acquisition Reform

significantly impact weapon acquisition programs’ cost and schedule 

outcomes without a lot of benefit. This was the case for the KC-46 Tanker 

program, where program officials were concerned that additional testing 

recommended by the developmental test and evaluation, and operational 

testing offices, as part of the integrated test team, could have significant 

contractual implications during development. In this case, officials 

identified additional flight test opportunities without having to renegotiate 

the fixed price contract. However, the additional allotted test time is not 

equivalent to the 6 to 8 months the developmental testing office felt 

should be added. A similar situation occurred on the Ship to Shore 

Connector program. The Navy disagreed with a DT&E office 

recommendation to conduct full system testing prior to procuring 

additional craft during initial production. The DT&E office believed the 

program was high risk because the Ship to Shore Connector was a 

complete redesign of a previous system with no reuse of any major 

component (engines, gearboxes, hydraulics, command and control 

software). Navy officials, however, believe the program is low risk since it 

is an evolutionary program and has one critical technology, a fire 

suppression system, which has already been sufficiently demonstrated 

and qualified through test and evaluation. In addition, the Navy estimated 

that it would cost $15 million to revise the existing production schedule to 

accommodate the full system testing as recommended by the DT&E 

office. The DT&E office and the Navy reached a compromise whereby 

OSD would review available system test results before more craft are 

authorized.

Current fiscal pressures, along with the threat of more to come, have 

DOD officials looking for ways to increase buying power by controlling 

cost and schedule overruns on weapon acquisition programs. The offices 

established as a result of the Reform Act, as well as policy provisions 

have helped DOD make inroads towards putting weapon acquisition 

programs on more solid footing. Together, the offices and policy 

provisions place more attention on requirements, costs, testing, and 

reliability as early as Milestone A. The provisions of the act, when 

specifically focused on newer programs, are having a positive impact on 

the programs and the acquisition process. They show that expert 

attention to the cost and achievability of capability requirements, the 

assumptions made for cost and funding of programs, and the amount of 

systems engineering knowledge that is brought to bear early make 

programs more executable.

Conclusions 
 

Page 29 GAO-13-103 Weapons Acquisition Reform

Although senior officials we spoke with throughout the department are 

receptive to the broad principles of the Reform Act, it is too early to tell if 

the Reform Act is going to result in systemic change to DOD’s weapon 

acquisition process. DOD faces several challenges that must be 

addressed to get lasting change—organizational capability constraints, 

the need for additional cost estimating and implementation guidance, the 

possibility of insufficient systems engineering and developmental testing 

resources, limited dissemination of lessons learned, and cultural barriers 

between OSD and the services. Some challenges appear to be straight 

forward to address, such as providing guidance for estimating operating 

and support costs, providing additional guidance for conducting 

preliminary design reviews and competitive prototyping activities, and 

disseminating lessons learned to the broader acquisition community. 

However, they may require more resources, which have been difficult to 

obtain.

For Reform Act policies and practices to have a systemic effect across 

the entire portfolio of weapon system acquisition programs, the

department must also address challenges related to systems engineering 

and developmental testing resources and cultural barriers between OSD 

and the services. This begins with the services identifying key leaders at 

the headquarters level and within program offices to guide systems 

engineering and developmental testing efforts and then ensuring that 

there are enough trained staff to carry out these activities. OSD will need 

to continue monitoring the services’ efforts. It will also require an 

environment where the services stop proposing new weapon systems 

with inflexible requirements, immature technologies, and cost, schedule, 

and funding assumptions that are too optimistic at the start of a program. 

Breaking down cultural resistance to change will take more cooperation 

between the Under Secretary for Acquisition, Technology and Logistics 

and other OSD offices, and service acquisition executives to address, as 

well as continuity of leadership. Efforts by the PARCA office to identify 

factors that correlate to good or poor acquisition outcomes, particularly as 

it relates to program manager tenure, will be beneficial. The services’ 

ability to demonstrate that the Reform Act is influencing all weapon 

acquisition programs, not just the biggest, will be a key indicator for 

determining whether the Reform Act has had a positive effect on DOD’s 

culture.
 

Page 30 GAO-13-103 Weapons Acquisition Reform

We recommend that the Secretary of Defense take the following four 

actions to enable systemic change across the entire portfolio of weapon 

acquisition programs:

• direct the Director of Cost Assessment and Program Evaluation to 

issue guidance for estimating weapon acquisition program costs at 

Milestone A and operating and support costs throughout the 

acquisition life cycle by the end of fiscal year 2013 and ensure that the 

office prioritizes its resources accordingly to accomplish this task; 

• designate responsibility for providing advice and guidance to program 

offices on competitive prototyping and preliminary design reviews to 

the appropriate organization within the Office of the Under Secretary 

of Defense for Acquisition, Technology and Logistics and ensure that 

the guidance is developed. The office(s) designated would be the 

focal point for addressing program office issues related to the practical 

implementation of these Reform Act provisions, such as the type of 

competitive prototyping to use, the timing and benefits of holding 

preliminary design reviews prior to milestone B, and if a preliminary 

design review should be held after milestone B; 

• direct the Deputy Assistant Secretaries of Defense for Systems 

Engineering and Developmental Test and Evaluation to assess and 

include in their annual report to the Congress beginning with the 

report on fiscal year 2012 activities:

• the extent to which the office can perform their required activities 

with allocated resources;

• the impact budget cuts are having on the military services total 

workforce (civilians, military, and contractors) and ability to meet 

program office needs; and

• progress the services have made filling leadership positions, such 

as chief engineers at the service level and technical leads for 

systems engineering and developmental testing at the program 

office level;

• direct the Director of Performance Assessments and Root Cause 

Analyses to make lessons learned collected during its root cause 

analysis evaluations available to the acquisition workforce and ensure 

that the office prioritizes its resources accordingly.

DOD provided us written comments on a draft of this report. DOD 

concurred with two recommendations and partially concurred with two 

others. DOD’s comments appear in appendix III. DOD also provided 

technical comments, which we incorporated as appropriate in the report.

Recommendations for 

Executive Action

Agency Comments 

and Our Evaluation
 

Page 31 GAO-13-103 Weapons Acquisition Reform

DOD agreed with the intent of our first recommendation, but noted that 

due to resource constraints, the Cost Assessment and Program 

Evaluation office could not guarantee that it would be able to issue 

guidance for estimating major defense acquisition program costs at

Milestone A and operating and support costs throughout the acquisition 

lifecycle by the end of fiscal year 2013. We continue to believe that the 

Cost Assessment and Program Evaluation office should issue the 

guidance by the end of fiscal year 2013. However, if that is not possible 

from a resource standpoint, the office should commit to a date and devote 

the resources to meeting that date. We will continue to monitor DOD’s 

efforts to develop the guidance. 

Although DOD concurred with our second recommendation, we revised 

this recommendation based upon discussions with DOD officials during 

the agency comment period. Our revision clarified the intent of this 

recommendation, which is to have the Secretary of Defense designate a 

specific organization within the Office of the Under Secretary of Defense 

for Acquisition, Technology and Logistics to provide advice and guidance 

on competitive prototyping and preliminary design reviews. We 

understand that the department has issued Reform Act implementation 

guidance and has incorporated aspects of competitive prototyping and 

preliminary design reviews in the Defense Acquisition Guidebook. 

Further, we recognize that program offices we visited are taking steps to 

implement the guidance that has already been issued. However, based 

on our discussions with senior level officials, we believe one or more 

offices need to be designated with the responsibility of developing 

additional guidance and answering program specific questions related to 

the practical implementation of the requirements. As noted earlier in our 

report, some officials questioned when to use prototyping or what type of 

prototyping should be used. In addition, there were questions about the 

timing of the preliminary design reviews.

DOD partially concurred with our third recommendation. DOD noted that 

the type of information we recommended be assessed and reported on 

should be included as part of DOD’s human capital strategic planning 

process and as such, be reported in DOD’s annual Acquisition Workforce 

Strategic Plan. We agree that the impact of budget cuts on the workforce 

and the status of leadership positions could be addressed in the annual 

strategic plan. However, we continue to believe that the Deputy Assistant 

Secretaries for Systems Engineering and Developmental Test and 

Evaluation should include an assessment in their joint annual report to the 

Congress on the respective offices’ ability to perform activities specified in 

the Reform Act with available resources.
 

Page 32 GAO-13-103 Weapons Acquisition Reform

DOD concurred with our fourth recommendation, which would make 

lessons learned from root cause analyses available to the acquisition 

workforce. 

We are sending copies of this report to the Secretary of Defense and 

appropriate Congressional Committees. In addition, the report is available 

at no charge on the GAO website at http://www.gao.gov. 

If you have any questions about this report or need additional information, 

please contact me at (202) 512-4841 or sullivanm@gao.gov. Contact 

points for our Office of Congressional Relations and Public Affairs may be 

found on the last page of this report. Key contributors to this report were 

Cheryl Andrew, Assistant Director; Laura Greifner, Julie Hadley, Megan 

Porter, Rae Ann Sapp, and Marie Ahearn.

Michael J. Sullivan

Director

Acquisition and Sourcing Management
Appendix I: Objectives, Scope, and 

Methodology

Page 33 GAO-13-103 Weapons Acquisition Reform

This report examines DOD’s continued implementation of the Weapon 

Systems Acquisition Reform Act of 2009 (Reform Act). Specifically, we 

examined (1) DOD’s progress in implementing Reform Act provisions; 

(2) the impact the Reform Act has had on specific acquisition programs; 

and (3) challenges remaining in improving the weapons acquisition 

process.

To assess DOD’s progress in implementing Reform Act provisions, we 

interviewed officials and analyzed documents, such as reports to the 

Congress and guidance issued from the Office of the Secretary of 

Defense (OSD) offices of the (1) Deputy Assistant Secretary of Defense 

for Systems Engineering (SE), (2) Deputy Assistant Secretary of Defense 

for Developmental Test and Evaluation (DT&E), (3) Cost Assessment and 

Program Evaluation (CAPE), and (4) Performance Assessments and 

Root Cause Analyses (PARCA) to determine the extent to which 

provisions have been implemented. We focused our review on the offices’ 

implementation of four fundamental Reform Act provisions: developing 

policy and guidance; approving acquisition documents; monitoring 

programs and conducting program assessments; and developing 

performance measures. In cases where provisions had not been 

implemented, we asked officials about the reasons for the delay and the 

expected time frame for completion. We also interviewed officials and 

analyzed documents from the office of the Assistant Secretary of Defense 

for Research and Engineering and the Defense Procurement and 

Acquisition Policy office, as well as four weapon acquisition programs, 

which had not yet started development to determine the progress DOD 

has made implementing Reform Act provisions related to preliminary 

design reviews, competitive prototyping, and competition. We believe 

these programs offer the best glimpse at how the OSD offices and 

Reform Act policies are influencing acquisition strategies. The weapon 

acquisition programs we chose for this analysis were part of a larger case 

study review that is described below.

To determine the impact the Reform Act has had on specific weapon 

acquisition programs, we selected 11 weapon system programs to use as 

case studies. For each program we reviewed relevant program 

documentation such as the test and evaluation master plans, 

assessments of operational test readiness, systems engineering plans, 

program support reviews, root cause analyses, analysis of alternatives 

reports and cost estimates as applicable. We also interviewed appropriate 

program officials and officials from the OSD offices for SE, DT&E, and 

CAPE to obtain their perspectives about (1) the level of interaction 

between the programs and OSD offices; (2) changes made to program 

Appendix I: Objectives, Scope, and 

Methodology 
Appendix I: Objectives, Scope, and 

Methodology

Page 34 GAO-13-103 Weapons Acquisition Reform

acquisition strategies as a result of interactions with the OSD offices; and 

(3) benefits and challenges with implementing Reform Act provisions on 

each of the programs. We also reviewed the performance assessments 

and root cause analyses office’s root cause analysis documentation for 

programs that incurred Nunn-McCurdy cost or schedule breaches.

We selected our case studies based on input from the officials in the OSD 

offices for SE, DT&E, CAPE, PARCA, and operational test and 

evaluation. We also discussed possible case studies with GAO 

employees who monitor and report on weapon acquisition programs on 

an annual basis. The programs we selected for review represent a variety 

of platforms, including sea vessels, manned and unmanned aircraft, and 

land systems. Specifically, we examined 11 programs at various stages of 

the acquisition process. Four programs had not yet passed Milestone B, 

development start, at the time we began our review. The remaining seven 

programs had completed their Milestone B review and were in 

development at the time of our case study selection. Of the seven 

programs, three have breached Nunn-McCurdy cost thresholds since the 

act was passed and have had to satisfy the act’s new requirements with 

regards to certification. The other programs had significant interaction 

with one or more of the OSD offices established by the Reform Act. A 

complete list of programs is provided below.

Table 4: List of Programs Selected for Case Study Review

Programs before Milestone B Service Platform 

Ground Combat Vehicle Army Ground

Joint Light Tactical Vehiclea Army/ Marines Ground

Ohio Class Replacement Navy Sea 

Ship to Shore Connectora Navy Sea 

Programs after Milestone B 

Global Hawk Air Force Air 

Gray Eagle Army Air 

Joint Strike Fighter Joint Air 

KC-46 Tanker Air Force Air 

Littoral Combat Ship Seaframe Navy Sea 

Remote Minehunting System Navy Sea 

Small Diameter Bomb Increment II Air Force Air 

Source: GAO analysis.

a

During the course of our review, the Joint Light Tactical Vehicle and Ship to Shore Connector 

programs held a Milestone B.
Appendix I: Objectives, Scope, and 

Methodology

Page 35 GAO-13-103 Weapons Acquisition Reform

While our sample of 11 case studies allowed us to learn about the impact 

the Reform Act offices have had on DOD acquisitions, it was designed to 

provide anecdotal information, not findings that would be representative 

of all the department’s weapon acquisition programs.

To determine challenges remaining in improving defense acquisitions we 

relied on information we collected and analyzed during our case study 

review of 11 weapon acquisition programs. We also solicited the opinions 

of the Under Secretary of Defense for Acquisition, Technology and 

Logistics; other senior level officials in the Office of the Secretary of 

Defense including the leaders of each of the offices created as a result of 

the Reform Act; as well as the military services’ Senior Acquisition 

Executives.

We conducted this performance audit from January 2012 to December 

2012 in accordance with generally accepted government auditing 

standards. Those standards require that we plan and perform the audit to 

obtain sufficient, appropriate evidence to provide a reasonable basis for 

our findings and conclusions based on our audit objectives. We believe 

that the evidence obtained provides a reasonable basis for our findings 

and conclusions based on our audit objectives.
Appendix II: Progress of Reform Act Offices in 

Implementing Weapon System Acquisition 

Reform Act Provisions

Page 36 GAO-13-103 Weapons Acquisition Reform

Table 5: Implementation of Select Reform Act Provisions- Systems Engineering

Reform Act Provisions Status 

Develop policy and guidance Complete: Issued a reliability, availability, and maintainability Directive-Type 

Memorandum, a development planning Directive-Type Memorandum, and a DOD 

Instruction for DASD (Systems Engineering), participated in JCIDS revisions, developed 

guidance for incorporating systems engineering into development contracts, streamlined 

the Systems Engineering Plan and Program Protection Plan, and released an update to 

the Defense Acquisition Guidebook, chapter 4.a Continue to refine policies and 

guidance as necessary. 

Approve documents Completing on annual basis: Approved 52 Systems Engineering Plans since 2009, 

including 15 in fiscal year 2011. 

Monitor programs / conduct assessments Completing on annual basis: Review portfolio of 234 programs. In fiscal year 2011, 

participated in 73 overarching integrated product team meetings, 6 peer reviews of 

acquisition contracts, and conducted 15 Program Support Reviews. 

Develop performance measures Complete: Developed a set of time-based metrics to assess each program’s ability to 

execute its system engineering plans and address risks the office had identified in prior 

reviews. The metrics measure program cost, schedule, staffing, reliability, availability 

and maintainability, software, integration, performance and manufacturing, and are to be 

incorporated into each program’s systems engineering plan and evaluated at various 

points in the development process.

Source: GAO analysis of DOD data.

a DOD Directive-Type Memorandum (DTM) 11-003,”Reliability Analysis, Planning, Tracking, and Reporting,” (Mar. 21, 2011); DOD 

DTM 10-017, “Development Planning to Inform Materiel Development Decision Reviews and Support of Analysis of Alternatives,” 

(Sept. 13, 2010, Incorporating Change 2, Dec. 9, 2011); DOD Instruction 5134.16, “Deputy Assistant Secretary of Defense for Systems 

Engineering (DASD(SE)),” (Aug. 19, 2011).

Table 6: Implementation of Select Reform Act Provisions- Developmental Test and Evaluation 

Reform Act Provisions Status 

Develop policy and guidance Complete: Updated guide for incorporating test and evaluation into acquisition contracts. 

Championed updates to DOD Instruction assigning responsibilities and authorities to 

Developmental Test and Evaluation office, which is in the process of being updated.

Updated guidance to include reliability factors in the Test and Evaluation Master Plan. 

Continue to refine policies and guidance as necessary. 

Approve documents Completing on annual basis: Reviewed and approved 186 Test and Evaluation Master 

Plans since 2009, including 44 in fiscal year 2011. 

Monitor programs / conduct assessments Completing on annual basis: Review portfolio of nearly 250 programs. In fiscal year 2011, 

participated in 22 defense acquisition board meetings and 59 overarching integrated 

product team meetings. The office has also conducted 16 Assessment of Operational 

Test Readiness reviews since 2009.

Develop performance measures In process: Piloted performance measures on two programs. The measures were then 

updated and are being applied to over 40 programs that were selected for reporting in the 

fiscal year 2012 joint annual report. The assessments are being used to support the writeup of the program engagement section. 

Source: GAO analysis of DOD data.

Appendix II: Progress of Reform Act Offices 

in Implementing Weapon System Acquisition 

Reform Act Provisions
Appendix II: Progress of Reform Act Offices in 

Implementing Weapon System Acquisition 

Reform Act Provisions

Page 37 GAO-13-103 Weapons Acquisition Reform

Table 7: Implementation of Select Reform Act Provisions- Cost Assessment and Program Evaluation 

Reform Act Provisions Status 

Develop policy and guidance In process: Issued first policy document in May 2012, which is the basis for additional 

policy documents. Updating its Operating and Support Cost Estimating Guidebook, which 

will address the Reform Act requirement for DOD to issue guidance related to full 

consideration of life cycle management and sustainability costs in major defense 

acquisition programs. 

Approve documents Not applicable: The Reform Act does not require that the office approve acquisition 

documents. 

Monitor programs / conduct assessments Completing on annual basis: Conducted independent cost assessments for Milestone A 

and B certification on 30 future and current major defense acquisition programs, since 

2009, including 8 in fiscal year 2011. The office conducted 3 Milestone C and 3 NunnMcCurdy certification reviews in fiscal year 2011. 

Develop performance measures Not applicable: The Reform Act does not require that the office develop performance 

measures.

Source: GAO analysis of DOD data.

Table 8: Implementation of Select Reform Act Provisions- Performance Assessments and Root Cause Analyses 

Reform Act Provisions Status 

Develop policy and guidance In process: Developing guidance to assist offices in conducting root cause analyses. The 

guidance for conducting performance assessments is expected to be released in early 

fiscal year 2013. 

Approve documents Not applicable: The Reform Act does not require that the office approve acquisition 

documents. 

Monitor programs / conduct assessments Completing on annual basis: Completed 14 Root Cause Analyses for programs which 

have undergone a Nunn-McCurdy breach or were requested by OSD and has completed 

26 semi-annual follow-up reports on these programs. Providing OSD with the execution 

status of DOD’s portfolio of acquisition programs through the Defense Acquisition 

Executive Summary process. 

Develop performance measures In process: Utilize Defense Acquisition Executive Summary information to identify cost 

performance, schedule, funding, and technical performance issues on Major Defense 

Acquisition Programs. Continuing to develop performance measures. 

Source: GAO analysis of DOD data.
Appendix III: Comments from the Department 

of Defense

Page 38 GAO-13-103 Weapons Acquisition Reform

Appendix III: Comments from the 

Department of Defense
Appendix III: Comments from the Department

of Defense

Page 39 GAO-13-103 Weapons Acquisition Reform
Appendix III: Comments from the Department 

of Defense

Page 40 GAO-13-103 Weapons Acquisition Reform
Appendix III: Comments from the Department 

of Defense

Page 41 GAO-13-103 Weapons Acquisition Reform (121024)
The Government Accountability Office, the audit, evaluation, and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and 

policies; and provides analyses, recommendations, and other assistance 

to help Congress make informed oversight, policy, and funding decisions. 

GAO’s commitment to good government is reflected in its core values of 

accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents at no 

cost is through GAO’s website (http://www.gao.gov). Each weekday 

afternoon, GAO posts on its website newly released reports, testimony, 

and correspondence. To have GAO e-mail you a list of newly posted 

products, go to http://www.gao.gov and select “E-mail Updates.”

The price of each GAO publication reflects GAO’s actual cost of 

production and distribution and depends on the number of pages in the 

publication and whether the publication is printed in color or black and 

white. Pricing and ordering information is posted on GAO’s website, 

http://www.gao.gov/ordering.htm. 

Place orders by calling (202) 512-6000, toll free (866) 801-7077, or 

TDD (202) 512-2537. 

Orders may be paid for using American Express, Discover Card, 

MasterCard, Visa, check, or money order. Call for additional information.

Connect with GAO on Facebook, Flickr, Twitter, and YouTube. 

Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. 

Visit GAO on the web at www.gao.gov. 

Contact:

Website: http://www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470

Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-

4400, U.S. Government Accountability Office, 441 G Street NW, Room 

7125, Washington, DC 20548

Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, DC 20548

GAO’s Mission

Obtaining Copies of 

GAO Reports and 

Testimony 

Order by Phone

Connect with GAO

To Report Fraud, 

Waste, and Abuse in 

Federal Programs

Congressional 

Relations 

Public Affairs

Please Print on Recycled Paper.