GAO

United States General Accounting Office

Report to the Chairman and Ranking 

Minority Member, Subcommittee on 

Readiness and Management Support, 

Committee on Armed Services,

U.S. Senate

August 1999 BEST PRACTICES

DOD Training Can Do 

More to Help Weapon 

System Programs 

Implement Best 

Practices

GAO/NSIAD-99-206

United States General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-280234 Letter

August 16, 1999

The Honorable James Inhofe

Chairman

The Honorable Charles Robb

Ranking Minority Member

Subcommittee on Readiness and Management Support

Committee on Armed Services

United States Senate

As you requested, this report assesses the extent the Department of Defense’s (DOD) training helps 

weapon system program offices apply best practices and whether such training can be of more help. 

We make recommendations to the Secretary of Defense on how DOD training can better support 

program offices in the application of best practices.

We are sending copies of this report to the Honorable William S. Cohen, Secretary of Defense; the 

Honorable Louis Caldera, Secretary of the Army; the Honorable Richard Danzig, Secretary of the 

Navy; the Honorable F. Whitten Peters, Acting Secretary of the Air Force; the Honorable Jacob J. Lew, 

Director, Office of Management and Budget; and to interested congressional committees. We will also 

make copies available to others upon request.

If you or your staff have any questions regarding this report, please call me at (202) 512-4841. 

Contacts and key contributors to this report are listed in appendix III.

Katherine V. Schinasi

Associate Director

Defense Acquisitions Issues

Letr
Page 2 GAO/NSIAD-99-206 Best Practices

Executive Summary

Purpose The Department of Defense (DOD) plans to increase its annual 

procurement investment to about $60 billion by fiscal year 2001. DOD has 

high expectations from this investment: that new weapons will be better 

yet less expensive than their predecessors and will be developed in half the 

time. Essential to getting these kinds of outcomes will be the adaptation of 

best commercial practices that have enabled leading commercial firms to 

develop new products faster, cheaper, and better. DOD has begun a number 

of acquisition reform initiatives based on commercial practices to help 

foster these outcomes. Their success depends greatly on the extent to 

which the program offices responsible for managing weapon acquisitions 

can implement the practices on individual programs. Training provided to 

the program offices serves as a key agent in both creating a culture that is 

receptive to new practices and in providing the knowledge needed to 

implement new practices at the workplace. At the request of the Chairman 

and the Ranking Minority Member, Subcommittee on Readiness and 

Management Support, Senate Committee on Armed Services, GAO 

evaluated the role DOD training is playing in implementing best practices in 

weapon system programs. This report addresses (1) the contribution DOD 

training makes to program offices’ ability to apply best practices, (2) the 

different methods used by DOD and leading commercial firms in training 

on best practices, and (3) the strategic approaches that underlie DOD’s and 

leading commercial firms’ training methods for best practices. 

Background GAO’s review focused on weapon system program offices because they 

comprise a key component of DOD’s acquisition workforce. In planning, 

managing, and executing acquisition programs, these program offices are 

responsible for managing about $80 billion of DOD’s annual research, 

development, and procurement funds. As an entry point for DOD 

acquisitions, program offices greatly influence the work of the rest of the 

acquisition workforce. The primary responsibility for training the 

acquisition workforce falls within the Office of the Under Secretary of 

Defense for Acquisition and Technology. In 1992, the Defense Acquisition 

University, a consortium of 13 schools, was created to develop and provide 

training for the acquisition workforce. Each service also has an acquisition 

reform office that provides the workforce with the latest information about 

practices and initiatives that apply to acquisitions. Based on personnel 

reductions mandated by the National Defense Authorization Acts for Fiscal 

Years 1996 and 1997, DOD expects the acquisition workforce of fiscal year 

2000 to be 25 percent smaller than that of fiscal year 1995. Thus, training 

Letr
Executive Summary

Page 3 GAO/NSIAD-99-206 Best Practices

will become even more important as new authority and responsibility is 

granted to those who remain in the workforce. 

To determine the extent to which DOD training and other sources helped 

program offices obtain the knowledge needed for implementing best 

practices, GAO focused on five specific practices: cost as an independent 

variable, integrated product teams, evaluation of contractors’ past 

performance, setting performance specifications, and managing supplier 

relationships. The first four are formal DOD initiatives that are based on 

best commercial practices, while supplier relationships is a best practice 

GAO has observed in leading commercial firms. GAO selected six program 

offices considered by DOD as leaders in implementing one or more of the 

practices. As such, they represented best case examples of marshaling 

training and other resources needed to implement new acquisition 

practices. The term “standard training” is used in this report to describe 

the training provided by the Defense Acquisition University and the 

services’ acquisition reform offices, as distinct from training that program 

offices provide on their own.

Results in Brief DOD’s standard training did not make a major contribution to the leading 

program offices’ ability to implement best practices. In evaluating their key 

sources of knowledge, none of the key officials from programs at the 

forefront of implementing best practices ranked standard DOD training 

first, with many ranking it last. DOD training either did not reach the right 

people when it was needed or did not reach them at all. When training on 

best practices was received, it did not contain the depth or practical 

insights program office people needed to implement the practices. It was 

primarily through their own efforts—learning on the job, finding external 

training, or developing their own training program—that they attained the 

knowledge needed to apply best practices. Thus, success depended on 

their having the foresight to see what was needed, the ingenuity to find 

good sources of knowledge, and the resources needed to attain that 

knowledge. Replicating this approach broadly on other programs is 

problematic. Managers may not see the relevance of a practice to their 

programs and thus may not realize what training is needed. Others may not 

be able to afford the needed training.

Leading commercial firms and DOD use different training methods to 

implement key practices. Commercial firms use targeted, hands-on 

methods to ensure that program offices are trained on key practices. Their 

training organizations conduct front-end analyses to determine the 

Letr
Executive Summary

Page 4 GAO/NSIAD-99-206 Best Practices

programs’ training requirements and involve the program offices in 

designing the training. Training is customized to meet the specific needs of 

those implementing the practice. Company officials believe the targeted 

method results in more useful training, which helps to improve outcomes 

of the final product. DOD does not have a counterpart to this method. 

DOD relies primarily on its standard training, including classroom courses, 

videos, internet-based training, satellite broadcasts, and roadshows, to 

inform staff on best practices. These methods were designed for functional 

training, such as for engineers, and for increasing the awareness of new 

practices. As such, they do not provide the depth or reach enough of the 

right people at the right time to be of great help in implementing best 

practices at program offices. Also, DOD does not systematically involve 

program offices in the design of training. 

The intensive training methods leading commercial firms employ on new 

practices are the result of a strategic, institutionally driven approach to 

implementation. These firms commit their resources and attention to a few 

well-defined practices and make a significant front-end investment in the 

training to be provided to the workforce. Also, the firms strive to create an 

environment to put those responsible for implementing the practices in a 

good position to succeed. DOD’s training methods for best practices do not 

stem from such a strategic approach. DOD has promulgated as many as 40 

acquisition management initiatives in the past few years without 

communicating their relative priority to trainers or implementers. Often, 

the initiatives have not been accompanied by clear guidance or by the 

initial training needed for implementation. While DOD commits significant 

resources to training, it does not make a uniform front-end investment to 

ensure that program offices will succeed with the new practices. Since 

1997, two studies commissioned by DOD have recommended ways to make 

training organizations more effective in providing training of best practices. 

These recommendations were not adopted in favor of a more traditional 

training role. In June 1999, another DOD study proposed that DOD training 

organizations become change agents and be modeled after their corporate 

counterparts. If the latest study’s recommendations are adopted, DOD may 

be in a much better position to provide the type of help program offices 

need to successfully implement best practices. 

GAO makes recommendations to the Secretary of Defense on how DOD’s 

training on best practices can better support the needs of program offices.
Executive Summary

Page 5 GAO/NSIAD-99-206 Best Practices

Principal Findings

DOD Training Not a Major 

Catalyst for Best Practices 

Training often did not exist or was not provided when program officials 

began to implement an individual practice. For example, when the Joint 

Strike Fighter program office started to implement cost as an independent 

variable, there were no guiding documents, and no one, including the 

training community, knew what the initiative really meant. Some people 

involved with implementing best practices were missed altogether by DOD 

training offerings because they had not been required to take training since 

before the initiatives began. Others, such as requirements authors, fell 

outside the definition—and training curriculum—of the acquisition 

workforce. In one case, program officials reduced contractor reporting 

requirements to 2 items, in line with acquisition reform, only to have 40 

more added by another office with approval authority over the contract. 

Contractors are also essential to the application of best practices, but they 

are not part of the DOD defined acquisition workforce or the training 

offered. Consequently, they do not necessarily understand or know how to 

implement new practices. Program officials noted that DOD’s standard 

training typically provided a general awareness of the practices but not the 

“how-to” knowledge needed for implementation. Training was not tailored 

in such a way that individuals could see how the practices could be applied 

to their program. Program officials also observed that in some cases 

training suffered because instructors did not use up-to-date case studies 

and were not current on new practices themselves. They also believed they 

had little opportunity to influence the training they received from DOD. 

Program officials used a combination of ways—generally outside of 

standard DOD training offerings—to get the knowledge they needed to 

apply best practices. Several program officials relied on the cumulative job 

expertise of the staff and personal research to teach themselves how to 

implement new initiatives. For example, on the Advanced Medium-Range 

Air-to-Air Missile program, officials relied on their personal experiences to 

know how to set contract specifications at a performance, rather than a 

detailed, level. Officials also used their program funds to send staff to 

nongovernment sources or to bring experts in. One program manager 

sends people to outside training, such as university leadership courses, to 

develop creative thinking skills. One of his managers said some of his best 

training was from off-site sessions sponsored by the program office that 

dealt with the people issues critical to making integrated product teams 

work. For the Advanced Amphibious Assault Vehicle program, the prime 
Executive Summary

Page 6 GAO/NSIAD-99-206 Best Practices

contractor was responsible for providing the training on integrated product 

teams and cost as an independent variable. The contractor hired a third 

party to develop a training program tailored to the Advanced Amphibious 

Assault Vehicle program, and both program office and contractor staff 

were taught together, on-site as a team. 

DOD’s Training Methods for 

Best Practices Do Not Go as 

Far as Leading Commercial 

Firms’

For routine training, such as skill building, leading commercial firms have 

standard training offerings, including functional area courses and 

instruction on corporatewide issues, such as communications or ethics. 

However, when implementing key practices—such as those that change 

how product development and production are conducted—leading firms go 

beyond standard training offerings. Commercial firms use a targeted, 

hands-on approach to ensure program teams are put in a good position to 

implement a new practice. The elements common to how leading firms 

provide training on a key practice include (1) front-end analysis of program 

teams’ needs and training requirements, (2) involvement of program teams 

in key training decisions, (3) customized training to meet program team 

specific needs, and (4) targeted training for the implementation of specific 

practices. Program staff participate in and often influence a wide range of 

training decisions, including the amount of training provided for certain job 

descriptions, course topics, depth of course coverage, and identification of 

the appropriate course recipients. The involvement of the program staff 

has improved course depth, timeliness, and coverage of personnel in the 

commercial firms.

In the Boeing Company, training representatives develop a partnership with 

the program staff when a team is formulated to design and manufacture a 

new airplane. The training organization forms “drop teams” to colocate 

with the program staff to learn as much as possible about the business 

process and the staff’s concerns and to determine what training is needed 

to help the program staff implement a practice. Boeing officials stated that 

training was instrumental to the implementation of key practices, such as 

design build teams, on the 777 aircraft program. They noted that such 

teams were at odds with the company’s culture because employees were 

not accustomed to working in a team environment and sharing information 

across functional areas. Training officials worked side by side with the 

program staff to create a training program that provided team building and 

conflict resolution techniques as well as new technical skills training. To 

ensure all program staff were equally trained, employees were required to 

complete training before they reported for duty on the program. The 

professional employees—engineers and drafters—were required to 
Executive Summary

Page 7 GAO/NSIAD-99-206 Best Practices

complete 120 hours of start-up training on several key 777 practices before 

they were allowed to report for duty. Teams were often trained together at 

the work location. Ford followed a similar approach when it implemented 

the Ford Product Development System—a lean engineering process.

Responsibility for training on best practices is diffused among several DOD 

organizations, including the Defense Acquisition University and the service 

acquisition reform offices. However, GAO did not find an organization that 

was able to tailor and help deliver training on best practices to the program 

offices visited. Training provided by the university is designed primarily to 

enable people in individual career fields or functions, such as engineering 

and cost estimating, to meet professional certification requirements. The 

university incorporates best practices topics into these functionally 

oriented courses as drop-in modules that provide a survey of the topic, but 

not in-depth coverage. Although program offices see a greater need for 

training that cuts across functions to implement new practices and to 

manage in a team environment, it is difficult for a person in one career field 

to obtain training in another field. The usefulness of these courses for best 

practices is further hampered by limited availability; according to an 

official from the university, the member schools get about 10 percent of the 

workforce into training each year. 

DOD’s Acquisition Reform Communication Center and the acquisition 

reform offices in the services communicate best practice information 

through videos, periodic satellite broadcasts, roadshows, and Acquisition 

Reform Week. These methods can reach more people than Defense 

Acquisition University courses and are designed around practices—versus 

functions—but are not tailored to specific program offices and are not 

necessarily delivered at the time those implementing new practices or 

initiatives need them. For example, roadshows, traveling multiday training 

workshops provided to staff at a number of locations, typically provide 

awareness training on the practices. DOD officials estimate that only 10 to 

15 percent of the acquisition workforce attend the second day of 

workshops, where more detailed training is provided. The annual 

Acquisition Reform Weeks, which are a combination of satellite broadcasts 

and local presentations, mainly provide awareness-level training. Neither 

individual attendance nor the level of learning attained by attendees is 

tracked by these methods.
Executive Summary

Page 8 GAO/NSIAD-99-206 Best Practices

Differences Between DOD 

and Commercial Training 

Reflect Different Strategic 

Approaches

Leading commercial firms shared a common strategy for adopting and 

deploying key new practices. First, the firms’ corporate management 

committed to and adopted few key practices—seven or less—at any given 

time. In doing so, the companies were able to concentrate their attention 

and target resources to implementing the practices. It also signaled the 

importance of the practices to trainers and implementers. Second, the 

firms assessed which staff should be included in the implementation. For 

example, Ford’s training unit determined which engineering teams working 

on vehicle lines could benefit from the new production development 

system. Third, company leaders made implementing the practice 

mandatory for the target population. Lastly, companies developed 

well-defined learning objectives to better ensure that the target population 

consistently understood how to apply a new practice to improve 

production outcomes—the ultimate goal. According to officials from these 

firms, it was a corporate responsibility to ensure that those implementing 

the practice received the necessary training and other assistance to 

succeed. It was for this reason that the companies made a significant 

front-end investment to support the needs of program offices that would 

implement key practices. Company officials also pointed out that training 

is just one of the several components necessary for adopting new practices. 

They stressed that providing strong leadership and the right environment 

were key to ensuring the implementation of new practices and to 

developing quality training.

DOD’s approach to implementing best practices is less structured and more 

reliant on individual program offices to make the necessary training 

investment. DOD policies on individual reform initiatives are typically 

promulgated without indicating what components of the acquisition 

workforce or which programs are expected to implement the practice. In 

addition, the policies themselves are not always clear. For example, 

although the initiative on cost as an independent variable was promulgated 

in 1995, Advanced Amphibious Assault Vehicle program officials 

developed training in 1997 to define the initiative for the program. In 

November 1998, an Air Force workshop on the cost initiative reported that 

it was still not well understood or widely implemented. DOD has proposed 

over 40 reform initiatives since 1994, without an indication of relative 

priority, leaving educators and implementers to decide what is important. 

Program offices are not necessarily in a good position to sort through the 

initiatives to focus on those that are the most important to the job at hand. 

A service acquisition reform official observed that the combination of many 

reform initiatives and unclear priorities causes the office to guess what is 
Executive Summary

Page 9 GAO/NSIAD-99-206 Best Practices

the most important, which leads to emphasizing what is perceived to be 

popular. 

DOD is aware of the need to improve the means by which the acquisition 

workforce receives and implements new initiatives. A 1997 study by a DOD 

team and a 1998 study by the Logistics Management Institute pointed out 

several weaknesses in the focus and delivery of DOD training. Weaknesses 

included the need for the Defense Acquisition University to be more active 

in implementing best practices and reforms, the tendency for the training 

curriculum to be functionally stove-piped, and the need for the university 

to have more involvement with the workforce—the recipients of training. 

The studies made recommendations for significant organizational and 

operational changes in the university, including that it should follow the 

corporate university model of becoming a change agent and a proponent of 

best practices and put more emphasis on cross-functional training. In 

September 1998, the university proposed a revised structure, which has not 

been approved. While the proposal offers some improvements, it stays 

close to its roots of providing functional training. It does not reflect the 

corporate university model, a broadened role as a change agent, or a closer 

relationship with the acquisition workforce. In short, it does not 

discernibly address key weaknesses in providing training of best practices. 

In June 1999, a DOD study team chartered to identify training on 

commercial business practices for the acquisition workforce concluded 

that adopting the most effective commercial practices requires a cultural 

and organizational transformation within DOD. The team proposed a 

cross-functional plan for managing acquisitions that embraces best 

practices and calls for “learning organizations that seek out and adopt best 

practices that improve individual and organizational performance.” The 

plan proposes new roles for several organizations in fostering the adoption 

of best practices. Among these is the Defense Acquisition University. For 

example, it recommends that the university be broadly recast to adopt the 

corporate university model and become a change agent. This plan, while 

not specific about the help that program offices would receive, does call for 

a strategic approach that would make it more likely that DOD could 

provide its program offices tailored training—more help—in implementing 

best practices. However, the fact that the September 1998 and the June 

1999 proposals are still vying for approval indicates that DOD has not yet 

decided what role it wants acquisition training to play on best practices.
Executive Summary

Page 10 GAO/NSIAD-99-206 Best Practices

Recommendations GAO makes several recommendations to the Secretary of Defense that are 

intended to ensure that DOD’s approach to the training of new practices 

better supports the needs of program offices by (1) developing a strategy 

for a structured approach to training on new practices; (2) providing 

tailored training assistance to program offices; and (3) improving the 

standard training curriculum so that it is more timely, relevant, and 

accessible. These recommendations appear in full in chapter 5.

Agency Comments DOD concurred with the views expressed in the report and all of the 

recommendations. A discussion of DOD’s actions appears in chapter 5. 

DOD’s comments appear in appendix I.
Executive Summary

Page 11 GAO/NSIAD-99-206 Best Practices
Page 12 GAO/NSIAD-99-206 Best Practices

Contents

Letter 1

Executive Summary 2

Chapter 1

Introduction

Responsibilities for Training the Acquisition Workforce Within

DOD 16

Objectives, Scope, and Methodology 21

Chapter 2

DOD Training Is Not a 

Major Catalyst for Best 

Practices

DOD Training Did Not Reach the Right People at the Right Time 26

DOD Training Offerings Did Not Provide the Depth Needed to

Implement Best Practices 31

Limitations of Standard Training Led Program Offices to Develop

Their Own Training Solutions 34

Chapter 3

DOD Training Methods 

for Best Practices Do 

Not Go as Far as 

Leading Commercial 

Firms

Commercial Firms Use Targeted, Hands-on Methods to Improve

Training Usefulness 39

DOD Does Not Target Training on Key Practices to Program Offices 45

Chapter 4

DOD and Commercial 

Training Methods 

Reflect Different 

Strategic Approaches 

to New Practices

Leading Firms’ Strategic Approach Better Ensures Implementation

of Key Practices 50

DOD’s Training on Best Practices Does Not Stem From a Strategic

Approach 52

DOD Proposals to Alter Training Reflect Conflicting Approaches 54

Chapter 5

Conclusions and 

Recommendations

Conclusions 59

Recommendations 60

Agency Comments and Our Evaluation 62
Contents

Page 13 GAO/NSIAD-99-206 Best Practices

Appendixes Appendix I: Comments From the Department of Defense 64

Appendix II: Description of Program Offices Visited 67

Appendix III: GAO Contacts and Staff Acknowledgments 69

Related GAO Products 72

Tables Table 1.1: The Defense Acquisition and Technology Workforce

as of April 1999 18

Table 1.2: Best Practices Evaluated at Program Offices Visited 23

Figures Figure 1.1: Organizations Responsible for Training 

Development 20

Figure 2.1: JSF 28

Figure 2.2: JSTARS-JTT 30

Figure 2.3: AMRAAM 32

Figure 2.4: AAAV 35

Figure 2.5: JASSM 37

Figure 3.1: 777 Airplane 42

Figure 3.2: Ford Focus 43
Contents

Page 14 GAO/NSIAD-99-206 Best Practices

Abbreviations

AAAV Advanced Amphibious Assault Vehicle

AMRAAM Advanced Medium-Range Air-to-Air Missile

ARCC Acquisition Reform Communications Center

BCIS Battlefield Combat Identification System

CAIV cost as an independent variable

DAU Defense Acquisition University 

DAWIA Defense Acquisition Workforce Improvement Act

DOD Department of Defense

FPDS Ford Production Development System 

IBM International Business Machines

IPT integrated product teams

JASSM Joint Air-to-Surface Standoff Missile 

JSF Joint Strike Fighter

JSTARS-JTT Joint Surveillance Target Attack Radar System-Joint Tactical 

Terminal
Contents

Page 15 GAO/NSIAD-99-206 Best Practices
Page 16 GAO/NSIAD-99-206 Best Practices

Chapter 1

Introduction Chapte1r

The Department of Defense (DOD) plans to increase its annual 

procurement investment to $60 billion by fiscal year 2001. DOD has high 

expectations from this investment: that new weapons will be better and 

less expensive than their predecessors and will be developed in half the 

time. Essential to getting these kinds of outcomes will be the adaptation of 

best commercial practices that have enabled leading commercial firms to 

develop new products faster, cheaper, and better. To help foster these 

outcomes, DOD has begun a number of acquisition reform initiatives based 

on commercial practices. Success depends greatly on the extent to which 

the program offices responsible for managing weapon acquisitions can 

implement the practices on individual programs. 

The training DOD provides program office staff to help them implement 

best practices should play a central role in getting the desired outcomes. 

While first-hand experience and “learning by doing” are instrumental in 

adopting new practices, training serves as a key agent in creating a culture 

that is receptive to new practices and providing the knowledge to 

implement the new practices at the workplace. The relationship between 

training and implementing new practices was highlighted in a 1994 study of 

300 “improvement-driven” organizations conducted by Coopers & 

Lybrand—with the American Society of Quality Control, Rutgers University 

Center for Public Productivity, and the National Institute of Canada. The 

study found that training played a “critical, integrative role as driver of 

cultural change, process alignment, job redesign, and continuous 

improvement.” Organizations included in the study used training as the 

vehicle for implementing and sustaining the changes at the level where the 

work was done. The leading commercial firms we reviewed committed 

substantial investments to the training on key practices, underscoring its 

importance to getting the outcomes desired by the firms. The significant 

numbers of reform initiatives that DOD has introduced, which involve 

adopting a number of new practices in the acquisition of weapons, present 

implementation challenges that training can help meet.

Responsibilities for 

Training the 

Acquisition Workforce 

Within DOD

For nearly 50 years, the importance of an educated professional DOD 

acquisition workforce has been emphasized by government leaders and 

reflected in the work of key studies and reform commissions. The First 

and Second Hoover Commissions (1949 and 1955), the Fitzhugh 

Commission (1970), and the Commission on Government Procurement 

(1972) all recognized the importance of high quality, well-educated 

acquisition professionals to the successful operation of DOD. The Packard 

Commission, which undertook a broad examination of DOD management 
Chapter 1

Introduction

Page 17 GAO/NSIAD-99-206 Best Practices

practices and procedures, reported in its June 1986 report that the DOD 

acquisition workforce was undertrained and inexperienced. One of its 

recommendations was to improve the education and training of the 

acquisition workforce for the purpose of enhancing the defense acquisition 

process. 

Based in part by the recommendations of the Packard Commission, the 

Defense Acquisition Workforce Improvement Act (DAWIA) was passed in 

1990 as part of the National Defense Authorization Act for Fiscal Year 

1991.1

 Its primary objective was to improve the DOD acquisition system by 

enhancing the education, training, and career development of members of 

the acquisition workforce. Accordingly, DAWIA established the Defense 

Acquisition University (DAU) to provide for the professional educational 

development and training of the DOD acquisition workforce. The act also 

charged DOD officials with the responsibility to designate certain positions 

as acquisition positions, to set qualification requirements, and to establish 

policies and procedures for training the acquisition workforce.

The program offices that manage weapon system acquisitions are a key 

component of DOD’s acquisition workforce. This workforce is generally 

defined as those people who are responsible for managing the wide array of 

DOD acquisitions, including contracting professionals, program managers, 

engineers, scientists, logisticians, and other occupational fields, from the 

earliest phases of basic research to the logistical support and disposal of 

old systems. However, there have been several definitions of what 

comprises the DOD acquisition workforce, which have led to varying 

estimates of the workforce size, ranging from about 106,000 to 355,000 

people. At the end of fiscal year 1997, DOD estimated the acquisition 

workforce covered by DAWIA included approximately 106,000 positions, of 

which about 90,000 were civilian. Its most recent definition places the 

workforce, now called the acquisition and technology workforce, at about 

150,000 people2

 and includes people from science and technology 

organizations. Table 1.1 lists the different acquisition and technology 

workforce occupations and the number in each occupation based on this 

definition.

1

 P. L. 101-510, Nov. 5, 1990.

2

 This definition was based on the study, “Identification of the Department of Defense Key Acquisition 

and Technology Workforce,” April 1999, Jefferson Solutions, Washington, D. C.
Chapter 1

Introduction

Page 18 GAO/NSIAD-99-206 Best Practices

Table 1.1: The Defense Acquisition and Technology Workforce as of April 1999

Source: DOD.

Definitions aside, the DOD acquisition workforce has been undergoing 

restructuring and downsizing. The National Defense Authorization Acts for 

Fiscal Years 1996 and 1997 mandated reductions in the number of civilian 

and military employees in acquisition organizations. DOD estimates that, 

as a result, the acquisition workforce will be 25 percent smaller at the end 

of fiscal year 2000 compared to fiscal year 1995. DOD understands that 

effective training will become even more important as the workforce is 

reduced and new authority and responsibility are granted to those who 

remain. We have previously reported that decisions to restructure or 

reduce this workforce should be linked to getting better outcomes from the 

acquisition process; doing otherwise would miss an opportunity to address 

the deep-seated causes of acquisition problems.3

Acquisition and technology workforce occupations Total persons

Engineers (electronics, aerospace, mechanical, civil, and general) 44,117

Contracting 19,387

Management 15,509

Business and industry 12,989

Communications and computers 9,370

Administration and programs 5,116

Scientists 4,476

Auditing 3,692

Financial management 3,618

Procurement assistants 2,650

Mathematics and statistics 2,400

Purchasing 2,158

Supply program management 1,753

Inventory management 944

Equipment specialists 858

General supply 326

Miscellaneous 3,698

Military 16,378

Total 149,439

3

 Defense Acquisition Organizations: Linking Workforce Reductions With Better Program Outcomes

(GAO/T-NSIAD-97-140, Apr. 8, 1997).
Chapter 1

Introduction

Page 19 GAO/NSIAD-99-206 Best Practices

Although weapon system program offices comprise a subset of this 

workforce, they are a highly leveraged subset. In planning, managing, and 

executing acquisition programs, program management offices are 

responsible for about $80 billion of DOD’s annual research, development, 

and procurement funds—about 30 percent of the fiscal year 1999 DOD 

budget. They influence much of the work of the rest of the acquisition 

workforce because they are bringing new equipment into the inventory, 

which must be managed, budgeted for, maintained, and supplied. In a 

sense, they are a point of entry for DOD acquisitions. In addition, these 

offices are made up of a cross section of people that draw from most of the 

acquisition and technology workforce occupations cited in table 1.1. As 

such, they are a medium in which the training of new initiatives in different 

career fields converge.

The primary responsibility for training the acquisition workforce in 

general—and new initiatives in particular—falls within the Office of the 

Under Secretary of Defense for Acquisition and Technology. That office is 

responsible for setting the standards that the workforce must meet. DAU, 

which is responsible for designing and conducting the training to meet 

those standards, also reports to that office. These and other organizations 

responsible for setting training standards and providing training are shown 

in figure 1.1.
Chapter 1

Introduction

Page 20 GAO/NSIAD-99-206 Best Practices

Figure 1.1: Organizations Responsible for Training Development

The Under Secretary of Defense for Acquisition and Technology delegated 

responsibility for developing career paths and establishing educational 

standards to the Office of the Director of Acquisition Education, Training, 

and Career Development. That office is supported by four functional 

boards that have established experience, education, and training standards 

for the acquisition workforce and for professional certification levels in 

each of the acquisition career fields. A DOD manual, DOD 5000.52M, 

“Acquisition Career Development Program,” implements and prescribes 

procedures for career development of the acquisition workforce. The 

Under Secretary of Defense for 

Acquisition and Technology

Director for Acquisition Education, 

Training, and Career Development Defense Acquisition University

Functional Boards

• Acquisition management

• Contracting

• Technical management (e.g.,

 engineering and testing

•Business, cost estimating, and

 financial management

Consortium of Schools

• Defense Systems Management

 College

• Industrial College of the Armed

 Forces

• Service Schools
Chapter 1

Introduction

Page 21 GAO/NSIAD-99-206 Best Practices

manual establishes experience, education, and training standards for 3 

certification levels in each of the 11 acquisition career fields. Level I is 

basic or entry level, level II is the intermediate or journeyman level, and 

level III represents the advanced or senior level. The specific training to 

increase competency or attain higher certification levels within a career 

field is often referred to as functional training. There are also standards for 

specific acquisition workforce positions (such as program managers), 

position categories, and membership in the acquisition corps. 

DAU was created by DAWIA to develop the training curriculum to meet 

these standards and to coordinate the efforts of its consortium of 13 

DOD-wide and service schools that conduct the training courses. The 

Defense Systems Management College, a DOD-wide school, is dedicated to 

providing acquisition-related training. Other member schools, including 

those run by individual military services, provide a variety of training, of 

which acquisition training is only a part. While the service schools provide 

information on new initiatives, they do not have primary responsibility for 

educating the acquisition workforce. Each service also has an acquisition 

reform office that helps make people aware of the latest practices and 

initiatives that apply to acquisitions, although this office does not play a 

significant role in designing or providing the training. The Acquisition 

Reform Communications Center is an organization related to DAU that has 

the mission of sharing knowledge about acquisition reform by providing 

and disseminating information on how DOD is changing the way it acquires 

goods and services. 

Objectives, Scope, and 

Methodology

The Chairman and the Ranking Minority Member, Senate Committee on 

Armed Services, Subcommittee on Readiness and Management Support, 

asked us to review how DOD is training its acquisition workforce to 

implement best practices for acquiring weapon systems. The 

Subcommittee’s request is part of a broader interest in seeing that best 

practices are incorporated into DOD’s acquisition process as a way of 

saving money for modernization, increasing efficiency, and improving 

quality. The objectives of this report are to assess (1) the contribution DOD 

training makes to program offices that are applying best practices, (2) the 

different training methods DOD and leading commercial firms use in 

providing training on practices, and (3) the different strategic approaches 

that underlie the training methods DOD and leading commercial firms use 

in implementing practices. 
Chapter 1

Introduction

Page 22 GAO/NSIAD-99-206 Best Practices

To meet these objectives, we focused on five specific practices and 

identified program offices recognized as being leaders in applying them. 

We chose the five practices based on discussions with DOD and on our 

previous work in the application of best practices to weapon acquisitions. 

The first four are formal DOD initiatives that are based on best commercial 

practices, while supplier relationships is a best practice we observed in 

leading firms and in some leading weapon system programs.4

 The selected 

practices are the following:

• Cost as an independent variable (CAIV): An acquisition management 

practice in which aggressive life-cycle cost goals are achieved through 

trade-offs between requirements and performance.

• Integrated product teams (IPT): Teams composed of members from 

functional disciplines such as engineering, test, and contract 

management. All members contribute their particular expertise to team 

decisions and to resolve issues.

• Performance specifications: States requirements in terms of required 

results without stating the methods to achieve those results. They 

define the functional requirement for operation, interface, and 

interchange characteristics, and have criteria for verifying performance 

compliance.

• Past performance: Information on a contractor’s past performance on 

relevant prior work is used as a factor in source selection. The 

information is used to evaluate risk and the potential for future 

contractor success.

• Supplier relationships: A commercial practice in which maximum 

participation of suppliers and their suppliers is encouraged to promote 

product excellence. The best suppliers are selected and supported in a 

number of ways that ensure a mutually beneficial partnership. 

We focused on weapon system program offices because of the significant 

role they play in implementing best practices. Initially, we considered 

gathering information from a cross section of program offices that had a 

range of experience in implementing best practices. However, we found 

that a program office that had little or no experience with a new practice 

was not in a good position to comment on the training needed to implement 

the practice. Consequently, we selected program offices that DOD 

considered to be leaders in one or more of the five practices. 

4

 See Best Practices: DOD Can Help Suppliers Contribute More to Weapon System Programs

(GAO/NSIAD-98-87, Mar. 1998).
Chapter 1

Introduction

Page 23 GAO/NSIAD-99-206 Best Practices

We based our selections on extensive consultation with DOD officials to 

ensure that the program offices had experience in implementing the 

practices and thus were in a good position to comment on the training 

resources that enabled them to implement the practices. (See app. II for a 

description of the programs.) These program offices represented best case 

examples in DOD for specific practices, although none of the programs was 

considered a leader in all five practices. The program offices selected were 

the Advanced Amphibious Assault Vehicle (AAAV), Advanced 

Medium-Range Air-to-Air Missile (AMRAAM), Battlefield Combat 

Identification System (BCIS), Joint Air-to-Surface Standoff Missile 

(JASSM), Joint Strike Fighter (JSF) and Joint Surveillance Target Attack 

Radar System-Joint Tactical Terminal (JSTARS-JTT). Table 1.2 shows 

which of the program offices were considered leaders for the different 

practices.

Table 1.2: Best Practices Evaluated at Program Offices

Source: GAO analysis of DOD information.

At the program offices, we used structured questions to interview the key 

people responsible for implementing an individual practice. Through the 

interviews, we determined the various sources they used to develop the 

knowledge needed to implement best practices and the extent to which the 

training DOD provided to the program office contributed to this 

knowledge. 

To assess strategy and the methods DOD uses to train program office staff 

on the use of best practices, we concentrated on the DAU training 

organizations and resources established by DAWIA to provide for the 

professional educational development and training of the DOD-wide 

acquisition workforce. We also assessed how these resources met program 

office needs in implementing best practices. In addition, we assessed other 

CAIV IPT

Performance

specifications

Past

performance

Supplier

relationships

AAAV XX X

AMRAAM X X

BCIS X X

JASSM X XX

JSF X X

JSTARS-JTT X X
Chapter 1

Introduction

Page 24 GAO/NSIAD-99-206 Best Practices

acquisition training sources, including service and agency specific schools 

and the services’ acquisition reform offices. However, our report focuses 

on the training approach for best practices and is not intended to describe 

all DOD training practices. We met with the Director of Acquisition, 

Education, Training, and Career Development; the President of the DAU; 

officials of the Defense Systems Management College; and representatives 

from the Office of the Under Secretary of Defense for Acquisition and 

Technology. We also met with and collected data from officials at the Air 

Force, the Army, and the Navy. We reviewed the organizational structure 

and responsibilities for DAU consortium members and service 

organizations, the process for establishing the DAU training curriculum, 

and survey data on DOD training recipients. 

We evaluated the strategies and methods of commercial firms recognized 

for their training excellence by examining how these leading companies 

used training to implement new practices at their program management 

organizations. To identify these companies, we conducted literature 

searches, consulted with and collected data from professional associations, 

and spoke with university faculty specializing in corporate organizations. 

We visited the following four companies recognized as being leaders in the 

area of training:

• The Boeing Company is the largest manufacturer of commercial 

jetliners and military aircraft with 234,000 employees worldwide. 

Boeing’s Employee Training and Development organization is 

responsible for training all Commercial Airplane Group employees, 

approximately 97,000. There are 500 to 600 employees in the office, with 

roughly 300 serving as trainers.

• Ford Motor Company is one of the largest U.S. manufacturers of 

automobiles, trucks and provider of automotive services with 345,000 

employees worldwide. Ford’s Product Development Process 

Leadership organization was created to provide assistance, including 

training, to engineers implementing the Ford Production Development 

System (FPDS). 

• International Business Machines (IBM) Corporation is one of the 

world’s top providers of computer hardware and software with 290,000 

employees worldwide. Different internal organizations provide training 

that serves employees worldwide, including Learning Services and the 

Center for Excellence. 

• Motorola is one of the world’s leading providers of wireless 

communications, semiconductors, advanced electronic systems, and 

services with over 150,000 employees worldwide. Motorola University 
Chapter 1

Introduction

Page 25 GAO/NSIAD-99-206 Best Practices

has a staff of over 600 operating through account managers that are 

assigned to each business unit to provide a “one-face” education and 

training contact.

These companies, recognized as industry leaders, place strong emphasis on 

training. At these companies, we reviewed company documents and 

training data and met with individuals responsible for designing and 

developing programs to educate and train employees on major new 

practices. We also met with representatives from major program offices 

who were involved in key training decisions and were recipients of the 

training. Our report highlights the best commercial training approaches for 

implementing key new practices. As such, they are not intended to 

describe all commercial training practices or suggest that commercial firms 

are without flaws. 

Finally, we reviewed several studies on DOD’s training organizations and 

methods. We used these, as well as the previous information and analysis, 

to determine the extent to which DOD’s proposals to reshape DAU and 

continuous learning policy held potential for better delivery of training to 

foster implementation of best practices by program offices.

We conducted our review between April 1998 and June 1999 in accordance 

with generally accepted government auditing standards.
Page 26 GAO/NSIAD-99-206 Best Practices

Chapter 2

DOD Training Is Not a Major Catalyst for Best 

Practices Chapte2r

Key officials from weapon system programs at the forefront of 

implementing best practices did not find that standard DOD training 

offerings provided the information they needed to apply the practices to 

their programs. In evaluating their key sources of knowledge for 

implementing best practices, none of the program officials ranked required 

DOD training first, with many ranking it last. DOD training either did not 

reach the right people when it was needed or did not reach them at all. 

When training on best practices was received, it did not contain the depth 

or practical insights program office staff needed to implement the 

practices. It was primarily through their own efforts—learning on the job, 

finding external training, or developing their own training program—that 

they attained the knowledge needed to apply best practices. 

Programs that became leaders in applying best practices did so primarily 

because their managers realized that the practices were key to the 

programs’ success. In so doing, the managers were able to identify what 

knowledge they needed to apply the practices. Thus, their success 

depended on having the foresight to see what was needed, the ingenuity to 

find good sources of knowledge, including training, and the resources 

needed to attain that knowledge. Replicating this approach broadly on 

other programs is problematic. Other managers may not realize the 

significance of a practice to the success of their programs and the need for 

additional training. Also, they may be uncertain about testing new 

initiatives on their programs. Some may recognize a practice’s importance 

but be unable to fund their own training efforts and be left relying on 

standard DOD training.

DOD Training Did Not 

Reach the Right People 

at the Right Time

For training to facilitate the adoption of a new practice, it must be received 

by those responsible when they are ready to begin implementing the 

practice. Program officials stressed that on a new initiative or practice, 

training must begin when a practice is to be introduced. If training is 

provided too late, people will be forced to devise their own means of 

getting the knowledge needed to begin implementation or risk improper 

implementation. If training is provided too soon, knowledge could fade 

before it is applied at the workplace. For the programs we reviewed, those 

responsible for managing or implementing best practices noted that in 

some cases, training did not exist when they began implementing the 

practices; in other cases, the training was received too late for the job at 

hand. Some were missed altogether by DOD training offerings because 

they were too senior to be required to take courses, training was not 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 27 GAO/NSIAD-99-206 Best Practices

provided locally, or they fell outside the definition—and training 

curriculum—of the acquisition workforce.

Training Came at the Wrong 

Time for Some People 

Program office officials believed it is essential that training on new 

initiatives accompany implementation to explain what the initiatives are 

and to come up with a common understanding and way to tackle 

program-specific issues. For the program offices leading the way in best 

practices, officials reported relevant DOD training was not offered at the 

time implementation began. For example, the BCIS program applied 

performance-based specifications soon after the 1994 DOD initiative was 

announced. However, specific training on performance specifications had 

not been developed. The AAAV program had already implemented 

performance specifications without DOD provided training. A top manager 

for JSF similarly reported that there was no training for program office 

staff when they started to implement CAIV. There were no guiding 

documents and no one, including the training community, knew what CAIV 

was. Similarly, on the JSTARS-JTT program, CAIV training was not 

available when the program office began applying the practice in 1995. 

Now, some training is available, but it was not when it was needed for the 

program. Nor was training available when the JASSM program office 

began assessing contractor past performance. Training could have helped 

avoid a protest of the prime contract award, according to an official. 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 28 GAO/NSIAD-99-206 Best Practices

Figure 2.1: JSF

Implementation of CAIV on the JSF program began with a clean slate—no advance training.

Source: DOD, artist rendition.

Training can also come too early. A program official questioned the staff’s 

ability to retain the information when training is not provided at a practical 

time for the assignment. For example, he noted that symposiums are good 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 29 GAO/NSIAD-99-206 Best Practices

ideas, but people may not have an opportunity to apply the ideas at the time 

and may not remember when they need to.

Key People Were Missed by 

Training on Best Practices

Applying best practices on a weapon system program involves reaching not 

only program office staff but also other members of the acquisition 

workforce, DOD people outside the acquisition workforce, and 

contractors. Experience on the programs we reviewed shows that it is 

hard to reach all of these people with best practices information through 

standard DOD training.

The promotion of many best practices took place after the implementation 

of DAWIA course requirements so that senior people have not taken formal 

acquisition training that includes exposure to best practices. Several of the 

senior program officials we spoke with did not receive training on new 

practices because they had not taken courses that incorporated best 

practices. These officials had been acquisition officials for many years 

before DAWIA and were grandfathered into their certification level. Their 

job experiences were applied to meet professional certification 

requirements and thus they did not have to take any DAWIA training. For 

example, an official who helped establish JSF performance specifications 

noted that he had his last class 3 years earlier, a contracting refresher 

course. The course may have had a short section on performance 

specifications, but he could not remember it.

Training misses some program office people because it is not offered 

locally. According to program officials, training needs to be conducted at 

the local level by subject matter when it is needed. This is especially true 

for small programs, for which it is difficult to spare people from the 

programs for long periods. One official from a small program said that 

although senior management stresses the importance of training, it is 

difficult to attend classes that run over 14 weeks. He added that these 

classes have not been offered locally. Another program official asked why 

DOD training organizations do not bring some of the harder to get courses 

to the field, as is done by private organizations.
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 30 GAO/NSIAD-99-206 Best Practices

Figure 2.2: JSTARS-JTT

JSTARS-JTT officials believe training could be improved by being provided on-site.

Source: DOD.

Training is not reaching people outside program offices that also play key 

roles in successfully implementing best practices. For example, program 

offices are typically supplemented with people from separate functional 

organizations, such as engineering directorates. These people may not 

receive training on new initiatives in their home organizations and can be 

unfamiliar with the initiatives when they come to a program. People that 

set requirements early in the process and those in logistics must also be 

knowledgeable and committed for practices to be successful. Exercising 

flexibility in requirements, for instance, is critical to the success of CAIV 

and performance specifications. The need for training was extended 

further to those that play a role in a weapon system’s approval. Officials 

from one program reported that they had worked with a prime contractor 

to streamline contract reporting requirements down to four items, in line 

with acquisition reform. However, based on a review by a separate office 

with approval authority over the contract, 40 contract requirements were 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 31 GAO/NSIAD-99-206 Best Practices

added, returning the contract to traditional reporting methods. The 

program official said that the reviewing official did not know what the 

program office was trying to accomplish and did not care.

Prime contractors and subcontractors are also essential to the application 

of best practices but are not part of the DOD defined acquisition workforce 

or otherwise targeted for training on best practices. Several program 

officials we met with believed that some means of informing contractors is 

needed to make them aware of important DOD initiatives. Contractors do 

not necessarily understand or know how to implement new DOD 

initiatives; for example, they may not know how an integrating IPT or 

overarching IPT works. While DOD is not responsible for training 

contractors, it does have to ensure that contractors understand best 

practices, as well as give them an opportunity to help shape how these 

practices are applied on programs. The AAAV program management 

ensured their prime contractor was knowledgeable of best practices by 

requiring training for contractor and program office staff in the prime 

contract.

DOD Training 

Offerings Did Not 

Provide the Depth 

Needed to Implement 

Best Practices

Program officials reported that standard DOD training did not prepare 

them well for implementing the practices at the workplace. DOD’s training 

typically provided only an awareness of the practices, not the knowledge 

that is needed for actual implementation. Also, the training was not 

tailored to allow program offices to see how individual practices applied to 

their specific programs. Further, program officials noted that trainers did 

not have the practical experience to share and were not up to date with the 

most recent examples of programs that had implementation experience.

Program officials stated that they need to go beyond the theoretical 

concepts covered in most training courses. They believed that the “how to” 

is missing on all the initiatives and that they need to know how to move 

from traditional practices to the new practices. DOD courses were 

described in general as too esoteric and not relevant to the tasks at hand. 

For example, a JASSM official noted that a contractor’s past performance is 

often equated with DOD’s Contractor Performance Assessment Rating 

System, but it involves much more. The JASSM program office evaluated 

contractors’ past products, and in doing so, learned about the quality of 

their design, management, and production processes. Program officials 

said they went well beyond the performance assessment rating system 

covered in standard training courses. Nonetheless, they said that they 

could not avoid a protest by the losing contractor. They believed they 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 32 GAO/NSIAD-99-206 Best Practices

needed guidance on how to collect past performance information, which 

was not covered by training.

Program officials believed, overall, that training should be designed more 

for the customer because acquisitions are unique and that programs may 

have different implementation issues based on program size, stage of the 

acquisition, or type of weapon. A senior AAAV official said general courses 

can provide information about individual practices, but to implement a 

practice, the training must be tailored to the program. JSF and BCIS 

officials concurred. Some program officials have observed that training 

courses overemphasize the application of new practices for larger and 

newer programs and believe that not enough coverage is given to teaching 

how the new practices apply to programs that are smaller or older. For 

example, officials noted that newer programs have an advantage in starting 

with a clean slate and that training should also show how the practices 

apply to older programs and the benefits to be obtained. 

Figure 2.3: AMRAAM

AMRAAM officials believe training should help program managers in deciding how initiatives apply to 

their particular circumstances.

Source: DOD.
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 33 GAO/NSIAD-99-206 Best Practices

Program officials also found the presentation methods for training on best 

practices were not helpful in applying the practices in the workplace. They 

believed that instructors lacked practical experience and current 

information. For example, a JSF manager reported that the executive 

course he took in 1997 on CAIV was not valuable because the instructors 

had no experience with CAIV and did not know how to explain it. Another 

program official thought that instructors were left in place too long and had 

only old experiences to share. Officials noted that course material was 

frequently out of date and sometimes incorrect. For example, one JASSM 

program official attended an engineering course in which the instructor’s 

answer on CAIV was wrong. Another said the acquisition reform example 

the instructor used—from the official’s own program—was about 2 years 

behind what was being applied on the program. The official brought up 

more current examples of practices being used on the program, but the 

instructor did not want to discuss them. 

Training also suffered from limited use of case studies that would allow 

students to see how an initiative in the abstract might apply to their own 

programs. Officials from several programs added that current case 

histories should show the application of acquisition principles in a program 

context. Case studies would include the successes, as well as the pitfalls 

and solutions. The case studies should be designed for the customer, 

another said, and be applied at multiple levels. For example, programs 

could benefit if training allowed students to play off the risks and benefits 

of how new initiatives apply to their particular program. Real case 

experiences help others visualize how practices could apply. BCIS officials 

reported that during the presentation of their case study for the 1998 

Acquisition Reform Week, they realized, as they explained what they did to 

implement initiatives, that the audience gained better insight on the issues. 

Another official said he shared his program’s experiences with those in his 

class, but those in other classes would not have the benefit of his 

experiences. 

Program officials did not believe that there was an effective means for 

providing feedback on the quality and usefulness of courses, such as the 

need to update course materials, or course relevance. They would like to 

see the students—the practitioners—have more impact on what 

adjustments are needed to courses. The only means of feedback program 

office people were aware of was the end of class survey form. However, 

these surveys did not allow students to give in-depth feedback or ask the 

questions that got to the larger issue of whether students’ training needs 

were met.
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 34 GAO/NSIAD-99-206 Best Practices

Limitations of 

Standard Training Led 

Program Offices to 

Develop Their Own 

Training Solutions

In evaluating their key sources of knowledge for implementing best 

practices, none of the program officials ranked required DOD training first, 

with many ranking it last. To obtain the knowledge needed to implement 

key practices, officials in leading programs developed their own solutions. 

They had a vision of what they wanted to accomplish and they devised a 

variety of methods, such as funding training beyond standard DOD 

offerings, creating their own internal training programs, and learning on the 

job. Strong program managers, supported by the executives above them 

and strong working relationships with their contractors, were key to the 

implementation of the practices. Program managers of the leading 

programs cautioned, however, that not all officials have the vision or the 

resources to mirror this approach. 

Implementation Success 

Tied to Vision and Support 

at All Levels

All the leading programs had at least one element in common—strong 

leadership committed to implementing practices that would help their 

programs succeed. Leaders were described as having vision and knowing 

what they had to do to realize that vision. This included accepting the risks 

for trying new approaches. For example, the original AAAV program 

manager conceived of the unique approach of collocating the program 

office and the contractor because he thought it was essential to making 

IPTs work. IPTs became the backbone of the AAAV program. 

Consequently, team training was viewed as so important that it was 

incorporated into the prime contract as a requirement. Support for 

collocation and the team approach came from top management of both the 

contractor and the program office. According to program officials, it does 

not make any difference how good the training is without management 

support; junior people can come back from training with new ideas and 

have them go nowhere if they do not get management support.
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 35 GAO/NSIAD-99-206 Best Practices

Figure 2.4: AAAV

Training for AAAV was tailored to support contractor and program office collocation.

Source: DOD.

In turn, managers point to the support that they receive from their senior 

management as an important factor in their ability to be successful. One 

official described his manager as aggressive in his efforts to pave the way 

for trying new practices. To do that, the senior manager said that he is 

willing to go up against the established bureaucracy and provide cover for 

his program managers to try new things. One program manager cited a 

personal commitment to a reform-minded DOD official as part of his 

motivation for making acquisition reform work. He said that with his 

bosses’ support, he can do what he thinks is right on his program, such as 

making past performance central in selecting a prime contractor. However, 

he noted that other managers have not had that ability because they are not 

supported when they propose doing things differently. 

Program managers believe that a key element for adopting best practices 

was developing trust or partnership with the contractor. An AMRAAM 

manager believes the ability to make radical program changes has 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 36 GAO/NSIAD-99-206 Best Practices

depended on working closely with the prime contractor, and seeing that the 

same relationship exists between the prime contractor and subcontractors. 

The JASSM office organized its IPTs to mirror the structure of the 

contractor’s IPTs, rather than along functional lines like most offices. A top 

JASSM official credited the high level of trust between the program office 

and the contractor, along with the program office’s creativity and 

innovation, for enabling their use of new practices. This perspective is 

essential to managing programs in today’s environment and to make teams 

work, another official noted, but it is not taught in standard DOD training 

offerings.

More managers would attempt to apply best practices if training 

encouraged it, a program official said, particularly the training provided for 

junior program managers. More general training is needed to support a 

new program environment for adopting new practices that goes beyond the 

current training emphasis on the mechanics of management and theory. 

Program officials reported that new program manager training as recent as 

1997 did not reflect the new approach they have practiced on their 

programs. One believed that it was a lost opportunity for DOD to not 

impress upon new managers their role in adopting new practices and the 

potential benefits. 

Leading Program Officials 

Innovated to Get Needed 

Training

With the support of senior management, program officials used a mixture 

of strategies—generally outside of standard DOD training offerings—to get 

the knowledge they needed to apply best practices. A number of programs 

used their own resources, including cumulative job expertise of the staff 

and personal research to learn how to implement new initiatives. For 

example, JSF officials stated they learned how to apply CAIV by organizing 

warfighters, engineers, and analysts together in a team. They learned as 

they went, developing materials on their own. The AMRAAM program had 

a young staff member gather information on CAIV from various sources 

such as the internet and conferences. From that starting point, they 

learned as they went along. In setting performance specifications, it was 

the personal experience of AMRAAM officials, coupled with commitment, 

that allowed the specifications to be set at a performance, rather than 

detailed, level. Despite their own experiences, program officials did not 

recommend the learn-as-you-go approach; everyone should not have to 

reinvent the wheel.

Programs officials used their own program funds to go beyond standard 

DOD training by sending staff to nongovernment training or to bring 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 37 GAO/NSIAD-99-206 Best Practices

experts in. For example, frustrated with DOD training, the JASSM program 

manager sends people to outside training, such as Harvard leadership 

courses, to have them learn and grow beyond the basic training and 

develop creative and innovative thinking. He has his staff take 80 hours of 

general, nonfunctional training. Another manager said some of his best 

training was from off-site sessions sponsored by JASSM that dealt with 

people issues that were critical to making IPTs work. JSF officials took a 

private sector course on performance-based specifications and used a 

model developed by their contractor. BCIS officials hired an outside firm 

to teach program officials on two occasions, both of which included a CAIV 

component.

Figure 2.5: JASSM

JASSM officials sponsor training to develop staff leadership skills.

Source: DOD. 

AAAV officials developed their own training program tailored to the 

program’s work environment and training needs. A key characteristic of 

the program is the collocation of government and contractor program 
Chapter 2

DOD Training Is Not a Major Catalyst for 

Best Practices

Page 38 GAO/NSIAD-99-206 Best Practices

officials, making team dynamics an important factor. The prime contract 

specified that teams must be used, and IPT training was charged to the 

contract. The contractor hired a third party to develop a training program 

tailored to the AAAV program, and program office, contractor, and 

subcontractor staff were taught together, on-site as a team. Joint training 

was used to establish a common culture for participants. Training was 

given one time to the team initially and then to every new person. The team 

training started with mandatory 10 hours of IPT training, with sessions 

covering the AAAV program, trust, communications, and other team 

dynamics. CAIV training was added as part of the training for 

implementation within the team context. 

The team approach to training was supported by other leading programs as 

well. On the JASSM program, the manager reported that he just had his 

entire office (about 30 staff) take 1 week of leadership training, which 

included topics such as stress management, critical thinking, and 

decision-making. JSF officials took the “train the trainer” approach and 

developed a team of experts within JSF and then the experts trained 

everyone that would be involved in implementing performance 

specifications. The experts used multiple sources to train themselves, such 

as published guidance and talking with their peers. They then developed 

basic guidance for the staff, such as engineers, who needed a common 

understanding on how to write requirements at a performance level. 

Other Programs May Lack 

Leading Programs’ Ability to 

Innovate

Program officials reported that they were fortunate to have staff that could 

use their collective experiences to work through problems in implementing 

initiatives. However, they noted that not all programs will have the same 

advantages. JASSM officials said top service officials handpicked program 

officials for the program team because their personal characteristics 

supported flexibility and creativity. Similarly, JSF officials reported that 

the program office was staffed with multifaceted people, as well as strong, 

senior management support and upfront money for training. They noted 

that smaller programs may be staffed by junior officials with less 

experience to draw upon or be unable to devote staff to research 

information on how practices might apply to their situation. Limited 

funding may also be an issue for some programs. As one program official 

said, his program office could pay for external training, but for many 

programs, the cost would be an impediment since training is one of the first 

items cut in a program budget. Consequently, smaller programs might need 

to rely more heavily on standard DOD training as their main source of 

information.
Page 39 GAO/NSIAD-99-206 Best Practices

Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms Chapte3r

Leading commercial firms and DOD apply different training methods to 

implement key practices. Commercial firms use targeted, hands-on 

methods that include conducting a front-end analysis to determine the 

teams’ training requirements and regularly involve those implementing the 

practices in making important training decisions to ensure program teams 

are trained on key practices. Thus, the training is customized to meet the 

specific needs of the teams. These methods also involve providing many 

hours of training—beyond standard skill-based or functional training—

focused on the implementation of a single practice. Company officials 

believe the targeted approach results in more useful training—improving 

the personnel coverage, course depth, and timing—which helps to improve 

outcomes of the final product. 

In contrast, DOD training on best practices is delivered through traditional 

DAWIA certification courses and vehicles such as videos and 

computer-based training, which are limited in reaching the right people at 

the right time and in providing the needed depth to implement best 

practices. DOD does not have a counterpart to the commercial method of 

providing customized, hands-on assistance to support program office staff 

and other implementers of key practices. Although exceptions exist, there 

is no systematic effort—or responsible organizations—within DOD to 

directly assist key implementers to use new practices. Further, DOD does 

not have a comprehensive means for allowing program staff and others to 

influence training decisions in a way that could improve the relevance of 

training.

Commercial Firms Use 

Targeted, Hands-on 

Methods to Improve 

Training Usefulness

For routine training, such as skill-building, leading commercial firms 

provide standard training offerings, including functional area courses and 

instruction on corporatewide issues, such as communications or ethics. 

However, when implementing key new practices—such as those that 

change product development and production—the firms go beyond the 

standard training offerings. Commercial firms use a targeted, hands-on 

training approach to ensure program teams are in a good position to 

implement a new practice. They provide numerous hours of training, 

typically through a single company organization, targeted to the 

implementation of a key practice. The practice-specific training hours are 

targeted to the program teams most likely to implement the new practice. 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 40 GAO/NSIAD-99-206 Best Practices

The elements common to how the four leading firms provide training on a 

key initiative or practice include

• a front-end analysis of program teams’ needs and training requirements;

• involvement of program teams in key training decisions;

• customized training to meet program teams’ specific needs;

• targeted training for the implementation of specific practices; and

• improved training outcomes, including better course depth, timeliness, 

and reach.

The training organizations of leading commercial firms conduct a front-end 

analysis to determine the needs and training requirements of program 

offices implementing new practices. The analysis is also used to identify 

and address barriers each program office faces when implementing new 

practices. According to the Director of the Benchmarking Forum for the 

American Society for Training and Development, this type of analysis is 

crucial for an organization to be able to institute performance-improving 

measures. Using information from the front-end analysis, the training 

organizations customize the training to ensure that it directly assists 

program teams in implementing new practices. One company official told 

us that training is costly and when it misses the mark, the company pays a 

big price. Given the importance of training when implementing a key 

practice, company officials believe that it is crucial to ensure that the 

training is beneficial to the key implementers of the practice. To ensure that 

the training will address the needs of the program teams, the training 

organizations involve the staff in making important training decisions. 

Program staff help decide the amount of training to be provided for certain 

job positions, course objectives, and depth of course coverage. Company 

officials believe their training approach, which includes program staff, has 

resulted in the right amount of course depth, timeliness, and coverage of 

personnel in the commercial firms. Following are descriptions of the 

training methods employed by companies on key initiatives or new 

practices. 

The Boeing Company Officials from Boeing’s Employee Training and Development organization 

state that their primary goal is to support their customers—employees 

assigned to the Commercial Airplane Group. The training representatives 

develop a partnership with the staff from the beginning of the program to 

design and manufacture a new airplane. The training representatives form 

“drop teams” to collocate with the program to conduct a front-end analysis 

and learn as much as possible about the business process and the staff’s 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 41 GAO/NSIAD-99-206 Best Practices

concerns. The analysis allows the drop team to determine what training is 

needed to support the staff implementing new practices. 

Boeing training officials said they worked side by side with the program 

staff to create a training program that provided team building and conflict 

resolution techniques and technical skills training that specifically focused 

on improving work competencies that would change as a result of the 777’s 

new digital environment. To ensure all 777 staff were equally trained, 

employees were required to complete training before they reported to the 

program. For example, the professional employees—engineers and 

drafters—were required to complete 120 hours of start-up training on 

several key 777 practices, including design build teams and computer-aided 

three-dimensional interactive application1

 software. Teams were often 

trained together at the work location. Boeing officials stated that training 

was instrumental to the implementation of key practices on the 777 

program, such as design build teams—essentially IPTs. The officials stated 

that design build teams were at odds with the company’s culture because 

employees were not accustomed to working in a team environment and 

sharing information across functional areas. 

1

 This application is a computer-based design tool that allows designers the opportunity to view design 

drawings and the interface of the millions of airplane parts as three dimensional.
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 42 GAO/NSIAD-99-206 Best Practices

Figure 3.1: 777 Airplane

Boeing’s use of design build teams created a major culture change for the staff assigned to the 777 

program.

Source: The Boeing Company.

Boeing officials believe their partnership approach improved their training. 

Training representatives stated that the partnership resulted in program 

management support, which ultimately led to acceptance from the program 

staff. The representative stated that since Boeing has involved program 

staff in decisions regarding training, course “no-show” rates have 

decreased. A senior manager for program operations for the 777 program 

stated that because he and other senior program leaders drove key training 

decisions, the training was tailored to the staff’s needs and provided the 

necessary skills and orientation to work in the new environment.
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 43 GAO/NSIAD-99-206 Best Practices

Ford Motor Company The Ford Motor Company created an organization, Product Development 

Process Leadership, with the singular purpose of supporting its 100 

program offices in designing new or modifying vehicle lines in the 

implementation of the FPDS--a lean engineering process. The organization 

provides internal communication regarding FPDS to the teams, and its 

training representatives work with the teams to conduct an analysis to 

learn first hand if impediments to FPDS implementation exist. Other 

support, such as team coaching, is provided to facilitate the engineering 

team’s implementation of FPDS in the workplace. 

Figure 3.2: Ford Focus 

Ford uses training to improve the timeliness and quality of new vehicle launches.

Source: Ford Motor Company.

Ford officials told us that their training focus is to provide practical skills 

just in time—that is, when it coincides with the need to apply the skills on 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 44 GAO/NSIAD-99-206 Best Practices

the job. Ford provides over 50 hours of training to instruct engineers on 

how to do their jobs using the FPDS process. To that end, Ford uses 

internal subject matter experts—engineers who have been on teams 

designing new or modifying existing vehicle lines—to help tailor FPDS 

course topics to make them relevant to the work environment. Ford 

officials stated that the subject matter experts understand the details of the 

FPDS and are needed to ensure that the training developed is practical at 

the working level. According to the manager responsible for training 

employees to launch new vehicles at Ford, his office pulls together the 

training that is necessary to get the job done. To further improve 

practicality and timeliness, the manager stated that on-the-job-training at 

the worksite is used to the extent possible. 

IBM IBM’s Center for Excellence provides in-house education consultants to 

personally understand the business situation and product. The consultants 

interview business unit staff to assess the staff’s training needs and identify 

inhibitors to implementing new practices. The assessments can take

2 weeks to 4 months, depending on the size of the business unit. An IBM 

trainer for Object Orientation Project—a companywide software 

development practice—explained that providing practical training is one of 

the Center’s guiding principles. To achieve this goal, the company first 

stresses the importance of using experienced instructors who have 

practical knowledge in “doing” what they are teaching. Second, employees 

are assigned to an Object Orientation Project before they take the

5- to 7-week immersion training course. According to the IBM trainer, this 

requirement has improved practicality because the students have better 

knowledge retention, as opposed to having to wait 6 months to apply the 

information. Lastly, to further enhance practicality and relevance, IBM 

integrates case studies with real examples related to the student’s next 

assignment into the Object Orientation Project training. According to an 

official from IBM’s Center of Excellence, most training is provided and 

tailored to entire work teams or, at a minimum, to individuals with common 

responsibilities. 

Motorola University Motorola University focuses on providing training and education solutions 

for its business units. The university recently began to assign an account 

management team to consult and advise senior leadership for each 

business sector in order to anticipate and provide appropriate training. 

The management team partners with the business unit staff to identify their 

training needs for implementing new practices, such as Five Nines—an 

effort to improve product reliability to the level of 99.999 percent. The 

account team works with the business units to assess their training needs 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 45 GAO/NSIAD-99-206 Best Practices

and develop a plan to meet those needs. This partnership enables the 

university to customize its training to the specific needs of the various 

types of engineers, such as software engineers, within the business units. A 

software engineer stated that the university provides separate training 

courses tailored to meet the different and often distinct needs within the 

engineering community implementing Five Nines. For example, a software 

engineer could receive up to 120 hours of training targeted to implementing 

Five Nines. Although the account management structure is relatively new, 

the engineer observed that most business unit staff are reacting favorably 

to the effort because the coordinated approach provides one-stop 

shopping.

DOD Does Not Target 

Training on Key 

Practices to Program 

Offices 

DOD does not have a counterpart to the commercial hands-on approach for 

directly assisting key implementers of the best practices. DOD relies 

primarily on its standard training, including DAWIA courses, augmented by 

videos, internet-based training, satellite broadcasts, and roadshows, to 

inform staff on best practices. These venues were designed to focus on 

functional training and to increase the awareness of new practices. As 

such, they do not provide the necessary depth or reach enough of the right 

people at the right time to be of help in implementing best practices at 

program offices. Responsibility for training on best practices is diffused 

among several DOD organizations, including DAU and the service 

acquisition reform offices. We did not find an organization that was able to 

tailor and help deliver training on best practices to the program offices we 

visited. Furthermore, DOD does not systematically involve program office 

staff and other implementers in key decisions regarding best practices 

training. Currently, no feedback mechanism exists to determine the effect 

of DOD’s training on the implementation of new practices at the program 

office level.

DAWIA Training Not Well 

Suited as a Primary Means 

for Conveying Best 

Practices

DAU training is designed primarily for employees seeking career level 

certification in the acquisition workforce, as required by the DAWIA 

standards. DAU incorporates best practice topics into the DAWIA courses 

as drop-in modules that provide a survey of the topic. The information 

conveyed is enough to provide a general awareness of the concept but not 

enough to implement the practice at the workplace. While this approach 

may provide sufficient information for the target audience—newer 

acquisition employees—it has inherent limitations when it comes to 

providing best practices’ implementation training to the entire acquisition 

workforce. For example, DAU courses have been developed by functional 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 46 GAO/NSIAD-99-206 Best Practices

boards that teach skill-based competencies for functional career fields 

versus best practices. As a result, DAU courses are primarily aligned along 

career fields, such as engineering and cost estimating. Because personnel 

assigned to a particular functional area are given priority for training in that 

function, program officials told us that it is difficult for personnel outside 

of their areas to take courses. On one program, an official noted that 

because of limited space, only 5 to 10 percent of the program staff can take 

training outside of their functional areas each year. 

One program manager believed that the functional training, while 

important to career fields, was no longer as relevant to the role of today’s 

program office. He noted that the combination of best practices, 

delegation of key responsibilities to contractors, and fewer staff has altered 

the program office’s role. In his opinion, the program office used to be 

closely involved with managing the design of a weapon system and 

double-checking the prime contractor’s work. In this role, the program 

office was organized and operated along functional lines, and functional 

training was relevant to how a program office operated. He observed that 

today the program office is not as involved with the designing of the 

weapon system, nor is it able to mirror the contractor’s functions. Rather, 

the program office must be expert at what the government can control—

which the program manager referred to as key leverage points. These 

leverage points include the requirement trade-off process, the selection of a 

prime contractor, and the establishment of key relationships that enable 

the program office staff to have insight into the contractor’s actual 

progress. Program office staff, working in an IPT environment, must have 

knowledge of multiple career fields and work in a cross-functional setting. 

As a result, he believes that functional training alone no longer covers the 

things most critical to a program manager. The kind of training that is 

needed—such as on the leverage points—must be obtained elsewhere by 

the program office. 

The usefulness of the DAU courses is further hampered by limited 

availability, which restricts program offices from receiving training when 

needed. According to a DAU school representative, the consortia of 

schools can train about 10 percent of the workforce each year. 

Furthermore, more senior staff may have limited exposure to best 

practices because the majority of them have already met training 

requirements and are not taking the certification courses that introduce 

best practices. While these staff are not prohibited from taking 

certification courses as part of their continuing education requirements, 

availability is limited and priority is given to individuals seeking 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 47 GAO/NSIAD-99-206 Best Practices

certification. These inherent limitations are consistent with the training 

shortfalls noted by program office staff in chapter 2.

Other Training Methods Are 

More Dedicated to Best 

Practices but Are 

Awareness-oriented

DAU’s Acquisition Reform Communication Center (ARCC) is a key avenue 

for disseminating information on the best practices to the acquisition 

workforce. ARCC provides training through videos and periodic satellite 

broadcasts on a variety of best practice topics. However, ARCC does not 

track attendees and has no assurance that the workforce adequately 

receives the training. For those that do attend, the introductory nature of 

the training may not provide the depth or specificity to implement the 

practice at the workplace or in a time frame that is helpful.

The acquisition reform offices in the services communicate best practice 

information through acquisition reform courses, periodic satellite 

broadcasts, and informational videos, which are sufficient for broad 

exposure on best practices but again are of limited depth for practical 

application. Roadshows, traveling multiday training workshops provided to 

staff at a number of locations, and Acquisition Reform Week, designated by 

DOD as an opportunity for all service organizations to cease their normal 

operations and focus on acquisition and logistics reform initiatives, are also 

used to provide best practices training to a wide range of the acquisition 

workforce. These methods also have limitations in depth and workforce 

coverage. For example, roadshows typically provide awareness training on 

the practices and do not provide in-depth information needed for 

implementation at the workplace. A program official believes that only

10 to 15 percent of the acquisition staff attend the second day of roadshow 

workshops, where more detailed training is provided. The annual 

Acquisition Reform Weeks also provide awareness level training. Neither 

method is tailored to specific program offices or provides assurance that it 

is delivered at the time most needed by the workforce.

DOD Training Organizations 

Not Set Up to Help Design 

or Deliver Tailored Training 

to Program Offices

DOD does not have organizations that are comparable to those in 

commercial firms and that work with program offices in identifying best 

practices applicable to a particular program, designing an approach to 

training the program office staff, or delivering the training to the program 

offices. DOD training organizations—those that deliver DAWIA and 

acquisition reform training—are not set up or have the resources to deliver 

best practices quickly and easily to program staff, and then ensure 

implementation at the working level. 
Chapter 3

DOD Training Methods for Best Practices Do 

Not Go as Far as Leading Commercial Firms

Page 48 GAO/NSIAD-99-206 Best Practices

The DAU consortium integrates best practices topics into DAWIA 

certification or acquisition reform training but does not develop training 

specifically for implementing the practices. Although DAU has a 

substantial full-time faculty, the faculty members’ main priority has been to 

teach in the classroom. One course director informed us that faculty 

members are evaluated on the basis of hours of training provided in the 

classroom, which implicitly discourages work outside the classroom, such 

as consulting with program offices. Also, little or no consultation occurs 

between the course designers and the implementers on how to best 

implement a practice at the program office level. The Director of DOD’s 

International and Commercial Systems Acquisition unit stated that DOD’s 

current structure for defining and developing training courses does not 

have clear accountability to ensure that training on best practices is 

provided at the program level. Several program office staff informed us 

that they have not been given the opportunity to provide input to influence 

the content of the courses. Instructors for the courses, who do have 

frequent contact with program office staff, also believed that their ability to 

influence the content of the courses was limited. DOD does use internet 

site surveys to obtain staff feedback on acquisition reform training, which 

could help tailor the courses more to the needs of the program offices. 

However, program office staff believed the survey forms were inadequate 

for addressing the specific training needs of individual program offices.

Organizations with more direct responsibility for providing training on best 

practices are staffed to the level needed to design distance learning 

courses, such as web-based training, satellite broadcasts, and instructional 

videos. For example, the services’ acquisition reform offices each employ 

from 8 to 20 people, not nearly enough to provide on-site training or 

in-depth consulting to individual program offices. Some DOD organizations 

have recognized the value of providing training tailored to specific program 

offices and are attempting to go beyond traditional training roles. 

However, these are largely ad hoc efforts that do not go as far as 

commercial methods. For example, the Navy is creating Total Ownership 

Cost teams, that are to advise program offices on the use of CAIV. Navy 

officials stated that due to limited resources, the teams will only reach 

about 15 of the 300 program offices per year. The Air Force plans to begin 

using training support teams to address best practices issues from the 

program office perspective. However, the teams have not yet begun and 

full operation is not scheduled to begin until fiscal year 2001. Also, an Air 

Force official stated that another team, created to address acquisition 

reform training issues, will not involve program managers, which raises the 

question of whether program office needs will be adequately considered.
Page 49 GAO/NSIAD-99-206 Best Practices

Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to New 

Practices Chapte4r

The intensive, hands-on training methods leading commercial firms employ 

on new practices are the result of a systematic, institutionally driven 

approach to implementation. These firms commit to and concentrate their 

resources and attention on a few well-defined practices and make a 

significant front-end investment in providing the training to the primary 

implementers. Additionally, the firms strive to create a supportive 

environment to put the implementers of the practice in a good position to 

succeed. Their objective is to have successful implementation at the work 

level; training methods are shaped to meet that objective. DOD’s training 

methods for best practices do not benefit from a similarly strategic 

approach for deploying new best practices and provide little assurance that 

the practices are consistently implemented in the workplace. DOD has 

promulgated numerous initiatives in the past few years without 

communicating their relative priority to trainers and implementers. Often, 

the initiatives were not accompanied by clear guidance or by the initial 

training needed for implementation. While DOD commits significant 

resources to training, it does not make a uniform front-end investment to 

ensure the implementers of new practices will succeed. This approach 

depends more on the individual program offices to recognize the need for 

and to make this investment.

Recent DOD actions reflect its recognition that training improvements are 

needed. Two plans proposing divergent improvements to the training of 

the acquisition workforce are now being considered. One plan does not 

focus on best practices training specifically, offering instead an incremental 

approach to changing the DAU structure. The other targets 

implementation of best practices directly, with a broad scope of efforts that 

would require substantial changes from the current approach to training. 

The latter plan is much more aligned with the commercial approach to 

training described in this report. However, if it is adopted, it faces many 

challenges to providing better training for program offices. Another recent 

DOD action—creation of a continuous learning policy—may provide more 

training, especially for more senior members of the acquisition workforce, 

but it does not ensure that best practices will be included.
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 50 GAO/NSIAD-99-206 Best Practices

Leading Firms’ 

Strategic Approach 

Better Ensures 

Implementation of Key 

Practices

Although the four companies we contacted initiated very different types of 

practices, they exhibited a similar strategic approach to ensuring that the 

key practices were implemented. The elements of this approach can be 

summarized as

• clear vision,

• adoption of few practices at any one time,

• assessment and identification of target population for implementation,

• well-defined goals,

• significant investment in training and other vehicles to aid 

implementation, and

• supportive environment to facilitate implementation.

In general, we found that the firms shared a common strategy for 

implementing key practices that were intended to change company culture. 

First, the firms’ corporate management committed to and adopted seven or 

less key practices at any given time. One firm we contacted adopted only 

one or two key new practices, which enabled the company to concentrate 

its attention and target resources, including training, to the implementation 

of the practice, and signaled the importance of the practices to trainers and 

implementers. Second, for practices that were not companywide, the firms 

assessed which staff should be included in the implementation. For 

example, Ford’s training unit assessed each existing program team working 

on vehicle lines beyond the initial development phase to determine if the 

team could benefit from adopting portions of FPDS or if some older teams 

should be excluded. An IBM official stated that while the company 

currently promotes six practices, not all will apply to every segment of the 

employee population. Third, once the target population was identified, 

company leaders made the implementation of the practice mandatory. For 

example, the Ford Motor Company required that all new vehicle lines built 

for the year 2000 and beyond use FPDS. Lastly, to further assist in the 

implementation, the companies developed well-defined goals to better 

ensure that the target population consistently understood how to apply a 

new practice to improve product production—the ultimate goal. 

The firms we contacted ensured that the implementers of the practice 

received the assistance necessary to succeed. Consequently, these 

companies made a significant investment, including a comprehensive 

front-end effort, to support the needs of program offices that would 

implement the practice. According to a Ford training official, if the 

initiative is important enough to the organization, then the investment is 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 51 GAO/NSIAD-99-206 Best Practices

justified. Other companies agreed with that way of thinking. Boeing’s 

Learning Program Development Director summarized the corporate 

training strategy for implementing new practices as one that includes a 

clearly stated vision or mission statement, well-defined goals, and enablers, 

such as training and good processes, to support the implementers. This 

philosophy enabled Boeing to take a year to develop the training program 

tailored to the 777 program—which was intended to change the corporate 

culture and encourage employees to rethink how they did their jobs. Both 

Boeing training and program officials believe that the training investment 

resulted in the successful implementation of the key 777 program 

practices. 

While the company officials acknowledged that training was instrumental 

in the implementation of the key practices, nearly every official we spoke 

with stated that training is just one of the necessary components. They 

noted that creating the right environment is key to the successful 

implementation of new practices and that the quality of training was 

dependent on this environment. Company officials stressed that strong 

leadership is often another key. An IBM official stated that, at inception, 

top leaders need to provide sufficient funding for training, well-defined 

expectations, clear direction, oversight, continued interest, and incentives 

to ensure that the new practices are possible to implement. The manager 

for the 777 program stated that Boeing’s management works in teams—a 

key practice. He believed that it was management’s ability to lead by 

example that helped to prevent a return to the former functional way of 

operating. These companies believe that other factors, such as an 

accommodating organizational structure, good internal communication, 

consistent application, and supportive technology, are needed to foster the 

implementation of key new practices.

DOD’s Training on Best 

Practices Does Not 

Stem From a Strategic 

Approach

DOD does not have a uniform or defined strategic approach to 

implementing best practices. Accordingly, training methods are not shaped 

by the same drive for implementation found in leading commercial firms. 

DOD’s approach commonly begins with policy memorandums widely 

distributed to the entire acquisition workforce. However, the policy 

provides little specific guidance on how or which program offices should 

implement the practice. Since 1994, DOD has proposed close to 40 

acquisition reform initiatives without an indication of relative priority. The 

absence of priorities makes it difficult to not only develop training courses 

but also to determine which courses are most appropriate for the needs of 

program offices. 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 52 GAO/NSIAD-99-206 Best Practices

DOD’s Policy on Best 

Practices Is Not Coupled 

With a Strategy for 

Implementation 

DOD’s policy on implementing the best practices is promulgated without a 

strategy or specificity as to whom within the acquisition workforce is 

expected to implement the practices or which program offices are in the 

best position, in a program life cycle, to implement the practices. Typically, 

the policy is not promulgated in conjunction with a detailed plan that 

outlines the support and training required by the program offices tasked 

with implementing the practice. Similarly, a 1997 Coopers & Lybrand study 

on acquisition reform implementation found inconsistencies in DOD’s 

implementation of new practices. It stated that these inconsistencies may 

have resulted from a lack of an integrated, cohesive DOD strategic plan to 

put them in context to each other and to larger strategic goals. While DOD 

has made some progress in establishing such goals through its Government 

Performance Reporting Act plans, organizations responsible for training 

and implementing the practices have not experienced the effects.

In addition, the policies themselves are not always clear. For example, 

although the CAIV initiative was promulgated in 1995, training officials 

reported they did not have a clear understanding of what CAIV means. 

AAAV program officials said they developed training for 1997 to clarify the 

CAIV concept for the program. A November 1998 Air Force workshop on 

CAIV reported that the concept is still not well understood or widely 

implemented. 

Similarly, the IPT policy states that IPTs should be implemented “when it 

makes sense.” The policy for the use of past performance data states that 

the practice should be implemented to the “degree possible.” While a fair 

degree of latitude is good for implementers, some DOD officials believe 

that this level of generality is insufficient to ensure implementation of the 

practice. For example, a training official observed that policies are not well 

defined because of the desire to bring new practices quickly to the 

workforce and so some are deployed when they are not ready for 

implementation. According to a DAU official, unclear and general policies 

can contribute to ineffective training because it makes it difficult to 

develop a course of instruction that teaches a unified, cohesive way to 

implement a policy. A program official noted that general policy 

statements provide little guidance or assistance in the program offices’ 

implementation of the practice, even though program offices are often the 

primary implementers of best practices. 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 53 GAO/NSIAD-99-206 Best Practices

DOD Attempts to 

Implement Many Practices 

Without Prioritization

In addition to IPTs, performance specifications, CAIV, and past 

performance, DOD has proposed numerous reform initiatives since 1994. 

In 1997, the Secretary of Defense proposed several new reform initiatives 

under the umbrella of the Defense Reform Initiative. These initiatives have 

been promulgated without an indication of relative priority, leaving 

educators and implementers to decide on what is important. The lack of 

prioritization makes it difficult to determine training needs, especially 

when the number of initiatives continues to increase. While the prolific 

offering of new ideas can stimulate innovation, the absence of priorities 

makes it difficult to focus training on the specific initiatives that are the 

most important to implement. Moreover, program management offices are 

not necessarily in a good position to sort through the initiatives to focus on 

those that are the most important to the job at hand.

Several DOD officials expressed concerns regarding the number of 

initiatives without prioritization. For example, a DAU training official 

noted that it is impractical to expect to train the entire workforce on all of 

the initiatives and that the Office of the Secretary of Defense should set the 

priorities for implementation of initiatives. A DAU consultant echoed the 

view that there are too many new reform initiatives and that DOD needs to 

set priorities on which initiatives to address. A service acquisition reform 

official observed that the large numbers of best practices promoted by 

DOD overwhelms the services. This official told us that the three services’ 

acquisition reform offices meet quarterly to share progress on acquisition 

reform initiatives. He believes these meetings could serve as an 

opportunity to prioritize the practices. However, the current focus of the 

meetings is unclear and the offices’ charters are under review. Another 

acquisition reform office representative stated that the combination of a 

large number of reform initiatives and unclear priorities puts the office in 

the position of having to guess at what is the most important. This leads to 

emphasizing what is perceived to be popular, he added.

Other officials explained that due to the frequency of new initiatives being 

introduced, training courses should be regularly revisited. However, 

training personnel have not been able to consistently keep up with the pace 

of change. For example, a service acquisition reform official stated that it 

requires at least a full year to determine the elements of a new initiative and 

then the challenge of how to train the initiative still exists. A 1997 DOD 

study team that reviewed DOD’s acquisition education and training 

structure and process found that curriculum development often lagged 

behind the rapidly changing policy requirements. The study also noted that 

instructors were not up to date on current acquisition policies and that 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 54 GAO/NSIAD-99-206 Best Practices

because they were out of touch with DOD policy makers, the instructors 

found it difficult to develop up-to-date training courses. 

DOD Proposals to Alter 

Training Reflect 

Conflicting 

Approaches

DOD is aware of the need to improve the means by which the acquisition 

workforce receives and implements new initiatives. Two proposals being 

considered by the Under Secretary of Defense for Acquisition and 

Technology that are intended to improve workforce training, and a new 

continuing education policy has been put in place. The first proposal is a 

DAU transition plan for improving DAU’s structure. The plan is based on a 

1997 study by a DOD team and a 1998 study by the Logistics Management 

Institute. Both studies confirmed strengths in the DAU approach, 

particularly relating to functional or skill-based training and the 

weaknesses in some of the same areas that surfaced in our work. The 

studies made recommendations for significant organizational and 

operational changes in DAU to better meet the educational needs of the 

acquisition workforce. However, a more moderate approach was adopted 

in preparing the transition plan for DAU. While the proposed structure 

offers improvements, it does not discernibly address key weaknesses in the 

training of best practices. The plan has yet to be approved. The second 

proposal, a report from the DOD Section 912(c) Commercial Business 

Environment Group directly advocates incorporating best practices and is 

much broader in recommending changes. It proposes that DOD’s 

acquisition structure be transformed into a team type of organization with 

members drawn from across the DOD procurement enterprise. A new 

learning organization would be created specifically to support accelerated 

change. 

DOD has also instituted a continuous learning policy, which should help 

people obtain current training, even if they are already fully certified. 

However, the required levels of continuous learning can be met in many 

ways and do not place any particular emphasis on training in best 

practices. These DOD efforts are a step in the right direction for seeing 

that program offices have the information they need, but none focus on the 

program offices’ needs. Given the pivotal role they play in weapons 

acquisition, a more direct approach may be required.
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 55 GAO/NSIAD-99-206 Best Practices

DAU Transition Plan Is 

Largely Silent on 

Improvements to Best 

Practices Training

The DOD transition plan for restructuring DAU was based on 

recommendations of a 1997 process action team that DOD chartered to 

create a clear vision and a structure for future workforce training and 

education. Several findings of the study related to training for new 

initiatives and practices, including (1) the existing curriculum design 

process was functionally driven, did not address the needs of people 

outside the functional career fields, and did not lend itself to successful 

development of cross-functional curricula; (2) a mechanism was needed to 

validate and prioritize requirements before they are submitted; (3) faculty 

members did not incorporate up-to-date case studies; (4) DAU schools 

used procedures for measuring performance that focused on student 

satisfaction with a course, not whether the user community’s acquisition 

education requirements were being met; and (5) DAU’s use of technologybased learning was insufficient.

The team recommended that DAU be replaced with a unified Defense 

Acquisition Institute that would have responsibility for development and 

delivery of acquisition training, the consortium be streamlined to reduce 

duplication, and lines of authority between DAU and the members be 

strengthened. The team envisioned the institute as, among other things, (1) 

fostering innovation and facilitating reform and continuous improvements, 

(2) having a clear message of support from DOD acquisition leadership that 

education is the key to meeting DOD goals, (3) identifying acquisition 

training needs that are cross-functional areas or that are multifunctional in 

nature, (4) focusing on the needs of the learner, and (5) employing a 

process to validate the effectiveness of the training received. However, a 

dissenting opinion offered by one of the team members to evolve the 

current DAU organization structure through a series of near-term and 

long-term actions was accepted by the Under Secretary of Defense. 

Subsequently, the Logistic Management Institute conducted a review for 

DOD and developed a structural reorganization for DAU that was to be 

used as the starting point for the transition plan. The Institute noted that 

DAU training was generally skill and specialty oriented, with limited 

overlap among “stove-piped” career fields. It recommended that the new 

structure be designed along the lines of a corporate university. As such, its 

curriculum would include cross-functional training and business executive 

development and act as an agent of change, being more involved with the 

workforce in determining how changes would be carried out.

The DAU president submitted the draft transition plan for restructuring in 

September 1998. As of June 1999, DOD had not approved the plan. The 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 56 GAO/NSIAD-99-206 Best Practices

plan has several features that offer promise for more effective training 

overall. These include (1) a unified structure in which DAU has full 

responsibility for ensuring that the overall course structure meets the 

needs of the acquisition workforce, (2) consolidation of the consortium 

schools into four main campuses under the direct authority of DAU, (3) an 

increased emphasis on distance learning techniques, (4) steps to improve 

the qualifications of faculty, and (5) replacement of functional boards with 

functional IPTs that have representation from both functional experts and 

instructors and that will design curricula together. 

Other features of the plan raise doubts about its ability to provide a 

strategic approach to shape training that will help implementation of best 

practices at program offices. The transition team that prepared the plan 

concluded that DAU was unique and could not be matched with other 

institutions, like corporate universities. Thus, the corporate university 

model recommended by the Institute was not adopted. While the 

curriculum may benefit from the functional IPTs, the plan does not mention 

the need for cross-functional or best practices training or for DAU to 

accept the role of a change agent. It is silent on prioritizing initiatives or 

improving feedback. While it is possible that under the proposed plan DAU 

could play a more active role in the design and delivery of tailored training 

to the workplace, the plan does not that suggest that, other than increased 

distance learning, training will be offered in a manner substantially 

different from what has been traditionally offered or that the relationship 

between DAU and the acquisition community will be any closer.

Commercial Business 

Environment Plan Proposes 

Major Changes to Hasten 

Reform

The National Defense Authorization Act for Fiscal Year 1998, section 912, 

directed the Secretary of Defense to submit to Congress an implementation 

plan to streamline the acquisition organizations, workforce, and 

infrastructure. As part of that mandate, the Deputy Under Secretary of 

Defense for Acquisition and Technology formed a team to develop a plan 

for ensuring that new practices would be incorporated by DOD. The draft 

plan, dated June 1999, concludes that adopting the most effective 

commercial practices requires a cultural and organizational transformation 

within DOD. The draft proposes a very broad approach to making this 

change, which includes a training regime to implement key commercial 

business practices and a change model to accelerate the implementation of 

new practices. The model calls for establishing goals and a scorecard to 

measure whether they are met. To inform everyone of the goals, a tiered 

approach is envisioned, starting with senior leaders, rolling down to teams 

with specific reform target goals. The teams are to report on their 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 57 GAO/NSIAD-99-206 Best Practices

outcomes within 60 days. A mission support office will support the teams 

as well as capture results across DOD and make them available to others. 

This office is to be led by a senior DOD official and staffed by management 

and facilitation experts, and possibly outside contractors. 

The plan proposes a team approach for managing acquisitions that 

embraces best practices and maintains “learning organizations that seek 

out and adopt best practices that improve individual and organizational 

performance.” The plan also calls for new organizational roles in adopting 

of best practices. Among these is DAU. The plan recommends that DAU be 

broadly recast to adopt the corporate university model and become a 

change agent. This plan, while not specific about the help that program 

offices would receive, calls for a strategic approach that would make it 

more likely that DOD could provide its program offices tailored training—

more help—in implementing best practices. Nonetheless, translating this 

plan into the type of training and other help program offices need to 

implement best practices is challenging. Key elements, such as the 

cascading of goals from senior levels on down and the revision of what 

constitutes teams, are major undertakings and challenge long established 

patterns of interaction. How the mission support office would reach 

specific program offices is unclear, as is how it would interface with DAU. 

Also, whether initiatives are to be prioritized and how DAU would be 

restructured to provide functional training and become a change agent for 

new practices remains to be delineated. 

Continuous Learning Policy 

Does Not Ensure Best 

Practices Training

The objective of DOD’s recently issued continuous learning policy is to 

augment the acquisition training standards for career field certification of 

acquisition professionals. The policy requires all civilian and military 

members of the acquisition workforce to complete 80 hours of continued 

education within a 2-year period. As such, it is a good step for reaching 

more senior people in the acquisition workforce who would normally not 

receive this kind of training because they are fully certified. The policy also 

places increased emphasis on distance learning, which may address some 

of the timeliness and availability issues raised by program office officials in 

obtaining useful best practice information. One distance learning course 

that deals directly with a best practice was jointly developed last year by 

the National Contract Management Association and the National 

Association of Purchasing Management at the request of DOD. It is an 

internet-based course to integrate best commercial practices for managing 

suppliers into DOD acquisitions. It entitles participants to 24 hours of 
Chapter 4

DOD and Commercial Training Methods 

Reflect Different Strategic Approaches to 

New Practices

Page 58 GAO/NSIAD-99-206 Best Practices

continuous education credit and participants are encouraged to take the 

course as teams. 

While the policy does not preclude staff from receiving training on new 

practices, given the array of options for meeting requirements, it does little 

to ensure that best practices training will be received. An earlier version of 

the policy recommended 16 units of acquisition reform training that could 

have included best practices, but the requirement was deleted. The policy 

encourages acquisition reform training but allows for requirements to be 

met in many ways—including courses, professional activities, and on the 

job experience—which may or may not include best practices information. 

According to a training official, crediting acquisition workforce members 

for almost anything dilutes the policy.

Officials of one program noted that no one advises them on what courses to 

take. They can take courses or attend conferences that they believe would 

benefit them, but they believed this may be inadequate for ensuring that 

senior officials are aware of new ideas. Another program official said that 

her program office could meet continuing education requirements by 

mandatory attendance of Acquisition Reform Week. While this exposure 

would increase awareness of new practices, such training was 

characterized as providing just a “flavor” for the initiative—not providing 

enough information to implement the practice.
Page 59 GAO/NSIAD-99-206 Best Practices

Chapter 5

Conclusions and Recommendations Chapte5r

Conclusions Leading commercial firms and weapon system program offices have 

successfully implemented best practices or new initiatives by taking a 

tailored, on-site training approach. Such an approach is a concentrated, 

up-front effort that creates an environment for accepting the practice 

within a program and provides the impetus and know-how to apply the 

practice at the workplace. It is warranted by the investment and 

importance that large programs represent. The difference is that within 

DOD, the approach is allowed to happen, while leading commercial firms 

ensure it happens. These firms ensure that initiatives are successfully 

implemented at the program level by customizing the initiatives to fit the 

programs, working with program office staff to develop training that meets 

the needs of the program, and providing on-site assistance to the program. 

This approach would not necessarily be used every time by commercial 

firms to train their entire workforce or to impart every skill; it is instead an 

intensive approach they reserve for key initiatives and individual programs. 

It stems from an overall training strategy that restricts the number of 

initiatives that are brought to the workforce, ensures that the initiatives 

have well-defined objectives, and makes training organizations responsible 

for seeing that policy is turned into practice at the program level. Little 

regarding implementation is left to chance. 

Within DOD, this approach can be taken if a program’s management has the 

insight to recognize the importance of an initiative, the necessary resources 

needed to implement the initiative at the work site, and support from top 

management. Even though DOD has drawn its reform initiatives from the 

practices of leading commercial firms, it has not adopted the 

accompanying training strategy to ensure a concerted effort is made to 

implement key initiatives at the program level. As a result, more is left to 

chance. In the past 4 years, over 40 initiatives have been introduced to the 

DOD acquisition workforce, without delineating which are the most 

important for weapon system programs. Several organizations are 

responsible for training and each service has an office that promotes 

acquisition reform initiatives. Although these organizations are doing 

valuable work and are contributing to workforce training, it is unclear how, 

collectively, they can drive key initiatives to the program office level. 

DAU’s main tool for training the acquisition workforce—functional 

training—is aimed at increasing the expertise of people in their career 

fields, but it does not have the throughput and ability to reach the program 

office as a work unit, nor the practical depth needed to imbed an initiative 

at the program office. Other tools, such as satellite broadcasts, distance 
Chapter 5

Conclusions and Recommendations

Page 60 GAO/NSIAD-99-206 Best Practices

learning, and Acquisition Reform Week, make program office staff aware of 

new initiatives, but they do not provide the depth or environment to 

implement them because DOD’s training organizations and methods have 

not been designed to provide this kind of support. 

DOD recognizes the importance of training as a tool that can help infuse 

best practices into weapon acquisitions and has several draft plans to 

reorganize how training is delivered. However, it is not evident that the 

planning and methods called for by the DAU transition plan will change the 

way such training is conceived and delivered to weapon system program 

offices. The Commercial Business Environment Plan does call for a 

strategic approach that would make it more likely that DOD could provide 

its program offices tailored training—more help—in implementing best 

practices. The fact that neither plan has been approved indicates that DOD 

has not decided what role it wants acquisition training to play on best 

practices. 

Whether through existing organizations or through new ones, DOD needs 

to increase its capability to provide tailored training of specific initiatives at 

the program office level. Developing this additional capability will require 

a strategy for implementation and collaboration with program offices on 

the design of training. Without a concerted approach to foster the 

implementation of best practices by providing for customized training at 

the program offices, better outcomes in weapon programs will be more 

difficult to achieve. The recommendations that follow address ways DOD 

can provide tailored training on selected practices to program offices. The 

recommendations are made without prejudice toward the functional and 

other training DOD provides to the acquisition workforce.

Recommendations We recommend that the Secretary of Defense develop a strategy for 

delivering targeted training on selected new practices to program offices to 

ensure the practices will be implemented. This strategy should 

accomplish the following.

• Identify those initiatives most worthy of a targeted training investment 

by screening the numerous initiatives to separate those for which a 

general awareness is sufficient from those that warrant a targeted 

approach. Those to be targeted should be relatively few in number at 

any given point in time. The process of setting priorities would be of a 

continuing nature and would benefit from the input of current program 

office members.
Chapter 5

Conclusions and Recommendations

Page 61 GAO/NSIAD-99-206 Best Practices

• Decide whether DAU is to play a more traditional role, as called for by 

the DAU Transition Plan, or a more proactive role, as called for by the 

Commercial Business Environment Plan and clearly communicate to 

DAU and other providers of DOD training their responsibilities in 

supporting a targeted approach to training.

• Identify the key organizations and people that are critical to the program 

offices’ ability to implement best practices, including those not currently 

defined as part of the acquisition workforce and contractors, as the 

potential audience of targeted training.

We also recommend that the Secretary make the Under Secretary of 

Defense for Acquisition and Technology responsible for taking steps to 

institutionalize the methods for tailoring training on key initiatives that 

have been shown to be successful by leading program offices and 

commercial firms. This involves having proactive organizations and the 

tools to inform, prepare, and assist program offices to implement the 

initiatives most applicable to their programs. This approach should include 

the following.

• Making new program managers aware of the initiatives that could 

significantly affect the outcomes of their programs and the role the 

managers play in having their staffs trained to implement those 

initiatives. This could be done through existing program managers’ 

courses.

• Working with individual program managers to tailor the initiatives to 

their programs.

• Developing an approach, in conjunction with the program managers, to 

create a culture—and necessary incentives—within the program office 

to make it receptive for adopting the initiatives.

• Helping the program managers determine the best methods for making 

initial and sustaining resource investments in training the program 

office staff. This help could consist of providing advice, assisting in the 

identification of experts in the needed areas, and culling lessons learned 

from other programs that have implemented the initiative. 

To introduce and reinforce new practices, we recommend that the 

Secretary require the Under Secretary of Defense for Acquisition and 

Technology more effectively use existing training vehicles to

• incorporate new practices more quickly,

• better reflect the changing role of program managers,
Chapter 5

Conclusions and Recommendations

Page 62 GAO/NSIAD-99-206 Best Practices

• ensure that instructors’ knowledge keeps pace with the latest practices, 

and

• provide more case study material that gives current implementation 

examples for a variety of situations.

Agency Comments and 

Our Evaluation

DOD concurred with a draft of this report and all of its recommendations 

(see app. I). DOD noted that it was taking steps to develop a strategy for 

delivering targeted training on selected new practices to program offices, 

consistent with our report. It cited the Commercial Business Environment 

study team’s vision for accelerating cultural change within the acquisition 

community, in conjunction with a strategy of delivering team training, to 

implement best practices. DOD stated that it anticipates adopting and 

launching many of the study’s recommendations without delay. DOD also 

recognized that DAU should augment its capability, consistent with the best 

practices of the private sector corporate universities, by embracing the role 

of “change agent” and by designating “performance consultants” who focus 

on developing tailored training to meet program team needs. 

DOD noted that it has also begun the restructuring of DAU to provide a 

more centralized and integrated education and training program that will 

also be integrated with the initiatives identified by the Commercial 

Business Environment report. We agree that the DOD study frames a 

strategic approach that would make it more likely that DOD could provide 

its weapon system program offices tailored training to help them 

implement new practices. We are also encouraged by the speed with which 

DOD is implementing the study’s recommendations. We do note that the 

study is not specific about the help that program offices would receive and 

that translating the study into such specifics will be a significant challenge. 

As DOD proceeds with implementation, we reiterate the importance of 

screening the numerous initiatives to identify those most worthy of 

targeted investments and to involve the staff of current program offices in 

setting these priorities.

In agreeing on the need for targeted training, DOD noted that DAU’s 

communications arm will work to provide for immediate delivery of 

tailored training on key initiatives to meet the needs of program offices. 

DAU’s current communications arm—the ARCC—provides training 

primarily by distributing videos and periodic satellite broadcasts. As such, 

ARCC may not be well-poised to take on the responsibilities of a proactive 

organization with the tools needed to inform, prepare, and assist program 

offices to help them implement the initiatives most applicable to their 
Chapter 5

Conclusions and Recommendations

Page 63 GAO/NSIAD-99-206 Best Practices

programs, as we have recommended. Regardless of which organization is 

assigned the responsibility, it is important that the organization go beyond 

the traditional approach of making standard training available and instead 

work with program offices to (1) tailor the initiatives to their programs;

(2) develop an approach to help make their staffs receptive to adopting 

new practices; and (3) help the program managers—on site, if necessary—

determine the best methods for making the investments in training the 

program office staff. 

Regarding making more effective use of existing training vehicles, DOD 

stated that during a review of core curriculum requirements, it will be 

completely reviewing the principal program management training toward 

implementing a fully integrated strategy. It noted that the strategic plan 

will address the development of tailored case study materials, practical 

exercises, and assessment criteria for the adoption of new practices. We 

believe that DOD’s proposed review has the potential to more effectively 

promote best practices training. In addition to these actions, as DOD 

conducts its review, it needs to have existing training vehicles incorporate 

new practices more quickly, better reflect the changing role of program 

managers, and ensure that instructors’ knowledge keeps pace with the 

latest practices. Greater involvement of weapon system program office 

staff in the design and content of these training vehicles could link the 

training more closely with the job at hand. While DOD plans to focus its 

continuous learning activities on key acquisition reforms, currently, 

members of the acquisition workforce do not necessarily have to take 

training on acquisition reform to meet continuous learning requirements.
Page 64 GAO/NSIAD-99-206 Best Practices

Appendix I

Comments From the Department of Defense ApenxIdi
Appendix I

Comments From the Department of Defense

Page 65 GAO/NSIAD-99-206 Best Practices
Appendix I

Comments From the Department of Defense

Page 66 GAO/NSIAD-99-206 Best Practices
Page 67 GAO/NSIAD-99-206 Best Practices

Appendix II

Description of Program Offices Visited ApenxIdi

Following is a description of each of the program offices we visited.

Advanced Amphibious 

Assault Vehicle 

The Advanced Amphibious Assault Vehicle (AAAV) is to be used by the 

Marine Corps to provide high-speed transportation of troops from ships 

located beyond the horizon to the beaches. It will also provide armor 

protection, land mobility, and fire support during combat operations on the 

shore. It is a category I1

 major acquisition program for the Marine Corps. 

The program office is staffed by about 75 people.

Advanced 

Medium-Range 

Air-to-Air Missile 

The Advanced Medium-Range Air-to-Air Missile (AMRAAM) is a new 

generation radar-guided missile that fighter aircraft are to use against 

enemy aircraft. It can be launched beyond visual range, day or night and in 

all weather. The missile is used on the Air Force F-15 and F-16 and the 

Navy F-14 and F/A-18. It is a category I acquisition program for the Air 

Force. The AMRAAM program started over 20 years ago. About 70 people 

staff the program office.

Battlefield Combat 

Identification System 

The Battlefield Combat Identification System (BCIS) is a millimeter wave 

electronic, question, and answer combat identification system capable of 

identifying friendly ground combat vehicles. The BCIS interrogation is 

triggered automatically by activation of the shooter, which sends an 

encrypted query message to the targeted vehicle. If the targeted vehicle is 

friendly and equipped with BCIS, its transponder answers with an 

encrypted friend message that is illuminated in the shooter’s sights. It is a 

category II2

 acquisition managed by the Army. The program office is 

staffed by 16 core people and 30 additional people who are assigned from 

other government organizations or contractor support. 

1

 Category I programs are defined as major defense acquisition programs estimated to cost over 

$355 million for research, development, test, and evaluation, or have procurement costs of more than 

$2.135 billion. 

2

 Category II programs are defined as acquisition programs estimated to cost over $75 million for 

research, development, test and evaluation, or have procurement costs of more than $300 million.
Appendix II

Description of Program Offices Visited

Page 68 GAO/NSIAD-99-206 Best Practices

Joint Air-to-Surface 

Standoff Missile

The Joint Air-to-Surface Standoff Missile (JASSM) is an autonomous, 

long-range cruise missile to be capable of launch from outside area 

defenses to hit ground targets. It is to be launched from a wide range of 

bomber, attack, and fighter fixed-wing aircraft. It is a joint Air Force and 

Navy category I major acquisition program.

Joint Strike Fighter The Joint Strike Fighter (JSF) program’s objective is to develop and field an 

affordable, highly common family of next generation multirole strike 

fighter aircraft for the Navy, the Air Force, the Marine Corps, and U.S. 

allies. The focus of the program is affordability. It is a joint service 

category I major acquisition program with Navy as lead. About 100 people 

staff the program office.

Joint Surveillance and 

Target Attack Radar 

System–Joint Tactical 

Terminal 

The Joint Surveillance and Target Attack Radar System– Joint Tactical 

Terminal (JSTARS-JTT) is to provide warfighters with near real-time 

tactical intelligence and targeting information. It is a terminal that supplies 

the critical data link from various intelligence sources to battle managers 

across all services. The terminal is integrated into other weapon systems 

and is to be mounted in fixed- and rotary-wing aircraft, surface ships, and 

fixed or mobile ground platforms and vehicles. It is managed by the Army 

and is a category III3

 program. About 10 people staff the program office.

3

 Category III programs are defined as those acquisitions programs that do not meet the criteria for 

categories I or II.
Page 69 GAO/NSIAD-99-206 Best Practices

Appendix III

GAO Contacts and Staff Acknowledgments ApenIxdi

GAO Contacts Louis J. Rodrigues (202) 512-4841

Paul L. Francis (202) 512-2811

Acknowledgments In addition to those named above, Charles Cannon, Monica L. Kelly, 

Gordon W. Lusby, Elisabeth G. Ryan, Sally Shipman, and Yelena K. 

Thompson made key contributions to this report.
Page 70 GAO/NSIAD-99-206 Best Practices
Page 71 GAO/NSIAD-99-206 Best Practices
Page 72 GAO/NSIAD-99-206 Best Practices

Related GAO Products

Best Practices: Commercial Quality Assurance Practices Offer 

Improvements for DOD (GAO/NSIAD-96-162, Aug. 26, 1996).

Major Acquisitions: Significant Changes Underway in DOD’s Earned Value 

Management Process (GAO/NSIAD-97-108, May 5, 1997).

Best Practices: Successful Application to Weapon Acquisitions Requires 

Changes in DOD’s Environment (GAO/NSIAD-98-56, Feb. 24, 1998).

Best Practices: DOD Can Help Suppliers Contribute More to Weapon 

System Programs (GAO/NSIAD-98-87, Mar. 17, 1998).

Defense Acquisition: Improved Program Outcomes Are Possible

(GAO/T-NSIAD-98-123, Mar. 18, 1998).

Defense Acquisition: Best Commercial Practices Can Improve Program 

Outcomes (GAO/T-NSIAD-99-116, Mar. 17, 1999).

Best Practices: Better Management of Technology Development Can 

Improve Weapon System Outcomes (GAO/NSIAD-99-162, July 30, 1999).

(707343) Letr
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the 

following address, accompanied by a check or money order made 

out to the Superintendent of Documents, when necessary, VISA and 

MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

For information on how to access GAO reports on the INTERNET, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at: 

http://www.gao.gov
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00