GAO

United States General Accounting Office

Testimony

Before the Subcommittee on Readiness and Management 

Support, Committee on Armed Services, U.S. Senate 

For Release on Delivery on

Wednesday, March 17, 1999 DEFENSE ACQUISITION 

Best Commercial Practices 

Can Improve Program 

Outcomes

Statement for the Record by Louis J. Rodrigues, Director, 

Defense Acquisitions Issues, National Security and 

International Affairs Division

GAO/T-NSIAD-99-116

Page 1 GAO/T-NSIAD-99-116 Best Practices

Mr. Chairman and Members of the Subcommittee:

Thank you for the opportunity to submit this statement for the record, 

which discusses the best practices that can improve the way the 

Department of Defense (DOD) buys major weapon systems. With DOD’s 

annual research, development, and production spending for major systems 

at about $85 billion, the Subcommittee’s oversight of acquisition policy can 

have a major impact on the value the taxpayer gets for that expenditure.

For several years, the leadership in the Office of the Secretary has been 

committed to instituting reforms to improve the outcomes of the 

acquisition process. In particular, the Under Secretary of Defense’s 

(Acquisition and Technology) focus on shorter cycle times is welcome. 

Shorter cycle times make for a more agile acquisition process, which is 

critical if DOD is to respond quickly in a national security environment of 

unknown threats. Shorter cycle times are also important for solving 

problems associated with readiness by ensuring that the industrial base 

will be able to support systems once they are fielded.

Despite good intentions and some progress, our ongoing reviews of DOD’s 

major system acquisitions are showing that these efforts at systemic 

change have not yet been reflected in the management and

decision-making on individual programs. The flagship systems, as well as 

many other top priorities in each of the services, continue to cost 

significantly more, take longer, and deliver less than was promised. Our 

work for this Subcommittee shows that lessons for major system 

acquisitions can be learned from the best commercial practices and applied 

in the DOD environment. Adopting these practices will require a dramatic 

change in behavior—a change that must be supported by incentives. In this 

context, the Office of the Secretary of Defense, the Congress, and the 

services’ organizations for requirements, research, and acquisition, each 

play a critical role in getting the better outcomes sought on major weapon 

system programs. 

Results in Brief On the basis of the work we have done over the past 3 years, we believe the 

best practices of leading commercial firms can be used to improve the 

development of technology and weapon systems in DOD. In particular, 

knowledge standards that are rigorously applied, coupled with the practice 

of keeping technology development separate from product development, 

stand out as key factors in the most successful commercial examples. 

These practices have put managers in the best position to succeed in 
Page 2 GAO/T-NSIAD-99-116 Best Practices

developing better products in less time and producing them within 

estimated costs. DOD programs, with some exceptions, proceed with 

lower levels of knowledge available about key factors of product 

development, such as proof of design maturity and production readiness. 

In addition, DOD allows technology development to take place during 

product development. These practices put DOD program managers in a 

much more difficult position to deliver better weapons more quickly and 

within cost projections.

Getting better outcomes on weapon system programs will take more than 

attempting to graft commercial best practices onto the existing acquisition 

process. There are underlying reasons and incentives for why such 

practices are not a natural part of how weapon systems are bought. 

Environmental factors, such as the intense competition for funding when a 

program is launched, encourage lower standards of knowledge and the 

acceptance of higher, but unrecognized, risks. What we offer to help the 

adoption of best practices is not a cookbook recipe, but a series of actions 

aimed at fostering an environment in DOD that encourages or rewards such 

practices. These actions will put managers of DOD programs in a better 

position to succeed, for we believe they are as informed and capable as 

their commercial counterparts.

Following the discussion of our work on best practices, we present 

information on the status of other work we are doing that is of interest to 

the Subcommittee. Most of this work deals with DOD initiatives related to 

acquisitions. The work includes acquisition workforce training in best 

practices, best practices for test and evaluation of weapon systems, pricing 

of sole-source commercial items, spare parts price trends, the Cost 

Accounting Standards Board, other transactions, and government-wide 

information technology contracts.

A Best Practices Model 

for Acquisition

For an acquisition process that meets DOD’s goal of developing and 

producing militarily superior weapons in a resource-constrained 

environment, we look to answer the basic question of how a capability can 

best be provided to the customer. The characteristics of best practices, as 

we have analyzed them, suggest a process for developing new 

capabilities—whether they are commercial or defense products—that is 

based on knowledge. It is a process in which technology development and 

product development are treated differently and managed separately. The 

process of developing technology culminates in discovery and must, by its 

nature, allow room for unexpected results and delays. The process of 
Page 3 GAO/T-NSIAD-99-116 Best Practices

developing a product culminates in delivery, and therefore, gives great 

weight to design and production. Discipline is inherent because criteria 

exist, tools are used, and a program does not go forward unless the strong 

business case on which the program was originally justified continues to 

hold true. 

In the past several years, we have examined best practices used by world 

class commercial firms such as Boeing, Chrysler, Hughes, Ford, and 3M, 

and individual DOD acquisition programs for weapons such as the F-22, the 

C-17, the Comanche, the New Attack Submarine, and the Advanced 

Amphibious Assault Vehicle with the objective of finding best practices for 

developing and producing major weapon systems. Our completed work 

has examined best practices for quality assurance, earned value 

management, supplier management, and transitioning products from 

development to production. A listing of these reports is included in the 

appendix. We are currently reviewing best practices for readying 

technology for inclusion in product development programs, best practices 

for test and evaluation of weapon systems, and how well DOD training 

supports the implementation of best practices. 

We have learned that a knowledge-based process is essential to getting 

better cost, schedule, and performance outcomes. This means that 

decisionmakers must have virtual certainty about critical facets of the 

product under development when needed. Such knowledge is the inverse 

of risk. The commercial and military programs we reviewed did not all 

follow the same processes in their development cycles. However, at some 

point, full knowledge was attained about a completed product, regardless 

of what development approach was taken. This knowledge can be broken 

down into three junctures that we refer to as knowledge points: when a 

match is made between the customer’s requirements and the available 

technology, when the product’s design is determined to be capable of 

meeting performance requirements, and when the product is determined to 

be producible within cost, schedule, and quality targets. In addition, we 

have identified metrics that indicate the knowledge levels associated with 

best practices and can thus help forecast problems as a development 

program progresses. An important corollary to having a knowledge-based 

process is that technology development should take place separate from an 

acquisition program and its related product development process. The 

knowledge points and their associated metrics are depicted in figure 1.
Page 4 GAO/T-NSIAD-99-116 Best Practices

Figure 1: Levels of Knowledge Attained in Best Practices for Developing Technology and Products

The leading commercial firms we visited gained more knowledge about a 

product’s technology, design, and producibility much earlier than DOD in 

the acquisition programs we reviewed. In fact, product development in 

commercial ventures is a clearly defined undertaking for which firms insist 

on having the technology in hand that is needed to meet customer 

requirements before starting. Once underway, the firms demand—and 

receive—specific knowledge about the design capability and producibility 

of a new product before production begins. The process of discovery—the 

accumulation of knowledge and elimination of unknowns—is completed 

well ahead of production. There is a synergy in this process, as the 

attainment of each successive knowledge point builds on the preceding 

one. In contrast, DOD programs are started earlier and allow technology 

development to continue into product development and even into 

production. Consequently, the programs proceed with much less 

knowledge available—and thus more risk—about required technologies, 

design capability, and producibility. Proceeding with lower levels of 

knowledge available explains much of the turbulence in DOD program 

outcomes. Metrics, such as those associated with knowledge points, show 

this to be a predictable consequence.

Technology

Development Product Development Production

Knowledge

Point 1

Technology

readiness

levels

Percent of

drawings

complete

Percent of key

production

processes in

control

Metrics

Knowledge 

Point 2

Knowledge

Point 3
Page 5 GAO/T-NSIAD-99-116 Best Practices

Knowledge Point 1: 

Requirements and 

Technology Are Matched 

Technology development has the ultimate objective of bringing a 

technology up to the point that it can be readily integrated into a new 

product and counted on to meet requirements. As a technology is 

developed, it moves from a concept to a feasible invention to a component 

that must fit onto a product and function as expected. In between, there 

are increasing levels of demonstration that can be measured. In our 

ongoing review of best practices for including new technology in products, 

we applied a scale of technology readiness levels—from one to nine—

pioneered by the National Aeronautics and Space Administration and 

adapted by the Air Force Research Laboratories. Without going into the 

details of each level, let me note that a level four equates to a laboratory 

demonstration of a technology that is not in a usable form. Imagine, if you 

will, an advanced radio technology that can be demonstrated with 

components that take up a table top. A level seven is the demonstration of 

a technology that approximates its final form and occurs in an environment 

outside the laboratory. A level eight is a technology that has been proven to 

work in its final form and in its intended operating conditions. The same 

radio at this level would have been installed in the instrument panel in the 

aircraft cockpit, integrated with other aircraft systems, and flown under all 

expected conditions. 

The lower the level of the technology at the time it is included in a product 

development program, the higher the risk that it will cause problems in the 

product development. According to the people that use the technology 

readiness levels in DOD, level seven enables a technology to be included on 

a product development with acceptable risk. In applying these standards 

to leading commercial firms, we have observed that the firms do not let a 

new technology onto a product development until it reaches level eight. On 

weapon systems that experienced cost and schedule problems, we 

observed that they were started with key technologies at levels three and 

four. By the time the same programs reached a point DOD considers 

analogous to beginning product development, key technologies were still at 

level five or lower.

We also observed that three factors contribute to the successful maturation 

of technology for inclusion on a product development. These are: 

flexibility in resources and requirements to allow for the uncertainties of 

technical progress; disciplined paths for technology to take toward 

inclusion on products, with strong “gatekeepers” to decide when to allow it 

onto a product development program; and high standards for judging the 

maturity and readiness of technology. The commercial technologies and a 

few of the DOD technologies we reviewed exhibited these factors and were 
Page 6 GAO/T-NSIAD-99-116 Best Practices

successfully included in product developments. Technologies that caused 

problems in product development did not exhibit all of these factors. In 

essence, these technologies were still going through discovery in a 

delivery-oriented environment. As one might imagine, the most difficult 

situation occurs when a product development is launched with inflexible 

requirements that can only be met with a new and immature technology. 

Knowledge Point 2: The 

Design Will Perform as 

Required

The commercial firms we visited had achieved near certainty that their 

product designs would meet customer requirements and had gone a long 

way toward ensuring that the products could be produced by the halfway 

point of product development. Both DOD and the commercial firms hold a 

critical design review (CDR) to review engineering drawings, confirm the 

design is mature, and “freeze” it to minimize changes in the future. The 

completion of engineering drawings and their release to manufacturing 

organizations signify that program managers are confident in their 

knowledge that the design performs acceptably and is mature. The 

drawings are critical to documenting this knowledge because they are not 

only precision schematics of the entire product and all of its component 

parts—they also reflect the results of testing and simulation, and they 

describe the materials and manufacturing processes to be used to make 

each component.

Both DOD and commercial companies consider the design to be essentially 

complete when about 90 percent of the engineering drawings are 

completed. Officials from commercial companies such as Boeing and 

Hughes told us that they typically had over 90 percent of these drawings 

available for the CDR. Two DOD programs we reviewed had less than

60 percent—one had less than one-third—of the drawings done at the time 

their CDRs were held. Thus, these programs had significantly less 

knowledge available about their designs. The programs did not get or were 

not expected to get to the 90-percent level of completion on the drawings 

until late in development or in production. Nonetheless, at the time of the 

CDRs, the risks of proceeding with the rest of development on these 

programs as planned were deemed acceptable. Both programs 

encountered significant design problems in testing that occurred after the 

CDR.
Page 7 GAO/T-NSIAD-99-116 Best Practices

Knowledge Point 3: 

Production Units Will Meet 

Cost, Quality, and Schedule 

Objectives

Leading commercial firms reached the point at which they knew that 

manufacturing processes would produce a new product conforming to 

cost, quality, and schedule targets before they began fabricating production 

articles. Reaching this point meant more than knowing the product could 

be manufactured; it meant that all key processes were under control, such 

that the quality, volume, and cost of their output were proven acceptable. 

Commercial firms relied on good supplier relationships, known 

manufacturing processes, and statistical process control to achieve this 

knowledge early and, in fact, had all their key processes under statistical 

process control when production began. The ability to establish control for 

key processes before production began was the culmination of all the 

practices employed to identify and reduce risk. All of the companies we 

visited agreed that knowledge about technology and design early in the 

process makes the control of processes possible.

One weapon system program that had been in production for nearly 9 years 

at the time of our 1998 review still had less than 13 percent of its key 

manufacturing processes in control. Another program had 40 percent of its 

key manufacturing processes in control 2 years before production was 

scheduled to begin, but was not scheduled to have all key processes in 

control until 4 years into production. Both programs experienced basic 

producibility problems that were not discovered until late in development 

or early in production. These risks went unrecognized even though the 

DOD had established criteria for determining whether risks were 

acceptable and whether enough knowledge had been gained to enter the 

next development phase. 

Impediments to 

Adopting Best 

Practices

The most important factors in the adoption of best practices are the 

incentives the development or acquisition process offer to managers of 

technology and product developments. The differences in the practices 

employed by the leading commercial firms and DOD reflect the different 

demands imposed on programs by the environment in which they are 

managed. The way success and failure are defined for commercial and 

defense product developments differs considerably, which creates a 

different set of incentives and evokes different behaviors from the people 

managing the programs. Specific practices take root and are sustained 

because they help a program succeed in its environment.

Leading commercial firms launch a product development only when a solid 

business case can be made. The business case basically revolves around 
Page 8 GAO/T-NSIAD-99-116 Best Practices

the ability to produce a product that the customer will buy and that will 

provide an acceptable return on investment. The point of sale occurs in 

production after product development is complete; program success is 

determined when the customer buys the finished product. If the firm has 

not made a sound business case, or has been unable to deliver on one or 

more of the business case factors, it faces a very real prospect of failure—

the customer may walk away. Also, when each program is delivered as 

promised, the company does not put at risk resources invested in other 

products. Because the match between technologies and product 

requirements is made before the product development is launched, the cost 

and schedule consequences associated with discovery are minimized. 

Production is a dominant concern throughout the commercial product 

development process and forces discipline and trade-offs in the design 

process. This environment encourages realistic assessments of risks and 

costs; doing otherwise would threaten the business case and invite failure. 

For the same reasons, the environment places a high value on knowledge 

for making decisions. Program managers have good reasons to want risks 

identified early, be intolerant of unknowns, and not rely on testing as the 

main vehicle for discovering the performance characteristics of the 

product. By protecting the business case as the key to success, program 

managers in leading commercial firms are conservative in their estimates 

and aggressive in risk reduction. Ultimately, preserving the business case 

strengthens the ability of managers to say “no” to pressures to accept high 

risks or unknowns. Practices, such as maturing technologies to high 

readiness levels before inclusion in a program, having 90 percent of 

engineering drawings done by the CDR, and achieving statistical process 

control before production, are adopted because they help ensure success.

The basic management goal for a weapon system program in DOD is 

similar: to develop and deliver a product that meets the customer’s needs. 

However, the pressures of successfully competing for the funds to start and 

sustain a DOD acquisition program make for a much different business 

case. Compared with commercial programs, the DOD environment 

encourages launching product developments that embody more technical 

unknowns and less knowledge about the performance and production risks 

they entail. Although DOD is attempting to create more flexibility in how 

technical requirements are set for programs, a new product development is 

encouraged to possess performance features that significantly distinguish 

it from other systems. Consequently, aspiring DOD programs have 

incentives to include performance features and design characteristics that 

rely on immature technologies. These unknowns place a much greater 
Page 9 GAO/T-NSIAD-99-116 Best Practices

reliance on maturing technology during product development than we 

found on commercial programs. 

Even though less information about a new product development is 

available at the time of launch, the competition for funding requires 

detailed projections to be made from what information does exist. A 

product development deemed worthy cannot be launched unless the 

program’s development and production cost, as well as timing, fall within 

available funding. Because DOD relies largely on forecasts of cost, 

schedule, and performance that are comparatively soft at the time, success 

in funding competition encourages the cost and schedule estimates to be 

squeezed into profiles of available funding. Additional requirements, such 

as high reliability and maintainability, serve to make the fit even tighter. As 

competition for funding will continue throughout the program’s 

development, success is measured in terms of ability to secure the next 

installment.

Untempered by knowledge to the contrary, the risks associated with 

developing new technologies together with the product within tight 

estimates are deemed acceptable. Production realities, critical to matching 

technological capabilities with customer requirements on commercial 

programs, are too far away from the DOD launch decision to have the same 

curbing effect on technology decisions. Thus, the environment for 

managing weapon system programs is a particularly difficult environment 

for managing technology development. The ups and downs and resource 

changes associated with the technology discovery process do not mesh 

well with a program’s need to meet cost, schedule, and performance goals. 

Problems with developing technologies, which are to be expected, can 

actually threaten the support for a program if they become known.

These pressures and incentives explain why the behavior of weapon 

system managers differs from that of managers of commercial product 

developments. Problems or indications that the estimates are decaying do 

not help sustain funding support for the program in subsequent years, and 

thus, their admission is implicitly discouraged. An optimistic cost estimate 

makes it easier to launch a product development and sustain annual 

approval; admission that costs are likely to be higher could invite failure. 

Rewards for discovering and recognizing potential problems early in a DOD 

product development are few. Less available knowledge makes it harder 

for program managers to say “no.” In contrast with leading commercial 

firms, not having attained knowledge—such as on the full performance of a 

key technology or the true risks facing manufacture—can be perceived as 
Page 10 GAO/T-NSIAD-99-116 Best Practices

better than knowing that problems exist. For these reasons, the practices 

associated with managing to knowledge points—such as applying 

knowledge standards for technology, design, and production maturity—are 

not readily adopted in DOD.

Our ongoing work on technology inclusion shows that in addition to the 

incentives provided by the acquisition environment, some structural and 

budgetary impediments exist that make it difficult for technology to be 

matured to a high level of readiness before being included in a DOD 

weapon system development. First, it is not clear that organizations 

exist—other than program offices—that have the role to take technology 

from a readiness level of three or four, where new technologies are often 

put into a program, to the level seven needed to be readily included in a 

product development. We found that for the DOD cases in which high 

technology readiness levels were attained before program launch, 

organizations stepped in and played atypical roles in bridging the gap from 

science and technology to product development. Second, budget 

realities—the fact that programs attract much higher levels of funding than 

science and technology projects—make programs a favored venue for 

funding technology development through the higher and more expensive 

readiness levels. Third, if science and technology organizations became 

responsible for managing technology development to the higher levels, 

they most likely would need additional funding. 

These observations about the differences between the commercial and 

DOD environments should not be interpreted to mean that commercial 

managers are somehow more skilled or knowledgeable than their DOD 

counterparts. Nor do DOD program managers act irrationally. They see 

the acquisition of the weapons under their purview as aligned with national 

interests, and they do what they believe is right, given the pressures they 

face. All of the numerous participants in the acquisition process play a part 

in creating these pressures. In fact, the weapon systems acquisition 

process asks much more of DOD program managers than commercial firms 

do. Perhaps they are asked to do too much: develop advanced technology, 

manage product design and production, and champion the program’s cause 

through budget and other decisions over many years. In fact, one 

commercial executive observed that it is unreasonable to expect people to 

focus on a goal (such as production start-up) that is 4 or more years away. 

Commercial program managers are put in a better position to succeed; they 

have to worry only about product design and production within the cost, 

schedule, and performance demands of the business case. 
Page 11 GAO/T-NSIAD-99-116 Best Practices

Charting a Course for 

Adopting Best 

Practices to Get Better 

Outcomes

Commercial practices for gaining knowledge and assessing risks can help 

produce better outcomes on weapon systems. Collectively, such individual 

outcomes will help attain DOD’s modernization goals and improve funding 

stability for programs. For such practices to work, however, the 

knowledge they produce must help a DOD program succeed in its 

environment. Thus, the DOD environment must become conducive to such 

practices. At least two factors are critical to fostering such an 

environment. First, program launch decisions must be relieved of the need 

to overpromise on performance and resource estimates. The pressure to 

amass broad support at launch creates incentives for new programs to 

embrace far more technology development than commercial programs do. 

Separating technology development so that it does not have to be managed 

within the confines of a weapon systems program would go a long way to 

relieving this pressure. The objectives of technology development, as well 

as what is demanded of knowledge and estimates, differ from those of 

product development. Clearly, DOD has to develop technology, particularly 

the technology that is unique to military applications. However, by 

separating technology development from weapon programs, DOD could 

insist on higher standards for knowledge on its programs and get better 

results when those programs transition to production.

Second, once a program is underway, the participants in the acquisition 

process must make it acceptable for managers to identify unknowns as 

high risks so that they can be aggressively worked on earlier in 

development. Commercial firms insist on knowledge measured against a 

criterion for assessing risk. Firms then make decisions to preserve the 

business case by eliminating risks. The result is discipline provided from 

within. We believe that if the Congress and DOD weighed program launch 

decisions and subsequent progress on weapon systems by applying a 

common set of knowledge standards, like those gleaned from leading 

commercial firms, they could create a better “business” case for weapon 

systems. By developing technology separately to high readiness levels 

before including it in a program and by adhering to knowledge standards in 

product development, DOD program managers can be put in a better 

position to succeed in the timely design and production of weapon 

systems. 

The real test of the participants’ resolve to get better outcomes by applying 

best practices will be the decisions made on individual weapon systems, 

such as for launch and funding. These decisions define what success 

means in DOD and what practices contribute to success. Decisions made 
Page 12 GAO/T-NSIAD-99-116 Best Practices

by DOD or the Congress to advance or fund programs that do not have 

enough knowledge to meet agreed-upon standards send signals to 

managers that not having the necessary level of knowledge is acceptable. 

On the other hand, decisions to not start new programs that need 

technology advances to meet unforgiving requirements or to recognize 

early that a change in a program is necessary to attain desired knowledge 

levels merit support.

Managing technology development differently and applying knowledge 

standards to both technology development and product development will 

have implications for organizational roles and budgeting. For example, if 

DOD were to attempt to develop technologies to higher readiness levels 

before including them in a product development, then organizations other 

than weapon system program offices will have to be made responsible for 

bridging the gap from the traditional science and technology role to the 

redefined program manager’s role. Likewise, the research and 

development funding attendant to those bridging responsibilities may have 

to be budgeted and accounted for differently. This does not necessarily 

mean more research and development money is needed in the aggregate. 

Rather, taking the foregoing actions could actually lower costs in the long 

run, thereby freeing funding for other needs.

The best practices we have described and the changes needed to adopt 

them are not concepts that are foreign to DOD. One of the 

recommendations from a 1996 Defense Science Board study called for 

DOD to aggressively pursue high-risk technology before inclusion in a 

weapon research and development program.1

 We found several 

instances—such as the photonics mast for the Navy’s New Attack 

Submarine, propulsion and related technologies for the Marine Corps 

Advanced Amphibious Assault Vehicle, and the Air Force’s Integrated High 

Performance Turbine Engine Technology Program—where DOD has put 

the organizations and funding in place to bring technologies to a high level 

of readiness before they were included in programs. Another program of 

interest is the Army’s Future Scout and Cavalry System, which is being 

managed as an Advanced Technology Demonstration, a DOD initiative to 

demonstrate immature technologies so they can be more easily 

incorporated into a product development. Several other DOD initiatives, 

1

A Streamlined Approach to Weapon Systems Research, Development, and Acquisition: The

Application of Commercial Practices, a Report from the Defense Science Board Task Force on Defense 

Acquisition Reform, May 1996.
Page 13 GAO/T-NSIAD-99-116 Best Practices

like cost as an independent variable and integrated product teams, are 

attempting to draw lessons from commercial practices.

Ongoing Work of 

Interest to the 

Subcommittee

The Subcommittee also asked for information on several assignments we 

have underway. Most of this work deals with DOD initiatives related to 

acquisitions. Summaries follow.

Acquisition Workforce 

Training in Best Practices

DOD has a number of reform initiatives that are related to the best 

practices that we have reported are possible to adopt in the DOD 

environment. At the request of this Subcommittee, we are evaluating the 

role of DOD’s training in getting best acquisition practices applied to 

weapon system programs and whether training could be improved. 

We have focused our work on weapon system program offices because 

they are where the practices are applied. We selected six program offices 

cited as having been successful in applying one or more of the following 

practices: cost as an independent variable, past performance, performance 

specifications, integrated product teams, and supplier relations. We are 

assessing how DOD’s training supported their use of the practices. For 

comparative purposes, we are meeting with four leading commercial firms 

noted for their excellence in workforce training. We expect to issue a final 

report in August 1999.

Best Practices for Test and 

Evaluation of Weapon 

Systems

At the request of this Subcommittee, we have recently begun reviewing 

best practices as they can be applied to the test and evaluation of weapon 

systems. The objectives of this work are to determine whether (1) the 

testing practices of leading commercial firms offer improvements to DOD’s 

testing practices on weapon acquisition programs, (2) a particular area of 

best testing practices stands out as a leverage point that could offer a 

significant improvement for weapon acquisitions, (3) the role or purpose of 

testing in best commercial product developments shape actual testing 

practices and to what extent testing plays a different role in major weapon 

acquisitions, and (4) obstacles hinder the implementation of best practices 

on DOD weapon acquisition programs. We plan to complete the design 

phase of this assignment in June 1999.
Page 14 GAO/T-NSIAD-99-116 Best Practices

Pricing of Sole-Source 

Commercial Items

DOD, with the support of the Congress, is increasing its use of 

commercially available products and services. While the current level of 

commercial purchasing is relatively small and sole-source purchases even 

smaller, we expect commercial purchases to increase in the future. We are 

currently reviewing the price analysis tools DOD contracting personnel use 

to arrive at fair and reasonable prices for commercial sole-source items and 

the guidance and training available to assist them in determining price 

reasonableness. In March 1998, we reported our preliminary observations 

on the pricing of commercial sole-source spare parts during testimony 

before the former Subcommittee on Acquisition and Technology.2

 

Currently, we are nearing completion of that work and plan to issue our 

report to you in June. Our tentative findings are summarized below.

The current contracting environment for sole-source commercial items 

presents negotiating challenges for DOD contracting personnel. While the 

Federal Acquisition Regulation grants DOD contracting officers wide 

latitude on the type and extent of price analysis techniques they can use, 

they are, nevertheless, required to perform sufficient price analyses to 

determine whether offered prices are fair and reasonable. Our review of 

the price analyses they perform found a number of weaknesses. For 

example, some contracting personnel accepted initially offered prices 

because they had a misperception that if the offered prices were the same 

as the catalog or list price, the offered prices were fair and reasonable. 

Some contracting personnel did not seem to use pertinent contract file 

information on historical pricing in their price analysis. Others did not 

appear to understand the makeup of catalog prices and paid prices that 

were based on rapid delivery service when it was not needed because the 

purchases were for restocking inventories. 

DOD continues to provide training and guidance to assist contracting 

personnel in understanding the requirements of a sound price analysis in a 

commercial contracting environment. In addition, recent legislation 

requires increased guidance for contracting personnel on price analysis 

tools, the appropriate use of information other than certified cost or pricing 

data, and the role of support agencies. In time, as more contracting 

personnel are trained and additional guidance is provided, we expect to see 

improvement in the quality of price analyses for sole-source commercial 

items.

2

Defense Acquisition: Improved Program Outcomes Are Possible (GAO/T-NSIAD-98-123, Mar. 18, 1998).
Page 15 GAO/T-NSIAD-99-116 Best Practices

Spare Parts Price Trends Recently, the services have been concerned that the prices they are paying 

for critical spare parts have been increasing above the rate of inflation. As 

a result, we have begun a review of the prices DOD end-users are paying for 

consumable spare parts and repairable items and the change in these prices 

over time relative to inflation. Where there are significant increases over 

and above the rate of inflation, we will attempt to ascertain the factors that 

most contribute to those increases.

Cost Accounting Standards 

Board 

The Congress asked us to establish a panel of experts to review and make 

recommendations regarding the Cost Accounting Standards (CAS) Board 

and the CAS system in light of recent procurement reforms. More than

25 years ago, the Congress established the CAS Board to protect the 

government from certain risks inherent in cost-based contracts and to 

improve communications between the government and contractors with 

regard to those contracts. Cost-based contracts continue to represent the 

majority of all federal contracting dollars. The CAS Board Review Panel is 

expected to recommend that there is a continuing need for the CAS Board.

The Review Panel is also expected to recommend reforms to encourage the 

participation of new commercial companies in government procurement 

and to reduce the burden of government unique accounting system 

requirements on smaller companies. The Review Panel’s report is expected 

to be released within the next few weeks with its recommendations aimed 

at significantly reducing the burdens and costs of the CAS system without 

diminishing its benefits. 

Other Transactions In 1993, the Congress passed section 845 of the National Defense 

Authorization Act for fiscal year 1994, which authorized the Defense 

Advanced Research Projects Agency to conduct prototype projects of 

weapons or weapon systems under the authority of 10 U.S.C. 2371. This is 

commonly referred to as “other transaction” authority. An other 

transaction is distinct from other instruments, such as contracts, grants, 

and cooperative agreements, and is generally not subject to the laws and 

regulations that apply to these instruments. Subsequently, section 804 of 

the fiscal year 1997 National Defense Authorization Act extended the 

authority to enter into other transaction agreements to the military 

departments and other DOD components. Other transactions can be used 

even if a contract would be feasible or appropriate. Further, contractors
Page 16 GAO/T-NSIAD-99-116 Best Practices

are not required to share in the costs of the agreements. DOD’s authority 

under section 845 is to expire on September 30, 2001.

In response to a Subcommittee request, we are determining (1) the extent 

to which DOD has used its section 845 authority, (2) why other transactions 

were selected as the procurement instrument, and (3) the extent to which 

other transactions address key areas of risk normally covered by Federal 

Acquisition Regulations. The scope of our review includes all other 

transactions that DOD entered into under the authority of section 845, as 

amended, between fiscal years 1994 and 1998 and two other transactions 

that were awarded in October 1998 for the Evolved Expendable Launch 

Vehicle. Our concerns on the potential risks of using other transactions for 

that program were discussed in a prior report.3

 We will issue our report 

this summer.

Governmentwide 

Information Technology 

Contracts

We were asked to examine selected governmentwide agency contracts, 

typically for various information technology resources, to determine 

whether competition requirements were being met. In 1994, the Congress 

directed that agencies consider awarding these task and delivery order 

contracts to multiple firms—rather than a single firm—to provide for 

competition in ordering.4

 Federal agencies are to provide each of the 

multiple contractors a fair opportunity to be considered for orders placed 

under the contracts.

In September 1998, we reported that efforts to provide a fair opportunity—

and thereby promote competition—varied among the six organizations 

reviewed.5

 Four agencies had experienced difficulty obtaining 

competition, while two had achieved consistent competition. Because 

multiple award task and delivery order contracts are a relatively new 

contracting mechanism, and because of the large size of some orders that 

had been awarded pursuant to the new authority, we were requested to 

continue a review of the implementation of these provisions. We have 

3

Evolved Expendable Launch Vehicle: DOD Guidance Needed to Protect Government’s Interest

(GAO/NSIAD-98-151, June 11, 1998).

4A task- or delivery-order contract provides for an indefinite quantity, within stated limits, of supplies or 

services to be furnished during a fixed period, with deliveries or performance to be scheduled by 

placing orders with the contractor.

5

Acquisition Reform: Multiple-Award Contracting at Six Federal Organizations (GAO/NSIAD-98-215, 

Sept. 30, 1998)
Page 17 GAO/T-NSIAD-99-116 Best Practices

selected four organizations that have awarded multiple award contracts for 

information technology products and services that are available for 

governmentwide use. Our work will provide a perspective on how agencies 

ensure contractors a fair opportunity to be considered for interagency 

orders.

Three of the four organizations were included in our prior work and have 

attempted to strengthen their processes for providing contractors a fair 

opportunity. The Department of Transportation—which sometimes 

accepted unconvincing rationales for placing sole-source orders—now 

provides customers placing interagency orders specific guidance about 

acceptable rationales to justify sole-source orders. The National Institutes 

of Health—which had identified preferred contractors when announcing 

plans to place orders—has eliminated any reference to a preferred 

contractor from such announcements. In addition, a proposed change to 

the governmentwide procurement regulations would prohibit designating 

preferred contractors. Finally, the Defense Information Systems Agency—

which had not required that all multiple award contractors be notified of 

planned competitive orders—modified its procedures to require that all 

contractors be notified of orders.

Although our current review is now in its initial phases, our preliminary 

work shows that competition for the largest orders under multiple award 

contracts has not been routinely achieved. During our work, we will 

identify factors that might have deterred contractors from competing for 

these orders and assess how changing agencies’ fair opportunity processes 

might broaden competition for orders.

This concludes our statement. We appreciate the opportunity to have it 

placed in the record. 
Page 18 GAO/T-NSIAD-99-116 Best Practices

Appendix I

GAO Products on Best Practices Applicable to 

Weapon Acquisitions ApenIxdi

Best Practices: Commercial Quality Assurance Practices Offer

Improvements for DOD (GAO/NSIAD-96-162, Aug. 26, 1996).

Major Acquisitions: Significant Changes Underway in DOD’s Earned Value

Management Process (GAO/NSIAD-97-108, May 5, 1997).

Best Practices: Successful Application to Weapon Acquisitions Requires

Changes in DOD’s Environment (GAO/NSIAD-98-56, Feb. 24, 1998).

Best Practices: DOD Can Help Suppliers Contribute More to Weapon

System Programs (GAO/NSIAD-98-87, Mar. 17, 1998).

Defense Acquisition: Improved Program Outcomes Are Possible

(GAO/T-NSIAD-98-123, Mar. 18, 1998).

(707408) Lert
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the 

following address, accompanied by a check or money order made 

out to the Superintendent of Documents, when necessary, VISA and 

MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are 

discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 37050

Washington, DC 20013

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000

or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and 

testimony. To receive facsimile copies of the daily list or any list 

from the past 30 days, please call (202) 512-6000 using a touchtone 

phone. A recorded menu will provide information on how to obtain 

these lists.

For information on how to access GAO reports on the INTERNET, 

send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at: 

http://www.gao.gov
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. GI00