United States General Accounting Office

GAO Report to the Subcommittee on

Acquisition and Technology, Committee

on Armed Services, U. S. Senate

August 1996 BEST PRACTICES

Commercial Quality

Assurance Practices

Offer Improvements

for DOD

G O A

years

1921 - 1996

GAO/NSIAD-96-162

GAO United States

General Accounting Office

Washington, D.C. 20548

National Security and

International Affairs Division

B-272039 

August 26, 1996

The Honorable Robert C. Smith

Chairman

The Honorable Jeff Bingaman

Ranking Minority Member

Subcommittee on Acquisition and Technology

Committee on Armed Services

United States Senate

The Department of Defense (DOD) spends about $1.5 billion extra per year

on military-unique quality assurance requirements for major acquisitions

and billions more on cost and schedule overruns to correct problems

caused partly by poor quality practices. To help improve DOD’s quality

assurance program, we reviewed world-class commercial organizations to

determine what practices they had adopted to more efficiently produce

quality products. Specifically, this report describes (1) the historical

problems DOD has had in improving quality assurance practices, (2) some

private sector practices that could benefit DOD, and (3) a current plan for

improving quality assurance activities.

Background Quality assurance has a simple goal: to ensure that products perform the

way they are supposed to. For many years, the traditional way DOD and

commercial companies achieved quality was through systematic final

inspection. But now, intense competition has led some U.S. companies to

adopt total quality management practices that are prevention based.

Consequently, quality assurance has taken on a broader meaning, to

include virtually all key design and engineering elements during

development, the transition to production, and production itself.

There is general agreement across government and industry that DOD’s

inspection-based quality assurance practices have added unnecessary

costs to acquisitions because they require DOD and contractor personnel

and resources for oversight that are separate from the production process.

Until recently, DOD’s quality requirements were based on MIL-Q-9858A, a

military standard established in 1963. This standard requires a contractor

to establish a quality program with documented procedures and processes

that are subject to approval by government representatives throughout all

areas of contract performance. Quality is theoretically ensured by

requiring both the contractor and the government to monitor and inspect

products. In June 1994, the Secretary of Defense announced that

Page 1 GAO/NSIAD-96-162 Best Practices
B-272039 

commercial quality standards such as ISO-9000 should replace MIL-Q-9858A

where it makes sense. DOD will not require MIL-Q-9858A on contracts

awarded after October 1996. Until then, DOD will be approving the quality

standards on a case-by-case basis at the request of industry.

The Defense Contract Management Command (DCMC) has primary

responsibility for setting and overseeing quality assurance standards

within DOD. As of September 1995, DCMC had about 5,000 quality specialists

located in contractor facilities or in regional offices across the country.

Results in Brief Based on studies performed for DOD, we estimate that it spends more than

$1.5 billion annually beyond what is necessary to support its quality

assurance approach. Despite this outlay, it has had long-standing problems

with significant cost and schedule overruns that have been needed to

correct manufacturing and quality problems on weapon system programs.

Historically, numerous acquisition programs have had quality problems in

production because designs were not complete. More recently, the B-2

bomber program and the C-17 Airlifter program encountered major

manufacturing problems because they went forward with unstable designs

and relied on inspections to find and rework defects once in production.

Nonvalue-added costs have increased in part because DOD has taken a

narrow approach to implementing its quality standard.

On the other hand, a number of successful commercial manufacturers

have adopted a dramatically different approach. Driven by the competitive

imperative, they have significantly improved quality in their products,

while reducing oversight and inspection costs. The striking difference

between the way DOD’s weapon system programs and world-class

companies practice quality assurance is that the latter defines quality

assurance much more broadly, making it an integral part of the entire

process from development through production to sales. Several key

techniques are common to this approach: (1) focusing on achieving robust,

producible designs before production begins by requiring communication

between key players; (2) using process controls to design products and

control the production process as it occurs; and (3) establishing programs

with key suppliers to ensure the quality of incoming material. The

manufacturers we visited reported that these techniques have helped

reduce defects from 34 to 90 percent and the number of inspectors from 

25 to 94 percent.

Page 2 GAO/NSIAD-96-162 Best Practices
B-272039 

In the past 2 years, DOD has developed policies and procedures that reflect

a broader approach to quality assurance. They are based on teaming with

the contractor to control processes while reducing reliance on inspection.

DOD faces a challenge in changing the inspection-based quality practices

that have become ingrained, particularly when it comes to placing a

greater emphasis on building quality into weapon systems during design.

DOD and industry have joined in an initiative to develop a plan that, if

implemented, could reduce the costs of redundant quality assurance

processes. According to the plan proposed by the Government and

Industry Quality Liaison Panel, DOD would allow its contractors to use a

single quality management system based on process controls for all of its

contracts, military and commercial.

We believe the results of this system could be enhanced if DOD also

implemented plans it has developed for some of the advanced quality

concepts we found in the commercial world. Right now, DOD’s Single

Process Initiative is focused on allowing defense contractors to replace

the military standard with a more common commercial one. It must now

encourage the defense industry to use more advanced commercial

techniques, such as design for manufacturing, statistical process control,

and supplier quality programs. Achieving the same results as world-class

companies would also require DOD to consider quality assurance as an

integral part of the entire acquisition process and diffuse responsibilities

accordingly.

Changing DOD’s

Quality Assurance

Practices Is

Challenging Given

Long-standing

Problems

DOD faces a formidable challenge in changing its quality assurance culture.

This culture has been characterized by a narrow approach to quality

assurance, in both DOD and the defense industry, which has led to a focus

on detecting defects and recording corrective actions. In the past, DOD’s

practices have reflected a narrow approach to quality assurance.

Responsibility for meeting military quality standard MIL-Q-9858A has been

assigned to one organization—DCMC, which is outside much of the

acquisition decision-making process. DCMC has quality specialists stationed

in contractor plants across the country to inspect material and end items.

These specialists have relied heavily on quality system documentation,

various reviews and audits of the quality process, and corrective action

plans submitted by contractor quality assurance personnel to ensure

quality on weapon systems.

Page 3 GAO/NSIAD-96-162 Best Practices
B-272039 

Based on information from studies performed for the Secretary of

Defense,1

 we estimate that the extra cost associated with military-unique

quality assurance requirements for DOD acquisitions is $1.5 billion annually.

The studies assessed the cost impact of DOD regulations on contractors

and DOD and found that the contractors’ cost to implement MIL-Q-9858A

and to comply with DOD requirements represented at least 1.7 percent of

DOD’s acquisition cost, or about $1 billion. Most of this cost occurred as the

result of contractor quality assurance and operations personnel devoting

time to such activities as preparing quality plans and procedures,

conducting and documenting inspections, documenting deviations,

proposing corrective actions to government concerns, and supporting

government audits and reviews. Further, DOD’s own costs for quality

assurance oversight were about $687 million annually. This estimate does

not include what DOD has spent to correct the manufacturing and quality

problems that have contributed to historical cost and schedule overruns

on weapon system production programs. These past problems occurred

partly because manufacturing processes were not considered during the

design phase of the program, and inspection, rather than process control,

was the predominant method of ensuring quality.

Like many commercial manufacturers, DOD is now being expected to cut

costs and do more with less. Commercial manufacturers have adopted

far-reaching quality strategies as one way to become more competitive,

efficient, and economical. Even though DOD faces the same challenge,

particularly since it can no longer expect the budget increases it had

during the 1980s, it has yet to effectively achieve the same level of

efficiency from its manufacturers. A major difference between DOD and

commercial manufacturers is that DOD has, until recently, maintained its

practice of inspecting rather than designing quality into a product while

world-class companies have broadened their definition of quality to

include design.

DOD is attempting to change its approach to quality by including design in

the definition of quality. In translating this approach into practice, DOD will

have to overcome a history in which many weapon system acquisitions

have encountered significant cost and schedule overruns because of

design and manufacturing problems. These problems usually resulted from

acquisition strategies that began production before the design was

1

DOD Regulatory Cost Premium: A Quantitative Assessment (Dec. 1994) and The DOD Regulatory Cost

Premium — Response to Secretary Perry’s Follow-on Taskings (Sept. 1995), prepared jointly by

Coopers & Lybrand and TASC for Dr. William J. Perry, Secretary of Defense. The studies stated that

the findings provide an adequate basis for assessing the general impact of the DOD regulatory

environment on acquisition cost.

Page 4 GAO/NSIAD-96-162 Best Practices
B-272039 

complete and key manufacturing processes were in place and tested for

capability. Some examples we have reported on follow:

• In 1985, we reported on four major weapon system acquisitions—the

Army’s Copperhead projectile and Blackhawk helicopter, and the Navy’s

High Speed Anti-Radiation Missile and Tomahawk missile—that had

encountered substantial problems in early production, resulting in cost

overruns and schedule delays. At least part of these cost and schedule

overruns were caused by several untried manufacturing processes that

were not studied to see if they could produce quality components.

Symptoms of these problems were high scrap rates, parts shortages, and

changes to engineering drawings.2

• In 1988 and 1990, we reported that the B-2 bomber program’s immature

design led to manufacturing problems in production that, in turn, led to

schedule delays and cost increases. The contractors initiated thousands of

changes to engineering drawings, some of which required new parts and

tooling. Because the bomber was not ready for production, actual labor

hours exceeded planned labor hours on the first three aircraft by 84, 86,

and 94 percent, respectively, and the three major contractors estimated it

would take over 10 million quality assurance labor hours to develop and

produce 21 production aircraft, not including hours estimated for

government oversight. These manufacturing problems delayed the first

flight of the B-2 by 19 months and contributed to significant cost

increases.3

• In 1991, we reported that 12 tactical missile programs had cost and

schedule overruns, partly because program offices had not adequately

considered the risk associated with the weapons’ design, development,

and production. One missile, the Advanced Medium Range Air-to-Air

Missile, encountered problems in transitioning to production partly

because certain electronic components proved too complex and had to be

redesigned. This contributed to cost increases of 285 percent and a 5-year

delay to the missile’s operational capability date.4

• In 1994, we reported that quality problems with the C-17 program

increased unit cost to production aircraft and delayed scheduled

deliveries. Labor hours for rework and repair of production items made up

40 percent of the labor on the first five production C-17s, and scrap,

rework, and repair cost about $44 million in 1993. In addition to these

2

Why Some Weapon Systems Encounter Production Problems While Others Do Not: Six Case Studies

(GAO/NSIAD-85-34, May 24, 1985).

3

Strategic Bombers: B-2 Program Status and Current Issues (GAO/NSIAD-90-120, Feb. 22, 1990).

4

Tactical Missile Acquisitions: Understated Technical Risks Leading to Cost and Schedule Overruns

(GAO/NSIAD-91-280, Sept. 17, 1991).

Page 5 GAO/NSIAD-96-162 Best Practices
B-272039 

costs, some production aircraft were delivered to the Air Force with

unfinished work and known deficiencies that had to be corrected after

government acceptance. The Defense Science Board reviewed the C-17

program in 1993 and concluded that the production schedule could not be

maintained unless the contractor changed its manufacturing and quality

assurance processes.5

 An Industry Review Panel on Specifications and

Standards found that the C-17 production process had many quality

problems that were adding cost to the program. It found that the

production aircraft were being produced in a “development environment,”

with unqualified processes, and with a reactive rather than proactive

quality management system that did not analyze the causes of quality

problems. It also found that a high number of engineering changes were

making production less efficient. The review panel concluded that the

program had an “inspection mentality” and that only 10 percent of quality

cost was being spent on prevention.

These examples represent a persistent problem in DOD’s major

acquisitions. All of the programs began producing weapon systems before

their key characteristics were fully designed and the key processes for

building the system were understood. Some started production before

doing a significant amount of testing to determine if these systems would

perform their required mission. The consequences of beginning production

before completing testing have repeatedly included procurement of

substantial inventories of unsatisfactory weapons that require costly

modifications to achieve satisfactory performance and, in some cases,

deployment of substandard systems to combat forces.6

These examples do not necessarily condemn DOD to a repetition of the past

or prejudge the potential success of DOD’s current efforts to better manage

quality. However, they do underscore the challenge DOD faces in changing

quality assurance practices. The systems in the examples were developed

and produced using inspection-oriented quality assurance practices and

significant DOD oversight. Yet, in each case, we reported that beginning

manufacturing before the design was understood and manufacturing

processes were controlled led to quality problems, cost increases, and

schedule delays. The narrower interpretation of quality assurance that

prevailed at the time may not have included the various design and

engineering elements that played a part in the weapons’ eventual problems

5

Military Airlift: The C-17 Program Update and Proposed Settlement (GAO/T-NSIAD-94-166, Apr. 19,

1994).

6

Weapons Acquisition: Low-Rate Initial Production Used to Buy Weapon Systems Prematurely

(GAO/NSIAD-95-18, Nov. 21, 1994).

Page 6 GAO/NSIAD-96-162 Best Practices
B-272039 

in production. However, the broader view of quality assurance practiced

by leading commercial manufacturers includes these elements and thus

covers a much larger range of responsibilities.

Companies Developed

New Mindset: Prevent

Problems Rather Than

Fixing Them

Many companies have effectively evolved from an inspection-oriented

quality assurance process, and the culture and infrastructure to support it,

to a standard in which quality is an integral part of each stage of the design

and production process.

The difference between how a manufacturer operates on a military

contract versus how it operates on a commercial contract is often

startling. For example, we visited a company that manufactured military

and commercial products with similar specifications and uses; both

manufacturing processes were in the same building. On one side of the

hallway, the commercial process used automation and process control

throughout the production process to continually reduce nonvalue-added

inspections. On the other side of the hallway, the military process included

two large test facilities at the end of the production process, used at least

four test stations, and added about 10 days to the process. Company

officials told us that the military process continued to have 100-percent

end-item inspection, even though the quality of these products was not in

question.

In response to increased competition in the 1980s, companies had to

dramatically improve quality while reducing cost. They accomplished this

by shifting paradigms. Rather than focusing on identifying and correcting

problems, they began trying to focus on preventing the problems. Quality

assurance changed from being a postmanufacturing step, done at the end

of each process, to being part of the process itself. This was a significant

culture change for these companies. Figure 1 illustrates the shift in

paradigms from inspection-oriented to prevention-oriented quality

practices.

Page 7 GAO/NSIAD-96-162 Best Practices
B-272039 

Figure 1: Detection Versus Prevention-Based Quality Strategies

Suppliers Outgoing 

inspect

Incoming

inspect

Original

equipment

manufacturer

Outgoing

inspect

Incoming

inspect Customer

Rework

Scrap

Rework

Scrap

Rework

Scrap

Rework

Scrap

Detection-based, after-the-fact

Relationship

gaps

Relationship

gaps

Design

Suppliers

Relationship

building

Identify key

characteristics

and process

Original

equipment

manufacturer 

Relationship

building

Customer

Process control

quality function 

during the 

manufacturing 

process

Prevention-based quality

Traditional quality assurance techniques relied upon many after-the-fact

inspections, increasing costs in time and money. To remain profitable,

manufacturers switched from detection to prevention-based quality

Page 8 GAO/NSIAD-96-162 Best Practices
B-272039 

strategies. These strategies teamed suppliers, manufacturing staff, and

engineers to design quality into the product and to identify and control key

characteristics. Prevention-based process control replaced end-item

inspections.

To reduce their costs and gain an edge on the competition, companies

have found that they have had to not only establish a basic commercial

quality system, such as ISO-9000, as their baseline, but also consistently

exceed it. Most of the companies we visited had obtained ISO-9000

certification,7

 a basic quality management system that commercial

customers are beginning to require. This standard ensures that a

manufacturer has a well-documented commercial quality system. In

addition, they all had developed or adopted advanced quality concepts that

brought the concept of quality to the design phase of a product’s life. They

began by using two advanced concepts called design for manufacturing

and process control techniques. After a company had successfully

incorporated each technique, it began to eliminate inspections—and the

cost associated with them—and significantly reduced the amount of

defects in its products. A third ingredient for success involved developing

relationships with key suppliers, which ensured that parts and

subcomponents were appropriate for a product’s design and arrived for

production at consistently high quality and expected cost. Companies we

visited had dramatic reductions in product defects—ranging from 34 to

90 percent—resulting from these techniques. Appendixes II and V contain

detailed examples.

Design for Manufacturing

Controls Product

Development

Design for manufacturing represents a culture change that involves a

whole new mindset. Rather than continuing the old practice of engineers

designing a product in isolation and then handing it off to the

manufacturing process, design for manufacturing involves all stakeholders

in the process. The stakeholders form cross-functional teams that include

representatives from the customer, marketing, research and development,

engineering, manufacturing, key suppliers, quality assurance, finance, and

customer support. Under the old practice, a product would require many

design changes in full-scale production, creating additional defects,

rework, and scrapped material. But now, the teams identify the

requirements for a product’s performance and ensure that manufacturing

processes are in place to meet those requirements within specified cost

7

ISO-9000 is the commercial standard for quality assurance. In the United States, the recognized

certifying body is the Registrar Accreditation Board. Independent, certified quality consultants

approved by the Board conduct on-site audits of commercial applicants and give registration approval

to those passing the audits.

Page 9 GAO/NSIAD-96-162 Best Practices
B-272039 

targets before production begins. Their objective is to build quality in up

front rather than fix problems during some stage of the manufacturing

process or discover there is no profitability. These cross-functional teams

conduct phased development processes to ensure that a product’s design

is producible and profitable. Continuous communication between the

engineers who design the product and the people responsible for

manufacturing it is a key to the process.

Using design for manufacturing techniques, these companies review

projects to prevent a potentially unprofitable design from entering

full-scale production. Teams use modeling and prototyping to determine

the capability of existing manufacturing processes. As the development

process continues, team members must make trade-offs between a

product’s performance and its cost to meet strategic targets. For example,

Texas Instruments’ semiconductor facility reviews a product’s potential

profitability at each milestone of a five-phased development process. If

any of these reviews indicate that cost targets cannot be met, the product’s

development can be terminated. This eliminates additional investment in

full-scale production tooling and facilities. Likewise, Varian Associates’

managers are given cost and schedule targets during the design phase, and

special “out-of-bounds” reviews are held for behind-schedule or

over-budgeted items throughout the development process. Figure 2

provides a conceptual model of the design for manufacturing process

based on what we observed at these two companies.

Page 10 GAO/NSIAD-96-162 Best Practices
B-272039 

Figure 2: Design for Manufacturing Model

0

Evaluate

concept

Informal profit analysis done at every step

Program 

manager

Marketing

Customer

R&D

Design

engineers

Key 

players

3

Concurrent

engineering

Manufacturing

and others

4

Capability

study

5

Production

Integrate design and

manufacturing, and 

consider

support structure and

process necessary to

service the product

Conduct formal profit 

analysis, conduct

manufacturing 

5-sigma

capability study to 

determine if 

processes are in 

control

Commit 

funding and

build plant, 

tools

Quality

Manufacturing

2

Develop

model

Rapid control

prototype 

customer

involvement

Design

engineers

Key suppliers

Program 

managers

1

Define 

and

plan

Understand

 product

requirements, 

translate

them into utility 

functions, and

complete commitment 

of resources

Design

engineers

Program managers

Marketing

R&D

Customer

Phase 0 Phase1 Phase 2, 3 and 4 Phase 5

Profit?

No.

Stop. 

Profit?

Yes.

Go. 

Process Controls Help to

Ensure Consistently

High-Quality Products

Process control means controlling the production process by checking the

quality while the work is being done. Beyond that, companies rely on final

inspections of completed lots. Leading companies rely on “total process

control,” which demands that every process is controlled by checking the

Page 11 GAO/NSIAD-96-162 Best Practices
B-272039 

quality during production. Rather than employing a lot of inspectors,

however, they entrust the production workers to do it themselves.

The companies we visited had implemented similar systems. In addition to

a basic quality system, such as ISO-9000, and an emphasis on including

design as an element of the quality process, we found they (1) diffused

responsibility for quality across the production line and (2) trained

employees to use analytical diagnostic tools such as statistical process

control, process mapping, or continuous flow manufacturing to maintain

predictable processes.

The companies had developed or adopted advanced quality concepts that

went beyond the basic ISO-9000 standard and that emphasized finding the

cause of quality problems by gathering data and then eliminating those

causes from the manufacturing process. Once root causes are discovered

and processes are controlled, the likelihood of consistently high-quality

products is significantly increased and the need for inspection at the end

of the production line is reduced. For example, Cherry Electronics uses

QS-9000, an advanced quality guide developed jointly by General Motors,

Ford, and Chrysler as a required supplement to ISO-9000 for their suppliers.

It includes comprehensive instructions for implementing design for

manufacturing in new products and using process controls to reduce

defects in final products. Cherry is undergoing certification for QS-9000

now and plans to require its own suppliers to become certified in QS-9000

as well. Also, Motorola and Varian Associates used the Malcolm Baldrige

National Quality Award criteria8

 to create advanced quality systems for

themselves and their suppliers.

Representatives from both companies credit the use of these advanced

quality guidelines for reducing defects while eliminating the need for

end-item inspection. For example, Varian’s Nuclear Magnetic Resonance

Instruments business unit heavily inspected both its product and its

suppliers’ components coming into final assembly until the mid-1980s. At

that time, it instituted process control and gradually reduced the number

of its inspectors by 92 percent, from 26 to 2. Similarly, John Deere

eliminated its quality assurance department by dividing the production line

8

The Malcolm Baldrige National Quality Award’s purposes are to promote awareness of quality

excellence, to recognize quality achievements of U.S. companies, and to publicize successful quality

strategies. The award criteria examines leadership, information and analysis, strategic planning,

human resource development and management, process management, business results, and customer

focus and satisfaction. Heavy emphasis is placed on demonstrated business excellence and quality

achievement.

Page 12 GAO/NSIAD-96-162 Best Practices
B-272039 

into “focused factories” and giving the inspection responsibility directly to

each product manufacturing team.

Suppliers Become Part of

the Team

Material from suppliers typically represents from 60 to 85 percent of the

final product cost at the companies we visited. Because of this, companies

focused on improving supplier quality so they could eventually reduce

material defects and inspections. They began by reducing their number of

suppliers. In addition, they created qualification and certification

procedures that relied on periodic evaluations of supplier quality systems.

Finally, they developed long-term relationships with valued suppliers,

increasing communication and creating a partnership, when possible.

These practices helped companies reduce suppliers’ defects by as much as

90 percent and inspections of incoming material by as much as 76 percent.

Appendix III describes the results of our visits to specific companies that

had implemented some of these supplier quality programs.

Commercial companies significantly reduced the number of

suppliers—from 50 to 85 percent—to manage the cost and quality of

incoming material more closely. Generally, a reduction in the supplier

base eliminates poorer performers, increases the importance of top

performers, and allows closer cooperation with suppliers in continuous

improvement practices and in new product development. To help reduce

the number of suppliers, companies generally have a qualification and

certification process. The qualification process typically begins with an

evaluation of each supplier’s current quality system and is often carried

out by small cross-functional teams. For example, Varian’s Oncology

Systems qualifies all new suppliers through an evaluation process that

includes an assessment of suppliers’ defect rates, delivery performance,

and a review of their internal quality systems.

Once a company determines that a supplier has processes in place to

guarantee continued quality, it eliminates inspections and depends on

periodic supplier quality information and reviews, generally not more than

once annually. For example, Motorola uses its Quality System Reviews to

assess the suppliers’ quality system every 2 years and score the supplier on

such items as process controls and its ability to develop new products.

The reviews are accomplished by a team of four or five quality

management experts and take 2 to 3 days. The review guidelines use a

scoring approach patterned after the Malcolm Baldrige National Quality

Award criteria and include audit guidelines to determine how well a

supplier controls quality in new product development and deploys good

Page 13 GAO/NSIAD-96-162 Best Practices
B-272039 

process controls. Both Motorola and Varian provide consulting teams and

optional training courses to certified suppliers who either need or request

assistance.

Most of the companies had strategies to enter into long-term relationships

with world-class, high-quality suppliers, resulting in reduced inspection,

planning, rework, and contracting costs. As suppliers progress in meeting

cost and quality goals with companies, they are included on design teams,

given schedule forecasts for future orders, and included in the company’s

quality training program. In addition, longer term contracts typically help

to reduce costs further and encourage valued suppliers to deliver material

more efficiently. When companies introduce new products, valued

suppliers are given the key characteristics of the part they are being asked

to supply and then use this information to make sure their manufacturing

processes are capable of making the component with consistently high

quality.

DOD Moves to

Improve Quality

Assurance Practices

Beginning in the early 1990s, DCMC implemented Process Oriented Contract

Administration Services (PROCAS) as a method of moving away from

inspection-oriented quality assurance practices and toward process

control. The intent of PROCAS is to team with contractors to identify,

analyze, and manage production processes and reduce the need for

oversight and inspection where it makes sense. In recognition that more

advanced quality concepts are needed, a group of DOD officials and defense

company representatives have joined in an initiative that, if implemented,

could eliminate the costs of redundant quality assurance processes.

According to the plan created by the Government and Industry Quality

Liaison Panel in April 1995, DOD would allow a contractor to use the same

quality management system—based on process controls—for its military

contracts that it uses on its commercial contracts. Related to the panel’s

goals, DOD’s Single Process Initiative, introduced in December 1995, also

supports moving from multiple government-unique management systems

to a single management system common to both commercial and military

contracts. In addition, DOD has established a policy for using integrated

product and process development concepts to integrate all acquisition

activities, from product concept through production and field support.

How well it will support the panel’s goals remains to be seen.

The Liaison Panel’s plan has three overall goals. The first goal is to

develop a single quality system that can meet commercial or military

quality requirements. It envisions a multitier quality management

Page 14 GAO/NSIAD-96-162 Best Practices
B-272039 

framework based on criteria similar to the commercial ISO-9000 standard

that would be recognized by all government and commercial entities. It

begins with a basic quality system that must use process controls from

design to delivery and would be reviewed on a regular basis by a single,

unified government audit team. As a contractor moves to advanced quality

concepts, such as design for manufacturing, and commits to continuous

improvement of all its processes, its advanced quality system would be

reviewed by government and industry representatives. One objective of

this goal is to create a culture that can ensure effective design, production,

and delivery processes without constraining suppliers with a set of

inspection and oversight requirements.

The plan’s second goal is to have government and industry share the most

advanced quality concepts in defining requirements for, designing, and

manufacturing military products. It envisions a system that will encourage

continuous improvement across the industry by identifying the most

advanced methods of ensuring quality, making these concepts available to

all contractors, and helping train personnel in these concepts. DOD and

industry agreed that contractors who can provide evidence of using these

advanced concepts to ensure quality should be given credit during source

selection.

The plan’s third goal is to establish and implement efficient oversight

methods. It envisions the government and industry developing a single set

of criteria to evaluate contractors’ quality management systems. This

criteria should be implemented in a unified evaluation process and should

promote effective and efficient innovation. Most importantly, DOD believes

the criteria and its implementation method should be accepted by all

government customers to avoid inconsistency and duplication of quality

evaluations. The Co-chair of the Government and Industry Liaison Panel

stated that this evaluation criteria would be used by government audit

teams, perhaps from DCMC, in registering contractors’ basic and advanced

quality systems rather than current methods of oversight. Appendix IV lists

the 15 specific tasks that the panel has undertaken to achieve its goals,

their current status, and their projected completion dates.

In December 1995, DOD began the Single Process Initiative, managed by

DCMC, that allows contractors with military contracts to transition their

quality system from MIL-Q 9858A to their best practice, such as a quality

system based on ISO-9000, the basic commercial standard. The response to

date has been slow; as of June 5, 1996, 38 contractors had submitted

proposals to change their quality systems, 5 of which had been approved

Page 15 GAO/NSIAD-96-162 Best Practices
B-272039 

by DCMC. In discussions with government officials, we found that the

biggest reason for this slow response is cultural. The defense community’s

traditional quality assurance practices have been inspection-based; the

newer, more advanced concepts advocate process control. In addition,

some contractor officials believed there is an understandable fear among

DOD employees that changes in the quality assurance strategy will translate

to a loss of jobs. Also, defense contractors do not see the benefit of

implementing quality systems that may result in payment to the

government for savings resulting from more efficient practices.

DOD’s approval of five contractors’ commercial quality systems means that

they are accepted as the basic quality system on all government contracts.

This is a positive step toward introducing more advanced quality concepts

from the commercial world that force quality considerations during the

design phase of an acquisition. However, in agreeing to this change, DCMC

reserved the right to review all quality documentation at any time; perform

any inspections, verifications, or evaluations it deems necessary; review

any supplies from other facilities; and disapprove the quality system or any

portion of it. The method or frequency with which DCMC invokes these

rights will be as important as a change in the military standard. Also, it is

important that DOD continue to move contractors beyond changing basic

quality systems, toward advanced concepts.

According to DCMC officials, DCMC approved changes to commercial quality

systems on 4,255 out of 158,000 existing military contracts—less than

3 percent. It approved ISO-9000 on 398 new military contracts between May

1995 and April 1996. DCMC approves an average of 36,000 contracts per

year, meaning that about 1 percent of new contracts in the past year have

been approved with ISO-9000 as the quality standard.

Recommendations The commercial world has proven that it can design and manufacture

consistently high-quality products by focusing on building quality into the

product’s design, understanding the key characteristics of the product and

the manufacturing processes necessary to build it, training production

personnel to control those processes throughout production, and

instituting quality programs with suppliers based on these same principles.

We saw convincing evidence that these practices improve product quality

and reduce time and labor spent on quality assurance oversight by making

it unnecessary.

Page 16 GAO/NSIAD-96-162 Best Practices
B-272039 

DOD recognizes the benefits of taking a broader approach to quality

assurance. It has made some policy changes and is beginning to implement

new practices. If DOD provides incentives for implementation of the

advanced commercial practices, such as those identified in this report, we

believe it can significantly improve quality, reduce costs of its acquisition

programs, and apply savings to future modernization efforts. However, we

do not believe DOD’s quality assurance culture will change easily. Our

conclusion is based on discussions with DOD and service representatives

and our review of past acquisitions where DOD and we repeatedly

identified unstable design, poor process controls, and poor transition to

production as causes for manufacturing quality problems and made

recommendations that have not been implemented.

Therefore, we recommend that the Secretary of Defense (1) establish

measurable steps to implement and monitor the progress of the

Government and Industry Quality Liaison Panel plan closely;

(2) periodically assess its success in implementing basic standards such as

ISO-9000; (3) develop ways to encourage the adoption of advanced quality

concepts of design for manufacturing, process controls, and supplier

quality programs throughout the defense industry, using commercial

practices as a guide; and (4) as suggested in the plan, establish incentives

for defense contractors to participate, such as providing credit during

source selection for successful implementation of these advanced quality

practices. Such selection criteria could also provide world-class

companies greater opportunities to participate in DOD’s weapons

acquisition programs.

To assist DOD in changing its own quality assurance culture, we

recommend that the Secretary of Defense expeditiously determine who in

DOD’s acquisition community can best oversee the advanced quality

functions used by defense contractors in developing and producing

weapon systems, using commercial practices as a guide in assigning these

functions, and provide all necessary training for any new responsibilities

that DOD personnel need to perform.

Agency Comments

and Our Evaluation

DOD agreed with the intent of the report and stated that it has already

implemented many changes in line with our recommendations. (See 

app. VI.) These include the institution of integrated product and process

teams in revised acquisition policy directives and the development of

metrics to measure improvement in the overall acquisition process. DOD

Page 17 GAO/NSIAD-96-162 Best Practices
B-272039 

stated that it believed that no additional actions were required in response

to our recommendations at this time.

We agree that these and other actions initiated by DOD represent positive

steps in reforming the quality assurance process and are consistent with

the intent of our recommendations. However, time is needed to determine

whether these steps translate into tangible changes in quality assurance

practices on individual programs. As we note in the report, a number of

major acquisition programs over time have failed to include quality

considerations in the design phase. Although the prevailing

standard—MIL-Q-9858A—for these programs allowed some latitude for

interpreting how quality assurance could be carried out, it was the actual

practice—not the guidance—that was more narrowly focused on

inspections. These experiences underscore the challenge DOD faces in

implementing advanced quality concepts.

Scope and

Methodology

To develop information for this report, we interviewed and obtained

documents from officials of the Office of the Secretary of Defense in the

Pentagon and DCMC at Fort Belvoir, Virginia, because of the quality

assurance policymaking responsibilities and initiatives that are ongoing at

that level. We also held several discussions about quality assurance at the

service level. We discussed commercial quality assurance practices with

officials from the following commercial manufacturing organizations:

• Delco Electronics, Milwaukee, Wisconsin;

• John Deere Horicon Works, Horicon, Wisconsin;

• Cherry Electrical Products, Waukegan, Illinois;

• Texas Instruments Lubbock Metal-Oxide Semiconductor, Lubbock, Texas;

• Varian Nuclear Magnetic Resonance Instruments, Palo Alto, California;

• Varian Chromatography Systems, Walnut Creek, California;

• Varian Oncology Systems, Palo Alto, California; and

• Motorola Paging Products, Boynton Beach, Florida.

A detailed description of the companies we visited is contained in

appendix I.

We then developed a data collection instrument that would assist us in

gathering uniform, quantifiable measurements about the techniques these

organizations used to improve operations and the results they

accomplished. We visited these manufacturing organizations, followed the

same agenda with each, and gathered the same data at each organization.

Page 18 GAO/NSIAD-96-162 Best Practices
B-272039 

We also visited Texas Instruments Defense Systems & Electronics, Dallas,

Texas, a defense contractor.9

 In addition, we visited DELCO in Milwaukee,

Wisconsin, to discuss differences between military and commercial

practices in a broader context.

We reviewed literature and various databases provided by the American

Productivity Quality Center’s International Benchmarking Clearinghouse

to identify manufacturing organizations that have shown significant

improvement in quality while reducing oversight functions such as

supplier oversight and end-item inspections. Our discussions centered

around their overall quality plan and the techniques they used to ensure

supplier quality, producibility of new products, and control of their final

assembly processes while reducing nonvalue-added cost.

We performed our review from August 1995 to July 1996 in accordance

with generally accepted government auditing standards.

We are sending copies of this report to congressional committees; the

Secretaries of the Army, the Navy, and the Air Force; the Director, Defense

Logistics Agency; and the Director, Office of Management and Budget. We

will also make copies available to others upon request.

Please contact me at (202) 512-4383 if you or your staff have any questions

concerning this report. The major contributors to this report were Michael

J. Sullivan, Shari A. Kolnicki, Gordon W. Lusby, and Carolyn S. Blocker.

Katherine V. Schinasi

Associate Director

Defense Acquisitions Issues

9

Texas Instruments’ Defense Systems and Electronics develops and assembles missiles for DOD, with

a small percentage of revenues from foreign military sales and subcontracting on DOD contracts. It

won the Malcolm Baldrige National Quality Award in 1992 and was the first defense contractor to do

so. In 1994, the company’s revenues were about $1.73 billion.

Page 19 GAO/NSIAD-96-162 Best Practices
Contents

Letter 1

Appendix I 

Companies We Visited

as Part of the Review

22

Delco Electronics 22

John Deere Horicon Works 22

Cherry Electrical Products 22

Texas Instruments Lubbock Metal-Oxide Semiconductor 22

Varian Chromatography Systems 22

Varian Nuclear Magnetic Resonance Instruments 23

Varian Oncology Systems 23

Motorola Paging Products Group 23

Appendix II 

Description of

Companies’ Process

Control Practices in

Design and

Manufacturing and

Their Benefits

25

Appendix III 

Examples of

Companies’ Supplier

Quality Practices and

Their Benefits

27

Appendix IV 

Status of Tasks in

Developing a Strategic

Plan for Quality to

Support Acquisition

Reform Initiatives

29

Page 20 GAO/NSIAD-96-162 Best Practices
Contents

Appendix V 

Examples of Savings

Using Statistical

Process Control

31

Appendix VI 

Comments From the

Department of

Defense

33

Related GAO Products 36

Figures Figure 1: Detection Versus Prevention-Based Quality Strategies 8

Figure 2: Design for Manufacturing Model 11

Figure V.1: Producing Integrated Circuits Using Proceeds Control

Techniques

31

Abbreviations

DCMC Defense Contract Management Command

DOD Department of Defense

DFARS DOD Federal Acquisition Regulation Supplement

FAR Federal Acquisition Regulation

ISO International Standards Organization

PROCAS Process Oriented Contract Administration Services

NMRI Nuclear Magnetic Resonance Instruments

Page 21 GAO/NSIAD-96-162 Best Practices
Appendix I 

Companies We Visited as Part of the Review

Delco Electronics Delco Electronics, Milwaukee, Wisconsin, manufactures products for both

commercial and military customers. Delco produces avionics circuit

boards and assemblies for several aircraft and light armored vehicles. For

its commercial automotive customers, it produces power train,

fuel-injection, and remote function computers.

John Deere Horicon

Works

John Deere Horicon Works, Horicon, Wisconsin, has worldwide

responsibility for engineering lawn and grounds care products.

Cherry Electrical

Products

Cherry Electrical Products, Waukegan, Illinois, designs and manufactures

a broad range of electrical and electronic components, assemblies, and

systems for the automotive industry. Applications include interior use,

weather-exposed devices, and under-the-hood functions. Cherry Electrical

received the General Motors’ Targets for Excellence Award and two Q1

Preferred Quality Awards from Ford Motor Company’s Diversified

Operations. Cherry Electrical has applied for the Malcolm Baldrige award

twice. The overall sales of the Cherry Corporation in 1994 were over

$339 million.

Texas Instruments

Lubbock Metal-Oxide

Semiconductor

Texas Instruments Lubbock Metal-Oxide Semiconductor, Lubbock, Texas,

designs and manufactures integrated circuits that are packaged and

assembled overseas. Electronic Programmable Read-Only Memory, a chip

that can be used in computers, telephones, automobiles, and other

computerized systems, is Texas Instruments’ best selling item. In 1993, the

overall sales for Texas Instruments Corporation were $8.5 billion.

Varian

Chromatography

Systems

Varian Chromatography Systems, Walnut Creek, California, supplies gas

and liquid chromatography, gas chromatography, and mass spectrometry

instruments as high-precision, high-end analytical tools. Its customers are

pharmaceutical, chemical, environmental, and other research and

developmental laboratories and quality laboratories worldwide. The

company manufactures 11 different product lines, with a total volume

ranging from 200 to 1,500 units sold per year. The corporation’s total sales

in 1995 were over $1.5 billion.

Page 22 GAO/NSIAD-96-162 Best Practices
Appendix I 

Companies We Visited as Part of the Review

Varian Nuclear

Magnetic Resonance

Instruments

Varian Nuclear Magnetic Resonance Instruments (NMRI), Palo Alto,

California, manufactures scientific and medical instruments that

determine the composition of substances through atomical analysis using

nuclear magnetic resonance. Using such instruments, scientists determine

the atomic connectivity, atomic spatial orientation, and dynamic

characteristics of molecules. The analysis is useful for chemical,

biomolecular, material science, and biomedical fields. NMRI’s major

markets consist of three customer bases: academia; private industry,

including pharmaceutical, basic chemistry, biotechnology, and

petrochemical industries; and the federal government. NMRI manufactures

low-volume, high-precision products and sells approximately 250 units

each year. Prices range from $1,000 to $3 million. It was included in

Varian’s corporate application for the Malcolm Baldrige National Quality

Award in 1992, 1993, and 1994.

Varian Oncology

Systems

Varian Oncology Systems, Palo Alto, California, develops and

manufactures linear accelerators as its principal product. Linear

accelerators are high-technology instruments that produce X-rays,

electrons, and other high-energy particles used to treat cancer.

Approximately 30 percent of its remaining business is in computerized

treatment planning and information systems. Principal customers include

universities, hospitals, government institutions, and free-standing clinics.

Varian Oncology’s annual sales are approximately $350 million. Varian

Oncology Systems applied for the Malcolm Baldrige Award in 1995. It has

received other accolades, such as mention in the “Top Ten” plants in

Industry Week in 1993, the California Governor’s Golden State Award for

Quality in the Marketplace in 1994, the Best Factory Award from

Management Today in 1992, and the European Quality Award

Commendation in 1993.

Motorola Paging

Products Group

Motorola’s paging plant in Boynton Beach, Florida, is part of Motorola’s

Messaging Information and Media Sector business unit. This business unit

designs, manufactures, and distributes a variety of messaging products,

including pagers and paging systems, wireless and wireline data

communication products, handwriting recognition software, and

infrastructure equipment, systems, and services. The plant in Boynton

Beach handles the strategic and tactical dealings with suppliers. Motorola

developed a successful six-sigma manufacturing process and was the first

Malcolm Baldrige award winner in 1988. Motorola’s resulting successes

have inspired many other U.S. corporations to use it as a quality

Page 23 GAO/NSIAD-96-162 Best Practices
Appendix I 

Companies We Visited as Part of the Review

benchmark. As a corporation, Motorola’s net sales in 1995 were

$27 billion.

Page 24 GAO/NSIAD-96-162 Best Practices
Appendix II 

Description of Companies’ Process Control

Practices in Design and Manufacturing and

Their Benefits

Texas Instruments Lubbock Metal-Oxide Semiconductor - Lubbock, Texas

The following processes are used to ensure a product’s profitability and producibility before it enters production:

— Profit and quality team is involved throughout the process.

— Manufacturing and design departments work together. 

— Manufacturing staff determine critical parameters to achieve consistent quality while engineers use statistical process control to

 discover whether a parameter remains in control if a change in the process or product occurred.

— Engineers consult with manufacturing staff during the design phase in an effort to prevent critical failures from occurring in

 production.

Process controls have helped Texas Instruments 

— eliminate end-item inspection that no longer adds value and reduce inspectors from 16 to 12,

— reduce manufacturer’s defects by over 68 percent, and

— increase productivity by over 120 percent.

Varian Nuclear Magnetic Resonance Instruments - Palo Alto, California

Varian’s NMRI business unit’s approach is as follows:

— Baldrige Award criteria and ISO-9000 standards are used as the foundation of its quality system.

— Cross-functional teams integrate design, manufacturing, and field service issues during product development. 

— Statistical process control is used to develop statistically significant boundaries within which the product remains functional.

— Software has been instrumental in improving the productivity and accuracy of the testing process.

— NMRI has sought to eliminate inspections by relying on self-testing and continuous improvement.

Process control initiatives have allowed NMRI to

— eliminate its quality assurance department,

— empower on-line operators with in-process control,

— decrease the number of inspectors from 26 to 2, and

— increase productivity by 97 percent over 6 years.

Texas Instruments Defense Systems and Electronics - Dallas, Texas

Texas Instruments Defense Systems and Electronics won the Malcolm Baldrige Award in 1992 with the following approach:

— Design for manufacturing was used in defense programs to reduce part counts, increase opportunities for automation, and simplify

 assembly.

— Multifunctional teams formed during the concept phase of a product’s life cycle remain together throughout the production

 program. 

— Production process is controlled by statistical process control, which identifies the critical parameters.

— Continuous flow manufacturing is used to identify bottlenecks and nonvalue-added costs in production lines.

Since 1991, Defense Systems and Electronics has

— reduced defects by almost 70 percent,

— reduced scrap rates by 50 percent,

— reduced inspectors by 50 percent, and

— increased productivity by about 30 percent.

(continued)

Page 25 GAO/NSIAD-96-162 Best Practices
Appendix II 

Description of Companies’ Process Control

Practices in Design and Manufacturing and

Their Benefits

Varian Oncology Systems - Palo Alto, California

Varian’s Oncology Systems business unit used the Malcolm Baldrige Award criteria as a foundation for its quality strategy. 

— Five-phased product development process uses cross-functional teams.

— Program reviews check progress and readiness, and senior management reviews the project every 6 weeks. 

— Capability of key manufacturing processes is measured before production begins.

— All production processes are mapped to determine nonvalue-added steps.

Using these techniques, Varian was able to

— reduce final test hours of each high-energy linear accelerator from 700 to 200;

— increase the maximum production of these items from 110 to 200;

— reduce significantly, the defects at the end of assembly; and

— reduce inspectors by 94 percent.

Cherry Electrical Products - Waukegan, Illinois

Cherry Electrical Products experienced decreased labor costs and increased quality in its stamping department using statistical

process control. For example:

— Using statistical data, the company diagnosed a problem with the start-up of the machine, compensated for it, and reduced the

 number of inspectors monitoring the process from four to two. Furthermore, roving inspectors in the department and final

 inspection on the end item were eliminated, and scrap and waste were significantly reduced.

— Cherry reduced the number of operators from 24 to 1 on its spring generator machines while increasing quality. Diagnostic,

 analytical problem-solving using statistical process control showed the company’s defects and scrap problems were not a

 result of machine variability as previously thought, but due to variability introduced by numerous operators. Because of

 statistical process control, labor was reduced, quality was increased, and waste was cut. 

— Because of product consistency resulting from process controls, one department was able to reduce its headcount from 20 to 12.

Overall, Cherry, by using statistical process control, has brought its quality processes under control and decreased inspectors by 

78 percent, while reducing defects by 90 percent.

Varian Chromatography Systems - Walnut Creek, California

Varian Chromatography used “Andon,” a Japanese quality process control, to improve its processes. Using Andon, each operator is

empowered to signal a problem to the rest of the manufacturing team. Each station is equipped with red, yellow, and green lights, with

a red light empowering the employee to stop production.

— Operators record any problems found on the production line on an opportunity board. 

— Weekly meetings are held with operators to resolve problems in cross-functional teams.

— Varian cross trains its employees to increase their skill base so that they are able to understand the entire process. 

Varian reduced waste and increased savings through these process control techniques. For example, it

— reduced quality test time by 98 percent, from a 96-hour test to a 2-hour test;

— reduced defects by 34 percent; 

— decreased the number of inspectors by over 75 percent; and

— improved productivity by 39 percent since 1992.

Page 26 GAO/NSIAD-96-162 Best Practices
Appendix III 

Examples of Companies’ Supplier Quality

Practices and Their Benefits

Varian Chromatography Systems - Walnut Creek, California

Varian Chromatography Systems’ supplier quality practices are to

— concentrate its business with a few, best-qualified suppliers;

— conduct just-in-time manufacturing with them, as part of its Value Managed Relationships program;

— perform zero-receiving inspection and not stock the suppliers’ parts; instead, components from suppliers are coordinated daily,

 and the supplier’s pretested, unboxed parts are delivered directly from the truck to Varian’s manufacturing assembly line;

— provide suppliers a total year goal and give a rolling 12-month forecast instead of the company preordering its material in a

 traditional “push” system; 

— give copies of its master production schedule to suppliers; and 

— implement a certification process whereby all new components must pass first article inspection and three defect-free lot inspections

 prior to approval. In addition, approved suppliers are subject to annual review and quarterly feedback reports.

Through its supplier quality program,

— inventory has been reduced by 68 percent;

— supplier defects have decreased 75 percent;

— suppliers were reduced by 78 percent, down from over 2,000 to approximately 440; and

— suppliers are kept for “life” as long as they provide the technological capabilities, and some have been supplying the company for

 as long as 30 years, with many over 10 years.

Varian Oncology Systems - Palo Alto, California

Varian Oncology Systems’ supplier quality practices are to

— deliver 100-percent quality parts directly to the factory floor;

— eliminate receiving inspections for certified suppliers and conduct periodic on-site reviews;

— assist any certified supplier experiencing quality or delivery problems with supplier corrective action teams;

— provide annual performance reports, periodic report cards, and performance awards as feedback to its suppliers;

— require certified suppliers to maintain competitive prices, high quality levels, and excellent delivery performance to retain

 certification; and

— provide certified suppliers with long-term contracts, access to Varian training in process controls, and improved schedule visibility.

Oncology Systems conducts business with 45 certified suppliers and estimates that these relationships have saved $3.3 million over

the past 3 years due to eliminated inspections, reduced planning and purchasing requirements, and reduced defects. 

It regularly invites suppliers to join continuous process improvement teams that are focused on process control techniques. Since 1990,

supplier defects have been reduced by 73 percent, and receiving inspections have been significantly reduced.

Varian Nuclear Magnetic Resonance Instruments - Palo Alto, California

Varian Nuclear Magnetic Resonance Instruments has increased both savings and supplier quality by

— switching from an inspection-oriented environment to one of process control;

— training its workers in statistical process control factorywide and using it to internally track the performance of its suppliers;

— reducing its supplier base by over 60 percent by weeding out those with poor performance records and those it seldom used; and

— conducting certification programs where suppliers are required to pass five to six lot inspections. Varian-approved suppliers’parts

 completely bypass receiving inspection, but if they do not meet quality standards as a product is assembled on the production

 line, they are immediately rejected.

Due to the supplier program, supplier defect rates have decreased by 96 percent, and inspections have decreased by 76 percent. In

fact, the company is so confident in the supplier’s quality for one critical component that comprises 80 percent of the total cost of

supplier material, that Varian does not perform final assembly and test until after the product is delivered to the customer. Thus, Varian

and its customer view the results of final assembly and test simultaneously.

(continued)

Page 27 GAO/NSIAD-96-162 Best Practices
Appendix III 

Examples of Companies’ Supplier Quality

Practices and Their Benefits

Motorola Paging Products Group - Boynton Beach, Florida

Motorola’s Paging Products Group supplier quality practices are to

— conduct certification programs requiring that each supplier undergo an on-site evaluation and submit sample parts for approval,

— conduct a quality system review at the end of the first year to determine the supplier’s status,

— eliminate receiving inspections after suppliers have passed inspection defect-free on three lots of material,

— require preferred suppliers to apply for the Baldrige Award,

— emphasize long-term relationships with suppliers,

— train all of its suppliers in total quality techniques,

— share its schedule forecasts with suppliers, and

— conduct early supplier involvement programs on new products.

As a result of its supplier quality assurance program, Motorola has

— eliminated all inspection of incoming material from all but new suppliers; 

— reduced suppliers by 85 percent, from 800 to 118; and

— decreased supplier defects by over 90 percent over a 7-year period.

Texas Instruments Lubbock Metal-Oxide Semiconductor - Lubbock, Texas

Texas Instruments Metal-Oxide Semiconductor ensures the quality of supplier products by

— relying upon third-party certification to periodically review its suppliers;

— using a Sematech database, allowing TI to choose from among the best suppliers; Sematech is a semiconductor consortium of 

 10 large companies that periodically conducts quality system audits of suppliers and retains past performance information

 so that its members can increase their effectiveness in selecting and retaining the few best ones; and 

— depending upon the Texas Instruments purchasing center located in Dallas, which screens and approves suppliers and parts as it

 does for all wafer fabrication facilities.

Because of this supplier process control program, Texas Instruments has experienced virtually zero defects from its suppliers. It is also

able to bypass inspecting parts from high quality suppliers whose processes are in control. These parts are directly used by Texas

Instruments-Lubbock in its products.

Cherry Electrical Products - Waukegan, Illinois

Cherry Electronics uses the following supplier quality practices:

— Most of Cherry’s 400 suppliers are “approved.”

— The approved suppliers follow Cherry’s Supplier Quality Assurance Manual.

— New suppliers and/or new items are audited, via a layout inspection, and statistical process control analysis is performed.

— Cherry matches its findings against those of the suppliers’ and if a match results, the supplier’s item becomes “approved.”

— The relationship with approved suppliers is based on trust and self-monitoring by the suppliers, with statistical process control

 data available from suppliers upon request.

As a result of its supplier quality practices, Cherry has 

— reduced the number of suppliers by 50 percent, from 800 to 400 suppliers;

— reduced the number of defects from suppliers by 90 percent; and

— reexamined and eliminated unnecessary specifications.

John Deere Horicon Works - Horicon, Wisconsin

John Deere’s Horicon Works supplier practices are the following:

— Suppliers of critical parts must have established +/-3 sigma quality. (A sigma unit is a measure of scale that can be used for

 quality data. Three sigma quality (+/-3) is equal to three standard deviations away from an average, and the area covered

 by three sigma is 99 to 100 percent of the data. Under three-sigma quality, a supplier product’s values will lie within a

 known, predictable range 99 to 100 percent of the time.) 

— Deere measures the parts for conformance and then requires process variability reports for the key characteristics of the part from

 the supplier on a periodic basis.

— Deere looks for strategic partners for its critical parts who will share management philosophies, be industry leaders, and require

 very little communication or reviews.

As a result, Deere has reduced its supplier base 81 percent, from over 800 to 151, and has qualified 40 of them. It has been able to

eliminate receiving inspections as a result, a savings of 25 positions.

Page 28 GAO/NSIAD-96-162 Best Practices
Appendix IV 

Status of Tasks in Developing a Strategic

Plan for Quality to Support Acquisition

Reform Initiatives

Task description Actions Milestone Comments

1. Develop charter, principles,

and memorandum of

understanding.

-Panel charter

-Memorandum of understanding

To be signed. Signed on

April 24, 1995.

2. Identify pilot program. Develop pilot program

characteristics and candidates.

To be implemented 

November 1995.

No status update.

3. Identify policy/

Federal Acquisition Regulation

(FAR) changes.

Identify changes required to

FAR parts 46 and 52 as well as

DOD FAR Supplement (DFARS)

part 246.

DFARS changes - December

1995

FAR changes - July 1995.

DFARS case 246 approved;

inputs needed to FAR part 46.

4. Develop data item

description.

-Identify key elements of a

quality plan.

-Write data item description

incorporating needed elements.

-Develop revisions needed to

ANSI/ASQC Q9000.

Obtain data item description

approval.

Data item description 81449

approved January 1995.

5. Develop training elements. Develop government/industry

training template for quality

system.

Plan of action for training - May

1995.

Completed draft - May 1996.

6. Develop evaluation criteria. Develop criteria for government

and industry to evaluate

contractor’s basic quality

management system.

Complete criteria August 1995. Completed draft - May 1996.

7. Develop evaluator guidelines. Determine if common evaluation

criteria exists among audit

community.

Complete guidelines March

1995.

Completed draft - May 1996.

8. Develop oversight guidelines

for basic quality management

system.

Determine the most efficient

way of performing the oversight

function to meet the criteria for

evaluating quality management

systems.

Develop guidelines for

oversight - September 1995.

Completed draft - May 1996.

9. Develop procurement

package criteria.

Develop a handbook that

contains criteria or guidelines

that can be used to encourage

the use of advanced quality

concepts.

Distribute handbook for

concurrence - September 1995.

Completed draft - May 1996.

10. Identify advanced quality

practices.

Recognize, improve, and

promote the use of

value-adding advanced quality

practices.

Final report - September 1995. Completed draft - May 1996.

11. Develop communication

clearinghouse.

Establish an advanced quality

practices clearinghouse to help

promote quality management

systems in government and

industry.

Implement clearinghouse and

publicize - October 1995.

Final report prepared February

1996.

(continued)

Page 29 GAO/NSIAD-96-162 Best Practices
Appendix IV 

Status of Tasks in Developing a Strategic

Plan for Quality to Support Acquisition

Reform Initiatives

Task description Actions Milestone Comments

12. Draft memorandum of

agreement to accept mutual

recognition.

Draft governmentwide

agreement to mutually

recognize a contractor’s quality

system based on defined

baseline requirements.

Memorandum ready October

1995.

Final draft contingent upon

completion of all other tasks.

13. Develop quality audit

system approach and

database.

Establish integrated

government audit system

whereby audit results are

shared by all participating

government agencies.

Not specified. Ongoing.

14. Develop quality systems

training courses.

Develop four courses based on

the approved ISO 9000 training

template.

Courses to start March 1996. Ongoing.

15. Review courses developed

in task 14.

Obtain review and approval of

course content and context.

Review to start December 1995. Ongoing.

Note: Source of information is a May 31, 1996, draft - “Phase I of the Government & Industry

Quality Liaison Panel.”

Page 30 GAO/NSIAD-96-162 Best Practices
Appendix V 

Examples of Savings Using Statistical

Process Control

Texas Instruments produced a quality, less expensive integrated circuit by

controlling its processes. Using a hierarchical process map that looked

like a “fishbone” diagram and statistical process control techniques, the

plant determined the few critical parameters that were responsible for

giving the customers the functions that they needed. (Figure V.I depicts

this process.) Texas Instruments found that although making the circuit

involved several steps, only a few key “process knobs” controlled the

outcome. It discovered that six quality measurements taken in the

Figure V.1: Producing Integrated Circuits Using Proceeds Control Techniques

Process steps

Coating process,

exposure of material

to ultraviolet light,

development process Customer

concerns Electrical

voltage

threshold Speed

eraseability,

programmability

Printing 

geometry

and other 

factors

Humidity level

and other key

parameters

Process

knobs

1

Functionally

What the 

customer 

needs

2

Electrical

parameters

3

Critical

process 

output

4

Process

input

Description of Process 

Steps

process were not critical to the product’s quality. The elimination of these

six measurements saved a total of 36 labor hours for each product

manufactured. Because of this application of statistical process control,

Page 31 GAO/NSIAD-96-162 Best Practices
Appendix V 

Examples of Savings Using Statistical

Process Control

rework decreased from 3 to 5 percent to 0.5 percent, the plant experienced

a 50-percent decrease in the product’s variability, and a 50-percent

decrease in cycle time. Because of this decreased cycle time, the resulting

throughput was increased, the plant became more efficient, and Texas

Instruments avoided purchasing an extra scanning electron microscope

costing $800,000, as well as saved the cost of a technician that would have

been needed to operate it.

Page 32 GAO/NSIAD-96-162 Best Practices
Appendix VI 

Comments From the Department of Defense

See pp. 17-18.

Page 33 GAO/NSIAD-96-162 Best Practices
Appendix VI 

Comments From the Department of Defense

Page 34 GAO/NSIAD-96-162 Best Practices
Page 35 GAO/NSIAD-96-162 Best Practices
Related GAO Products

Weapons Acquisition: Low-Rate Initial Production Used to Buy Weapon

Systems Prematurely (GAO/NSIAD-95-18, Nov. 21, 1994).

Military Airlift: The C-17 Program Update and Proposed Settlement

(GAO/T-NSIAD-94-166, Apr. 19, 1994).

Weapons Acquisition: A Rare Opportunity for Lasting Change

(GAO/NSIAD-93-15, Dec. 1992).

Tactical Missile Acquisitions: Understated Technical Risks Leading to Cost

and Schedule Overruns (GAO/NSIAD-91-280, Sept. 17, 1991).

Why Some Weapon Systems Encounter Production Problems While Others

Do Not: Six Case Studies (GAO/NSIAD-85-34, May 24, 1985).

Strategic Bombers: B-2 Program Status and Current Issues (GAO/NSIAD-90-120,

Feb. 22, 1990).

Effectiveness of U.S. Forces Can Be Increased Through Improved Weapon

System Design (GAO/PSAD-81-17, Jan. 29, 1981).

(705115) Page 36 GAO/NSIAD-96-162 Best Practices
Ordering Information

The first copy of each GAO report and testimony is free.

Additional copies are $2 each. Orders should be sent to the

following address, accompanied by a check or money order

made out to the Superintendent of Documents, when

necessary. VISA and MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address

are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office

P.O. Box 6015

Gaithersburg, MD 20884-6015

or visit:

Room 1100

700 4th St. NW (corner of 4th and G Sts. NW)

U.S. General Accounting Office

Washington, DC

Orders may also be placed by calling (202) 512-6000 

or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and

testimony. To receive facsimile copies of the daily list or any

list from the past 30 days, please call (202) 512-6000 using a

touchtone phone. A recorded menu will provide information on

how to obtain these lists.

For information on how to access GAO reports on the INTERNET,

send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov

PRINTED ON RECYCLED PAPER
United States

General Accounting Office

Washington, D.C. 20548-0001

Official Business

Penalty for Private Use $300

Address Correction Requested

Bulk Rate

Postage & Fees Paid

GAO

Permit No. G100