United States General Accounting Office

GAO Report to the Committee on

Agriculture, Nutrition, and Forestry,

U.S. Senate

December 2001 FOOD SAFETY

Weaknesses in Meat

and Poultry Inspection

Pilot Should Be

Addressed Before

Implementation

GAO-02-59
Page i GAO-02-59 Meat and Poultry Pilot Inspection Project

Letter 1

Results in Brief 2

Background 5

Pilot Project Makes Industry More Responsible and Accountable

for Product Safety and Quality, While Continuing Government

Monitoring With Fewer Resources 9

Design and Methodology Limitations Compromise Overall Validity

and Reliability of Pilot Project Results 15

Pilot Project Results Are Inconclusive, but FSIS Inspection

Personnel Generally Support Modified Inspections 25

Conclusions 33

Recommendations for Executive Action 35

Agency Comments and Our Evaluation 35

Appendix I Scope and Methodology 37

Appendix II Chronology of Pilot Project Litigation 40

Appendix III Performance Standards for Hog Plants 43

Appendix IV Survey Results 44

Appendix V Australia’s Meat Safety Enhancement Program 59

Background and Trends That Led to the Program 59

How the MSEP System Works 60

Prerequisites for Plants to Participate in MSEP 61

AQIS’ Evaluation of Its Program 62

Appendix VI Canada’s Modernized Poultry Inspection Program 63

Background 63

How the MPIP Pilot Works 63

Prerequisites for Plants to Participate in the MPIP Pilot 64

Performance Standards 65

Contents
Page ii GAO-02-59 Meat and Poultry Pilot Inspection Project

MPIP Pilot Success Attributed to Extensive Training of Inspectors

and Industry 66

Under MPIP Pilot, CFIA Officials Monitor Plant Performance

Trends 67

Current Status of MPIP 68

Appendix VII Comments From the U.S. Department of Agriculture 69

GAO’s Comments 76

Appendix VIII GAO Contact and Staff Acknowledgments 81

GAO Contact 81

Acknowledgments 81

Tables

Table 1: Number and Types of Pilot Plants That Participated in

Traditional and Modified Inspections 11

Table 2: Traditional Inspection System’s Results (Baseline Data)

From 16 Chicken Pilot Plants: Defects per Shift in Percents 13

Table 3: Performance Standards for Inspection Systems at Chicken

Plants in the Pilot Project 26

Table 4: Eleven Chicken Plants’ Inspection Systems Compliance

With Performance Standards (RTI’s Data) 27

Table 5: Eleven Chicken Plants’ Inspection Systems Performance

After Shifting to Modified Inspections (RTI’s Data) 28

Table 6: Ten Chicken Plants’ Inspection Systems Compliance With

Performance Standards (FSIS’ Data) 29

Table 7: Changes in Chicken Plants’ Inspection Systems

Performance Over Two Time Periods After Shifting to

Modified Inspections (FSIS’ Data) 30

Table 8: Summary of Responses to GAO’s Survey of USDA

Inspectors and Veterinarians 31

Table 9: Performance Standards for Inspection Systems at Hog

Plants in the Pilot Project 43

Table 10: Division of Responsibility Under Australia’s Meat Safety

Enhancement Program 60
Page iii GAO-02-59 Meat and Poultry Pilot Inspection Project

Figures

Figure 1: Traditional and Modified Inspection Systems on a

Chicken Slaughter Line 8

Figure 2: Location of Chicken Pilot Plants 18

Figure 3: Fecal Noncompliance Records Under Traditional and

Modified Inspections 23

Abbreviations

CCP Critical Control Point

CDC Centers for Disease Control and Prevention

FSIS Food Safety and Inspection Service

HACCP Hazard Analysis and Critical Control Point

HIMP HACCP-Based Inspection Models Project

RTI Research Triangle Institute

USDA U.S. Department of Agriculture
Page 1 GAO-02-59 Meat and Poultry Pilot Inspection Project

December 17, 2001

The Honorable Tom Harkin

Chairman

The Honorable Richard G. Lugar

Ranking Minority Member

 Committee on Agriculture, Nutrition, and Forestry

United States Senate

While the food supply in the United States is generally safe, foodborne

bacteria, viruses, and other pathogens can cause a significant number of

illnesses, a small fraction of which are life-threatening. According to

estimates from the Centers for Disease Control and Prevention (CDC),

foodborne diseases cause 76 million illnesses in the United States each

year, including 325,000 hospitalizations and 5,000 deaths. Meat and poultry

products contaminated with bacteria such as Salmonella, Campylobacter,

Escherichia coli (E. coli), and Listeria cause most foodborne illnesses

and deaths. To control the spread of foodborne illnesses through meat and

poultry products, approximately 3,400 U.S. Department of Agriculture

(USDA) inspectors at 1,300 slaughter plants are stationed along slaughter

lines to provide continuous inspection and conduct organoleptic

examinations—using sight, touch, and smell—of each and every carcass.

In 1997, USDA announced the need to modify its meat and poultry

slaughter inspection program to make industry more responsible for

identifying carcass defects. This approach is consistent with the agency’s

previous adoption of the Pathogen Reduction: Hazard Analysis and Critical

Control Point (HACCP) regulations. The HACCP approach is risk-based

and makes industry, rather than federal inspectors, responsible for

identifying steps in food production where food safety hazards are most

likely to occur and for establishing controls that prevent or reduce it.

USDA had not extended the HACCP principles to slaughter inspections

because the agency has traditionally provided continuous inspection of

each and every carcass. However, USDA believes that changing its

traditional inspection system would also reduce inspectors’ reliance on

organoleptic inspections, allow for a shift to prevention-oriented

inspection systems based on risk, and permit redeployment of its

resources to better protect the public from foodborne diseases.

Before making a permanent change to its slaughter inspections system,

USDA developed a model to test whether such a change would continue to

ensure the safety of meat and poultry products. This model is being tested

United States General Accounting Office

Washington, DC 20548
Page 2 GAO-02-59 Meat and Poultry Pilot Inspection Project

at several volunteer chicken and hog plants. At plants participating in the

project, plant personnel, instead of USDA inspectors, examine each

carcass to make an initial determination whether it is unacceptable and

should be removed from the slaughter line. At the participating plants,

USDA first measures the performance of traditional inspection systems

and then compares it with the performance of modified inspections. USDA

will propose regulations to modify its inspection system nationwide if the

project shows that the modified inspection system performs as well as or

better than traditional inspections. A reduced number of USDA inspectors

are still at each plant to verify that safety and quality standards are met. In

addition, at least one USDA inspector is positioned near the end of each

slaughter line in order to comply with a federal court ruling that USDA

cannot delegate the inspection of carcasses to plant employees. As of July

2001, the federal cost associated with this project was approximately

$5.7 million, and USDA estimates that it will spend another $1.2 million.

Australia and Canada have tested similar modified inspection programs for

meat and poultry and approved their use on a nationwide basis.

Concerned about the design, methodology, and reliability of the pilot

project, you asked us to (1) describe the objectives, design, and scope of

the project; (2) identify limitations, if any, in the project’s design and

methodology; and (3) any design and methodology limitations

notwithstanding, determine if the data generated by the project will allow

USDA to reach valid conclusions on the relative effectiveness of modified

and traditional inspection methods in ensuring food safety and quality. In

addressing these objectives, you also asked us to obtain information from

similar projects in Australia and Canada.

In response to your request, we reviewed USDA’s pilot project as

implemented at chicken and hog plants. However, we were able only to

analyze data from chicken plants because complete data from hog plants

were not yet available. We also visited Australia and Canada to learn about

similar pilot projects. Appendix I describes our methodology in detail.

USDA’s original objective in implementing the pilot project was to test

whether a prevention-oriented inspection system that uses plant personnel

to examine each carcass for safety and quality and USDA inspectors to

verify that safety and quality standards are met can provide a level of

product safety and quality equal to or better than traditional inspections.

At 11 chicken and 3 hog plants that are voluntarily participating in the

before and after phases of the project, plant personnel, instead of USDA

inspectors, initially determine which carcasses and parts are unacceptable

Results in Brief
Page 3 GAO-02-59 Meat and Poultry Pilot Inspection Project

and should be removed from the slaughter line because they are diseased

or unwholesome. USDA is, therefore, able to use fewer inspection

personnel at these plants. Using safety and quality performance standards

developed for the project, an independent contractor measured how well

pilot plants’ inspection systems performed against these standards. The

contractor measured how well USDA inspectors identified carcass defects

under traditional inspections and then conducted similar measurements

once plant personnel assumed those duties. Also, as a part of the pilot

project, USDA inspectors at the project plants have examined carcasses to

evaluate how well plant employees detect carcass defects. USDA will

analyze these data to determine if at least the same level of safety and

quality is maintained under the modified inspections. USDA has

announced that preliminary results from the chicken pilot project show

that plants’ inspection systems perform better under the modified

inspections, and that it expects to propose regulations to modify its

slaughter inspection system for all chicken plants early in 2002. Both

Australia and Canada have recently tested and adopted modified

inspection programs that resemble USDA’s pilot project in that they

include the replacement of some government inspectors with plant

personnel.

USDA’s pilot project for chickens has several design and methodology

limitations that compromise the overall validity and reliability of its

results. Hence, it is questionable whether the data generated by the project

are indicative of how all of the chicken plants’ inspection systems would

perform if modified inspections were adopted nationwide. First, the

chicken pilot that USDA designed lacks a control group—a critical design

flaw that precludes a comparison between the performance of the

inspection systems at those plants that volunteered to participate in the

pilot and that of plants that did not participate. Without a control group,

USDA cannot determine whether changes in inspections systems are due

to personnel changes or other possible explanations, such as the addition

of chlorine rinses. Second, the chicken plants that volunteered to

participate in the baseline measurement phase of the pilot were not

randomly selected, and they did not include plants from all chickenproducing areas or plants of all sizes. Thus, the results cannot be

generalized to the entire population of chicken slaughter plants in the

United States. Third, the pilot project’s methodology did not take into

account variables such as seasonal changes and plant modifications that

could affect project results. For example, after the project began, many

plants added antimicrobial rinses and washers, which usually reduce the

levels of microbial contamination. USDA acknowledges some of the

project’s limitations but maintains that the design is consistent with that of
Page 4 GAO-02-59 Meat and Poultry Pilot Inspection Project

other USDA studies and that it is difficult to conduct this type of

experiment without relying on volunteer participants. Finally, USDA’s

pilot project did not include features of the modified inspection systems in

Australia and Canada that would be important considerations in ensuring

the successful implementation of a modified inspection system

nationwide. For example, during the pilot project, USDA did not require

the training of plant employees. Such training would be important in

ensuring that plant personnel are as competent as federal inspectors in

identifying carcass defects that can affect product safety and quality.

Similarly, the pilot project did not require the use of statistical process

controls as required in Australia and Canada. Statistical process controls

provide a means to determine whether the plant’s production processes

are performing within established performance standards. This technique

would enable USDA and the plants to better measure and control their

performance daily and over time to ensure continuous improvement.

Additionally, USDA allowed plants with repeated noncompliance records

to continue participating in the pilot project. If USDA decides to

permanently modify its inspection system, a phased-in approach such as

Canada’s would help to ensure that plants continually improve their ability

to ensure product safety and quality and that those that have difficulty

return to traditional inspections.

Notwithstanding the project’s design problems, which we believe make

the results unreliable, we found that, so far, the data themselves do not

conclusively demonstrate that modified inspections are at least equal to

traditional inspections. Part of the difficulty is that the two data sets

collected for this pilot show somewhat different results. These data show

whether the pilot plants met the seven organoleptic performance

standards—two food safety standards and five other consumer protection

(quality) standards—that USDA developed to compare, first the inspection

systems’ performance at plants under traditional inspections and then

under modified inspections. The modified inspection systems at plants in

the pilot project were expected to perform at least as well as they did

under traditional inspections. However, the contractor’s data show that

while the majority of the chicken plants’ inspection systems met or

exceeded four or more of the seven organoleptic standards adopted by

USDA, none met all seven. Specifically, the contractor’s data show that

most of the inspection systems at these plants did not meet the safety

standard for the presence of fecal material, which could contain harmful

bacteria such as E. coli. However, the traditional inspection system at

these plants was also unable to meet this standard. Most inspection

systems also did not meet quality standards for defects such as feathers

and oil glands, but USDA officials pointed out that such defects are of a
Page 5 GAO-02-59 Meat and Poultry Pilot Inspection Project

lesser concern because they are not associated with foodborne illness and

they can generally be removed from the carcass after it leaves the

slaughter line and moves to the processing area. USDA’s contractor also

measured whether the modified inspection systems at these plants met

two other standards that existed before the pilot project—the pathogen

reduction performance standard for Salmonella and generic E. coli. The

data show that 9 of the 11 inspection systems met the Salmonella standard

and that 80 percent of the samples collected for generic E. coli were

within the acceptable ranges. On the other hand, the data collected by

USDA inspectors at these chicken plants show better results than those

shown by the contractor’s data. According to USDA’s data, 7 of 10 plants’

inspection systems met 6 of the 7 standards. While the data generated by

the pilot project yielded inconclusive results, the pilot project’s inspectors

and veterinarians that we surveyed for the most part believe that a

modified inspection system results in safer products. More than half of

those surveyed said that modified inspections are equal to or somewhat

better than traditional inspections in ensuring product safety and quality.

Modified inspections allow for additional time to inspect slaughter line

operations more thoroughly because inspectors, except for the carcass

inspector, are no longer tied to a fixed location.

This report reiterates our previous recommendation for legislative

revisions aimed at reducing the potential for further legal challenges by

providing USDA with clear authority to modify its inspection system. In

addition, this report makes several recommendations to the Secretary of

Agriculture aimed at ensuring that, if USDA decides to implement a

modified inspection system, such a system will effectively ensure product

safety and quality. In commenting on a draft of this report, USDA said that

the report’s recommendations for executive action are appropriate and

that the issues noted in the report can be satisfactorily addressed. USDA

also provided technical comments which we incorporated as appropriate.

The Federal Meat Inspection Act and the Poultry Products Inspection Act,

21 U.S.C. sections 604 and 455, respectively, give USDA overall

responsibility for ensuring the safety and wholesomeness of meat and

poultry products that enter interstate commerce. Acting under these

legislative authorities, USDA has engaged in continuous government

inspection of each and every carcass at slaughter plants throughout the

United States. Within USDA, the Food Safety and Inspection Service

(FSIS) is responsible for inspections at all meat and poultry slaughter and

processing plants and for ensuring plants’ compliance with regulatory

requirements.

Background
Page 6 GAO-02-59 Meat and Poultry Pilot Inspection Project

At slaughter plants, FSIS inspectors are located at fixed stations and

examine carcasses to identify and remove defects that impair product

safety and quality. Defects that are likely to present a public health risk,

such as the presence of potentially pathogenic fecal material, are

considered food safety hazards. Product quality defects, such as feathers

or bruises on the carcasses, are aesthetic flaws that rarely present a direct

safety risk.

Inspectors perform different tasks, depending on the animal species being

slaughtered. For example, in chicken plants, the inspectors are located at

fixed stations immediately after the place on the slaughter line where

carcasses are opened and the organs removed. From these positions, the

inspectors examine the exterior, interior cavity, and organs of each

carcass. The inspectors then determine which carcasses must be

condemned, which have conditions that may be trimmed, and which may

proceed to the next stage in the slaughter line. Carcasses are then

trimmed, rinsed, and, as the final step of the slaughter line, placed into a

chiller. From here, carcasses move to the processing area of the plant,

where they are cut up and packaged. In contrast, in hog plants, the

inspectors are located at three fixed stations on the slaughter line. At the

head station, the inspectors excise and evaluate the head lymph nodes; at

the viscera station, they evaluate most organs; and at the final carcass

station, they examine each carcass for possible contamination, disease

conditions, and other adulterants before it goes into the chiller or cooler.

For a variety of reasons, including responding to recommendations from

the National Academy of Sciences and GAO that FSIS shift its resources to

a prevention-oriented, risk-based inspection system, in 1996 USDA

embarked on regulatory reform efforts to reduce foodborne hazards in

meat and poultry products. The agency adopted a new science-based

process control system, the Pathogen Reduction: Hazard Analysis and

Critical Control Point system. HACCP is a risk-based approach that makes

industry, not federal inspectors, responsible for identifying steps in food

production where contamination is most likely to occur and for

establishing controls that prevent or reduce contamination.

USDA anticipated that the HACCP framework would also require changes

in the roles and responsibilities of its inspection workforce, as industry,

not government, becomes increasingly responsible for the safety of its

products. For slaughter line operations, however, making industry more

directly responsible for the safety and quality of their products was

problematic because of a legal requirement for continuous carcass-bycarcass inspection. USDA believed that, to more fully integrate the HACCP
Page 7 GAO-02-59 Meat and Poultry Pilot Inspection Project

concept into meat and poultry slaughter operations, its traditional

inspection system would have to be modified to transfer some of the

carcass defect detection responsibilities to industry personnel. Before

implementing such an approach, FSIS decided to conduct a pilot to test

and evaluate whether product safety and quality could be maintained at a

comparable level if plant employees perform the carcass defect detection

duties and a reduced number of federal inspectors remain to verify

product safety and quality through increased testing and observation.

The pilot project is known as the HACCP-Based Inspection Models Project

(HIMP) and its original design was intended to measure the effect of

completely removing FSIS inspectors from their fixed locations on the

slaughter line while maintaining one oversight and one verification

inspector. In April 1998, the inspectors’ union filed suit in the United

States District Court for the District of Columbia to enjoin the USDA from

proceeding with the pilot project, on the grounds that it violated the

Federal Meat Inspection Act, 21 U.S.C. section 604, and the Poultry

Products Inspection Act, 21 U.S.C. section 455, by not requiring federal

government officials to perform carcass-by-carcass postmortem

inspections. In June 2000, the United States Court of Appeals for the

District of Columbia Circuit ruled that delegating the task of inspecting

carcasses to plant employees violated the acts because both statutes

require that federal inspectors, rather than private employees, determine

whether a product is adulterated. As a result of this ruling, in September

2000, FSIS redesigned the pilot project for chickens and hogs and placed

at least one FSIS inspector back at a fixed location on each slaughter line

to inspect each carcass.

Following a district court determination in January 2001 that the

redesigned pilot project does not violate the acts, the inspectors’ union

appealed in February 2001. That appeal is still pending. A detailed

description of the litigation surrounding the pilot project is contained in

appendix II.

Figure 1 depicts inspection responsibilities under FSIS’ traditional and

modified inspections on a chicken slaughter line.
Page 8 GAO-02-59 Meat and Poultry Pilot Inspection Project

Figure 1: Traditional and Modified Inspection Systems on a Chicken Slaughter Line

Note: Under the traditional inspection system, the slaughter line’s speed at most chicken plants is 91

birds per minute. Each inspector is responsible for about 35 birds per minute. Therefore, three FSIS

inspectors are required to inspect carcasses on a slaughter line that runs at a speed of 91 birds per

minute. The figure above shows only one FSIS inspector’s station in detail. Under the pilot project, a

plant employee (sorter) replaces the FSIS inspector who examined the carcasses to determine their

condition.

As figure 1 shows, under traditional inspection systems, three FSIS

inspectors are present at fixed locations on the slaughter line. In contrast,

at chicken plants that participate in the pilot project, one FSIS inspector is

present at a fixed location on each slaughter line, while a verification

inspector monitors the entire line. The verification inspector is free to

move along the slaughter line to continuously observe and evaluate the

plant’s implementation of its HACCP system and process controls. This

inspector is also responsible for randomly selecting and examining 80

carcasses per line per shift to verify that the plant is complying with the

performance standards for food safety and quality. A slaughter line
Page 9 GAO-02-59 Meat and Poultry Pilot Inspection Project

averaging 91 chickens per minute would process over 43,000 chicken

carcasses during each shift. The carcass inspector is located at the end of

the slaughter line, immediately before carcasses enter the chiller. This

inspector visually examines each carcass to comply with a court mandate

that each carcass should receive FSIS inspection. At participating hog

plants, carcass inspectors are positioned at the head, viscera, and final

carcass inspection stations. Most of the inspectors and some veterinarians

at the pilot project plants received promotions.

The pilot project’s hypothesis was that a prevention-oriented inspection

system that uses plant personnel to examine each carcass can provide a

level of product safety and quality equal to or better than traditional

inspections. At selected chicken and hog plants that voluntarily agreed to

participate in the project, plant personnel, instead of FSIS inspectors,

initially determine which carcasses and parts are unacceptable and should

be removed from the slaughter line. Under this project, FSIS is able to use

fewer inspection personnel at each plant. FSIS used an independent

contractor to collect and analyze organoleptic and microbial data under

the traditional inspection system first and then under the modified

inspection system. FSIS developed organoleptic performance standards to

measure how well the inspection system at participating plants performed

after shifting from traditional to modified inspections. In addition, as part

of their duties, FSIS inspectors continued to collect organoleptic

performance data at these plants. Recently, both Australia and Canada

tested and adopted modified inspection programs that resemble USDA’s

pilot project.

Pilot Project Makes

Industry More

Responsible and

Accountable for

Product Safety and

Quality, While

Continuing

Government

Monitoring With

Fewer Resources
Page 10 GAO-02-59 Meat and Poultry Pilot Inspection Project

To initiate the pilot project, FSIS invited meat and poultry plants that

slaughter young, healthy animals—market hogs, fed cattle, and young

poultry (including young turkeys)—to volunteer to participate in the

project.1

 Sixteen chicken, 5 hog, and 5 turkey plants initially volunteered to

participate.2

 No cattle plants volunteered. In general, plant managers that

volunteered to participate in the pilot said that they elected to do so

because they believe they can enhance product safety and quality if they

have better control of slaughter line operations and, at the same time,

increase productivity.

FSIS contracted with the Research Triangle Institute3

 (RTI) to collect and

analyze organoleptic and microbial data at plants that volunteered to

participate in the pilot project. The purpose of this data collection is to

evaluate whether the inspection systems at plants operating under

modified inspections perform at least as well as the traditional inspection

systems. In August 2000, RTI completed its collection of organoleptic and

microbial data at 16 chicken, 5 hog, and 5 turkey plants under the

traditional inspection. These baseline data documented the

accomplishments of the traditional inspections system, and FSIS decided

to adopt the results as performance standards for measuring the

accomplishments of the same plants’ inspection systems after they shifted

to modified inspections. After a transition period during which plant

personnel practiced their new roles, RTI again collected organoleptic and

microbial data under the modified inspection system to provide a beforeand-after comparison.4

After RTI collected data on the performance of the traditional inspection

system at the volunteer plants (baseline data), 5 of the 16 chicken plants

 1

FSIS designed the pilot project for meat and poultry plants that slaughter young animals

because this group comprises approximately 90 percent of the animals slaughtered and

those carcasses do not have complex pathology or other problems associated with older

animals.

2

Originally, 17 chicken plants volunteered to participate, but 1 of the chicken plants elected

to drop out of the project shortly thereafter.

3

The Research Triangle Institute is an independent nonprofit organization that conducts

multidisciplinary research on issues including health and pharmaceuticals, environment,

and education and training. RTI’s contract with USDA for this pilot project contains

modifications for increased future work.

4

After FSIS placed at least one carcass inspector back on the slaughter line, RTI had to

return to collect a new set of food safety and quality data.

Under Pilot, Industry Is

More Responsible and

Accountable for Product

Safety and Quality, While

FSIS Provides Inspection

Using Fewer Government

Resources
Page 11 GAO-02-59 Meat and Poultry Pilot Inspection Project

and 2 of the 5 hog plants dropped out of the project.5 As a result, RTI has

collected modified inspections data at 11 chicken and 3 hog plants. These

data were collected over a 5- to 6-week period. Some managers from

plants that dropped out cited uncertainty about the future of the pilot

project because of the court’s action, which resulted in modifications to

the pilot’s original design.

Ten additional chicken plants have since volunteered to participate in the

pilot, bringing the total number of participating plants to 21, but RTI will

collect data at only 5 of the additional 10 plants. According to FSIS, no

traditional inspection performance data (baseline data) will be collected at

these additional plants. Table 1 shows the number of slaughter plants that

participated both in the baseline data collection phase (traditional

inspections) and in data collection after they switched to modified

inspections.

Table 1: Number and Types of Pilot Plants That Participated in Traditional and

Modified Inspections

Species

Plants that completed baseline data

collection under traditional

inspections

Plants that completed data

collection under modified

inspections

Chickens 16 11 (10) a

Hogs 5 3

Turkeys 5 0

a

One plant dropped out after it completed data collection under the modified inspection system.

Source: USDA, FSIS.

Because turkey plants have not participated in modified inspections and

because data from hog plants under modified inspections are not yet

available, this report discusses pilot project results from chicken plants

only. (See app. III for information on the development of performance

standards for hogs.)

At the pilot project plants, FSIS has been able to use fewer inspection

resources than would otherwise be used under traditional inspection

systems, resulting in an overall 22 percent reduction (from 259 to 202

inspectors) in the number of FSIS inspectors on the slaughter floor. At

most of the 14 chicken and hog plants, the reduction of inspectors ranged

 5

All five turkey plants dropped out after the baseline data were collected.
Page 12 GAO-02-59 Meat and Poultry Pilot Inspection Project

from 1 and 10 inspectors, depending on the number of slaughter lines and

the speed of those lines. Under traditional inspections, there were three

inspectors per line on average, compared with two inspectors per line

under modified inspections. At three of the pilot project plants, the change

in the number of inspectors was not entirely due to the modified

inspection system.

To collect and analyze organoleptic data on the defects that the FSIS

inspectors had overlooked under traditional inspections at chicken plants,

RTI veterinarians at each volunteer plant examined approximately 2,000

randomly selected carcasses that had passed FSIS’ slaughter inspection.

RTI veterinarians examined carcasses to record the same type of defects

that FSIS inspectors identify when they examine each carcass. These

included safety defects, such as visible fecal material and evidence of

septicemia and/or toxemia, and other consumer protection defects (quality

defects) such as feathers, hairs, or bruises. FSIS grouped this information

into two food safety and five other consumer protection (quality)

categories:

• Food Safety 1: Infectious conditions such as septicemia and/or toxemia.

• Food Safety 2: Contamination with fecal material.

• Other Consumer Protection 1: Animal diseases such as airsaculitis.

• Other Consumer Protection 2: Defects such as bruises and sores.

• Other Consumer Protection 3: Contamination with digestive tract contents

(ingesta).

• Other Consumer Protection 4: Dressing defects such as feathers and oil

glands.

• Other Consumer Protection 5: Dressing defects of the digestive tract tissue

such as bursa and cloaca.

FSIS then developed performance standards for each of these seven

categories. The quality performance standards were set at position 12 (out

of 16) on the basis of measurement of the traditional inspection system at

the 16 chicken plants. That is, 25 percent of the plants’ inspection systems

(4 different plants) would have to improve on their baseline results in each

of the 7 performance categories. FSIS officials explained that they decided

to adopt the 12th position of the baseline results as the standard to ensure

that most participants would be able to meet them. The officials stated

that this represents a reasonable tightening of the traditional system’s

accomplishments.

The Pilot’s Scope Included

the Development of Safety

and Quality Standards to

Measure Performance of

Traditional Inspections as

Compared With Modified

Inspections
Page 13 GAO-02-59 Meat and Poultry Pilot Inspection Project

To better protect consumers from defects that may be harmful, FSIS set

the two food safety standards at zero rather than adopting the results of

the 12th position. Thus, a performance standard of zero for fecal

contamination means that FSIS expects all carcasses slaughtered during a

shift to be free of fecal material. Conversely, an 80-percent performance

standard for dressing defects such as feathers and oil glands means that if

80 of 100 carcasses have these defects, the performance standard is met.

The performance standards represent the percentage of carcasses per shift

with each defect that FSIS considers acceptable. Table 2 shows the

traditional inspection system’s baseline results for the 16 chicken plants.

Table 2: Traditional Inspection System’s Results (Baseline Data) From 16 Chicken

Pilot Plants: Defects per Shift in Percents

Note: Ranking numbers represent different plants for different food safety and OCP standards.

Although the 12th position’s performance was 0.05 percent for Food Safety 1 and 1.5 percent for

Food Safety 2, FSIS adopted a zero tolerance policy for these two standards. For Food Safety 2, the

zero tolerance standard was used because it is already a regulatory requirement.

Source: RTI.

RTI also measured the prevalence of generic E. coli and Salmonella under

traditional inspections and modified inspections at the 11 chicken plants

to determine if changes in the inspection system would have an effect on

the microbial profile of carcasses. RTI collected and analyzed 300 samples
Page 14 GAO-02-59 Meat and Poultry Pilot Inspection Project

for both microorganisms over a 6-week period. RTI analyzed the

Salmonella samples to determine if they met FSIS’ already existing

regulatory standard for Salmonella and the generic E. coli samples to

determine if the results were in the acceptable, marginal, and

unacceptable ranges established by FSIS.

FSIS is also collecting data on product safety and quality defects at

chicken plants in the pilot project and is analyzing these data to compare

how well traditional inspection systems perform versus modified

inspection systems. FSIS inspectors collect the data as they conduct

scheduled and unscheduled verification tests at each chicken plant. These

tests evaluate, among other things, how well plant employees detect

carcass defects. When the pilot project is concluded, FSIS plans to analyze

these data to determine if at least the same level of safety and quality is

maintained under the pilot program. FSIS has announced that preliminary

results from the chicken pilot project show that inspection systems

perform better after the plants shifted to modified inspections, and that it

expects to propose regulations to modify its traditional slaughter

inspection system for all chicken plants in 2002.

The Australian Quarantine and Inspection Service and the Canadian Food

Inspection Agency have tested and adopted alternative inspection systems

similar to that tested in the FSIS pilot project. (See apps. V and VI for

additional details.)

Australia has tested and adopted a modified inspection system for all meat

plants that produce products for domestic consumption. In addition,

Australia is now in the process of implementing the Meat Safety

Enhancement Program at plants that export meat. Under the program,

plants are responsible for plant process controls, including sanitation, the

microbial monitoring of pathogens, and detecting product defects. The

program incorporates the use of government-licensed company

employees, acting under the supervision and oversight of government

veterinarians, to replace government inspectors. In order to be licensed by

the government, the company’s employees are required to undertake

about 600 hours of training and pass an examination. The Australian

Quarantine and Inspection Service sets public health product standards

and audits plant operations to ensure that food safety standards are met. A

key feature of the program is the mandatory use of statistical process

controls to track plants’ performance over time.

Pilot Projects in Australia

and Canada Used a Similar

Strategy
Page 15 GAO-02-59 Meat and Poultry Pilot Inspection Project

Canada has adopted a new poultry inspection system, known as the

Modernized Poultry Inspection Program that allows plant personnel to

assume defect detection duties and remove defective carcasses from the

slaughter line under the supervision of government inspectors. The

program is being implemented in a phased approach. Performance

standards for Canadian poultry plants participating in this program are

similar but not identical to FSIS’ standards for chicken plants in the U.S.

pilot project. Like the Australian program, the Canadian program has

stringent training requirements for industry personnel replacing

inspectors. Industry personnel are required to demonstrate competency

through testing and certification.

The pilot project has several design and methodology limitations that

compromise the overall validity and reliability of its results. For example,

the lack of a control group prevents valid comparisons between the

inspection system of participating plants and that of nonparticipating

plants. In addition, the participating plants were not randomly selected;

therefore, results from these plants cannot be generalized to the entire

population. Finally, the pilot project does not appropriately explain how

variables, such as seasonal changes and plant modifications, could affect

the project’s results. In addition, the pilot project does not include

important features of similar programs in Australia and Canada.

FSIS officials characterized the design of the pilot project as a classic

before-and-after experiment that is typical of other FSIS experiments. A

classic before-and-after experiment involves both a control and an

experimental group and thus provides a basis for comparing the results of

the two groups. Ideally, the groups are as similar as possible, except for

the variable that the experiment is attempting to measure. However,

because this level of rigor is rarely achievable in applied settings, quasiexperimental designs are often used in which comparison groups are

established to compare the impact of a new program with the status quo.

According to our discussion with FSIS officials, the volunteer plants were

not separated into a control group and an experimental group because the

agency felt that it would have been unfair to exclude volunteer plants from

taking part in the modified inspection aspect of the experiment.

Nevertheless, under a quasi-experimental design, FSIS could have used as

a control group the five chicken plants that decided to discontinue

participation in the pilot project after their baseline performance under

the traditional system was measured. The agency could have then

compared the results from these plants with the results obtained from the

Design and

Methodology

Limitations

Compromise Overall

Validity and

Reliability of Pilot

Project Results

Pilot Project Lacks a

Control Group
Page 16 GAO-02-59 Meat and Poultry Pilot Inspection Project

plants that participated in the modified inspections. Lacking such a

comparison, it is difficult to determine how the inspection systems at the

plants that did not participate in the pilot project would have performed

under the performance standards established for the pilot project.

The managers of two large chicken plants that dropped out of the pilot

said that they proposed another approach to FSIS. Namely, these

managers wanted to conduct what could have been a controlled

experiment within their own plant. Both of these plants have multiple

slaughter lines, and the managers wanted to operate half of their slaughter

lines under traditional inspections and the other half under modified

inspections. However, FSIS would not authorize such an arrangement

because the agency wanted the entire plant to be under one inspection

system. Officials said that it would have been too difficult to maintain two

separate inspection systems in the same plant. As a result, both of these

plants withdrew from the pilot project.

The Australian pilot project compared three plants that volunteered to

participate in the pilot project with three plants operating under

traditional inspections, so the pilot had both an experimental group and a

control group. The plants in the experimental group removed the

government inspectors from the slaughter line and replaced them with

plant personnel, while the control group plants made no changes to their

inspection system. The pilot project measured both microbial and

organoleptic data at plants in both groups to determine how well they

complied with standards. The results were then compared in order to

determine how the modified inspections affected food safety and quality.

The Canadian Food Inspection Agency did not use a control group.

Because the volunteer chicken plants were not randomly selected, the

results cannot be generalized to the population of plants that slaughter

young chickens. FSIS is not authorized to compel plants to participate in

any pilot project; therefore, it had to rely on volunteer plants instead of

randomly selecting participants. However, FSIS did have the option to

randomly select a subgroup to serve as a control group from the volunteer

plants, but decided not to pursue that option. Australia and Canada also

did not select plants on a random basis.

FSIS officials acknowledge that the number of plants in the pilot was not

statistically derived. The officials believe, however, that the 16 chicken

plants that originally volunteered constitute a sufficient number to

estimate results for the population. However, only 11 chicken plants have

Plants Participating Are

Not Randomly Selected

and Do Not Represent the

Population
Page 17 GAO-02-59 Meat and Poultry Pilot Inspection Project

participated in all phases of the pilot project. These plants represent about

6 percent of the population of young chicken plants nationwide, or 11 of

186 plants in 1999. We estimate that 125 randomly selected plants would

need to participate in the project in order for FSIS to obtain results that

can be generalized to the population of chicken plants.6

As a result of the pilot project’s reliance on volunteer plants, the results of

the pilot do not provide sufficient geographic representation. That is, the

11 chicken plants that have participated in all phases of the pilot project

are located in 7 of 35 states that produce young chickens, or about

20 percent of the chicken-producing states. These plants are concentrated

in the southern region of the United States and account for about

57 percent of U.S. young chicken production. (See fig. 2.) Unlike FSIS, the

Canadian Food Inspection Agency designed its pilot to include chicken

plants from the western, eastern, and central provinces. As a result, the

Canadian pilot project has better geographic representation.

 6

This sample size is calculated to provide for results of plus or minus 5 percent at the

95-percent confidence interval.
Page 18 GAO-02-59 Meat and Poultry Pilot Inspection Project

Figure 2: Location of Chicken Pilot Plants

Note: Numbers indicate how many chicken plants participated in the baseline data collection phase of

the pilot project.

Source: USDA, FSIS.

In addition, there are eight large plants, three small plants, and no very

small plants in the pilot project. According to FSIS, there are 112 large, 61

small, and 13 very small plants that slaughter young chickens in the United

States. We discussed with officials at all participating plants in the pilot

project whether the plants represent the population of chicken plants in

the United States. Most of them said that, in their opinion, the plants are

representative of the industry nationwide. FSIS officials are also of the

opinion that all plants that volunteered to participate are similar to

nonparticipants in all respects except for the fact that they volunteered.

However, since only volunteer plants participated in the pilot, potential

selection bias is introduced because such plants may be predisposed to

better performance.
Page 19 GAO-02-59 Meat and Poultry Pilot Inspection Project

The pilot project does not adequately control for the effect that seasonal

variations and plant modifications can have on the its results. Because

seasonal changes can cause animals to develop diseases and/or poor

health, two poultry science experts that we consulted said that it is

imperative to collect samples over sufficiently long periods of time to

cover all seasons. Seasonal variations significantly affect microbial results,

especially in the case of bacteria such as Salmonella, which was measured

in the pilot project. However, the pilot project’s design called for

6 consecutive weeks of microbial data collection. Accordingly, the experts

we consulted believe that microbial data collected during the pilot project

are of limited value because the data cover less than 2 months at each

plant.

The pilot project’s results may also be affected by another set of variables.

Most of the plant managers at pilot project plants told us that they

introduced changes to their slaughter lines to improve product safety and

quality after the baseline performance of the traditional system was

measured. These changes included the addition of new brushes and

scrubbers, plain water washes, and antimicrobial washes. Some managers

also modified their evisceration equipment and developed computerized

systems to track carcass safety and quality defects. All the inspectors and

veterinarians at pilot project plants that we surveyed indicated that at least

one change had been made after modified inspections were implemented

at the plant. In effect, it is not possible to discern whether the before-andafter results measured by the two food safety and five quality standards

are to be solely attributed to the shift from traditional to modified

inspections at these plants.

According to FSIS officials, the pilot project was designed to measure the

accomplishments of the traditional inspection system and the

accomplishments of the models inspection system allowing for measuring

multiple process changes—not isolated comparisons between FSIS

inspector performance and plant employee performance. If this is the case,

then the pilot project should have included the measurement of all the

process changes introduced, not just the only change introduced by the

agency; namely, the transfer of carcass defect detection from FSIS

inspectors to plant personnel. We understand that, at their initiative, plants

decided to make changes to their process during this pilot. However, FSIS

did not collect data on what changes were made or on what the effect of

those changes may be on the overall pilot project results.

USDA officials disagree that the pilot project does not take into account

variables that could affect its results. They stated that, if the results from

Pilot Project’s

Methodology Does Not

Take Into Account

Variables That Could

Affect Results
Page 20 GAO-02-59 Meat and Poultry Pilot Inspection Project

all plants are viewed collectively, their sample extended over a sufficient

period of time to consider the effect of seasonal variations, evisceration

line effects, and other non-quantifiable variables. Even if all plants are

considered collectively, FSIS is still missing microbial data for the entire

year. According to officials, from January through April, no data were

collected at any of the pilot plants. Moreover, according to the experts we

consulted, FSIS should have a year’s worth of microbial data from each

plant in the pilot to be able to account for seasonal variations.

In discussing these limitations with USDA officials, they explained that the

focus of the project is studying an entire system, including all changes

made by the plants, and not simply comparing the work of company

employees with that of government inspectors. However, we note that the

project’s stated objective was to determine whether a modified inspection

system is at least as good as the traditional system. In order for such a

comparison to be valid, USDA would have to determine that other

variables introduced into the slaughter line, such as the addition of rinses

or brushes, are not the cause of changes noted after the shift in inspection

systems took place.

The pilot project did not address three important features of similar

programs in Australia and Canada. Those features would help to ensure

that plant personnel are able to discharge their carcass defect detection

duties as well as federal inspectors and that plants are capable of

continuing to ensure the safety and quality of their products under a

modified inspection system. First, the pilot project did not require that,

prior to assuming their new duties, plant personnel receive any training

and pass competency tests. Second, the pilot project did not require the

use of statistical process controls as a means to ensure that plants

continually improve their processes under the modified inspection system.

Third, the plants continued in the pilot project irrespective of repetitive

noncompliance with regulatory requirements.

FSIS did not require that plant employees complete training before

assuming the carcass defect detection duties that FSIS inspectors’

performed. Furthermore, FSIS did not establish a way to measure plant

employee’s knowledge and competence. FSIS officials told us that training

for plant personnel will not be required if the system is adopted

nationwide. Plant personnel and FSIS inspectors at each pilot project plant

did participate, however, in a transition phase during which they practiced

their new roles. FSIS officials told us that the transition phase was also

Pilot Project Lacks Key

Elements

Project Does Not Require

Formal Employee Training and

Certification
Page 21 GAO-02-59 Meat and Poultry Pilot Inspection Project

used to explain how and when data were to be collected during the

project.

If FSIS implements modified inspections nationwide, plant personnel will

be assuming carcass defect detection duties previously performed by FSIS

inspectors who have received formal training on how to identify carcass

defects and determine whether they are suitable for human consumption.

However, unless FSIS institutes training requirements, plant personnel will

not have received comparable training. We asked plant managers

participating in the pilot project about the training that they have provided

for plant employees. They replied that they provided various levels of

training. Individual plants have sole discretion for determining the type of

training provided for their employees as FSIS does not approve or monitor

plants’ training programs. According to plant officials, FSIS is supportive

but does not endorse a training program that the International HACCP

Alliance developed for plant personnel participating in this pilot project.7

In some instances, the Alliance, not FSIS, has accredited plants’ training

programs but it does not certify trainees who complete these courses.

Seven of the 14 pilot project managers whom we interviewed support a

training program for plant personnel. About two-thirds of the managers

also were in favor of developing a joint training program between industry

and FSIS. At nine pilot project plants, plant employees were required to

pass a test before being assigned to their new jobs, and seven plants

adopted some kind of continuous on-the-job training. The managers told

us that classroom training for plant employees ranged from 4 to 80 hours.

The duration of on-the-job training ranged from 8 to 160 hours.

In contrast, the Australian modified inspection program requires plant

personnel to complete about 600 hours of classroom training as a

prerequisite to participation in the modified inspection program. Plant

personnel must also pass a competency test and be certified. Upon

completion of the on-the-job training, state authorities license the plant

personnel. The Australian government’s veterinarian-in-charge at each

plant is authorized to withdraw or deny license renewal of plant personnel

on the basis of performance.

 7

The International HACCP Alliance membership includes over 120 members from nine

countries representing industry associations, professional associations, educational

foundations, universities, third party/private companies, and government cooperators.

Their mission is to promote international public health and safety by facilitating uniform

development and implementation of HACCP programs from farm to table.
Page 22 GAO-02-59 Meat and Poultry Pilot Inspection Project

Similarly, under Canada’s modified inspection program, government

inspectors train plant personnel. The required training includes

standardized classroom training, in-plant training, a final examination, and

a demonstration of competency on verification tests and finished product

standards. At the completion of the training, the Canadian Food Inspection

Agency accredits plant employees. Plant employees must be reaccredited

every 6 months, and the agency’s inspectors reserve the right to conduct

additional reaccreditation of plant employees at any time.

Sixty-seven percent of the FSIS inspectors and veterinarians whom we

surveyed believe FSIS should develop a training program that includes

mandatory testing and certification of plant employees.

Although FSIS has encouraged plants in the pilot project to use statistical

process controls, it is not a requirement of the pilot. However, threequarters of the inspectors and veterinarians responding to our survey

indicated that their plants use statistical process controls. About a third of

these respondents believe that the use of statistical process controls have

the effect of making food safety and quality better, while a quarter of them

believe that it does not make a difference. At the pilot project plants, FSIS

inspectors do not review or analyze plant data using statistical process

controls. However, by applying process controls, FSIS could obtain a more

comprehensive assessment of the effectiveness of the modified inspection

system because it could review the performance of an individual plant’s

data to determine whether the plant’s systems are continually improving

over time.

In contrast, the Australian and Canadian food inspection agencies use

statistical process control methods to audit a plant’s performance over

time. This method entails verification by the plant’s management that its

production process is stable, that it is capable of producing products that

meet performance standards, that it takes appropriate actions if changes

occur in the process before it results in unacceptable products, and that it

takes actions to continuously improve its process.

Project Does Not Require Use

of Statistical Process Controls
Page 23 GAO-02-59 Meat and Poultry Pilot Inspection Project

FSIS allowed all plants to continue participating in the pilot project even

though several had multiple instances of noncompliance with regulatory

requirements for fecal contamination.8

 Our analysis of noncompliance

records issued by FSIS inspectors at 11 chicken pilot plants showed an

increased number of such records after the plants shifted to modified

inspections for fecal noncompliance. In two instances, however, FSIS took

additional action and notified plants of problems with repeated

noncompliance. FSIS officials told us that they are now considering how

many repeated instances of noncompliance will result in further regulatory

action.

Figure 3: Fecal Noncompliance Records Under Traditional and Modified Inspections

Note: Data are provided for a 12-month period before and after the pilot project’s implementation.

However, at some plants, data reported after the implementation of the pilot project covered periods

of less than 12 months, data for plant 7 cover 8 months, data for plants 1 and 10 cover 10 months,

and data for plant 9 cover 11 months. It is likely that a full twelve months of data for these plants

would result in an even higher number of noncompliance records.

Source: USDA, FSIS.

As figure 3 shows, the number of noncompliance records resulting from

fecal material increased significantly at several plants after they shifted

from traditional inspections to modified inspections. At one plant, FSIS

 8

FSIS inspectors document each instance of noncompliance with regulatory requirements

in a noncompliance record that plant managers must address immediately. If this is done,

the plant continues to operate without interruption. When deficiencies occur repeatedly,

FSIS may take further action to withhold products or to suspend inspections. If FSIS

suspends inspections, the plant cannot operate.

Project Does Not Fully Take

Into Account Plants’ Repeated

Instances of Noncompliance

With Regulatory Requirements
Page 24 GAO-02-59 Meat and Poultry Pilot Inspection Project

inspectors issued 469 fecal material noncompliance records during a

10-month period after the plant shifted to modified inspections. Many

noncompliance records reflect multiple violations recorded within the

same record. At one plant, FSIS inspectors reported that over 90 percent

of the fecal material noncompliance records reflected repetitive failures.

FSIS officials acknowledge the rise in the number of noncompliance

records and attribute it to the fact that at pilot project plants, FSIS

inspectors have more time to verify compliance with regulatory

requirements and to document instances of noncompliance when they

occur. The officials also point out that, as part of the their verification

duties, FSIS inspectors now examine 80 carcasses per line per shift

compared with 20 carcasses under the traditional inspection system.9

 In

addition, unscheduled verification tests of additional carcasses are

conducted under each inspection system. Officials also told us that

staffing shortages existed under traditional inspections and that in some

instances, it was not possible to complete verification activities on 20

carcasses for each line. Even accounting for the fourfold increase in the

number of verification checks taking place under the modified inspections

system, one plant experienced a more than twentyfold increase in the

number of noncompliance records for fecal contamination. At other

plants, however, the number of noncompliance records did not even

double.

Another factor may also account for the increases in noncompliance

records issued for fecal failures. Although inspectors and veterinarians

working at pilot project plants who responded to our survey were not

comparing line speeds to those of the traditional inspection system, about

60 percent of them think that line speeds are too fast under modified

inspections to ensure product safety. For example, one inspector stated

that fast line speeds cause inexperienced employees to miss carcasses

with visible fecal contamination. FSIS does not regulate line speeds under

this pilot. However, under traditional inspections, each inspector is

responsible for about 35 carcasses per inspector per minute. Canadian

inspectors told us that line speeds need to be regulated under modified

inspection systems.

 9

Each carcass is inspected under both traditional and modified inspections. In addition,

under both systems, inspectors also select a sample of carcasses at the end of each line to

verify that the inspection system is working as intended.
Page 25 GAO-02-59 Meat and Poultry Pilot Inspection Project

In contrast to FSIS, the Canadian Food Inspection Agency decided to

proceed with a phased approach to nationwide implementation of

Canada’s modified inspection program, the Modernized Poultry Inspection

Program. Under this approach, poultry plants must pass a preliminary

assessment before being accepted into the program and are required to

pass additional audits as they move from one phase of the program to the

next. Producers that choose not to participate in or do not qualify for a

partially modified or fully modified inspection system are inspected under

the traditional system.

Notwithstanding the design and methodology limitations, we found that

data from the chicken pilot that RTI and FSIS inspectors collected show

somewhat different results and do not provide a conclusive basis for FSIS

to make a decision regarding the merits of one inspection system

compared with another. However, 71 percent of the FSIS inspectors and

veterinarians that we surveyed believe that product safety is equal to or

somewhat better under modified inspections. Some of them commented

that under the modified system, they are able to oversee the entire

slaughter line and have more time to collect carcass samples for detailed

examination. However, they also raised concerns about the modified

inspection system.

RTI’s data do not conclusively show that inspection systems at chicken

plants perform as well as or better than they did after they switched to

modified inspections. Under modified inspections, none of the inspection

system at these plants met all of the performance standards that FSIS

developed to measure their performance. Table 3 shows the performance

standards that the inspection systems at these plants were expected to

meet after shifting to modified inspections.10

 10These standards, set by data gathered at the 16 plants under the traditional inspection

system, reflect the performance level of the 12th position. (See p. 13.)

Pilot Project Results

Are Inconclusive, but

FSIS Inspection

Personnel Generally

Support Modified

Inspections

Contractor’s

Data Do Not Conclusively

Show That Modified

Inspection Systems at

Chicken Pilot Plants

Performed “As Well As or

Better Than” Traditional

Inspection Systems
Page 26 GAO-02-59 Meat and Poultry Pilot Inspection Project

Table 3: Performance Standards for Inspection Systems at Chicken Plants in the

Pilot Project

Categories

Performance standards

(percentage of defects

allowed per shift)

Food Safety (FS) 1:

Infectious condition such as septicemia and toxemia

0

FS2:

Digestive content contamination (visible fecal material)

0

Other Consumer Protection (OCP) 1:

Animal diseases such as airsaculitis

1.7

OCP2:

Miscellaneous such as bruises, sores, etc.

52.5

OCP3:

Digestive content contamination (ingesta)

18.6

OCP4:

Dressing defects such as feathers and oil glands

80.0

OCP5:

Dressing defects of digestive tract such as bursa and cloaca

20.8

Source: USDA, FSIS.

Data from the inspection systems at the 11 chicken plants whose

performance RTI measured under both inspection systems show that,

while the majority of the systems met four or more of the performance

standards for food safety and quality under modified inspections, none

met all seven of the standards.11 The data show that 10 of the 11 plants’

inspection systems met the food safety standard for septicemia and/or

toxemia, but only 1 of the 11 systems met the zero-tolerance food safety

standard for visible fecal material, which could contain harmful bacteria

such as E. coli. Most plants’ inspection systems, however, were also

unable to meet the fecal material standard under the traditional inspection

system. All 11 systems at these plants met the quality standard for bruises

and sores (OCP2), but only 3 met the quality standard relating to the

presence of feathers (OCP4). According to FSIS officials, carcasses with

product quality defects are of a lesser concern than those with food safety

defects because they are not associated with foodborne illness and quality

defects can generally be removed from the carcass after it leaves the

slaughter line and moves to the cut-up and processing area.

 11We analyzed the statistical significance of inspection system changes at each plant using

the 95-percent confidence level.
Page 27 GAO-02-59 Meat and Poultry Pilot Inspection Project

Three of the 11 plants’ inspection systems met six of the seven standards,

3 plants met five of the standards, 4 plants met four standards, and 1 plant

met two standards. Table 4 shows which systems met the performance

standards that FSIS set for the pilot project.

Table 4: Eleven Chicken Plants’ Inspection Systems Compliance With Performance Standards (RTI’s Data)

Food safety standards Food quality standards (in percent)

Plant

FS1:

septicemia

and/or

toxemia (zero

allowed)

FS2: visible

fecal material

(zero

allowed)

OCP1: animal

disease, e.g.,

airsaculitis

(1.7 allowed)

OCP2: bruises,

sores, etc.

(52.5 allowed)

OCP3: ingesta

(18.6 allowed)

OCP4:

feathers, oil

glands, etc.

(80.0 allowed)

OCP5: bursa,

cloaca, etc.

(20.8 allowed)

1 Yes No Yes Yes Yes No Yes

2 Yes No Yes Yes Yes No No

3 Yes No Yes Yes Yes No No

4 Yes No Yes Yes Yes Yes Yes

5 Yes No Yes Yes Yes Yes Yes

6 Yes No Yes Yes Yes No No

7 Yes No Yes Yes Yes No No

8 Yes Yes Yes Yes Yes No Yes

9 No No Yes Yes No No No

10 Yes No Yes Yes Yes No Yes

11 Yes No Yes Yes No Yes Yes

Plants

that met

standard

10 1 11 11 9 3 6

Source: RTI.

As discussed earlier, under its contract with FSIS, RTI collected microbial

data as well. Specifically, it analyzed samples for the presence of

Salmonella and generic E. coli. The results show that 9 of the 11 chicken

plants’ inspection systems met the performance standard for Salmonella.

Also, about 80 percent of the samples collected at the11 plants were within

the acceptable ranges that FSIS specifies for generic E. coli.

Irrespective of whether plants’ inspection systems met the performance

standards, we analyzed these data to determine if, under modified

inspections, the systems performed at least as well as they did under

traditional inspections. Table 5 summarizes the results of this analysis.
Page 28 GAO-02-59 Meat and Poultry Pilot Inspection Project

Table 5: Eleven Chicken Plants’ Inspection Systems Performance After Shifting to Modified Inspections (RTI’s Data)

Microbial standards Food safety standards Food quality standards

Performance

Changes

Generic

E. coli Salmonella

FS1:

septicemia

and/or

toxemia

FS2:

visible

fecal

material

OCP1: animal

disease, e.g.

airsaculitis

OCP2:

bruises,

sores, etc.

OCP3:

ingesta

OCP4:

feathers,

oil glands,

etc.

OCP5:

bursa,

cloaca

, etc.

Better 2 2 1 4 5 5 4 3 4

Same 9 4 10 7 5 3 1 1 1

Worse 0 5 0 0 1 3 6 7 6

Total 11 11 11 11 11 11 11 11 11

Source: GAO’s analysis of RTI’s data.

The data show that some of the plants’ inspection systems improved or

remained unchanged in certain categories such as the food safety standard

for septicemia and/or toxemia. On the other hand, the systems’ ability to

meet the quality performance standards deteriorated over time. In general,

most measures of performance showed deterioration in one or more

plants. In fact, every inspection system showed deterioration for at least

one of the nine standards measured. At 5 of the 11 plants, results for

Salmonella were worse under modified inspections than they were under

the traditional inspection system—only 2 improved their performance in

this category. FSIS has stated that any new inspection system should

perform at least as well as the current system of inspections.

FSIS’ data for 10 chicken plants in the pilot project also show that none of

their inspection systems met all seven standards.12 In contrast to the RTI

data, however, the FSIS data show that 7 of the 10 systems met six of

seven standards, and the remaining 3 plants met five of the seven

standards. FSIS officials stated that the differences in results might be

caused, at least in part, by differences in data collection. As part of FSIS

inspectors’ verification duties at the chicken plants participating in the

pilot project, the inspectors sample and analyze carcasses to determine

whether they meet performance standards. Inspectors are not required to

collect and analyze microbial samples, as RTI was required to do for the

pilot project. The data collected cover a longer period than RTI’s and

include much larger samples per plant. The FSIS officials acknowledge,

 12We analyzed FSIS data from 10 of the 11 plants at which RTI collected data for both

traditional and modified inspections because one plant dropped out of the pilot shortly

after RTI completed data collection.

FSIS’ Data Do Not

Conclusively Show That

Plants’ Inspection Systems

Perform “As Well As or

Better” Under Modified

Inspections
Page 29 GAO-02-59 Meat and Poultry Pilot Inspection Project

however, that disparities in the FSIS and RTI data for some performance

standards, such as dressing defects (e.g., feathers and oil glands), suggest

a need for further investigation.

The FSIS data show that 7 of the 10 chicken plants’ inspection systems

met six of the seven performance standards, but none of the plants met the

zero-tolerance standard for fecal material over time. It should be noted,

however, that the actual performance of the 12th position for this standard

was 1.5 percent, not zero as required by FSIS’ performance standard. So

noncompliance with this standard may not provide a basis for comparing

performance of the modified with traditional inspection systems. After

shifting to modified inspections, each of the 10 plants’ systems had

reached a performance that was better than 1.5 percent for this standard.

We analyzed data on individual inspection systems’ performance from the

time that each plant implemented modified inspections through June of

2001. Table 6 shows a summary of the inspection systems’ performance at

individual plants.

Table 6: Ten Chicken Plants’ Inspection Systems Compliance With Performance Standards (FSIS’ Data)

Food safety standards Food quality standards

Plant

FS1:

septicemia

and/or

toxemia

FS2: visible

fecal material

OCP1: animal

disease, e.g.,

airsaculitis)

OCP2:

bruises,

sores, etc.

OCP3:

ingesta

OCP4:

feathers, oil

glands, etc.

OCP5: bursa,

cloaca, etc.

1 Yes No Yes Yes Yes Yes Yes

2 Yes No Yes Yes No Yes Yes

3 Yes No Yes Yes Yes Yes Yes

4 Yes No Yes Yes Yes Yes Yes

5 Yes No Yes Yes Yes Yes Yes

6 No No Yes Yes Yes Yes Yes

7 Yes No Yes Yes Yes Yes Yes

8 Yes No No Yes Yes Yes Yes

9 Yes No Yes Yes Yes Yes Yes

10 Yes No Yes Yes Yes Yes Yes

Plants that

met

standard

9 0 9 10 9 10 10

Note: FSIS inspectors do not collect and analyze generic E. coli and Salmonella as part of their

verification duties in this pilot project.

Source: FSIS inspectors’ verification data from November 2000 through June 2001.
Page 30 GAO-02-59 Meat and Poultry Pilot Inspection Project

Table 6 shows which plants’ inspection systems did not meet specific

performance standards. That is, plant 6 did not meet the food safety

standard for infectious conditions (e.g., septicemia and/or toxemia); plant

8 did not meet the food quality standard for animal diseases such as

airsaculitis; and plant 2 did not meet the food quality standard for digestive

content contamination (ingesta). At the pilot plants, FSIS inspectors are

issuing noncompliance records when plants do not meet the performance

standards; however, FSIS officials have not yet decided how many

instances of noncompliance will be tolerated before the agency can decide

to take further action to ensure regulatory compliance.

We reviewed data collected by FSIS inspectors after the plants shifted to

modified inspections during two different time periods. In comparing

these two periods, we found that in several categories, the inspection

system’s performance at each plant deteriorated over time. Table 7

summarizes the changes for these 10 plants.

Table 7: Changes in Chicken Plants’ Inspection Systems Performance Over Two Time Periods After Shifting to Modified

Inspections (FSIS’ Data)

Food safety standards Food quality standards

Performance

FS1:

septicemia

and/or toxemia

FS2: visible

fecal material

OCP1:

animal

disease, e.g.,

airsaculitis

OCP2:

bruises,

sores, etc.

OCP3:

ingesta

OCP4:

feathers, oil

glands, etc.

OCP5:

bursa,

cloaca, etc.

Better 0 1 1 5 2 0 5

Same 10 9 9 0 2 1 2

Worse 0 0 0 5 6 9 3

Total 10 10 10 10 10 10 10

Note: During the first period, the plants were implementing the original pilot project design in which

there were no FSIS inspectors stationed at fixed positions on the slaughter lines. During the second,

period, a carcass inspector was reinstated at each slaughter line to comply with a court ruling. The

first period starts just after each plant shifted from traditional to modified inspections and ends on

October 31, 2000. The second period starts November 1, 2000, and ends on June 30, 2001.

Source: GAO’s analysis of FSIS inspectors’ verification data from implementation of modified

inspections until June 2001. We used a difference of plus or minus 0.5 percent to analyze the

changes for the seven categories described in the table. We estimated that, on average, 43,000

chickens are processed per line, per shift.

On the basis of this information, in addition to providing plant managers

with feedback on a daily basis, FSIS inspectors could identify plants

whose inspection systems need to improve. In addition, FSIS could

aggregate data from these plants and use this type of analysis to evaluate

whether a specific performance standard needs to be revised. For

example, we made two observations by analyzing the data as follows:
Page 31 GAO-02-59 Meat and Poultry Pilot Inspection Project

• As shown in table 7, 9 of the 10 plants’ performance under the OCP4

standard deteriorated over time. While this standard permits 80 percent of

carcasses per shift to have this defect and still meet the standard, this

trend may suggest the need for FSIS to investigate why the inspection

system’s performance at an individual plant is deteriorating.

• As shown in table 7, 9 of the 10 plants maintained their performance and

one improved under the FS2 standard.

Australian officials told us that under their modified inspection system, the

Australian Quarantine and Inspection Service requires plants to use

process controls to demonstrate continuous improvement in plant

sanitation, microbial pathogen reduction, and quality defect reduction. The

agency uses these data to monitor plant performance and enforce

compliance with standards.

About 70 percent of the respondents to our survey of FSIS inspectors and

veterinarians believe that modified inspections are equal to or somewhat

better than traditional inspections for product safety while more than half

believe that modified inspections are the same as or better than traditional

inspections for product quality. Table 8 summarizes inspectors’ and

veterinarians’ responses to major topics covered by our survey.

(See app. IV for further detail on the survey responses.)

Table 8: Summary of Responses to GAO’s Survey of USDA Inspectors and

Veterinarians

Topic Responses in percentages

Product safety 71: same as or better under modified inspections versus

traditional inspections

24: better under traditional inspections

 5: unsure

Product quality 57: same as or better under modified inspections versus

traditional inspections

39: better under traditional inspections

 5: unsure

Effect of plants’ adding

anti-microbial rinses and

washesa

62: food safety the same as or better than without rinses and

washes

 3: food safety worse than without rinses and washes

10: don’t know its effect on food safety

Line speeds:

Concerning product safety 59: line speed is too fast

39: line speed is about right

 0: line speed is too slow

 2: unable to determine

Most FSIS Inspectors and

Veterinarians Believe That

Modified Inspections Are

Equal to or Better Than

Traditional Inspections for

Ensuring Product Safety
Page 32 GAO-02-59 Meat and Poultry Pilot Inspection Project

Topic Responses in percentages

Concerning product

quality

64: line speed is too fast

33: line speed is about right

 0: line speed is too slow

 2: unable to determine

Original modified

inspection system versus

modified system after

court order:

Concerning product safety

Concerning product

quality

57: modified system after court order is better

27: original modified system is better

 8: they are about the same

 8: no opinion on which is better

45: modified system after court order is better

25: original modified system is better

19: they are about the same

11: no opinion on which is better.

Noncompliance records 76: increased under the modified inspections versus

traditional inspectionsb

11: decreased under the modified inspections versus

traditional inspections

10: the same under the modified inspections and traditional

inspections

 3: don’t know

Training of plant

personnel:

Concerning adequacy of

plant sorter training

Concerning USDA’s

development of

standardized training

curriculum for plant

personnel

Concerning USDA’s

development of a testing

and certification program

for plant personnel

15: plant’s training helped to a great extent

51: plant’s training helped to some extent

28: plant’s training helped to little or no extent

 6: unable to determine

67: FSIS should develop curriculum

17: FSIS should not develop curriculum

15: neutral

 2: unable to determine

64: FSIS should develop this program

18: FSIS should not develop this program

16: neutral

 1: unable to determine.

FSIS training for pilot

project inspectors

40: helped to a great extent

51: helped to some extent

 8: helped to little or no extent

 1: have not received training for the pilot project

 0: unable to determine.

a

This change was experienced by 75 percent of respondents at their plants after modified inspections

were implemented.

b

Some inspectors cited more time and freedom to find defects and write noncompliance records and

more carcasses are sampled under modified inspections.

Note: Because of rounding, numbers may not add up to 100 percent.
Page 33 GAO-02-59 Meat and Poultry Pilot Inspection Project

Nearly all of the inspectors and veterinarians provided additional written

comments on several issues of concern and made suggestions for

improvement. The inspectors commented that under the modified system,

they are able to oversee the entire slaughter line and have more time to

collect carcass samples for detailed examination. In addition, 27 of the 210

inspectors and veterinarians commented that they were concerned about

regulatory noncompliance and enforcement issues. Particularly

noteworthy are comments that indicate that stronger measures should be

in place to reduce repetitive instances of fecal material noncompliance

and that inspectors should have more authority to hold a plant

accountable for multiple and/or repetitive problems. In addition, four

inspectors would like each failure of the zero-tolerance standard for fecal

material to be documented in separate noncompliance records rather than

recording multiple instances in a single record, which may obscure the

frequency of the problem. Thirty-nine inspectors and veterinarians also

raised concerns regarding product quality. In addition, seven inspectors

stated that the quality performance standards need to be tightened so that

fewer defects are overlooked.

In addition, 30 inspectors and veterinarians commented that they are

concerned that the zero-tolerance standard for fecal material is not

adequately enforced, especially since the carcass inspector examines only

the back of the carcass and does not look inside the cavity. Seven

suggested that FSIS place a mirror or provide some other mechanism to

facilitate observation by the carcass inspector.

Finally, 62 inspectors commented that working conditions under the

modified inspection system are better because the inspectors no longer

need to make repetitive motions, as they did when they continuously

examined each carcass on the slaughter line by touch. Sixteen inspectors

also said that they like the ability to physically move from their location at

least once every hour. Twenty-five of them mentioned, however, that the

carcass inspectors’ location might expose them to agents, such as chlorine

or trisodium phosphate that could adversely affect their health.

We believe that a risk-based inspection system—such as the one that

USDA is pilot-testing at chicken plants and is starting at hog plants—has

merit in concept and is consistent with the existing risk-based framework

for HACCP. However, while we support this approach, we believe that the

design of this pilot will not permit USDA to reach conclusions about

whether the new system of modified inspections performs as well as the

traditional system.

Conclusions
Page 34 GAO-02-59 Meat and Poultry Pilot Inspection Project

If, as planned, USDA undertakes to modify its current inspection system

for chicken plants, it will need to consider several factors that we discuss

in this report and that are also key features of the Australian and Canadian

modified inspection systems. First, if USDA issues regulations that require

all chicken plants to modify their inspection systems concurrently without

first determining whether individual plants are able to meet standards, it

may risk including those plants with repeated records of noncompliance.

Such plants may have difficulty meeting their new responsibilities.

Continued participation in a modified inspection system should depend on

the plants’ ability to maintain good performance. Second, adding a

requirement that plants use statistical process control systems to identify

variations in performance will allow the plants to better manage and

control their production processes and will also allow USDA to

appropriately monitor and verify inspection systems’ performance at each

plant over time. Third, without requiring that plant personnel receive

adequate training to undertake carcass defect detection responsibilities,

USDA may jeopardize product safety and quality. At the very least, USDA

should ensure that personnel conducting carcass defect detection duties

have knowledge and training comparable to that of USDA inspectors who

were previously responsible for these duties. Last, inspectors and

veterinarians responding to our survey provided several useful comments

on how to enhance a modified inspection system. For example, they

believe that USDA needs to address multiple instances of plant

noncompliance with regulatory requirements. As discussed in this report,

many plants had repeated instances of noncompliance with a critical food

safety performance standard.

In addition to the current legal challenges, USDA faces future potential

legal challenges unless its statutory inspection authorities are revised.

These challenges will likely hinder the Department’s objective of

modifying its inspection system at meat and poultry slaughter plants. We

continue to believe that, as we have recommended before, the Congress

should consider revising the Meat and Poultry Acts, 21 U.S.C. sections 604

and 455, to provide FSIS with the flexibility and discretion to target its

inspection resources for the most serious food safety risks. Such revisions

would eliminate the requirements that USDA has traditionally

implemented through continuous carcass-by-carcass government

inspection and replace them with a risk-based inspection system that

includes government oversight and verification.
Page 35 GAO-02-59 Meat and Poultry Pilot Inspection Project

If USDA decides to implement modifications to its inspection system, we

recommend that the Secretary of Agriculture direct FSIS to

• phase in the implementation of modified inspections so that only plants

with a good history of regulatory compliance are eligible to participate and

continue in the program;

• require plants to adopt statistical process control systems to manage and

control their production and require FSIS personnel to monitor and verify

these systems;

• in conjunction with industry, develop a training and certification program

for personnel involved in tasks previously performed by federal inspectors

and require that only trained and certified plant personnel perform these

duties; and

• consider the merits of adopting suggestions from inspectors and

veterinarians at pilot project plants contained in this report, such as how

to address repetitive instances of noncompliance with regulatory

requirements.

We further recommend that, if in addition to the current pilot project for

chickens, USDA decides to conduct similar pilots for other species—hogs,

turkeys, or cattle—the Department take steps to ensure that the pilot’s

design and methodology are sufficiently rigorous to allow more valid

conclusions than those in this chicken pilot.

We provided USDA with a draft of this report for review and comment. In

written comments, USDA said that the report’s recommendations were

appropriate and that the agency will address them when it seeks public

comment prior to making any regulatory changes. USDA provided

technical comments, which we incorporated in the report as appropriate.

USDA’s comments and our responses are contained in appendix VII.

We performed our review from December 2000 through October 2001, in

accordance with generally accepted government auditing standards.

As agreed with your offices, unless you publicly announce its contents

earlier, we plan no further distribution of this report until 30 days from the

date of this letter. At that time, we will send copies to congressional

committees with jurisdiction over food safety issues; the embassies of

Recommendations for

Executive Action

Agency Comments

and Our Evaluation
Page 36 GAO-02-59 Meat and Poultry Pilot Inspection Project

Australia and Canada; the Secretary of Agriculture; the Director, Office of

Management and Budget; and other interested parties. We will make

copies available to others on request.

If you have any questions about this report, please contact me at

(202) 512-3841. Major contributors to this report are listed in

appendix VIII.

Lawrence J. Dyckman

Director, Natural Resources

 and Environment
Appendix I: Scope and Methodology

Page 37 GAO-02-59 Meat and Poultry Pilot Inspection Project

To describe the objectives, design, and scope of the pilot project, we

interviewed cognizant government and industry officials. Specifically, we

interviewed officials and/or reviewed documents from the U.S.

Department of Agriculture’s (USDA) Food Safety and Inspection Service

(FSIS); Office of Policy, Program Development, International Programs,

and Field Operations; and Foreign Agriculture Service. We also consulted

with the National Joint Council of Food Inspection Locals, Research

Triangle Institute (RTI), Center for Science in the Public Interest,

Government Accountability Project, European Union Commission,

Consumer Federation of America, National Chicken Council, National

Turkey Federation, National Veterinarian Association, and American Meat

Institute. We obtained information as well from Federal Register notices

and court filings. Additionally, we visited and interviewed plant managers,

USDA inspectors, and veterinarians at five chicken plants and one hog

plant participating in the pilot project. To provide information on similar

projects in Australia and Canada, we visited and toured their meat and

poultry plants. We also interviewed Australian and Canadian government

officials, inspectors, and veterinarians, consumer groups, and plant

managers to obtain their perspectives on their country’s modified

inspection systems. We did not evaluate the validity of the Australian and

Canadian projects’ design and methodology.

To identify limitations, if any, in the project’s design and methodology, we

reviewed USDA’s contract with RTI and discussed data collection and data

analysis methods with statisticians from USDA, the Australian Quarantine

and Inspection Service, and the Canadian Food Inspection Agency. We

obtained input on the pilot project’s design from consumer groups,

industry associations, poultry science experts, and our design

methodologists. We also reviewed the sample sizes selected by USDA and

determined whether the results generated from the volunteer plants can be

generalized to entire population. We interviewed managers of pilot project

plants to identify the length of industry training provided for plant

personnel who replaced USDA inspectors on the slaughter line and any

modifications made to the slaughter line before and during the pilot

project that could affect the project’s results. Moreover, we participated in

FSIS’ three-day training course covering pilot project inspections.

Additionally, we analyzed noncompliance records issued by USDA

inspectors for failures to meet the zero-tolerance standard for fecal

material at chicken plants. We asked USDA to provide a complete set of

noncompliance records for each of the 11 chicken plants that participated

in the baseline and modified inspection phases of the pilot project. That is,

we asked for noncompliance records 1 year before and 1 year after

Appendix I: Scope and Methodology
Appendix I: Scope and Methodology

Page 38 GAO-02-59 Meat and Poultry Pilot Inspection Project

implementation of the modified inspections at each plant. To the best of

our knowledge, we received complete data from all the plants. However,

for some plants, we did not receive 12 months’ worth of data reflecting the

modified inspections because the plants had been operating under

modified inspections for less than a year.

We began our analysis by removing duplicate noncompliance records and

noncompliance records resulting from sanitation and other nonfecal

problems. We then counted the number of noncompliance records

resulting from fecal material violations by month for each plant to

determine whether the total number of noncompliance records was

increasing or decreasing after implementation of the modified inspections.

Since each noncompliance record can document multiple violations, we

also analyzed, on a monthly basis, whether the total number of carcasses

contaminated with fecal material for each plant was increasing or

decreasing under the modified versus traditional inspections.

Furthermore, we counted and compared the number of noncompliance

records generated from the same type of system failure (repetitive

failures) before and after implementation of the modified inspections. For

some months, we did not have any noncompliance records documenting

problems with fecal material.

To evaluate whether the data generated by the pilot project will allow

USDA to reach valid conclusions on the relative effectiveness of the

modified inspection system tested in the pilot project, we reviewed,

compared, and contrasted pilot project data gathered by USDA inspectors

and RTI. We requested and received pilot project data gathered by RTI at

the 11 chicken plants that participated in both the baseline and modified

inspection phases of the pilot project. We compared the individual plants’

results with the two food safety and five food quality performance

standards to determine if each plant was able to meet the standards. We

also determined whether plants improved their performance over time

under modified inspections by comparing individual plants’ results under

modified inspections with their results under traditional inspections. We

also requested USDA inspectors’ verification data from the same 11

chicken plants. Because one plant dropped out of the pilot project in early

2001, we did not evaluate the data from this plant. USDA averaged the

verification results from all of the plants together to determine if the plants

collectively met the performance standards. We chose to average each

plant’s results individually to determine its ability to meet or exceed the

performance standards over time.
Appendix I: Scope and Methodology

Page 39 GAO-02-59 Meat and Poultry Pilot Inspection Project

In addition, we surveyed all pilot project inspectors and veterinarians who

the USDA identified as working at pilot project plants to obtain their views

and concerns regarding the modified inspection system. USDA officials

provided the survey respondents with 1 hour of compensatory time for

completing our survey. In designing our survey, we interviewed pilot

project program officials, inspectors, and veterinarians to determine what

issues should be included in the survey. During the pretesting phase of our

survey, it became evident that respondents considered the survey

questions to be highly sensitive. To address these concerns, we used

procedures to guarantee the anonymity of all survey responses. However,

the use of a separate return postcard for follow-up purposes allowed us to

track which respondents did and did not mail back survey responses. After

developing an initial draft of the survey questionnaire, we visited and

pretested the questionnaire with 10 respondents at two chicken plants and

one hog plant to obtain comments from pilot project inspectors and

veterinarians and to ensure that the survey was appropriate for both

species of animals. We revised the questionnaire in accordance with their

comments. Then we incorporated input from union officials and USDA

officials before sending out the questionnaires. We distributed them by

mail according to a list provided by USDA officials. However, we sent

additional questionnaires to those pilot project inspectors and

veterinarians who were missing from the initial list and who called us to

say they did not receive one. We also sent three follow-up reminders in the

weeks following the initial distribution of the questionnaire. We reviewed

each survey response to identify internal data inconsistencies. We sent out

questionnaires to 225 persons and received 210 responses, for a

93-percent-response rate.

We performed our review from December 2000 through October 2001, in

accordance with generally accepted government auditing standards.
Appendix II: Chronology of Pilot Project

Litigation

Page 40 GAO-02-59 Meat and Poultry Pilot Inspection Project

USDA’s Food Safety and Inspection Service sought public comment on the

development of new inspection models for the slaughter and processing of

meat and poultry in connection with the implementation of its previously

promulgated HACCP rule.1

 FSIS stated that the inspection models would

not include fixed FSIS inspection stations on slaughter lines, as under

existing procedures. Instead, FSIS inspectors would provide oversight at

the slaughter lines and verification that plants were properly implementing

HACCP, among other things. FSIS further stated that, upon completion of

the models project, it will initiate rulemaking, as appropriate, to change

existing inspection procedures.

American Federation of Government Employees (AFGE) filed a complaint

in the U.S. District Court for the District of Columbia seeking to enjoin the

USDA from proceeding with the pilot project, on the grounds that it

violated the Federal Meat Inspection Act, 21 U.S.C. section 604, and the

Poultry Products Inspection Act, 21 U.S.C. section 455, by not requiring

federal government officials to perform carcass-by-carcass postmortem

inspection.

U.S. District Court (Lamberth, R.) granted USDA’s motion for summary

judgment, holding that neither the Federal Meat Inspection Act nor the

Poultry Products Inspection Act was violated by the HACCP-Based

Inspection Models Project. American Federation of Government

Employees v. Glickman, No. 98-0893 (D. D.C. Sept. 23, 1999). The court

reasoned that even though the statutes require postmortem inspections by

USDA inspectors, the word “inspection” in these statutes does not

necessarily mandate direct, physical “organoleptic”—involving sight,

touch, and smell—examination by USDA inspectors.

AFGE appealed the district court’s ruling to the U.S. Court of Appeals for

the District of Columbia Circuit, which reversed the district court’s

decision. The court of appeals held that delegating the task of inspecting

carcasses to plant employees violated the Federal Meat Inspection Act and

the Poultry Products Inspection Act because both statutes require that

federal inspectors, rather than private employees, determine whether a

product is adulterated. American Federation of Government Employees

v. Glickman, 215 F.3d 7, 9 (2000). The court of appeals remanded the case

to the district court for further proceedings consistent with its opinion.

 1

“HACCP-Based Meat and Poultry Inspection Concepts,” 62 Fed. Reg. 31553 (June 10, 1997).

Appendix II: Chronology of Pilot Project

Litigation

6/10/97

4/8/98

9/23/99

6/30/00
Appendix II: Chronology of Pilot Project

Litigation

Page 41 GAO-02-59 Meat and Poultry Pilot Inspection Project

AFGE moved the district court to enter an order declaring that the pilot

project violated the Federal Meat Inspection Act and the Poultry Products

Inspection Act and permanently enjoining USDA from delegating its

statutory duty to inspect each meat and poultry carcass and its parts to

anyone other than federal inspectors.

USDA responded to AFGE’s motion, stating that to comply with the court

of appeals’ decision, FSIS was modifying the pilot project so that an FSIS

inspector would directly inspect every carcass. Specifically, USDA stated

that in the 11 chicken plants in the pilot project, one FSIS inspector would

be stationed toward the end of the slaughter line, between the final wash

and the chiller. This inspector would be responsible for examining each

carcass and determining whether it is adulterated. In the three hog plants,

FSIS inspectors would be placed at up to three fixed locations on the

slaughter line and would be responsible for examining the carcass, head,

and viscera, and determining whether they were adulterated.

All pilot project plants had implemented the redesigned inspection model.2

U.S. District Court for the District of Columbia ruled that the redesigned

pilot project does not violate the court of appeals ruling in American

Federation of Government Employees v. Glickman, 215 F.3d 7 (2000) and

does not violate the Federal Meat Inspection Act and the Poultry Products

Inspection Act. American Federation of Government Employees v.

Glickman, No. 98-893 (Jan. 17, 2001). According to the district court, the

redesigned pilot project is consistent with the court of appeals decision

because the thrust of that opinion was that to satisfy its inspection duties,

FSIS must do more than merely observe others performing inspections,

and in the redesigned project, federal inspectors will make the critical

determination as to whether a product is adulterated. Furthermore, the

redesigned pilot project satisfied the requirements of the federal meat and

poultry inspection statutes, according to the district court, because it

reflects a reasonable interpretation of the meaning of the term

“inspection” within those statutes.

 2

See FSIS Briefing Paper: The History of the HACCP-Based Inspection Models Project

(Jan. 19, 2001).

8/25/00

9/15/00

11/6/00

1/17/01
Appendix II: Chronology of Pilot Project

Litigation

Page 42 GAO-02-59 Meat and Poultry Pilot Inspection Project

AFGE filed a notice of appeal to the U.S. Court of Appeals for the District

of Columbia Circuit. The appeal is still pending.

2/5/01
Appendix III: Performance Standards for Hog

Plants

Page 43 GAO-02-59 Meat and Poultry Pilot Inspection Project

For hog plants, FSIS developed three food safety and three food quality

standards from traditional inspections data (baseline data) gathered at five

plants. (See table 9.) These standards were set slightly below the

performance of the fourth (out of five) plants.1

 At the time of our review,

FSIS did not have complete pilot project data available for the hog plants.

Table 9: Performance Standards for Inspection Systems at Hog Plants in the Pilot

Project

Categories

Performance standards

(percentage of defects

allowed per shift)

Food Safety 1:

Infectious conditions such as septicemia and toxemia

0

Food Safety 2:

Digestive content such as fecal material, ingesta, milk

0

Food Safety 3:

Antemortem symptoms such as neurologic conditions

0

Other Consumer Protection 1:

Carcass pathology such as arthritis, emaciation, etc.

4.1

Other Consumer Protection 2:

Visceral pathology such as enteritis/gastritis, fecal

contamination of viscera, etc.

7.2

Other Consumer Protection 3:

Miscellaneous such as anemia, bile, bruises, scabs, etc.

20.5

Source: USDA, FSIS.

USDA is using the safety and quality performance standards developed for

this project to determine, among other things, if the plants can perform at

least as well as they did under traditional inspections.

 1

USDA developed the hog performance standards from the following formula: (0.25 * 3rd

plant’s performance) + (0.75 * 4th plant’s performance). These standards are comparable to

the 25th percentile established for the chicken pilot.

Appendix III: Performance Standards for Hog

Plants
Appendix IV: Survey Results

Page 44 GAO-02-59 Meat and Poultry Pilot Inspection Project

Appendix IV: Survey Results
Appendix IV: Survey Results

Page 45 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 46 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 47 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 48 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 49 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 50 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 51 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 52 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 53 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 54 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 55 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 56 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 57 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix IV: Survey Results

Page 58 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix V: Australia’s Meat Safety

Enhancement Program

Page 59 GAO-02-59 Meat and Poultry Pilot Inspection Project

Australia’s Meat Safety Enhancement Program (MSEP) is a pilot meat

inspection program developed by the Australian Quarantine and

Inspection Service (AQIS) for use in meat export plants. MSEP is

patterned after the Australian domestic meat inspection system, which is

grounded in the theory of Hazard Analysis Critical Control Point

procedures and uses a quality systems approach to control for hazards in

food processing. Under the Australian inspection approach, slaughter

plants are clearly identified as being responsible for plant process control,

including sanitation, microbial monitoring, and inspection of the product

for quality defects. AQIS sets public health product standards and audits

plant operations to ensure that food safety standards are being achieved.

Australia’s traditional system of meat inspection had an end-product

inspection focus and employed government staff to perform inspections.

However, Australian authorities were concerned that this system did not

adequately address all of the risks to food safety. For example, public

health officials, among others, thought that proper emphasis was not being

given to microbial contaminants, such as E. coli and Salmonella, or

residues of veterinary and agricultural chemicals and antibiotics, which

had emerged as threats to the safety of fresh meat and meat products.

Australia first moved in 1991 to address these emerging food safety issues

with a comprehensive review of domestic meat inspection. Upon the

completion of this review, the review task team outlined changes to

existing meat inspection procedures that it thought would result in a more

cost-effective system while maintaining adequate safeguards to public

health. The recommended changes included strategies for implementing

quality assurance programs and HACCP procedures.

Over an 18-month period, commencing in early 1992, three domestic

slaughter plants developed quality assurance systems that included

replacing regulatory agency inspectors with company employees. AQIS

subsequently approved this new inspection approach for trial. In the trial,

each plant was paired with a nearby similar plant as a control plant. The

control plants continued operations under the traditional meat inspection

system. According to AQIS, the trial results showed that all three plants

operating under their quality assurance systems and using companyemployed meat inspectors were capable of maintaining standards of food

safety at least equal to those of the control plants.

Appendix V: Australia’s Meat Safety

Enhancement Program

Background and

Trends That Led to

the Program
Appendix V: Australia’s Meat Safety

Enhancement Program

Page 60 GAO-02-59 Meat and Poultry Pilot Inspection Project

These findings were examined and endorsed by a scientific panel

composed of officers from AQIS, universities, and meat inspector

representatives of the Public Sector Union and the Australian Meat

Research Council. All Australian domestic meat establishments in each

state subsequently moved to a self-regulated system of meat inspection,

whereby companies assumed responsibility for meat safety. MSEP, which

was implemented in 1996 for meat export facilities, was an outgrowth of

the domestic movement to quality assurance systems using HACCP

procedures and company-employed inspectors.

The MSEP model incorporates the use of routine meat inspection

performed by qualified company employees working under a system

approved and legislated by the government. The government, acting

through AQIS, maintains full-time oversight, verification, and certification

of the final product. As a part of Australia’s MSEP system, a full-time AQIS

veterinary officer, whose salary is reimbursed by the company, is required

to be on duty at all times in each participating export plant. At

establishments producing products for export to the United States, an

AQIS inspector assists the veterinary officer in performing the zero

tolerance and end-product inspection. Company management is

responsible for on-line inspection using company-paid inspectors that

have been trained in accordance with a standardized curriculum and are

under the general supervision of the full-time AQIS veterinary officer. The

officer provides continuous oversight, verification, and daily audit of plant

operations and production control systems. The company-paid inspectors

must also be licensed by their respective states before they may act as

inspectors. To be eligible for a license, they must complete about

600 hours of formal classroom training and pass examinations. The

division of responsibilities between AQIS and companies under MSEP are

as shown in table 10.

Table 10: Division of Responsibility Under Australia’s Meat Safety Enhancement Program

Function Company’s responsibility AQIS’ responsibility

Antemortem inspection Sort healthy animals for slaughter and

separate animals identified as potential

suspects for inspection by AQIS.

Inspect suspect animals and decide on their

disposition.

Postmortem inspection Develop procedures for carcass, head, and

offal inspection using all the inspection

activities required by AQIS.

Set up monitoring checks for all inspection

activities conducted by company-employed

inspectors.

Conduct regular oversight and verification

checks of post-mortem inspection activity at

least daily.

Conduct independent checks of the inspection

procedures during the monthly audit review.

How the MSEP

System Works
Appendix V: Australia’s Meat Safety

Enhancement Program

Page 61 GAO-02-59 Meat and Poultry Pilot Inspection Project

Function Company’s responsibility AQIS’ responsibility

E. coli and Salmonella monitoring Company Quality Assurance officers or

inspectors will take samples for testing.

Check the E. coli samples to ensure that

processing is consistent with good laboratory

practice.

Supervise each Salmonella test.

Review microbiological results in the monthly

review of the plant.

End-of-line inspection Take appropriate corrective action if the AQIS

veterinary officer detects defects.

Perform regular “check the checker” audits of

company inspection and quality assurance

personnel.

Monitoring Monitor all procedures and all aspects of the

HACCP program.

Continuously monitor and recheck

• antemortem and postmortem inspections

and

• microbiological testing and the HACCP

program.

Source: AQIS.

According to AQIS officials, the service has established several

prerequisites for companies that wish to participate in MSEP and sell

products for export. In addition, plants must demonstrate the following

before they can be a part of the program. Specifically, plants must

• have a good operational/coregulatory trade record;

• have demonstrated a commitment to and appreciation of HACCP/Quality

Assurance principles, to include an adequate company infrastructure and

training competencies;

• accept responsibility for performing the sorting (inspection) function;

• develop and maintain a plant-specific, AQIS-approved Meat Safety Quality

Assurance System;

• act as a promotional site for industry/overseas officials; and

• maintain a comprehensive plant quality assurance manual that was

developed by and is “owned” by relevant areas of plant operations. The

manual must be a working document and receive AQIS’ approval. When

approved, the HACCP/Quality Manual acts as a legal contract between the

plant and AQIS on the specific procedures, controls, and standards at the

plant.

Participation in the program is currently voluntary. However, eventually,

all plants will be required to move to the new inspection system.

Prerequisites for

Plants to Participate

in MSEP
Appendix V: Australia’s Meat Safety

Enhancement Program

Page 62 GAO-02-59 Meat and Poultry Pilot Inspection Project

According to AQIS officials, much empirical evidence on the effectiveness

of HACCP quality assurance systems has accumulated since MSEP was

implemented in Australia. AQIS officials told us they are fully confident

that the MSEP approach to regulatory inspection control achieves

guarantees of food safety that are at least equivalent to those of the

traditional approach to meat inspection. They said the system incorporates

a reengineering of meat inspection practices and results in various

changes to the culture or philosophy of food inspection, including changes

that

• prevent rather than correct,

• build quality in rather than rely on end-product inspection,

• place more reliance on microscopic verification,

• transfer responsibility from government to industry for performing certain

functions and maintaining standards, and

• incorporate government verification and certification.

AQIS’ Evaluation of

Its Program
Appendix VI: Canada’s Modernized Poultry

Inspection Program

Page 63 GAO-02-59 Meat and Poultry Pilot Inspection Project

The Canadian Food Inspection Agency (CFIA) recently adopted a new

poultry inspection system that allows industry to progressively assume

defect detection duties and remove defective carcasses from the slaughter

line. The agency began pilot testing this modified inspection system in

September 1997. The pilot project is similar in design to USDA’s pilot

project for poultry. The new system, known as the Modernized Poultry

Inspection Program (MPIP), has goals to reduce microbial pathogens in

raw poultry products and to enhance the use of science and risk-based

management in Canada’s poultry inspection system. MPIP builds on a

precursor program—the Canadian Poultry Inspection Program (CPIP)—

which also allows industry operators to assume some defect-detection

duties. The significant difference between the two programs is that under

MPIP, industry personnel are also responsible for detecting viscera

defects. Under both programs, CFIA inspectors monitor poultry plants’

slaughter operations and verify industry’s compliance with food safety and

quality standards. For poultry plants that choose not to participate in

either of the programs, government inspectors continue to perform all

carcass defect detection and removal duties.

Before CFIA amended its meat inspection regulations to adopt the MPIP

program in May 2001, the agency pilot tested the modified inspection

system at four chicken plants that represented about 10 percent of the

total chicken production. CFIA officials told us that several factors

contributed to their decision to modify the poultry inspection system,

including the desire to make inspections more consistent with the HACCP

framework. The use of the HACCP system is not yet mandatory at meat

and poultry plants in Canada, but it is a prerequisite for participation in

MPIP. In addition, CFIA officials noted that the United States was

engaging in a similar effort to modify its slaughter inspection system and

said that Canada, which was modifying its poultry inspection system,

wanted to obtain equivalency status with the United States to facilitate

bilateral trade. Finally, CFIA officials noted that the Canadian poultry

industry is investing in new evisceration equipment with technical

improvements that permit faster line speeds without compromising food

safety and quality.

MPIP was a voluntary pilot project, and now that Canada has issued

regulations to implement the program nationwide, participation continues

to be voluntary. Prior to being accepted into MPIP, poultry plants must

pass a preliminary assessment. Later, they must pass additional audits as

they move from one phase of the program to the next. The audits, which

Appendix VI: Canada’s Modernized Poultry

Inspection Program

Background

How the MPIP Pilot

Works
Appendix VI: Canada’s Modernized Poultry

Inspection Program

Page 64 GAO-02-59 Meat and Poultry Pilot Inspection Project

are performed by a CFIA audit team, are an important tool for the agency

and plant managers. Clearly stated audit objectives help plant managers to

plan for and address potential deficiencies before they occur, allowing the

plant to succeed in the program. Plants participating in the program move

through three phases.

Phase 1 begins after a CFIA preliminary assessment of a volunteer plant’s

eligibility to participate in the modified inspection program. All volunteer

plants must have been operating under CPIP before participating as an

MPIP pilot plant. As with USDA’s pilot project, baseline data are collected

and analyzed before MPIP implementation. That is, microbial and quality

defect-detection data are collected at volunteer plants while they are

operating under CPIP. In addition, plant personnel are trained and

accredited in their new role as plant defect detectors.

Phase 2 resembles the transition phase in USDA’s pilot project. It is the

trial phase during which CFIA inspectors back up plant personnel in

detecting defects on the slaughter line. Plant personnel are required to

pass four weekly practical on-line tests as viscera detectors. In addition,

the plant must also pass a national compliance and verification review.

Phase 3 entails operating under MPIP and the collection and analysis of

data after MPIP’s implementation at the plant. Accredited plant personnel

detect carcass, cavity, and viscera defects, while plant preselectors cull

carcasses with specific processing and pathology defects, such as

contamination from the intestines and septicemia. On average, a Canadian

poultry plant runs one evisceration line and will have one veterinarian-incharge and two CFIA inspectors (one evisceration floor inspector per line

working on the slaughter floor and one processing area inspector). If an

equivalency agreement with the United States requires federal inspection

of each and every carcass, CFIA will place one on-line government

inspector on each slaughter line at plants that want to export to the United

States.

CFIA has established several prerequisites that plants must meet before

they are accepted to participate in MPIP. CFIA considers, at a minimum,

the following criteria for each plant:

• The state of the current plant inspection system, its quality control

operations, and the gap between these and MPIP.

Prerequisites for

Plants to Participate

in the MPIP Pilot
Appendix VI: Canada’s Modernized Poultry

Inspection Program

Page 65 GAO-02-59 Meat and Poultry Pilot Inspection Project

• The stage of plant-specific HACCP system development and

implementation—at a minimum, the plant must have a functioning HACCP

system in place to commence MPIP.

• The implementation of E. coli and Salmonella testing protocols.

• The types of poultry processing conducted by the plant, as it is desirable

to have several classes of poultry slaughtered to improve efficiency in

training staff, data collection, and implementing future pilots in the area.

• The geographic location (desire to implement MPIP and collect data from

across Canada).

• The compliance history of the plant.

• The range of the health status of flocks and average condemnation rates in

the plant selected to pilot MPIP to ensure adequate training and

accreditation of plant defect detectors to handle condemnation rates and a

range of disease conditions under an HACCP-MPIP system.

CFIA’s MPIP national performance standard categories for poultry are

similar but not identical to USDA’s pilot project standards for chickens.

The poultry standards address the following factors, among others:

• Microbial contamination.

• Processing defects, including all types of contamination—such as fecal

and bile—extraneous material, extensive bruising, extensive

overscald/mutilation, and inadequate bleeding.1

• Pathology defects, including septicemia/toxemia, airsacculitis, ascites,

emaciation, cellulitis, dark-colored carcasses, and other diseases.2

• Finished product defects, such as bruises, feathers, trachea, oil glands,

lungs, intestines, and crop.3

To analyze microbial contamination as part of the MPIP project, in 1997

and 1998, CFIA performed a national baseline survey at 36, or 55 percent,

of the 65 federally registered chicken plants nationwide over a 1-year

period. CFIA’s performance standards for microbial contamination are

 1

USDA’s food safety performance standard 2 and other consumer protection (food quality)

performance standards 2 and 3 for the pilot project include these types of defects.

2

USDA’s food safety performance standard 1 and other consumer protection (food quality)

performance standard 1 for the pilot project include these types of defects.

3

USDA’s other consumer protection (food quality) performance standards 4 and 5 for the

pilot project include these types of defects.

Performance

Standards
Appendix VI: Canada’s Modernized Poultry

Inspection Program

Page 66 GAO-02-59 Meat and Poultry Pilot Inspection Project

identical to USDA’s Pathogen Reduction: HACCP regulations for generic

E. coli and Salmonella.

To evaluate both processing and pathology defects, in 1997 and 1998, CFIA

conducted another national survey to establish the average level of

overlooked processing and pathology defects on carcasses passed by CFIA

inspectors or by trained and accredited plant detectors under the

traditional and CPIP methods of chicken inspection. Performance

standards for processing and pathology defects were developed from data

gathered at 35, or 54 percent, of the 65 federally registered chicken plants

nationwide. The performance standards for processing defects were set at

4.0 percent; however, as part of the MPIP project, CFIA subsequently

tightened them to 2.5 percent, meaning that 97.5 percent of the carcasses

passed by plant employees must be free of processing defects. The

performance standards for pathology defects were set at 0.4 percent,

meaning that 99.6 percent of the carcasses passed by plant employees

must be free of pathology defects.

CFIA conducted an additional national survey to amend its Finished

Product Standards program. This is the same program developed and used

by USDA in all chicken plants operating under the traditional carcass-bycarcass inspection systems. CFIA, however, has tightened the Finished

Products Standards’ pass/fail criteria for plants operating under CPIP’s

and MPIP’s inspection systems. In addition, CFIA continues to enforce a

zero-tolerance policy for fecal contamination on ready-to-chill carcasses

as part of this program.

CFIA developed an extensive training program for government and

industry personnel to ensure that everyone participating in MPIP has the

necessary skills and knowledge to perform his/her new duties. The

training includes classroom training, a final exam, in-plant training, and

the demonstration of competency. After successfully completing the

training, CFIA trainers accredit the trainees.

The entire MPIP training program was developed by CFIA. CFIA officials

believe that no outside organization can replicate in-house expertise. In

addition, in order to encourage union cooperation, CFIA inspectors

conduct the training. At MPIP plants, every CFIA employee must

participate in the training program.

MPIP Pilot Success

Attributed to

Extensive Training of

Inspectors and

Industry
Appendix VI: Canada’s Modernized Poultry

Inspection Program

Page 67 GAO-02-59 Meat and Poultry Pilot Inspection Project

CFIA officials modeled MPIP’s industry-training program on the training

program they developed for their own staff. The agency adopted a trainthe-trainer approach for industry. That is, CFIA inspectors train the plant

trainers who in turn train the plant employees (defect detectors). The

training is conducted in two phases. Phase 1 covers classroom and on-thejob training and concludes when the defect detector is accredited to

perform his/her new duties under MPIP. The plant managers decide which

plant employees will become defect detectors and receive MPIP training.

The training covers three types of defect detection: (1) preselection before

evisceration, (2) internal examination of carcasses, and (3) poultry viscera

examination. On-the-job training ensures that defect detectors learn how

to identify carcass pathology conditions that are indicative of disease.

Once the defect detectors are comfortable with identifying diseases, they

are tested on 20 carcasses and the corresponding viscera to see if they can

correctly decide whether the carcasses pass or need to be examined by the

veterinarian. In addition, the defect detectors have to demonstrate the

ability to detect defects on fast-moving lines. The defect detector must

pass three 15-minute tests to be accredited.

After the defect detector is certified, he/she enters a 4-week trial period

(phase 2) where he/she performs his/her new MPIP duties on the slaughter

line. Defect detectors can be trained and accredited in more than one

station (cavity, carcass, and/or viscera) and will have weekly evaluations

at each of these stations during the trial phase. In addition, the trainer

evaluates defect detectors on a periodic basis, as specified in the plant’s

HACCP plan. Plant defect detectors are reevaluated on an ongoing basis

and lose their accreditation if they fail a periodic review test. Ongoing

testing ensures continuing competence as line speeds and/or disease

patterns change. CFIA reserves the right to conduct additional

reaccreditation tests of plant detectors at any time. In addition, CFIA

reserves the right to conduct additional tests for missed defective

carcasses at any time.

According to CFIA officials, MPIP is successful because the data

generated from the pilot plants show a downward trend in carcass defects.

The plants that participate in the MPIP pilot are performing better than the

performance set by the national standards. Part of the program’s strength

is that CFIA officials periodically review data results with plant managers

to discuss the plant’s record of performance against the standards.

Through discussions with plant managers, CFIA officials learn what types

of corrective actions are successful in addressing the plant’s system

Under MPIP Pilot,

CFIA Officials

Monitor Plant

Performance Trends
Appendix VI: Canada’s Modernized Poultry

Inspection Program

Page 68 GAO-02-59 Meat and Poultry Pilot Inspection Project

failures. In turn, CFIA officials share this knowledge with managers of

other pilot plants to help them improve their own process controls.

CFIA modified its meat-inspection regulations to implement MPIP on a

nationwide basis on May 23, 2001. The agency will continue to maintain

three types of poultry inspection systems (traditional, CPIP, and MPIP).

CFIA does not plan to make changes to the MPIP program and will

continue to use a phased approach for implementing MPIP at volunteer

plants. Plants will still be required to pass a preliminary assessment before

being accepted into MPIP. Under MPIP, CFIA officials will continually

monitor the plants’ performance through informal discussions and formal

audits. The agency expects MPIP plants to perform better than the

performance set by the national standards for the microbial, food safety,

and other consumer protection defects. If a plant fails to meet these

standards, the plant will lose its license to operate under MPIP and must

be relicensed to operate either under CPIP or traditional inspections. In

addition, the agency expects MPIP plants to continually improve their

performance over time. Furthermore, CFIA will continue to emphasize the

importance of training through its mandatory training program and

certification requirements for both CFIA employees and industry

personnel.

Since CFIA has formally adopted MPIP, the agency will maintain an

oversight inspector and a veterinarian-in-charge at each MPIP plant.

Unlike USDA’s pilot project, CFIA’s MPIP program does not require an

on-line carcass inspector performing carcass-by-carcass inspections.

However, if the equivalency agreement with the United States requires an

on-line carcass inspector, CFIA will comply with this requirement and

place a CFIA inspector on the slaughter line at those plants that export to

the United States.

Current Status of

MPIP
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 69 GAO-02-59 Meat and Poultry Pilot Inspection Project

Appendix VII: Comments From the U.S.

Department of Agriculture

Note: GAO’s comments

supplementing those in

the report’s text appear at

the end of this appendix.
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 70 GAO-02-59 Meat and Poultry Pilot Inspection Project

See comment 1.

See comment 2.
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 71 GAO-02-59 Meat and Poultry Pilot Inspection Project

Now on pp. 2-3.

See comment 9.

Now on p. 2.

See comment 8.

See comment 7.

See comment 6.

See comment 5.

See comment 4.

See comment 3.
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 72 GAO-02-59 Meat and Poultry Pilot Inspection Project

See comment 18.

See comment 17.

See comment 16.

See comment 15.

See comment 14.

See comment 13.

See comment 12.

See comment 11.

See comment 10.
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 73 GAO-02-59 Meat and Poultry Pilot Inspection Project

Now on pp. 27-28.

See comment 23.

Now on p. 26.

See comment 22.

Now on p. 24.

See comment 21.

Now on p. 24.

See comment 20.

Now on p. 23.

See comment 19.
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 74 GAO-02-59 Meat and Poultry Pilot Inspection Project

See comment 30.

See comment 29.

Now on p. 34.

See comment 28.

Now on p. 30.

See comment 27.

Now on p. 29.

See comment 26.

Now on p. 28.

See comment 25.

Now on p. 27.

See comment 24.
Appendix VII: Comments From the U.S. Department of

Agriculture

Page 75 GAO-02-59 Meat and Poultry Pilot Inspection Project
Appendix VII: Comments From the U.S.

Department of Agriculture

Page 76 GAO-02-59 Meat and Poultry Pilot Inspection Project

The following are GAO’s comments on the U.S. Department of

Agriculture’s letter dated November 30, 2001.

1. We believe that the report’s title describes the overall results of our

study. Therefore, we have not modified the title as USDA suggested.

2. We do not agree that our report oversimplifies the goals of the pilot

project. The pilot project’s objective was to compare the

accomplishments—as measured by USDA’s safety and quality

performance standards—of two inspection systems at volunteer

plants. We understand that the plants participating in the pilot project

made various other changes at different times during the pilot project.

However, the only change that USDA required of plants volunteering to

participate in the pilot was the replacement of USDA inspectors with

plant personnel. As USDA states, this project was designed as a

“before-and-after” experiment. The “before” refers to traditional

inspections with USDA inspectors performing the carcass defect

detection duties, and the “after” refers to the plant employees’

performance of the carcass defect detection duties. As explained in

our report, USDA’s independent contractor first went to each

participating plant to collect carcass defect data under traditional

inspections, waited until the plant employees went through a transition

period, and then returned to collect the same type of data after plant

employees assumed the inspectors’ duties. If USDA wants to compare

the accomplishments of two different inspection systems that include

many variables, it is critical to use a control group. Indeed, the lack of

a control group in USDA’s study prevents them from knowing the

extent to which plants that do not participate in the study (about 175

plants nationwide) made changes in their processing systems that

improved safety and quality while continuing to inspect under the

traditional system. We continue to believe that the design of the

USDA’s pilot project does not permit USDA to interpret what changes

account for the experiment’s outcomes.

3. We agree with the premise that the use of volunteer plants is a

legitimate approach and our report acknowledges that, once USDA

decided to go forward with this pilot project design, it could not

compel plants to participate and, therefore, had to rely on volunteer

plants. However, given that USDA decided to proceed with this design,

our report points out that USDA had the option of randomly selecting a

subgroup from the volunteer plants to serve as a control group. The

problem with relying on volunteers for this study is that there are a

GAO’s Comments
Appendix VII: Comments From the U.S.

Department of Agriculture

Page 77 GAO-02-59 Meat and Poultry Pilot Inspection Project

small number of plants participating—there are only 11 chicken plants

for which before and after data have been collected. Even if the plants

had been randomly selected, they would not represent a sufficient

number to enable USDA to generalize the results to the entire

population of chicken plants. Therefore, USDA cannot assume that the

results from these 11 plants would apply to all the plants that in the

future may participate in a modified inspection system. Finally, neither

USDA nor RTI provided GAO with any evidence that the pilot project

was either designed or analyzed as a time-series study.

4. We do not agree with USDA that the plants participating in the study

are typical of the industry in that they represent diversity in geography,

corporate structure, management styles, numbers of evisceration lines,

product distribution, and other variables. As figure 2 of our report

shows, the pilot project has before-and-after data for 11 plants that are

located in the southern region of the United States. Therefore,

particularly with respect to microbial data, the results from these

plants may not be indicative of how plants in other areas of the United

States may behave. Geographic distribution and seasonal variations

are important variables that cannot be ignored in assessing a program.

As a recent USDA study points out, Salmonella and E. coli 0157:H7

rates appear to differ dramatically by season, while geography seems

to affect only Salmonella results. When, as part of this study, USDA

researchers investigated the regional patterns, they found higher

positive rates for Salmonella in warmer climates. For example, in

southern operations, the rates were about 8 percent, whereas the rate

was just under 5 percent in northern operations. This study highlights

the importance of variables such as seasonal variations and geographic

distribution. As discussed in our report, the design of USDA’s pilot

project did not appropriately consider these variables.

5. Our report accurately describes the duties of FSIS inspectors and plant

personnel as they have changed during the course of the litigation

outlined in appendix II. However, as noted in comment 9, we have

modified our characterization of current plant personnel activities in

accordance with USDA comments.

6. We made the technical changes as appropriate.

7. We disagree. Our report does not describe the program as being

designed as a cost-saving initiative.
Appendix VII: Comments From the U.S.

Department of Agriculture

Page 78 GAO-02-59 Meat and Poultry Pilot Inspection Project

8. We modified our report to state that there are only 11 chicken plants

with before-and-after data.

9. We have made the first change suggested. However, USDA has not

provided any information to demonstrate that inspection resources are

being utilized elsewhere in the farm-to-table continuum other than

performing for plant inspections.

10. We disagree. We believe that the report consistently describes the

purpose and type of data collected by RTI. See comment 2.

11. We made the technical changes as appropriate.

12. We made the technical changes as appropriate.

13. We made the technical changes as appropriate.

14. We disagree. According to the documentation provided by USDA, there

is a model for fed cattle and, as of October 2001, no cattle plants had

volunteered to participate.

15. Table 1 and its footnotes describe the number of plants participating in

the project.

16. We made the technical changes as appropriate.

17. We made the technical changes as appropriate.

18. We agree with the agency that it should look at this during the

rulemaking process.

19. We disagree. Our report acknowledges that inspectors’ verification

checks increased from 20 to 80 at pilot project plants, a fourfold

increase in verification checks. However, the number of

noncompliance records issued by inspectors at some of the plants far

exceeded the level expected by the increase in verification checks. At

one plant, there was a twentyfold increase in noncompliance reports.

Our intent is to alert USDA of the fact that, after shifting to modified

inspections, some of the plants appear to have problems with fecal

contamination. USDA’s HACCP Rules of Practice contemplate the

Department’s taking action for noncompliance by withholding the

marks of inspection or by suspending inspections in instances where it

decides that it is necessary to do so because of multiple or recurring
Appendix VII: Comments From the U.S.

Department of Agriculture

Page 79 GAO-02-59 Meat and Poultry Pilot Inspection Project

regulatory noncompliances evidenced by noncompliance records. We

understand from FSIS officials that they are now considering how

many repeated instances of noncompliance will result in further

regulatory action.

20. We agree that actions taken when there are multiple instances of

noncompliance are based on the same decision-making criteria in both

project pilot plants and in traditional plants.

21. About 60 percent of our survey respondents said that line speeds are

somewhat too fast or much too fast for food safety.

22. We made the technical changes as appropriate.

23. We agree with USDA’s comment that, as shown in table 5 of our report,

the generic E. coli results show improvement after the plants shifted to

modified inspections. As the table shows, 2 of the 11 plants improved

while 9 remained unchanged.

24. We disagree. As stated in our report, we believe that looking at

individual plant inspection systems’ performance is relevant and

appropriate because each plant is expected to meet the standards.

Therefore, in table 4, our report shows how each plant performed, as

measured by USDA’s performance standards.

25. We made the technical changes as appropriate.

26. We disagree. As stated in our report, we believe that looking at

individual plant inspection systems’ performance is relevant and

appropriate. Therefore, in table 6, our report shows how each plant

performed, as measured by USDA’s performance standards.

27. We agree. Our original criterion for analyzing changes in the data

presented in—table 7 was .01 percent—which we used because it

reflects the level of precision in the data that USDA provided to us.

However, in response to USDA’s comment that declines in

performance of fractions of 1 percent may not be meaningfully

different, we modified our criterion to 0.5 percent. This means that a

plant’s change from “same” to “worse” or “better” would now

represent 215 chickens per line per shift compared with 4.3 chickens

per line per shift in terms of quality and safety. By changing our

analysis, we now find that in 16 instances, the plants’ performance

under modified inspections moved from “better” to “same” and in 8
Appendix VII: Comments From the U.S.

Department of Agriculture

Page 80 GAO-02-59 Meat and Poultry Pilot Inspection Project

instances the plants’ performance moved from “worse” to “same.” In

summary, even after changing our criterion, we find that in several

instances, the plants are not meeting USDA’s goal—that is, that

modified inspections should achieve at least the same level of

performance as traditional inspections. We modified table 6 to reflect

these changes. In table 6, we clearly identify which plants have or have

not met each individual performance standard.

28. We agree. As stated in our report, we believe that training of plant

personnel is an issue that must be addressed by USDA.

29. We made the technical changes as appropriate.

30. We made the technical changes as appropriate.
Appendix VIII: GAO Contact and Staff

Acknowledgments

Page 81 GAO-02-59 Meat and Poultry Pilot Inspection Project

Robert C. Summers (404) 679-1839

In addition to the name above, Maria Cristina Gobin, John M. Nicholson,

Diana P. Cheng, Shana B. Wallace, Fran A. Featherston, Karen K. Keegan,

and Cynthia Norris made key contributions to this report.

Appendix VIII: GAO Contact and Staff

Acknowledgments

GAO Contact

Acknowledgments

(360024)
The General Accounting Office, the investigative arm of Congress, exists to

support Congress in meeting its constitutional responsibilities and to help

improve the performance and accountability of the federal government for the

American people. GAO examines the use of public funds; evaluates federal

programs and policies; and provides analyses, recommendations, and other

assistance to help Congress make informed oversight, policy, and funding

decisions. GAO’s commitment to good government is reflected in its core values

of accountability, integrity, and reliability.

The fastest and easiest way to obtain copies of GAO documents is through the

Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-text files of

current reports and testimony and an expanding archive of older products. The

Web site features a search engine to help you locate documents using key words

and phrases. You can print these documents in their entirety, including charts and

other graphics.

Each day, GAO issues a list of newly released reports, testimony, and

correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site

daily. The list contains links to the full-text document files. To have GAO E-mail

this list to you every afternoon, go to our home page and complete the easy-to-use

electronic order form found under “To Order GAO Products.”

The first copy of each printed report is free. Additional copies are $2 each. A

check or money order should be made out to the Superintendent of Documents.

GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a

single address are discounted 25 percent. Orders should be sent to:

U.S. General Accounting Office

P.O. Box 37050

Washington, D.C. 20013

To order by phone: Voice: (202) 512-6000

TDD: (301) 413-0006

Fax: (202) 258-4066

GAO Building

Room 1100, 700 4th Street, NW (corner of 4th and G Streets, NW)

Washington, D.C. 20013

Contact:

Web site: www.gao.gov/fraudnet/fraudnet.htm,

E-mail: fraudnet@gao.gov, or

1-800-424-5454 (automated answering system).

Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800

U.S. General Accounting Office, 441 G. Street NW, Room 7149,

Washington, D.C. 20548

GAO’s Mission

Obtaining Copies of

GAO Reports and

Testimony

Order by Mail or Phone

Visit GAO’s Document

Distribution Center

To Report Fraud,

Waste, and Abuse in

Federal Programs

Public Affairs