Review of the HACCP-Based Inspection Models Project 

by the National Alliance for Food Safety Technical Team 

B.M.Hargis1, D.V.M., Ph.D., P.A. Curtis2, Ph.D., M.G. Johnson3, Ph.D., 

and J.D. Williams

4

, Ph.D. 

l Professor and Director, JKS Poultry Health Research Laboratory, Center of Excellence for 

Poultry Science, University of Arkansas, Fayetteville, Arkansas 72701 

2

Professor and Director, Poultry Product Safety & Quality Program, Department of Poultry

Science, Auburn University, Auburn, AL 36849

3

Professor of Food Microbiology and Safety, Department of Food Sciences, University of 

Arkansas, Fayetteville, Arkansas 72701 

4

Professor of Biostatistics (retired), Department of Veterinary Pathobiology, College of 

Veterinary Medicine, Texas A&M University, College Station, Texas 77843

Summary 

In September and October 2002 a technical review team selected by the National Alliance for Food 

Safety was contracted by the United States Department of Agriculture Food Safety Inspection Service (FSIS) to 

review and evaluate the data collected by Research Triangle Institute (RTI) with regard to the HACCP 

Inspection Models Project (HIMP) in young chicken plants. Data collected by RTI and FSIS as related to the 

HIMP Models Project were provided by FSIS for technical review. These included data collected during the 

Baseline phase reflecting quality and safety of poultry inspected under the traditional system, data collected 

during the Models phase as volunteer plants implemented their new HACCP and Process Control Plans and 

began to make necessary adjustments to these procedures. Also included in this review were data collected 

during the Models phase by FSIS personnel with regard to carcass inspection and verification tests, and data 

collected after the Models phase as plants continued to operate under the new procedures as Model plants. 

As charged, the review team has focused on the validity of study design and methodology to permit an 

interpretation of organoleptic and microbial data to access the accomplishments of traditional vs. HIMP 

systems. Overall, the review team found that the study design and methodology were indeed valid and provide a 

useful and legitimate comparison of the two systems. The review team then characterized the observed 

differences in food safety performance data between the inspection systems for young chickens using data from 

Food Safety categories 1 and 2/microbia l testing, and also using data from Other Consumer Protection 

categories 1-5. The review team noted some issues related to optimal design and interpretation, but finds that 

overall the data collected were quite meaningful and useful and that the study was well designed and conducted 

under real-world conditions and limitations. Overall the HIMP system compared favorably to the traditional 

system of inspection with regard to meaningful pa11ameters to consumers of poultry and poultry products. 

Several specific observations and conclusions are discussed below. 
Introduction 

In 1985 and 1987, the National Academy of Sciences published two reports providing strong 

suggestions to progress from traditional organoleptic inspection systems to a risk-based (HACCP) approach to 

meat and poultry inspection (NAS; 1985,1987), and in a subsequent report indicated that the mandatory 

inspection of all carcasses is an impediment to improving the safety of meat and poultry (NAS, 1998). Because 

under the traditional system, FSIS inspectors have some responsibilities related to sorting of carcasses, 

arguments have been made that these activities should appropriately be assigned to the plant under FSIS 

oversight (FSIS, 1997). 

To evaluate a model system of inspection, operating under HACCP principles, the HIMP project was 

initiated to evaluate the ability of the HIMP model to improve the safety of processed animals and poultry 

(FSIS, 1998). The present review has focused on the validity of study design and methodology to permit an 

interpretation of organoleptic and microbial data to assess the accomplishments of traditional vs. HIMP systems 

and reviewed the observed differences in food safety performance data between the inspection systems for 

young chickens using data from Food Safety categories 1 and 2/microbial testing, and also using data from 

Other Consumer Protection categories (1-5). Data reviewed include the Baseline and Model data collected at 16 

volunteer plants that process young chickens collected by RTI International (RTI) under contract with FSIS. 

Materials and Methods 

The technical review team was selected by the National Alliance for Food Safety under contract with 

FSIS. The review team consisted of nationally and internationally recognized experts in the area of poultry 

microbiology, food safety, poultry health, poultry processing, and statistical evaluation (please see attachment). 

These individuals reviewed the documents provided by FSIS. 

Documents reviewed: 

An Overview of the HACCP-Based Inspection Models Project -(USDA/FSIS generated) 

Chronological Order of Events -Young Chicken -(USDA/FSIS generated) 

The HACCP Based Inspection Models Project -(USDA/FSIS generated) 

Clarification of Agency Issuances (Models I vs. Redesign) -(USDA/FSIS generated) 

Federal Register Notice: HIMP Concepts June 1997 

HIMP: Diseases and Conditions Observable in Meat and Poultry (USDA/FSIS generated) 
Organoleptic and Microbial Study Design and Collection Protocols (USDA/FSIS generated) 

RTI Individual Establishment Data -11 Establishments (Bar Graphs) (RTI generated) 

RTI Organoleptic Summary Data -11 and 16 Establishments (RTI generated) 

RTI Analysis Per Performance Category (Incomplete) (RTI generated) 

Traditional Versus HACCP-Based Inspection: Results from a Poultry Slaughter Project. Journal of Food 

Protection 2001; 64(6): 826-832. (RTI generated) 

RTI HIMP Presentation for the National Advisory Committee on Meat and Poultry Inspection, June 5, 2002. 

(Narrative) (RTI generated) 

RTI HIMP Presentation for the National Advisory Committee on Meat and Poultry Inspection, June 5, 2002. 

(Power Point) (RTI generated) 

FSIS HIMP Presentation for the National Advisory Committee on Meat and Poultry Inspection, June 5, 2002. 

(Organoleptic and Microbial Verification Data) (FSIS generated) 

Review of HACCP-based Inspection Model Project In-Plant Slaughter. (Dr. Phil Kott) (USDA/National 

Agricultural Statistics Service generated) 

Summary and General Conclusions of HIMP Study Design. (Dr. Surak) (Professor of Food Science, Clemson University) 

Statistical Review of Salmonella Data. (Dr. Chris Wiesen) (RTI generated) 

FSIS HIMP Microbial Verification Data. (Through September 30, 2002) (USDA/FSIS generated) 

Results and Discussion 

As requested, the initial focus of this review is on the specific questions posed by FSIS, as follows: 

Question 1: Does the design and methodology of the study permit an interpretation of organoleptic and microbial data to 

assess the accomplishments of traditional and HIMP inspection systems? 

In considering this question, the review team has considered the tremendous merit of actual field 

evaluation as opposed to laboratory monitoring and considers that this value outweighs specific criticisms 

related to problematic implementation of a real-world comparison of this magnitude. With regard to 

comparisons of the 
acquired data, the relatively large data sets acquired over time greatly reduce concerns over many variables such 

as antemortem flock health status. Nevertheless, this issue presents an unknown with regard to the comparisons 

made. While generic E. coli tend to be relatively stable over seasons, there is evidence of seasonal variation 

with regard to both antemortem Salmonella infection in poultry and carcass contamination levels, and several 

have observed corresponding seasonal differences within plants. As such, there is a potential of seasonal bias 

especially related to the albeit intensive but compressed Salmonella sampling during only a 6 week period 

regarding the RTI-generated data sets. 

From a pure science-based perspective, one can argue that a side-by-side comparison of the two systems in 

individual plants using split flocks would provide valuable data not obtainable fro~ the study. This type of design would 

have negated potential criticisms related to variability of 4lltemortemhealth status and seasonal conditions and would have 

provided a specific control set for each plant evaluated. While this would most appropriately allow for direct comparison 

of two established systems, the HIMP model system was in fact under development and, importantly, HACCP-based 

inspection is expected to change and react to variable conditions over time. 

From a statistical perspective, several flaws were noted in the RTI-collected data. These differences include a time 

span of approximately 2 years between collection of the Baseline (control) data and the data for the HIMP model system. 

It is also important to note that many variables changed over this time span so there are multiple factors be tween the 

compared groups. Several other issues were also addressed in the manuscript that resulted from the 8 plant comparison 

(Cates et al., 2001). These flaws are possibly overcome by the 21 HIMP and 21 Traditional plants that are currently under 

comparison by FSIS. However, information related to design, basis for selection, and statistical analysis were not 

available for this review. 

When considering the hypothetical design of simultaneous evaluation, at any given time a comparison 

between the two systems could theoretically be simultaneously evaluated. However, as the HIMP model system 

in fact requires continuous HACCP-based change and adjustment, this design would not completely resolve the 

question of appropriate controls. Indeed, as in-plant HACCP allowed for identification of useful and appropriate 

interventions, simultaneous evaluation of the two systems would require disregard of important potential 

interventions that could improve quality and safety of the product in line(s) operating under the traditional 

system. Corresponding adjustments to the traditional system would then negate the value of those lines 

functioning as controls in the traditional system. Alternatively, disregard of appropriate HACCP-based 

intervention steps identified by the HIMP model within a plant, which could lead to improved product quality 

and safety in carcasses presented to inspection under the traditional system, would be a difficult-to-defend 

process. 

Although not a perfect statistical model, the intended and appropriate comparison was between the 

traditional inspection system and the HIMP model system, and these systems were made during implementation 

of the new system, with ongoing adjustments as needed. Based on kno wn constraints within poultry processing 

plants relating to personnel, physical space, available equipment, and investments required to adopt a model 
inspection system, the proposed experimental design is perhaps the only workable model for comparing systems 

over a large geographic area with the large data sets collected. It is the conclusion of the review team that the 

implemented design was appropriate in most respects. 

As discussed above, the nature of HACCP-based inspection will require ongoing modifications and adjustments 

to ensure optimal performance and to minimize risks associated with public health hazards and product defects. As such, 

following adoption of the HIMP model, the adoption of many changes in process control were required by plants. Among 

these, required and appropriate HACCP -based changes in some cases were addition of microbial rinses, changes in major 

equipment and product flow, personnel changes, and others. As such, any plant adopting the HIMP model system was 

substantially changed, and the identification of pre-modification controls for individual plants is not strictly necessary for 

analysis and comparison. While the continued participation of 11 of the 16 plants from Baseline data collection to HIMP 

Model data collection is conceptually attractive, there is no sound basis for excluding data from the additional 5 HIMP 

model plants that volunteered to replace those that did not choose to continue with the project. In fact, with 16 total plants 

participating during data collection under both traditional and HIMP inspection systems, it can be argued that individual 

plants serving as appropriate controls for within-plant comparisons is in fact not necessary, and that 11 plants participated 

in all phases of data collection is somewhat remarkable given the voluntary nature of the project. Simple comparison of 

data from the two groups is not wrong, but the fact that 11/16 of the plants also serve as internal controls does reduce 

experimental error, increases confidence in the data, and reduces potential criticism that bias was introduced with regard 

to plant selection for the study. 

Regarding geographic distribution of volunteer plants selected for this study, it is important to note that selected 

plants were largely distributed across the Southeastern United States and, in fact, represent the states supplying the 

majority (~80%) of domestic chicken production. Furthermore, plant design, equipment, and procedures within poultry 

plants are relatively uniform and large geographic influences on comparison of these inspection systems are not 

expected. Similarly, as the size range of plants included in this study are representative of almost 90% of chickens 

slaughtered in FSIS-inspected facilities within the United States, the selected plants are appropriate for this comparison. It 

is indeed very difficult to hypothesize a geographical bias in this study. 

In conclusion, the review team has determined that the overall design and methodology were generally 

appropriate, and were perhaps the best available options to allow for comparison of organoleptic data between 

the traditional and HIMP systems. The design and methodology seems likely to detect differences in generic 

E. coli recoveries related to differences between the HIMP model system and the traditional system as these are 

believed to be much more stable with regard to seasonal changes than Salmonella (antemortem infections). 

Potential seasonal bias in microbial sampling times could bias the Salmonella recovery data in this study, as 

discussed below. 
Question 2: Evaluate and characterize the differences in food safety performance data between inspection systems for 

young chickens, using data from Food Safety categories 1 and 2 and microbial testing. 

For both FS-l and FS-2, clear and important reductions in defects were observed in carcasses processed 

under the HIMP model system. The review team agrees with the presentation and interpretation by RTI. 

Question 3: Evaluation of aggregate and an individual establishment basis by comparing the accomplishments in 

traditional and HIMP systems for the 11 establishments participating in both RTI traditional and RTI models sampling. 

Regarding the organoleptic data of these 11 establishments, the primary question asked is which of these systems 

results in fewer organoleptic defects. As such, it is perhaps more useful to focus on the actual differences in defect scores 

rather than the more arbitrary performance standards. Significant (P<0.05) differences in each of the organoleptic 

parameters evaluated were noted between systems. Apparent improvement in the average score was noted with carcasses 

processed under the HIMP models system for FS-l, FS-2, OCP-l, and OCP-2. Relatively consistent with these data, a 

numerical increase in the percentage of plants which met performance standards for each category were noted for FS-l, 

and OCP-l through 3, with no difference in the percentage of plants meeting performance standards of FS-2. However, 

average score for carcasses processed under the HIMP models system was higher for OCP-3, OCP-4, and OCP-5. It is 

perhaps important to consider that there is a very real difference between statistically significant changes and those that 

are likely to be meaningful. For example, the average score was reduced in the HIMP models group by 18-fold for FS-l, 

almost 5-fold for FS-2, almost 3-fold for OCP-l, and almost 2-fold for OCP-2. In contrast, the increase in average score in 

the HIMP models group was only 10% for OCP-3, 15% for OCP-4, and 44% for OCP-5 as compared with the Baseline 

data. It is also important to note that the Food Safety categories are considered to reflect much more important defects as 

related to product safety. 

Overall, the review team considers these data to evidence marked improvement in the organoleptic defect scores 

of carcasses processed under the HIMP model system as compared to the Baseline data collected under the Traditional 

system. 

Similar to the organoleptic data (above), marked and significant reductions in generic E. coli recovery were 

reported in carcasses processed under the HIMP model system as compared to those processed under the Traditional or 

Baseline system. However, a statistically significant increase in Salmonella recovery from carcasses processed under the 

HIMP model system (9.2%) was observed as compared to the Baseline data (4.6%). However, this observation was not 

consistent with overall data from the combined 16 plants (see below) or with the FSIS data (also discussed below). 
Question 4: Descriptive analysis of individual establishment performance in traditional and HIMP systems. 

A complete data set provided by the "RTI Individual Establishment Data" was available only for the 

initial 11 plants for most categories. Review of these data did not affect the conclusions apparent in the 

summary data for these 11 plants (please see Question 3, above). There was no evidence that the summary data 

were unduly weighted by extreme variations, with the exception of some very high individua l Salmonella 

recovery incidence numbers (see below). 

Question 5: Aggregate analysis comparing the accomplishments of the 16 establishments participating in the 

RTI traditional sampling to the accomplishments of the 16 establishments participating in RTI models 

sampling. 

As mentioned above for the slightly more limited organoleptic data from the 11 establishments with preand post-HIMP implementation (above), the primary question asked is which of these systems results in fewer 

organoleptic defects, and focus on the actual differences in defect scores rather than the more arbitrary 

performance standards is perhaps most useful. Significant (P<0.05) differences in each of the organoleptic 

parameters evaluated were noted between systems. Apparent improvement in the average score was noted with 

carcasses processed under the HIMP models system for FS-l, FS-2, OCP-l, OCP-2, and OCP-3. Relatively 

consistent with these data, a numerical increase in the percentage of plants which met performance standards for 

each category. As discussed above, average scores for carcasses professed under the HIMP models system were 

significantly (p < 0.05) increased by a small factor for OCP-4 and OCP-5 as compared to the Baseline data. 

Overall, the review team considers these data to evidence marked improvement in the organoleptic defect scores 

of carcasses processed under the HIMP model system as compared to the Baseline data collected under the Traditional 

system. 

Similar to the organoleptic data, marked and statistically significant reductions in generic E. coli recovery were 

reported in carcasses processed under the HIMP model system as compared to those processed under the Traditional or 

Baseline system. As with the 11 original plants (above) Salmonella recovery was less noticeably but more frequently 

recovered from carcasses processed under the HIMP model system. As mentioned above, this observation was unexpected 

given the clear reductions in FS-l and FS-2 scores, and marked decrease in generic E. coli recovery attributed to carcasses 

processed under the HIMP model system. As these latter parameters are factors that can be readily controlled by a 

HACCP-based system, it is perhaps not surprising that improvements were noted. As mentioned above, one potential 

pitfall of the microbiological data collected by RTI is the compressed time frame during collection (6 weeks per plant). 

Salmonella contamination of broiler carcasses has been conclusively shown to be of antemortem origin, and flocks that 

have been identified as highly infected antemortem have been associated with greatly increased carcass contamination at 
processing (Hargis et al., 2001 for review). Seasonal and intermittent patterns of Salmonella infection of broiler chickens 

have also been reported. With these considerations in mind, the authors suggest that the Salmonella data for this set of 16 

plants are inconclusive and that more seasonally balanced data should be considered (please see below). 

Question 6: Consider additional data provided for this analysis including FSIS organoleptic and PR/HACCP 

microbial verification data for young chicken establishments participating in HIMP and national Salmonella 

PR/HACCP verification data for young chickens. 

Information provided for this review did not include design or methodology information and only limited 

statistical analysis. The FSIS PR/HACCP data, current to September 30, 2002, rela ted to Salmonella recovery from 21 

establishments operating under the Traditional system and 21 establishments operating under the HIMP model system 

were provided by plant and in summary form for review. Importantly, these data apparently reflect 'rolling' consecutive 

sampling dates representing at least 51 working days, and in practice approximately 3 months. Thus, the potential effect of 

seasonal bias is reduced by this expanded time frame of sample collection as compared to the RTI-collected data. In this 

case, the Salmonella prevalence in samples from plants operating under the traditional system (8.0%) is not significantly 

different than the prevalence in samples from plants operating under the HIMP system (8.2%). Furthermore, of the 

completed sample sets, 94% of plants operating under the traditional system and 96.9% of the plants operating under the 

HIMP system passed the testing criteria. 

These data suggest that implementation of the HIMP system does not affect Salmonella recovery frequency. 

Conclusions 

Intensive review of the experimental design and methodologies allows for several conclusions. The experimental 

design is generally appropriate for a field study of this nature, and the methodologies employed generally allow for 

interpretation and comparison of these systems. Overall, adoption of the HIMP model system has clearly improved 

important scores related to the more important organoleptic parameters described as FS-l (septicemia, toxemia) and FS-2 

(fecal contamination), and has markedly reduced contamination of carcasses with generic E. coli as a generally-accepted 

parameter related to plant hygiene and process control.. Adoption of the HIMP model system has also resulted in 

improved scores related to OCP-l (animal diseases), OCP-2 (miscellaneous conditions) and OCP-3 (ingesta 

contamination), but has resulted in slightly increased scores for dressing defects (OCP-4 and OCP-5). 

Salmonella recovery seemed to be increased in plants inspected under the HIMP system when the smallest data 

set available was considered (11 plants) but less-so when the larger data set were considered (16 plants). Although it can 

be argued that inclusion of the 5 replacement plants not included in the Baseline study is a potential bias in this study, 

there are no founded reasons to exclude them from consideration. While not impossible, the authors are unable to identify 

any possible factor associated with the HIMP system that could be responsible for increased recovery of Salmonella. A 

much more believable hypothesis is that seasonal or random influences affected the Salmonella data set collected in a 
single short time frame. This hypothesis is supported by lack of consistency of this data with the more recently FSIS 

generated PR/HACCP Salmonella data consisting of 21 plants under the traditional system and 21 plants under the HIMP 

system, with data collected over at least 51 working days. The authors strongly suggest that these data be carefully further 

evaluated and considered. The overall results of the Salmonella testing data are equivocal at best without inclusion of the 

FSIS-collected data. We therefore urge a careful evaluation of the design, plant selection criteria, methodologies, and 

resulting data to assure that this interpretation is correct. Nevertheless, the data as presented would argue that 

implementation of the HIMP system is not contributing to increased Salmonella contamination. Conversely, at this time 

there is no evidence that implementation of the HIMP system is reducing the incidence of Salmonella recovery from 

chicken carcasses. 

The authors also reviewed the GAO document which commented on the results of this project (GAO, 2001). In 

general, we found the conclusions and recommendations of this report to be confusing, inconsistent, sometimes 

contradictory, and frequently inconsistent with the methodologies employed and data generated by this study. Criticisms 

of the study are largely based on 4 points, the first point being that the "chicken pilot that the USDA designed lacks a 

control group". The principle criticism here appears to be related to the fact that multiple factors were simultaneously 

changed as the HIMP system was adopted. The authors believe that this criticism does not take into account the concept 

that it is a system, and that constant adjustments to varying conditions is in fact the goal of a HACCP -based system. Even 

considering the statistical issues noted above, if it is considered that "systems", and not "who did the inspection", were 

compared, the controls are indeed appropriate for a study of this type. 

The second major criticism of the GAO report is related to the fact that "the chicken plants that volunteered to 

participate in the baseline measurement phase of the pilot were not randomly selected, and they did not include plants 

from all chicken-producing areas or plants of all sizes". Inability to randomly select and require that plants participate in 

an expensive study of this type is indeed a limitation of the study. However, the plants selected represent the states 

supplying the majority (~80%) of domestic chicken production and the size range of plants included in this study are 

representative of almost 90% of chickens slaughtered in FSIS-inspected facilities within the United States. Furthermore, 

plant design, equipment, and procedures within poultry plants are relatively uniform and large geographic influences on 

comparison of these inspection systems are not expected. It is indeed very difficult to hypothesize a geographic or plantselection bias in this study. Furthermore, the remarkable fact that 11 of the 16 original volunteer plants persisted 

throughout the study actually enhances the ability to compare the two systems as discussed above. 

The third major GAO criticism of the study is a claim that "the pilot project's methodology did not take into 

account variables such as seasonal changes and plant modifications that could affect project results. For example, after 

the project began, many plants added antimicrobial rinses and washers, which usually reduce the levels of microbial 

contamination. ". As discussed above, seasonal-related Salmonella levels and compressed sampling time for microbial 

surveillance could indeed be a factor limiting interpretation of this specific parameter. However, the addition of HACCPbased interventions during HIMP model implementation was in fact the goal. 

The fourth and final major GAO criticism of the study is the “...pilot project did not include features of the 

modified inspection systems in Australia and Canada that would be important considerations in ensuring the successful 

implementation of a modified inspection system nationwide. For example, during the pilot project, USDA did not require 
the training of plant employees." While not specifically a part of this review, this statement is not entirely consistent with 

the authors' understanding of HACCP training that was occurring prior to and during implementation of the HIMP models 

project. For example, the InternationalHACCP Alliance indicates that more than 20,000 people have participated in their 

volunteer HACCP training courses for the entire food industry and 190 participants have completed the HIMP training 

courses in this program alone since inception. Nevertheless, if training were inadequate prior to implementation of this 

project, this would seem to enhance the relative effectiveness of this system and argue that improvements could be made 

with increased training. 

Of special interest in the GAO report were the responses to GAO's survey of USDA inspectors and veterinarians 

as related to this review (p. 31). Of the 210 inspectors and veterinarians surveyed, 71% indicated that product safety was 

the same or better under HIMP system as compared to the traditional system, and 57% indicated that product quality was 

the same or improved. Based on evaluation of the data generated from this study, the authors must agree with the majority 

of USDA inspectors and veterinarians in that safety and quality of young chickens inspected under the HIMP system is 

either the same or improved. 

Summary Conclusions 

1. The authors urge continued FSIS oversight and continuous re-evaluation as HIMP is more broadly 

implemented. 

2. At this time, no convincing arguments were identified which indicate that adoption of the modified 

system, under regulatory supervision, would increase risk. 

3. More importantly, the authors find that there are several lines of evidence that strongly argue process 

improvements from the consumer perspective as related to adoption of the HIMP system. 

References 

Cates, S.C., D.W. Anderson, S.A. Karns, and P.A. Brown. Traditional Versus HACCP-Based Inspection: 

Results from a Poultry Slaughter Project. Journal of Food Protection 2001; 64(6): 826-832. 

Hargis, B.M., D.J. Caldwell, and J.A. Byrd. 2001. Chapter 8 Microbiological Pathogens: Live Poultry 

Considerations in: Poultry Meat Processing. CRC Press LLC, Boca Raton, FL. 

National Academy of Sciences (NAS). 1985. Meat and poultry inspection: the scientific basis of the nation's 

program. National Academy Press, Washington, DC. 

National Academy of Sciences (NAS). 1987. Poultry inspection: the basis for a risk-assessment approach. 

National Academy Press, Washington, DC. 

National Academy of Sciences (NAS). 1987. Ensuring safe food from production to consumption. National 

Academy Press, Washington, DC. 
U.S. Department of Agriculture, Food Safety and Inspection Service (FSIS). July 7, 1998. HACCP-based 

inspection models project in-plant slaughter, [Internet,www], ADDRESS: 

http://www.fsis.usda.gov/OA/haccp/himp.htm.