# Presentation: 705523

## Across Establishment Ranking Concept
For Processing and Slaughter 

February 5, 2008

Curtis Travis, PhD
Science Applications International Corporation

**For Processing and Slaughter **
- February 5, 2008
- Curtis Travis, PhD
- Science Applications International Corporation

## Goal of Ranking Algorithm

- Focus FSIS resources to ensure food safety systems are working effectively to further achieve FSIS’ public health mission
- Across establishments---focus on establishments with evidence of a lack of process control
- Within establishments---focus on most vulnerable food safety system areas
- Remain Resource Neutral

## Prioritize Establishments Based on Need for Inspection

- Establishments

-  In-Depth Inspection

-  Focused Inspection

-   Routine Inspection

## Risk Has Two Components: Magnitude and Hazard

- Risk Has Two Components: Magnitude and Hazard

- Risk = Magnitude x Hazard
- Both components help FSIS better focus its inspection activities
- Magnitude (attribution)
- Focus on pathogen-product pairs that most contribute to human disease
- Hazard (effectiveness of process control)
- Focus on establishments with less than optimal food safety process control

## Conceptual Approach

- Establishment
- Public Health
- Risk Ranking

- =

- Establishment  Volume  /
- National
- Volume

- Public Health Attribution

- Measures over time (i.e., verification testing, health based NRs)

- Episodic Measures
- (i.e. FSAs, recalls, enforcements)

- Magnitude
- Public Health Impact

- Hazard
- Indicators of Process Control

- Fraction of human disease an establishment might cause if a contamination event were to occur

- Indicator of how well establishment is maintaining process control

## Determining Level of Inspection (LOI)

- Sort establishments into LOI 3 based on specified criteria
- Sort establishments into LOI 1 based on specified criteria
- Remaining establishments are placed into LOI 2
- Within LOI 2, rank order establishments by their contribution to public health

## Levels of Inspection

**Levels of Inspection**

- Routine Inspection (LOI 1):
- Maintain routine in-plant inspection
- Focused verification activities, prompted by in plant results to identify and prevent possible problems (i.e. new with-in establishment inspection system) [For Cause Prompts]
- Focused Inspection (LOI 2):
- Focus in-plant verification activities at vulnerable points to identify whether there is a food safety system problem [Directed Procedures and For Cause Prompts]
- In-Depth Inspection (LOI 3):
- Focused in-plant verification activities [Directed Procedures and For Cause Prompts]
- Deploy highly trained resources for in depth assessments and verification (i.e. EIAO/PHV performing FSAs and IVTs)

## Slide 8

## Overview Ranking Algorithm

**Overview Ranking Algorithm**

**Establishments**

**LOI 1**

**LOI 2**

**LOI 3**

**Separate Based on **
**Process Control Effectiveness**

**___**
**___**
**___**
**___**
**___**
**___**

**Routine**
**Inspection Activities**

**Focused**
**In-plant Inspection activities**** ***

**In-Depth Inspection**
** ****Activities**

**Rank LOI 2 on**
**Potential Public Health Impact**

*****Focus in-plant verification activities at points where greatest microbial contamination or growth occurs if process control is not maintained

## In-Depth Inspection (LOI 3) Criteria*Satisfies ANY of the following criteria to be in LOI 3:

**In-Depth Inspection (LOI 3) Criteria*****Satisfies ANY of the following criteria to be in LOI 3:**

- A positive *E. coli* O157:H7 verification test in past month
- A positive *Lm*, *Salmonella* or *E. coli *O157 in RTE products in past month
- Establishment in *Salmonella *Category III
- Establishment is linked to a disease outbreak
- Establishment has sustained structural damage due to a natural disaster
- *Establishments remain in LOI 3 until their FSA and IVT results demonstrate they are in compliance or an enforcement action is taken.
- *Algorithm will be run monthly

## In-Depth Inspection (LOI 3) Criteria (Cont) Satisfies ANY of the following criteria to be in LOI 3:

- In STEPS database more than once in past 120 days
- Shipment of Specified Risk Material
- Enforcement action (i.e. NOIE) or adulterated or misbranded product shipped (captures recalls)
- Highest percentile of health-related NR rates (e.g. SRM, Insanitary Dressing, Zero Tolerance, Residue) over some time period to be determined
- Use of NRs justified through predictive analysis
- Repetitive *Salmonella* serotypes of human health concern or  PFGE matches**
- ** This criterion is not currently applied. FSIS will collect this data as part of the *Salmonella *Initiative Program.

## Predictive Analysis of Utility of NRs

- If NR occurs, what is increased probability of positive *Salmonella*
- 	in next two weeks?
- Health-related NRs---probability 3 times higher
- Industry-proposed NRs---probability about
- 	 2.3 times higher
- All NRs---probability about 1.9 times higher
- Differences are statistically significant
- All are statistically greater than 1.0

## Routine Inspection (LOI 1) Criteria Must satisfy ALL of the following criteria to be in LOI 1:

**Routine Inspection (LOI 1) Criteria**** ****Must satisfy ALL of the following criteria to be in LOI 1:**

- No positive FSIS *E. coli* O157:H7 in past 120 days or until establishment determined *E.coli* free from follow up sampling*
- No positive FSIS *Lm*, *Salmonella* or *E. coli* O157:H7 in RTE products in past 120 days
- No Enforcement action (i.e. NOIE) in past 4 months or adulterated or misbranded product in commerce in past 4 months (captures recalls including those related to human illness)
- *120 days is based upon the approximate time required for 16 follow up *E. coli *samples

## Routine Inspection (LOI 1) Criteria (Cont) Must satisfy ALL of the following criteria to be in LOI 1:

- Establishment not linked to disease outbreak in past 6 months
- Lower percentile of *Salmonella *percent positives on most recent sample set, unannounced sampling or other *Salmonella* testing programs
- Lower percentile of health-related NR rates (e.g. SRM, Insanitary Dressing, Zero Tolerance, Residue) over a period of time to be determined
- Use of NRs justified through predictive analysis

## Routine Inspection (LOI 1) Criteria (Cont) Must satisfy ALL of the following criteria to be in LOI 1:

- Lower percentile on most recent FSA score**
- Lower percentile of scores on focused in-plant verification questions—vulnerable points**
- Lower percentile of *Salmonella* serotypes of human health concern or PFGE matches***
- ** FSIS will collect this data in its new Public Health Information System
- *** FSIS will collect this data as part of the *Salmonella Initiative Program. *

## Focused Inspection (LOI 2) CriteriaEstablishments not in LOI 3 or LOI 1

**Focused Inspection (LOI 2) Criteria****Establishments not in LOI 3 or LOI 1**

*E. coli* positive within last 120 days or still undergoing follow-up sampling, for which FSA has been completed
- Positive *Lm*, *Salmonella* or *E. coli* O157 sample within last 4 months, for which FSA has been completed
- Enforcement action (e.g., NOIE) or adulterated or misbranded product shipped (captures recalls including those related to human illness) in past 4 months, for which FSA has been completed and corrective actions have been verified

## Focused Inspection (LOI 2) Criteria (Cont)

- Based on past history of *Salmonella* testing, above the lower percentile cut-point for LOI 1 for percent positives on most recent sample set, unannounced sampling or other *Salmonella* testing programs
- Above the lowest health-related NR rate percentile (cut-point for LOI 1) and below the highest health-related NR rate percentile (cut-point for LO3)
- In STEPS database more than once in past 120 days, for which FSA has been completed

## Focused Inspection (LOI 2) Criteria (Cont)

- Above lower percentile (cut-point for LOI 1) on most recent FSA score
- Above lower percentile (cut-point for LOI 1) of scores on focused in-plant verification questions—vulnerable points
- Above lower percentile (cut-point for LOI 1) of *Salmonella* serotypes of human health concern or PFGE matches
- Establishment confirmed to be cause of outbreak in past 6 months, for which FSA has been completed

## Rank LOI 2  Establishments Based on Public Health Impact

**Rank LOI 2  Establishments Based on ****Public Health Impact**

- Rank order LOI 2 establishments based on public health impact (fractional volume x attribution)
- Product fractional volume = Vi / ∑Vi ,where sum is over product class (e.g. broilers, ground beef)
- Attribution for pathogen-product class (e.g. ground beef consumption causes 34% of all E. coli O157:H7 illnesses)
- Potential public impact = Vi / ∑Vi x attribution
- If establishment produces more than one product with same pathogen of concern, select max potential public impact

##  Rank LOI 2  Establishments (Cont)

-  **Rank LOI 2  Establishments (Cont)**

- Sort the ranked establishments into one of four pathogen categories–*Salmonella*, *Lm*, *E. coli, Campylobacter*)–or place in fifth category–no pathogen results
- For each pathogen category, place upper and lower 50th percentile into categories LOI 2a and LOI 2b, respectively*
- *Depending upon FSIS priorities (e.g. performance standards, seasonality) the categorization of LOI 2a and LOI 2b may be amended for specific pathogens.

## Summary 

- Summary

- PHRBI algorithm is designed to:
- Focus inspection on establishments most needing attention
- Focus inspection on most vulnerable food safety system areas
- Verify that food safety systems are working optimally

## Summary

- Approach has multiple advantages
- Transparent
- Focuses on plants with evidence of lack of process control
- All plants with high pathogen levels are ranked high
- All plants with health-related problems (recalls, outbreaks, enforcement actions) are ranked high
- Categorization independent of production volume
- Compatible with FSIS sampling programs

## Next Steps

- Apply algorithm to existing FSIS data
- External reviews
- Examine relationship to pathogen-specific sampling programs