# Presentation: 204667

## Expert Elicitation and Its Role in RBIDaniel Engeljohn, Ph.D.Office of Policy, Program, and Employee Development, FSIS, USDA

## It’s use:  For ranking the relative public health risk of FSIS inspected products

## Goal of Risk Based Inspection

- To allocate FSIS resources in a manner that best protects public health
- Estimations of the contribution of a particular food to a subsequent human illness is difficult
- There are several ways to attribute food to illness; each has its own strengths and caveats

## Methods for Attribution

- Three different methods for illness attribution.
- Expert elicitation
- Predictive models
- Epidemiological data analysis
- By comparing the results of multiple methods FSIS can improve the final rankings.

- Please Note::  Many of the following ideas were summarized by the food attribution working group and published in Emerging Infectious Diseases,  Michael B. Batz, Michael P. Doyle, J. Glenn Morris, Jr., John Painter, Ruby Singh, Robert V. Tauxe, Michael R. Taylor, and Danilo M.A. Lo Fo Wong, for the Food Attribution Working Group

## Expert Elicitations

- FSIS recently conducted two independent expert elicitations to define the relative risks posed to public health by processed meat and poultry products
- Two elicitations:
- One with 22 experts and the other with 12 experts from industry, academia and public health sector
- Both generally defined 24 meat and poultry product categories
- Both ranked these foods to relative risk to public health
- Strengths
- Can be performed even when there is little available data
- Can help to resolve discrepancies between other methods
- Caveats:
- Judgment based
- May be less objective than data driven decisions

## Predictive Models

- Estimate the public health risk of a food based on a variety of data inputs. By estimating illnesses attributed to each product a ranking of FSIS foods can be determined.
- FSIS has developed predictive models to estimate the number of illnesses attributed to meat, poultry, egg products (including raw and ready-to-eat products).
- Strengths:
- Objective
- Based on observable data
- Can help identify data needs
- Caveats:
- Reliance on human dose-response curves or surveillance data can create uncertainty in predictions of human illness
- Modeling post production mitigations (i.e. consumer handling and cooking) is complex

## Predictive Models:  Molecular Subtype Data

- Molecular markers for microbial source tracking
- PFGE, serotypes, subtypes, phage types, and genotype assays
- Denmark *Salmonella* process
- A working model for how *salmonella* serotypes are used to attribute salmonellosis to particular foods
- Caveats:
- Some cases of salmonellosis do not have a serotype or subtype strongly associated with a single food or species of source animal

## Predictive Models: Estimated Human Salmonellosis Cases from Meat, Poultry, and Eggs, 1998-2003*

- Human case data, by serotype
- CDC Public Health Laboratory Information System, 1998 – 2003
- Food prevalence data, by serotype
- FSIS in-plant samples of beef, ground beef, chicken, turkey, pork, and processed egg products, 1998 – 2003
- Pennsylvania Pilot Project samples of shell eggs, ca. 1995
- Food consumption data
- USDA Economic Research Service, 1998 – 2003

- * Approximately 28,000 to 30,000 culture-confirmed salmonellosis cases per year, 1998-2003

## Attribution of the Predicted Median Cases of Listeriosis for US Population on per Serving and per Annum Basis

**Attribution of the Predicted Median Cases ****of Listeriosis for US Population ****on per Serving and per Annum Basis**

## 
Factors in volume production, product risk, plant sanitation practices, and historical culture results from product, food contact surfaces, and the production environment
Model is updated on a monthly basis using current culture results and ranks all eligible RTE establishments according to Lm risk for the purpose of sample allocation
Model can be used to predict the total number of illnesses expected annually for each level of sanitation 

- Factors in volume production, product risk, plant sanitation practices, and historical culture results from product, food contact surfaces, and the production environment
- Model is updated on a monthly basis using current culture results and ranks all eligible RTE establishments according to Lm risk for the purpose of sample allocation
- Model can be used to predict the total number of illnesses expected annually for each level of sanitation

- The Lm model predicts which RTE establishments are likely to produce contaminated product based on prevalence in FSIS testing

**Predictive Models:**

## Epidemiological Data Analysis

- Uses actual human illness events clinically diagnosed and attributed to a food vehicle by epidemiological evidence
- Foodborne Outbreak Reporting System (eFORS)
- CDC FoodNet (active surveillance of foodborne illness)
- Strengths:
- Objective
- Based on observed human illnesses
- Caveats:
- Foodborne illness grossly under diagnosed
- Uncertainty about the coefficients used to compensate
- Interviews, surveys, and case control studies used to attribute clinical cases to foods rely on human reporting and can be “anecdotal”
- Data sets tend to be small. (see first caveat).

## Epidemiological Data Analysis:

**Step**

** **

**Input**

** **

***Salmonella***

** **

***Campylobacter***

*** ***

**Data Source**

** **

- 1

- Incidence

- 14.4/100,00

- 12.6/100,000

- FoodNet Annual

- Report

-  for

-  2003

- 6

- 2

- Populati

- on estimate 2003

- 290,788,976

- 290,788,976

- US Census Bureau

- 16

- 3

- Underreporting multiplier

- 38

- 38

- Mead et al.

- 7

- 4

- Foodborne fraction

- 0.95

- 0.8

- 0

- Mead et al

- .

- 7

- 5

- Poultry attribution fraction

- 0.3351

- 0.6936

- Food Safety Research

- Consortium

- 17;18

- 6

- Young poultry fraction

- 0.838

- 0.838

- ERS

- 19

- 7

- Total illnesses

- 1,591,197

- 1,392,297

- Step = (1)(2)(3)

- 8

- Total foodborne illnesses

- 1,511,637

- 1,113,838

- Step = (4)(7)

- 9

- Total foodborne

-  illnesses

- from poultry

- 498,840

- 772,558

- Step = (5)(8)

- 10.

- Total

- foodborne illnesses

- from

- young chicken

- s

- 424,389

- 647,251

- Step = (6)(9)

- 1

- 1.

- Costs per illness

- $

- 1800

- $610

- ERS

- 19

- 12.

- Total Cost of illnesses fro

- m

- product and pathogen

- $

- 759 million

- $

- 39

- 0 million

- Step = (10

- )(11

- )

**Estimating illnesses due to young chickens using**
**epidemiological data from the United States 2003**

**Epidemiological Data Analysis:**

## Summary:  Current Status 

- In the immediate future, FSIS intends to base estimates of food item inherent risk on the results of the first expert elicitation
- The risk rankings in RBI are currently based on the results of two expert elicitations with close agreement
- FSIS will continue to combine expert rankings with other information (e.g., volume) to arrive at an overall establishment’s inherent risk

## Summary:  FSIS Next Steps 

- FSIS will continue to assess expert elicitation results with current knowledge from epidemiological studies and predictive modeling via risk assessments regarding attribution of foodborne disease risk
- FSIS is further assessing whether/how to collapse/expand the 24 food categories from the expert elicitations into categories more closely informed by other work (e.g., the Center for Science in the Public Interest outbreak groupings)

## Summary:  FSIS Next Steps (continued)

- FSIS will assess the use of epidemiological data to incorporate severity of illness
- FSIS will evaluate methods to incorporate cost of illness into expert risk rankings
- FSIS will conduct sensitivity analyses to ensure appropriate weights are given to components of RBI, including the expert risk rankings
- FSIS will update the algorithm, its components, and the expert risk rankings, as appropriate (e.g., to incorporate emerging pathogens, as warranted)
- FSIS will maintain ongoing communication with all stakeholders, state and local partners in order to maintain a shared understanding of attribution