# Presentation: 798944

## Ranking Foodborne Risks Under Uncertainty: Attribution Using Outbreaks and Expert Judgment

**Michael Batz**
- University of Maryland School of Medicine
- mbatz@epi.umaryland.edu
**USDA FSIS Public Meeting: Attributing Illness to Food**
- Arlington, VA  |  5 April 2007

## The Food Safety Research Consortium

- Develop analytic and decision tools towards a more risk- and science-based food safety system
- Interdisciplinary collaboration of seven institutions:

| University of Maryland, Baltimore   -- Mike Taylor (Chair)   -- Glenn Morris Jr.   -- Mike Batz (Executive Director)
University of California at Davis   -- Juliana Ruzante
University of Georgia   -- Mike Doyle

 | Iowa State University   -- Helen Jensen
University of Massachusetts   -- Julie Caswell
Michigan State University   -- Ewen Todd
Resources for the Future   -- Alan Krupnick   -- Sandy Hoffmann |
| --- | --- |

## FSRC and Food Attribution

- FSRC Food Attribution Workshop
- October 2003 – Atlanta, GA
- Funding from FDA, USDA, and CDC
- Resulted in EID article*
- International Conference: Priority Setting of Foodborne and Zoonotic Pathogens
- July 2006, Berlin, Germany
- Convened with EU’s MED-VET-NET
- Attribution was central part of program
- Funding from FDA and USDA

- * Batz MB, Doyle MP, Morris JG Jr, Painter J, Singh R, Tauxe RV, Taylor MR, DLF Wong. 2005. “Attributing illness to food.” *Emerging Infectious Diseases*. 11(7): 993-999.

## Foodborne Illness Risk Ranking Model

- A first step in priority setting
- Ranks pathogen-food combinations
- 28 pathogens
- 13 food categories (46 sub-categories)
- 5 measures of annual public health impact: illnesses, hospitalizations, deaths, cost ($), and QALY loss
- Project team:
- UMB: Glenn Morris, Mike Taylor, Mike Batz, others
- RFF: Alan Krupnick, Sandy Hoffmann, others
- Iowa State: Helen Jensen
- Funded by RWJ (v1) and USDA CSREES (v2)

## Food Attribution in FIRRM

- Our definition of “food attribution” is broad: for each pathogen, determine _proportion (percentage)_ of foodborne cases in each food category

| Food 1
Food 2
Food 3
     :
     :
Food 10
Food 11 | 10%
5%
30%

15%
10% |
| --- | --- |
| TOTAL | 100% |

- 1000 foodbornecases of
- Pathogen A

- X

| Path-Food A1
Path-Food A2
Path-Food A3
     :
     :
Path-Food A10
Path-Food A11 | 100
5
300

150
100 |
| --- | --- |
| TOTAL | 1000 |

- =

## The Point of Attribution

- Different approaches (and data) address attribution at different points in the farm-to-fork continuum:
- Point of consumption
- Point of production/reservoir
- Point of contamination
- Point of attribution affects interpretation:
- Outbreak data is point of consumption attribution because food vehicles are those that were eaten, and may include cross-contamination during preparation or earlier
- Microbial fingerprinting/sub-typing approaches are point of production because they identify the reservoir species, but not the route (e.g. produce left out)

## Attribution for FIRRM

- Want “point of consumption” attribution
- Want to address many pathogens across all foods
- Two available data sources qualify
- Outbreak data from CDC and CSPI:
- Pros: Large national dataset, can interpret/aggregate using decision rules
- Cons: Misrepresents sporadic cases, geographic/temporal/selection biases
- FIRRM uses CDC line listings and CSPI dataset
- Expert judgment from FSRC elicitation
- Pros: Large number of experts, wide expertise
- Cons: Not “data driven” in traditional sense, potential for circularity, biases in survey approaches
- Attempted an exposure assessment approach, but data was too lacking for wide range of pathogens/foods

## Food Categories and Binning Outbreaks

- Sounds easier than it is to develop categories that are consistent, compatible, and tractable for risk ranking. For example:
- Is a tomato a fruit or a vegetable?
- Are turkey slices “poultry” or “luncheon meat”?
- Many foods as consumed are “complex” in that they include multiple ingredients - for some pathogens, as many as 50% of outbreaks may be complex
- How to deal with complex foods?
- Include complex foods category or exclude from analysis?
- Bin all multi-ingredient dishes into complex food category, or use less conservative approach to bin these outbreaks into the primary ingredient in the dish (e.g. omelette as egg)?

## Complex Foods Example: Salmonella

- Complex Foods Example: *Salmonella*

- Multi-Source outbreaks excluded (single food vehicle only), 1990-2004

- Conservatively bin all complex dishes into “complex foods” category

- Or bin some complex dishes into categories with their primary (and most likely) ingredient

- “Complex foods” redistributed into likely ingredient vehicle

**Binning option:**

**Include/exclude?**

## Complex Foods Example: Salmonella

- Multi-Source outbreaks excluded (single food vehicle only), 1990-2004

**Binning option:**

**Include/exclude?**

## Uncertainty Due to Binning: Salmonella

- Multi-Source outbreaks excluded, “complex foods” dropped from percentages.

## Comparing Outbreaks and Experts

- For some pathogens, percentages are quite similar
- For others, percentages significantly different
- Outbreak data might have informed expert opinion
- Expert opinions might also reflect other data, such as case-control studies

## Campylobacter: Experts & Outbreak

- μ = mean of expert attribution
- o = mean of outbreak attribution

- Major differences between outbreak data and expert judgments for Campylobacter

**Note: preliminary data shown for illustrative purposes only**

## Attribution Affects Rankings: Example

**Note: Preliminary results, shown for illustrative purposes only. Toxoplasma cannot be attributed via outbreaks (1 outbreak in dataset), but can be attributed via experts, thus the large number of unattributable hospitalizations by outbreaks are broken up by expert attribution.**

## Conclusions: Challenges

- Significant problems with outbreak attribution
- Can manage some with uncertainty & sensitivity analysis
- Can’t do much about non-representitiveness or sparseness
- Expert elicitation is informative
- Even if you don’t trust the percentages themselves, if done properly can give you the state of expert perception
- We’ll never have perfect attribution:
- Surveillance pyramid problem is multiplied – Getting incidence and pathogens is difficult enough, food is harder
- Dynamic system that changes over time
- How to interpret trends, account for interventions, or deal with changes in food consumption?
- How to measure or deal with changes in durable immunity of population and antimicrobial resistance of pathogens

## Conclusions: Future Needs

- Even though we won’t have perfect attribution, there are a few things we can work on
- Common terminology
- What do we mean by “attribution”?
- Consensus on food categories
- Find ways to combine, connect, and compare attribution data and results:
- Connect “top-down” and “bottom-up” results
- Connect human surveillance with microbial testing of animals, plants, and foods
- Different approaches for different pathogens
- More data, more research! Surprise!
- More sampling of products/animals/farms?
- Large epidemiological studies?

## Decisions Under Uncertainty

- We can’t wait forever...
- We need to figure out what is “good enough” for the purpose at hand and make decisions in the face of uncertainty
- That said, we should take care to analyze and present uncertainties, limitations, and biases in our results
- In the ideal world, we would estimate accurately and precisely, but in reality, we must find ways to communicate risks in quantitative if qualified ways

## Thanks

- For more information on the Foodborne Illness Risk Ranking Model, including a downloadable version, visit the FSRC website:	_[http://www.rff.org/fsrc/](http://www.rff.org/fsrc/)_
- 	Michael Batz	mbatz@epi.umaryland.edu	410.706.3756