FOR THE FUTURE 

RESOURCES 

Using Expert Elicitation to 

Foodborne Illness to Food Co

Sandra Hoffmann, Paul Fischbec

Alan Krupnick 

FSIS Public Meeting: Attributing 

to Food 

April 5, 2007 
RESOURCES 

F O R THE FUTURE 

Take Home Messag

• Know why you’re doing an attrib• Expert elicitation can improve th

information basis for risk manag

• Our expert elicitation study con

information relevant to risk-basesafety management. 
RESOURCES 

F O R THE FUTURE 

Know why you’re doing an a

1. You can attribute foodborne illness to m

factors 

3⁄4 pathogens (Mead et al., 1999) 3⁄4 food consumption (Hoffmann et al., 2007) 3⁄4 food contamination 3⁄4 food production, processing and handling 

Decision needs must drive attribution. Mixing purposes can result in inconsistcategories. 

2. Having attributions that attribute all foo

to broad categories are a useful starting 
RESOURCES 

F O R THE FUTURE 

Expert Elicitation Is Widely 

Inform Risk-based Decision 

• Systematic methods for eliciting 

integrating expert judgment 

• Widely used in government and • Methods vary 

 See Clemen (1996), Cooke (1991

and Henrion (1990) 
RESOURCES 

F O R THE FUTURE 

Expert Elicitation Can Shed 

Food Attribution Data G

• Outbreak data is improving, but • Studies indicate outbreak cases a

cases may have different associafood 

• FoodNet is not yet nationally rep• Experts have relevant knowledge 

experience 
RESOURCES 

F O R THE FUTURE 

What we did. 

• Surveyed 44 nationally recognized foexperts • Asked experts for their best estimates and low bound •11 food categories: modified CSPI foo•11 pathogens: FoodNet pathogens, Togondii, and Noroviruses •Used mean expert food attribution percattribute Mead pathogen estimates to fo
RESOURCES 

F O R THE FUTURE 

Multiple Uncertainty Me

Characterize Knowledge ab

Attribution 

• Agreement among experts’ best • Agreement between experts’ and 

data-based attribution estimates 

• Experts’ mean individual uncerta• Variability in experts’ individual 

uncertainty 
,,,_.,,- --....

/ 

l 

,,..__ __.

~

--

• .~ 

,) 

• 

-

• • .. -

• 

RESOU RCES 

FOR THE FUT U RE 

1.00 

Uncertainty by Foo

Average Expert-Expert Correlation0.80 

0.60 0.40 

Poultry

Beef

Dairy

Pork Luncheon

Game

Seafood

Eggs

Breads Produce

Beverages

Average Expert-Expert Correlation 

1.00.90.80.70.60.50.40.30.2

Seafood

Poultry

Eggs

Breads 

Beef

Beverages

Lun

Dairy

cheon

Game

Por

0.20 

0.10.0

0 5 

0.00 

0.00 0.20 0.40 0.60 0.80 1.00 Average C

Average Expert-Outbreak Correlation 
RESOURCES 

F O R THE FUTURE 

Regressions Analys

• Uncertainty measures are smaller wit

outbreaks and more years of experien

• Expert characteristics do not affect be

but do affect individual confidence 

• Pathogens are more important than fo

explaining outbreak/expert difference 

• Foods are more important than patho

explaining variability in individual co
RESOURCES 

F O R THE FUTURE 

Major Stories about Attri

• 15 food/pathogen pairs cause 90

foodborne illnesses in the U.S. 

• 22 food/pathogen pairs cause 90

from foodborne illness in the U.S

• Outbreak-based and expert attrib

estimates agree on many, but not exposure routes for foodborne illdeath 
RESOURCES 

F O R THE FUTURE 

Differences in Top Causes o

•Noroviruses/produce (1 for both) •Noroviruses/seafood (2 for both) •Campylobacter/produce (3 EE), (not in top 1

•Campylobacter/poultry (3 OB), (not in top 15 

•Noroviruses-RTE Meat (15 EE), (4 OB) 
RESOURCES 

F O R THE FUTURE 

Take Home Messages -- 

• Know why you’re doing an attributio• Expert elicitation can improve the inf

basis for risk-based food safety reguldecisions. 

• Our expert elicitation study: 

 provides food attribution estimates;  depicts the state of knowledge about th

between food consumption and foodbo

 suggests places where outbreak data m

supplemented.