FSIS Risk Assessment for 

Listeria monocytogenes 

in Deli Meats 

Prepared by: 

Daniel L. Gallagher 

Dept. of Civil and Environmental Engineering 

Virginia Polytechnic Institute and State University 

Eric D. Ebel and Janell R. Kause 

Risk Assessment Division 

Food Safety and Inspection Service, USDA 

May 2003 
This report provides information on the risk assessment model developed, including the sources 

of data used, underlying assumptions, model equations, and techniques applied, to provide 

estimates of the number of deaths from L. monocytogenes in deli meats in response to specific 

risk management questions. This report is organized into the following sections: 

1. Public Health Regulatory Context 

2. Risk Management Questions 

3. FSIS Listeria Risk Assessment 

a. Model Overview 

b. Model Parameters 

c. Conceptual Model 

d. FDA/FSIS Risk Ranking Model 

e. In-Plant Dynamic Model 

f. Model Implementation and User Interface 

g. Calibration of the In-Plant Dynamic Model 

4. Listeria Risk Assessment Outputs 

5. Sensitivity Analysis 

FSIS Listeria Risk Assessment May 26, 2003 

FSIS LISTERIA RISK ASSESSMENT REPORT 

SCOPE AND MANDATE 

This risk assessment was initiated in February 2002 in response to public comments on the Food 

Safety and Inspection Service (FSIS) proposed rule: Performance Standards for the Production 

of Processed Meat and Poultry Products [66 FR 12589, February 27, 2001]. Several comments 

indicated a need for a stronger scientific basis for the proposal to require testing and sanitation of 

food contact surfaces for Listeria species.1 In general, the scientific literature indicated that the 

relationship between the prevalence and level of Listeria species in the plant environment (e.g., 

food contact and non-food contact surfaces) to the prevalence and level of Listeria 

monocytogenes (L. monocytogenes) in ready-to-eat (RTE) meat and poultry products is not well 

understood. To better understand this relationship, FSIS requested public input as part of the 

proposed rule for RTE meat and poultry products (66 FR 12609). In addition to the public 

request for data, FSIS initiated the planning and development of this risk assessment to: 1) 

provide insight into the relationship between Listeria species on food contact surface(s) and L. 

monocytogenes in RTE meat and poultry products; and 2) to evaluate the effectiveness of food 

contact surface testing and sanitation regimes, pre- and post-packaging interventions, use of 

growth inhibitors, and combinations of these interventions to mitigate contamination on RTE 

meat and poultry products and reduce the subsequent risk of illness or death from L. 

monocytogenes. 

1 The purpose of risk assessment as a public health tool is to use available data and information in a model to predict 

outcomes (i.e., effectiveness of an intervention in reducing illnesses) to inform decision-making. Without risk 

assessment, the public health benefit of selecting one policy intervention over another would be unknown. On 

the other hand, waiting to have all the data would prevent public health measures from being implemented in a 

timely manner. The risk assessment methodology is a tool designed to inform decision-makers when all of the 

data or information are not known. Risk assessment allows there to be scientifically-based informed decisionmaking. 

3
FSIS Listeria Risk Assessment May 26, 2003

6. References 

7. Appendix A: Revisions to the 2001 FDA/FSIS Risk Ranking Model 

8. Appendix B: Predicted Growth Between Processing and Retail 

9. Appendix C: Evaluation of FSIS RTE Survey Data for Volume of Production

for Establishments Producing Deli Meats 

10. Appendix D: Risk Assessment Model Outputs Stratified by High, Medium and 

Low Production Volume Establishments & Consecutive Positive FCS Samples. 

PUBLIC HEALTH REGULATORY CONTEXT 

This section provides background information on the health risks posed by L. monocytogenes

and the regulatory context for this pathogen in FSIS-regulated RTE meat and poultry products. 

Public Health Background 

L. monocytogenes is a pathogen that occurs widely in both agricultural (e.g., soil, water, and 

plants) and food processing environments (e.g., air, drains, floors, machinery) (Ryser 1999). 

L. monocytogenes grows at low oxygen 

conditions and refrigeration temperatures, and Definition: Ready-to-Eat (RTE) 

therefore survives for long periods of time in RTE meat and poultry products are the environment, on foods, in processing plants, products that are in a form that is edible and in household refrigerators. Although without additional preparation to achieve frequently present in raw foods (dairy, meat, food safety and may receive additional poultry, fruits, and vegetables), it can also be preparation for palatability or aesthetic, present in RTE foods due to post-processing gastronomic, or culinary purposes (9 contamination (Mead 1999a, CDC 2000).2 In CFR Part 430). 2001, the Food and Drug Administration and 

the Food Safety and Inspection Service 

completed a draft risk ranking of RTE foods for L. monocytogenes (FDA/FSIS, 2001). Of 

the 20 RTE food categories evaluated, deli meats posed the highest per annum risk of illness 

and death from L. monocytogenes, while hot dogs (i.e., frankfurters, wieners, etc.) posed a 

moderate public health risk. Since the release of the FDA/FSIS risk ranking of RTE foods, 

public comments and additional data have been made available to update the exposure 

assessment for deli meats3 and the L. monocytogenes dose-response relationship (see 

Appendix A). 

In general, consumption of food contaminated with L. monocytogenes may cause listeriosis, 

which can result in serious human illness (Ryser 1999). In 1999, the Centers for Disease 

 

2 In 1991, after a series of outbreaks of human illness associated with the consumption of a variety of foods (e.g., 

meats, coleslaw, pasteurized milk, soft cheese), the National Advisory Committee for Microbiological Criteria in 

Foods (NACMCF) recommended control strategies to minimize the presence, survival, and multiplication of L. 

monocytogenes in foods (NACMCF 1991). These control strategies included the development of an effective 

national surveillance system for listeriosis and inclusion of this pathogen in industry HACCP systems to ensure 

the safety of foods from production to consumption. 

3 The exposure assessment for hot dogs was also updated based on public comments and additional data since the 

release of the FDA/FSIS risk ranking of RTE foods. 

4
FSIS Listeria Risk Assessment May 26, 2003 

Control and Prevention (CDC) reported that of all the foodborne pathogens under 

surveillance in the United States, L. monocytogenes had the second highest fatality rate 

(20%) and the highest hospitalization rate (90%). Those at greatest risk of illness were the 

elderly (i.e., those 60 years and older), those with suppressed or compromised immune 

systems (e.g., those who have received a bone marrow transplant, cancer treatment, etc.), and 

fetuses or newborns (Slutsker and Schuchat 1999).4 Each year, L. monocytogenes causes an 

estimated 2,500 cases of foodborne listeriosis, including approximately 500 fatalities (Mead 

1999a, b). 

Policy Context 

Prior to initiating this risk assessment, FSIS has taken a number of regulatory steps to protect 

the public’s health, including the following: 

Microbiological Testing for L. monocytogenes in RTE Meat and Poultry Products. Since 

1987, FSIS has randomly sampled and tested RTE meat and poultry products5 produced in 

federally inspected establishments for L. monocytogenes. During the 1980s, when L. 

monocytogenes emerged as a public health problem associated with deli meats and other 

processed foods, FSIS established a “zero tolerance” (e.g., no detectable level of viable 

pathogens permitted) for L. monocytogenes in RTE meat and poultry products. Such 

products testing positive for L. monocytogenes are considered “adulterated” under the 

Federal Meat Inspection Act (FMIA) or the Poultry Products Inspection Act (PPIA) (21 USC 

453(g) or 601(m)).6 The combinatio

poultry products an adulterant and conti

L. monocytogenes may have contributed

7 

n of declaring L. monocytogenes in RTE meat and 

nued microbiological sampling of these products for 

 to t

onocytogenes.

he 44 percent decline from 1989 to 1993 in the 

rate of illness from L. m

PR/HACCP. On July 25, 1996, FSIS published its final rule on Pathogen Reduction and 

HACCP (PR/HACCP) Systems (61 FR 38806), which established new requirements for 

establishments producing meat and poultry products to improve food safety. Under HACCP, 

establishments must analyze their production systems, identify where hazards such as 

microbial contamination (e.g., L. monocytogenes) can occur, and establish controls to prevent 

or reduce those hazards. For hazards that are considered an adulterant in certain products, a 

”zero tolerance” is followed, and if the pathogen is detected in product, a recall of product 

may ensue if the product is in the market place. FSIS also requires establishments to adopt 

and follow written Sanitation Standard Operating Procedures (Sanitation SOPs) to reduce the 

likelihood that harmful bacteria will contaminate finished products (e.g., RTE meat and 

poultry products) that are exposed to the environment post-lethality treatment, particularly 

those products that support the growth of this pathogen. 

4 Perinatal listeriosis results from in utero exposure of the pregnant mother, causing fetal infection that leads to fetal 

death, premature birth, or neonatal illness, or death (Lennon 1984, Souef 1981). 

5 These products include cooked and fermented sausages, cooked corned beef, sliced ham and luncheon meats, beef 

jerky, cooked uncured poultry, and meat salads and spreads. 

6 Adulterated products are usually recalled voluntarily by the manufacturer. 

7 FSIS believes that while testing approximately 7,000 RTE meat and poultry products for L. monocytogenes each 

year helped to reduce the incidence of listeriosis, improved sampling methods (e.g., sampling design) are needed 

to effectively prevent illness from RTE meat and poultry products. See current RTE sampling directive: 

http://www.fsis.usda.gov/OPPDE/rdad/FSISDirectives/10240.3.htm. 

5
FSIS Listeria Risk Assessment May 26, 2003 

FSIS Notice/L. monocytogenes in HACCP Plans. In February 1999, during a large outbreak 

of listeriosis associated with hot dogs and deli meats, FSIS issued a notice advising 

manufacturers of RTE meat and poultry products of the need to reassess their HACCP plans 

to ensure that the plans were adequately addressing L. monocytogenes (64 FR 27351). FSIS 

believes that L. monocytogenes contamination is reasonably likely to occur in the production 

of most RTE meat and poultry products. 

Food Contact Surface Testing for Listeria Species. FSIS acknowledges that there may be 

certain processing operations in which L. monocytogenes is not a hazard reasonably likely to 

occur because of control procedures addressed in the Sanitation SOPs and other programs. 

In these cases, the hazard is, therefore, not addressed in an establishment’s HACCP system. 

In such establishments, verification through microbiological testing of food contact surfaces 

to ensure the establishment’s Sanitation SOP in controlling Listeria species may be 

appropriate.8 Were an establishment to find Listeria species on a food contact surface, that 

finding may be indicative of a sanitation problem that could cause adulteration of the product 

(e.g., cross-contamination).9,10 Establishments may need to take certain actions after food 

contact surfaces test positive for Listeria species (e.g., those defined in its Sanitation SOP 

according to §416.15).11 

Proposed RTE Rule. On February 27, 2001 FSIS issued a proposed rule (66 FR 12590) to 

require that all establishments that produce RTE meat and poultry products conduct 

environmental testing of food contact surfaces for Listeria species after lethality treatment 

and before final product packaging. Establishments were given the option to avoid testing if 

they established a critical control point (CCP) addressing possible L. monocytogenese 

contamination after lethality treatment. The focus on the non-pathogenic indicator was made 

because these organisms would be found more frequently in the environment than L. 

monocytogenes and because test results would be available more quickly. Finding Listeria 

species would be indicative of a sanitation problem even though the contaminant may not be 

L. monocytogenes. The establishment and FSIS would use the test results to verify the 

efficacy of the establishment’s “Sanitation SOPs” in preventing RTE product contamination 

by L. monocytogenes. FSIS also suggested an increased frequency of Listeria species testing 

on food contact surfaces for larger establishments. Since neither the suggested frequency of 

testing nor the relationship between testing for Listeria species on food contact surfaces and 

L. monocytogenes on the product was based on either scientific data or a risk assessment, the 

agency requested comment from the public regarding this ruling and initiated this risk 

assessment. 

8 On January 13, 2000, the Center for Science in the Public Interest (CSPI) requested that FSIS require all RTE meat 

and poultry processing establishments, including those that address L. monocytogenes as part of their HACCP 

system, to conduct environmental testing for Listeria spp. and product testing for L. monocytogenes. 

9 Notably, Tompkin et al. (1986) recommended plant-wide environmental testing for a non-pathogenic “indicator” 

(e.g., Listeria spp.) instead of testing for L. monocytogenes. An indicator organism is one that occurs frequently 

in the environment or food and the presence of which is correlated to the pathogen of concern. 

10 Recurring test positives for Listeria spp. may indicate that the establishment has a serious sanitation problem, 

even if L. monocytogenes is never found. FSIS enforcement action will vary depending on the establishment’s 

efforts to correct its sanitation and processing problems and its disposition of affected product. 

11 Sanitation SOP corrective actions may include “procedures to ensure appropriate disposition of product(s) that 

may be contaminated, restore sanitary conditions, and prevent the recurrence of direct contamination or 

adulteration of product(s).” (66 FR 12604). 

6
FSIS Listeria Risk Assessment May 26, 2003 

Technical Public Meetings. On May 15, 2000, FSIS held a public meeting to discuss: current 

Agency initiatives to prevent human illness from L. monocytogenes in RTE meat and poultry 

products; the use of Listeria species as an indicator organism for L. monocytogenes; and the 

efficacy of environmental testing for Listeria species.12 On May 8, 2001, FSIS held a public 

meeting to discuss scientific research and new technologies relevant to the L. monocytogenes 

in RTE meat and poultry products. At this meeting, FSIS requested data relevant to the 

proposed regulation regarding frequencies of testing for environmental Listeria species and 

the correlation with volume of production.13 

Listeria Summit. On November 18, 2002, FSIS held a public meeting to provide a forum for 

experts from government, academia, industry, and elsewhere to discuss current research and 

information related to improving the safety of RTE products. The topics discussed included 

the role of environmental and product testing, decontamination strategies, and consumer 

behaviors related to RTE foods. 

Risk Assessment Public Meeting. On February 26, 2003, FSIS held a public meeting to 

discuss the FSIS Listeria risk assessment model, underlying data and assumptions, and to 

garner data and information through public input. 

RISK MANAGEMENT QUESTIONS 

In the Fall of 2002, FSIS risk managers requested that a risk assessment be designed in order to 

evaluate the following specific questions: 

1) How effective are various food contact surface14 testing and sanitation (corrective 

action) regimes (e.g., vary the frequency of testing by plant size – large, small, and 

very small plants) on mitigating L. monocytogenes contamination in finished RTE 

product, and reducing the subsequent risk of illness or death?; 

12 The National Food Processors Association (NFPA) agreed that establishments should implement an 

environmental monitoring program for an indicator organism such as Listeria species. However, NFPA insists 

that such programs must be highly flexible in order that appropriate actions can be taken by industry. NFPA felt 

that mandating environmental testing was likely to be counterproductive, as it may discourage establishment 

efforts to find the Listeria species due to concerns of overly severe enforcement and compliance requirements by 

FSIS. Furthermore, NFPA noted that since there is no available scientific data correlating the frequency of 

environmental testing for Listeria species (and subsequent corrective actions) to reduced prevalence of L. 

monocytogenes in RTE meat and poultry products, establishments should be allowed flexibility in testing and 

frequency of testing. NFPA supported revision of the FSIS directive for plants operating under a HACCP 

system to incorporate options for industry testing for environmental Listeria species that would be verified by 

FSIS such that these establishments would be subject to a reduced frequency of product testing for L. 

monocytogenes by FSIS. 

13 In response to this request for input, the National Meat Association (NMA) submitted comments on September 

10, 2001, indicating that, because of the absence of evidence, they cannot support a regulation that would require 

plants to test product contact surfaces for Listeria species at prescribed frequencies based on plant size. 

14 In-plant food contact surfaces include conveyor belts, tables, counter tops, machinery (peeler, slicer, packing 

equipment) that contact product (9 CFR 301, 303). In-plant non-food contact surfaces tested during in-depth 

verification of establishments associated with L. monocytogenes outbreaks or where RTE product was found 

positive for L. monocytogenes during routine monitoring include: (1) air samples; (2) floor surfaces immediately 

below production lines; (3) machine parts; and (4) walls. 

7
FSIS Listeria Risk Assessment May 26, 2003 

2) How effective are other interventions (e.g., pre- and post-packaging interventions or 

the use of growth inhibitors) in mitigating L. monocytogenes contamination in finished 

RTE product, and reducing the subsequent risk of illness or death?; and 

3) What guidance can be provided on testing and sanitization 

s risk assessment, three diffe

of food contact surfaces for 

Listeria species (e.g., the confidence of detecting a positive lot of RTE product given a 

positive food contact surface test result)?15 

Note: For purposes of thi rent types of testing were 

considered. The first was environmental testing. This would include air ducts, walls, 

floor drains, etc. The second was food contact surface testing. This includes tables, 

rollers, or any other surface that the RTE product comes in contact with after cooking 

or other lethality treatment. The third type of testing was direct testing of the RTE 

product itself. Based on these specific risk management questions and the available 

data, this risk assessment focused on testing of food contact surfaces and considered a 

few scenarios for testing RTE product. 

IS LISTERIA RISK ASSESSMENT 

address these risk management questions, a dynamic in-plant Monte Carlo model (referred to 

the in-plant model) quantitatively characterizing the relationship between Listeria species in

 in-plant environment and L. monocytogenes in deli meats at retail was developed using

rently available data. The outputs of the in-plant model (e.g., concentration of L. 

nocytogenes on deli meat at retail) were used as inputs into the updated FDA/FSIS retail-tole exposure pathway for deli meats. This output was calibrated to the concentration of L. 

nocytogenes in RTE product at retail in the FDA/FSIS exposure assessment pathway, which 

luded recently available retail survey data (Gombas, 2003). The FDA/FSIS exposure

essment then tracks the level of L. monocytogenes in deli meat from retail to table, and

vides estimates of the subsequent risk of illness or death from consuming these RTE

ducts. These two connected models – the in-plant model and the updated retail-to-table

A/FSIS exposure assessment and FDA/FSIS dose-response relationship – comprise the

rall FSIS Listeria risk assessment model. 

 in-plant model is unique in that it is a dynamic model with spatial and temporal components, 

ich track the movement of Listeria contamination from food contact surface to RTE product 

ing processing. In general, there are few published studies that discuss microbial

tamination within a food processing plant. den Aantrekker et al. (2002) develop, but do not 

ually apply, a detailed model of bacterial recontamination within a food processing

FS

To 

as 

the

cur

mo

tab

mo

inc

ass

pro

pro

FD

ove

The

wh

dur

con

act

environment for different exposure pathways. As a result, the FSIS Listeria risk assessment 

15 The efficacy of microbiological testing is unclear in the literature. Brown et al. (2000) argue that, under HAACP, 

enumeration of indicator organisms is more appropriate that pathogen detection, and that batch testing for 

pathogens is not an effective method for evaluating food safety. Swanson and Anderson (2000) argue that 

microbial testing is needed to validate critical control points, but that once this is accomplished microbiological 

testing is ineffective. Nestle (2003) argues that additional testing would produce safer food. Sugarman (2003), 

in an interview with Jack in the Box VP for Quality and Logistics David Theno, states that Jack in the Box is 

currently testing ground beef production every 15 minutes at 3 processing plants ten years after the E. coli 

O157:H7 outbreak. Theno states that testing can be used to control contamination levels. Given this uncertainty 

concerning testing effectiveness, the goal of this risk assessment was to quantify the relationship between testing 

and public health. 

8
FSIS Listeria Risk Assessment May 26, 2003 

model provides a useful tool to evaluate the effectiveness of various interventions to control 

contamination of RTE product from in-plant sources of Listeria and reduce the subsequent risk 

of illness or death from L. monocytogenes on RTE product. By modeling changes in plant 

practices such as: the frequency of testing and sanitation of food contact surfaces, the 

effectiveness of pre- and post-packaging interventions16, the effectiveness of growth inhibitors, 

effectiveness of enhanced sanitation, as well as combinations of these practices, this risk 

assessment can provide numerous outputs to address specific risk management questions. The 

in-plant risk assessment model was also developed with user-friendly interfaces to allow users to 

change scenario conditions and assumptions. As a result, this risk assessment model can be used 

as a tool to explore a variety of risk management scenarios beyond those developed for this 

report. 

Note: An implicit assumption in this risk assessment is that all L. monocytogenes on RTE 

product comes from food contact surfaces and not from an inadequate lethality treatment. This 

assumption is necessary to evaluate the specific risk management question provided by FSIS risk 

managers. Also, in developing the FSIS Listeria risk assessment model, FSIS has generally left 

unchanged the components of the current FDA/FSIS exposure assessment for deli meats and the 

FDA/FSIS dose-response relationship for use in this risk assessment.

17 

Model Overview 

The FSIS Listeria risk assessment model includes a dynamic in-plant Monte Carlo model 

that predicts L. monocytogenes concentrations at retail. Dynamic means that the bacterial 

concentrations are predicted in each lot of RTE product over time. Monte Carlo means that 

many of the parameters for the model are stochastic random variables, and that different 

values are selected for each lot produced. For example, the fraction of Listeria that transfer 

from the food contact surface to the lot varied from lot to lot, but fell within a limited range 

and matched the probability distribution of the available data. 

Monte Carlo sampling is used throughout the FSIS Listeria risk assessment, in both the inplant dynamic model and the FDA/FSIS retail-to-table exposure assessment for deli meats. 

The inputs for the in-plant dynamic model of the FSIS Listeria risk assessment are modeled 

as variability distributions without the inclusion of parameter uncertainty. Inclusion of 

parameter uncertainty would have required substantial computational time requirements. 

This was a reasonable simplifying assumption in the model given that it is a generally 

accepted practice to exclude uncertainty in a model input if variability is thought to dominate 

(e.g., Small, 2000). In cases, as seen in this risk assessment, where parameter uncertainty is 

swamped by model uncertainty, it is not useful or pragmatic to invest a substantial amount of 

time required to draw fine distinctions between uncertainty and variability that may not be 

credible or useful. Instead, use of simpler modeling strategies may be more meaningful and 

pragmatic (Casman, 1999). Therefore, FSIS finds it reasonable, pragmatic and sufficient to 

use a simple, broad distribution to characterize in-plant model parameters 

16 Pre- and post-packaging interventions are those implemented after the potential pathogen transfer from food 

contact surface to RTE product has occurred. 

17 The FDA/FSIS risk-ranking model has undergone extensive review and public input. As a result, FSIS did not 

change any of the components of that retail-to-table exposure assessment for deli meats or hot dogs, including 

the dose-response relationship updated based on public comment. 

9
FSIS Listeria Risk Assessment May 26, 2003 

In the FSIS Listeria risk assessment, model inputs are assumed to be independent of one 

another. Without empirical information, specifying dependencies of inputs would be purely 

hypothetical. It seems reasonable to assume that variable model inputs (e.g., frequency, 

duration, and level of contamination) are independently distributed. 

The primary output of the in-plant model is the concentration of L. monocytogenes in RTE 

meat and poultry products at retail. This output was then coupled with the FDA/FSIS retailto-table exposure assessment for deli meats and the current FDA/FSIS dose-response model 

to predict human health impacts. 

A mass balance approach was used as the basis of the in-plant model. The number and 

disposition of Listeria organisms are tracked for both food contact surface area and the 

product over time. For example, as Listeria organisms move from the food contact surface 

area to the product, the concentration on the food contact surface area decreases and the 

product lot concentration increases so that the same total number of Listeria organisms is 

present. The total number of organisms can change due to growth of new organisms, die-off 

from sanitation, or transfer from external sources such as harborage sites. 

The in-plant model incorporates food contact surface testing, product testing, sanitation, preand post-packaging interventions, and the effect of growth inhibitors (or product 

reformulation18). The output of the in-plant model is combined with the updated version of 

the 2001 FDA/FSIS exposure retail-to-table pathway for deli meats and Listeria doseresponse relationship to estimate the risk of illness or death on a per serving and per annum 

basis from L. monocytogenes in RTE product. Risk estimates are provided as a function of: 

testing (Listeria species) and sanitation frequency (based on plant size) of food contact 

surfaces (FCSs), testing (L. monocytogenes) and disposition of RTE product, pre- and postpackaging interventions, and growth inhibitors. The conditional likelihood of detecting L. 

monocytogenes in products, given that the FCS tests positive for Listeria species, was also 

evaluated. 

To date, the model has been run for deli meats. Deli meats were selected because the 2001 

FDA/FSIS risk ranking analysis determined that this food category posed the greatest risk of 

illness and death among consumers. The model may also be run for hot dogs/frankfurters in 

the future. 

Model Parameters 

The data available within the published literature dealing with Listeria in the processing plant 

environment is rather sparse. Data limitations, the limited time available for model 

development, and the intended use of the model, dictated the following: 

1) The model only considers food contact surface as source of Listeria species/L. 

monocytogenes in product. In practice, Listeria could also arise from inadequate lethality 

treatment or from direct deposition from non-food contact surfaces. 

18 Product reformulation is another process for achieving inhibition of growth and is treated the same as using other 

growth inhibitors in this model. 

10
FSIS Listeria Risk Assessment May 26, 2003 

2) Only a generic food contact surface is modeled. A lot, for purposes of this analysis, 

consists of product produced in a shift or 8-hour period. There is no spatial component 

within the plant (e.g., slicer, convey belt, etc.). 

The model assumes Listeria species are evenly distributed across food contact surfaces, 

and L. monocytogenes are evenly distributed within a lot of product. In other words, the 

variability across a food contact surface or within a lot is not accounted for in this model. 

The model operates on a RTE product lot basis. This is the smallest unit of RTE product 

for which model results are available. 

Interventions, such as sanitation and testing, would affect the distribution of Listeria at 

retail, but did not change the timing, duration, or concentrations transferred during a 

3) 

4) 

5) 

contamination event. 

Updated FDA/FSIS Risk Ranking Model 

The 2001 FDA/FSIS risk ranking model was developed to identify the relative risk of illness 

or death posed by RTE foods in 20 categories (FDA/FSIS, 2001). This assessment indicated 

that deli meat posed the greatest public health risk for listeriosis of all the RTE foods. 

Roughly 80% of all deaths and cases are caused by deli meats according to the FDA/FSIS 

risk ranking model. This model was originally released for public comment and review in 

January, 2001. Based on review and comments, the exposure assessment for deli meats (and 

hot dogs) and the dose-response relationship have been updated. 

The current FSIS Listeria risk assessment is designed to simulate RTE food production 

within the processing plant and predicts the L. monocytogenes concentrations at retail. It 

uses the updated FDA/FSIS exposure assessment for deli meats and the updated doseresponse relationship to model distributions of the concentration of L. monocytogenes on 

RTE product at retail through consumption and estimates the subsequent annual number of 

deaths and illnesses. 

The 2001 FDA/FSIS risk ranking model is comprised of two major components – an 

exposure assessment and a dose-response relationship. A separate retail to table exposure 

assessment pathway was constructed for each of the RTE food categories. Results from all 

the RTE food categories were then carried forward to the dose-response simulations, where a 

separate simulation was constructed for each of the three population groups: elderly, 

intermediate, and perinatal.19 The exposure assessment for deli meats incorporated new 

data, including retail survey data from the National Food Processors Association (NFPA) on 

the prevalence and level of L. monocytogenes on RTE products (Gombas, 2003). 

A two-dimensional Monte Carlo simulation was used to integrate the components for each of 

the twenty exposure assessment pathways for each of the RTE food categories, with 100,000 

19 For the purposes of this model: elderly were defined as being 60 years of age or older; the intermediate 

population were those older than 30 days and less than 60 years old; and the perintal included fetuses and 

newborns from 16 weeks after fertilization to 30 days after birth (i.e., the pregnancy-associated cases where the 

mother experiences a foodborne L. monocytogenes infection during pregnancy, exposing her fetus to the 

pathogen). 

11
FSIS Listeria Risk Assessment May 26, 2003 

variability iterations and 300 uncertainty iterations. The end result of each exposure 

simulation is the fraction of servings that occur at designated dose levels (broken out at halflog10 intervals) for each food category and population group. The conversion to dose bins 

was necessary in order to integrate the exposure simulation, which evaluated the exposure 

from individual servings, with the dose-response model, which predicted the number of cases 

at a population level. For more information on the 2001 FDA/FSIS risk ranking model see: 

http://www.foodsafety.gov/~dms/lmrisk.html. 

The simulation in the FDA/FSIS risk-ranking model was carried out in several steps. First, a 

two-dimensional Monte Carlo simulation was used to integrate the variability and uncertainty 

of the initial RTE contamination levels, predicted growth of L. monocytogenes per serving, 

and serving size, with 100,000 model variability iterations and 300 model uncertainty 

iterations. The variability dimension for the estimated doses was then condensed to halflog10 increments, which ranged from -5 to +10 logs for each of the 300 model uncertainty 

iterations. Second, a one-dimensional (uncertainty only) dose-response simulation was run 

by selecting, one of the 300 exposure distributions for each food category, then adjusting 

these distributions for strain-virulence and host susceptibility factors. The dose-adjusted 

exposure distributions (i.e., the concentration of L. monocytogenes in servings of RTE 

product) were then integrated with a dose-response function to predict the total number of 

deaths per annum for each food category. The total number of listeriosis deaths per annum 

was estimated by summing the deaths across all food categories. On each uncertainty 

iteration, the dose-response function was adjusted until the total number of listeriosis deaths 

was equivalent to CDC surveillance estimates. 

The dose-response simulations consisted of 4000 model uncertainty iterations. During the 

model simulation, a dose-response scaling factor was determined to equate the deaths 

predicted by the dose-response function and the exposure distribution for each of the food 

categories, with the public health estimates for current annual rates of listeriosis. Since the 

2001 FDA/FSIS risk ranking model is calibrated such that the overall predicted incidence of 

listeriosis is in line with the actual incidence of listeriosis based on CDC surveillance data, an 

implicit assumption is that the foods encompassed by the food categories account for all 

cases of foodborne listeriosis. 

In order to facilitate scenario comparisons, the same sequence of random numbers were used 

in different simulations to permit comparisons. 

In-plant Dynamic Model 

Conceptual Model 

A schematic overview of the conceptual model is provided in Figure 1 below. The model 

assumes that a Listeria reservoir exists in the plant and is capable of contaminating the food 

contact surface. This reservoir can be harborage sites such as floor drains or air conditioning 

ducts, or other surfaces/equipment in the plant. 

12
FSIS Listeria Risk Assessment May 26, 2003 

Figure 1. Conceptual Model for the “In-plant” Component of the FSIS Listeria Risk 

Assessment. t ne Ev

cy enequFr

Listeria Reservoir 

(niches, harborage sites, drains, ...) 

Contamination 

Event 

ion aminat ont

C

Number of Lspp Organisms 

Food Contact 

Surface 

RTE Product 

before pre and 

post packaging 

interventions 

co

t

e

r

f

a

fi

n

c

s

i

f

e

e

n

r 

t 

transport to 

retail: growth 

Distribution at 

retail for FDA/ 

FSIS model 

m L of

Concentration 

Concentration of 

Listeria Spp 

L spp Testing 

& Sanitation 

on Food 

Contact 

Surface 

Positive? Apply 

Corrective action 

Pre and post 

packaging 

interventions 

RTE Product 

after pre and 

post packaging 

interventions 

RTE product 

testing Positive? 

Dispose 

Time and Duration 

13 
FSIS Listeria Risk Assessment May 26, 2003 

The concept of long-term Listeria reservoirs (harborage sites) in plants assumed in the FSIS 

Listeria risk assessment is supported by recent studies published in the literature. Lunden et 

al. (2002) described sequential L. monocytogenes contamination at three plants as a dicing 

machine was moved from plant to plant. This study provides an example where food 

processing equipment can act as long-term harborage sites over a long period of time even 

while typical sanitation measures are being taken. 

The FSIS Listeria risk assessment model supposes that Listeria species move from this 

reservoir onto the food contact surface during what is termed a contamination event. The 

key parameters defining a contamination event are: 1) the time between initialization of 

events (i.e., How often is a food contact surface contaminated?); 2) the duration of the event 

(i.e., How long does it last?); and 3) the amount of Listeria species transferred from the inplant reservoir to the food contact surface. 

Once on the food contact surface, Listeria species can be transferred to the lot of RTE 

product being processed, be removed from the food contact surface through sanitation at the 

end of each lot processing, or stay on the surface. Published studies support the concept that 

RTE product is primarily contaminated by food contact surface. In a study of L. 

monocytogenes in French delicatessen plants, Salvat et al. (1995) found that contact of 

cooked product with contaminated surfaces was a major route of product contamination, as 

was cross contamination between raw and cooked product.20 

If the contamination event is continuing, the new Listeria species transferred from the 

reservoir will be added to the Listeria species already on the food contact surface. For each 

lot processed, the food contact surface can also be tested for Listeria species and various 

mitigation steps taken if the surface tests positive. 

A positive food contact surface test can trigger a required lot of RTE product to be tested for 

L. monocytogenes. It can also trigger a more intensive sanitation (i.e., enhanced sanitation) 

of the food contact surface at the end of lot processing. 

Some fraction of the Listeria species on the food contact surface is transferred to the lot. 

This fraction is the transfer coefficient, which can range from 0 to 1. A transfer coefficient 

of 0 indicates that none of the Listeria species are transferred. A transfer coefficient of 1 

indicates that all the Listeria species is transferred to the product lot being processed. 

Once the number of Listeria species present in the product lot is calculated, the concentration 

of Listeria species per gram is then calculated. This must be converted to a concentration of 

L. monocytogenes. A ratio of L. monocytogenes to Listeria species is used for each lot to 

estimate this concentration. 

20 Air was tested and not found to be a source of contamination. Inadequate cleaning was also indicated as a reason 

for contamination. 

14
FSIS Listeria Risk Assessment May 26, 2003 

At this point the product lot can undergo post-lethality treatment (i.e., pre- and postpackaging intervention(s)21), which will reduce the concentration of L. monocytogenes. 

After these interventions, the lot can then be tested for L. monocytogenes, either because of 

routine lot testing or because a food contact surface tested positive for Listeria species. If a 

test-and-hold procedure is in place, the lot tested for L. monocytogenes, based on a food 

contact surface positive for Listeria species, is the lot produced at the time the food contact 

surface sample was collected. If a test-and-hold procedure is not in place, the lot testing 

response is lagged by the time it takes to analyze a food contact surface sample for Listeria 

species and obtain results of this test, i.e., lot testing is applied to a lot lagging behind the 

tested food contact surface. The model assumes a lag time of about 3 days. The model also 

assumes that product lots of RTE product that test positive for L. monocytogenes are 

removed from the food supply. Operationally, this would be accomplished by re-processing 

the lot for human food, diversion of the lot into products not intended for human 

consumption, or disposal of the lot. 

After pre- and post-packaging interventions and possible additional RTE product testing, the 

lot proceeds to retail. Using the deli meat component of the updated FDA/FSIS risk ranking 

model, the growth of L. monocytogenes during the transport stage was estimated. A constant 

logarithmic growth factor is applied in the model (see Appendix B). Because three different 

plant sizes are modeled, the final step in the model is to select the lots that appear at retail 

from among the lots produced by each plant size. The resulting distribution of L. 

monocytogenes concentrations on RTE product at retail serves as an input for the updated 

FDA/FSIS risk ranking model to estimate the public health impacts (illnesses and deaths). 

The FSIS Listeria risk assessment team considered including additional detail, such as 

modeling various types of food contact surfaces, additional operational steps based on the 

type of ready-to-eat product, additional interventions, and pathways of contamination of food 

contact surface or product from the plant environment. However, the current model was 

designed specifically to answer the risk management questions posed by FSIS risk managers. 

The current level of detail in the FSIS Listeria risk assessment is adequate to inform 

decision-making based on these risk management questions. To incorporate additional 

operational steps and variability in the FSIS Listeria risk assessment model would require the 

availability of additional data adequate to provide this level of detail. Such data are not 

available in the published literature and have not been made available to the Agency. 

Sources of Data and Assumptions 

Based on the conceptual model for the FSIS Listeria risk assessment (Figure 1), a summary 

of the data and assumptions used in this model are provided below (Table 1). 

21 Either immediately before packaging or after being sealed in the final package, the lot can undergo additional 

post-lethality treatment, which is intended to further reduce the level of potential pathogens, such as L. 

monocytogenes, in RTE products. 

15
FSIS Listeria Risk Assessment May 26, 2003 

Table 1. Available data and assumptions for the “plant to table” FSIS Listeria risk assessment. 

Model Step Data Required Available Data Assumptions 

Occurrence of a 

“contamination 

event”22 

Distribution (mean and 

shape) for time between 

contamination events 

FSIS in-depth verification 

investigation – number of 

food contact surface 

samples that test positive 

for Listeria spp. over a 

specified time period 

Distribution does not 

change by size of plant. 

Interventions do not 

change time between 

contamination events. 

Duration of a 

contamination event 

Tompkin (2002) provides 

table of number of plants 

with successive weekly 

positive Listeria food 

contact surfaces. 

Duration does not change 

by size of plant. 

Intervention does not 

change duration. 

Number of Listeria spp. 

transferred to food 

contact surface during 

each lot production. 

None. Levels calibrated to 

match FDA/FSIS risk 

exposure assessment 

concentration distribution 

for L. monocytogenes on 

deli meat at retail (includes 

recent NFPA data in FSIS 

Docket 03-005N). 

Distribution assumed log 

normal. Intervention does 

not change number 

transferred. 

Food contact surface area None. Assumed to vary by plant 

size in proportion to mean 

lot weight. 

Fraction of deli meats 

produced by plant size. 

FSIS RTE survey results 

(FSIS 2003). 

Lot assumed to be 1 shift 

production per line. 

Model assumes 2 shifts 

per day and 30 days per 

month. Minimum lot 

weight for any plant size 

assumed to be 1000 lbs. 

Testing of food 

contact surface 

Area swabbed 

Probability of detecting 1 

organism 

Area swabbed provided by 

industry (Dr. Brie Wilson, 

National Turkey 

Federation, personal 

communication, November 

2002). Information also 

provided by Dr. Sharar, 

FSIS/OPPDE, November 

2002. 

Transfer of Listeria 

species from food 

contact surface to 

RTE product 

Transfer coefficients for 

the transfer of pathogens 

from food contact 

surfaces to RTE products 

Scientific literature: 

Montville et al. (2001); 

Chen et al. (2001); and 

Midelet and Carpentier 

22 A "contamination event" is defined as Listeria spp. contaminating a food contact surface from workers hands, 

through environmental disruption, etc. 

16
FSIS Listeria Risk Assessment May 26, 2003 

(2002) 

Sanitation of food 

contact surface 

Sanitation timings and 

effectiveness 

The frequency of sanitation 

and sanitation effectiveness 

can be input into the model 

Convert food 

contact surface 

concentrations for 

Listeria spp. into L. 

monocytogenes 

surface 

concentrations on 

RTE product. 

Proportion of Listeria 

spp. (levels) that are L. 

monocytogenes (levels) 

Scientific literature: 

Tompkin, 2002 and 1992 

Assume that the 

prevalence distribution 

provided by Tompkin are 

similar to those for 

concentration 

Lot weight (production 

volume per line per shift) 

by plant size 

FSIS RTE survey results 

(FSIS 2003) 

Post Processing Fraction of industry 

implementing controls 

and their effectiveness 

(Input provided by 

FSIS/OPPED, December 

2002) 

Varied by scenario 

analyzed 

Product testing for 

L. monocytogenes 

Sample mass 

Frequency of testing 

Mass from USDA 

guidelines. Frequency of 

testing varied by scenario. 

Transportation of 

RTE product to 

retail 

Growth multiplier FDA/FSIS exposure 

assessment for deli meats 

Growth multiplier fixed at 

1 log unit for all lots. 

Fraction of industry 

employing growth 

inhibitors or product 

reformulation and its 

effectiveness 

Varied by scenario 

analyzed 

L. monocytogenes 

in RTE product 

from retail to 

consumer 

None. Model output. Use the updated FDA/FSIS 

exposure assessment for 

deli meats for L. 

monocytogenes in RTE 

products as calibration 

values for Listeria added 

during contamination event. 

Public health 

impacts 

No additional data. Uses the updated 

FDA/FSIS dose-response 

model 

Note: Keep in mind that the quality of the assessment of data is distinct from the sufficiency of 

the available data. While there was limited data for this risk assessment, key uncertainties (e.g., 

the dose-response relationship developed in the FDA/FSIS risk ranking) are likely to remain for 

quite some time until additional data becomes available. FSIS used the “best available” data to 

conduct this risk assessment. The option not to use risk assessment in decision-making was 

clearly not acceptable to the public based on comments received by the Agency for the Feb. 27, 

2001 proposed rule. Moreover, the decision not to make any decision in light of the number of 

illnesses associated with L. monocytogenes, particularly from deli meats, which compromise 

about 80% of cases based on the FDA/FSIS risk ranking of illnesses/deaths associated with RTE 

products, is not acceptable in light of Healthy People 2010 goals. Risk assessment organizes 

data into a systematic framework to evaluate the marginal public health benefits (e.g., lives saved 

17
FSIS Listeria Risk Assessment May 26, 2003 

or deaths prevented) associated with a potential intervention relative to the decision to maintain 

the status quo. Such information was deemed useful for risk management decision-making. 

Model Calculations and Base Values 

This type of risk assessment model is a dynamic model and has spatial and temporal interactions 

that make it somewhat difficult to present as pure mathematical equations. However, the major 

equations and base values are provided below. The justifications for the base values are 

provided later. 

The model starts by stochastically generating the start time and duration for each contamination 

event that will be needed for the simulation. These parameters are simply random variates drawn 

from distributions described below. The model also stochastically generates the timing for the 

requested testing of lots and FCS. These too are simply random variates. 

For each RTE lot produced during a contamination event, the concentration of Listeria species 

on the food contact surface is increased by a stochastic amount to account for the transfer of 

organisms from the harborage site to the food contact surface. The Listeria species 

concentration on the food contact surface at the end of the time period LSj is calculated as: 

LS j = (LS j−1 + δ ( j)) (1 − TC j ) (1 − sj )

where 

Table 2. Variables and Base Values for Listeria Concentration on Food Contact Surface. 

Variable Definition Type Base 

Value* 

LSj Listeria spp concentration on food contact surface at end of 

lot j (cfu/cm2

) 

stochastic, 

calculated 

NA 

TCj transfer coefficient for lot j (dimensionless) stochastic, 

input 

LN(-0.14, 

1), 

truncated 

to 

between 0 

and 1 

δ(j) added Listeria spp. concentration added to the food contact 

surface if a contamination event is ongoing (cfu/cm2

) 

( ) 





− = if during contamination event

if not during contamination event

~ ( 6,3.5)

0 

RN LN

δ j

stochastic, 

input 

LN(-6, 

3.5) 

sj Sanitation effectiveness calculated See 

below 

*LN indicates log10 normal distribution with mean and standard deviation given on the log10 

scale 

The sanitation effectiveness sj for each time period (or lot produced) is 

18
FSIS Listeria Risk Assessment May 26, 2003 

 swipe if 1st lot of day 

sj =  ssop if 2nd lot of day 



senhan if LSj-slag tested, positive, and enhanced sanitation option selected 

where 

Table 3. Variables and Base Values for Sanitation of Food Contact Surface. 

Variable Definition Type Base Value 

sj sanitation effectiveness for lot j 

(dimensionless) 

calculated NA 

swipe between-lot sanitation effectiveness 

(dimensionless) 

fixed, input 0.50 

ssop end of day sanitation effectiveness 

(dimensionless) 

fixed, input 0.75 

senhan enhanced sanitation effectiveness if a 

previous FCS was tested, found 

positive, and the enhanced sanitation 

option is selected (dimensionless) 

fixed, input 0.95 

slag 

lotsofnumber produced dayper

s lag = *daysinlagreportFCS

(lot units, i.e. time) 

fixed, input 6 

(3 days * 2 lots 

per day) 

The L. monocytogenes concentration in the RTE lot is then calculated as: 

LM = (LS + δ ( )j )*TC * j*

j j −1 j * R

M j

j 

A

where 

Table 4. Variables and Base Values for the Concentration of L. monocytogenes in a RTE 

Product Lot Produced in the Plant. 

Variable Definition Type Base Value* 

LMj 

* 

L. monocytogenes 

concentration in 

RTE product lot j 

(cfu/g) 

stochastic, 

calculated 

NA 

Aj food contact surface 

area at lot j , 

stochastic (* only 

varies for new 

contamination 

event) 

(cm) 

stochastic, input U(100000, 

1000000) 

Mj mass of lot j 

(lb, internally 

stochastic, input varies by plant size 

large: N(19371, 

19
FSIS Listeria Risk Assessment May 26, 2003 

converted to g) 14000) 

small: N(7100, 

10600) 

very small: N(2800, 

9500) 

Rj L. monocytogenes / 

Listeria spp ratio for 

lot j (dimensionless) 

stochastic, input N(0.52, 0.26), 

truncated to 

between 0 and 1 

* U() represents a uniform distribution with minimum and maximum given 

N() represents a normal distribution with mean and standard deviation given 

Post processing interventions are then applied which can reduce the concentration of L. 

monocytogenes in the RTE lot. 

 LM j if RN j ≥ FPPk LMPPj = 

LM j * (1 − PPk ) if RN j < FPPk 

where 

Table 5. Variables and Base Values for the Concentration of L. monocytogenes in a RTE 

Product Lot With Consideration of Post-Processing Interventions. 

Variable Definition Type Base Value 

LMPPj L. monocytogenes concentration in RTE 

lot j after post processing interventions 

(cfu/g) 

stochastic, 

calculated 

NA 

PPk Post processing intervention 

effectiveness for plant size k 

(dimensionless) 

Stochastic, 

input 

0 

U(PPmin, PPmax) 

when applied 

FPPk Fraction of lots for plant size k that 

undergo post processing interventions 

(dimensionless) 

Fixed, 

Input 

0 

RNj Uniform random number used to test if 

lot j should undergo post processing 

Stochastic, 

calculated 

U(0,1) 

Post processing interventions were not modeled for the base run. The different plant sizes were 

allowed to have different minimum and maximum values. Note: only a percentage of the lots 

produced by each different plant size were assumed to undergo post processing interventions. 

The decision on which lots undergo post processing was a simple binomial test based on the 

fraction of lots appropriate for the given plant size. 

During the transport from the processing plant to retail, bacterial growth could occur which 

increased the concentration of L. monocytogenes. The product or packaging could be formulated 

to reduce the growth. 

20
FSIS Listeria Risk Assessment May 26, 2003 

 LMPPj if RN j ≥ FGI k LMPPj = 

LMPPj 

*10GF +log10(1−GI ) if RN j < FGI k 

where 

Table 6. Variables and Base Values for Modeling Growth of L. monocytogenes in Product. 

Variable Definition Type Base Value 

LMGIj L. monocytogenes concentration in lot j 

after growth and growth inhibition 

during transport to retail (cfu/g) 

Stochastic, 

calculated 

NA 

GF Growth factor applied to all lots Fixed, input 1 

GI Growth inhibition factor Stochastic, 

input 

0 

UN(GImin, GImax) 

when applied 

FGIk Fraction of lots for plant size k that 

undergo growth inhibition 

(dimensionless) 

Fixed, 

Input 

0 

RNj Uniform random number used to test if 

lot j should undergo growth inhibition 

Stochastic, 

calculated 

U(0,1) 

Growth inhibition was not modeled for the base run. Note that, based on the values of GF and 

GI, it was possible for growth inhibition to actually reduce the concentration of L. 

monocytogenes in the lot. 

The model actually generates the requested number of lots for each plant size, and then selects a 

continuous run to combine for the retail distribution. The number of lots in the run is determined 

by the fraction of production for each plant size. 

 LMGI k

l arg e ∀ k = start, FPl arg e * NSim ∪

LMCombi = 

 LMGI k

small ∀ k = start, FPsmall * NSim ∪

LMGI k

verysmall ∀ k = start, FPverysmall * NSim 

where 

Table 7. Variables and Base Values for Modeling Retail Concentration of L. monocytogenes in 

a Product Lot. 

Variable Definition Type Base Value 

LMCombi L. monocytogenes concentration in lot 

i after combining lots from different 

plant sizes (cfu/g) 

Stochastic, 

calculated 

NA 

start Starting lot number for run Fixed, builtin 100 

FPk Fraction of pounds produced by each 

plant size k (dimensionless) 

Fixed, input Large = 0.48 

Small = 0.48 

Very small = 0.04 

NSim Number of lots to simulate for each Fixed, input 1000000 

21
FSIS Listeria Risk Assessment May 26, 2003 

plant size 

The first lot produced assumed that the FCS Listeria concentration is 0 cfu/gram. To prevent this 

initial value from biasing the final results, the first 100 lots simulated for each plant size are 

excluded from further consideration. In effect, this seeds the starting FCS concentration. 

The final retail distribution is based upon the combined distribution but filtered depending on 

whether the lot was tested and the corresponding result. Any lot that was not tested and any lot 

that was tested and returned a negative passes on to retail. Any lot that was tested and found 

positive is removed. 

LMRetaili = LMCombi |

i not tested ∪ LMCombi |i tested negative 

FCS and RTE Lot Testing 

The testing procedure for L. monocytogenes in a lot was calculated by first generating a Poisson 

random number using a population mean as mean cfu’s within the sample (sample mass * 

concentration): 

LM sample j = Poisson(SM j * LM j )

The RTE lot sample is judged positive by 

 positive if LMsample > 0 and (1 − pDLM)

LMsample < U(0,1)j LMRsample j =





negative otherwise 

where 

Table 8. Variables and Base Values for Testing for L. monocytogenes in Product. 

Variable Definition Type Base Value 

LMsample j total L. monocytogenes 

cfu in test sample j 

(cfu) 

stochastic, 

calculated 

NA 

pDLM probability of detecting 

1 L. monocytogenes 

cfu in test if present 

(dimensionless) 

fixed, input 0.75 

U(0,1)j uniform random 

number between 0 and 

1 (dimensionless) 

stochastic, 

calculated 

NA 

LMRsample j L. monocytogenes test 

result for lot j 

(positive or negative) 

stochastic, 

calculated 

NA 

The testing procedure for food contact surfaces was similar, with the relevant substitutions. 

LS sample j = Poisson(Aswab j * LS j )

22 
FSIS Listeria Risk Assessment May 26, 2003 

The FCS sample is judged positive by 

 positive if LSsample j > 0 and (1 − pDLS)

LMsample j < U(0,1)j LSRsample j = 

negative otherwise 

where 

Table 9. Variables and Base Values for Testing for Listeria on Food Contact Surface. 

Variable Definition Type Base Value 

LSsample j total Listeria species 

cfu in test sample j 

(cfu) 

stochastic, 

calculated 

NA 

pDLS probability of 

detecting 1 Listeria 

species cfu in test if 

present 

(dimensionless) 

fixed, input 0.75 

U(0,1)j uniform random 

number between 0 

and 1 

(dimensionless) 

stochastic, 

calculated 

NA 

LSRsample j LS test result for lot 

j 

(positive or 

negative) 

stochastic, 

calculated 

NA 

Parameter Descriptions and Baseline Values 

1) Frequency of a Contamination Event [How often does a ‘contamination event’ occur?] 

Time series Listeria species prevalence on various pieces of equipment were available from an 

FSIS in-depth verification conducted in a plant that was associated with an L. monocytogenes 

outbreak in humans (Hynes 2000). These data are shown in Table 10, and summarized in Table 

11. The data were analyzed using survival analysis and distribution fitting using NCSS23 

statistical software (Hintz, 2001). Several distributions were compared, and the log10 normal 

distribution had the greatest likelihood (Table 12). On a log10 scale, the mean time between 

contamination events was 1.08 with a standard deviation of 0.46. This is approximately 20 days 

± 29 days. Figure 2 shows the resulting fit. 

This analysis should be considered as an estimate only. Samples were not taken on a daily basis, 

and in some cases a considerable number of days passed between samples. Nor does the data 

provide sufficient sampling evidence to estimate the duration of contamination in comparison to 

23 Reference herein to any specific commercial products, process, or service by trade name, trademark, 

manufacturer, or otherwise, does not necessarily constitute endorsement or imply its endorsement, 

recommendation, or favoring by the United States Government. 

23
FSIS Listeria Risk Assessment May 26, 2003 

other data (i.e., Tompkin, 2002). Finally, these data were taken at a plant associated with an L. 

monocytogenes outbreak. How representative this plant’s data are compared to other plants is 

not known. 

Table 10. FSIS in-depth verification time series data for estimating time between contamination 

* Censoring refers to the type of observation that was made. An F or failed observation is one in which the time 

until the terminal event was measured exactly. An R or right censored observation provides a lower bound for the 

actual failure time. An L or left censored observation provides an upper bound for the actual failure time. An I or 

interval censored observation is one in which we know that the failure occurred between two time values, but we do 

not know exactly when (Hintz, 2001). 

events (Hynes 2000). 

Day Sequential Number 

Positive 

Line Days Between 

Positives 

Censor* 

Type 

12 2 1 11 F 

16 3 1 4 F 

31 4 1 15 F 

49 1 18 R 

3 2 2 2 F 

11 3 2 8 F 

19 4 2 8 F 

44 5 2 25 F 

57 2 13 R 

5 2 4 4 F 

16 4 11 R 

18 2 5 17 F 

95 3 5 77 F 

97 4 5 2 F 

117 5 5 20 F 

124 6 5 7 F 

138 5 14 R 

Table 11. Summary of Mean Time Between Start of Contamination Events 

Type of 

Observation 

Count Minimum 

(days between) 

Maximum 

(days 

between) 

Failed 13 2 77 

Right Censored 4 11 18 

Left Censored 0 

Interval Censored 0 

Total 17 2 77 

Table 12. Maximum Likelihood Fits to Mean Time Between Contamination Events for Various 

Distributions 

Distribution Likelihood Shape Scale Threshold 

Lognormal10 -50.89 1.08 0.46 0.0 

24
Log10(Time) 

1.50 

1.00 

0.50 

0.00 

-2.00 -1.13 -0.25 0.63 1.50 

Lognormal10 Quantile 

2.00 

Lognormal10 Probability Plot 

FSIS Listeria Risk Assessment May 26, 2003 

Lognormal -50.89 2.50 1.06 0.0 

Loglogistic -51.19 2.50 0.62 0.0 

Weibull -51.71 1.05 19.85 0.0 

Exponential -51.74 1 19.69 0.0 

Logistic -57.21 14.82 8.23 0.0 

Normal -59.05 19.10 18.97 0.0 

Extreme Value -63.73 32.23 25.68 0.0 

Figure 2. Fit of mean time between contamination events to log normal distribution. 

FSIS selected input distributions based on a maximum likelihood fit and a visual fit of data. 

Given the shortcomings of goodness of fit tests, this approach was considered reasonable for 

ascertaining the adequacy of fit. Frey (1999) pointed out that the most important approach for 

ascertaining the adequacy of fit due to the shortcomings of goodness of fit is to consider the 

visual fit of the data. FSIS believes that its selection of input distributions was reasoned, 

transparent, and reproducible. 

The available data to estimate the time between contamination events came from an in-depth 

verification investigation of an establishment producing ready-to-eat meat and poultry product 

associated with an outbreak of L. monocytogenes. This was the only data available for this 

model parameter. Besides the log normal distribution, an exponential distribution is often used 

to model a mean time between failure, and this theoretical approach was considered. 

To compare the two approaches, 10000 random numbers were generated using the best fit 

parameters for both the log normal distribution and the exponential distribution. The summary 

statistics comparing the distributions are shown in Table 13 below. 

25
FSIS Listeria Risk Assessment May 26, 2003 

Table 13. Selection of a Distribution for the Time Between Contamination Events. 

Parameter Log normal 

deviates 

Exponential 

deviates 

minimum 0.22 0.000117 

Q25 5.88 5.40 

Q50 (median) 12.05 13.25 

mean 20.56 19.49 

Q75 23.61 26.93 

maximum 717.40 197.00 

The values within the middle quartile range are quite similar. The distributions differ most in the 

tails. A quantile-quantile plot comparison of the two approaches is shown in Figure 3. Clearly, 

the log normal distribution is much more right skewed than the exponential. This implies that, at 

times, the random number generated will mean that several years can pass between 

contamination events if the log normal distribution is used, but not if the exponential distribution 

is used. Because the observed data fall much nearer the central value of the distribution, it is 

difficult to use the data alone to decide. Discussions with deli meat producers suggest that fairly 

long time between contamination events are possible for some plants. Thus it seemed 

appropriate to use a log normal distribution for this parameter. 

Figure 3. Quantile-Quantile Plot Comparison of the Lognormal and Exponental Distributions 

for the Time Between Contamination Events. 

The selection of the most appropriate distribution has been discussed at length in the agency and 

the lognormal distribution was selected because it was considered to be biologically more 

26
FSIS Listeria Risk Assessment May 26, 2003 

plausible. That is, it is conceivable that the movement of Listeria contamination from a biofilm 

in the in-plant environment is a multi-step process and that the probability of this movement 

occurring increases over time as the biofilm accumulates. This process would be better 

represented by the lognormal distribution rather than an exponential distribution. 

2) Duration of a Contamination Event [How long does a contamination event last?] 

Tompkin (2002) provided a table of sequential weekly Listeria species testing results and the 

number of weeks that Listeria species positives persisted. While the data available to estimate 

this parameter was limited, the Tompkin (2002) data was peer-reviewed, represented industry 

data, and was likely more representative than targeted environmental sampling data. Therefore, 

FSIS concludes that its reliance on these data was appropriate. These data were analyzed using 

survival analysis and distribution fitting with NCSS (Hintz 2001). Table 14 shows the data and 

Table 15 summarizes it. Table 16 provides the maximum likelihoods estimates for a variety of 

parameters. The log10 normal distribution had the second greatest likelihood (behind the log 

logistic). On the basis of consistency, ease of interpretation and ease of implementation, the 

lognormal distribution was used during the simulation. On a log10 scale, the mean 

contamination event duration was 0.60 with a standard deviation of 0.57. This is approximately 

9 days ± 20 days. Figure 4 illustrates the fit 

Table 14. Data for Contamination Event Duration Analysis. (Adapted from Tompkin 2002) 

Number of Weekly 

Tests 

Time (Days) Start Time 

(Days) 

Censor Type 

483 7 0 L 

136 14 7 I 

36 21 14 I 

32 28 21 I 

44 35 28 R 

Table 15. Summary of Duration of Contamination Event 

Type of 

Observation 

Count Minimum 

(days) 

Maximum 

(days) 

Failed 0 

Right Censored 44 35 35 

Left Censored 483 7 7 

Interval Censored 204 7 28 

Total 731 7 35 

Table 16. Maximum Likelihood Fit to Distributions for Contamination Event Duration 

Distribution Likelihood Shape Scale Threshold 

Loglogistic -777.5997 1.455336 0.7245711 0.0 

Lognormal10 -780.1027 0.6019546 0.5728621 0.0 

Lognormal -780.1027 1.386052 1.319064 0.0 

27
Lognormal10 Probability Plot 

1.60 

1.40 

Log10(Time) 

1.20 

1.00 

0.80 

FSIS Listeria Risk Assessment May 26, 2003 

Weibull -785.0569 0.6291547 5.966346 0.0 

Logistic -805.0837 -0.5512639 10.47769 0.0 

Normal -815.7148 -2.161562 20.28963 0.0 

Exponential -828.398 1 8.356113 0.0 

Extreme Value -830.9927 3.349459 26.03331 0.0 

0.40 0.70 1.00 1.30 1.60 

Lognormal10 Quantile 

Figure 4. Log normal distribution fit for duration of contamination event. 

3) Listeria spp. added to FCS during a contamination event 

There was no reported literature available to estimate this parameter. The FSIS Listeria risk 

assessment team decided that calibration of the model to obtain this input was preferable to other 

options (e.g., expert elicitation where the there is no knowledge, expert or otherwise, to estimate 

the level of L. monocytogenes transferred from a harborage site to food contact surface). 

Model calibration consists of changing values of model input parameters in an attempt to match 

the model’s output with independently derived values within some acceptable criteria. 

Calibration has been used for decades as a standard step in the modeling process, particularly 

when specific parameter values are unknown and relevant data do not exist. Calibration is wellfounded in the scientific literature. While it would be desirable to have data regarding, for 

example, the concentration of Listeria spp. on food contact surfaces, such data do not exist. In 

this case, it was entirely appropriate to use calibration methods to estimate the distribution of the 

concentration of Listeria spp. on food contact surfaces by matching the model’s output with the 

FDA/FSIS risk ranking model’s estimated input for L. monocytogenes contamination at retail. 

The FDA/FSIS risk ranking model is calibrated, or “anchored” on human health surveillance 

data that currently provide the best estimate of the magnitude of the public health problem 

associated with L. monocytogenes in food. Taking the FDA/FSIS risk ranking model as a given, 

28
FSIS Listeria Risk Assessment May 26, 2003 

the FSIS calibration procedure used to infer the initial concentration distribution makes good use 

of the available scientific information. Ideally, additional information on the initial L. 

monocytogenes levels would be useful, but the very low concentrations estimated for the vast 

majority of RTE product would frustrate additional efforts to collect better data at this point in 

the production process. Ideally, risk assessment models would be validated against independent 

observed data, but this is often not practicable. Model calibration or “anchoring” is a generally 

accepted practice in health risk assessment and environmental modeling (National Academy of 

Sciences 2002). The practice is most appropriate when the primary objective of the risk 

assessment is, as in this case, to provide a risk management tool for analyzing how to mitigate 

risk rather than to predict risk. Model calibration has been employed in one fashion or another in 

the prior USDA microbiological food safety risk assessments (Salmonella Enteriditis and 

Escherichia coli O157:H7), as well as the Joint Food and Agriculture Organization/World Health 

Organization risk assessments of Vibrio species (FAO/WHO 2001) and Salmonella (FAO/WHO 

2002). There is a trade-off, however, since data used to calibrate the model are unavailable for 

independent model validation. In the future, it may be possible to use a portion of surveillance 

data for model calibration and withhold a portion for model validation.24 

4) Sanitation Effectiveness 

Clean-up effectiveness measures the proportion of bacteria on the food contact surface that is 

removed through sanitation procedures. The model assumes the effectiveness of clean-up 

between lots is 50% and end of day clean-up is 75%. Therefore, total effectiveness of routine 

cleaning is actually 1-[(1-50%)*(1-75%)]=87.5%, or just less than a one log10 reduction in the 

amount of contamination remaining on food contact surfaces. A similar level of effectiveness 

was estimated for cleaning of stainless steel surfaces experimentally inoculated with a biofilm of 

Pseudomonas aeruginosa and Staphylococcus aureus by Gibson et al. (1999). While some plants 

may achieve greater log reductions from their cleaning practices, the effectiveness levels 

assumed in this risk assessment seem reasonable as averages across the entire industry. 

Regarding enhanced cleaning, it seems unreasonable to assume an infinite log reduction. Such a 

level of effectiveness could never be proven experimentally. Nevertheless, an analysis of these 

inputs suggests the model is insensitive to higher effectiveness levels because much of the 

contamination on food contact surfaces is transferred to RTE deli meats during the time of 

processing. 

5) Transfer of Listeria species from Food Contact Surface to RTE Product 

24 Note that model calibration is distinct from model validation. Model validation is a process for assessing how 

accurately the model predicts actual phenomena in nature. Validation involves the comparison of model predictions 

with empirical data not used in developing the model. See Law and Kelton (1991) for a further discussion of the 

distinction between calibration and validation. Given the limited data available to develop this risk assessment 

model, validation was not accomplished. Nevertheless, because annual mortality from L. monocytogenes in RTE 

foods is expected to be reasonably constant from year to year (absent some purposeful intervention to prevent such 

mortality), this model’s predictions about annual mortality are expected to be reasonably consistent with estimates 

from future public health surveillance data. Such consistency provides a limited validation of this model. 

29
FSIS Listeria Risk Assessment May 26, 2003 

Montville et al. (2001) and Chen et al. (2001) found that transfer coefficients of bacteria were 

log normally distributed based on testing a variety of foods and surfaces such as hands, lettuce, 

and spigots. The range of transfer coefficients varied from 0.01% to 10%, with a standard 

deviation of about 1 log. 

Midelet and Carpentier (2002) prepared L. monocytogenes biofilms by contacting meat exudates 

with 5x107

 cfu/mL to stainless steel slides for 3 hours. The planktonic bacteria were then 

removed by washing. The resulting L. monocytogenes surface concentrations were estimated in 

the range 106.1

 cfu/cm2 for stainless steel to 106.4 cfu/cm2 for PVC. Twelve sequential contacts 

with beef were then conducted. After 12 contacts, the study results suggested that approximately 

a) log 6.1 transferred from log 6.1 initial population for stainless steel, for a transfer 

coefficient of 1 

b) log 6.45 transferred from log 6.8 initial population for PU for a transfer coefficient of 

0.45 

c) log 6.25 transferred from log 6.4 initial population for PVC for a transfer coefficient of 

0.71 

The mean transfer coefficient used was 0.72, which is equivalent to a mean log transfer 

coefficient of -0.14. The standard deviation reported from Montville et al. (2001) and Chen et al. 

(2001) is assumed to apply for this input. Variability about the transfer coefficient, therefore, 

was assumed to be log normally distributed (normally distributed on the log scale) with the mean 

of –0.14 and a standard deviation of 1. Values generated above 0 (i.e. 100% transfer) were 

simply truncated to 0. These values imply that the majority of the Listeria species on food 

contact surfaces readily transfer to product. 

Estimations for the three different materials were used in the Midelet and Carpentier (2002) 

paper. Because in the risk assessment the food contact surface was modeled as a single 

representative surface, a stochastic transfer coefficient varied from lot to lot based on these 

estimates was deemed appropriate. 

Although the distribution is truncated at 100% transfer, the actual approach used does not result 

in a two-peaked distribution to any noticeable extent. It is certainly true that because of the 

truncation in the generation of the transfer coefficients, the resulting distribution is not normal on 

the log scale. Figure 5 presents a histogram of 10000 simulations for the transfer coefficient 

using the approach in this risk assessment. There is no evidence of a bimodal distribution. 

30
FSIS Listeria Risk Assessment May 26, 2003 

Figure 5. Histogram for the fit of the transfer coefficient data. 

An alternative approach was considered – to simply draw with replacement from the three 

transfer coefficient values provided by Midelet and Carpentier (2002). The empirical cumulative 

density functions for both approaches are shown below (Figure 6). In both cases, 10000 values 

for the transfer coefficient were generated. The black curve (below) represents the algorithm 

selected for this risk assessment. The impact of the truncation can be seen in the jump at a log 

transfer coefficient of 0. Approximately 45% of the log values are set to 0. Twenty percent of 

the values are less than –1. The alternative approach is shown in red. Only 3 values are 

available, so the resulting curve resembles a step function. Using this approach, 33% of the data 

have a log transfer coefficient of 0, 33% have a value of –0.14, and 33% have a value of –0.34. 

31
FSIS Listeria Risk Assessment May 26, 2003 

Figure 6. Empirical Cumulative Density Distribution for the Transfer Coefficient. 

Obviously, the method chosen results in more variable transfer coefficients, with the possibility 

of much lower values than available from the alternative approach. This seemed an appropriate 

approach given the limited data. 

There is often a great deal of confusion about the use of prevalence data in estimating transfer 

coefficients. There are some studies available that examine transfer from food contact surface to 

RTE product, and these were considered for this risk assessment. However, they are based on 

prevalence rather than concentrations, and so are of limited usefulness. For example, Deaver 

(2002) evaluated transfer from inoculated equipment to RTE product, but little useful data could 

be obtained in estimating a transfer coefficient for this risk assessment. There are two reasons 

for this. The first is that the study was conducted at the package level, not a lot level as used in 

32
FSIS Listeria Risk Assessment May 26, 2003 

the risk assessment. The second and more important reason is that only prevalence was 

examined, making it impossible to calculate a transfer coefficient. 

The following examples illustrate this point. They are based on Day 1 25-gram sampling for 

Trial 2, but similar examples could be constructed for any of the results. The slicer was 

inoculated with 1080 cfu L. monocytogenes. Ten of the 100 samples tested positive for Lm. The 

table below presents 3 possible scenarios consistent with the data, assuming that 10 cfu 

transferred to the package would be sufficient to find the sample positive. (This number is 

probably higher than needed, but only affects the minimum transfer coefficient calculated.) 

In Case A, the minimum number of cfu is transferred to each sample. The vast majority of the 

cfu’s remain on the slicer, for an overall transfer coefficient of 0.09. In Case B, all the cfu’s are 

transferred to the samples, leaving none on the slicer and resulting in a transfer coefficient of 1.0. 

Table 17. Examples Illustrating Why Prevalence Data is Insufficient for Constructing a Transfer 

Coefficient for the FSIS Listeria Risk Assessment 

C ase A C a se B 

P ack ag e # L m S lic er L m P a cka g e L m S licer L m P a cka g e 

In oculum 108 0 108 0 

1 107 0 1 0 9 0 990 

3 106 0 1 0 8 0 1 0 

5 105 0 1 0 7 0 1 0 

7 104 0 1 0 6 0 1 0 

9 103 0 1 0 5 0 1 0 

39 102 0 1 0 4 0 1 0 

117 101 0 1 0 3 0 1 0 

195 100 0 1 0 2 0 1 0 

197 99 0 1 0 1 0 1 0 

199 98 0 1 0 0 10 

T ran s fer C o ef 0.0 9 1.0 0 

The observed prevalence of 10% (i.e. 10 packages out of 100 positive) are consistent with a 

transfer coefficient that ranges from 0.09 to 1.00. Thus, prevalence data cannot be used to 

impute a transfer coefficient. Because of this range, the study was not used directly in the risk 

assessment, especially since a relevant quantitative study was available in the peer-reviewed 

literature. 

A prevalence of 0 can still imply a non-zero transfer coefficient if the number of organisms 

transferred to each package is below the detection limit. A prevalence of 100% can still imply a 

transfer coefficient near 1 if only a small number of organisms are transferred to each package. 

Thus, the Deaver (2002) study had little relevance to this risk assessment. 

5) Ratio of Listeria monocytogenes to Listeria species 

No data were available on the ratio of concentrations of L. monocytogenes to Listeria species. 

Data, however, were available on the prevalence of L. monocytogenes to Listeria species (i.e., 

data on when a food contact surface was found positive for Listeria species, whether or not the 

surface was also positive for L. monocytogenes). These prevalence data were available from the 

published literature (Tompkin 2002) and some unpublished industry data provided to FSIS 

(Cornell University, November 2002). Table 18 summarizes these values. 

33
FSIS Listeria Risk Assessment May 26, 2003 

Table 18. Prevalence Data for L. monocytogenes to Listeria species Ratios 

Number of Samples 

Positive for Listeria 

species 

Percent of Samples also 

Positive for L. 

monocytogenes 

1 100 

115 96 

11 82 

90 71 

142 71 

128 62 

328 57 

237 54 

204 47 

46 41 

85 38 

90 34 

3 33 

219 27 

241 23 

318 5 

These data concerning the ratios for Listeria species to L. monocytogenes were tested and found 

not to be significantly different from a normal distribution. Therefore, this input was modeled as 

a variability distribution. The distribution fit was not weighted by the number of samples. Each 

ratio in the table above was given equal weight. The mean was 52% and the standard deviation 

was 26%. Values outside 0-100% were rounded to 0% or 100% appropriately. 

The model uses this ratio of Listeria species/L. monocytogenes prevalence and applies it to 

Listeria species/L. monocytogenes concentration ratios. Given the lack of more specific data, the 

assumption that the ratio of L. monocytogenes to Listeria species prevalence applies to the ratio 

of the concentrations is a reasonable use of available data. Given a random distribution of L. 

monocytogenes amongst all Listeria species, and the expectation that all Listeria behave in a 

roughly similar manner, this assumption is a reasonable default in the absence of specific 

information to the contrary. Moreover, in a peer review of this risk assessment, it was found that 

the truncated normal (52%, 26%) distribution of the species prevalence ratio values assumed in 

the risk assessment, compared to a non-parametric empirical cumulative distribution of such 

prevalence data, provides a reasonable fit (ORACBA 2003). 

6) Probability of detecting 1cfu in a sample 

This probability of detecting 1 cfu in a RTE sample or FCS swab sample is different from the 

test sensitivity. Test sensitivity is the probability that a contaminated sample tests positive. A 

contaminated sample may contain anywhere from one organism to a very large number of 

organisms. To calculate a test sensitivity would require consideration of the population of 

contaminated samples. Therefore, test sensitivity is density dependent and differs from the 

probability of the test successfully detecting 1 cfu. Specificity is the probability that a non34
FSIS Listeria Risk Assessment May 26, 2003 

contaminated sample tests negative and would be estimated at <100% if laboratory error 

information were available and could be considered. 

For both contact surface testing and product testing, the modeled concentration of the organism 

was multiplied by the sample size to estimate the mean of a Poisson distribution -- a probability 

distribution that is appropriate for modeling such concentrations. (For food contact surfaces, the 

concentration is measured in cfu/cm2

 and the sample size is measured in cm2. For RTE product, 

the sample size is measured in cfu/gram, and the sample size in grams.) A random number was 

generated from this distribution that represented the number of cfu’s in the sample itself. 

Once the number of organisms in the sample was known, the probability that a test to detect the 

presence of the pathogen would yield a positive or negative result could be determined by using 

a binomial distribution. The Agency did this by using the following expression: 

1 − (1 − p)

n 

where p is the probability of detecting 1 cfu in the sample, and n is the number of cfu’s in the 

sample from the Poisson calculation. The p probability is based on the detection limit and 

microbiological test sensitivity, and is the input parameter to the risk assessment model. 

As for the limit of detection, the value for this input was obtained from the FSIS Microbiological 

Laboratory Guidebook (http://www.fsis.usda.gov/OPHS/microlab/mlg8.03.pdf), which reports 

the detection limit for L. monocytogenes testing as better than 1 cfu in a 25-gram sample. Thus, 

the p value should be fairly high for L. monocytogenes testing, conceptually near 1, because the 

base data set assumed a 25-gram sample. 

Moreover, a study by Hayes et al. (1992) reported that the USDA method for L. monocytogenes 

had an overall sensitivity of 74%, with a sensitivity of 75% for the luncheon meat subcategory. 

While the Hayes et al. work is reporting test sensitivity, many of the foods included in their 

analysis had concentrations below the limit of detection for MPN method (<0.3 CFU per gram). 

Nevertheless, these samples were at or above the limit of detection for the qualitative culturing 

methods (i.e.., 0.04 CFU per gram or 1 CFU per 25 grams). It is reasonable to argue that these 

results are mostly indicative of the likelihood of detecting samples containing very few (or even 

a single) organisms. 

Assuming that the concentration of L. monocytogenes in RTE product at processing is distributed 

as 10^Normal(-9, 3.5) cfu/gram, the levels in 50,000 25-gram samples were simulated as a 

random Poisson process (Haas et al. 1999). Approximately 2% of the simulated 25-gram 

samples were contaminated with one or more cfu of L. monocytogenes, and, as illustrated in 

Figure 7, approximately 70% of the simulated contaminated 25-gram test samples contained 

more than 1 cfu of L. monocytogenes. 

35
FSIS Listeria Risk Assessment May 26, 2003 

Figure 7. Levels of L. monocytogenes in 25-gram samples of RTE product. 

As shown in Figure 8, for the roughly 2% of simulated test samples that were contaminated, the 

mean likelihood of detection exceeds 80% for pdetect 1 values between 0.5 and 0.95. This 

suggests that given its presence in a 25-gram sample, there is a reasonable likelihood that L. 

monocytogenes would be present at levels sufficiently high to make the probability of detecting a 

single organism of minor importance. This is due to the fact that the likelihood of detection 

becomes insensitive to this probability as the numbers of L. monocytogenes in the sample 

increase: likelihood(detection, given presence at level of n) = 1-(1-pdetect 1)

n

. 

Sensitivity to Uncertainty about p(detect 1 organism) 

0.45 

0.50 

0.55 

0.60 

0.65 

0.70 

0.75 

0.80 

0.85 

0.90 

0.95 

1.00 

0.45 0.5 0.55 0.6 0.65 0.7 0.75 0.8 0.85 0.9 0.95 1 

assumed p(detect 1 organism) 

likelihood detect +ve 25 g 

sam

ple 

Figure 8. Likelihood of detecting L. monocytogenes-positive samples. 

A baseline value of 75% probability was used for both FCS sampling and RTE lot sampling. 

Simulated Test Samples 

0 

0.1 

0.2 

0.3 

0.4 

0.5 

0.6 

0.7 

0.8 

0.9 

1 

0 1 4 6 7 8 9 10 

log cfu in +ve 25 g samples 

cu

m. freq. 

2 3 5 

36
FSIS Listeria Risk Assessment May 26, 2003 

7) Homogeneity of L. monocytogenes within a RTE Product Lot and on FCS 

Homogeneity of contamination is a reasonable default assumption often used within the field 

of microbial and environmental risk assessment. The degree of spatial cluster is unknown 

and selection of the extent of cluster would be arbitrary. Furthermore, an assumption of 

clustering should be coordinated with assumptions of sampling design strategies. For 

example, if we know the agent is limited to a specific fraction of the food contact surface 

area, sampling strategies might be designed to ensure at least sampling of that area. It should 

be recognized that a clustered distribution assumption would require recalibration of the 

concentration distribution and result in higher concentrations in the contaminated area. This 

heightens the likelihood of detection if any portion of this contaminated region is sampled. A 

sampling plan with many composited samples each over a very small sampled area, would 

compensate for the clustering. 

The nature of the potential clustering in RTE product is more complicated than it might 

initially appear. Chae and Schraft (2000) found that L. monocytogenes biofilms grown under 

static conditions occur in two distinct layers. Different L. monocytogenes strains also 

exhibited different biofilm growth rates and different adhesion strengths. Deaver (2002) 

studied transfer coefficients from an inoculated slicer to RTE product. The slicer was 

inoculated with 103

 cfu L. monocytogenes on 1 square inch of the slicer blade and allowed to

air dry for 20 minutes, and 200 packages were then processed. The entire package (~125 g) 

and 25 g samples were then analyzed for L. monocytogenes, and the prevalence reported. 

Odd number samples were tested on day 1. (The even number samples were tested on day 

30. These results are not shown.) Figure 9 depicts the results for 25-gram samples of salami 

and turkey. As might be expected, both show that samples from packages processed early 

were often positive. However, positives were also detected during the middle of the 200 

package run. Strangely, both products also had one or more positive samples at the end of 

the 200 package run. These results suggest that no simple approach to clustering will be 

valid. Because only prevalence was reported, it is not known if the concentration of L. 

monocytogenes in the “negative” samples was truly zero, or merely below detection. 

37
FSIS Listeria Risk Assessment May 26, 2003 

Positive Turke

y Samples Salami S

ampl

es 

Da

y 1 Results Day 1 Results 

Negative 

Positive 

Negative 

0 50 100 150 200 

Sample No. (Only odd numbers tested) 

Figure 9. L. monocytogenes contamination on samples of RTE product to evaluate the effect 

assumption of clustering or homogeneity of contamination in a product lot. 

The second aspect of clustering that must be considered is the dynamic nature of the 

contamination event. Contamination events which occur over a very short time frame are 

more likely to produce clustering in the RTE product. Contamination events which occur 

over longer time frames are more likely to produce a more uniform concentration 

distribution. Based on the available data for the duration of a contamination event, this risk 

assessment model uses contamination events that occur over several days duration. 

38
FSIS Listeria Risk Assessment May 26, 2003 

Yet a third consideration is the possibility that after some portion of a lot becomes

contaminated, the RTE product may transfer the bacteria to a food contact surface further

down the production line. These bacteria can then move to a later portion of the lot. In

effect, cross-contamination between the product and the entire food production chain would 

likely disperse the bacterial contamination among the lot more than the initial contamination 

location might imply. 

8) Growth of L. monocytogenes on RTE Product During Distribution from Plant to Retail 

The 2001 FDA/FSIS Listeria risk ranking model includes an option for growth from the plant 

to retail for FSIS-regulated products (e.g., deli meats). Based on a time-temperature submodel, a growth of 1.9 log units (a multiplier of about 79) was applied to deli meats based on 

plant monitoring data. While the sub-model itself was stochastic, the final multiplier applied 

to appropriate data sets was a constant. 

Levine et al. (2001) report 1999 prevalence levels of L. monocytogenes in various deli meat 

products at the processing plant: these levels were 2.71% for cooked, roast and corned beef, 

and 4.58% in sliced ham and other pork luncheon meats. The National Food Processors

Association (NFPA) survey of RTE deli meats at retail found an L. monocytogenes

prevalence of 0.9%. Although these L. monocytogenese prevalence levels in deli meats are 

not directly comparable, these values were used to justify a lowering of the growth factor in 

this risk assessment. A growth of 1.0 log units (i.e., a factor of 10) was used for all lots,

rather than the 1.9 used in the FDA/FSIS risk ranking model (see Appendix B for further

discussion). 

Note that the limited understanding of growth during shipment to retail, and the nonstochastic nature of the growth model used in this analysis increases the uncertainty of the

risk assessment outputs regarding the effectiveness or the use of growth inhibitors or

reformulating product. 

9) Line production

FSIS (2003) reports a survey among RTE processors of deli meats (and hot dogs) to evaluate 

the fraction of the deli meat food supply produced by large, small and very small plants. 

Additionally, the pounds per shift per line for each plant size were also estimated. The 

survey found that for deli meats, about 48% of the food supply is produced by large plants, 

48% by small plants, and the remaining 4% by very small plants. The estimated average 

production volume in pounds of deli meats per line per shift is shown in Table 19. 

[Note: The data from the FSIS survey of RTE processors of deli meats was also used to 

stratify establishments according to those that produce a high (upper 25th percentile of 

industry), medium (50th-75th percentile), or low (lower 50th percentile) volume of product. 

Analysis of this data and risk estimates by plant production volume are provided in 

Appendixes C and D.] 

Table 19. Lot (per line per shift) weight by plant size. 

Plant size Lot weight (lbs) Lot standard deviation (lbs) 

Large 19371 14000 

39
FSIS Listeria Risk Assessment May 26, 2003 

Small 7100 10600 

Very Small 2800 9500 

Lot weights (i.e., pounds of deli meat per line per shift) were varied stochastically from lot to 

lot. These distributions were assumed to be normal. Simulated lot weights less than 1000 

pounds were rounded up to 1000 pounds. 

While the survey found that the average mass of a lot of RTE product varied by plant size. 

But there is no evidence of a difference in the occurrence of L. monocytogenes in RTE 

product by plant size. To reconcile differences in lot mass with equivalency in L. 

monocytogenes occurrence by plant size, the model was adjusted for food contact surface 

sizes. This adjustment eliminated the unintended bias that would have resulted from 

assuming the same food contact surface size regardless of plant size. 

No survey data of plant characteristics (e.g., line configuration, Listeria control program 

implementation, and packaging technology) or corresponding data on the prevalence and/or 

level of Listeria species in the establishment was provided to the Agency. Therefore, these 

factors cannot be further evaluated at this time. As already noted, however, data on 

production volume from the FSIS survey on processors of deli meats was analyzed and risk 

estimates provided in Appendixes C and D. 

10) Post Processing and Growth Inhibition 

Neither post processing interventions nor growth inhibition product formulation and packaging 

were considered for the base run. However, their impact was evaluated during the different 

scenarios in the same manner as different FCS testing frequencies. The default assumptions 

regarding efficacy of post-processing interventions used in the model may very well be lower 

than efficacies observed in plants or laboratories. Simulating a higher efficacy will illustrate 

greater benefits for these interventions. The current model settings, therefore, are conservative. 

For example, the current model predicts that post-processing interventions are at least as 

effective as a testing program that tests every lot of product. Therefore, the model already gives 

Agency decision makers the useful information that post-processing interventions that are 90% 

to 95% efficacious are as effective as, or more effective than, testing. 

Data on interventions, such as the use of lactate and diacetate to prevent growth during 

distribution, which has been published (Seman et al., 2002) were reviewed during the 

development of the risk assessment. However, since the risk management questions that were 

presented to the risk assessors at the outset of the assessment did not deal with specific product 

formulations, the risk assessors decided to model growth inhibition in a manner that could easily 

be applied to any product reformulation or packaging. Moreover, as mentioned previously in the 

context of post-processing controls, the efficacy of growth inhibitors assumed in the risk 

assessment model may be conservative. Nevertheless, FSIS risk managers can conclude from 

this model’s results that growth inhibitors are as effective as, or more effective than, testing food 

contact surfaces. Simulating higher efficacy from growth inhibitors only serves to reinforce this 

conclusion. In addition, greater percent reductions were modeled as part of the sensitivity 

analysis and did show greater public health impacts. 

40
FSIS Listeria Risk Assessment May 26, 2003 

Model Implementation and User Interface 

The FSIS Listeria risk assessment in-plant dynamic model was written in Microsoft Visual 

Basic 6.0. Three additional third-party add-ons were used and are necessary to recompile the 

model: Videosoft vsFlex 6.0, Videosoft vsOCX 6.0, and Graphic Server 5 for Windows. In 

addition, several subroutines from Numerical Recipes (Press et al. 1992) were used. The 

model is designed so that almost all the required data are entered through the graphical user 

interface and can be easily changed by the user. Tabs separate the major data entry screens. 

Each data entry or result screen is described below. 

Several portions of the model not directly related to the risk assessment have not yet been 

completed. These include the printing and help functions. 

The Project Data screen shown in Figure 10 is used to store information about the specific 

model run. None of the data are used within the simulation itself. 

Figure 10. Project Data Entry Screen. 

The Plant Data screen shown in Figure 11 is used to enter information on plant production, 

lot size, sanitation and testing controls. All of these inputs can be modified to perform 

sensitivity analysis or update the model with more recent data. 

41
FSIS Listeria Risk Assessment May 26, 2003 

Figure 11. Plant Data Entry Screen 

There was little available data on the effectiveness of sanitation in reducing the level of 

Listeria species on food contact surfaces. The base model assumes a brief cleaning or wipe 

down between the first lot of the day with an efficiency of 50%, i.e. 50% of the Listeria 

species remaining on the food contact surface at the end of the lot production are removed by 

sanitation controls. The base model assumes greater sanitation effectiveness after the 2nd lot 

production, since many plants run a 3rd shift as a sanitation shift. The end of day sanitation 

efficiency was assumed to be 75% in the base model. Therefore, overall effectiveness of 

routine cleaning is assumed to be 87.5% (i.e., 1-[(1-50%)*(1-75%)]). 

Finally, if a food contact surface was found positive for Listeria species, the base model 

assumes that the plant would conduct a more effective or enhanced cleaning to remove the 

bacterial contamination. This effectiveness was set at 95% for the base model. The 

enhanced cleaning was always lagged in time to allow for the time between the testing and 

when the results would be available. 

The frequency of food contact surface testing for Listeria species varied depending on the 

scenario being analyzed. Different frequencies were allowed for different plant sizes (i.e., 

for large, small, and very small establishments). Two interventions based on testing results 

were allowed. First, if a food contact surface tests positive for Listeria species, then the RTE 

product lot would be tested for L. monocytogenes. If the RTE product lot was positive for L. 

42
FSIS Listeria Risk Assessment May 26, 2003 

monocytogenes, then this lot is disposed of and not used for human consumption. Second, if 

a food contact surface tested positive for Listeria species, then the food contact surface 

would undergo enhanced cleaning. The base model runs had both options selected. 

The model also allowed for the simulation of a test-and-hold procedure for the RTE product 

lot. If this was selected and a food contact surface was found to be positive for Listeria 

species, the product lot that was produced at the same time the food contact surface was 

sampled and later found positive for Listeria species would be tested for L. monocytogenes. 

If the test-and-hold option was not selected, then the RTE product lot that would be tested for 

L. monocytogenes would be one that was produced after the results from the food contact 

surface sampled earlier were obtained. 

RTE product lot testing for L. monocytogenes was similar in concept. Only one intervention 

was considered: disposal of a product lot found to be L. monocytogenes positive. Disposal 

implies that the lot was removed from the food supply, but could include reprocessing the 

affected RTE product lot. The base model always had this option selected. 

Note that the total number of lots produced per line is fixed at 60 per month (2 lots per day 

per line multiplied by 30 days per month) within the model. Thus the maximum testing 

frequency for any size plant is 60 per month. 

The model allows for food contact surface testing and lot testing to be performed either 

randomly or systematically. Random testing would randomly select the specified number of 

lots to be tested from among the 60 available that month. Systematic testing would keep a 

constant time interval between the lots being tested, with a random start. For example, a 

systematic sample might take the first lot produced each Tuesday to obtain 4 lots per month. 

The base model assumed systematic sampling. Note that systematic sampling has 

implications for use of test-and-hold procedures. At 16 samples per month, the timing 

between systematic samples matched the lag between sample analysis and reporting, and 

simultaneous sampling of food contact surfaces and lots took place even if the test-and-hold 

option was not selected. 

The Contamination Data screen, shown in Figure 12, is used to enter data relating to 

contamination event timing, duration, levels, transfer coefficients, area swabbed, and product 

lot mass sampled. Most of these data have been described previously. The “number of 

composites” was not implemented in this version of the model. 

43
FSIS Listeria Risk Assessment May 26, 2003 

Figure 12. Contamination Data Entry Screen 

The Post-Processing Data screen shown in Figure 13 is used to enter data relating to product 

pre- and post-packaging interventions, growth inhibitors, and product reformulation. A 

variety of these interventions have been studied. Example of interventions include: addition 

of sodium lactate or sodium diacetate in frankfurter formulations. (Bedie et al. 2001, Glass et 

al. 2002), steam/hot water pasteurization (Murphy and Berrang 2002), vacuum-steamvacuum (Kozempel et al. 2000, Sommer et al. 2002), high pressure technology (Avure 

Technologies studies), and antimicrobial packaging (Cagri et al. 2002). 

44
FSIS Listeria Risk Assessment May 26, 2003 

Figure 13. Post Processing Data Entry Screen 

For this risk assessment model, the specific pre- and post-packaging interventions are not 

required. The fraction of production by plant size and the effectiveness of these interventions 

are required inputs. The effectiveness of a pre- and post-packaging intervention is treated as 

a uniform random number between the ranges given and reduces the arithmetic scale 

concentration of L. monocytogenes in product by that amount. The effectiveness of growthinhibitors is also a uniform random number between the specified ranges and is used to 

adjust the exponential growth predicted between processing and retail. 

The base model assumed that none of these measures are used by the industry. Scenarios 

were run where the impact of these measures were evaluated. 

The Advanced Data tab shown in Figure 14 is used to enter data that should not be changed 

during most scenarios. These include testing lags and detection limits, L. monocytogenes to 

Listeria species ratios, food contact surface areas, and growth of L. monocytogenes from the 

processing plant to retail. The model requires the probability of detecting 1 cfu of Listeria 

species for food contact surface testing and 1 cfu of L. monocytogenes for product testing. 

The total number of cfu’s in the sample provided are generated as a Poisson random number 

with the mean of Listeria species concentration multiplied by the total area swabbed for food 

contact surface tests or L. monocytogenes concentration multiplied by sample mass for 

product testing. This sampled cfu number is then used to determine if the sample tests 

45
FSIS Listeria Risk Assessment May 26, 2003 

positive or negative based on the probability of the test successfully detecting 1 cfu. For the 

base runs, both probabilities were set at 75%. 

Figure 14. Advanced Data Entry Screen 

The L. monocytogenes to Listeria species ratio has been described above. The model 

assumed that the distribution was normally distributed but truncated to fall between 0% and 

100%. 

The area of the food contact surface was needed to convert between concentration of Listeria 

species on the surface and total number of organisms present on the food contact surface. 

Limited data was available for this parameter. Base runs assumed that the area varied as a 

uniform random number from 100,000 cm

2

 to 1,000,000 cm2. While treated as a random 

variable, the value was held constant while a contamination event was occurring. 

The Simulation screen shown in Figure 15 is where the model is actually run. The number of 

product lots to be simulated is the only required input. Results are based on a run of 

1,000,000 lots, although early calibration runs were based on fewer lots. The current 

implementation of the model is rather inefficient in that the model actually simulates the 

number of lots for each of the 3 plant sizes, then randomly selects the lots to go to retail 

based on the percentage of the food supply provided by each plant size. The user can 

46
FSIS Listeria Risk Assessment May 26, 2003 

optionally request that all the information for each lot simulated be output to a commadelimited file that can be read by a spreadsheet or database. Note that these output files can 

become quite large. 

Figure 15. Simulation Screen 

The percentiles of the L. monocytogenes concentrations at retail and after pre- and postpackaging interventions are provided in conjunction with the updated FDA/FSIS exposure 

assessment levels for L. monocytogenes in deli meats at retail. This portion of the model was 

used primarily during calibration. The mean and standard deviation of the Listeria species 

levels added to the food contact surface were varied in order to match the levels of L. 

monocytogenes in deli meats observed in the updated FDA/FSIS exposure assessment. 

Empirical cumulative density functions are provided as part of the output on the Graphs tab 

shown in Figure 16 for either the L. monocytogenes concentration in product at retail or the 

Listeria species concentration on food contact surfaces. These graphs were used primarily 

during the calibration phase. The option box selection controls which graph is displayed. 

Only the non-zero concentrations are shown on either plot. The graph software can only 

display about 32,000 points, and therefore the graphs are not available if a large number of 

lots are simulated. 

47
FSIS Listeria Risk Assessment May 26, 2003 

Figure 16. Graph Output Screen 

The Output Stats screen shown in Figure 17 summarizes the testing results. It provides the 

numbers of RTE product lots simulated for each plant size, the number chosen for retail, the 

number of food contact surfaces and lots tested and the number that failed. Some of the 

quantiles from the Simulation tab are also given. Finally, two contingency tables are 

provided to summarize the testing results. The contingency tables shown n Figure 17 break 

down the food contact surface and RTE product lot testing in a 2 dimensional matrix, and are 

used to estimate the overall prevalence of food contact surface samples positives for Listeria 

species, RTE product lots positive for L. monocytogenes, and the likelihood of finding a RTE 

product lot positive for L. monocytogenes if the corresponding food contact surface sample is 

positive for Listeria species. The first of the contingency tables is used when the test-andhold procedure is in place, and the RTE product lot tested for L. monocytogenes is the one 

that is produced at the same time the food contact surface is tested for Listeria species. The 

second contingency table is the results for the likelihood of detection of L. monocytogenes in 

a RTE product lot when a food contact surface tests positive for Listeria species when the 

test-and-hold procedure is not in place (i.e., this option was not selected in the model). 

Again, when the test-and-hold procedure is not in place, the RTE product lot tested is one 

that lagged in time after the food contact surface was tested for Listeria species and later 

found to be positive (i.e., once the test results are obtained from the laboratory). 

48
FSIS Listeria Risk Assessment May 26, 2003 

Figure 17. Output Statistics Screen 

Calibration of the In-plant Dynamic Model 

As described earlier, the values for the mean and standard deviation of the number of Listeria 

species transferred to food contact surfaces at the beginning of lot production, while a 

contamination event is ongoing, are unknown. The distribution was assumed to be lognormal. Values were initially selected for these parameters and the resulting simulated 

distribution of the concentration of L. monocytogenes in deli meat at retail was compared to 

the updated FDA/FSIS exposure assessment values for the concentration of L. 

monocytogenes in deli meats at retail. The updated FDA/FSIS exposure assessment model 

for deli meats actually estimates 300 plausible lognormal distributions (one for each iteration 

of the model) for L. monocytogenes contamination in deli meats at retail. A single set of 

parameters was estimated by calculating the average of the mean and standard deviation 

across the 300 sets of parameters. 

By comparing the distribution for the concentration of L. monocytogenes in deli meats at 

retail predicted by the FSIS in-plant model to the distribution estimated by the updated 

FDA/FSIS exposure assessment values for deli meats at retail, the two parameters for the 

input distribution (i.e., number of Listeria species transferred to the food contact surface) 

49
FSIS Listeria Risk Assessment May 26, 2003 

were changed on a iterative basis until the two distributions were deemed sufficiently close.

Lm i

n R

TE product at r

etail 

Figure 18 provides the comparison of the final FSIS in-plant model calibration distribution (cf

u/g) with the updated FDA/FSIS exposure assessment concentration of L. monocytogenes in deli 

meats at retail. Note that only two parameters were treated as unknowns. All other model 

parameters were kept at their base values. The final estimates of the organisms transferred 

had a mean on the log10 scale of – 6 cfu/cm2

 and a standard deviation on the log scale of 3.5 

cfu/cm2

. 

1e+6 

1e+5 

1e+4 

1e+3 

1e+2 

1e+1 

1e+0 

1e-1 

1e-2 

1e-3 

1e-4 

1e-5 

1e-6 

1e-7 

75 80 85 90 95 100 105 

FDA 

LMRA Calibration 

Percentile 

Figure 18. Final FSIS Listeria Risk Assessment In-plant Model Calibration to the Updated 

FDA/FSIS Exposure Assessment Concentrations of L. monocytogenes in Deli Meats at 

Retail. The mean and standard deviation of the log number of Listeria species transferred to 

the food contact surface at the beginning of each lot production during a contamination event 

were used to fit this distribution. 

Model Stability 

Twenty separate runs were made using the 4-2-1 scenario. 

“4-2-1” means that food contact surfaces are tested for Listeria species at one of the following 

frequencies, depending on establishment size: 

• If the plant is large, at least four tests, per line, per month; 

• If the plant is small, at least two tests, per line, per month; 

• If the plant is very small, at least one test, per line, per month. 

50
FSIS Listeria Risk Assessment May 26, 2003 

The variability of the quantiles is shown in Figure 19 below as a box plot. The interquatile range 

is shown as a rectangular box, with the median value as a line within the box. The 95th 

percentiles are shown as vertical lines extending from the box. These graphs then indicate 

central tendency (the median), spread (both the interquartile range and the 95th percentiles), and 

an indication of symmetry/skewness (the location of the median within the box.) The results 

indicate very little spread among the 20 replicate model runs. As expected, the 99.99th quantile 

exhibited more variability than the lower quantiles. Overall however, the variability appears 

small among replicate simulations. 

Variability of 20 runs of 4-2-1 scenario 

(1,000,000 lots per run) 

1e+7 

1e+6 

1e+5 

1e+4 

1e+3 

1e+2 

1e+1 

1e+0 

1e-1 

1e-2 

1e-3 

1e-4 

1e-5 

1e-6 

1e-7 

Q80 Q99 Q99.99 

Percentiles 

Figure 19. Stability of the FSIS Listeria risk assessment model simulated quantiles based on 20 

runs of the 4-2-1 scenario. Lm C

oncentr

ation at R

etail 

51
FSIS Listeria Risk Assessment May 26, 2003 

FSIS LISTERIA RISK ASSESSMENT OUTPUTS 

The FSIS Listeria risk assessment outputs provided in this report are only those that inform risk 

management decision-making in regards to the following policy questions: 

1) How effective are various food contact surface testing and sanitation (corrective action) 

regimes (e.g., vary the frequency of testing by plant size – large, small, and very small 

plants) on mitigating L. monocytogenes contamination in finished RTE product, and 

reducing the subsequent risk of illness or death?; 

2) How effective are other interventions (e.g., pre- and post-packaging interventions or the 

use of growth inhibitors) in mitigating L. monocytogenes contamination in finished RTE 

product, and reducing the subsequent risk of illness or death?; and 

3) What guidance can be provided on testing and sanitization of food contact surfaces for 

Listeria species (e.g., the confidence of detecting a positive lot of RTE product given a 

positive food contact surface test result)? 

Listeria monocytogenes concentrations at retail (outputs of the FSIS Risk Assessment inplant model).’ 

Figure 20 below shows 3 quantile (i.e., the 80th, 99th, and 99.99th percentiles) concentrations of L. 

monocytogenes in deli meats at retail for the scenarios analyzed. Test and hold was used for all 

food contact surface testing and if a lot tested positive for L. monocytogenes it was assumed not 

to be sold for retail. 

Most of the scenarios are given as triplet numbers, e.g. 4-2-1, and represent the number of 

monthly food contact surface samples per line for large, small, and very small plants. 

The “60-60-60” triplet represents testing the food contact surface for every lot that is produced, 

because the model assumes that each line produces 60 lots per month. The “60-60-60 Lot” 

scenario represents testing every lot produced for L. monocytogenes, rather than a food contact 

surface for Listeria species. “PP” represents post-processing intervention/control, assuming that 

100% of the industry incorporates some form of post-processing that is 90-95% effective. The 

“GIP” represents that 100% of the industry incorporates growth inhibiting packaging or product 

reformulation that is 90-95% effective. Finally, the “PP&GIP” scenario represents a 

combination of the previous two scenarios: 100% of the industry incorporates both postprocessing and some form of growth inhibition, each of which is 90-95% effective. 

52
FSIS Listeria Risk Assessment May 26, 2003

53

Scenario

FDA

Baseline

4-2-1

8-4-2

10-10-10

16-8-4

32-16-8

40-20-10

60-60-60

60-60-60 Lot

PP

GIP

PP & GIP

Lm Concentration at Retail

(cfu/g) 1e-9

1e-8

1e-7

1e-6

1e-5

1e-4

1e-3

1e-2

1e-1

1e+0

1e+1

1e+2

1e+3

1e+4

1e+5

1e+6

1e+7

Q80 

Q99 

Q99.99 

Figure 20. Quantiles of L. monocytogenes at Retail for Various Scenarios Tested. 

The data generally show a decline in the L. monocytogenes concentration in RTE product at

retail as the food contact surface testing and sanitation effort increases. The decline is more

noticeable for the 80th and 99th percent quantiles. th percent 

quantile is more variable. te the slight drop in the 80th percent quantile from the baseline to 

the initially proposed 4-2-1 testing level. so note that testing and corresponding sanitation 

alone is not sufficient to effect a complete removal of L. monocytogenes from retail deli meats. 

Testing either every RTE lot that is produced or the food contact surface (along with 

corresponding sanitation) for every lot that is produced greatly reduces the extreme tail of the 

distribution (Q99.99) but has little impact on the 80th percent quantile. cessing 

interventions and growth inhibition (e.g., via the use of growth inhibitors/product reformulation) 

each have lower 80th percent quantiles than complete testing (i.e., testing every single lot of RTE 

product; 60-60-60 testing). ular, note the decrease in the 80th percent quantile when

post-processing and growth inhibition are combined. Reminder: that these scenarios assume that 

100% of the industry adopts such practices. 

As previously described, the 99.99

No

Al

Post-pro

In partic
FSIS Listeria Risk Assessment May 26, 2003

54

Public Health Impacts 

Figure 21 depicts estimated numbers of deaths among the elderly for the scenarios tested. 

the proposed minimal amount of food contact surface testing (i.e., the 4-2-1 scenario ; FSIS, 66 

FR 12589, February 27, 2001), the estimated median number of deaths among the elderly is 

reduced by about 20 per year. 

Scenario

Baseline

4-2-1

8-4-2

10-10-10

16-8-4

32-16-8

40-20-10

60-60-60

60-60-60 RTE

PP

GIP

PP & GIP

A

nnual deaths among elderly

M

edian Prediction

0

50

100

150

200

250

300

Figure 21. Estimated number of deaths among the elderly for the various scenarios tested.

Tables 20-23 provides the estimated retail concentration of L. monocytogenes in deli meats and

the resulting number of deaths in the U.S. population among the elderly, intermediate age, and 

neonatal populations. bination of post-processing and growth inhibitors is the only 

scenario tested where the total estimated number of deaths falls below 100 per year at the median 

of the uncertainty distribution. 

The FDA/FSIS results in Tables 20-23 include uncertainty about the retail concentration

distribution which the FSIS baseline predictions do not. 

substantial but is the result of the in-plant model being calibrated to a singular, average, 

distribution predicted by the updated version of the 2001 FDA/FSIS risk ranking model. 

For 

The com

This reduced uncertainty is not 
FSIS Listeria Risk Assessment May 26, 2003 

Table 20. Quantiles of L. monocytogenes Concentrations in Deli Meat at Retail for Scenarios Tested 

% 

FDA/FSIS 

exposure 

assessment 

Model 

FSIS 

Baseline 

Model 8-4-2 32-16-8 40-20-10 60-60-60 

60-60-60 

Lot GIP PP&GIP 

80.00 7.40E-06 1.39E-06 8.38E-07 8.68E-07 6.29E-07 7.67E-07 1.12E-07 1.22E-07 8.67E-09 85.00 3.70E-05 1.41E-05 8.98E-06 9.02E-06 6.13E-06 7.52E-06 1.18E-06 1.25E-06 9.06E-08 90.00 2.70E-04 1.81E-04 1.18E-04 1.09E-04 6.88E-05 8.34E-05 1.59E-05 1.69E-05 1.23E-06 95.00 5.50E-03 5.05E-03 3.19E-03 2.71E-03 1.35E-03 1.53E-03 5.22E-04 5.60E-04 3.93E-05 99.00 1.50E+00 2.60E+00 2.47E+00 1.98E+00 1.40E+00 5.26E-01 3.42E-01 6.10E-02 6.51E-02 2.03E-01 2.24E-01 1.56E-02 99.50 1.10E+01 1.78E+01 2.20E+01 1.70E+01 1.27E+01 4.50E+00 2.61E+00 1.47E-01 1.54E-01 1.70E+00 1.90E+00 1.32E-01 99.90 7.90E+02 8.04E+02 1.70E+03 1.24E+03 1.01E+03 4.52E+02 3.02E+02 5.04E-01 5.08E-01 1.39E+02 1.47E+02 1.08E+01 99.99 1.40E+05 2.06E+05 3.53E+05 3.31E+05 1.80E+05 7.76E+04 5.62E+04 

1.25E+0 

0 

1.31E+0 

0 2.47E+04 2.09E+04 1.67E+03 

4-2-1 16-8-4 PP 

1.15E-06 1.50E-06 2.95E-06 1.25E-05 1.57E-05 2.66E-05 1.70E-04 2.07E-04 3.06E-04 5.34E-03 6.47E-03 8.86E-03 

Table 21. Estimated Uncertainty in Annual Deaths Among the Elderly Population (> 60 years of age) for Scenarios Tested* 

Percentile 

FDA/FSIS 

doseresponse 

Model 

FSIS 

Baseline 

Model 4-2-1 32-16-8 40-20-10 60-60-60 

60-60-60 

Lot GIP PP&GIP 

5% 44 79 73 70 69 61 58 42 43 43 43 21 

50% 230 250 230 220 220 190 180 130 130 130 140 64 95% 300 290 270 260 260 230 210 150 160 160 160 76 

Average 220 210 200 200 170 170 120 120 120 120 59 

16-8-4 8-4-2 PP 

200 

Table 22. Estimated Uncertainty in Annual Deaths Among the Intermediate Age Population (> 30 days old and less than or equal to 60 years 

of age) for Scenarios Tested* 

Percentile 

FDA/FSIS 

doseresponse 

Model 

FSIS 

Baseline 

Model 4-2-1 32-16-8 40-20-10 60-60-60 

60-60-60 

Lot GIP PP&GIP 

5% 11 19 17 N/A N/A N/A 14 10 11 10 10 5 

50% 53 56 52 N/A N/A N/A 41 29 30 30 31 15 95% 65 64 60 N/A N/A N/A 47 34 35 35 36 17 

Average 51 48 N/A N/A N/A 37 27 28 28 28 13 

16-8-4 8-4-2 PP 

47 

Table 23. Estimated Uncertainty in Annual Deaths Among “Perinatal” Population (between 16 weeks before delivery and up to 30 days after birth)

for Scenarios Tested.* 

Percentile 

FDA/FSIS 

doseresponse 

Model 

FSIS 

Baseline 

Model 8-4-2 16-8-4 32-16-8 40-20-10 60-60-60 

60-60-60 

Lot GIP PP&GIP 4-2-1 PP 

55 
FSIS Listeria Risk Assessment May 26, 2003 

5% 3.7 6.4 6 N/A N/A N/A 4.7 3.3 3.4 N/A 3.5 1.7 

50% 13 14 13 N/A N/A N/A 10 7 7.3 N/A 7.6 3.5 95% 16 15 14 N/A N/A N/A 11 8 8.3 N/A 8.5 4 

Average 12 13 12 N/A N/A N/A 9.3 6.6 6.8 N/A 7.1 3.3 

*Baseline model calibrated to 310 deaths per year among the elderly, 67 intermediate age deaths per year, and 16 neonatal/newborn 

deaths per year in the U.S. population. 

56 
FSIS Listeria Risk Assessment May 26, 2003 

Table 24 summarizes the predicted median lives saved per year for each of the age groups for the 

difference testing and pre and post packaging interventions analyzed. 

Table 24. Summary of predicted median lives saved relative to baseline 

Scenario Elderly Intermediate Neonates/Newborns Total 

4-2-1 20 4 1 25 

8-4-2 30 NA NA ≥30 

10-10-10 40 NA NA ≥40 

16-8-4 30 NA NA ≥30 

32-16-8 60 NA NA ≥60 

40-20-10 70 15 4 89 

60-60-60 120 27 7 154 

60-60-60 RTE 120 26 7 153 

PP-95% 120 26 NA ≥146 

PP-99% 173 39 10 221 

GIP 110 25 NA ≥135 

PP-95% & GIP 186 41 11 238 

NA – not available. 

Based on a monotonic Kendall tau statistical test for trend, the increase in the number of lives 

saved with increasing frequency of testing is statistically significant at the 99% significance 

level. (tau=0.88, p=0.0028). 

Lot and Food Contact Surface Prevalence: Likelihood of Detection 

Table 25 illustrates the contingency results of a sample run of 1,000,000 lots tested with 60 food 

contact surface tests per month and 60 lot tests per month, i.e. all possible tests of both the food 

contact surface and the product was conducted. Test and hold was used, but no other 

interventions were implemented. 

Table 25. RTE Product Lot and Food Contact Surface Prevalences 

Lot positive Lot negative Sum 

FCS positive 

FCS negative 

21635 

8 

115940 

862417 

137575 

862425 

Sum 21643 978357 1000000 

This implies an overall RTE product lot prevalence for L. monocytogenes is 21643/1000000 or 

approximately 2.2%. The food contact surface prevalence for Listeria species is 

137575/1000000 or approximately 13.7%. The lot prevalence when the food contact surface is 

positive is 21635/137575 or approximately 15.7%. Thus, knowing that the food contact surface 

is positive increases the likelihood of finding a positive lot by a factor of 7. 

Test and Hold Effectiveness 

Table 26 below provides data for evaluating the effectiveness of test and hold at various testing 

frequency. Figure 22 provides a graphical comparison. Clearly, there is only a small impact at 

57
FSIS Listeria Risk Assessment May 26, 2003 

lower testing frequencies such as 4-2-1. At higher testing frequencies, test and hold greatly 

reduces the concentrations at retail. 

Table 26. Effectiveness of Test and Hold of RTE Product Lot 

Description Q80 Q85 Q90 Q95 Q99 Q99.5 Q99.9 Q99.99 

4-2-1 1.50E-06 1.57E-05 2.07E-04 6.47E-03 2.47E+00 2.20E+01 1.70E+03 3.53E+05 

4-2-1 no test and hold 1.40E-06 1.50E-05 2.04E-04 6.63E-03 2.74E+00 2.28E+01 1.92E+03 4.04E+05 

8-4-2 1.15E-06 1.25E-05 1.70E-04 5.34E-03 1.98E+00 1.70E+01 1.24E+03 3.31E+05 

8-4-2 no test and hold 1.21E-06 1.32E-05 1.82E-04 6.00E-03 2.31E+00 1.90E+01 1.80E+03 3.18E+05 

16-8-4 1.39E-06 1.41E-05 1.81E-04 5.05E-03 1.40E+00 1.27E+01 1.01E+03 1.80E+05 

16-8-4 no test and hold 2.04E-06 1.97E-05 2.41E-04 7.03E-03 2.54E+00 2.12E+01 1.76E+03 2.42E+05 

32-16-8 8.38E-07 8.98E-06 1.18E-04 3.19E-03 5.26E-01 4.50E+00 4.52E+02 7.76E+04 

32-16-8 no test and hold 1.07E-06 1.15E-05 1.59E-04 4.88E-03 1.75E+00 1.51E+01 1.31E+03 2.69E+05 

60-60-60 6.29E-07 6.13E-06 6.88E-05 1.35E-03 6.10E-02 1.47E-01 5.04E-01 1.25E+00 

60-60-60 no test and hold 1.28E-06 1.24E-05 1.53E-04 4.11E-03 9.62E-01 8.74E+00 8.02E+02 1.29E+05 

Q80 

Q99 

Q99.99 

1e+7 

1e+6 

1e+5 

1e+4 

LM C

oncentration at R

etail 

(cf

u/g) 1e+3 

1e+2 

1e+1 

1e+0 

1e-1 

1e-2 

1e-3 

1e-4 

1e-5 

1e-6 

1e-7 

H&

- 2

H 

- 8

H 

2 N - 4

H 

61 - H 

4 N - 8

H&

- 2

H 

8 N - H 

06

H 

0 N 6

H

o T& 

2 T& 

o T& 4 T& 

8 T& -

o T& 

6 

0 T& 

o T&

1 T 

o T

-2- 4

1 N 

4-

8 -

6

-6

3

1 

6 

- 0

-

- 4

-

61 -

- 23

8 

1

0 - 06 6

Scenario 

Figure 22. Comparison of Test and Hold Effectiveness for Difference Testing Frequencies. 

58
FSIS Listeria Risk Assessment May 26, 2003 

This changing impact can be best illustrated in Table 27, which shows the comparison of the 

percentage of food contact surface positives and the lot positives for 2 sampling frequencies with 

and without test and hold. 

Table 27: Example comparison of % food contact surface positives and lot positives under 

ifferent test and hold scenarios 

FCS Test and FCS FCS Lot Tests Lot % FCS % Lot 

Sample Hold? Tests Positives Positives Positives Positives 

Frequency 

4 Yes 66667 9171 9171 1432 13.8 15.6 

4 No 66666 9442 9442 422 14.2 4.5 

60 Yes 1000000 132914 132914 20560 13.3 15.5 

60 No 1000000 131867 131867 5268 13.2 4.0 

d

The percentage of food contact surface positives is approximately constant at about 13-14% 

regardless of the test and hold option. The percent of positive lots varies significantly depending 

on whether or not test and hold is implemented. When test and hold is implemented, positive 

lots occur approximately 15-16% of the time. When test and hold is not implemented, the lot 

percentage drops to 4-5 %. This decrease is caused by not being able to sample the lot during a 

period of known food contact surface contamination. The 3 day lag before a lot test is conducted 

greatly reduces the probability of finding a contaminated lot. These prevalence levels can also 

be compared to the overall lot prevalence described earlier, which was about 2.2%. The 4% 

prevalence when test and hold is not implemented is still almost twice what the overall lot 

prevalence is. In other words, knowing that the food contact surface was positive 3 days prior 

doubles the likelihood of finding a positive lot. 

With test and hold enabled, for the smaller testing frequency, only 1432/1000000 lots (0.14%) 

tested positive and were removed from the food supply. For the more frequent testing, 

20560/1000000 lots (2%) tested positive and were removed. The higher percentage removal 

leads to lower values for the given percentiles at retail. 

SENSITIVITY ANALYSIS 

A sensitivity analysis involves varying parameter inputs and assumptions to determine how they 

affect the estimated risk of illness. A preliminary sensitivity analysis of the FSIS Listeria risk 

assessment model has been conducted and the initial results are presented below. 

Figure 23 evaluates the model results for a variety of pre and post packaging intervention level. 

The L. monocytogenes concentrations in deli meat at retail for different industry participation and 

intervention effectiveness are graphed. As expected, the retail concentrations decrease as both 

participation and effectiveness increase. 

59
FSIS Listeria Risk Assessment May 26, 2003

60

Post processing (Industry participation, effectiveness)

LM concentrati

on at ret

ail (cf

u/g) 10-9

10-8

10-7

10-6

10-5

10-4

10-3

10-2

10-1

100

101

102

103

104

105

106

80.00%

99.00%

99.99%

70-75% Effective 90-95% Effective 99% Effective

50% 75% 90% 100% 50% 75% 90% 100% 50% 75% % Industry

Participarting

No FCS testing, no lot testing

Figure 23. Sensitivity to Pre and Post Packaging Interventions. 

Figure 24 presents the changes in retail L. monocytogenes concentrations for different sample

masses used for RTE product lot testing. ncentrations decrease over all the sample masses 

tested, and the percent of positive lots increases. The change in the lot prevalence emphasizes 

that prevalence data is tied to detection limits. 

In practice, 25 grams is consistently used for the sample mass, and the largest sample mass that 

can easily be used is about 100 grams. ples, at greater cost, would have to be 

analyzed to achieve the same effect as the larger RTE product lot sample masses modeled. 

90% 100%

The co

Multiple sam
FSIS Listeria Risk Assessment May 26, 2003 

60-60-60 Lot testing, test and hold, dispose product 

101 

100 

10-1 

10-2 

10-3 

10-4 

10-5 

10-6 

10-7 

5 

4 

3 

2 

1 

0 

25 50 100 200 400 800 

Q80 

Q99 

Q99.99 

RTE Sample Mass (grams) 

Figure 24. Sensitivity to RTE mass sampled. 

Figures 25 and 26 show the impacts of varying the surface area swabbed during food contact 

surface testing. The retail concentrations initially decrease as larger areas are swabbed, but this 

effect levels off when 100-1000 cm2

 are sampled. Larger areas do not provide additional 

benefits. This is confirmed in Figure 26. The total number of positive lots found reaches its 

maximum when about 100 cm2

 is sampled, at about 2% of all the lots produced. This is the same 

as the overall lot prevalence. In other words, this area is sufficient to identify all the positive lots 

that are present. Sampling larger areas increases the percentage of food contact surface 

positives, but does not change the number or percentage of positive lots. Percent Lots Positi

v

e (

%) 

LM C

oncentr

ation at R

etail (

cfu/g) 

61
FSIS Listeria Risk Assessment May 26, 2003 

It is important to keep in mind that these conclusions are based on the assumption that Listeria 

species contamination is uniformly spread across the entire food contact surface. In practice, 

there is likely to be spatial variability, which might change the results. 

60-60-60 FCS Testing, enhanced cleaning, test lot, dispose lot 

103 

102 

101 

100 

10-1 

10-2 

10-3 

10-4 

10-5 

10-6 

10-7 

0.01 0.1 1 10 100 1000 10000 

Q80 

Q99 

Q99.99 

Sampled Surface Area (cm2) 

Figure 25. Retail L. monocytogenes concentrations in deli meats for different food contact 

surface area tested. Lm C

oncentration at R

etail (

cfu/g) 

62
FSIS Listeria Risk Assessment May 26, 2003

63

N

o. F

CS Positives =

N

o. Lots T

e

ste

d

No. Lot Positiv

e

s

0.0

2.0e+4

4.0e+4

6.0e+4

8.0e+4

1.0e+5

1.2e+5

1.4e+5

1.6e+5

1.8e+5

FCS Positives = Lots Tested

Lot Fails

0.01 0.1 1 10 100 1000 10000

P

ercent 

F

CS Positiv

es (%)

0

2

4

6

8

10

12

14

16

FCS Surface Area Tested (cm2)

0.01 0.1 1 10 100 1000 10000

Per

cent 

of Tested Lots Positive (%)

0

20

40

60

80

60-60-60 FCS Testing, enhanced cleaning, test lot, dispose lot

Figure 26. Sensitivity of positive RTE product lots and food contact surface area found to be 

positive based on the area of food contact surface tested. 
FSIS Listeria Risk Assessment May 26, 2003 

L monocytogenes to Listeria species ratio 

A very preliminary evaluation of the FSIS risk assessment model results to changes in the L. 

monocytogenes to Listeria species ratio is presented in Table 28. 

Table 28. Evaluation of the concentration of Listeria species added to food contact surface and 

the prevalence of Listeria species on food contact surface or L. monocytogenes in RTE product 

lots as a function of different L. monocytogenes (Lm)/ Listeria species (Listeria species) ratios 

Parameter Low Ratio Baseline High Ratio 

Mean Lm/Listeria species ratio 0.052 0.52 0.95 

Std dev Lm/Listeria species ratio 

Mean Listeria species/cm2

 added during 

0.026 0.26 0.026 

contamination event (log scale) 

Std dev Listeria species/cm2

 added 

-5 -6 -6.4 

3.5 3.5 3.5 

overall lot prevalence (%) 2.2 2.2 2.0 

overall FCS prevalence (%) 18.7 13.8 12.0 

contingent lot prevalence when FCS is positive 

(%) 

11.7 15.7 17.0 

Improvement 5.3 7.1 8.5 

Each column in the table requires a separate calibration of the level of Listeria species added to 

the food contact surface during a contamination event, and except for the baseline, the results are 

from initial calibrations only. 

The overall lot prevalence, whether the mean ratio is 5%, 52%, or 95% is relatively constant at 

about 2%. This is consistent with the fact that all 3 simulations need to meet the same observed 

prevalence of L. monocytogenes at retail. The food contact surface prevalence changes however, 

with higher prevalences found for lower ratios. This result is because lower ratios require more 

Listeria species added to the food contact surface to match observed Lm concentrations. A ratio 

of 5% implies that approximately 10 times as many Listeria species are added to the contact 

surfaces compared to the baseline case. The contingent lot prevalence, i.e. the prevalence of 

positive lots when the food contact surface is positive increases as the ratio increases. As more 

of the organisms on the food contact surface are Lm, a positive food contact surface is more 

indicative of a positive lot. The improvement over the baseline lot prevalence (i.e. the ratio of 

contingent lot prevalence to overall lot prevalence) also increases as the ratio increases. At very 

low ratios, lot testing is 5 times more likely to find a positive lot if the food contact surface was 

positive. At very high ratios, lot testing is 8.5 times more likely to find positive lots. 

The baseline ratio is based on prevalence data, not actual concentration data. The model has 

simply made this assumption in the lack of any better data. A concentration ratio of 5% is 

possible, however a concentration ratio of 95% seems unlikely when almost half of samples 

collected contain only Listeria species other than Lm. 

The efficacy of food contact surface increases with higher ratios. However, even at very low 

ratios there is still a marked improvement achieved in sampling efficiency by knowing the results 

of the food contact surface test. 

64
FSIS Listeria Risk Assessment May 26, 2003 

Uncertainty versus Variability 

The in-plant model only considers variability, while the FDA/FSIS model considers both 

uncertainty and variability. Figure 27 depicts the 5th, 50th, and 95th percentile values for annual 

deaths in the elderly population are represented for four scenarios. These ranges reflect the 

uncertainty about the true number of deaths per annum. The first scenario is the prediction using 

the FDA/FSIS risk ranking model without any modification. For this scenario the 5th and 95th 

percentile values are about 50 and 300, respectively. The second scenario replaces the 

uncertainty about the concentration of L. monocytogenes per gram at retail in the FDA/FSIS risk 

ranking model with a single distribution that only describes variability. This variability 

distribution was calculated as the average distribution among 300 uncertain choices. For this 

scenario, the 5th and 95th percentiles are about 75 and 290, respectively. Therefore, removing the 

uncertainty about the concentration of L. monocytogenes per gram at retail has slightly reduced 

the uncertainty implied by the model. It has also increased the median value from about 230 

deaths per annum to about 250 deaths per annum. The third and fourth scenarios use the in-plant 

model predictions for the variability in concentration of L. monocytogenes per gram at retail. 

The variability distributions for L. monocytogenes concentration per gram predicted by the inplant model were calibrated to the average distribution calculated from the FDA/FSIS risk 

ranking model. These final two scenarios suggest that the predicted uncertainty in deaths per 

annum is not affected by the choice of a particular baseline from the in-plant model. 

Although the baseline median value changes from 230 to 250 by not including uncertainty in the 

L. monocytogenes concentration per gram at retail, this effect is not substantial. The primary 

quantitative output of the risk assessment is the predicted deaths averted by interventions relative 

to the baseline. This marginal effect should be equivalent for baseline median of 230 or 250. 

D

e

aths i

n elderly per a

n

n

um 

Lm. Concentration 350 Lm. Concentration In-plant model output uncertainty NOT uncertainty included calibrated to FDA/FSIS included 300 model 

250 

200 

150 

100 

50 

0 

Original FDA/FSIS Baseline 1 Baseline 2 

FDA/FSIS modified 1 

Scenario 

Figure 27. Per Annum Deaths Among the Elderly – A Comparison of FDA/FSIS Model 

Estimates with the FSIS Listeria Risk Assessment Baseline. 

Model Validation 

65
FSIS Listeria Risk Assessment May 26, 2003 

Although the data are not available to formally validate the model, the prevalence under the base 

model run was compared to preliminary USDA surveillance data. Prevalence was not used as 

part of the calibration process. Based on HAACP code 03G, which represents fully cooked, not 

shelf stable product that is sliced, diced or shredded, 23 out of 997 samples were positive for L. 

monocytogenes. This represents a prevalence of 2.3%. The base model’s prevalence for L. 

monocytogenes in deli meats was 2.2%. 

Several caveats apply to this comparison. The product categories do not overlap exactly. The 

O3F category includes products like diced chicken that would not be considered a RTE deli 

meat. The USDA values are still undergoing QA/QC and can only be considered preliminary. 

The Gombas et al. study (2003) found a lower average prevalence in deli meats of 0.9% then the 

USDA surveillance. Finally, the agreement between simulated and measured prevalence may be 

more of an indication that the upper tail of the FDA retail distribution , which the risk assessment 

match well during calibration, agrees with the observed USDA prevalence. Nonetheless, the 

agreement is supportive of the risk assessment model. 

SUMMARY 

• Food contact surfaces found to be positive for Listeria species greatly increased the 

likelihood of finding RTE product lots positive for L. monocytogenes. 

• Frequency of contamination of food contact surfaces with Listeria species encompasses a 

broad timeframe, and the duration of a contamination event lasts approximately a week. 

• The proposed minimal frequency of testing and sanitation of food contact surfaces, as 

presented in the proposed rule (66 FR 12569, February 27, 2001), is estimated to result in 

a small reduction in the levels of L. monocytogenes on deli meats at retail 

• Increased frequency of food contact surface testing and sanitation is estimated to lead to a 

proportionally lower risk of listeriosis. 

• Combinations of interventions (e.g., testing and sanitation of food contact surfaces, preand post-packaging interventions, and the use of growth inhibitors/product reformulation) 

appear to be much more effective than any single intervention in mitigating the potential 

contamination of RTE product with L. monocytogenes and reducing the subsequent risk 

of illness or death. 

• The FSIS Listeria risk assessment clearly provides information important for comparing 

the relative effectiveness of interventions (e.g., testing and sanitation, post-lethality 

interventions, use of growth inhibitors, and combinations of these interventions; see 

Tables 10-14). 

66
FSIS Listeria Risk Assessment May 26, 2003 

REFERENCES 

Bedie GK, Samelis J, Sofos JN, Belk KE, Scanga JA, Smith GC (2001). “Antimicrobials in the 

formulation to control Listeria monocytogenes postprocessing contamination on frankfurters 

stored at 4 degrees C in vacuum packages.” J Food Prot 64(12):1949-55 

Brown, MH, Gill CO, et al. (2000). The role of microbiological testing in systems for assuring 

the safety of beef. International Journal of Food Microbiology 62: 7-16. 

Casman, EA, Morgan MG, Dowlatabadi H (1999). Mixed Levels of Uncertainty in Complex 

Policy Models. Risk Analysis 19:33-42 

Centers for Disease Control and Prevention, CDC (December 22, 2000). Multistate Outbreak of 

Listeriosis --- United States, 2000. MMWR Weekly 49(50):129-1130. 

Chae, MS, Schraft H (2000). Comparative evaluation of adhesion and biofilm formation of 

different Listeria monocytogenes strains. International Journal of Food Microbiology 62: 103-

111. 

Chen Y, Jackson KM, Chea FP, Schaffner DW (2001). Quantification and variability analysis of 

bacteria cross-contamination rates in common food service tasks. J Food Prot 64(1):72-80. 

Cornell University (November 2002). Dr. M. Wiedmann provided blinded industry data on the 

ratio of Listeria spp. to Listeria monocytogenes. 

den Aantrekker, ED, Boom RM, et al. (2002). Quantifying recontamination through factory 

environments - a review. International Journal of Food Microbiology 80: 117-130. 

Doyle, M. (December 2002). Environmental Testing and Sanitation for Listeria. Presentation 

by Kraft Foods and the Center for Food Safety,University of Georgia to USDA, FSIS, 

Washington, DC. December 15, 2002. 

FAO/WHO (2001). Hazard identification, exposure assessment and hazard characterization of 

Vibrio spp. in seafood. Preliminary document. MRA 01/03/04. (Available at: 

ftp://ftp.fao.org/es/esn/food/vibrio.pdf.) 

FAO/WHO (2002). Risk assessments of Salmonella in eggs and broiler chickens. 

Microbiological Risk Assessment Series 2. (Available at: ftp://ftp.fao.org/es/esn/food/RA 

_Salmonella _report.pdf) 

Food and Drug Administration, Food Safety and Inspection Service (FDA/FSIS, January 2001). 

Draft Relative Risk to Public Health form Foodborne Listeria Monocytogenes Among Selected 

Categories of Ready-to-Eat Foods. FDA Docket No. 99N-1168 and FSIS Docket No. 00-048N. 

Available at Website http://www.foodsafety.gov/~dms/lmrisk.html.25 

25 Note: This relative risk ranking has been received public review and comment. As a result, the model has been 

updated with additional data supplied by industry and other stakeholders. The FSIS Listeria risk assessment used 

components of the updated version of the 2001 FDA/FSIS risk ranking of RTE foods (i.e., those pertaining to deli 

meats and hot dogs/frankfurters). 

67
FSIS Listeria Risk Assessment May 26, 2003 

Food Safety and Inspection Service (February 27, 2001). Performance Standards for the 

Production of Processed Meat and Poultry Products. Proposed Rule. 66 Federal Register 

39:12590-12636. 

Food Safety and Inspection Service, Office of Policy, Planning, and Employee Development 

November 2002). Dr. Amelia Sharar provided estimates of food contact surface area and the 

sampling area for these surfaces. November 5, 2002, personal communication. 

Food Safety and Inspection Service, Office of Policy, Planning, and Employee Development 

(December 2002). Dr. Felix Spinelli and Mr. Michael Matthew provided information on postprocessing interventions, including use of high pressure and irradiation. Personal 

communication, December 2002. 

Food Safety and Inspection Service (January 2003). FSIS Survey of Establishments Producing 

Ready-To-Eat Meat and Poultry Products. Survey of FSIS inspectors in establishments 

producing RTE deli meats and hot dogs/frankfurters/wieners. FSIS survey design, 

implementation, and analysis from October 2002-January 2003. Analyzed data supplied by Mr. 

Richard Cope, Office of Policy, Planning, and Employee Development, Food Safety and 

Inspection Service. 

Frey, HC (1999). Chairperson’s Summary. Report of the Workshop on Selecting Input 

Distributions For Probabilistic Assessments, New York, NY, April 21-22, 1998. EPA/630/R98/004. U.S. Environmental Protection Agency, Risk Assessment Forum: Washington, DC. 

Gibson H, Taylor JH, Hall KE, Holah JT (1999). Effectiveness of cleaning techniques used in 

the food industry in terms of the removal of bacteria biofilms. J. Applied Microbiology 87:41-

48. 

Gombas DE, Chen Y, Clavero RS, Scott VN (2003). Survey of Listeria monocytogenes in 

Ready-to-Eat Foods. Journal of Food Protection: Vol. 66, No. 4, pp. 559-569. April, 2003 

Hayes, PS, et al. (1992) Comparison of Three Selective Enrichment Methods for the Isolation of 

Listeria monocytogenes from Naturally Contaminated Foods. J. Food Protection 55(12): 952-

959. 

Hintz, J (2001). NCSS and Pass. Number Cruncher Statistical Systems. Kaysville, UT. 

Hynes N (2000). Draft: Multistate Listeriosis Outbreak – Findings of Epidemioloigcal In-Plant 

Investigation; Waco, TX. Letter submitted to the Record. Unpublished Report. December 18, 

2000. 

Kozempel M, Goldberg N, Radewonuk ER, Scullen OJ, Craig JC Jr. (2000). Rapid hot dog 

surface pasteurization using cycles of vacuum and steam to kill Listeria innocua. Journal of 

Food Protection 63(4):457-61. 

Law AM, Kelton WD (1991). Simulation Modeling and Analysis. McGraw-Hill, New York. 

68
FSIS Listeria Risk Assessment May 26, 2003 

Lennon D, Lewis B, Mantell C, et al. (1984). Epidemic perinatal listeriosis. Pediatr Infect Dis 

3(1):30-34. 

Levine P, Rose B, Green S, Ransom G, Hill W (2001). Pathogen testing of ready-to-eat meat and 

poultry products collected at federally inspected establishments in the United States, 1990 to 

1999.” Journal of Food Protection 64(8):188-1193. 

Mead PS, Slutsker L, Dietz V, McCraig LF, Bresee S, Shapiro C, Griffin P, Tauxe RV (1999a). 

“Food-related illness and death in the United States.” Emerging Infectious Diseases 5:607-625. 

Mead P (1999b). Multistate outbreak of listeriosis traced to processed meats, August 1998-

March 1999. May 27, 1999, CDC unpublished report. 

Midelet G, Carpentier B (2002). Transfer of microorganisms, including Listeria monocytogenes, 

from various materials to beef. Applied and Environmental Microbiology 68(8):4015-4024. 

Montville R, Chen Y, Schaffner DW (2001). Glove barriers to bacterial cross-contamination 

between hands to food. J Food Prot 64(6):845-849. 

Murphy RY, Berrang ME (2002). Thermal lethality of Salmonella senftenberg and Listeria 

innocua on fully cooked and vacuum packaged chicken breast strips during hot water 

pasteurization. J Food Prot.65(10):1561-4. 

National Academy of Sciences (2002). Escherichia coli O157:H7 in Ground Beef: Review of a 

Draft Risk Assessment. National Academy Press: Washington, DC. 

National Advisory Committee for Microbiological Criteria in Foods (NACMCF) (1991). 

Listeria monocytogenes: Recommendations by the National Advisory Committee on 

Microbiological Criteria for Foods. International Journal of Food Microbiology 14:185-246. 

Nestle, M. (2003). “Not good enough to eat”. New Scientist. 177: 25. 

ORACBA (USDA Office of Risk Assessment and Cost Benefit Analysis). 2003. Technical 

Comments on FSIS February 26, 2003 Draft Risk Assessment for Listeria in Ready-to-Eat Meat 

and Poultry Products. 

Press WH, Teukolsky SA, Vetterling WT, Flannery BP (1992). Numerical Recipes in C, 2nd 

edition, Cambridge University Press. 

Ryser ET (1999). Foodborne Listeriosis. In: Listeria, Listeriosis and Food Safety. Ryser ET, 

Marth EH, eds. Food Science and Technology, New York:Marcell Dekker Inc., pp. 75-95. 

Salvat G, Toquin MT, et al. (1995). Control of Listeria monocytogenes in the delicatessen 

industries: the lessons of a listeriosis outbreak in France. International Journal of Food 

Microbiology 25: 75-81. 

69
FSIS Listeria Risk Assessment May 26, 2003 

Seman et al. (2002). Modeling the growth of L. monocytogenes in cured ready-to-eat processed 

meat products by manipulation of sodium chloride, sodium diacetate, potassium lactate and 

product moisture content. J. Food Protection 65: 651-658. 

Slutsker L, Schuchat A (1999). Listeriosis in humans. In: Listeria, Listeriosis and Food Safety. 

Ryser ET, Marth EH, eds. Food Science and Technology, New York:Marcell Dekker Inc., pp. 

75-95. 

Small M (2000). Using Disparate Data in Exposure Assessment. Advanced Methods for DoseResponse Assessment: Bayesian Approaches. Resources for the Future: Washington, DC. 

Sommers C, Kozempel M, Fan X, Radewonuk ER (2002). Use of vacuum-steam-vacuum and 

ionizing radiation to eliminate Listeria innocua from ham. Journal of Food Protection 

65(12):1981-3. 

Souef P, Le N, Walters BNJ (1981). Neonatal listeriosis--a summer outbreak. Med J Austral 

2:188-191. 

Sugarman C (2003). Jack in the Box's Dave Theno takes a 10-year look back. Food Chemical 

News 44(50): 10. 

Swanson KMJ, Anderson JE (2000). Industry perspectives on the use of microbial data for 

hazard analysis and critical control point verification. Journal of Food Protection 63(6): 815-818. 

Tompkin RB (2002). Control of Listeria monocytogenes in the food-processing environment. 

Journal of Food Protection 65(4):709-25. Review. 

Wilson B (November 20, 2002). Estimates for production volume by plant size. National 

Turkey Federation. Personal communication, November 20, 2002. 

70
FSIS Listeria Risk Assessment May 26, 2003 

Appendix A. Revisions to the 2001 FDA/FSIS Risk Ranking Model 

The exposure assessments for deli meats and hot dogs and the dose-response relationship of the

January 2001 draft FDA/FSIS risk ranking model (see 

http://www.foodsafety.gov/~dms/lmrisk.html) was updated in response to public comments and 

the availability of additional data. Below is a list of the changes made to the exposure 

assessments for deli meats and hot dogs and the dose-response relationship. The updated 

FDA/FSIS exposure assessment for deli meats and updated dose-response relationship was used 

in the FSIS Listeria risk assessment. 

Food Category Changes 

3⁄4 Split frankfurters into two categories: not reheated and reheated. 

Contamination Data Changes 

3⁄4 Additional contamination data for deli meats from published studies (see the table on p. 

48). 

3⁄4 New contamination data was incorporated. This included: updated FSIS data (meats and 

meat products; included in Docket 03-005N), and the NFPA L. monocytogenes retail data 

for deli meats (also included in Docket 03-005N). 

3⁄4 Percent hot dogs eaten uncooked was modeled using a triangle distribution (4, 7, 10) 

based in part on information provided in the America Meat Institute (AMI) survey. The 

AMI data has been submitted to the Listeria docket (Docket 03-005N). 

Growth Data Changes 

3⁄4 Frankfurters that are frozen before consumption were considered by excluding growth of 

L. monocytogenese during consumer handling for this portion of the frankfurters. A 

uniform distribution (3, 8.7) was used based information provided in the AMI survey and 

the FDA Food Safety Survey. 

3⁄4 The storage temperature distribution applicable to deli meats is shown below. This data 

was developed from Audits International surveys (see: 

http://www.foodriskclearinghouse.umd.edu/pversion/Audits-FDA_temp_study.htm). 

Post-retail Storage Duration Changes 

71

Fraction of delimeat 

servings

30.00% 

25.00% 

20.00% 

15.00% 

10.00% 

5.00% 

0.00% 

8

20 

24 

2

32 

36 

40 

44

48 

52 

56 

60

64 

68 

Temperature (F) 
FSIS Listeria Risk Assessment May 26, 2003 

3⁄4 Frankfurter and deli meats food categories. A survey sponsored by AMI provided data 

allowing the use of a semi-empirical distribution. Inter-household variation was based on 

the AMI data (they asked average storage time). These results are shown below (also 

included in Docket 03-005N). 

1.00 

0.80 

0.60 

0.40 

0.20 

0.00 

0 10 20 30 40 50 

Average storage time for delimeats (days) 

A log normal distribution was applied at the empirical data points to introduce intrahousehold variation. The magnitude of the intra-household variation, expressed as the 

Geometric Standard Deviation (GSD) ranged from 0.4 to 0.6 to be consistent with the 

'last storage time' data from the FSIS hotline study. 

Changes to Modeling L. monocytogenes Levels in Food at Retail 

3⁄4 The models were fit to log dose (log cfu) instead of dose (cfu). A normal distribution 

was used exclusively; a range of parameters was used to represent the uncertainty. 

3⁄4 The algorithm used to calculate percentiles by ParamFit (used to develop the Log-Growth 

models) is (x-0.5)/n instead of (x-l)/(n-1). 

3⁄4 Quantitative modeling of Listeria distributions was applied to individual studies. Only 

studies with 10 or more enumerated samples were modeled. Group-specific 

generalizations about the shape of the L. monocytogenes concentration distributions (i.e. 

the geometric standard deviation with an uncertain range) were based on these analyses. 

The NFPA survey data (see Listeria Docket 03-05N) were used for deli meats. These 

results are summarized below. 

Cumulative frequency 

72
FSIS Listeria Risk Assessment May 26, 2003 

1.000 

0.998 

0.996 

0.994 

0.992 

0.990 

0.988 

0.986 

0.01 0.1 1 10 100 1000 10000 

Concentration of Lm per gram of delimeats 

Gombas, 2001 (CA) Gombas, 2001 (MD) 

3⁄4 The geometric means used to produce an estimate were based on the prevalence value 

from a randomly selected individual study and a randomly selected geometric standard 

deviation. The probability interval assigned to each study was proportional to its weight, 

which was a function of the number of observations, the date of the study, and the 

geographic area of the study. 

Prevalence data used for deli meats are summarized below. Cu

mulati

ve frequency 

REFERENCE Country Total samples % Positive 

Aguado et al., 2001 Spain 369 9.2% 

Baek et al., 2000 Korea 50 0.0% 

Bersot et al, 2001 Brazil 30 26.7% 

Daley et al., 1999 Canada 19 5.3% 

Gillespie et al., 2000 UK 3455 0.4% 

Gombas, 2001 NFPA-CA USA 4600 0.6% 

Gombas, 2001 NFPA-MD USA 4599 1.2% 

Gomez-Campillo et al., 1999 Spain 20 0.0% 

Kamat and Nair, 1994 India 2 0.0% 

Lahellec et al., 1996 France 45 2.2% 

Levine, 2000 USA 9864 2.3% 

Levine, 2001 USA 9037 1.9% 

Miettinen, M., et al., 2001 Finland 25 0.0% 

Ng and Seah, 1995 Singapore 17 17.6% 

Ojeniyi, et al 2000 Denmark 55 7.3% 

Oregon State Dept of Agriculture, 2001 USA 451 1.1% 

Qvist and Liberski, 1991 240 14.2% 

Samelis and Metaxopoulos, 1999 Greece 52 5.8% 

Soriano et al.,2001 Spain 15 0.0% 

Uyttendaele et al., 1999 Belgium 879 7.1% 

3⁄4 Data from geographic areas outside the United States were weighted to predict L. 

monocytogenes concentrations in foods in the United States. Group 1: North America 

73 
FSIS Listeria Risk Assessment May 26, 2003 

including US, Canada and Mexico; EU countries, Japan; Australia and New Zealand. 

Data from other countries will also be included in group 1 if they an important source for 

the food in the study. Weight =1. Group 2: All remaining data. Weight = 0.3. The 

decision of whether a country was an important import source depended on the level of 

imported product and the level of US consumption of the product. This decision was 

made on a case-by-case basis for each food category but general criteria for identifying 

an important import source is at least 1000 MT or $1 million/year. 

3⁄4 Data from older studies was weighted. A step-wise weighting was used for three time 

periods: pre-1993 to 1993, 1994 to 1998, and 1999 to current. The weighting for the 

step-wise approach will be 0.4, 0.7, and 1.0, respectively. 

3⁄4 Analogies about L. monocytogenes distribution shape was drawn from one food category 

to another, if there are no significant differences in distribution shapes among foods. 

3⁄4 The impact of truncating the contamination distribution prior to the growth model at the 

low (cold) end of the maximum growth values (i.e., at approximately 105

) was evaluated. 

Changes to Dose-response Modeling 

3⁄4 Instead of targeting the median value that is the result of multiple simulations, the doseresponse adjustment factor was individually generated for each of the uncertainty 

iterations. 

3⁄4 The hospitalization /mortality ratios were calculated separately for each population 

group. 

General Model Change 

Although the model still uses Excel worksheet functions (e.g., statistical distribution 

functions, data indexing functions), it has been completely rewritten in Visual Basic for 

Applications (VBA). 

74
FSIS Listeria Risk Assessment May 26, 2003 

Appendix B. Predicted growth between processing and retail 

In the 2001 FDA/FSIS risk-ranking model exposure assessment for deli meats, prevalence data 

from processing plants were adjusted to account for growth in L. monocytogenes between the 

processing plant and the retail outlet. Based on simulated growth predictions, an adjustment of 

1.9 logs (a multiplier of roughly 79) was assumed. 

The available sampling evidence at processing and retail creates some confusion as to what is 

actually occurring between these two points in time and space. For example, FSIS reports a 

prevalence of 1%-3% L. monocytogenes-positive 25 gram samples at deli meat processors. In 

contrast, a large survey of deli meats at retail completed by NFPA found 0.9% of 25 gram 

samples positive for L. monocytogenes. Because the sampling and culturing methods were the 

same for both surveys, these results suggest that fewer servings are contaminated at retail than at 

processing. Seemingly, instead of growth making the problem worse between processing and 

retail, these data imply that the situation is better at retail than processing. This conclusion, 

however, is highly counterintuitive. Given the capacity of L. monocytogenes to survive and 

grow even at low temperatures, it is difficult to argue that there is no growth, or a reduction, in 

the numbers of L. monocytogenes in servings between processing and retail. As the 2001 

FDA/FSIS risk ranking model predicts, this amount of growth is predicted to be, on average, 1.9 

logs. 

The FDA/FSIS exposure assessment for deli meats used both the FSIS and NFPA data in 

estimating the distribution for concentration of L. monocytogenes at retail. The conflicting 

effects of these data, however, are subsumed in the uncertainty about this distribution. This 

uncertainty is ignored in calibrating the in-plant model and, therefore, the effect of growth is 

more explicit for the in-plant model. This creates a problem that must be addressed. 

To illustrate the problem, a series of three examples are presented. These examples are based on 

the following assumptions. 

The log concentration of L. monocytogenes at retail is the sum of the log concentration at 

processing and the log of growth. 

(1.1)RetailLog(Lm per gram) = ProcessingLog(Lm per gram) +GrowthLog(Growth multiplier) 

The retail concentration distribution is assumed in the FDA/FSIS risk ranking to be a 

lognormal. Therefore, the log of concentration is normally distributed. The logs of the 

processing and growth distributions are also assumed to be normal distributions for these 

examples. Consequently, the following equation results. 

(1.2) Normalretail(μ1 + μ2 , 2 

1 σ 2 

2 σ + ) = Normalprocess (μ1,σ 1) + Normalgrowth (μ2 ,σ 2 ) 

The FDA/FSIS exposure assessment model for deli meats provides the parameters for the 

Normalretail distribution. The mean is approximately -8 and the standard deviation is 

about 3.5. Given these parameters, the parameters of the Normalprocess distribution are 

calculated for different cases of growth. These cases are defined by assuming different 

parameters for the Normalgrowth distribution. 

75 
FSIS Listeria Risk Assessment May 26, 2003 

As assumed in the FDA/FSIS exposure assessment for deli meats, a threshold 

concentration of one L. monocytogenes in 25 grams is needed for a test to be positive. 

This concentration is equivalent to -1.4 logs. The proportion of the retail and processing 

distributions above this threshold provides an estimate of the prevalence of positive 

samples at each of these locations. 

Case 1 

The 2001 FDA/FSIS exposure assessment model for deli meats predicts an average growth of 

1.9 logs with a standard deviation of 1.4 logs. Figure A-1 illustrates the outcome in this case. 

The grey line shows the threshold above which any sample would be positive. In this case, 

although 3% of samples would be positive at retail, only 0.3% of samples would be positive at 

processing. 

Processing concentration X >=-1.4 

0.3% 

0 

0.02 

0.04 

0.06 

0.08 

0.1 

0.12 

-25 -20 -15 -10 -5 0 5 

Log(Lm per gram ) 

Probability 

=1/25=0.04 

Lm per gram 

Normal 

mean= -9.9 

s.d=3.2 

Grow th m ultiplier 

0 

0. 05 

0. 1 

0. 15 

0. 2 

0. 25 

0. 3 

-4 -2 0 2 4 6 8 

Lo g( Gr owt h) 

Normal 

mean=1.9 

s.d=1.4 

Retail concentration X >=-1.4 

3% 

0 

0.02 

0.04 

0.06 

0.08 

0.1 

0.12 

-25 -20 -15 -10 -5 0 5 10 

Log(Lm per gram ) 

Probability 

=1/25=0.04 

Lm per gram 

Normal 

mean= -8 

s.d=3.5 

Figure A-1. Case 1 example where growth multiplier is assumed to be a normal distribution with 

a mean and standard deviation consistent with those predicted by the growth model in the 2001 

FDA/FSIS exposure assessment model for deli meats. 

Case 2 

While the 2001 FDA/FSIS exposure assessment model for deli meats predicts a distribution of 

growth (mean = 1.9 logs and s.d.= 1.4 logs), the model only uses the central tendency value 

when predicting growth between processing and retail. Figure A-2 illustrates the outcome when 

76
FSIS Listeria Risk Assessment May 26, 2003 

growth is a scalar adjustment. In this case, 3% of samples would be positive at retail and 0.8% 

of samples would be positive at processing. 

+ 

= 

Process ing concentration 

X >= -1.4 

0.8% 

0 

0.02 

0.04 

0.06 

0.08 

0.1 

0.12 

-25 -20 -15 -10 -5 0 5 

Log(Lm per gram ) 

Probability 

Normal 

Mean= -9.9 

s.d.=3.5 

Growth multiplier 

0 

0.2 

0.4 

0.6 

0.8 

1 

-2 -1 0 2 5 

Log(Grow th) 

Probability 

Scalar value 

= 1.9 

Retail concentration X >= -1.4 

3% 

0 

0.02 

0.04 

0.06 

0.08 

0.1 

0.12 

-25 -20 -15 -10 -5 0 5 10 

Log(Lm per gram ) 

Probability 

=1/25=0.04 

Lm per gram 

Normal 

mean=-8 

s.d=3.5 

1 3 4 

Figure A-2. Case 2 example where growth multiplier is a constant value of 1.9 logs. This is the 

assumption made when accounting for growth in the FDA/FSIS exposure assessment model for 

deli meats. 

Case 3 

Instead of a 1.9 logs scalar adjustment for growth, a 1 log adjustment is considered. Figure A-3 

illustrates the outcome for this case in which 1.5% of samples would be positive at processing. 

77
FSIS Listeria Risk Assessment May 26, 2003 

+ 

= 

Processing concentration 

X >= -1.4 

1.5% 

0 

0.02 

0.04 

0.06 

0.08 

0.1 

0.12 

-25 -20 -15 -10 -5 0 5 

Log( Lm pe r gr a m ) 

Normal 

mean= -9 

s.d.=3.5 

Growth multiplier 

0 

0.2 

0.4 

0.6 

0.8 

1 

-2 -1 0 1 4 

Log(Grow th) 

P ro b a b ilit y 

Scalar value 

= 1.0 

Retail conce ntration 

X >=-1.4 

3% 

0 

0.02 

0.04 

0.06 

0.08 

0. 1 

0.12 

-25 -20 -15 -10 -5 0 5 10 

Log( Lm pe r gr a m) 

2 3 5 

Normal 

mean= -8 

s.d=3.5 

=1/25=0.04 

Lm per gram 

Figure A-3. Case 3 example where growth multiplier is a constant 1 log. This is the assumption 

used in the in-plant model. 

Of the three cases considered, Case 3 is most consistent with the 1%-3% prevalence of positive 

samples found by FSIS at processing. In both Cases 1 and 2, the prevalence of positive samples 

at processing are below this observed range. None of the cases match the NFPA results of 0.9% 

positive samples at retail, but these results are included in the algorithm for estimating the L. 

monocytogenes concentration distribution for deli meats at retail in the FDA/FSIS exposure 

assessment model. 

For the in-plant model, the scenario presented for Case 3 is used. A one log adjustment for 

growth seems most consistent with the available data at processing, as well as the L. 

monocytogenes concentration distribution in deli meats at retail estimated in the FDA/FSIS 

exposure assessment model for deli meats. 

78
FSIS Listeria Risk Assessment May 26, 2003 

Appendix C. Evaluation of FSIS RTE Survey Data for Volume of Production 

for Establishments Producing Deli Meats 

Purpose 

The purpose of this analysis is to estimate probability distributions of product mass per line per 

shift for plants categorized as large, medium, and small volume plants. These three probability 

distributions are necessary inputs to the in-plant FSIS Listeria risk assessment model. 

Data 

The data were collected during a November 2002 survey of RTE processors. A total of 1139 

processing establishments provided responses to this survey. While the survey included 

questions about hot dog production, this analysis only focuses on production of deli products. 

There were four classes of deli products considered: 

1. Unpeeled other sausage type product 

Examples: bologna, mortadella, cooked salami 

2. Large mass chopped and formed type product 

Examples: turkey roll, pickle & pimento loaf, cooked ham (sectioned and formed) 

3. Large mass whole muscle type product 

Examples: cooked roast beef, cooked whole birds, cooked corned beef, whole cooked turkey 

breast, cooked whole ham 

4. Sliced type product 

Examples: sliced ham/bologna/chicken or turkey breast/olive loaf 

Regarding production amounts, the survey asked processors to estimate production per shift of 

operation. One shift is assumed to refer to the time of production between clean-up in a 

processing plant. A single day in a large processing plant may comprise two shifts; the first two 

occurring between 6 am and 12 pm, and the second between 1 pm and 6 pm. These shifts are 

separated by work stoppage, cleaning of equipment, and a lunch break for personnel. 

Nevertheless, a shift may represent the continuous production of a specific deli product on one or 

more lines in the processing plant. Therefore, the survey also asked processors for the number of 

lines simultaneously operating in the processing plant. 

To estimate total annual production, the survey also asked processors to provide the number of 

shifts per week, and weeks per year, the plant was producing a particular deli product. 

PRODUCTION PER SHIFT 

Each processing plant completed production per shift questions for each deli product it 

produces. Responders selected one of the following choices to signify production per 

shift for each deli product they produced. 

a. < or = 1,000 lbs 

79
FSIS Listeria Risk Assessment May 26, 2003 

b. 1,000 – 10,000 

c. >10,000-50,000 

d. >50,000-100,000 

e. >or =100,000 

does not produce Æ skip to next row 

For the purposes of analysis, the responses were converted into point estimates by 

assuming the median value of intervals. For choice e (> 100,000 pounds per shift), a 

value of 100,000 was assumed. Therefore, the following values were entered into the 

database for the selected choice. 

a. 500 lbs 

b. 5500 lbs 

c. 30,000 lbs 

d. 75,000 lbs 

e. 100,000 lbs 

LINES PER SHIFT 

For each deli product, responders indicated the number of lines producing this product 

per shift. 

Number of lines 

producing this product per shift:

(select one) 

a. 1 

b. 2 

c. 3 

d. 4 

e. > or = 5 

In this case, responders who selected choice e (>5) were assumed to have 5 lines per 

shift. 

SHIFTS PER WEEK 

For each deli product, responders indicated the number of shifts that produced this 

product per week. 

Number of 

shifts per week: 

(select one) 

a. 1-3 

b. 4-5 

c. 6-8 

d. 9-10 

e. >or = 11 

The midpoint value of each interval was selected as a point estimate for the database. 

Therefore, the following values were assumed. 

80
FSIS Listeria Risk Assessment May 26, 2003 

Number of 

shifts per week: 

(select one) 

a. 2 

b. 4.5 

c. 7 

d. 9.5 

e. 11 

NUMBER OF WEEKS OF PRODUCTION 

For each deli product, responders indicated the number of weeks that this product was 

produced each year. 

Number of production weeks per year: 

(select one) 

a. < or = 12 weeks 

b. 13 – 24 

c. 25 – 42 

d. 43 – 51 

e. 52 weeks 

The midpoint value of each interval was selected as a point estimate for a database entry. 

Therefore, the following values were assumed. 

Number of production weeks per year: 

(select one) 

a. 6 weeks 

b. 18.5 weeks 

c. 33.5 weeks 

d. 47 weeks 

e. 52 weeks 

Methods 

The analysis began by estimating each processing plant’s total annual production of all deli 

products. The plants were then ranked and categorized into large, medium, and small volume 

processors based on this total annual production. 

For each volume category, the production per line per shift was initially characterized for each 

deli product. The production per line per shift for the entire volume category was then estimated 

by combining the deli products within the category. 

TOTAL ANNUAL PRODUCTION PER PROCESSING PLANT 

Responding processing plants were ranked by their estimated total annual production of 

all deli products. For each deli product produced in a processing plant, the total annual 

production was estimated as; 

Production per shift x Shifts per week x Weeks per year 

81
FSIS Listeria Risk Assessment May 26, 2003 

The total annual production per processing plant was estimated as the sum of the annual 

production for all deli products produced in that processing plant. 

CATEGORIZING LARGE, MEDIUM, AND SMALL VOLUME PROCESSING PLANTS 

Responding plants were ranked by their total annual production of all deli products and 

assigned to large, medium of small volume plant categories. Definitions for the categories 

were provided by OPPD. Large volume plants were defined as those whose total annual 

production of all deli products was within the top quartile of plants (> 75th percentile). 

Medium volume plants were defined as those whose total annual production of all deli 

products was between the 50th percentile and the 75th percentile. Small volume plants were 

defined as those whose total annual production of all deli products was less than the 50th 

percentile. 

ESTIMATING PRODUCTION PER LINE PER SHIFT 

The in-plant Listeria monocytogenes risk assessment model randomly selects a production 

line during a shift and characterizes the production of deli product in terms of lots. 

Therefore, a lot is the mass of deli product produced by a single line during one shift. 

Because the lot is the unit modeled in the risk assessment, the survey data were analyzed to 

estimate the distribution of production per line per shift by volume category. 

For each deli product produced in a processing plant, the production per line per shift was 

estimated as: 

Production per shift 

Lines per shift 

For each volume category, various distributions were fit to the (non-zero) production per 

line per shift estimates for each of the deli products. Fitting of continuous probability 

distributions to the data was accomplished using the maximum likelihood estimation 

algorithm in BestFit. The choice of distributions was limited by forcing the distributions to 

have non-negative domains. 

In each volume category, the selected distributions for the four deli products were combined 

using Monte Carlo simulation. On each iteration of a simulation, one distribution was 

randomly selected according to the percent of total annual production represented by the deli 

product (Table x), and a value from the selected distribution was randomly selected. At 

10,000 iterations per simulation, the mean and standard deviation converged sufficiently so 

that there was <1% change in these parameters. 

The 10,000 values, or pseudo-data, generated from each simulation (one each for large, 

medium, and small volume plants) were then entered into BestFit and several plausible 

82
FSIS Listeria Risk Assessment May 26, 2003 

distributions were fit to the data. Chi-square, Anderson-Darling (AD), and KolmogorovSmirnov (KS) goodness of fit statistics were also calculated. 

Table C-1. Estimated annual production (pounds) for four deli products within three 

volume categories. 

Volume 

Category Deli-1 Deli-2 Deli-3 Deli-4 Total 

Large 591,002,625 1,499,120,250 2,002,267,375 2,091,023,125 6,183,413,375 

(%) 9.6% 24.2% 32.4% 33.8% 

Medium 35,188,375 31,824,250 69,002,125 52,759,000 188,773,750 

(%) 18.6% 16.9% 36.6% 27.9% 

Small 7,282,125 5,118,000 9,495,375 12,223,000 34,118,500 

(%) 21.3% 15.0% 27.8% 35.8% 

Results 

The results of this analysis summarize the total annual production for all processing plants, the 

statistical fitting of probability distributions to the deli product classes within each production 

volume category, and the statistical fitting of probability distributions to the combined data 

within volume categories. 

TOTAL ANNUAL PRODUCTION ACROSS ALL PLANTS 

After ranking processing plants from largest to smallest total annual production of deli 

products, it was noted that processors in the upper 25th percentile of production are 

responsible for >95% of total annual production (Figure C-1). In other words, 285 (25%) 

of the 1139 processors surveyed produced a total of 6.2 billion (96%) of the 6.4 billion 

pounds all processors were estimated to produce in a year. It is also notable that the top 

10% of processors are responsible for about 85% of total annual production of deli meats. 

100% 

80% 

60% 

40% 

20% 

0% 

0% 20% 40% 60% 80% 100% 

Fraction of processing plants 

Figure C-1. Relationship between the fraction of processing plants and the cumulative total 

annual production of deli products by the processing plants. Fra

ction of t

otal a

nnu

al 

production of deli 

products 

83
FSIS Listeria Risk Assessment May 26, 2003 

INITIAL FITTING OF DISTRIBUTIONS TO DATA BY DELI PRODUCT CLASSES 

WITHIN VOLUME CATEGORIES 

Statistical fits of lognormal distributions to the data from each of the four deli products 

within each processing plant volume category suggested substantial differences in the 

average pounds of production per line per shift (Table C-2). For example, the average 

production per line per shift of deli product category 1 is 12,637 pounds from large 

volume plants, 1,251 pounds from medium volume plants, and 532 pounds from small 

volume plants. Similar patterns are noted for the other deli product categories. 

Goodness of fit tests did not support a conclusion that the production per line per shift 

data originated from any of the parametric distributions tested, including the lognormal. 

Such a finding is not surprising given the nature of the data. Figures C-2 and C-3 are 

examples of estimated lognormal distributions and empirical cumulative frequency 

distributions based on the survey data. The empirical distributions shown in these graphs 

are not characteristic of any smooth cumulative function. Instead, these distributions 

suggest a “lumpy” pattern of data points. This clustering of data points is a result of the 

ordinal, multiple choice format of the survey. There were only 5 choices for production 

per shift – and 5 choices for lines per shift – available to those surveyed. Therefore, only 

a total of 25 data points were possible. This limitation of the data is responsible for the 

stair-stepping pattern evident in the empirical cumulative distribution. Such a pattern 

would be very difficult for any smooth, well-behaved function to statistically fit, yet the 

lack of fit does not necessarily rule out the hypothesis that the data originated from a 

lognormal distribution. For the purposes of this analysis, the lognormal distribution was 

selected for ease of implementation and plausibility relative to other parametric 

distributions available from BestFit, e.g., loglogistic, Inverse Gaussian, Weibull, gamma, 

or exponential. 

Table C-2. Maximum likelihood estimates of lognormal distribution parameters for production 

per line per shift. Large, medium, and small volume processors are defined based on total annual 

production of all deli products 

Deli category 1 Deli category 2Deli category 3 Deli category 4 

Large volume 

mean 19,384 23,766 12,501 

s.d. 49,396 49,470 29,710 

Medium volume 

mean 1,041 2,087 1,303 

s.d. 1,137 3,409 1,672 

Small volume 

mean 560 639 555 

s.d. 219 355 220 

12,637 

28,468 

1,251 

1,580 

532 

162 

84
FSIS Listeria Risk Assessment May 26, 2003 

Lognorm(12637, 28468) 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 

0 10 20 30 40 50 60 70 80 

Pounds produced per line per shift (Values in Thousands) 

Data Lognorm 

Figure C-2. Comparison of lognormal distribution to data on production per line per shift from 

large volume processing plants’ production of deli product category 1. 

Lognorm(1250.6, 1580.0) 

1 

0.9 

0.8 

0.7 

0.6 

0.5 

0.4 

0.3 

0.2 

0.1 

0 

0 5 10 15 20 25 30 35 

Pounds produced per line per shift (Values in Thousands) 

Data Lognorm 

Figure C-3. Comparison of lognormal distribution to data on production per line per shift from 

medium volume processing plants’ production of deli product category 1. 

Cumul

ative freq

u

ency 

C

u

m

ulati

ve freq

u

e

n

cy 

85
FSIS Listeria Risk Assessment May 26, 2003 

STATISTICAL FITTING OF DISTRIBUTIONS TO COMBINED VOLUME 

CATEGORY DATA 

Following the combining of the four deli product distributions within each volume 

category by Monte Carlo simulations, the resulting pseudo-data were entered into BestFit 

to determine plausible distributions. A priori, it could not be determined what 

distribution would describe each volume categories’ production per line per shift. 

Nevertheless, the lognormal distribution best fit the data from the large and medium 

volume categories, and was the second best-fitting distribution in the small volume 

category (Tables C-3 – C-5). The lognormal distribution was a statistically significant fit 

(P=0.58) in the large volume category, but none of the distributions’ fits were significant 

for the medium and small volume categories. 

Because the lognormal distribution was the only statistically significant fit, and this 

distribution could be readily implemented in the Listeria monocytogenes risk assessment 

model, this distribution was assumed applicable for the three volume categories. The 

lognormal parameters shown in Tables C-3 – C-5 were used. Nevertheless, it should be 

noted that uncertainty about the true distribution type and the parameters of the 

lognormal could influence the results of the in-plant model. 

Table C-3. MLE parameters for production per line per shift in large volume plants. 

Parameters and goodness of fit statistics were generated from analysis of pseudo-data resulting 

from combining four deli product categories in large volume category of plants. 

MLE’s Lognormal LogLogistic Inverse Gaussian Weibull Exponential 

parameter 1 18,420.35 0.00 18,067.37 0.72 18,067.37 

parameter 2 45,155.71 6,982.84 3,106.66 13,963.21 

parameter 3 1.25 

Goodness of fit 

Chi-sqr value 69.80 213.69 963.57 1,077.15 2,843.44 

p-value 0.58 0.00 0.00 0.00 0.00 

AD value 0.35 7.77 196.25 108.55Infinity 

p-valueN/A N/A N/A < 0.01 < 0.001 

KS value 0.01 0.02 0.10 0.06 0.18 

p-valueN/A N/A N/A < 0.01 < 0.01 

Table C-4. MLE parameters for production per line per shift in medium volume plants. 

Parameters and goodness of fit statistics were generated from analysis of pseudo-data resulting 

from combining four deli product categories in the medium volume category of plants. 

MLE’s Lognormal LogLogisticInverse Gaussian Weibull Exponential 

parameter 1 1,487.93 0.00 1,532.48 0.92 1,532.48 

parameter 2 2,115.42 846.98 744.94 1,455.15 

parameter 3 1.67 

Goodness of fit 

Chi-sqr value 99.43 175.28 288.81 1,428.84 1,500.43 

p-value 0.02 0.00 0.00 0.00 0.00 

AD value 1.89 5.98 28.26Infinity Infinity 

p-valueN/A N/A N/A < 0.01 < 0.001 

86
FSIS Listeria Risk Assessment May 26, 2003 

KS value 0.01 0.01 0.04 0.07 0.09 

p-valueN/A N/A N/A < 0.01 < 0.01 

Table C-5. MLE parameters for production per line per shift in small volume plants. 

Parameters and goodness of fit statistics were generated from analysis of pseudo-data resulting 

from combining four deli product categories in the small volume category of plants. 

MLE’s LogLogistic Lognormal Inverse Gaussian Gamma Weibull 

parameter 1 0.00 573.46 574.52 5.74 2.25 

parameter 2 523.58 251.29 2,971.02 100.14 648.76 

parameter 3 4.24 

Goodness of fit 

Chi-sqr value 121.02 126.09 159.89 454.54 1,590.99 

p-value 0.00 0.00 0.00 0.00 0.00 

AD value 2.27 4.51 7.67 36.67Infinity 

p-valueN/A N/A N/A < 0.005 < 0.01 

KS value 0.01 0.02 0.02 0.04 0.08 

p-value N/A N/A N/A N/A < 0.01 

Conclusions 

• Large volume processing plants – the upper quartile of all plants – are responsible for 

>95% of all deli meat produced per year. 

• Deli product classes 3 and 4 – large mass whole muscle and sliced meats – comprise the 

largest share of deli products produced by all processing plants. 

• While large volume processors produce the greatest total product per year, these plants 

also have a much larger average production per line per shift than medium and small 

volume processors. 

• After combining the four deli products, the resulting production per line per shift 

distribution can be modeled as a lognormal for each of the volume categories. 

87
FSIS Listeria Risk Assessment May 26, 2003 

Appendix D 

Volume Based Testing and Lot Testing Based on Sequential Positive

Food Contact Surface Results

1) VOLUME BASED TESTING 

Using the results from Appedix C above, the in-plant model was rerun. Because the probability 

distribution used to generate lot masses changed from a normal to a log10 normal, no direct 

comparison between the volume based sampling and the HAACP based sampling is possible 

without a model recalibration, which was not performed. Although the mean lot mass increased 

based on the volume classification, the median lot mass actually decreased. Because of this, the 

Lm concentrations at retail are higher than the baseline in the HAACP classification approach. 

A recalibration would be need to reduce these concentrations back to the FDA distributions. 

Because of this, comparisons should only be made within the same classification strategy 

(volume based or HAACP based). 

The classification strategy only makes a difference if there is a differential testing frequency 

among the categories. For example, a 4-2-1 testing frequency requires more tests of the larger 

category, and the retail concentrations and public health impacts can change depending on how 

the categories are defined. A 4-4-4 testing frequency tests all producers at the same frequency, 

and the category definition is immaterial. 

The survey analysis in Appendix C found that over 96% of servings were produced by the top 

25% of production facilities. The HAACP category of large plants found that 48% of servings 

were produced by the large HAACP category. Switching to volume based testing (with 

equivalent testing frequencies for the “large” category) implies that more servings are tested at a 

higher frequency under the volume based approach than under the HAACP based approach. The 

earlier risk assessment found that increased testing frequency was statistically significantly 

correlated with greater number of lives saved. Thus, switch to volume based sampling categories 

would have a corresponding benefit for the number of lives saved. 

Figure D-1 illustrates the Lm concentrations at retail from a volume based categorization under 

different FCS sampling frequencies and interventions. The trends are similar to the HAACP 

based results. Increased FCS testing results in lowered Lm concentrations at retail, particularly 

among the highest quantiles. Post-processing and growth inhibiting formulation and packaging 

decrease the lower quantiles, and the combination has the greatest impact overall. 

88
Lm Concentr

ation at r

etail 

(cf

u/g) 1e+8 Q80 

Q99 1e+7 Q99.99

1e+6 

1e+5 

1e+4 

1e+3 

1e+2 

1e+1 

1e+0 

1e-1 

1e-2 

1e-3 

1e-4 

1e-5 

1e-6 

1e-7 

1e-8 

04

4

8

1

3

6

0

0

0

-0-2

-4

-4

6

2

0

-

-

-

-

-

-

0

0

0

-0 

-1 

-4 

-2 

8 

1 

6 

-

-

-

-

6 

0

0 

0 0

4 

-8 

-6 

P G 

P

0 

P 

IP 

P& GIP

Testing and Intervention 

Volume Based Sampling 

FSIS Listeria Risk Assessment May 26, 2003 

2) LOT TESTING BASED ON SEQUENTIAL POSITIVE FOOD CONTACT SURFACE 

RESULTS 

Description of Lot Testing Based on Sequential Positive Food Contact Surface 

Take FCS samples on a regular basis. Upon notification that a FCS was positive for Lspp, 

immediately take a second FCS sample. Anytime two reported FCS samples are positive 

(regardless of the time between samples): 

a) take corrective sanitation action, 

b) immediately take a lot sample, 

c) take FCS samples continuously until a negative is reported 

d) hold any additional lots until the FCS sample result is available. If the FCS result is 

reported as positive, release held product lots. If the FCS sample is positive, test all 

product lots being held. 

Note that product lot testing is triggered by FCS positives, not by any product lot results. The 

timing between FCS samples can vary because sequential positives can trigger additional FCS 

testing. Two examples are shown below. 

89
FSIS Listeria Risk Assessment May 26, 2003 

Below is part of the run with the interpretation of what sequential means in terms of the actual 

actions undertaken. The model assumes that 6 lot production before a result is returned (3 days 

reporting lag * 2 lots per day). 

Abbreviations: 

“h,y”: initially held, then tested based on a later FCS positive result 

“h,n”, initially held, then released without testing based on a later negative FCS result 

Table D-1: Simple Consecutive Positive Example 

Lot FCS 

Sampled? 

Result 

Reported

from 

FCS Test 

6 lots 

previous 

Consecutive 

positive 

count 

Action 

Lot test? 

625 n n 

626 y n 

627 n n 

728 n n 

629 n n 

630 n n 

631 n n 

632 y pos 1 take additional FCS sample n 

633 n n 

634 n n 

635 n n 

636 n n 

637 n n 

638 y pos 2 trigger lot test 

trigger FCS test until result 

available 

y 

639 y hold lot h,n 

640 y hold lot h,n 

641 y hold lot h,n 

90
FSIS Listeria Risk Assessment May 26, 2003 

642 y hold lot h,n 

643 y hold lot h,n 

644 n neg 0 release held lots n 

645 n neg 0 n 

646 y pos 1 take additional FCS sample n 

647 n neg 0 n 

648 neg 0 n 

649 neg n 

650 n 

651 n 

652 neg 0 n 

653 n 

Table D-2: Complex Consecutive Positive Example 

Lot FCS 

Sampled? 

Result 

Reported

from FCS 

Test 6 lots 

previous 

Consecutive 

positive 

count 

Action 

Lot 

test? 

896 n n 

897 y n 

898 n n 

899 n n 

900 n n 

901 n n 

902 n n 

903 y pos 1 take additional FCS sample n 

904 n n 

905 n n 

906 n n 

907 n n 

908 n n 

909 y pos 2 trigger lot test, 

trigger FCS until result 

y 

91
FSIS Listeria Risk Assessment May 26, 2003 

available 

910 y hold lot h,y 

911 y hold lot h,y 

912 y hold lot h,y 

913 y hold lot h,y 

914 y hold lot h,y 

915 y pos 3 test held lots 

test current lot 

y 

916 n neg 0 n 

917 n neg 0 n 

918 y pos 1 take additional FCS sample n 

919 y pos 2 trigger lot test 

trigger FCS until result 

available 

y 

920 y pos 3 test lot y 

921 y pos 4 test lot y 

922 y hold lot h,y 

923 y hold let h,y 

924 y pos 5 test held lots 

test current lot 

y 

925 n neg 0 n 

926 n neg 0 n 

927 n neg 0 n 

928 neg 0 n 

929 neg 0 n 

930 neg 0 n 

Note in Table D-2 that upon the receipt of the 3rd positive FCS result, the number of product lots 

tested increases dramatically as the held lots are tested. The more important impact of requiring 

sequential FCS positives before a product lot sample is taken is that two Lspp reporting lags 

occur before a product sample is taken. For the Lm risk assessment, each reporting lag was 

taken as 3 days, so this approach does not take a lot sample 6 days after the first FCS positive 

occurred. 

The duration of a contamination event in the model has a mean of about 9 days and a median of 

about 4 days. (Recall the parameter is log10 normally distributed.) Thus the majority of the 

contamination events are over before a lot sample is taken. Only long term contamination events 

are likely to be detected. The problem is compounded by the fact that even within a 

contamination event, not all FCS samples are positive. One negative FCS sample is enough to 

reset the number of consecutive positives. 

The model results bear out these concerns. Figure D-2 illustrates the Lm concentrations at retail. 

Increased testing does not reduce the concentrations, even at the higher quantiles. 

The nonconsecutive positive approach for Figures D-3 and D-4 used test-and-hold, so that the lot 

sampled corresponded to the FCS positive. 

92
FSIS Listeria Risk Assessment May 26, 2003 

Figure D-3 compares the number of FCS and product lot tests between the two approaches. The 

consecutive positive approach often requires more FCS tests. In Table D-2 for example, 5 FCS 

tests were required while waiting for the result after the second FCS positive. Figure D-3 also 

shows, however, that in general fewer products lots are tested. The proposed approach, as 

defined, results in more FCS samples taken and fewer product lot samples than the alternative 

approach of not requiring consecutive positives. 

Figure D-4 illustrates the likelihood of detecting a positive once a FCS or product lot sample has 

been taken. The consecutive positive approach consistently has higher probability of finding a 

positive FCS once a FCS sample is taken. The efficiency drops off with increasing testing 

frequencies. The original approach had lower, and more constant efficiency levels. 

The lot testing level efficiencies are quite different. The consecutive positive approach always 

resulted in lower likelihood of finding a positive product lot by about a factor of 4. Lot samples 

appear to be taken too late compared to when the contamination even is ongoing. 

CONCLUSIONS 

The overall effect of requiring consecutive FCS positives before taking a product lot sample 

results in fewer lot samples being collected and a much lower likelihood of finding positives lots 

for those that are collected. Based on these findings, and the lack of any decrease in the Lm 

concentration at retail, the consecutive positive requirement appears to be ineffectual in 

protecting public health 

93
0

4

0

0

0

FSIS Listeria Risk Assessment May 26, 2003 

1e+7 

Q80 

Q99 

Q99.99 1e+6 

1e+5 

1e+4 

1e+3 

1e+2 

1e+1 

1e+0 

1e-1 

1e-2 

1e-3 

1e-4 

1e-5 

1e-6 

1e-7 

1e-8 

Lm concentr

ation at r

etail 

(cf

u/g) 

0- -

4-2-

4-4-

8- -

16-8 

32-1 

60-6 

0- -

0- -

0-

0 

1 

4 

2 

-

6 

0

0 

0 

-0

4 

-8 

-6 

P 

G 

P

0 

P 

IP 

P& GIP

Testing and Intervention 

Volume Based Sampling

Consecutive Positives Required 

Figure D-2. Lm Concentrations at Retail Under Various FCS Sampling Frequencies When 

Consecutive FCS Samples are Required to Trigger a Product Test. 

94
FSIS Listeria Risk Assessment May 26, 2003 

1000000 

800000 

600000 

400000 

200000 

1.4e+5 0

1.2e+5 

1.0e+5 

8.0e+4 

6.0e+4 

4.0e+4 

2.0e+4 

0.0 

4-2-1 8-4-2 16-8-4 32-16-8 60-60-60 

Consecutive + not required 

Consecutive + required 

Testing Frequency 

Figure D-3. Comparison of the number of FCS tests and product lot tests when consecutive 

positives are required and not required to trigger a product lot test. 

N

o. of FCS Tests

N

o. Lot Tests 1,000,000 Simulations 1,000,000 Sim

ulations 

95
0 

FSIS Listeria Risk Assessment May 26, 2003 

20

15

10

5

25

20

15

10

5

0 

4-2-1 8-4-2 16-8-4 32-16-8 60-60-60 

Consecutive + not required 

Consecutive + required 

Testing Frequency 

Figure D-4. Comparison of the efficiency of FCS and product lot sampling when consecutive 

positives are required and not required to trigger a product lot test. 

96

Per

cent Lot Positi

v

es Perc

e

nt F

CS Po

siti

v

e Sampl

es