FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Assessing the Effectiveness of the 

“Listeria monocytogenes” Interim Final Rule 

Summary Report 

This report was prepared by the Agency’s Listeria monocytogenes Assessment Team. 

Members of this team were Agency employees who served on one or more Project 

Assessment Teams (PAT’s). Each PAT was assigned to one of seven aspects of the 

assessment. The Summary Report contains an executive summary of major findings and 

recommendations, as well as the Agency’s accomplishments associated with the 

implementation of the L. monocytogenes interim final rule. 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Executive Summary 

In January 2004, the Food Safety and Inspection Service (FSIS) assembled a team to 

assess and measure the effectiveness of the new regulation to control L. monocytogenes 

in ready-to-eat (RTE) meat and poultry products. The Listeria monocytogenes (L. 

monocytogenes) Assessment Team (hereafter referred to as the Team) was charged with 

identifying these measures of effectiveness as well as making recommendations for 

further evaluations. This report presents the major findings and recommendations of the 

Team. This information will be considered in finalizing the interim final rule. The Team 

also began planning for long term Agency initiatives in the areas of retail sampling, RTE 

shelf life and public health issues. 

The 28-member Team represented staff organizations from the Agency. The Team 

established an overall project plan and assigned members to smaller Project Assessment 

Teams (PAT). Each PAT designated a project leader, developed its own project 

assessment plan, and prepared a report of findings and recommendations for its assigned 

area addressing one of the seven aspects of the rule. 

This Summary Report presents the major findings and recommendations as prioritized by 

the Team. There are additional findings and recommendations with prescriptive 

suggestions for implementation in each of the PAT’s individual reports (attached). 

In general, the Team found that the L. monocytogenes interim final rule demonstrated a 

continual positive impact during its short implementation period from October 2003 to 

June 2004. Sampling based upon risk has begun and a process is in place to improve the 

scientific basis for sample selection. Plans are underway to continually improve the 

Agency’s efforts to assure that all size establishments and consumers, especially those at 

risk, are aware of the necessary actions needed to assure RTE products are produced, 

labeled, and consumed in a safe manner. 

Initiatives are continuing to focus on improving the safety of RTE meat and poultry 

products at retail deli counters. Information sources are being refined to measure the 

impact of the Agency’s policies on the prevalence of L. monocytogenes in RTE product 

and its relationship to public health. The Team also lays the groundwork for future 

Agency initiatives in retail food handling safety and monitoring the prevalence of L. 

monocytogenes in RTE meat and poultry products to protect the public health. 

Establishments are aware of, and responding positively to the new rules. FSIS intends to 

update the economic impact analysis for the final rule using, among other sources, data 

from FSIS Form 10,240-1, Production Information on Post-Lethality Exposed Ready-toEat Products, which will help measure the capital costs of the alternative compliance 

approaches. Inspection personnel were trained and are carrying out their responsibilities 

to ensure compliance. However, additional training needs of inspection personnel are 

being identified and prioritized. 

2 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The Team recommends new initiatives and continued improvement in the Agency’s 

efforts. A new focus on the baseline sampling to support risk analysis is suggested. L. 

monocytogenes labeling statements and a consumer education campaign based upon an 

analysis of focus group research results is needed. The Team also sees significant 

potential for the Agency to undertake regulatory and other strategies needed to reduce the 

prevalence of L. monocytogenes at retail deli counters. 

To assure that the Agency’s regulatory strategies continue to protect the public health, the 

Team identified three important sources (National Surveillance Data, Outbreak Data, and 

FoodNet Data) of illness information about the occurrence of listeriosis in the U.S. 

population and, in some cases, the food vehicle responsible for those illnesses. The Team 

suggests the need for periodic analysis of this data to create a listeriosis profile and to 

make recommendations of food safety and public health issues. 

Recommendations have been made to address economic impact, training needs, and 

communications with small businesses. Priorities are listed to provide training to a large 

number of inspection personnel with ideas on how to expand the Agency’s efforts using 

cost conscious, proven educational technologies such as web-based training to deliver 

and track progress. Ideas are also presented to improve efforts to reach small businesses, 

revise the Agency’s Compliance Guidelines, and specifically provide assistance on how 

to assure products are properly classified as RTE. 

The Agency provided a briefing on the Team’s activities to the National Advisory 

Committee on Meat and Poultry Inspection (NACMPI) during its June 2004 meeting and 

obtained its suggestions. The Team’s PATs incorporated the Committee’s advice into 

their planned activities and their revised reports that are attached to this document. 

3 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Major Findings and Recommendations 

The following sections summarize the reports of the seven Project Assessment Teams 

(PAT). Each section presents the findings and recommendations for the respective PAT. 

VERIFICATION SAMPLING 

Findings 

Current Sampling. The Agency continued its sampling program of RTE meat and poultry 

products and is currently scheduling randomized samples for all RTE establishments and 

products regardless of risk. The Agency initiated an additional sampling program at the 

beginning of the 2004 that targets the riskiest product, at the inspector’s discretion, within 

any given establishment. 

Conclusions from available L. monocytogenes testing data conducted since January 2004 

indicate that the prevalence for both sampling programs is essentially equivalent, less 

than or approximately 1 percent. Interestingly, some low-risk products have been found 

to be positive for L. monocytogenes. 

Risk-Based Sampling. Overall, the relative risk posed by RTE meat and poultry products 

produced by an establishment varies by the type of product (i.e., deli meat versus dry 

sausage) and the type of process controls in place after the lethality step. 

The Agency’s L. monocytogenes Risk Assessment (FSIS 2003) indicated that use of both 

post-lethality interventions and use of growth inhibitors has the greatest impact on 

lowering the risk of illness/death from L. monocytogenes in RTE meat and poultry 

products. The next category is the use of post-lethality interventions or growth inhibitors, 

followed by testing and sanitation of food contact surfaces. 

Recommendations 

Current Sampling. Modifications to the current sampling program should be made to 

drive sampling toward baseline surveillance. This allows FSIS to gain knowledge of the 

pattern of occurrence and potential of occurrence in a community. This will enable the 

Agency to develop ways to control and prevent disease in a community and collect data 

to accurately track trends and improvements in L. monocytogenes prevalence from year 

to year. 

The current Agency sampling program should also be modified to be consistent with the 

baseline study design using weighted sampling based on production volume or weighted 

estimate analysis after the fact. 

4 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Risk-Based Sampling. Currently, the Agency is working to complete a risk-based 

sampling scenario that will be used to revise the Agency’s Verification Sampling 

program, as discussed by the L. monocytogenes interim final rule. This scenario will take 

into account the special needs of small businesses with respect to production volume 

while protecting the public health. 

Currently under development is a model that categorizes production into the three 

alternatives categorized by the L. monocytogenes interim final rule. This model is being 

developed incorporating the risk analysis of the available management interventions such 

as: testing frequency, number of consecutive positives prior to product testing (test-andhold), sample size and other factors. This work will develop and incorporate a 

quantitative risk assessment using the three alternative categories presented in the L. 

monocytogenes interim final rule. 

It is likely that other risk factors contributed to L. monocytogenes contamination, but the 

available data precludes this analysis for the present. These factors might include 

construction at the processing plant, age of the processing plant, building material used in 

the plant, and degree of separation of incoming product with post lethality treated 

product. 

Other Recommendations. If a sample is found to be positive for L. monocytogenes after 

testing by the Agency, inspection personnel should evaluate the establishment’s Hazard 

Analysis and Critical Control Points (HACCP) plan, Standard Sanitation Operating 

Procedures (SSOP), and prerequisite programs to confirm that the establishment’s 

proposed corrective actions appear reasonable and insure that the establishment begins 

environmental testing. 

Standardizing the test sampling procedure is inherently difficult. Agency inspection 

personnel may use discretion when selecting samples and performing sampling 

techniques. Each plant is a unique environment that affects the development of a 

standardized and statistically-based approach to sampling procedures and the analysis of 

sampling results. The introduction of Agency microbiologists in each of the Agency’s 

District Offices should bring additional uniformity to the sampling process and mitigate 

this problem. 

The Agency should conduct Intensified Verification Testing (IVT) of Food Contact 

Surface (FCS) and Environmental Sampling in response to samples tested positive for L. 

monocytogenes. These test results should not be incorporated into monitoring or baseline 

data. 

IVT should be triggered when product or contact surfaces test positive for L. 

monocytogenes (for all alternatives presented in the L. monocytogenes interim final rule). 

IVT should also begin if continuing sanitation issues are identified by Agency inspection 

5 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

personnel, or when multiple contact or product positives for Listeria spp. or Listeria-like 

organisms occur. 

The Agency should conduct an annual survey of establishments to gather information on 

production volume and the L. monocytogenes control alternative presented in the L. 

monocytogenes interim final rule selected by establishments that produce RTE meat and 

poultry products. This information should be incorporated into the establishment profile 

captured by the Performance Based Inspection System (PBIS). 

The Agency should verify whether growth inhibition ingredients or anti-microbial agents 

(AMAs) are used appropriately and that product incorporating such ingredients does not 

provide an opportunity for significant microbial outgrowth. One option to assure 

compliance may be for inspection personnel to review the establishment’s documentation 

of any AMA validation study conducted, the parameters used and the study’s findings. 

The Agency should design an audit procedure based upon the Agency’s L. 

monocytogenes guidelines. Assuming the Agency adopts such a procedure, the Agency 

can use its Food Safety Regulatory Essentials (FSRE) training program (FSRE) or other 

approaches to ensure that effectiveness of an establishment’s Standard Sanitation 

Operating Procedures (SSOP) are adequate to control L. monocytogenes. 

LABELING/CONSUMER EDUCATION 

Findings and Recommendations 

Labeling. The industry is not currently using L. monocytogenes incentive labeling 

statements. The incentive labeling provision should remain in the final version of the L. 

monocytogenes interim final rule as an encouragement to industry to declare that their 

product has undergone post-lethality treatments or was treated with anti-microbial agents 

or processes to destroy L. monocytogenes. The Agency should further develop L. 

monocytogenes labeling statements by conducting focus group research studies to 

develop statements that provide flexibility to the industry while still remaining truthful 

and not misleading. 

Consumer Education. The Agency actively continues to base its consumer education 

messages on the latest available science and incorporates social marketing and 

educational principles to reach their targeted at-risk audiences for L. monocytogenes. 

The L. monocytogenes interim final rule stated that in addition to providing education on 

safe handling of food, the Agency would provide information to consumers regarding 

new labels that processors may voluntarily use to inform consumers of interventions used 

to reduce contamination. By continuing these outreach efforts and conducting focus 

6 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

group research, the Agency should be able to develop a consumer education campaign on 

incentive labeling. 

RETAIL ASPECTS 

Findings 

Evidence indicates that slicing and packaging of luncheon meats at retail deli counters 

presents a significant source of exposure to L. monocytogenes. Prevalence reported from 

these sources ranges from 3 to 5 percent in deli meat sliced at retail. Further studies of 

this are needed, since the samples that produced these data were insufficient for statistical 

analysis. 

Delicatessen operations present a potential public health concern but are not currently 

under the active purview of the Agency. Retail delis complied with all Food Code 

controls 73 percent of the time. Proper holding time and temperature were observed over 

43 percent of the time. The opportunity for equipment to become contaminated was seen 

over 79 percent of the time. These are critical controls for L. monocytogenes in RTE 

food. 

Recommendations 

The Agency should increase comparisons of the levels of L. monocytogenes in RTE meat 

and poultry products at the establishments producing these products with the levels of L. 

monocytogenes after the products are sliced at retail. The result of this assessment will 

provide data on the prevalence of L. monocytogenes in RTE meat and poultry at retail. 

This study should include the collection of national and state retail data on the prevalence 

and level of L. monocytogenes in deli meats and risk factors such as retail sanitation, 

product co-mingling, and product formulation. 

Two possible Agency strategies to mitigate risk in retail establishments are suggested: 

Food Service and Retail Training. This effort should focus specifically on L. 

monocytogenes issues, pertaining to proper sanitation, refrigeration, and products of 

concern (for example, uncured poultry rolls). 

Antimicrobial Agent (AMA) Formulations. Seemingly low-risk deli products containing 

AMAs may represent a significant hazard in retail deli operations using sub-optimal 

refrigeration. Even with modest temperature increases, AMAs are far less effective in 

inhibiting outgrowth. Options for federally inspected establishments in preventing 

product contamination and outgrowth in retail operations appear to be limited and may 

not be effective in significantly reducing the likelihood of foodborne listeriosis from deli 

counter products. 

7 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

PUBLIC HEALTH 

Findings 

The Public Health PAT reviewed data on human illness caused by listeriosis and 

determined that there are three important sources of illness information. Each of these 

data sources provides useful information about the occurrence of listeriosis in the U.S. 

population and, in some cases, the food vehicle responsible for those illnesses. 

Recommendations 

National Surveillance Data. The Center for Disease Control (CDC) issues an annual 

surveillance summary of nationally reported diseases that represents the national case 

count of such illnesses. This report contains retrospective data with a two year 

surveillance lag. The surveillance system provides data on the numbers of cases, 

incidence rates (cases per population), and demographic profile (age, race, sex, ethnicity, 

and geography). The Agency should review this data to update listeriosis patient profiles. 

Outbreak Data. The CDC also maintains the Electronic Foodborne Outbreak Reporting 

System (EFORS). This data results from summary reports on outbreaks that were 

investigated by State public health agencies. Although EFORS data may be incomplete, 

investigated outbreaks provide the best opportunity to identify the food vehicle associated 

with illness. As the quality of this data improves, the Agency should match cases with 

food products. This data along with the information collected from the case interview 

form for lab-confirmed cases may enable the Agency to correlate changes in its 

regulations with changes in illness. The Agency should also periodically analyze the 

EFORS data for other opportunities to construct policy to reduce the levels of listerosis. 

FoodNet Data. The Agency should participate in the analysis of FoodNet1 data. The core 

activity of FoodNet is to collect all laboratory confirmed cases of foodborne illness, as 

well as track the trends in such illness. Although FoodNet data may not be nationally 

representative, it provides a rough estimate of the U.S. burden of illness, specifically, the 

incidence of laboratory confirmed listeriosis. For example, in 2003 there were 629 cases 

of listeriosis provisionally reported to CDC (data is not final), yet the FoodNet incidence 

rate for listeriosis (3.3 cases/1 million) provides an estimate of 951 cases. Thus, FoodNet 

may provide a more precise measure for monitoring trends in listeriosis. FoodNet also 

conducts epidemiological studies comparing illness cases to healthy controls. This 

provides a way of identifying demographic characteristics of food exposures that are 

associated with illness. Such ongoing studies allow the identification of emerging food 

vehicles for pathogens that cause foodborne illness, including L. monocytogenes. 

8 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

ECONOMIC IMPACT 

Findings 

Establishment profile data shows a slight increase in the number of federally inspected 

establishments producing RTE meat and poultry product since the publishing of the L. 

monocytogenes interim final rule. There are approximately 2,983 federally inspected 

establishments currently producing RTE meat and poultry products. 

Compliance with the L. monocytogenes interim final rule, as measured by PBIS 

noncompliance data indicates that approximately 76 percent of establishments subject to 

the rule received no Noncompliance Records (NRs) related to L. monocytogenes between 

October 6, 2003 and July 7, 2004. The majority of the L. monocytogenes-related NRs 

(about 53 percent) concern fully-cooked, perishable products. 

Approximately 51 percent of all establishments that produce RTE products are classified 

under the HACCP regulations as very small. About 56 percent of all L. monocytogenesrelated NRs have occurred in these very small establishments. 

FSIS Form 10,240-1, “Production Information on Post-Lethality Exposed Ready-to-Eat 

Products”, will be used to collect data from establishments regarding alternatives they use 

to control L. monocytogenes, the types of RTE products produced, and volumes of the 

production. This information will enable the Agency to update the initial findings used in 

preparation of the economic impact analysis of the L. monocytogenes interim final rule 

to determine capital costs of the alternatives available to establishments to control L. 

monocytogenes. This form will be a more reliable source for this type of information 

than inference from NRs. This data will also enable Agency economists to examine the 

specific variables suggested by National Advisory Committee on Meat and Poultry 

Inspection (NACMPI) when estimating cost of compliance. 

Recommendation 

The Economic Impact PAT was established to support the revision of the Economic 

Impact Analysis conducted for the L. monocytogenes interim final rule. The PAT 

recommends further data gathering to support that analysis in preparation of the final L. 

monocytogenes regulation. 

9 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

TRAINING 

Findings 

Training provided to the Agency inspection workforce to aid in implementation of the L. 

monocytogenes interim final rule included a CD-ROM which focused on the L. 

monocytogenes interim final rule and FSIS Directive 10,240.4, “Verification Procedures 

for the Listeria monocytogenes Regulation and Microbial Sampling of Ready-to-Eat 

(RTE) Products for the FSIS Verification Testing Program”, dated 10/03/2003. A smaller 

pool of inspection personnel were provided with more in-depth training including Food 

Safety Regulatory Essentials (FSRE) classroom training, Intensified Sampling Training, 

and Enforcement, Investigations, and Analysis Officers (EIAO) training. The Agency 

plans to provide additional training when the L. monocytogenes Final Rule is issued. 

In April 2003, the Agency developed and presented a course in College Station, Texas to 

approximately 35 EAIOs. The course, “Intensified Verification Sampling Training” 

covered regulations pertaining to L. monocytogenes. This course also provided an 

overview of L. monocytogenes and presented techniques used in aseptic sampling that 

included a sampling practicum, as well as other information related to sampling. 

A survey conducted by the Training PAT indicated that the District Managers felt the 

implementation of the L. monocytogenes interim final rule is progressing well. They 

recommended that the Agency should continue to distribute new information on L. 

monocytogenes as it becomes available. Although the amount of Intensified Sampling 

conducted by Districts varied, the District Managers felt that the number of trained 

inspection personnel currently available is adequate to address the sampling needs in their 

District. District Mangers also informed on L. monocytogenes issues by attending the 

industry workshops, reviewing the video for the industry workshop, reading the directive 

and attending briefings by members of their District Office staff. Although training 

within individual districts by the Districts Offices varied, District Managers rely on the 

FSRE training of the Consumer Safety Inspectors (CSI) to implement the L. 

monocytogenes interim final rule and the associated Agency directives and to train 

District inspection personnel. 

Recommendations 

FSRE training should be provided to all CSIs. The Agency should consider training all 

in-plant supervisors in FSRE. In addition, CSIs who completed FSRE prior to October 

2003 and employees who have not had the opportunity to attend FSRE training should 

attend this course. Training could be facilitated through technologies such as interactive 

CD-ROM modules. When the rule is finalized, this CD-ROM training module should 

include the full text of the final rule, related Agency directives, and the Agency 

Compliance Guidelines. Additional CD-ROMs should be considered for specific training 

aspects such as L. monocytogenes sampling and guidance when issuing NR related to 

10 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

noncompliance with the L. monocytogenes requirements. One alternative for the delivery 

of L. monocytogenes training to the workforce would be the use of web-based training. 

This delivery mechanism for training offers an interactive format and delivers the training 

much like training delivered by CD-ROM, but enhances this method of training by 

enabling the use of a “Question and Answer” format. 

The Agency should establish a certification program in the methodology for performance 

of intensified and specialized sampling. Certification would require successful mastering 

of the subject’s knowledge and the demonstration of the correct application of sampling 

techniques and other requirements. 

If it is not possible to certify all EIAO inspection personnel, an alternative is to develop a 

cadre of EIAO’s to be certified within each District based upon the number of RTE 

establishments within the District. Once data is obtained on the alternative selected by 

establishments producing RTE meat and poultry product as required by the L. 

monocytogenes interim final rule, this information may also provide a means to assure an 

adequate number of EIAO certified to conduct sampling. Having qualified and certified 

inspection personnel to serve as experts in intensified sampling would enhance the 

Agency’s ability to conduct enforcement actions if required. 

It is recommended that all future training covering L. monocytogenes be tracked and 

mechanisms be put in place to document the successful completion of required training. 

The Agency currently has such tracking mechanisms in place such as “Aglearn” that may 

be adapted to track L. monocytogenes training. 

SMALL PLANT GUIDANCE 

Findings 

The majority of the small and very small plants did not receive or were unaware of the 

Agency Compliance Guidelines document associated with the L. monocytogenes interim 

final rule. Of the small and very small establishments that did receive the Agency’s 

Compliance Guidelines, many needed additional guidance to understand the 

recommendations or experienced difficulty complying with the rule. For example, some 

of these establishments also needed guidance when considering whether their product 

met the criteria of RTE and whether the rule pertained to their operations. 

Recommendations 

FSIS should recognize that very small establishments face special challenges when 

attempting to comply with new requirements. To meet the needs of these establishments, 

the Agency should develop outreach to assure that new information is received in a 

timely manner. FSIS should explore ways to use available technology when providing 

11 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

information such as remote broadcasting and the distribution of videotapes and 

videodiscs of these broadcasts. 

The Agency should examine its current procedures that rely heavily upon the Internet to 

distribute the Agency’s Compliance Guidelines. Some small and very small 

establishments do not have capability or expertise to access documents posted on the 

Agency’s Website. 

The Agency should also conduct additional workshops targeting small and very small 

establishments. These workshops should be scheduled well in advance of the publication 

of the final L. monocytogenes rule. 

The Agency should simplify the Agency’s Compliance Guidelines to enable small and 

very small establishments to easily understand the recommendations. The Agency should 

also provide establishments with guidance on how to reclassify their products from RTE 

to non-RTE products. 

12 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Agency Accomplishments 

The Team was requested to catalog the significant accomplishments of the Agency 

during the implementation of the L. monocytogenes interim final rule. The 

accomplishments listed are not exhaustive but representative of the Agency’s efforts to 

assure understanding of inspection personnel and the industry’s compliance with the new 

requirements. 

Inspector Survey 

The Agency surveyed 1,490 Inspectors-in-Charge (IIC) who cover the over 2,900 

establishments that produce RTE meat and poultry products to see if the establishments 

had made improvements to their L. monocytogenes controls since October 6, 2003. 

Overall, more than 87 percent of these establishments indicated at least one change to 

their process. In addition, about 59 percent of the establishments in the survey started 

sampling for Listeria or Listeria-like organisms on direct food contact surfaces and 27 

percent of the establishments in the survey began using an anti-microbial agent or other 

control process in one or more of the RTE products they produced. Finally the survey 

indicates that over 17 percent of the establishments initiated the use of a post-lethality 

treatment in their RTE production process. 

Verification Sampling 

The Agency’s current sampling program of RTE product schedules randomized samples 

for all establishments producing such product regardless of risk. From January 1, 2004 to 

June 6, 2004, 345 samples have been tested with three samples testing positive for L. 

monocytogenes. 

The Agency also initiated an additional RTE product sampling program at the beginning 

of the 2004 that targeted the riskiest product, at the inspector’s discretion, within any 

given establishment producing RTE meat and poultry product. As of June 6, 2004, 1,349 

of these samples have been tested with 11 samples testing positive for L. monocytogenes. 

Consumer Education 

The Agency actively continues to base its consumer education messages on the latest 

available science and incorporates social marketing and educational principles to reach 

their targeted at-risk audiences for L. monocytogenes. 

The Agency continued to distribute its consumer education messages using multi-media 

channels; e.g. the Hispanic Radio Network, the USDA Food Safety Mobile, “TodoBebe” 

(Telemundo Television Network), radio tours, Web sites, and community health fairs. 

13 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The Agency assisted in revising “Diagnosis and Management of Foodborne Illnesses: A 

Primer for Physicians and Other Health-Care Professionals”2 to include a major section 

on L. monocytogenes. The Agency also published a Spanish version of “Listeriosis & 

Pregnancy: What is Your Risk” printed in April 2003 and posted it on the Agency Web 

site. The Agency updated L. monocytogenes information through The Food Safety 

Educator, EdNet and other e-communications. The Agency’s L. monocytogenes flyer 

aimed at pregnant women was developed and revised to reflect the latest L. 

monocytogenes risk assessment. 

Agency staff members also participated in a joint effort with FDA and CDC to publish a 

comprehensive booklet on L. monocytogenes aimed at all the at-risk groups. The Agency 

is developing a series of educational materials for at-risk audiences, beginning with 

transplant patients, the first to be posted on The Agency Web site in September 2004 for 

National Food Safety Education MonthSM. 

Retail Aspects 

FSIS attended the Conference for Food Protection (CFP), presented a summary of the 

Agency’s recent efforts to combat L. monocytogenes, and included guidance material 

based upon recent risk assessments. FSIS employees also served on a CFP committee 

tasked with developing guidance on the control of L. monocytogenes for retail food 

operations. This guidance will be incorporated into the FDA Food Code, a model set of 

food regulations that has been adopted by more than 40 states and territories in the United 

States of America. 

Training 

The Agency produced and distributed a CD-ROM covering the requirements of the L. 

monocytogenes interim final rule for use by the inspection workforce. The Agency 

developed and presented a classroom course on intensified verification sampling for L. 

monocytogenes in April 2003 to approximately 35 EIAOs. 

The Agency also developed and conducted the Food Safety Regulatory Essentials 

(FSRE) course that addresses L. monocytogenes in RTE meat and poultry products. The 

course is offered routinely and made available to all CSIs. 

Small Plant Guidance 

The Agency conducted Workshops targeted for plant operators of small and very small 

establishments prior to the implementation of the rule. The Agency updated the Agency’s 

Compliance Guidelines to include responses to questions received at the workshops and 

by Agency staff members at the National Technical Services Center located in Omaha, 

Nebraska. 

14 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The Agency revised FSIS FORM 10,240-1, “Production Information on Post-Lethality 

Exposed Ready-To-Eat (RTE) Products”, based upon comments received from industry 

groups to assure its ease of proper completion by establishments. 

Agency staff members made presentations on the L. monocytogenes interim final rule to 

trade organizations, retail organizations, and at public meetings. The Agency submitted 

an article to the Conference for Food Protection that was successfully approved and 

included guidance to revise the Food Code to include methods to control L. 

monocytogenes at retail. The Agency also took steps to assist companies and food 

processing establishments by sharing information on methods and equipment used in 

post-lethality treatments, anti-microbial agents, sanitation, and other L. monocytogenes 

control procedures. The Agency also established a method for establishments to present 

their experiences with new methods and technology and their findings to the Agency. 

Endnotes 

1 FoodNet is an inter-Agency collaboration among CDC, FSIS, the Food and Drug 


Administration (FDA), and 10 State and local health departments.


2
 The Primer was produced collaboratively by the American Medical Association (AMA), the 


American Nurses Association (ANA), the Centers for Disease Control and Prevention (CDC), 


the Center for Food Safety and Applied Nutrition-Food and Drug Administration (CFSAN-


FDA), and the Food Safety and Inspection Service (FSIS) of the United States Department 


of Agriculture. This primer is intended to provide health care professionals with current 


and accurate information for the diagnosis, treatment and reporting of foodborne 


illnesses. The primer also provides health care professionals with patient education 


materials on prevention of foodborne illness.


Attachments 

A. Listeria Assessment Aspects 

B. Project Assessment Team Reports 

15 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Attachment A 

Listeria Assessment Aspects 

Purpose: to identify how to assess and measure the effectiveness of the new regulation, Control 

of Listeria monocytogenes in Ready-to-Eat Meat and Poultry Products; 68 Federal Register, 

34208; June 6,2003. The Agency will use this information in finalizing the interim final rule. 

Seven Aspects 

•

•

•

•

Verification Sampling 

Short term alternative to survey based frame 

meets the special needs of small business 

focus on high risk products 

target large volume establishments 

•

•

•

Labeling/Consumer Education 

incentive based labeling 

consumer education 

vulnerable groups 

•

•

Retail Aspects 

potential risk assessment 

true prevalence on meat and poultry products at retail 

•

•

Public Health 

the impact of the rule on public health? 

what will it take to demonstrate future impact? 

•

Economic Impact 

Verification of the assumptions made for the interim rule 

•

Training 

An explicit and well-articulated plan to evaluate inspection personnel training 

•

Small Plant Guidance 

Are the compliance guidelines assisting small business to meet the requirements 

16 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Attachment B 

Project Assessment Team Reports 

Sampling Verification 

I. Purpose 

Control of Listeria monocytogenes in Ready-to-Eat Meat and Poultry Products (1), FSIS 

Interim final rule, states that FSIS will move toward targeted, risk-based sampling. In this 

action, the Agency is recognizing differing levels of risk posed by ready-to-eat (RTE) 

products produced under Alternatives 1, 2, or 3. This PAT was charged with 

understanding the current sampling scheme, analyzing effectiveness of the Interim Rule, 

describing the movement of the industry since implementation of the Rule, and making 

recommendations for Rule and sampling scenario modifications. 

The PAT also provides plans for risk analysis that incorporate advice from The National 

Advisory Committee on Meat and Poultry Inspection (NACMPI) that FSIS should focus 

on assessment of the three alternatives presented in this report and evaluate their 

effectiveness for risk mitigation. 

II. Major Findings and Recommendations: 

A. Current Sampling: Assessment and Suggested Modifications 

Current activities: 

ALLRTE currently provides a randomized sampling program for all RTE regardless of 

risk. However, the current implementation does not provide a sound scientific basis for 

trend analysis because it is not based on relative production volume. Year to date for 

2004, 345 samples have been tested in ALLRTE, with 3positives. 

At the beginning of the 2004, an existing program called “Target” was replaced with a 

program called “RTERISK1”. RTERISK1 targets the riskiest product, at the inspector’s 

discretion, within any given establishment. As such, it provides some focus on public 

health priorities but is not ideal. Year to date 2004, 1349 samples have been tested in 

RTERISK 1, with 11 positives. 

Conclusions from Lm testing data in CY2004- Apparent prevalence for both sampling 

programs is essentially equivalent, less than or approximately 1%. Interestingly, some 

low-risk products have been found to be positive for Lm. OPHS is still awaiting data on 

the identity of some of these products. 

Issues for consideration with current sampling program: 

1. Baseline surveillance is “the systematic collection, analysis, interpretation, and 

dissemination of health data on an ongoing basis, data which is a description of 

existing conditions to provide a starting point against which progress can be 

17 
______________________________________________________________ 

FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

assessed or comparisons made, for the purpose of gaining knowledge of the 

pattern of occurrence and potential in a community, in order to control and 

prevent disease in the community”. This is not what the current program provides. 

Modifications should be made to drive sampling toward this goal. 

2. Provide data to accurately track trends and improvements in Lm prevalence from 

year to year. Because prevalence based on relative production volume is more 

likely to correlate with incidence of illness, the current ALLRTE monitoring 

program should be modified to be consistent with baseline study design; i.e., 

weighted sampling based on production volume or weighted estimate analysis 

after the fact. Redesigning ALLRTE as a baseline is intended to provide Agency 

management with a reliable tool for tracking progress on product Lm prevalence. 

3. Sampling should confirm acceptable performance by a given establishment. 

4. Surveillance will provide data for refining targeted sampling plans, risk 

assessments, and regulatory policies. 

5. Sampling increases incentive for establishments to implement interventions, to 

reduce the burden of sampling and to reduce the likelihood of positives in those 

samples. 

A statistically based program (as opposed to random regulatory monitoring) will 

provide highly reliable annual data on Lm prevalence for RTE products as a 

whole (i.e., all risk groups, cook-in-bag to Alternative 3). Baseline data illustrates 

a simple picture of a plant on a given day. The composite picture of 2450 plants 

will provide a movie of reality. 

B. Risk Based Sampling: Preliminary Conclusions and Directions 

Risk Factors Identified for L. monocytogenes 

In a prior review of the draft FSIS Directive 10,240.4, several risk factors (e.g., product 

type, etc.) were identified based on the FSIS Listeria risk assessment and the FDA/FSIS 

risk ranking. These risk factors can be used to support risk-based allocation of inspection 

resources. Overall, the relative risk posed by RTE meat and poultry products produced 

by an establishment is due to the type of RTE meat and poultry product produced (i.e., 

deli meat versus dry sausage) and the type of process controls in place after the lethality 

step, but not the volume produced by the establishment. [Note: volume is not a risk 

factor because it would assume that the contamination events, operations, etc. are the 

same among plants. Instead, these other factors override the issue of volume.] 

18 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The following information was used to support the development of language for 

Directive 10,240.4 for verification for L. monocytogenes in RTE meat and poultry 

products. 

1) Risk Based on Type of Product 

The FDA/FSIS Draft Assessment of the Relative Risk to Public Health from Foodborne 

Listeria monocytogenes (Lm) Among Selected Categories of Ready-to-Eat Foods (2002) 

indicated that deli meats posed the greatest per annum risk of illness/death from Lm, 

followed by frankfurters/hot dogs, pate and meat spreads, and dry/semi-dry fermented 

sausages. Note: Under this category, the use of growth inhibitors (discussed below) 

could be also considered (rather than under “processing environment”). 

2) Risk Based on Processing Environments 

The FSIS Listeria Risk Assessment (FSIS 2003) indicated that use of post-lethality 

interventions and use of growth inhibitors has the greatest impact on lowering the risk of 

illness/death from Lm in RTE meat and poultry products, followed by use of postlethality interventions or growth inhibitors, and testing and sanitation of food contact 

surfaces (highest to lowest frequency). 

The Interim Final Rule categorizes RTE meat and poultry product into three alternatives 

depending in their use of post lethality processing, antimicrobial agents and targeted 

sanitation: 

Alternative 1 – Employ both a post-lethality treatment and a growth inhibitor for Listeria 

on RTE products. Establishments opting for this alternative will be subject to FSIS 

verification activity that focuses on the post-lethality treatment effectiveness. Sanitation 

is important but is built into the degree of lethality necessary for safety as delivered by 

the post-lethality treatment. 

Alternative 2 – Employ either a post-lethality treatment or a growth inhibitor for Listeria 

on RTE products. Establishments opting for this alternative will be subject to more 

frequent FSIS verification activity than for Alternative 1. 

Alternative 3 – Employ sanitation measures only. Establishments opting for this 

alternative will be targeted with the most frequent level of FSIS verification activity. 

Within this alternative, FSIS will place increased scrutiny on operations that produce 

hotdogs and deli meats. In a 2001 risk ranking, FSIS and the Food and Drug 

Administration identified these products as posing relative high-risk for illness and 

death. 

The current FSIS risk assessment based its categorization on plant size as defined by 

USDA, which is based on the number of employees. A preliminary analysis was also 

prepared that categorized plants based upon production volume. Currently under 

development is a model that categorizes production into these three proposed alternatives. 

The current scope of work requires the development of such a model along with a risk 

19 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

analysis of the available management interventions: testing frequency, number of 

consecutive positives prior to product testing (test-and-hold), sample size, etc. 

It is likely that other risk factors are present for Listeria monocytogenes contamination, 

but the available data precludes their analysis for the present. These factors might 

include construction at the processing plant, age of the processing plant, building material 

used in the plant, and degree of separation of incoming product with post lethality treated 

product. This work will develop and analyze a quantitative risk assessment using the 

three alternative categories. It will build upon the existing FSIS risk assessment model 

and the FDA/FSIS risk ranking model. 

Due in mid-summer 2004, OPHS scientists are working to complete a risk-based 

sampling scenario, as described here. This scenario will be presented to Agency 

managers for use in drafting a Verification Sampling program, as called for by the 

Interim Final Rule. Many of the recommendations presented in this PAT report are in line 

with the expected outcome of that work. Indeed, the efforts and activities of this PAT will 

not cease with the submission of this report. Much is left to be done. 

III. Continuing Goals of the Verification Sampling PAT

• Refocus sampling on high risk products/as determined by Risk Assessment and 

Alternatives1, 2, 3. 

• Risk-based activity needs to take priority: target large volume establishments/ 

most potential exposure. 

• Outline a method for conducting verification testing until the Survey form clears 

and is answered. 

• Make explicit provisions for avoiding bias in verification testing towards very 

small production operations: a goal of 10 percent of the verification testing could 

include high-risk products from this very small production segment. 

IV. Background 

To protect public health, FSIS is seeking to focus testing and surveillance for Listeria 

monocytogenes in a science-based, risk-prioritized manner, along the requirements 

outlined in Agency Rules and Directives, to ensure that regulatory controls for L. 

monocytogenes are serving at expected levels of success. 

FSIS issued Directive 10,240.4 Verification Sampling Verification Procedures for the 

Listeria monocytogenes Regulation and Microbial Sampling of Ready-to-Eat (RTE)

Products for the FSIS Verification Testing Program.

This “Risk-based” sampling is interpreted as stratifying the total population of servings 

based on the associated risk and public health impact. 

20 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The Risk Assessment Division of OPHS is currently working to develop and analyze a 

quantitative risk assessment using the three alternative categories. It will build upon the 

existing FSIS risk assessment model and the FDA/FSIS risk ranking model. The results 

of this Risk Assessment will not be available until mid-summer 2004. 

Some historical information to keep in mind when considering a Verification Sampling 

Program: 

Microbiological Testing for Listeria monocytogenes in RTE Meat and Poultry 

Products. Since 1987, FSIS has randomly sampled and tested RTE meat and poultry 

products produced in federally inspected establishments for L. monocytogenes. Such 

products testing positive for L. monocytogenes are considered “adulterated” under 

the Federal Meat Inspection Act (FMIA) or the Poultry Products Inspection Act 

(PPIA) (21 USC 453(g) or 601(m)). The combination of declaring L. 

monocytogenes in RTE meat and poultry products an adulterant and continued 

microbiological sampling of these products for L. monocytogenes may have 

contributed to the 44 percent decline from 1989 to 1993 in the rate of illness from L. 

monocytogenes. 

FSIS Notice/Listeria monocytogenes in HACCP Plans. In February 1999, during a 

large outbreak of listeriosis associated with hotdogs and deli meats, FSIS issued a 

notice advising manufacturers of RTE meat and poultry products of the need to 

reassess their HACCP plans to ensure that the plans were adequately addressing L. 

monocytogenes (64 FR 27351). FSIS believes that L. monocytogenes contamination 

is reasonably likely to occur in the production of all RTE meat and poultry products. 

Environmental Testing for Listeria spp. FSIS acknowledges that there may be 

certain processing environments in which L. monocytogenes is not a hazard 

reasonably likely to occur and it is therefore not addressed in an establishment’s 

HACCP system. FSIS believes that in such establishments, verification through 

microbiological testing of food contact surfaces to ensure the establishment’s 

Sanitation SOP in controlling Listeria spp. would be necessary, at a minimum. FSIS 

believes that were an establishment to find Listeria spp. on a food contact surface, 

that finding would be indicative of a sanitation problem that could cause potential 

adulteration of the product (e.g., cross-contamination). FSIS is also proposing to 

require that establishments take certain actions after food contact surfaces test 

positive for Listeria spp. (e.g., those defined in its Sanitation SOP according to 

§416.5). 

Proposed RTE Rule. On February 27, 2001 FSIS issued a proposed rule (66 FR 

39:12590-12636) to require that all establishments that produce RTE meat and 

poultry products conduct environmental testing of food contact surfaces for Listeria 

spp. after lethality treatment and before final product packaging. The focus on the 

non-pathogenic indicator is made because these organisms will be found more 

frequently in the environment than L. monocytogenes and because test results are 

21 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

available more quickly. Finding Listeria spp. would be indicative of a sanitation 

problem even though the contaminant may not be L. monocytogenes. The 

establishment and FSIS will use the test results to verify the efficacy of the 

establishment’s “Sanitation SOPs” in preventing RTE product contamination by L. 

monocytogenes. FSIS also suggested an increased frequency of Listeria spp. testing 

on food contact surfaces for larger establishments. Since neither the suggested 

frequency of testing nor the relationship between testing for Listeria spp. on food 

contact surfaces and L. monocytogenes on the product was based on published 

scientific data, the agency requested comment from the public regarding this ruling. 

V. Summary Recommendations 

A. Suggested Response to an Lm-positive Sample 

If a sample is found to be positive for L. monocytogenes by FSIS testing, ideally a set 

sequence of events should occur. A standardized approach best informs all parties, both 

regulatory and not, of the expectations, provides the most rapid resolution of the 

establishment’s problem, provides for consistency and fairness, and allows for the most 

rapid and complete response in protecting the public health. 

This response should begin with the CSO for the relevant DO who will evaluate the 

HACCP, SSOP and prerequisite programs of the establishment. The CSO then confirms 

proposed corrective actions appear reasonable, consulting OPHS if necessary. CSO takes 

IVT FCS/environmental samples (total number based on number of relevant lines, i.e., 

past OPHS guidelines). 

For first production lot after corrective actions, FSIS should analyze samples consistent 

with ICMSF; Case that is consistent with Compliance Guidelines recommendations (e.g., 

Case 12 for Alternative 3 product distributing to general population). 

a. n = # number of samples taken 

b. for each sample (product?), one 25-g portion is analyzed by FSIS lab. Up 

to four samples may be composited (125 g total, Curiale et al.). For Case 

12, that would be four 125-g test portions each representing five retail 

units (i.e., at least 20 packages taken by the inspector). 

c. For a second Lm-positive finding within a set of 10 subsequent sample 

tests: 

1. FSA Team would evaluate establishment records, plans and 

activities. 

2. The establishment would be removed from the applicable 

targeted testing program above and temporarily placed into 

the intensified testing program (INTRTE). 

a. One 2-lb sample collected each month for 10 

months 

22 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

b. FSIS lab analyzes each monthly sample according 

to ICMSF Case consistent with Compliance 

Guidelines recommendations (see above). 

c. Special consideration should be given to 

circumstances where plant fails again. 

Extraordinary measures may be required. 

B. Suggested Food Contact Surface (FCS) and Environmental Sampling 

There is clear value for establishments to conduct FCS and environmental testing to 

identify possible sources of contamination in the processing environment. FSIS should 

conduct IVT FCS/environmental testing in response to Lm-positive findings, but this 

testing is not appropriate for monitoring or baseline purposes. 

Standardizing the test sample and procedure is inherently difficult. CSOs may introduce 

bias in choice of sampled area and sampling technique. Each plant is a unique 

environment, which confounds development of a standardized and statistically based 

approach to sampling or data analysis. As the number of tested establishments increases, 

the statistical differences may decrease. The introduction of staff microbiologists in each 

district would mitigate this problem (more detail to follow). 

MLG Chapter 8.03 (i.e., FSIS lab method) may not be ideally suited to this analysis as 

opposed to product testing. Products are often protective of sublethally injured Lm. In 

contrast, the presence of sanitizer residues in Lm sponge samples, even with “neutralizing 

agents”, might be relatively more difficult to detect. Establishment might heavily sanitize 

surfaces and mask problems easier in the environment than in product. The MLG 

method provides instruction for sponge analysis but has never been validated for this 

purpose compared to other potential methods. 

What should trigger Intensified Verification Testing (IVTs)? 

1. Product or contact surface Lm positives (for all alternatives). 

2. Continuing sanitation issues identified by CSO (for all alternatives). 

3. Multiple contact or product positives for Listeria spp. or Listeria-like 

organisms 

A. For Alt. 3: two or more positives in any one year 

B. For Alt. 2: three or more positives in any one year 

C. For Alt. 1: five or more positives in any one year 

4. Multiple positive results from an IVT should trigger a Food Safety 

Assessment. 

23 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

While the Listeria Rule mentions IVTs, it does not specify when they will occur. Further 

direction from FSIS managers is needed in order to increase the frequency of IVTs. At 

present, OFO personnel have been tasked with collecting samples for FSIS. This 

sampling requires increased training, funding, etc and may detract from other regulatory 

duties of these personnel. As the agency continues to increase the number of samples 

collected, workload issues could increase. These issues could be solved, in part, by 

hiring microbiologists in each District. The microbiologists could collect samples, 

interact with establishment personnel, and analyze in plant data. They could also 

communicate information to headquarters, and serve as experts on pathogenic organisms 

and processing interventions. If an outbreak were to occur they could meet with state 

health officials, and participate in epidemiological investigations of establishments. 

C. Annual Survey – criteria and justifications 

An FSIS survey of establishments (summer 2004 projected time to be cleared by OMB) 

is critical to gather information on production volume and the Lm control methods used 

by establishments that produce RTE meat and poultry products. This data will be used to 

modify the current FSIS Listeria risk assessment to evaluate risk-based verification 

sampling protocols for deli meats. Sensitivity analyses will include consideration of testand-hold of RTE product, sample size, etc. Since the FSIS survey of establishments is 

contingent on OMB clearance, development of a modified assessment for verification 

sampling, one that incorporates this data, will likely be available Fall or Winter 20041

. 

Such information would be useful for updating FSIS risk-based verification sampling 

protocol, building off ALLRTE and RTERISK1 programs and the recommendations 

made by this PAT. This will also capture reductions in prevalence as establishment 

moves toward Alternative 1 or 2, or new processes where product is no longer exposed in 

the post-process. Risk-based testing programs are dependent on accurate production 

volume data. FSIS Management should consider regulatory options for mandating and 

organizing this information as the establishment profile part of PBIS. 

D. Verification of growth inhibition ingredients or antimicrobial agents 

To verify that validation efforts for antimicrobial agents (AMAs) were appropriate and 

that product incorporating such ingredients does not provide an opportunity for 

significant outgrowth, the most cost-effective option and logistically feasible option is 

review of establishment documentation on validation study parameters and results. This 

PAT recommends the following plan: 

1. OPHS Microbiology Division develops guidance on expectations for proper 

validations based on NFPA guidelines currently in development. 

2. OPHS provides guidance and perhaps training to CSOs on how to review 

validation data. 

1

 OMB approval has been obtained and the survey was conducted after this report was prepared. FSIS is 

analyzing the survey results. 

24 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

3. CSOs screen establishment validation documentation for potential issues. 

4. Suspect validation documents are submitted to OPHS MD MIB for further 

scrutiny. 

5. If OPHS finds the validation design or results to be suspect, FSIS could send an 

OPHS Microbiologist on-site to the establishment to verify establishment 

practices and/or the private contract laboratory that conducted the testing to verify 

laboratory procedures and methods. 

Antimicrobial agents (AMA) 

FSIS could consider a requirement or incentive for FSIS-inspected establishments to 

formulate cook-in-bag or post-lethality treated RTE with AMA to prevent outgrowth 

while opened product is in use in the retail deli counter. However, it is unclear whether 

such a policy would provide a cost-effective public health benefit. With regard to 

certain high-risk products, there are constraints on the effectiveness of current AMAs, 

particularly for potential use of lactate/diacetate in product formulation. Establishments 

that produce certain high-risk products complain that only modest, and therefore likely 

ineffective, levels of lactate are acceptable for incorporation in to their product. 

Attempting to inhibit Lm outgrowth with greater lactate/diacetate combinations and levels 

alters the palatability and marketability of the product. Antimicrobial packaging or other 

antimicrobial surface treatments may not be effective protection for products that are 

eventually sliced at the retail environment. Therefore, current AMA options may not be 

an effective for reducing listeriosis from certain high-risk products, when much of the 

contamination is likely to occur in the retail environment. Other seemingly lower-risk 

deli products containing AMAs may represent a significant risk in retail settings that are 

using sub-optimal refrigeration; even with modest temperature increases, AMAs are far 

less effective in inhibiting outgrowth. In short, options for federally inspected 

establishments in preventing eventual product contamination and outgrowth in retail 

operations appear to be limited; regulatory strategies that that attempt to project the 

responsibility onto FSIS-inspected establishments may not be effective in significantly 

reducing the likelihood of foodborne listeriosis from products later processed in retail 

environments. 

The Potential Use of Formulation Validation 

In general, the only truly effective way to verify validation of an AMA is for FSIS 

laboratories to duplicate conditions of the reported challenge study using inoculated 

product held through the shelf life. Such a study may require packaging after inoculation 

and/or special equipment (e.g., pressure treatment system) that is not readily available to 

FSIS laboratories (i.e., where the testing would have to be conducted). In the event it is 

possible to duplicate the industry study, FSIS pursuit of such a study would be resourceintensive and is not likely to be cost-effective as routine policy. 

25 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

FSIS may want to explore quantitative re-testing of products found to be Lm-positive 

from regulatory qualitative testing. Although this would not necessarily address the 

validation study conducted by a specific establishment or their laboratory, such a study 

would provide valuable data for refining risk assessment conclusions in the future. 

Furthermore, product found to be positive and quantified could be held until the shelf life 

date for an additional quantitative test, which could provide some information on the 

potential for outgrowth in that product under refrigerated storage conditions. 

Although such testing would likely have value, data would have to be evaluated with 

caution. Each of the three tests required above would represent different test portions 

with potentially different levels of contamination; in fact, it is not uncommon for 

quantitative follow-up tests of companion samples to be negative due to heterogeneous 

and sparse levels of Lm. Using multiple or larger test portions may reduce the problem to 

some extent, but a limited amount of product reserve may be available for repeating the 

tests. Despite these obvious limitations, a policy of testing products that may represent a 

higher risk for outgrowth may provide useful data for future Agency efforts. OPHS 

MD/MIB and the FSIS laboratory system could coordinate these testing activities. 

In the past (i.e., early 1990s), FSIS investigated potential disparities in prevalence for 

recently produced product vs. the same product held to the shelf life date. This study 

was unable to conclude that there was a significant increase in Lm prevalence for 

refrigerated products held to their shelf life date. As the study proposed above, this past 

study was not intended to address product validation for a specific establishment and 

product. 

E. Assess the effectiveness of Sanitation SSOPs 

The Agency requires additional information regarding the status of SSOP programs in 

plants producing RTE meat and poultry products. 

Recommendations: Design an audit for SSOPs. The audit can be used by agency 

personnel to determine whether an establishment’s SSOPs are adequate to control L. 

monocytogenes. The audit can be based on FSIS L. monocytogenes Guidelines. 

VI. FSIS RTE Verification Sampling, 2004: A brief overview 

In January 2004, FSIS began two sampling projects, ALLRTE and RTERISK1. All 

establishments with HACCP process categories 03E, 03F, 03G, or 03I, but not the “No 

RTE” block checked in their plant profile were put into the sampling frame for each of 

these projects. There were approximately 2500 establishments meeting these criteria. 

26 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

In ALLRTE, each establishment has an equal chance of being selected each month. A 

predetermined number of establishments are selected, and one sample request form sent 

to the IIC at each selected establishment. On that form, instructions are provided to select 

randomly a product to sample from among all rte products produced at that 

establishment, with the exception of the products identified in the exemption list below. 

The sample is collected and sent to the FSIS laboratory for analysis. All samples are 

analyzed for the presence of Salmonella sp. and Listeria monocytogenes; Escherichia coli 

O157:H7 is also analyzed for in dry and semi-dry fermented sausages and fully cooked 

meat patties. 

As of Feb 18, 2004, 129 samples had been analyzed under the ALLRTE project, with 2 

(1.55%) positive for L. monocytogenes. 

In RTERISK1, each establishment has an equal chance of being selected each month. A 

predetermined number of establishments are selected, and one sample request form sent 

to the IIC at each establishment. On that form, instructions are provided to select a 

product to sample according to the priority list outlined in FSIS Directive 10,240.4, page 

8. The list is as follows: 

If a specific product is not pre-selected for sampling in Block 18 of the sample request 

form, the CSI should sample products based on the following priority: 

a. Post-Lethality Exposed RTE Products under Alternative 3: 

1. Deli meats 

2. Hotdogs 

3. Deli salads, pate, meat spreads 

4. other product 

b. If no post-lethality exposed RTE products are produced using Alternative 3 criteria, 

then sample post-lethality exposed RTE products using Alternative 2 criteria in the 

following order: 

1. Sample product produced using only a growth inhibitor 

2. Sample product produced using post-lethality treatment 

c. If no post-lethality exposed RTE products are produced using Alternative 3 or 2 

criteria, then sample post-lethality exposed RTE products using Alternative 1 

criteria. 

d. If no post-lethality exposed RTE products are produced, then sample any RTE 

product that is not produced using an antimicrobial agent or process and likely will 

be used as a deli-type item, such as a cook-in-bag roast beef. 

e. If none of the above is available, select any other RTE product. 

27 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The sample is collected and sent to the FSIS laboratory for analysis. All samples are 

analyzed for the presence of Salmonella sp. and Listeria monocytogenes; Escherichia coli 

O157:H7 is also analyzed for in dry and semi-dry fermented sausages and fully cooked 

meat patties. 

As of Feb 18, 2004, 494 samples had been analyzed under the RTERISK1 project, with 3 

(0.61%) positive for L. monocytogenes. 

Note, the “1” in RTERISK1 does not apply to anything other than it is the first risk-based 

rte sampling project. It is assumed that the next risk-based rte project would be 

RTERISK2, etc. 

The specific instruction pages for each of these verification sampling projects in FSIS 

Directive 10,210.1 lists products that FSIS would not be sampling on a regular basis, but 

would specifically sample if the need were to arise: 

The specific instruction pages for each of these verification sampling projects in 

FSIS Directive 10,210.1 lists products that FSIS would not be sampling on a 

regular basis, but would specifically sample if the need were to arise. CSIs are 

instructed to RANDOMLY select the product to sample from ALL RTE 

products produced at the establishment, with the exception of the following 

products: Lard, margarine, lard margarine, mixtures of rendered animal fats, 

popped pork skins, pork rinds, dried soup bases, concentrated (high salt content) 

soup mixes, pickled pig’s feet, or product labeled “For Further Processing” in 

which the product is expected to receive a lethality treatment at another 

federally-inspected establishment. (9CFR430.4 still applies to these products if 

they are post-lethality exposed) 

If a sample from either project is found to be positive for one of the pathogens analyzed 

for, the District Manager can request follow-up sampling in that establishment. Multiple 

product, product contact surface, and non-product contact surface, samples can be 

collected and sent to the FSIS lab for analysis. 

Reference: 

Control of Listeria monocytogenes in Ready-to-Eat Meat and Poultry Products 

http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/97-013F.htm 

Federal Register: June 6, 2003 (Volume 68, Number 109)][Rules and Regulations][Page 

34207-34254] From the Federal Register Online via GPO Access 

[wais.access.gpo.gov][DOCID:fr06jn03-20] 

28 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Page 34207 Part V Department of Agriculture Food Safety and Inspection Service 9 

CFR Part 430 “Control of Listeria monocytogenes in Ready-to-Eat Meat and Poultry 

Products”; Final Rule 

29 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Listeria monocytogenes Incentive Labeling Assessment 

I. Major Finding and Recommendation 

The industry is not currently using Listeria monocytogenes incentive labeling 

statements. The incentive labeling provision should remain in the final rule since it 

provides the industry a means of declaring that their product has undergone postlethality treatments or was treated with antimicrobial agents or processes to destroy 

Lm and conveys truthful information to the consumer. The Agency should further 

develop Lm labeling statements by conducting focus group research studies to 

develop statements that would provide flexibility to the industry while still remaining 

truthful and not misleading. 

II. Purpose 

To analyze the incentive labeling portion of the interim rule. 

III. Goals 

To analyze the feasibility of constructing labeling statements that the industry will use 

and, in conjunction with a consumer education program, will provide accurate useful 

information. 

The PAT also considered advice from The National Advisory Committee on Meat 

and Poultry Inspection (NACMPI) that FSIS should also assess the effects of 

incentive labeling activities. 

IV. Background 

The interim final rule provides for the use of voluntary incentive labeling statements 

on products that have been treated with an antimicrobial agent or process that 

eliminates, reduces, suppresses or limits the growth of Listeria monocytogenes (Lm) 

[9 CFR 430.4(e)]. The preamble of the interim rule offered examples of statement 

that could be made, “Sprayed with a solution of sodium lactate to prevent the growth 

of L. monocytogenes” or “Contains sodium diacetate and sodium lactate to prevent 

the growth of Listeria monocytogenes”. Several establishments have submitted labels 

with Lm statements; however, none of the labels used the statements provided as 

examples and, therefore, were returned for further validation information to 

substantiate the claims made. To date, the establishments that submitted labels with 

Lm claims have not further pursued label approval. Recently, industry sponsored a 

survey to evaluate consumer reaction to food safety information statements regarding 

Lm. Industry concluded that the results of the survey show that the use of such 

statements could be confusing or misinterpreted by consumers. Industry has concerns 

that these statements could have adverse affects in the marketplace, (e.g., promote 

30 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

questionable food safety handling practices, drive consumers from food safety 

enhanced products, or cause a reduction in sales these products). 

V. Summary Recommendations 

Conduct a two phased focus group research study on food safety labeling statements 

that would guide the Agency in the development of policy for food safety labeling 

statements. 

Phase I would: 

• Assess consumer’s perceptions, beliefs, and understanding of meat and 

poultry labeling features on which they rely for making purchase decisions 

about the safety of the product, and for food handling/safe preparation, and 

• Assess various types of informational labeling that would help consumers 

choose meat and poultry products that meet their safety concerns and help 

them understand how to safely handle and prepare the products, including 

safe handling statements and statements targeting vulnerable populations. 

Phase II would: 

• Assess the consumer’s perceptions of proposed statements and features, and 

• Collect views on the extent to which they value or would depend on the 

specifically designed statements 

The research protocol should be developed by the Agency and contracted out, similar 

to what has been done in the past with RTI for similar research of this type (e.g., 

conducted in several areas of the country, 4 locations, two groups in each location, 

one group with a high school education or less and one with individuals that have a 

college education, include individuals from various age groups, e.g., 8-30, 35-55, and 

60 years or older). The LCPS submitted the request for the 2006 budget. If 

approved, the research would be conducted and completed in 2006. 

Findings from the focus group research could be shared with industry or used in an 

Advanced Notice of Proposed Rulemaking for comment to help guide the 

development of informative labeling statements that would be useful and valued by 

consumers. The findings will help the Agency, as NACMPI suggested, to assess the 

effects of incentive labeling. 

VI. Summary Findings 

Lm incentive labeling may become a more palatable option for use by the industry if 

statements could be constructed to provide accurate, non-misleading information in 

conjunction with promoting the enhanced food safety features of the product. 

31 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Consumer Education Assessment Report 

I. Major Finding and Recommendation 

The Agency actively continues to base its consumer education messages on the latest 

available science and incorporates social marketing and educational principles to 

reach their targeted at-risk audiences for Listeria. The interim final rule also stated 

that in addition to providing education on safe handling of food, FSIS would provide 

information to consumers regarding new labels that processors may voluntarily use to 

inform consumers of interventions used to reduce contamination. Propose to develop 

a consumer education campaign on incentive labeling following analysis of focus 

group research results. 

II. Purpose 

To analyze the consumer education portion of the interim rule. 

III. Goals & Accomplishments 

To determine if Listeria education efforts committed in the interim final rule were 

carried out. 

• Disseminated Listeria messages through multi-media channels, e.g. 

Hispanic Radio Network, USDA Food Safety Mobile, “TodoBebe” 

(Telemundo Television Network), radio tours, Web sites, local health 

fairs. 

• Completed revision of AMA Primer – with major section on Listeria – 

(made available on April 7, 2004) in cooperation with AMA, ANA, CDC 

& FDA. 

• Spanish version of “Listeriosis & Pregnancy: What is Your Risk” printed 

in April 2003 and posted on FSIS Web site. 

• Updated Listeria information through The Food Safety Educator, EdNet 

and other e-communications. (FSIS Web site Listeria topics page receives 

over 3,500 requests each month.) 

• Worked on International Life Sciences Institute’s (ILSI) risk 

communication subgroup that developed a blueprint of how to reach key 

audiences (March 2004.) 

• Low literacy Lm flyer aimed at pregnant women developed and revised to 

reflect latest Listeria risk assessment. 

• Participating in a joint effort with FDA and CDC to publish a 

comprehensive booklet on Listeria aimed at all the at-risk groups (by the 

end of FY 2004.) 

• Beginning with transplant patients, developing a series of educational 

materials for at-risk audiences, the first to be posted on FSIS Web site in 

September 2004 for National Food Safety Education Month SM. 

32 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

• The PAT also considered advice from The National Advisory Committee 

on Meat and Poultry Inspection (NACMPI) that FSIS should continue to 

work with health professionals to disseminate food safety information. 

IV. Background 

The interim final rule stated that food safety education is one risk management 

strategy the Agency uses to reduce the incidence of illness associated with Lm in 

Ready-to-Eat (RTE) meat and poultry products. Safe handling, storage and 

preparation of RTE meat and poultry products can help reduce the risk of illness, 

particularly for those populations most at risk of contracting listeriosis: pregnant 

women, newborns, older adults, people with weakened immune systems caused by 

cancer treatment, AIDS, diabetes, kidney disease, organ transplants, etc. The Agency 

reaches these audiences through the USDA Meat & Poultry Hotline, the USDA Food 

Safety Mobile, the FSIS Website, most recently the Web-based virtual representative 

(vRep) “Ask Karen,” as well as printed material, electronic communication, the 

media and other information multipliers, in cooperation with other Federal agencies, 

educators, and healthcare professionals. 

V. Summary Recommendations 

Continue ongoing targeted consumer outreach beyond FY 2004 that focuses on 

safe handling, storage and preparation of RTE meat and poultry products for those 

populations most at risk of contracting listeriosis: 

• Continue targeted outreach to at-risk audiences, especially Hispanics and 

immune compromised individuals, as well as pregnant women, newborns 

and older adults, through print media, radio, TV, and Web-based and 

electronic information. 

• Continue to develop educational materials using latest scientific 

information and social marketing principles. 

• Continue to explore innovative methods of delivery utilizing the expertise 

of staff and contractors. 

• Continue to provide funding to ensure successful education and outreach 

programs. 

• Continue to leverage resources through partnerships and alliances. 

• Focus group research studies to assess effectiveness of Listeria messages 

and impact on consumer behavior(s). 

• Continued assessment of effectiveness of delivery methods, and 

revising/expanding outreach to reach wider target audiences, e.g. through 

health professionals, health care providers, caregivers, and peer 

counselors. 

• In FY 04, 05 and 06, continue to utilize print media, radio, TV, Web-site, 

and the USDA Food Safety Mobile in ongoing consumer food safety 

education and outreach activities. 

33 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

The success of these consumer education and outreach efforts will be measured 

by: 

• Continued reduction in the incidence of listeriosis cases per CDC Food 

Net data. 

• Monitoring the number of USDA FSIS Web-site hits; questions about 

Listeria to Web-based vRep, “Ask Karen;” electronic mailbox inquiries; 

publication requests; calls to the USDA Meat & Poultry Hotline, and 

media tracking reports. 

VI. Summary Findings 

The Listeria consumer education and outreach activities are successfully being 

carried out as prescribed in the interim final rule. 

34 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Retail 

I. Summary Findings and Recommendations 

A. Anecdotal evidence suggests that slicing and packaging of luncheon meats at the 

retail environment presents a significant source of exposure to Listeria 

monocytogenes. PAT members reviewed the published literature, Gombas et al., 

as well as unpublished data acquired from New York State. Prevalence reported 

from these sources ranges from 5-3% in deli meat sliced at retail. Further 

investigation of this problem is needed, as the sampling frames that resulted in 

these data were insufficient for statistical analysis. 

B. Retail operations are a significant concern for public health but are not currently 

under the active purview of FSIS. PAT team members reviewed the Database of 

Foodborne Illness Risk Factors2

, a publication from FDA that present Food Code 

compliance rates for the critical controls in the retail food setting. The retail delis 

complied with all Food Code controls only 73% of the time. Proper 

Holding/Time-Temperature was observed only 43.3% of the time and 

contaminated Equipment/Protection from Contamination was seen 79.4%. Both of 

these are critical controls for RTE food. Two possible FSIS strategies to mitigate 

this problem are presented below: 

1. Food Service and Retail Training

Perhaps the most effective strategy for mitigating the problem is offering 

Listeria-specific training to the retail managers, their deli personnel and the 

state officials that oversee their operations. This effort should focus 

specifically on Listeria issues, particularly with regard to proper sanitation, 

refrigeration, and products of particular concern (e.g., those products that 

allow rapid growth of Lm). Including a discussion of liability issues might 

improve attention of all parties to these issues. It is unclear whether FSIS can 

facilitate and/or encourage state public health departments to assume more 

responsibility for these efforts. 

2. Antimicrobial Agent (AMA) Formulations 

Some in the industry and in the published literature describe AMAs in almost 

panacea terms. Others, significant other sources, dispute the efficacy of these 

agents at anything other than unpalatable levels. Furthermore, seemingly lowrisk deli products containing AMAs may represent a significant hazard in 

retail environments using sub-optimal refrigeration; even with modest 

2 Report of the FDA Retail Food Program Database of Foodborne Illness Risk Factors in 

Selected Institutional Foodservice, Restaurant, and Retail Food Store Facility Types 

(2004) http://vm.cfsan.fda.gov/ 

35 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

temperature increases, AMAs are less effective in inhibiting outgrowth. In 

short, options for federally inspected establishments in preventing product 

contamination and outgrowth in retail operations appear to be limited. 

Regulatory strategies that that attempt to project the responsibility onto FSISinspected establishments may not be effective in significantly reducing the 

likelihood of foodborne listeriosis from deli counter products. 

C. PAT members attended the Conference for Food Protection, CFP, and presented a 

summary of the Agency’s recent efforts to combat L. monocytogenes and the best 

guidance gleaned from recent Risk Assessments. PAT members will represent the 

Agency on a CFP committee tasked with developing guidance on the control of 

Lm for retail food operations. This guidance will be incorporated into the 

FDA/FSIS Food Code, a model set of food regulations that has been adopted by 

more than 40 states and territories in the USA. 

II. Purpose 

The Retail Aspects PAT was formed to investigate the prevalence of Lm in RTE 

meat and poultry products at retail, to determine the impact of this contaminated 

product, to describe control programs that may reduce this prevalence and to 

evaluate and advance FSIS and FDA activities currently underway in these areas. 

III. Upcoming Recommendations 

Currently underway efforts in OPHS to compare the risk of listeriosis from deli meat 

sliced in plants versus sliced at retail will continue and the output of this assessment 

will inform FSIS managers regarding the prevalence of Lm in RTE meat and poultry 

at retail. This may include the collection of national and state retail data on the 

prevalence and level of L. monocytogenes in deli meats and associated risk factors 

(e.g., retail sanitation, product formulation, etc.). This assessment is projected to be 

completed in 2006. 

The PAT also considered advice from The National Advisory Committee on Meat 

and Poultry Inspection (NACMPI) that Agencies other than FSIS may have more 

experts on retail issues. FSIS will continue to work with other groups, such as the 

Food and Drug Administration (FDA), The Association of Food and Drug Officials 

(AFDO), the Conference on Food Protection (CFP), and state and local agencies 

experienced in the operations of retail facilities when addressing concerns relating to 

potential contamination of product further processed at retail facilities. 

36 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

IV. References: 

REPORT OF THE FDA RETAIL FOOD PROGRAM DATABASE OF 

FOODBORNE ILLNESS RISK FACTORS 

Prepared by the FDA Retail Food Program Steering Committee 

37 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Public Health Assessment 

I. Major Finding and Recommendation 

The Public Health Program Assessment Team, in its review of data on human illness 

caused by Listeria monocytogenes (listeriosis), determined that there are three important 

sources of illness information, which overlap in their content. Each of the three data 

sources provides useful information about the occurrence of listeriosis in the U.S. 

population and, in some cases, the food vehicle responsible for those illnesses. 

II. Purpose 

This assessment was conducted to determine the available appropriate data sources for 

measuring the impact on human health (morbidity and mortality) of the Lm Interim Final 

Rule. 

III. Goals 

• Identification of potential sources of data on the occurrence of listeriosis in the United 

States. 

• Analysis of the limitations of the identified sources of data. 

• Recommendation to FSIS of an approach to monitoring long-term trends in the 

incidence of listeriosis. 

• The PAT also considered advice from The National Advisory Committee on Meat 

and Poultry Inspection (NACMPI) that, as with Salmonella, FSIS should conduct 

molecular sub-typing and attempt to correlate positive product with actual cases of 

illness. 

IV. Background 

Listeria monocytogenes is a relatively rare cause of foodborne illness, estimated to cause 

about 2500 cases of listeriosis per year in the U.S., although in 2002 (the last year for 

which final national data is available) only 665 cases were reported by State public health 

authorities to the Centers for Disease Control and Prevention (CDC). The estimate of 

2500 cases of listeriosis includes both those persons with mild illness who do not seek 

medical care and those who do seek care but are not diagnosed. Although relatively rare, 

listeriosis is estimated to have a case-fatality rate of 20%, and is unusual among 

foodborne illnesses in that it can be transmitted from a woman to her fetus and 

consequently is associated with either spontaneous abortion, stillbirth, or perinatal illness 

in infants. 

Although approximately 99% of the cases of listeriosis are thought due to food 

exposures, identifying the specific food vehicle for a given case is particularly 

complicated for this illness. The most common illness presentation is not 

gastrointestinal, so there may be delay in identifying the illness as foodborne. Also, 

38 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

listeriosis has an unusually long incubation period (perhaps up to 70 days), so that the 

exposure that resulted in illness may be difficult to recall. Although most cases are 

sporadic and isolated, the increasing use of pulsed-field gel electrophoresis (PFGE) to 

subtype Listeria isolates has resulted in the recognition of clusters or outbreaks, which in 

turn sometimes results in identification of a common food exposure. 

Listeriosis has been associated with meat and poultry products, as well as dairy, produce, 

and other non-FSIS regulated products. For this reason, it will be difficult to identify the 

extent to which changes in reported cases of listeriosis are a result of FSIS regulation and 

the industry compliance with such regulation. 

V. Summary Recommendations 

• National Surveillance Data. The CDC issues an annual surveillance summary of 

nationally notifiable diseases, which represents the national case count of such 

illnesses. The data in this report can be analyzed by FSIS only retrospectively (e.g., 

the report on 2002 data was just published in May, 2004). The data on listeriosis is 

characterized by numbers of cases, incidence rates (cases per population), and 

demographic profile (age, race, sex, ethnicity, and geography). In the Office of 

Public Health Science, the Human Health Sciences Division (HHSD) should take 

the lead on extracting and customizing this data to create a listeriosis profile. 

• Outbreak Data. The CDC maintains the Electronic Foodborne Outbreak Reporting 

System (EFORS), to which State public health agencies contribute summary 

information on outbreaks, which they have investigated. Although EFORS data may 

be incomplete, investigated outbreaks provide the best opportunity to identify the 

food vehicle associated with illness. As the quality of this data improves, with the 

use of PFGE to match cases and food products and with the use of an extensive case 

interview form for all lab-confirmed cases, FSIS will be better able to correlate 

changes in its regulations with changes in illness. Again, HHSD should be 

responsible for periodic analysis of the EFORS data. 

• FoodNet Data. FoodNet is an interagency collaboration between CDC, FSIS, the 

Food and Drug Administration, and several State health departments. The core 

activity of FoodNet is to ascertain all lab-confirmed cases of foodborne illness, as 

well as to track the trends in illness. Although FoodNet data may not be nationally 

representative, it still provides a rough estimate of the U.S. burden of illness, 

specifically, the incidence of lab-confirmed listeriosis. As an example, in 2003 there 

were 629 cases of listeriosis provisionally reported to CDC (data is not final), yet the 

FoodNet incidence rate for listeriosis (3.3 cases/1 million) provides an estimate of 

951 cases. Thus, FoodNet may provide the most precise metric for monitoring trends 

in listeriosis. HHSD is the division responsible for leading the FSIS participation 

in FoodNet and can therefore provide the analysis of this data. 

FoodNet also conducts epidemiologic studies comparing illness cases to healthy 

controls, as a way of identifying the food exposures that result in illness. Studies of 

39 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

this kind, if conducted periodically, will allow identification of novel food vehicles. 

HHSD will be responsible for interpreting the results of these studies for FSIS. 

VI. Summary Findings 

• FoodNet data may be the best measure of the trend in incidence of listeriosis, though 

not precisely representative of the U.S. population. 

• National surveillance data, though not timely, will permit development of a 

demographic profile of listeriosis cases. 

• Outbreak reports on listeriosis provide the least information on incidence but often 

the best information on the associated food vehicle. 

• Two relatively recent innovations in listeriosis surveillance are likely to increase the 

precision of our knowledge about listeriosis cases and outbreaks and the exposures 

responsible for illness: 

• PFGE—this subtyping method allows investigators to link cases to other cases 

and to food products; it is important to note that all FSIS Listeria isolates are 

subtyped using PFGE and posted to the PulseNet database at CDC, a national 

database of all human and food isolates submitted by participating Agencies that 

allows comparison of food and human bacterial isolates. 

• Standardized case interview—use of a standard case interview form has just 

completed pilot testing in several States; when fully implemented, it will allow 

public health investigators to obtain detailed exposure information soon after 

laboratory confirmation of illness. 

VII. Summary Discussion of Findings 

All of the human health data that FSIS will use to evaluate the effect of the regulatory 

changes in ready-to-eat processes are best analyzed over a considerable period of time 

(i.e., years). Short-term, or snapshot, analyses of human illness data are not a reliable 

measure of such changes. A good example of this recently occurred. The 2003 FoodNet 

preliminary data showed a slight increase in listeriosis incidence after a 4-year downward 

trend (Centers for Disease Control and Prevention. Preliminary FoodNet Data on the 

Incidence of Infection with Pathogens Transmitted Commonly Through Food—Selected 

Sites, United States, 2003. MMWR 2004;53:338-343); however, the editorial note in this 

report omitted discussion of this change, since it is difficult to explain a one-year change 

in rates without explicit information of other factors that may have contributed to that 

change. Long-term trends in the incidence of listeriosis reflect changes in regulation, 

industry practices, food consumption, food safety education, demographics, diagnostic 

methodologies, and healthcare-seeking behavior. Consequently, the relative impact of 

the changes in FSIS regulations will be difficult to measure precisely. 

It is important that OPHS-HHSD have regular periodic meetings with OPPED to share its 

analysis of data trends and to respond to specific requests for interpretation of human 

health data. 

40 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

VIII. Attachments 

Here are links to the most recent year’s reports referred to above: 

National surveillance data 

http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5153a1.htm 

Outbreak Data 

http://www.cdc.gov/foodborneoutbreaks/us_outb/fbo2001/summary01.htm 

FoodNet Data 

http://www.cdc.gov/mmwr/preview/mmwrhtml/mm5316a2.htm 

41 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

(

** Data Collection ** 

Data Lm

Data Data Limi for 

Analysis 

Currently Available 

Data 

sed 

of 

the 

of in 

of in the 

population with caveats. 

of a 

the Lm

in 2003, the 

public 

for 2005). 

attributable to a specific food item, 

not

the Lm (

formulation of meat and 

) 

Yes Yes N/A 

National 

provisional 

same time period. 

to 

(s) 

provide the numbers of 

the a 

The annual 

data Yes Yes 

Contact with 

individual states or 

be

providing a more

of Yes, but not 

Yes 

CDC 

Reporting

) 

of either not 

foodborne illness. 

Yes, but 

Yes, and should 

be available 

in a

Pul

database, to which 

contributes 

pulsed-field patterns 

This database

 or 

of or in 

matching from 

and 

method for 

most state labs. 

May for

vehicle. Yes Yes 

i

Use of 

/

assessment/deli 

Provides supporting 

cases 

Supporting

 may

 to 

plant. 

Lm

 to be for a 

provide more and 

;

; data to be 

N/A (has 

proposed; in 

progress) 

1994-1994, 

1998 

devloped 

risk 

ranking 

cosnumption patterns for site 

consumption set is from 1998. food 

) 

No. is willing to No. 

)

 Risk Assessment 

contamination as risk factor 

Provides a 

the 

contamination on 

products, and 

date, and the data are from 

problems. Yes. Yes. 

Public Health Progam Assessment Team Public Health PAT) 

Source Usefulness for Evaluating the Public Health 

Impact of the RTE Rule Strengths tations Data Availability 

Expected to be 

Collected 

Current 

Gap/Propo 

Collection 

FoodNet MMWR 4/30/04 

Compare the incidence 

listeriosis in 2003 with

incidence listeriosis

previous years. 

The FoodNet data provides 

near complete ascertainment 

 listeriosis cases

FoodNet sites; rates may be 

extrapolated to U.S. 

Introduction 

standardized listeriosis case 

history form will increase 

the ability to determine the 

food vehicle associated with 

illness. 

Since RTE Rule went into 

effect October

requirements of this rule may not be 

fully implemented and the

heatlh impact not evident until later 

years (e..g, first quarter listeriosis data 

Listeriosis cases not always 

which means that FoodNet data may 

 have the specificity to directly 

correlate a reduction in listeriosis to 

 FSIS RTE Rule e.g., 

poultry 

products with antimicrobial agents, 

use of post-lethality interventions, etc.

Surveillance of 

Listeriosis 

MMWR Weekly 

Issues; MMWR 

Summary of 

Notifiable Diseases 

Each weekly report provides 

data on listeriosis 

cases from each State, as well 

as comparison to the previous 

year's reports of illness for the 

The weekly data provides 

ability compare case 

counts for the current year 

to the previous year "at-aglance". The annual reports 

cases by month, geography, 

age, sex, race, and ethnicity, 

and the incidence rates by 

age, sex, race, and ethnicity. 

Provisional data should not be used to 

formally evaluate impact of

regulatory change. 

summary data is available as historical 

State 

Surveillance of 

Listeriosis their websites 

For selected states, data may 

 available to FSIS more 

immediately, therefore 

 current 

measure changes in 

incidence of listeriosis cases 

More timely than national 

summary data. 

May require agreements with 

individual states or via a national 

organization such as the Council of 

State and Territorial Epidemiologists. 

directly 

accessible to FSIS. 

Listeriosis 

Outbreak Data 

Electronic 

Foodborne Outbreak 

 System 

(EFORS

Helpful when the food vehicle 

is identified 

Combination

epidemiologic, 

environmental, and lab data 

strengthen the identification 

of a food vehicle. 

Many outbreaks are

atrributed to a specific food or else are 

attributed to mixed foods. Outbreaks 

convey only part of the picture of only current 

through 2001. 

electronically 

 more 

timely manner 

in the future. 

seNet 

CDC's PFGE 

FSIS

 is useful in 

identifying confirming 

clusters illness 

 subtypes

clinical

food/environmental samples. 

Standardized 

subtyping bacterial 

pathogens and subscribed to 

by CDC, FDA, FSIS and 

detect clusters which the 

epidemiologic data is insufficient to 

ascribe cause to a particular food 

Human Illness Data 

Note: Linking listeriosis illness to a specific food exposure is especially problematic because of the unusually long incubation period (up to 30 

days) in this illness. This results in sign ficant limitations in recalling particular foods that might have resulted in illness. Food histories become 

food preferences and so are less useful. 

FDA FSIS risk 

meat pathway 

2003 FDA/FSIS risk 

ranking model 

information on the number of 

listeriosis associated 

with deli meat sliced and 

packaged in-plant vs. at retail 

 information 

that suggest more 

interventions needed at 

retail compared inNFPA retail data used in the FDA/FSIS 

risk ranking came from 2 locations (CA 

and MD). More retail contamination 

data collected FSIS by 

consortium in the next few years to 

representative data 

better estimates of the cases associated 

with deli meat packaged and sliced at 

retail vs. in-plant. 

FDAFSIS risk rankng 

model available NFPA 

data used improved 

retail data not available 

for a few years 

More retail 

collected. 

been 

CSFII 

data 

CSFII data set for 

ready-to-eat products 

by 

FDA/CFSAN for 

FDA/FSIS

If further stratified by region 

(northeast, southeast, etc.), 

may provide insight regarding 

ready-to-eat product by 

regions. 

Provide information that 

may indicate why there is 

a difference in incidence 

of listeriosis by FoodNet 

(e.g., variation in 

pattern/expsoure). 

FDA/CFSAN has this data set and would 

have to be willing to provide the analysis 

of this data set given its complexity. 

Also, the data cannot be stratified by 

state (only by region) and therefore 

cannot be easily correlated to FoodNet 

listeriosis data. Also, CSFII is no longer 

a USDA/ARS survey and the last data 

 Future 

consumption surveys will be conducted 

by DHHS (combined with NHANES

and data is not expected for a few years. 

Yes, if FDA 

analyze. 

Supporting Data 

FSIS 

Environmental 

Sampling Data 

Data from Intensified 

Verification Testing 

(IVT Program 

The FSIS

identified environmental 

for illness. 

connection 

between contamination in 

environment, 

human 

illness. 

Very little data havebeen collected to 

only 

establishments that are experiencing 

42 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Economic Impact 

I. Major Finding and Recommendation 

The analysis produced for this PAT was not designed to produce a major finding 

but to support the RIA. See recommendations for additional analysis below. 

II. Purpose 

To analyze compliance with the Listeria monocytogenes (Lm) regulations to 

support assessment of industry costs. Scope of this PAT’s work was limited after 

assigned economist left the Agency. 

III. Goals 

At a minimum, analysis was intended to update figures used in the interim final 

RIA and show if implementation of the regulations was having a disparate impact 

on firms of a certain size and which aspects of compliance firms are finding most 

difficult. This analysis should support further work estimating the extent and 

distribution of compliance costs to industry. 

The PAT also considered advice from The National Advisory Committee on Meat 

and Poultry Inspection (NACMPI) and agreed that it should focus on differences 

among small, very small and large plants and assess economic impact on very 

small versus large plants, especially whether the rule has caused firms to go out 

of business. In addition, the assessment will consider other variables such as 

product types and the frequency of production. 

IV. Background 

PBIS data was used exclusively for this analysis and the resultant limitations and 

ambiguities are discussed in the “Supporting Data” section of the full report. 

V. Summary Recommendations (list as many as you want) 

Because this analysis used data only from plants that have received Lm-related 

NRs (only 18% of all plants that produce RTE products have received NRs), it 

reveals nothing about what the majority of plants, presumably compliant, have 

done to comply. For example, which alternative for Lm control did each choose 

and how much did it cost to implement? Did these plants have to undertake 

capital improvements to comply? This data must be collected. 

VI. Summary Findings 

• PBIS plant profile data shows a slight increase in the number of official 

establishments subject to the Lm regulations, with approximately 2,951 Federal 

plants producing ready-to-eat products (the interim final RIA listed 2,930). 

43 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

• Compliance with the new regulations, as measured by PBIS data, has been very 

good, with approximately 82% of subject plants receiving no NRs related to Lm 

between 10/06/04 and 4/15/04. 

• Continuing compliance is indicated by these figures: roughly 35% of plants 

receiving Lm-related NRs were cited for not developing plans for controlling Lm 

and many of the NRs were written shortly after the effective date of the rule. Only 

19 of these plants (about 4% of total plants receiving NRs) received additional 

NRs. 

• Whether the rule has disproportionately affected very small plants in terms of 

compliance is uncertain. About 59% of all Lm-related NRs have gone to very 

small plants, but very small plants represent about 51% of plants that produce 

RTE products. And of course, this says nothing about the costs incurred by very 

small RTE plants in compliance. 

• The majority of the Lm-related NRs (about 53%) concerned fully-cooked, 

perishable products. Keep in mind, however, that many of these NRs may 

concern sanitation in general or other processes and may only be coded for this 

procedure. 

• Only a small percentage of plants that received Lm-related NRs (about 3%) need 

to make capital improvements to comply; none of these improvements seem major 

or costly. 

• This dataset does not show that in general plants with repeat NRs ultimately 

needed to make capital improvement to comply (data indicated this in previous 

economic work on the Lm proposal). However, this dataset is relatively small 

and extremely limited in detail regarding plant construction, equipment 

purchases, etc. 

• Of plants with Lm-control alternatives identifiable from NR narratives, most have 

chosen Alternative 3. This data is very inconclusive, however, given the number 

of NRs from which the alternative is not identifiable. 

VII. Summary Discussion of Findings 

none 

VIII. Accomplishments by the Agency 

N/A 

IX. Attachments 

• Analysis of Compliance Data 

• Updated Analysis of Compliance Data 

44 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

Economic Impact PAT – Analysis of Compliance Data 

Introduction 

Below is an analysis of compliance with the Listeria monocytogenes (LM) regulations in 

9 CFR 430 since the interim final rule became effective on October 6, 2003 through April 

15, 2004. Analysis is intended to show if implementation of the regulations is having a 

disparate impact on firms of a certain size, which aspects of compliance firms are finding 

most difficult, how plants are responding to LM-related NRs, and other information 

related to compliance. This analysis should support further work estimating the extent 

and distribution of compliance costs to industry. 

PBIS data was used exclusively for this analysis and the resultant limitations and 

ambiguities are discussed in the “Supporting Data” section. Further, because this 

analysis uses data only from plants that have documented noncompliance (only 18% of 

all plants that produce RTE products have received NRs), it reveals nothing about what 

the majority of plants, presumably compliant, have done to comply. For example, which 

alternative for LM control did each choose and how much did it cost to implement? Did 

these plants have to undertake capital improvements to comply? These and other 

questions are discussed in the “Additional Questions” section at the end of this document. 

Findings 

• PBIS plant profile data shows a slight increase in the number of official 

establishments subject to the LM regulations, with approximately 2,951 Federal 

plants3

 producing ready-to-eat products (the interim final RIA listed 2,930). 

• Compliance with the new regulations, as measured by PBIS data, has been very 

good, with approximately 82% of subject plants receiving no NRs related to LM 

between 10/06/04 and 4/15/04. 

• Continuing compliance is indicated by these figures: roughly 35% of plants 

receiving LM-related NRs were cited for not developing plans for controlling LM 

and many of the NRs were written shortly after the effective date of the rule. 

Only 19 of these plants (about 4% of total plants receiving NRs) received 

additional NRs. 

• Whether the rule has disproportionately affected very small plants in terms of 

compliance is uncertain. About 59% of all LM-related NRs have gone to very 

small plants, but very small plants represent about 51% of plants that produce 

RTE products. And of course, this says nothing about the costs incurred by very 

small RTE plants in compliance. 

• The majority of the LM-related NRs (about 53%) concerned fully-cooked, 

perishable products. Keep in mind, however, that many of these NRs may 

3

 See “Supporting Data” for a breakdown of plants by FSIS size classification. 

45 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

concern sanitation in general or other processes and may only be coded for this 

procedure.4 

• Only a small percentage of plants that received LM-related NRs (about 3%) need 

to make capital improvements to comply; none of these improvements seem 

major or costly. 

• This dataset does not show that in general plants with repeat NRs ultimately 

needed to make capital improvement to comply (data indicated this in previous 

economic work on the LM proposal). However, this dataset is relatively small 

and extremely limited in detail regarding plant construction, equipment purchases. 

• Of plants with LM-control alternatives identifiable from NR narratives, most have 

chosen Alternative 3. This data is very inconclusive, however, given the number 

of NRs from which the alternative is not identifiable. 

Supporting Data 

TABLE 1: Total Number of Plants Producing RTE Products as of 4/15/04 

Plant size # of Plants producing RTE products 

as of 4/15/04* 

% 

Total 

of 

VS 1510 51% 

S 1204 41% 

L 118 4% 

Unknown (“N” in PBIS) 119 4% 

Total 2951 100% 

* Approximate because a few of these plants may can product exclusively. 

TABLE 2: Individual Plants that Received LM-related NRs between 10/6/03 and 4/15/04 

* Approximate because a 

few of these plants may 

can product exclusively. 

TABLE 2: Individual 

Plants that Received LMrelated NRs between 

10/6/03 and 4/15/04 

Plant size 

Number of individual plants that 

received LM-Related NRs 

% of 

Total 

VS 329 62% 

S 178 34% 

L 15 3% 

Unknown (“N” in PBIS) 7 1% 

Total 529 100% 

4

 I also have yet to count the number of RTE plants that are scheduled for each inspection procedure; it 

could be that a disproportionate number of RTE plants produce fully-cooked, perishable product. 

46 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

TABLE 3: Total Number of LM-related NRs Issued between 10/6/03 and 4/15/04 

Plant size LM-related NR issued % of 

Total 

VS 419 59% 

S 258 36% 

L 21 3% 

Unknown (“N” in PBIS) 14 2% 

Total 712 100% 

TABLE 4: Percentage of Plants in Each Size Category that Received LM-related NRs 

between 10/6/03 and 4/15/04 

Plant Size 

# of Plants producing 

RTE products as of 

4/15/04* 

Number of individual 

plants that received LMRelated NRs 

% of total plants in size 

category that received 

NRs 

VS 1510 329 22% 

S 1204 178 15% 

L 118 15 13% 

Unknown 119 7 6% 

Total 2951 529 18% 

-- An example for interpreting data from TABLES 1 through 4 -- 51% of plants that 

produce RTE product are very small; 62% of plants that received LM-related NRs were 

very small; 59% of all LM-related NRs went to very small plants; 22% of all very small 

plants received LM-related NRs. 

TABLE 5: NRs by Procedure Code and Plant Size Issued between 10/6/03 and 4/15/04 

Procedure Code VS S L Unknown Total 

01A01 7 6 1 0 14 

01B01 11 10 0 1 22 

01B02 12 7 0 0 19 

01C01 22 13 0 1 36 

01C02 29 25 0 3 57 

03A01 23 17 2 0 42 

03B* 3 2 0 1 6 

03C* 1 1 0 1 3 

03F01 29 13 0 0 42 

03F02 28 4 0 0 32 

03G01 129 79 12 5 225 

03G02 96 49 4 0 149 

03H01 2 1 0 1 4 

03H02 0 1 0 0 1 

03I01 3 0 0 0 3 

03I02 1 0 0 0 1 

04C01 0 1 0 0 1 

05A01 0 3 0 0 3 

06D01 16 19 2 0 37 

06D02 0 1 0 0 1 

08S01 1 0 0 0 1 

No Code 6 6 0 1 13 

Total 419 258 21 14 712 

47 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

*These NRs, dealing with raw product are either erroneously attributed to the 

LM regulations by inspection personnel or written in such a way that they 

resulted from the PBIS queries designed for this study. 

TABLE 6: Percentages of LM-Related NRs by Procedure Element for 10/6/03 to 4/15/04 

53 

13 10 6 6 5 2 2 1 1 1 0 0 0 0 

0 

10 

20 

30 

40 

50 

60 

03G 

01C 

03F 

01B 

03A 

06D 

01A 

Unknown 

H 

03I 

03C 

04C 

05A 

08S 

f l

03B 

03

Procedure Code 

P e r c e n ta g e o T o ta N R s 

TABLE 7: Repeat LM-Related NRs (not necessarily linked) Issued between 10/6/03 and 

4/15/04 

Plant size 1 NR (no repeat) 2 NRs 3 NRs 4+ NRs 

VS 257 57 12 3 

S 132 28 8 10 

L 10 4 1 0 

Unknown (“N”) 5 0 1 1 

Total 404 89 22 14 

48 
FOOD SAFETY AND INSPECTION SERVICE 

WASHINGTON, DC 

September28, 2004 

TABLE 8: Needed Improvements to Control LM based on NR Issued between 10/6/03 

and 4/15/04: 

Distribution of Plants by Size, Repeat NRs, and Identified LM Control Alternatives 

Plant Size # of Need Need Only Changes to Plan Design or Need Capital Improvements to Control LM Despite Other 

NRs Plan* Implementation Plan *** 

Alternative 

Plan 1 

Alternative 

Plan 2 

Alternative 

Plan 3 

Plan 

Unknown 

** 

Alternative 

Plan 1 

Alternative 

Plan 2 

Alternative 

Plan 3 

Plan 

Unknown 

VS 1 124 2 21 42 44 4 20 

2 13 1 4 19 15 1 3 1 

3 3 8 1 

4+ 1 1 1 

S 1 37 6 4 23 32 4 26 

2 3 3 7 12 1 2 

3 1 1 2 4 

4+ 1 2 6 1 

L 1 1 1 7 1 

2 1 2 1 

3 1 

4+ 

Unknown 

(“N” in 

1 2 3 

2 

PBIS) 3 1 

4+ 1 

Total 1 164 8 26 65 86 9 46 

2 16 1 7 27 29 1 5 3 

3 3 1 1 11 5 1 

4+ 2 4 7 1 

All 183 10 36 107 127 1 16 49 

*Plants falling into this category of noncompliance had not yet developed any procedure 

to address LM and so, from analysis of the NR narrative, it is often impossible to 

determine which alternative they will choose. 

** It is impossible to tell from these written NRs which alternative the plant has chosen, 

although it is clear there is a plan in place. 

*** These NRs often reflect positive FSIS tests with no indication of the control 

alternative chosen or other violations which, in the opinion of the inspector, created a risk 

of LM contamination. 

Ambiguity of PBIS Data and Consequent Analysis Problems 

Much of the data in TABLE 8 was collected through analysis of NR narratives. In many 

cases, NR narratives contain ambiguous of insufficient data for determining the exact 

type of noncompliance recorded, although it is usually clear that noncompliance did 

occur. For example, one NR narrative in the dataset reads: At 1300 hours as I was 

performing a review of this establishment prerequisite program for the control of Listeria 

monocytogenes in post lethality RTE product, I observed the following deficiency. The 

program does not address the hold procedures to be implemented if a test result were to 

show positive for the presence of LM or L species on a product contact zone as per the 

requirements of CFR 430.4 b 3.The narrative states that the deficiency is with the 

establishment’s prerequisite program. The definition in 430.1 states that a prerequisite 

49 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

program is “considered by scientific experts to be prerequisite to a HACCP plan.” It is 

not a HACCP plan and may or may not be a Sanitation SOP. Yet, the inspector who 

recorded this noncompliance coded it as a violation of 9 CFR 417.3 (b), which concerns 

unforeseen deviations within HACCP, and also of 430.4(b)(3), which concerns the use of 

sanitation measures only to control Listeria monocytogenes contamination. So, we can 

infer that the establishment has chosen alternative 3 from the inspector’s citation of 9 

CFR 430.4(b)(3), but we cannot tell whether it chose to use sanitation measures as part of 

a prerequisite program, a Sanitation SOP, or a HACCP plan. This is significant in that 

we could assign costs to the revision of each of these plans. 

Also, it often is impossible to tell from NR narratives which regulatory alternative an 

establishment has chosen to control LM. In many cases, an inspector will describe an 

incident of noncompliance related to an LM control plan or testing, but provide too little 

information for determining which alternative the establishment has chosen. In other 

cases, narratives indicate that an establishment has chosen two plans (most often 2 and 3), 

which is hypothetically possible, but seldom illustrative of what the establishment is 

actually doing. 

Finally, inspection personnel often discuss several types of cited noncompliance in a 

narrative but can only enter one procedure code. Also, inspection personnel often 

perform a scheduled procedure, find an incident of noncompliance that would be 

classified under a different procedure code, but still code the noncompliance with the 

code of the scheduled procedures. For these reasons, the data in TABLE 5 should be 

used conservatively – the high level of O3G noncompliance does not necessarily mean 

that plants are having the most trouble controlling LM during the production of 

perishable, RTE products. 

Additional Questions 

As stated in the Introduction, this analysis uses data only from plants that have 

documented noncompliance regarding LM since 10/6/03. So, it reveals nothing about the 

82% of all plants that produce RTE products and that have received no NRs since then. 

Numerous questions about these plants and all plants that produce RTE products remain 

and some are suggested below. 

• What is the cost to revise a prerequisite program, Sanitation SOP, or HACCP to control LM? 

• What is the cost of each of the alternatives and the elements therein (e.g., testing, LM growth 

inhibitors, etc.)? Are there economies of scale associated with each? 

• As discussed in the interim final RIA, did plants already with LM controls or testing in place 

change practices because of the rule? Did their practices become more or less rigorous? 

• Did plants undertake capital improvements to better comply with the interim final rule? 

• Did plants have to purchase or lease additional storage space because of the hold-and-test 

requirements of the interim final rule? 

• Will FSIS of plant testing results cause plants to change alternatives? 

50 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

Economic Impact PAT – Updated Analysis of Compliance Data 

Introduction 

Below is an analysis of compliance with the Listeria monocytogenes (LM) regulations in 

9 CFR 430 from the effective date of the interim final rule on October 6, 2003 through 

July 12, 2004. This analysis updates the initial analysis conducted using compliance data 

for the period between October 6, 2003, and May 6, 2004. Analysis of compliance data 

shows, to some extent, whether implementation of the regulations is having a disparate 

impact on firms of a certain size, which aspects of compliance firms are finding most 

difficult, and other information related to compliance. This analysis should support 

further work estimating the extent and distribution of compliance costs to industry. 

PBIS data was used exclusively for this analysis and the resultant limitations and 

ambiguities are discussed in the original report. Further, because this analysis uses data 

only from plants that have documented noncompliance (only 24% of all plants that 

produce RTE products have received NRs so far), it reveals nothing about what the 

majority of plants, presumably compliant, have done to comply. For example, which 

alternative for LM control did each choose and how much did it cost to implement? Did 

these plants have to undertake capital improvements to comply? These and other 

questions need to be addressed by Agency economists. 

Findings 

• PBIS plant profile data shows a slight increase in the number of official 

establishments subject to the LM regulations, with approximately 2,983 Federal 

plants producing ready-to-eat products (the interim final RIA listed 2,930). 

• Compliance with the new regulations, as measured by PBIS data, has been very 

good, with approximately 76% of subject plants receiving no NRs related to LM 

between 10/06/04 and 7/12/04. Of course, this data indicates nothing about the 

cost of compliance, except that it seems to be affordable to a majority of plants. 

• Whether the rule has disproportionately affected very small plants in terms of 

compliance is uncertain. About 56% of all LM-related NRs have gone to very 

small plants, but very small plants do represent about 51% of all plants that 

produce RTE products. And of course, this data shows nothing about the costs 

incurred by very small RTE plants in compliance. 

• The majority of the LM-related NRs (about 53%) concern fully-cooked, 

perishable products. It is likely that a proportionate percentage of RTE products 

are perishable and fully-cooked (and unlikely that establishments are having a 

disproportionately high number of problems controlling LM in these products). 

51 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

The survey data on product type and production volume should better explain this 

trend in noncompliance. 

• We have not updated the initial findings regarding LM-control alternatives 

selected by plants or regarding capital costs. Forthcoming FSIS Form 10,240-1, 

Production Information on Post-Lethality Exposed Ready-to-Eat Products, will 

be used to collect data from plants regarding alternatives used, types of RTE 

products produced, and volumes of production. This survey will be a more 

reliable source for this type of information than inference from NR narratives 

(used in the previous report). This survey data also can be used by Agency 

economists to examine the specific variables suggested by NACMPI for 

estimating cost of compliance. 

Supporting Data 

TABLE 1: Total Number of Plants Producing RTE Products as of 7/8/04 

Plant size # of Plants producing RTE products 

as of 7/8/04* 

% 

Total 

of 

VS 1525 51% 

S 1214 41% 

L 119 4% 

Unknown (“N” in PBIS) 125 4% 

Total 2983 100% 

* Approximate because a few of these plants may can product exclusively. 

TABLE 2: Individual Plants that Received LM-related NRs between 10/6/03 and 7/12/04 

Plant size Number of individual plants that 

received LM-Related NRs 

% of 

Total 

VS 433 60% 

S 264 37% 

L 17 2% 

Unknown (“N” in PBIS) 10 1% 

Total 724 100% 

52 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

TABLE 3: Total Number of LM-related NRs Issued between 10/6/03 and 7/12/04 

Plant size LM-related NRs issued % of 

Total 

VS 608 56% 

S 423 39% 

L 34 3% 

Unknown (“N” in PBIS) 18 2% 

Total 1083 100% 

TABLE 4: Percentage of Plants in Each Size Category that Received LM-related NRs 

between 10/6/03 and 7/12/04 

Plant Size 

# of Plants producing 

RTE products as of 

7/8/04* 

Number of individual 

plants that received LMRelated NRs 

% of total plants in size 

category that received 

NRs 

VS 1525 433 28% 

S 1214 264 22% 

L 17 119 14% 

Unknown 10 125 8% 

Total 724 2983 24% 

-- An example for interpreting data from TABLES 1 through 4 -- 51% of plants that 

produce RTE product are very small; 60% of plants that received LM-related NRs were 

very small; 56% of all LM-related NRs went to very small plants; 28% of all very small 

plants received LM-related NRs. 

53 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

TABLE 5: NRs by Procedure Code and Plant Size Issued between 10/6/03 and 7/12/04 

Procedure 

Code VS S L Unknown Total 

01A01 7 10 1 0 18 

01B01 19 14 2 1 36 

01B02 16 11 2 0 29 

01C01 35 28 0 1 64 

01C02 51 33 0 3 87 

03A01 28 27 3 0 58 

03B* 3 2 0 1 6 

03C* 2 1 0 1 4 

03E01 1 5 0 1 7 

03E02 5 3 0 0 8 

03F01 45 20 0 0 65 

03F02 33 7 1 0 41 

03G01 190 132 13 6 341 

03G02 143 85 4 3 235 

03H01 2 1 0 1 4 

03H02 2 1 0 0 3 

03I01 3 3 0 0 6 

03I02 1 0 0 0 1 

04B01 1 3 0 0 4 

04C01 0 2 0 0 2 

05A01 0 3 0 0 3 

06D01 21 31 8 0 60 

06D02 0 1 0 0 1 

08S01 0 0 0 0 0 

No Code 0 0 0 0 0 

Total 608 423 34 18 1083 

*These NRs dealing with raw product are either erroneously attributed to the 

LM regulations by inspection personnel or written in such a way that they 

inadvertently resulted from the PBIS queries designed for this study. 

54 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

Training 

I. Major Finding and Recommendation 

Training provided to the FSIS inspection workforce to aid in implementation of 

the Lm (Listeria monocytogenes) interim rule has been limited in scope. The 

training that FSIS has provided is in several different forms, including FSRE 

classroom training, Intensified-Sampling Training, EIAO training, and the 

distribution of information in the form of a CD-ROM which focused on the Lm 

interim rule and directive. 

The Food Safety Regulatory Essentials (FSRE) course, week 3, Ready-to-Eat/Not 

Ready-to-Eat (RTE/NRTE) provides training to in-plant Consumer Safety 

Inspectors (CSIs). FSRE contains an approximately 3 1/2 hour module which 

explains the Lm Directive and Regulation 430 in depth, and focuses on the CSI’s 

inspection verification responsibilities relative to the Lm Directive and regulation. 

Another 3 hour module, RTE Product Sampling, covers the CSI’s responsibilities 

for taking samples, and reacting to positive results. Additionally, a 2 hour module 

on RTE Sanitation covers the establishment responsibilities for sanitation, 

including control and prevention of Listeria in RTE products. These modules are 

part of a three week course which provides the CSI with training in the FSIS 

Sanitation and HACCP regulatory process. CSIs that successfully complete this 

course should be prepared to enforce the Lm regulations and take RTE product 

samples. 

As of August 26, 2004, 1503 CSIs have completed FSRE training. 1,033 (one 

thousand thirty-three) students have completed FSRE training (week 3) since the 

release of the new interim directive in October 2003. 

The agency is in the process of providing the FSRE training to all current CSIs. 

Following completion of this process, all employees promoted from Food 

Inspector to CSI will be required to complete the FSRE course. New employees 

hired after May 1, 2004 are required to successfully pass the final exam as a 

condition of employment, although all students take the exam. Also, all new 

Public Health Veterinarians (PHVs) are required to successfully complete the 

FSRE course. Currently, the course is not required for other PHV, or Front-line 

Supervisors, although Districts are sending some of these current supervisors to 

the training. 

In conjunction with the effective date and implementation of the Lm Interim rule, 

FSIS provided a CD-ROM (October 2003) to the field inspection force. The CD 

contained the documents pertaining to the Lm Interim rule and was intended to 

assist field inspectors who normally use a dial-up connection to access these 

55 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

documents. The CD-ROM was sent to IICs and may not have reached all the 

CSIs who needed it. Plans were, at that time, to provide more comprehensive 

training when the Lm final rule is issued. 

In April 2003, a group of approximately 35 FSIS employees were trained in 

Intensified Sampling Procedures. This 3 day course provided these employees the 

training needed to take product, food contact surface, and environmental samples 

for the purpose of intensified sampling. This course was held prior to the release 

of the current Lm Directive and Regulation 430. The course was conducted once, 

in April 2003, and thus there is a limited pool of employees trained in intensified 

sampling procedures. 

The current EIAO (formerly CSO) training includes an overview of the Lm 

Directive and Regulation 430. All EIAOs are required to successfully complete 

this training, which includes this overview. 

II. Purpose 

To determine the extent of training information and activities provided to the 

FSIS inspection workforce in implementation of the Lm interim rule. Also, to 

make a determination of future training needs. 

III. Goals 

Our goal was to determine the amount of training to provide for implementation 

of the Lm rule, and to describe the adequacy and quality of that training. 

The PAT also considered advice from The National Advisory Committee on Meat 

and Poultry Inspection (NACMPI) and the Team decided to review whether 

accountability measures are adequate to ensure that those who participate in the 

training achieve some mastery over the subject. 

IV. Background 

Data was gathered about the amount of training that was provided to the FSIS 

inspection workforce, the adequacy and quality of that training. A survey of 5 

District Managers was also conducted to determine their views and satisfaction 

with the training that was provided in conjunction with the release and effective 

date of the Lm interim rule. 

56 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

V. Summary Recommendations 

FSRE Training 

The current project of training all CSIs in FSRE must continue until completed. 

Thereafter, all new CSIs must be trained in FSRE. Also, the Agency (FSIS) 

should consider training all the in-plant supervisors in FSRE. For CSIs who have 

completed FSRE prior to October 2003, and for employees who have not had the 

opportunity to attend FSRE yet, CFL will be issuing an interactive training CDROM to provide training to inspection personnel that do not attend the classroom 

training of FSRE (week 3). 

Interactive Training CD 

To complement the previous training on the Lm Interim rule and directive for 

ready-to-eat (RTE) products, an interactive training CD-ROM will be 

developed and issued. The CD-ROM training module will include the Lm Final 

Rule, associated directive(s), compliance guidelines, and also highlight the 

main points of these resources. The CD-ROM would be broken up into sections, 

each section ending with a set of questions to assure subject knowledge. 

For long term training, CD-ROMs or web-based training should be developed and 

used for specific training aspects such as Lm sampling, NRs relative to Lm, and 

the key elements and specifics of the Lm rule and directives. 

“Certification” 

The Agency should establish a “Certification” program in the methodology for 

performance of intensified and/or specialized sampling. When FSIS personnel 

are “certified”, one option would be to train all EIAOs in sampling and require 

their “Certification”. Certification would require successful completion of the 

subject knowledge and demonstration of correct application of sampling 

techniques, etc. 

This ‘certification’ in intensified sampling would require the development of 

training modules that contain core subject knowledge, methods and techniques of 

sampling, and demonstration of sampling techniques and proper application. 

An alternative to requiring certification for all would be to establish a minimum 

number of EIAO’s with “certification” in each district based upon the number of 

RTE establishments within the district. Once data is obtained associated with 

‘Alternative’ selection under the Lm rule, risk may also be a component factored 

into the equation to determine a minimum/adequate number of certified sampling 

EIAOs. Also, the potential of qualifying “certified” inspectors, EIAOs, as an 

expert in the subject of intensified sampling would be beneficial to FSIS in 

enforcement actions. 

57 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

Tracking Training 

It is recommended that all future training be tracked and mechanisms be put in 

place to show that an individual has successfully completed the required training. 

The new “AgLearn” system may possibly be used to track this training. FSIS also 

has current tracking mechanisms in place that can be used to track all Lm training. 

Web-based Training 

One alternative for the delivery of Lm training to the workforce would be the use 

of web-based training. There are two options for the web-based training. The 

website could: 

(1) Provide training in an interactive web-based format and deliver the 

training much like training delivered by CD-ROM. 

(2) Be designed to deliver training in a “Question and Answer” 

format. 

VI. Summary Findings 

• Training provided to the FSIS inspection workforce to aid in 

implementation of the Lm interim rule has been limited. In conjunction 

with the effective date and implementation of the Lm Interim rule, FSIS 

provided a CD-ROM to the field inspection force. The CD-ROM 

information was a recap of the Lm Interim rule. Plans were, at the time 

the CD-ROM was issued, to provide much more comprehensive training 

when the Lm final rule is issued. 

) • The third (3rd week of FSRE (Food Safety Regulatory Essentials) 

provides training and information relative to control and prevention of 

Listeria in RTE products. 

• In April 2003, FSIS developed and presented a course in College Station, 

Texas to approximately 35 EAIOs. The course subject was “Intensified 

Verification Sampling Training”. This course addressed regulations 

pertaining to Listeria monocytogenes, provided an overview of Listeria 

monocytogenes, taught techniques used in aseptic sampling, and provided 

a sampling practicum, as well as other information related to sampling. 

• A survey of District Managers was conducted to determine perceptions of 

training and guidance for implementing the Lm interim rule. The 

following questions were asked on the survey. Responses to the survey 

are included in Attachment 3. 

1. As a District Manager, do you feel that you were provided 

adequate guidance related to intensified sampling triggers? 

What type of actions, if any, have you taken subsequent to 

positive Listeria monocytogenes findings (FCS or product)? 

2. What was the specific case or situation in which action was 

taken? What criteria, if any, are used to initiate these actions? 

58 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

3. As a District Manager, how did you receive your training 

associated with the Lm interim final rule? Was it adequate? 

4. What training was done related to the issuance of the Lm 

Verification Directive in your district for CSIs? Was a district 

standard set to conduct work unit meetings, individual review of 

the directive, Q&A sessions? This training is in addition to 

formal FSRE material delivered by CFL. 

5. What tools were used to assess the district’s level of 

understanding by the CSI’s related to (about) Lm? 

verification/sampling? Based upon the assessment, how would 

you best categorize the CSI’s knowledge base? 

1) Most CSIs definitely need training 

2) Most CSIs could use training but are performing at an 

acceptable level (doing okay without it) 

3) Most CSI don’t need training 

6. As the District Manager, have you assumed that EIAOs are the 

subject matter experts related to Lm? Is additional training 

required for this job series? 

7. Do you have adequate access to employees trained in intensified 

sampling when you need their specialized skill set for sampling? 

How many trained individuals are at the District Manager’s 

disposal to initiate intensified sampling? 

8. As a District Manager, (related to the question above), are you 

confident in their abilities based upon observation or repeated, 

successful performance of this specific skill set (sampling)? 

Have other individuals (other than those attending the formal 

training at CFL/Texas) expressed interest in FCS/intensified 

sampling methodology? If so, were others trained within your 

District (e.g., train the trainer)? 

Summary of District Manager Questionnaire 

The Agency should continue to distribute new information on Lm as it becomes 

available. The new information can be used by District Office personnel to 

evaluate triggers for intensified sampling. The amount of Intensified Sampling 

conducted by Districts varies. One district had not done any Intensified 

Sampling, another had performed four. The number of individuals trained should 

be adequate to address the sampling needs in a district. When the District 

performs Intensified Sampling in an establishment, it is based on several factors 

including recurring sanitation non-compliances within RTE rooms, major 

construction, and positive Lm product samples. 

District Mangers were kept up-to-date on Lm material by attending the industry 

workshops, reviewing the video for the industry workshop, reading the directive 

and briefings by members of the District staff. 

59 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

Training within individual districts by the Districts Offices was not consistent. 

Some individuals used EIAOs to conduct training for FLS and to conduct work 

unit meetings among CSIs. 

District Managers are relying on the FSRE training of the CSIs covering the 

regulation and directive for training. Since there are many CSIs left to be trained, 

another faster method of reaching CSIs in the field with the required information 

may be needed. 

VII. Summary Discussion of Findings 

FSIS has made progress in providing training and education to the workforce in 

application of the Lm interim rule. Formal training has been provided in the Food 

Safety Regulatory Essentials (FSRE) week 3, Ready-to-eat, a CD-ROM was 

issued to the field inspection force to provide information on the implementation 

of Lm Interim Rule, and a 3 day training course in intensified verification 

sampling was completed by approximately 35 EIAOs. 

VIII. Accomplishments by the Agency 

a. FSIS produced and distributed a Lm Interim rule CD-ROM for use by the 

inspection workforce. 

b. A classroom course on intensified verification sampling for Lm was 

developed and delivered in April 2003 to approximately 35 EIAOs. 

c. The Food Safety Regulatory Essentials (FSRE – week 3) course 

addresses Listeria in ready-to-eat (RTE) products, is taught routinely by 

the Center for Learning (CFL) and is available to all CSIs. 

References 

1. FSIS Directive 10,240.4, “Resources and Compliance Guidelines for 

Controlling Lm, CD-ROM, issued October 2003 

2. FSRE, Food Safety Regulatory Essentials training manual, week 3, Center for 

Learning, FSIS 

60 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

SMALL PLANT GUIDANCE PAT REPORT 

I. Major Findings 

• Majority of the small and very small plants did not receive or did not know that 

there is a Compliance Guidelines document for the Listeria interim rule. 

• Those small and very small establishments that received the Compliance 

Guidelines need additional guidance in understanding the recommendations and 

complying with the rule. 

• Since the implementation of the Listeria interim rule, The Labeling and Consumer 

Protection Staff (LCPS) has seen an increase in the number of meat and poultry 

meals, entrees and dinners labeled as not-ready-to-eat (NRTE) although these were 

historically marketed as ready-to-eat (RTE). 

 Recommendations 

• Develop a system of distributing the Compliance Guidelines to small and very 

small establishments because most of these establishments do not have computers 

to access the guidelines on the FSIS web site. 

• Conduct more workshops targeting small and very small establishments and 

schedule workshops earlier prior to implementation. 

• Simplify the guidelines to enable small and very small establishments to easily 

understand the recommendations. 

• Guidance should be provided to establishments regarding the appropriate steps to 

follow in changing the HACCP category of their products from RTE to NRTE. 

II. Purpose 

• To determine if the Compliance Guidelines are assisting small and very small 

establishments in meeting the requirements of the Listeria Interim Final Rule. 

• To determine the establishments’ change of HACCP category of their products 

from RTE to NRTE. 

III. Goals 

• Gather and analyze sources of information to determine how the Compliance 

Guidelines helped small and very small establishments; and establishments’ 

change of the HACCP category of their products from RTE to NRTE. Sources of 

information to be analyzed are the reports on the questions received by FSIS in 

the Listeria interim rule, CSO/EIAO reports, NRs written, report on the 

workshops, and response to questions sent to University Extension Services, and 

FSIS offices and staff. 

• Provide recommendations to improve distribution of the Compliance Guidelines 

and its understanding by the small and very small plants. 

• Provide guidance for establishments that change the HACCP category of their 

products from RTE to NRTE. 

• The PAT also considered advice from The National Advisory Committee on Meat 

and Poultry Inspection (NACMPI). 

61 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

IV. Background 

The Compliance Guidelines for the Listeria Interim Rule were developed by FSIS in 

order to help establishments, especially small and very small establishments in complying 

with the rule. The guidelines were posted on the FSIS web site together with the 

Directive. As part of the Listeria rule assessment, a group was charged to determine if the 

Compliance Guidelines was indeed helpful to small and very small establishments. An 

additional charge to the group was to determine establishments’ change of the HACCP 

category of their products from RTE to NRTE. 

The Small Plant guidance PAT looked at documents and other sources of information, 

which may show compliance of small and very small establishments to the rule using the 

Compliance Guidelines. These documents and other sources of information are: 

• Report on the Workshops that FSIS conducted in 5 states prior to October 6, 

2004, the implementation date of the rule. 

• CSO/EIAO reports from January to February 2004 on establishment compliance 

to the rule. 

• NRs written from 1/22/04 to 3/22/04 to determine the kind of non-compliance to 

the rule 

• Summary of questions received by OPPED, the Technical Service Center and 

those received at the workshops. 

• Response to questions sent to Agricultural Extension Service at 2 universities. 

• Response to questions sent to pertinent FSIS offices and staff members 

. 

The Small Plant Guidance PAT met 4 times to report on the findings and discuss possible 

solutions and recommendations. 

V. Summary Recommendations 

FSIS should recognize that very small plants face special challenges in implementing 

new requirements and should devise ways to disseminate new information to small 

plants in a timely manner. Following are recommendations for distribution, 

dissemination and simplification of guidance materials. 

• Develop a system of distributing the Compliance Guidelines to small and 

very small establishments. 

Suggestions for distribution: 

1) Mailing to all small and very small establishments; 

2) Send through District Office, or CSO or EIAO; 

3) Send through university Agriculture Extension Service; 

4) Send through HACCP coordinators in each state program

FSIS should use available technology in training FSIS personnel and industry by 

using remote broadcast and videotapes of the broadcasts through distribution to 

62 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

small plants. Suggestions for disseminating information on availability of the 

Compliance Guidelines, in addition to the web site and the Constituent Update: 

1) Agriculture Extension Service live hook-up, phone –in 

2) C-Span presentation for government agencies- call-in numbers provided 

• Conduct additional workshops that reach small and very small establishments to 

distribute and explain compliance guidelines 

• Conduct workshops at a timely manner to give establishments enough time to 

prepare to comply with the rule 

• Simplify Compliance Guidelines so it is easily understood by small and very 

small establishments, using easily understood tables, graphs, and flow charts with 

only ‘need to know’ information. 

Suggestions for simplifying the compliance guidelines: 

1) Reorganize guidelines such that tables, flow charts, diagrams, scenarios, 

examples, bulleted/enumerated instructions or step by step instructions to 

comply with alternatives are in the main body of the guidelines and move the 

discussions/explanations after the tables/instructions, etc. 

2) Put all tables and instructions in the body of the guidelines and put discussions 

as attachments. 

3) Trim discussion on sections that may be superfluous. 

• Provide guidance to establishment regarding criteria for changing the HACCP 

category of products from RTE to NRTE. 

VI. Summary Findings 

• Questions received at OPPED, the Technical Service Center and from the 

workshops showed aspects of the rule and Compliance Guidelines that need to be 

clarified. 

• Attendees of the Workshops conducted by FSIS prior to implementation of the 

rule, showed that the Workshops were very helpful to attendees in understanding 

the rule. Attendees suggested that FSIS conduct more workshops and at earlier 

time before implementation. 

• The CSO/EIAO reports from January to February 2004 period showed that 22 

establishments reviewed are not complying with the rule due to various reasons. 

Most of the very small plants are waiting for the CSOs to talk to them on how 

they can comply with the rule. 

• Analysis of the NRs issued between 1/22/04 to 3/12/04, 3-6 months after 

implementation of the rule showed requirements of the rule that establishments 

need to comply with. Examples are hazard analysis, alternatives, sanitation 

measures, hold and test procedures, food contact surface testing, validation, 

recordkeeping, and verification. 

63 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

• Report from a University Extension Service showed that small and very small 

establishments need help because they may not have the patience and education in 

understanding the rule and the guidelines. These establishments contact the 

Agricultural Extension Service or state inspectors for help. 

• Report from LCPS estimated that one-fourth of the labels that are submitted to the 

Agency are for products that were classified as RTE but now may be categorized 

as NRTE. 

VII. Summary Discussion of Findings 

A. Questions received by OPPED, Technical Service Center, and at the Workshops 

A few of the questions received by FSIS were clearly from small and very small 

establishments and some from large establishments. However for most of questions, the 

size of the establishments could not be determined. Some plant management indicated 

that they were not aware of the compliance Guidelines and the Q and As. Some CSIs 

were also not aware of these resources accessible via the homepage or the CD format. 

Very small owners have problems understanding the language in the guidelines such as 

‘genus’ or ‘Listeria-like organisms’ and may need simpler language or definitions. 

Establishments believe that they can change the HACCP category of their products from 

RTE to NRTE by just adding cooking instructions so the products are not subject to the 

rule. 

Questions were received on the following subjects: acceptable documentation for 

validation, validation of jerky, post-lethality treatments, antimicrobial agents, Listeria test 

methods, hold and test procedures, food contact surface testing, lard and tallow, pork 

rinds, products affected after a positive test, recall procedures, changing Alternatives, 

temperature control for L. monocytogenes, changing HACCP category from RTE to 

NRTE, deli products, deli salads, among others. 

B. FSIS Workshops conducted prior to implementation of the rule 

The evaluation sheets showed positive responses to the workshops for content, notebook 

(hand-outs) and presentation. The attendees commended the Agency for conducting the 

workshops and suggested that more workshops be held and to hold them much earlier 

prior to implementation. Most of the attendees said that the workshops were well 

organized and informative and that they benefited from the workshops. Some attendees 

suggested that the topics were very general and that more details are needed. The 

attendees liked them to be able get responses to questions that they asked at the 

workshops. The report on the Workshops did not show the establishments’ size that the 

attendees represented. 

C. Non- Compliance Reports (NR) The NRs were issued from January 22, 2004 to March 

12, 2004, about 3 to 6 months after implementation. NRs were issued to 35 meat 

establishments and 2 poultry establishments. Nineteen (19) of the NRs were written as a 

result of review of HACCP plan features for completeness in 18 fully cooked not shelf 

64 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

stable products and in 1 heat treated shelf stable product. Seventeen (17) were cited for 

verification of HACCP plan implementation to determine if the establishment is 

following its HACCP plan for heat treated shelf stable products. The NRs showed that 

the requirements are not being followed in the following: hazard analysis, alternatives, 

sanitation measures, hold and test procedures, food contact surface testing, validation, 

monitoring, recordkeeping, and verification. The group looked into these reports to 

determine how many small and very small establishments were given NRs. However, the 

size of the establishment receiving NRs was not determined from the report. 

D. CSO/EIAO Reports

The CSO/EIAO reports were dated from January to February 2004. Out of 172 reports, 

22 were reviewed for compliance to the Listeria interim rule. Seven (7) were found to 

comply with the rule, 15 had deficiencies in complying with the rule, out of which 2 were 

given NRs, 8 were issued other enforcement action, and 5 had no action taken. 

Deficiencies found pertain to hazard analysis, alternatives, supporting documentation, 

food contact surface testing, changing product HACCP category from RTE to NRTE. 

One very small establishment was identified as waiting CSO to show how to comply to 

the requirements. These reports were analyzed to determine how many small and very 

small establishments are visited by the CSOs. The report identified 1 establishment as 

very small. 

E. Report from Agricultural Extension Service 

Two professors from university Ag Extension Service were sent questions regarding the 

Compliance Guidelines for the Listeria interim final rule. Response from one University 

Extension professor indicated that the establishments that they work with were not aware 

of the Compliance Guidelines until he and his associates talked to them about it in their 

workshops. He indicated that these establishments would not have the patience nor the 

education to read the guidelines. These establishments contact them (Extension Service) 

or state inspectors for questions on the rule. His group conducted 10 half-day workshops 

in their state (Wisconsin) to explain the rule and the compliance guidelines. He suggested 

simplifying the guidelines, using easily understood tables, sidebars, and charts with only 

the “need to know” information. 

F. Report from FSIS Offices and Staff 

Questions were sent to pertinent FSIS offices and staff regarding the Compliance 

Guidelines. Responses received: The notebook (hand-out) used in the workshops were 

mailed to small and very small establishments. Specific work with the guidelines still 

needs to be done. 

G. Report from LCPS on Establishments’ Change of HACCP Category of their Products 

from RTE to NRTE 

65 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

LCPS reported an increase in the number of meat and poultry meals, entrees and dinners 

labeled as NRTE. Although these products were historically marketed as RTE, 

establishments are now producing these products as NRTE (i.e. produced under a 

HACCP category that does not have a final lethality step). The establishments producing 

these products expect the consumer to thoroughly cook the product prior to consumption 

and thus provide the lethality treatment. They convey this message to the consumers in 

the product label. It is estimated that products that are historically marketed as RTE and 

are now marketed as NRTE are produced by large establishments and constitutes onefourth of the labels submitted to the Agency. 

LCPS developed appropriate steps to guide establishments in changing the HACCP 

category of their products from RTE to NRTE prior to submitting labels to FSIS for 

approval. FSIS conducted two focus groups to help explore consumers’ perception, 

understanding, and use of these food safety labeling features. 

VIII. Accomplishments by the Agency 

• Issued/published Listeria Interim Final rule, Directive 10,240.4, Compliance 

Guidelines and Questions and Answers for the control of L. monocytogenes in 

post-lethality exposed RTE meat and poultry products. 

• Responded to questions received at OPPED, Technical Service Center and at the 

Workshops 

• Conducted Workshops prior to implementation of the rule to help establishments, 

especially small and very small ones to comply with the rule. 

• Conducted FSRE training for FSIS inspection personnel to train them in enforcing 

the rule 

• Formed a team to assess the Listeria interim rule for the finalization. 

• Updated the Compliance Guidelines to include responses to questions received 

and further clarify some sections. 

• Revised the production volume form so it is easily understood and answered by 

establishments. 

• Presented talks on the Listeria Interim Rule to trade organizations, retail 

organizations, and in public meetings to disseminate information on the rule and 

control of L. monocytogenes. 

• Submitted an issue to the Conference for Food Protection, which was successfully 

approved, to include guidance to the Food Code for control of L. monocytogenes 

at retail. 

• Have companies/establishments with methods and equipment for post-lethality 

treatments, antimicrobial agents, sanitation, and other Listeria control procedures 

present their findings to FSIS. 

IX. Response to Recommendations from NACMPI 

66 
Summary Report 

Assessing the Effectiveness of the 

Listeria monocytogenes (Lm)Interim Final Rule 

NACMPI recommended the following: 

1) Include universities in disseminating guidance information to small plant 

2) Representatives of District Offices should be involved to help deliver messages to 

industry through timely training 

3) Train FSIS personnel and industry using available technology such as remote 

broadcast and videotapes of the broadcast for the small plants 

Response: 

1) Currently FSIS has awarded funding to university extension service so they can 

help small and very small plants to comply with the rules on meat, poultry and 

eggs. FSIS will send the guidelines to these and other university extension service 

for distribution to small and very small establishments. 

2) We’ll ask the district offices to have representatives (EIAO, CSO, CSI) deliver 

and disseminate messages to industry. 

3) Currently, FSIS is using remote broadcast and videotaping in the training of 

industry and FSIS for the newest Directive and compliance guidelines. 

67