**UNITED STATES DEPARTMENT OF AGRICULTURE**

**IN RE:**

**FSIS DRAFT LISTERIA RISK ASSESSMENT TECHNICAL MEETING DOCKET NO.
FSIS-03-01**

Meeting held on the 26th day of February, 2003

at 8:00 a.m.

10 Thomas Circle, N.W.

Massachusetts and 14^th^ Street

Washington, D.C.

TRANSCRIPT OF PROCEEDINGS

BEFORE: HONORABLE DANIEL L. ENGELJOHN, PH.D.

Acting Assistant Deputy Administrator OPPED

INDEX

Page

Opening Remarks - Karen Hulebak, Sc.D. 4

Welcome - Elsa A. Murano, Ph.D. 7

Overview - Garry L. McKee, Ph.D., M.P.H. 13

What is Risk Assessment? -

Janell R. Kause, M.P.H., M.P.P., Senior

Risk Assessment Division, OPHS 21

FDA/FSIS Risk Ranking of RTE Foods -

Robert L. Buchanan, Ph.D., Director,

Office of Science, CFSAN Food and Drug

Administration 26

Current Listeria Policy and Risk

Management Questions -

Daniel L. Engeljohn, Ph.D., Acting

Assistant Deputy Administrator OPPED 45

Introduction and Overview of the Draft

FSIS Risk Assessment - Carol Maczka, Ph.D.

Acting Director Risk Assessment Division,

OPHS 54

The FSIS Risk Assessment, Part I -

Contamination on Food Contact Surfaces,

Transfer to Ready-to-Eat Product,

Growth of Listeria from Plant to Retail

FDA/FSIS Model, Sanitation Efficacy and

(Verification) testing -

Daniel L. Gallagher, Ph.D., P.E. Associate

Professor of Civil and Environmental

Engineering, Virginia Tech 59

The FSIS Risk Assessment, Part II

Calibration of In-Plant Model

Retail to Table FDA/FSIS Model Aspects

Risk Assessment Outputs for Management

Questions 128

Sensitivity Analysis -

Daniel L. Gallagher, Ph.D.,

P.E. Associate Professor of Civil

and Environmental Engineering,

Virginia Tech 149

[PANEL DISCUSSION OF THE FSIS DRAFT]{.underline}

[LISTERIA RISK ASSESSMENT]{.underline} 190

Charlotte Christin, J.D., L.L.M.

Senior Food Safety Attorney, Center

for Science in the Public Interest 190

Jenny Scott, Senior Director, Food

Safety Program, National Food

Processors Association 196

Sophia Kathariou, Ph.D., Associate

Professor Food Science, North

Carolina State University, National

Alliance for Food Safety (given by

Karen Hulebak) 205

FSIS DRAFT LISTERIA RISK ASSESSMENT

TECHNICAL QUESTIONS AND ANSWERS 224

NEXT STAGES - Loren Lange, Acting

Associate Deputy Administrator OPHS 240

P R O C E E D I N G S

February 26, 2003

MS. HULEBAK:

Good morning. I welcome all you hearty souls in making it here in the
face of what used to be sort of a pleasant, delightful, because it was a
relatively rare event in Washington, and that was snow. I think I speak
for all of us when I say it's getting to get a little old. We are here
this morning to talk about the FSIS Draft Listeria Risk Assessment. The
focus of the meeting is discussion of the risk assessment on technical
aspects of the risk assessment. FSIS will present the risk assessment to
you that we have produced, and take this day to provide an opportunity
for you to ask questions and raise points, and for us to hear what you
have to say. Production of this risk assessment has been a top priority
for FSIS. We began work on the risk assessment last spring, and have
moved pretty quickly to produce a risk assessment faster, in fact, than
many of you may have been accustomed to seeing by way of risk assessment
development. And I think we can credit our excellent staff of risk
assessment professionals and the people who support them in achieving
this feat. Our goal in producing this risk assessment is to provide the
best possible scientific basis for regulatory decision making. Again,
today, this day, our focus is on the risk assessment, and not on the
agency's risk management thinking or policy proposals or plans. We've
got only a day for this public meeting, so let's focus on the science,
the data, how we use them in development of this assessment. We'll begin
the day with a series of remarks that will help set the context for a
discussion of the assessment. We'll begin hearing from Under Secretary
Elsa Murano, followed by FSIS Administrator, Garry McKee, and then we'll
hear from Janell Kause of FSIS on basic concepts in risk assessment, and
then from Bob Buchanan of FDA's Center for Food Safety and Applied
Nutrition, on a key part of the risk assessment as we use it, which is
the FDA/FSIS Risk Ranking. Finally, this morning's session will be
wrapped up by Dan Engeljohn, the Acting Assistant Deputy Administrator
for Policy in FSIS, on current Listeria policy and risk management
questions. The kinds of questions that shaped our approach in developing
this risk assessment. We've started a little late to allow people to get
here, so I will close up my remarks and turn to introduction of Dr.
Murano with this one note. Please, please turn off cell phones, egg
timers, whatever else you have in your possession, and switch them to
vibrate or some other relatively silent means of alert. The variety of
music available in cell phones these days is pleasant to all of us, I
know, but we've probably heard as many tunes as we need to.

I will turn now to Dr. Murano, introduce her, our Under Secretary for
Food Safety. In this position, Dr. Murano oversees the policies and
programs of the Food Safety Inspection Service. Dr. Murano, as you know,
has extensive public and private experience in food safety, both as a
manager and an educator. She's a professional microbiologist with a
Ph.D. in Food Science and Technology, and a Master's Degree in Anaerobic
Microbiology. She is doing an excellent job of leading this agency, and
we are pleased to call her Under Secretary. Dr. Murano.

DR. MURANO:

Thank you very much, Dr. Hulebak. Well, good morning, everybody. Welcome
to this technical meeting to discuss the Food Safety and Inspection
Service's Listeria Risk Assessment. Well, this risk assessment is, as
you heard Dr. Hulebak mention, extremely important to our efforts to
reduce illnesses associated with this pathogen. We know these efforts
are needed because, according to the Center for Disease Control and
Prevention, there has been a major listeriosis outbreak in the U.S.
about every two to four years. So rather than accept this status quo, we
must employ science in a way that breaks this all too familiar cycle.
And I think you all agree. Our efforts to reduce illnesses associated
with Listeria monocytogenes are also guided by the objectives spelled
out in the U. S. Department of Health and Human Services AHealthy People
2010.@ And that document sets goals every ten years for a variety of
health concerns, including food-borne illnesses. So these goals are also
a driving force to us in our assessment of current realities, and give
us resolve to be proactive and innovative in our thinking in order to
successfully achieve these goals. Well, knowing that a problem exists is
just the beginning. The tough part is coming up with solutions based on
scientific data. Making policy decisions without basing them on science,
in our opinion, is akin to shooting at a target in the dark without
night vision goggles. You have no idea if you hit your mark. So when
that mark is the safety of the food on American tables, accuracy is
essential. And that's why this risk assessment is so important. It gives
us a science to support our policy decisions. Many people don't realize
that risk assessment, in the pure sense of the word, has been practiced
for centuries in the establishment of early food handling practices and
laws. Even today, most of us practice a form of risk assessment, if you
will, in our daily lives, as we determine what we serve to our families
for dinner, or decide if we'll select a particular item from the salad
bar. Well, risk assessment has been a rapidly developing field for many
years, beginning in 1983, when the National Research Council published a
book that provided valuable information on chemical risk assessment.
Risk assessment is a complex process. It's one that organizes and
interprets scientific data. It estimates the risk for specific
scenarios, and presents findings in a format that facilitates informed
decision making. It's the use of scientific information to characterize
and estimate adverse effects from exposure to hazards. And what's more,
succinctly stated, input from epidemiologists, microbiologists and other
experts is merged in a risk assessment to develop a model that
represents the best available understanding of all factors affecting
public health. Perhaps more importantly, risk assessment helps stimulate
the impact of various control strategies which can be used in developing
risk management options. I cannot stress enough to you this morning that
in my estimation, we have no excuse for not making the best decisions
that will reduce the risk of food-borne illness. Ladies and gentlemen,
to me, traditional approaches which employ a piecemeal evaluation of
data are not going to give us the gains in public health that we need to
achieve. So this Listeria risk assessment that will be presented to you
today is a prime example of how we can use the scientifically based
process of risk assessment to quantitatively identify the risks posed by
this organism to public health and evaluate the impact over a wide
variety of possible mitigation strategies. Specifically, today, you will
see that during the course of our risk assessment, we learned that a
combination of environmental testing, sanitation and antimicrobial
interventions yielded greater benefits than any one strategy alone. In
addition, the risk assessment also demonstrated that the use of
intervention steps such as post-packaging pasteurization or the
introduction of growth inhibitors showed the most dramatic public health
benefit of all. While, as you all know, in December, we issued a
directive to our inspectors that laid out an aggressive and targeted
approach to further reduce the risk of listeriosis from ready-to-eat
meat and poultry products. This directive is a powerful tool in our
fight against Listeria. Through its implementation, we have been
achieving what I believe was its intended purpose. Which is to find this
pathogen, wherever it may be in the plant environment, before it can
contaminate product. Since its implementation, we are seeing an increase
in the sharing of testing data by industry with FSIS. And, more
importantly, in the execution of corrective actions to eliminate
Listeria from the plant environment. In spite of these advances, we are
not stopping here. Our scientists at FSIS have been working very hard to
develop the risk assessment that they will present to you today. They
know that it will provide us with valuable insight into the best
strategies that can be used to defeat this pathogen, and it will be a
vital tool in creating a regulation that will work to protect consumers
because it will be based on science.

Your participation is always as important as we develop and update our
food safety policies. I'm sure many of you were present at the Listeria
summit that we held in November of last year, where we heard from
government, academia, industry and consumer advocates on ways to address
Listeria monocytogenes. So I am glad that you have come to what may be
termed part two, and you're here today to participate in the discussion
on the technical merits of the risk assessment. We expect there will be
many technical questions about the assessment. That is why we're here
today, to allow you an opportunity to ask questions and offer
suggestions. You'll have time to carefully review the assessment and
provide your comments. Public comments will be accepted until March 14.
But I do urge you to take advantage of today. We are here because we
want an open exchange of ideas that will be constructive, and that will
help us do the best job possible in our quest to reduce the risk of
listeriosis from ready-to-eat meat and poultry products. So, once again,
I thank you for your participation, certainly for braving the weather,
and coming to this meeting today. I look forward, certainly, to the
discussions. And I'd like to now turn the program over to Dr. Garry
McKee, our FSIS Administrator.

DR. McKEE:

Thank you, Dr. Murano. On behalf of FSIS, I want to welcome all of you
here for this risk, Listeria Risk Assessment Technical Meeting. We have
a very full agenda today. I have a lot to cover, and a short time, and
I'll keep my remarks short. We have the importance, or we value the
importance of the public meetings in that they're vital to us as we
discuss the anticipated draft assessment, risk assessment for Listeria.
It's important that we have the input and the dialogue that will occur.
Three months ago, many of us came together for the Listeria summit to
discuss current research and information related to improving the safety
of ready-to-eat products. At that meeting, I shared my vision for the
agency, which is to make FSIS into a world-class public health agency
that is a model for all public health institutions. I believe that just
about everyone in this room has heard this vision by now. What we have
done to evolve toward this vision in combating listeriosis, I'd like to
discuss how FSIS has implemented the three important functions of a
model public health agency. The first function is assessment. And we
conducted an unprecedented investigation with CDC and other state and
local agencies to identify the source of the northeast listeriosis
outbreak last year. From this investigation, we found that some
establishments were not adequately addressing the potential for
bacterial contamination in their house of plans. Sanitation, SOPs, and
other control measures. This led us to the second function. Policy
development. Dr. Murano mentioned the directive we announced at the
Listeria summit and issued in December the directive outline and
aggressive and targeted approach to further reduce the risk of
listeriosis from consumption of ready-to-eat products. The last function
is assurance. And we are making sure that the increased activity, and
issued a directive that was implemented. Our inspectors are taking the
steps to ensure that establishment for producing ready-to-eat meat and
poultry products or preventing Listeria monocytogenes contamination.
They're also directing an intensified testing program consisting of
increased product and food contact surface testing, as well as
environmental testing inside the plant, and increased reviews of the
plants, records and data. We are also targeting plants with intensified
testing if they have Listeria control programs but do not choose to
share the testing data with us on an ongoing basis. Giving the gravity
of the listeriosis outbreak last autumn, we as a public health agency,
felt it was absolutely essential to take action with this interim
measure. The growth of Listeria monocytogenes is not affected by low
temperatures, like salmonella or ecoli 15787, and we could not afford to
wait through the winter for the results of our draft Listeria Risk
Assessment to finalize a rule. Furthermore, Listeria monocytogenes has
the second highest fatality rate for the food-borne illnesses, at 20
percent, and the highest hospitalization rate of 90 percent. FSIS took
the appropriate measures that had to be done to protect the publics'
health.

I mentioned in the last Listeria meeting that we can look at these
meetings as repair shops, and that we need to focus on long-term repairs
based on science. Short-term solutions don't solve the recurring cycle
of listeriosis outbreaks. In fact, listeriosis outbreaks are quite
analogous to a problem that may hit home with some of you. And I mean,
literally, your home. During the past weekend, here in Washington, we
experienced an incredible volume of water coming down from the sky from
either snow or rain. With so much water coming to the ground at one
time, the saturation threshold was quickly surpassed. Where else would
this water go? The answer, unfortunately, was to many of you, to your
basements. Even if the water didn't go into your basement, you certainly
feared that it might. After you get all of the water out of your
basement, you have to deal with the aftermath, the repairs. The choices
here are numerous, depending on a number of factors. The environmental
circumstances, type of damage, amount of damage that was incurred, and
where the water came in. After everything dries out, you could repaint
and make many of the necessary surface repairs to the walls and floors,
and then simply put everything back in place. However, when the next
deluge comes, will your basement and all your personal belongings be
secure? The answer likely may be no, if you only implemented short-term
solutions to a long-term problem. Repairing walls, floors, after the
flood is not solving your problem. Without taking a look at the
long-term solution, such as digging up the earth, inspecting the
foundation, fixing the drainage, installing a sump pump or improving the
grade around your home, you will keep your local Home Depot making money
hand over fist every time you come in for paint, materials to fix the
short-term problem. Just like trying to keep water out of your basement,
it takes continual vigilance to provide an environment where Listeria
won't exist. Listeria will be there some days, some days it won't. There
are so many factors to consider. The draft risk assessment which we have
come together today to discuss provides an important tool for evaluating
various control measures in the production of ready-to-eat and poultry
\-- ready-to-eat meat and poultry products. Like examining the
underlying factors that contribute to your flooded basement, this
Listeria Risk Assessment looked at a variety of issues such as the
relationship between the prevalence and level of generic Listeria on
food contact surfaces and the prevalence and level of Listeria in
ready-to-eat meat and poultry products. The public health impact of
different concentrations of Listeria in product and the availability of
testing programs. Sanitation processes and intervention steps to
mitigate the public health risks associated with Listeria. This
assessment is a vital step as we move forward toward our final Listeria
rule, which will lead us toward a long-term solution, significantly
reducing illness and death from this pathogen. Today we'll discuss all
sectors of the draft risk assessment, and I'd like to go over today's
agenda with you very quickly before we get started.

First, we'll hear from Ms. Janell Kause of our Risk Assessment Division,
who will explain a risk assessment. Next, Dr. Robert Buchanan, the
Director of the Office of Science within FDA's Center For Food Safety
and Applied Nutrition, will give us an overview of FDA's and FSIS's risk
ranking analysis of ready-to-eat foods. Then Dr. Dan Engeljohn, our
Acting Assistant Deputy Administrator of the Office of Policy Program
Development and Evaluation will cover our current Listeria policy and
the risk management questions from which FSIS risk managers requested
the risk assessment to be designed. After a short break, Dr. Carol
Maczka, the Acting Director of our Risk Assessment Division, will give
an overview of the Draft Listeria Risk Assessment. This will lead us
into a presentation on Part I of the risk assessment from Dr. Daniel
Gallagher before lunch. Dr. Gallagher is the Associate Professor of
Civil and Environmental Engineering at Virginia Tech. And after lunch,
Mr. Eric Ebel, from our Risk Assessment Division, will give an overview
of Part II of the Risk Assessment. Mr. Ebel and Ms. Kause will, whom I
mentioned before worked in partnership with Dr. Gallagher of Virginia
Tech in developing this risk assessment report. After a short break in
the afternoon, we'll head into the discussion section of the risk
assessment. Panelists of the discussion will include Dr. Karen Hulebak,
Deputy Administrator of FSIS's Office of Public Health and Science, Ms.
Jenny Scott, Senior Director of Food Safety Programs at the National
Food Processors Association, Ms. Charlotte Christin, Senior Food Safety
Attorney of the Center for Science in the Public Interest, and Dr.
Sophia Kathariou, Associate Professor of Food Science at North Carolina
State University National Alliance for Food Safety. After the panel
discussion, Dr. Maczka will cover the risk assessment's technical
questions and answers. Finally, Loren Lange, Acting Associate Deputy
Administrator of OPHS, will wrap up the meeting with the Next Stages
section. With that said, I look forward to a very constructive meeting
today. Keep in mind that we're looking for a long-term solution to
protect the American public from the ubiquitous pathogen Listeria
monocytogenes. Without further delay, I'd like to turn the program over
to Ms. Janell Kause. Ms. Kause?

MS. KAUSE:

Thank you.

MS. HULEBAK:

Let me briefly introduce Janell. I'm sure these are on, but I \-- okay,
ready to go. Janell is the Interim Acting Director for the FSIS Risk
Assessment Division, and is \-- has been the Project Lead on the
Listeria Risk Assessment that we're here to talk about today. She is a
Senior Risk Analyst with over ten years of experience in risk
experience. Much of this time focused on food safety microbial risk
assessments. Janell.

MS. KAUSE:

Good morning. It's certainly a pleasure to be here to do an exchange.
How is the mike situation? Is it clear? No. Okay, is that a little
better? Thank you. Before I begin, I'm going to go over some basic
concepts in risk assessment. Most of you are very familiar with these,
but we want to ensure that everyone here is on the same page. To begin,
I want to discuss what the difference is for what is risk. One of the
concepts between risk and hazard. Basically, risk is the likelihood and
the amount of harm that could occur. The hazard is the chemical,
physical or biological agent that could cause the harm. In this case,
we're talking about Listeria monocytogenes as the hazard, and we're \--
the risk assessment is designed to look at that likelihood of it
occurring. Basically, risk assessment is one component of a three-part
triad for risk analysis. Risk assessment, as I've already said, looks at
the likelihood of an adverse effect from a hazard. The type of questions
that get usually asked for risk assessment is what is the likelihood of
harm and how much harm could occur. The other two components will be
discussed today \-- that won't be discussed today are risk management
and risk communication. I believe that both Dr. Murano and \-- has done
a good job of going over what really risk assessment is in her talk. So
why do we use risk assessment? Well, we use risk assessment to focus on
hazards in meat and poultry that pose the greatest risk to public
health. We also use risk assessments, as you'll see today when we begin
to discuss the FSIS Listeria Risk Assessment to evaluate the public
health impact. For example, the reduction of the number of illnesses of
various interventions. It allows us to do a comparison. Risk assessments
are designed to answer specific risk management questions. And many
times we're asked why the model was built the way that it was. Well,
it's designed to answer the specific questions that come from our risk
managers. For the most part, we want everything to be transparent and
objective. And here, today, we hope to have the interchange between us
and the public so that people can see the data that was used to the
underlying assumptions and the modeling techniques used as part of the
transparency process. The slide that's here is showing you, basically,
there are many different types of risk assessment. And many of you are
familiar with the risk assessments we've done in the past that are farm
to table Monte Carlo models. They're what we call process risk models.
Those are what we consider highly quantitative in nature. And they were
designed that way to answer, again, very specific risk management
questions. And then the other end of the spectrum, we can have
qualitative risk assessments as well. Those that aren't very complex,
that can be done on the back of an envelope. Again, it depends on the
type of decision that needs to be made. Risk assessments vary in scope.
They can be farm to table, such as the ecoli 0157 risk assessment for
ground beef that we did a while back. They can be retail to table, which
is the \-- a risk assessment such as the FD/FSIS Listeria Risk
Assessment, which Dr. Buchanan will be talking about right after this
talk, or they can be plant to tables, such as the FSIS Listeria Risk
Assessment that was \-- that we're going to go into detail today. Risk
assessments, as I've already mentioned, can vary in their complexity.
They can be qualitative, meaning the answer can be high, medium or low
risk. They can be semi-quantitative, such as a hazard ranking, which we
did a few years ago for processing inspection operations. Or they can be
quantitative. They can be Monte Carlo. They can have an uncertainty
analysis and a sensitivity analysis. They can vary in type. They can be
point estimates, they could be risk rankings, they can be process risk
models, or they can be a combination. Today's FSIS Listeria Risk
Assessment is a combination of a dynamic in-plant model and a process
risk model that goes from retail to table. I want to always emphasize
what the purpose of risk assessments are. They're basically to be a
cytoic \[ph\] basis for regulatory decision making. They also allow us,
again, to evaluate various strategies to manage risks. And with that,
I'm going to turn it over to Dr. Buchanan. Thank you.

MS. HULEBAK:

Thank you, Janell. I'd like to now introduce to you Dr. Robert Buchanan.
Dr. Buchanan has a Bachelor's of Science, Master's of Philosophy, and
Ph.D. Degrees in Food Sciences from the Rutgers University. He spent 25
years teaching and conducting research in food safety first in academia,
then with the USDA's Agricultural Research Service, and most recently,
as the lead scientist for the FDA's food safety initiative. I will note
that during his time at USDA, he was one of the co-developers of the
widely used USDA Pathogen Modeling Program. Dr. Buchanan, please.

DR. BUCHANAN:

Thank you, and I'll try to adjust these for the short guy. And then
we're going to sit and pause for a second while I wait for my visual
aids, which seems to have successfully frozen the computer. Okay, here
we go. And what I'd like to do is provide a sort of quick summary of the
FDA/FSIS Risk Ranking Risk Assessment that we did for Listeria
monocytogenes. And I'd like to start off by, one, thanking Dr. Sherry
Dennis and Dr. Richard Whiting for helping me put this presentation
together and, in fact, helping to do a lot of the \-- most of the work
on the risk ranking. I will be covering this very fast, so hold onto
your seats. I have a, you know, a 500-page document to update you with.
But I do want to start off by re-emphasizing a point that Janell made
that risk assessments are performed to answer questions. And this was
the overall question that was the basis for doing the risk ranking, risk
assessment. That is to improve public health by determining which foods
should receive the most regulatory attention. And I threw that word
regulatory attention in, but I'm using that in the broadest framework.
It's not just regulatory attention in terms of regulations, but also
guidance, education, outreach, a variety of activities conducted by the
agencies.

I'd like to note also that we went to great pains in putting this risk
assessment together to do it on the basis of guidelines that have been
developed both nationally and internationally by organizations such as
Codex Alimentarious, the National Advisory Committee, ICMSF, on how to
conduct a microbial risk assessment, emphasizing concepts like
transparency, broad scientific and stakeholder input and extensive peer
reviews at different stages along the process.

We considered a great variety of data in putting this together. We did
make a decision that, for the needs of the risk assessment, we could
largely start at the retail level and move forward. We had data input
into this model on consumption surveys, contamination data, growth and
survival and activation data, animal studies, epidemiological
investigations. To date, I believe that last count I talked to Sherry, I
think that we've looked in excess at between 400,000 and 500,000
individual data points have been included in the data base that's within
that model.

We did have to limit the considerations in picking this. We did focus on
ready-to-eat foods, particularly, refrigerator ready-to-eat foods. In
selecting these categories, we did look at potential for Listeria
monocytogenes contamination. We did look at the history they had in
causing listeriosis. We did look to see if we had food consumption and
contamination data. No data didn't help to pick a category then. And we
did have to do some strategic lumping of foods so that we could get them
into a reasonable number of categories. The original risk assessment
considered 20 different food categories, each with a large number of
products within them.

Now, basically, you have three parts to a microbiological risk
assessment. The traditional model is hazard identification, hazard
characterization, the exposure assessment, and then risk
characterization. For many of the microbiological organisms, you pretty
well know that there's a hazard because there is extensive medical
literature on the problems. So, really, we're looking at three
components. The exposure assessment. And the exposure assessment is
really what goes into the consumer's mouth, because that's the only
number that counts when you're doing an exposure assessment. But in
order to deduce that, because we very seldom go into people's homes,
take the food away from them just as they're about to put it in their
mouth, and analyze it. So, often, we have to infer what is the level
that they're actually consuming. And we use different data sources in
this. We had to look at the frequency of contamination of the food, the
extent of that contamination, any growth that was likely to occur
between purchase and consumption. And that could be both positive
growth, but it could also be an activation in some instances. And then
an important thing is how frequently is that food consumed, if you're
looking at a risk on a national basis, and the amount of food consumed.
Because, as we've learned in, particularly on this risk assessment,
serving sizes count. We, in some instances, for certain products, had to
look at additional factors. Things like home refrigerator temperatures.
And that, itself, was a challenge, just finding the appropriate data.
When dealing with frankfurters, we had to estimate the percentages that
were reheated. We had to look at the effect of temperature on growth.
Itself, is a fairly complex biological phenomenon. And also, the effect
of temperature on the extent of growth. And as we found out, there's a
big difference between 2 degrees AC@ and 6 degrees AC@ on how high
Listeria monocytogenes will grow on food.

This is then coupled with a hazard characterization, and this is
basically the probability of illness or mortality as a function of the
number of Listeria consumed. We, to derive this kind of a relationship,
we had to use, again, various data sources. We got the dose response
curve shape from animal studies. We looked at variation in the virulence
of Listeria monocytogenes based on a variety of studies on, primarily,
immunocompromised mice. We had to account for the differences between
mice and men. There are differences, which is fairly obvious. And we
also had to attempt to anchor this to a reality check, because one of
the things that we've learned, had learned from earlier attempts to try
to work with Listeria monocytogenes, is you can very, with just a slight
hiccup, you can get a model that predicts four or five orders of
magnitude to many cases, so we had to find a way of anchoring it to real
health statistics. We looked at the variation of susceptibility within
age groups. We looked at the variation of susceptibility between age
groups. And we looked at factors that influenced different
sub-populations. We did decide to use three sub-populations in
considering the Listeria monocytogenes' ability to cause disease. These
included the perinatal group, which were fetuses from 16 weeks after
fertilization to 30 days after birth. The standing assumption here is
they weren't going out and eating most of these foods, is that the
transmission of the disease was via the mother. We look at the elderly,
and we use the CDC's definition of an elderly person. I might note this
was one of the most controversial things in the risk assessment. There
are a lot of you guys out there that I've known for a long time that are
approaching 60, that were really upset about this. And then the
intermediate age group was the group that was older than 30 days but
younger than 60. Okay, when you get those two bits and pieces put
together, no small feat itself, you use that to come up with a
characterization of the risk. And what we did in this case was we, the
first time that we looked at the model, we did this on the basis of
frequency of death, mortality, because it was the most definitive thing
that we could measure. When you start dealing with things like infection
rates, you can't get two microbiologists to agree on what is the
definition of infection. So we're looking for a metric here that was
clearly definable that we could get data on. We also found that we could
convert that to cases of severe listeriosis simply by multiplying by
five. It's been amazingly how constant this number has been despite
improvements in medical intervention, et cetera, that it's basically for
every five cases there is one fatality. And this is a value that we've
used consistently. We also spent a great deal of time characterizing the
variability and uncertainty associated with our rankings and the
individual risk assessments that underlie it, because, in fact, this
model actually involved doing 20 different risk assessments for the 20
food categories, and then combining them or integrating them into a
single risk ranking exercise. Just again to diagrammatically give you an
idea of what we did, we took the exposure assessment, which is the
number of Listeria monocytogenes consumed, developed a model for that.
We combined that with a model for the hazard characterization, the dose
response model. We were looking at, basically, a single end point, so we
weren't looking \-- severity was given. And then for each time we ran
the model, we did 30,000 inner rations. And we generated mortality cases
per serving as the primary metric. We then combined that \-- we then
multiplied that times five to get the Listeria cases per serving, and
then used that to calculate the risk per serving. We also used that, the
data, to generate, or combine it with the frequency of servings to get
the number of listeriosis cases predicted for that food group per annum,
or per year. I might note that this got chopped out at some of the
bottom. For each time we then ran a simulation, we repeated it 4,000
times, so that on a typical run with this model we repeat, we have
basically 120,000,000 inner rations to do a complete run on the
analysis. These high numbers of inner rations are necessary because

Listeria monocytogenes is a rare event. And in order to stabilize the
model, you're basically predicting a rare event. And you have to run it
enough times to get it to \-- the model to stabilize. That was a lesson
we learned early on. It did, in fact, allow us to do some fairly
sophisticated sensitivity analyses and uncertainty analyses. Oh, there's
the 4,000 times. We examined the results also once we got them. We
didn't stop with numbers alone. We, of course, looked at the
quantitative results. But we also evaluated very much the data
variability, the model uncertainty, and we provided estimates for each
of those. We also considered the results in light of a variety of
qualitative factors. The epidemiological record, the food
characteristics, to make sure that it made sense. And we also, in the
draft risk assessment, had an extensive discussion of each of the food
categories, their medical history in regard to this organism, et cetera.
This is what the data looked like. If you go pull this up, it's still
available on our web site, which I'll come into in a minute. Just to
make this a little bit more user friendly, this is the graph that looks
at the population on a per-serving basis, and it's the 20 food
categories. Today, we're going to be focusing a lot, I understand, on
deli meats, that's what I was asked to focus on, that value the fourth
from the right, on a per-serving basis as deli meats. You can see that
there is a substantial variation that goes from high to low. I think the
initial per serving risk assessment indicate pate, and ice cream was the
low at the other end. We also, again, took into account the total number
of servings consumed within the United States to come up with the risk
per annum. And the risk is influenced strongly by the number of servings
that are consumed each year. You can have a high per serving risk, but
if it's only consumed by three people, it's going to have a less of an
impact on the total country's public health stance. This is what it
looked like when we arrayed these from the original draft risk
assessment for the total population, and this case, the one of \-- the
product of interest for today, today's risk assessment, will be the deli
meats. It's on the far left.

Okay, there were some broad themes that were re-emphasized as a result
of this draft risk assessment. The disease is primarily a disease of
at-risk populations. It's rare, but severe. And there's a substantial
difference between food categories in terms of their relative risk. It
also emphasized another couple of other major factors that have not
changed, I might note, as I talk about updating. These general themes
and general factors still remain true. The amount and frequency of
consumption is an important factor in terms of the foods consumed. The
frequency and levels of contamination. And you have to deal with both.
The ability of the food to support growth is a critical factor. The
refrigerated storage time and the refrigerated storage temperature all
contribute as major factors determining the risk of any food category.

Just to review very quickly how we handled this in terms of getting
information and transparency and public input, et cetera, there were a
whole series of public meetings, advisory committee meetings, internal
and external scientific reviews at various stages along the process,
including a six-month public comment period, where we got a variety of
very useful comments. This is the process as it now stands. We're
anticipating that the final version of the risk assessment will be made
available in June or early July of this year. There will be, of course,
public meetings and potential future updates of the model as needed. We
did get a series of very helpful comments as a result of the comment
period. We got them from a wide variety of groups. I would like to thank
any of you that were involved in any of those comments. They were
extremely useful, and we did update the model in a number of ways in
response to those comments. Probably, we \-- a lot of comments on the
food categories, their definitions, what was in them, what was not
within them. There was such an ongoing debate both within FDA and within
the community on how to define cheeses, that I never want to hear about
cheeses again. We did split frankfurters into two categories, as we had
gotten more information that I'll talk about. We did move vegetables and
fruit salads around, taking them out of fruit salads and vegetable
salads and moving them into the deli category. And there are a series of
other categorical changes about specific things that should or should
not be in any of them. This is no small feat to move them around. Again,
it's a huge database that we're working with now. We also took the
advice of a number of commenters on how to weight contamination data
according to geographic location, year collected study. That is that we
didn't throw the data out, but we found a way of weighting it, or that
some got more importance than others.

We did receive, as a result of input from groups, and also from actively
going out and trying to get some additional information, some new data
that's been incorporated. I just would like to focus on two. AMI
provided a survey of information on several different factors. I just
wanted to highlight this one. Home storage of deli meats and
frankfurters, an area that we didn't have much information. We did get
some very good data on percentage of products that's held for a certain
amount of time in the home. And then also, we found out some data on
people that say, you know, what percent of the population don't eat
these products, period. We also got an extremely useful set of data, in
part because we helped pay for it, from a project that was done through
CFSAN in conjunction with the National Food Processors Association,
where we looked at the frequency and prevalence of Listeria
monocytogenes in a number of different foods, again at retail. We looked
at in excess of almost 32,000 additional samples of these foods
collected at a variety of supermarkets in Maryland and California. I'm
not at liberty to share all of this because it's about to be published,
but I did want to share the deli meats. There are almost ten \-- there
was over 9,000 of them tested. Out of that 9,000, 82 of them were
positive for Listeria monocytogenes. That works out to be just under 1
percent contamination rate. The values that you see below that are the
actual distributions because not only did we go out and determine how,
if the product had Listeria, but we also looked at the levels. So you
see that most of the contaminated product, if it was of the 82 samples,
were extremely low levels, less than 1 CFU per gram. But there was one
up there at the top that wound up having a level of between 1,000 and
10,000. And we found these distributions for a number of different
products. These were extremely useful in updating the risk assessment. I
might note, we did separate pre-packaged from deli packed, and those are
some of the data that we did find a difference between the two groups.

We have done some changes in the dose response model to make
improvements in that. I'm not going to get into it. This is stuff only a
modeler would love. And a modeler spent an hour trying to explain it to
me yesterday. The basic line is it didn't \-- it didn't impact the dose
response curve in any major way, or not much. What it did is gave us
better estimates of our uncertainty associated with it. And that is a
lot of the updating that I wanted to talk about. There are no major
changes in the results of the risk assessment. They still pretty much
say the same thing. There are some variations in rankings, is because
you've got five foods that rank out very close to each other. However,
what it did is it really decreased our uncertainty. And this is just an
example here. I looked again at the deli meats in an elderly population.
And that second column, the median value, is the predicted level of a
predicted number of cases associated with deli products. The original
model gave 650 with a range and a 95 percent confidence interval between
9 and 32,000, a fairly wide confidence interval. The changes in the
models have allowed, and the new data sets, have allowed us to hone that
in where it's 850, but now our confidence interval is 165 to 1100. So
the updating, you're going to see a lot of changes in our uncertainty.
Likewise, this is the per serving value on a per serving basis. For
these products, is 2 times 10 to the minus 7th. Again, you see a
substantial decrease in the uncertainty associated in the new revised
model.

Okay, and just summarizing, because Karen is busily passing me little
notes.

MS. HULEBAK:

Only one.

DR. BUCHANAN:

Only one.

MS. HULEBAK:

So far.

DR. BUCHANAN:

The revised model has taken in these new data sources. We've taken into
account the public comments that we've gotten. We've tried to address
each of them. And, in fact, we will, in the final model, have an
appendix on how we addressed each of the public comments. It is
completed. It's now undergoing scientific and management review in both
FDA and FSIS. The revised Risk Assessment Report is in the process of
being prepared. It's no \-- again, no mean feat to work with a document,
when you start counting the appendices and data sets, et cetera, is
about 500 pages long. But we will make a user-friendly summary, as we
did the first time. And I just want to, again, emphasize for the
purposes of today's discussion, that deli meats, the subject of the risk
assessment here, remain among the highest risk foods on both the per
annum and a per-serving basis, and that really has not changed. And,
hopefully, we'll have someone here. I'll be able to stay until noon, and
I was hoping that Sherry would be here at some point just for questions.
But, hopefully, she'll arrive shortly. Thank you. Oh, and if you \-- any
of you that don't have a copy of the original Draft Risk Assessment, it
is still available at this web site, but it takes a while to download.

MS. HULEBAK:

Thank you very much, Dr. Buchanan. And thank you for staying in your \--
within your allotted time. Our next speaker is Dan Engeljohn. Dan has
Bachelor's of Science and Master's of Science Degrees in Animal Science
and a Ph.D. in Nutrition from Howard University. He has worked at the U.
S. Department of Agriculture since 1979. Since 1998, he has been a
Senior Policy Manager in one position or another in FSIS. In September
of 2002, Dan began serving at the USDA, as in the FSIS Senior Management
Team as the Manager for Policy Analysis and Formulation. Dan Engeljohn,
please.

DR. ENGELJOHN:

Well, thank you very much, Karen, and good morning, everyone. Glad to
have the opportunity to be here today to give you an overview of the
role that risk management plays in the issue of the risk assessment. And
so I hope to answer, at least address, three issues for you this
morning, which are what is risk management, what is the problem that
we're trying to solve, and then what are the risk management questions,
specifically, that we within the Policy Office asked the risk assessors
to design the model to address.

Risk management, as I've defined it here, is taken from a Codex
definition, which is in a Codex document on principles and guidelines
for the conduct of microbiological risk management. This is from their
document from July of 2001, where it defines risk management as a
process of weighing policy alternatives in consultation with all
interested parties, considering risk assessment and other factors
relevant for the public health protection of consumers for promoting
fair trade practices and for selecting appropriate prevention and option
controls where they're needed. Risk management is, in fact, a distinct
activity from risk assessment. I serve as a risk manager within the
agency, being in the Policy Office. That's how we define ourselves. As a
risk manager, we're responsible as the primary users of the risk
assessment outputs. We formulate the food safety issues or problems, the
questions, objectives and goals that we present to the risk assessors.
And then we provide those risk assessors with the questions to be
answered. Back in the fall of 2002, where we actively worked to help
with the design of the risk assessment in terms of narrowing the focus
of what we needed to have answers for in order to pursue our rulemaking,
which we had initially published as a proposed rule in February of 2001,
one of the primary issues or problems or objectives that we need to
accomplish was that we needed to strengthen the scientific information
and the analytical data serving as the formulation of policy. And, in
this case, it was a proposed regulation that we issued acknowledging
that we did not yet have full scientific support for the way that we
crafted the proposed rule. However, under the circumstances, we realized
that we could not sit back and wait until all the science was available
to us. We needed to proceed with policy development. And we identified
those areas for which we believe there could be substantive input from
the stakeholders. For those of you who want to go back and look at what
that \-- many of those issues were, specifically, they're in Section 9
of the proposed rule, which was the Scientific Information and Data Need
Section. And that's on page 12,609 of the February, 2001 Federal
Register document. It outlined very specifically why, for the Listeria
portion of the rule, we, in fact, believe that testing in combination
with process control was an appropriate means for reducing
contamination, and ultimately affecting public health, and it identified
why we believe that we needed to address how distinct from sanitation
standard operating procedures controls and the data gaps that we believe
were significant and needed to be improved upon in order to pursue a
final rulemaking. I want to give you an overview of the 2001 proposed
rule. It's critical that I do that because it serves as a basis for how
we go forward with a final rule. For those of you not familiar with the
rulemaking process, when we issue a proposed rule, we lay forward the
concepts for which we intend to act upon with regard to a final action.
Our obligation is to provide the concept for which we intend to address
the rulemaking, and provide the stakeholders with sufficient information
to be able to make informed opinions and decisions about that particular
concept. In this case, we provided some fairly specific criteria that we
believed were necessary, identified that we did not have full scientific
support for many of the issues, and asked for comment on that. That's
where we are today in that in order to go forward with the final
rulemaking, we have to stay within the context of what we proposed and
address the comments that came in. If comments came in that help us to
modify the direction that we were going, then we can adjust the final
rule to address those comments. So we have some latitude as to how much
we can go beyond the proposed rule. But the basics where we start are in
that proposed rule language. And so for the Listeria component, I'd like
to just summarize that we identified that we believed it was necessary
to have process control for Listeria monocytogenes, and this could be
done in two ways. One would be through the HASA plan, in which a food
safety hazard would be reasonably likely to occur after lethality but
before final packaging. It could \-- all the components of HASA would be
applicable in that there would need to be a procedure designed, there
would need to be control procedures that were validated, monitored and
verified, along with corrective actions. And so that would be the
concept within HASA, but the control would be as if it were a critical
control point specifically related to Listeria monocytogenes.
Recognizing that Listeria is, in fact, an environmental issue, the
Agency accepted the fact that it could be addressed adequately through
the sanitation standard operating procedures. But a provision for that,
as we proposed, was that there would need to be testing of food contact
surfaces after lethality but before final packaging, and that that
testing would need to be completed within certain designs of a sampling
program. And just to remind you, the issues that we had were that, at
the time, we did not have information about production volume. We
believe that there may be, in fact, differences between large
establishments, those with 500 or more employees, small establishments,
who have fewer than 500 but more than 10 employees, or very small
establishments, that have fewer than 10 and less than 2.5 million
dollars of production a year. So we proposed that in order to be cost
effective with this proposed rule, that a minimum level of testing was
necessary, and based that on establishment size, where the large
establishments would take four tests a month per line per ready-to-eat
product, a small establishment would take two tests per line per
ready-to-eat product, and a very small establishment would take one test
per line per ready-to-eat product in that establishment.

Along with the sanitation SOP, to clarify, because we don't have all the
criteria specified, as we do in HASA, we would expect there to be
testing to verify the effectiveness of the sanitation SOPs. We would
expect that those results would be available to FSIS for review, upon
asking. And that positive results for Listeria species would, in fact,
be addressed to corrective action procedures. Those procedures would
include such things as identifying the disposition of the production
allotted product that was affected by the Listeria species positive. And
then identifying actions for what to do about future product that's
produced. And, in particular, we would required that product had to be
tested for Listeria monocytogenes after a Listeria species was found on
a product contact surface. That gets us then to designing the risk
management questions that we asked to have the risk assessment designed
around, and these questions are summarized as follows, and there are
three questions that we posed to the risk assessors. The first is how
effective are various food contact surface testing and sanitation
regimes on mitigating Listeria monocytogenes contamination in finished,
ready-to-eat product, and in reducing the subsequent risk of illness or
death. The second question that we posed was how effective are other
interventions such as pre and post-packaging interventions or the use of
growth inhibitors in mitigating Listeria monocytogenes contamination in
finished, ready-to-eat product, and in reducing the subsequent risk of
illness or death. And the final question, number three, was what
guidance can be provided on testing and sanitization of food contact
surfaces for Listeria species. For example, the confidence of detecting
a positive lot of ready-to-eat product given a positive food contact
surface test result. These are the three questions that we posed to the
risk assessors, for which they then took those questions, redesigned, or
at least designed the program that they were intending to provide as
outputs. We provided them the questions for which we wanted the model to
be designed around. They, then, will take data, design the model to
address these issues, and provide outputs to the risk managers, such as
myself, to be able to take, put into the form of options that we will
look at in the form of a cost benefit analysis to determine what are the
most effective ways to reduce public health risks from Listeria
monocytogenes within the resources available to industry and to
government. So that would be the next step that we, as risk managers,
will have once we receive those outputs from the risk assessors. I do
want to add that because of the way that the proposed rule was designed,
which focused on food contact surface testing for Listeria species, as
well as Listeria monocytogenes in a HASA plan, is that none of the
questions that we posed relate to the food contact surfaces. Thank you
very much for your attention, and we will have some opportunity to
address now the risk assessment as it's designed to answer these
specific questions. Thank you.

MS. HULEBAK:

We've moved along pretty effectively and efficiently this morning so we
have some time for some questions from the floor for this morning's
speakers before we take a break. Are there any questions for Ms. Kause,
Dr. Buchanan or MR. Engeljohn \-- Dr. Engeljohn? We have a caffeine
deficit. Well, in this case, we'll take a break, and please get a cup of
coffee.

\*\*\*

\[Brief recess\]

\*\*\*

DR. MACZKA:

I'd like to start by just doing a little recap from this morning. This
morning you heard from our Under Secretary of FSIS, Dr. Murano. You also
heard from our Administrator of FSIS, Dr. McKee. And they talked about
the public health challenge that LM and ready-to-eat foods presents to
both the industry and to FSIS. Dr. Murano also talked about the
importance of using risk assessment to inform risk management decisions.
You also heard from Dr. Buchanan. He made a presentation on FDA's FSIS
relative rethinking for LM in various food categories. That risk
assessment was designed to identify the foods that pose the greatest
risk to public health. Results from that assessment indicated that deli
meats are a high risk product on a per annum basis. This has been
confirmed by recent food-borne outbreaks involving Listeria in sliced
turkey and chicken deli meats. FSIS investigations of those outbreaks
provide evidence that LM in these products were due to contamination of
product contact surfaces and non-contact surfaces. So the question is
what is FSIS doing about this public health challenge? Dr. Engeljohn
spoke this morning about the proposed rule and the FSIS directive for
sampling of product contact surfaces. In addition to recent Listeria
policies and directives, FSIS also initiated this risk assessment. The
risk assessment, as Dan Engeljohn discussed this morning, was designed
to answer the following questions: What is the effectiveness of testing
and sanitation of food contact surfaces on mitigating product
contamination and reducing the subsequent risk of illness? What is the
effectiveness of other interventions such as pre and post-packaging
interventions? And how frequently should establishments test and
sanitize contact surfaces for Listeria species? In response to these
questions, we developed a model. And the model has two major components.
There's the dynamic in-plant model, which predict LM concentrations at
retail. And this was coupled with the updated version of the FDA/FSIS
Listeria Risk Assessment to predict human health impact. The risk
assessment that wad developed focuses on deli meats. And it considers
contamination only from food contact surfaces. In the next 2-1/2 hours,
we're going to describe the model in more detail, including the data
outputs, the model outputs, the data needs. And then this will be
followed by a panel discussion and a question and answer period. We will
end with our next steps. I'm going to do something a little
unconventional at this point. I'm going to tell you what the major
findings are coming out of the risk assessment. And they can be
summarized in five bullets. I'm doing this so that you're not sitting at
the edge of your chair in suspense as we unfold this 2-1/2 hours. So,
first bullet. Food contact surfaces found to be positive for Listeria
species greatly increase the likelihood of finding ready-to-eat product
lots positive for LM. The frequency of contamination of food contact
surfaces with Listeria species appears to encompass a broad timeframe
and the duration of contamination lasts about a week. The proposed
minimal frequency of food contact surface testing and sanitation as
presented in the proposed rule results in a small reduction in the
levels of LM in deli meats at retail. Fourth, increased frequency of
testing and sanitation leads to proportionately lower risk of
listeriosis. And fifth, one of my favorites, combinations of
interventions appear to be much more effective than any single
intervention in mitigating potential contamination of ready-to-eat
product with LM and reducing the subsequent risk of illness or death.
With respect to that last one, things like combining testing and
sanitation maybe with post-packaging interventions we found was a
factor. Now, I'd like to introduce our next two speakers. But before I
do that, I want to emphasize that the development of this model was a
real team effort. But special acknowledgements really need to go to the
people sitting at the table here. They are three Senior Risk Analysts.
Dr. Gallagher, Dr. Ebel and Ms. Janell Kause. I also want to acknowledge
the efforts of Under Secretary Murano, who works tirelessly to ensure
that FSIS regulations, policies and risk management decisions are based
on sound science.

So to introduce Dr. Gallagher, Dr. Gallagher is an Associate Professor
at Virginia Tech in the Department of Civil and Environmental
Engineering. He has extensive background in the development of models
for environmental testing. He has co-authored over 50 publications. Last
year, he was a Triple AS@ Fellow working with us at FSIS, and now he is
working with us on an IPA. Dr. Ebel has two degrees. He's a D.B.N. and
he also has an M.S. He is a Senior Risk Analyst at FSIS. He's been
involved in the development of several microbial risk assessments,
including a risk assessment model for salmonella intruders in eggs and
egg products, and a risk assessment model for ecoli in ground beef. Dr.
Ebel has gained international recognition for his work. He has played an
important role in many FAO/WHO expert consultations involving microbial
risk assessment. So again he's going to start to discuss the in-plant
model. Again, he'll discuss the data input, the model output, results
and data needs. And Dr. Ebel will discuss the coupling of the in-plant
model with the updated FDA at his site risk assessment model. We \--
these \-- both Dan and Eric have agreed to answer questions on
clarification during their presentation. But we all should really
restrict this to issues of clarification. There is a comment \-- a
question and comments period a little bit later on for more substantial
questions. So if you find that you're kind of moving towards a more
substantial question, I'm going to yank you. So let's keep it to points
of clarification.

DR. GALLAGHER:

Thank you. Thank you for this opportunity. Dr. Buchanan, a few minutes
ago, put up a slide and said, this is data only a modeler would love,
and kind of skipped over it. Fair warning, for the next two hours,
you're in the hands of the modelers. We're going to go through the model
in as much detail as I think we can in the timeframe. We'll present all
the data that we used, how we used it, what model results are. My goal
is to try and make this model as transparent, as usable as I can in the
next two hours. As Carol has said, I'm comfortable. If there's a concern
or \-- not a concern. If there's something that I say that you don't
understand that you need a clarification on, you know, raise your hand
and just shout it out. Let me get it while I can. Because if you sit for
the next hour and a half while we keep blathering on about graphs and
trends and things like that, it's not going to do you any good. At the
same time, we need to get through a lot of material now so that please
save general discussion questions, statements, until the end. There will
be time for that at that point.

All right. We're going to talk about the FSIS Listeria Risk Assessment
Model. The co-authors have already been introduced, so I'll just start
right down on it. To give you an overview of the talk, one more \-- I
have one slide again on the FDA model and to introduce how we used it,
because it is part of this model. I'll \-- you'll see one more time, the
risk management questions. Models are designed to answer specific needs.
So you have to judge the usefulness of the model as to whether it met
those questions. Then I'll start to describe our risk assessment model.
I'll give a conceptual diagram and go through it for a little while.
Then we'll talk about how we turned that conceptual model, what we think
is happening in a plant, into a set of computer code. What data was
available for us to use, what assumptions we had to make, and then what
the model actually looks like. At that point, we'll start to show you
some of the results from the model runs. You've seen the final results
coming up, but we'll show you the evidence where they came from.

This is the FDA risk ranking model, and since you're in the hands of the
model, let me put in plug for Clark Harrington. Clark Harrington is the
modeler at FDA, who has spent a lot of time sitting in front of a
computer screen actually coding this material up. You've seen the
rankings before. Dr. McKee gave a very good intro. Let me point out one
thing that was useful from our point of view. When we looked at the FDA
model, what we saw was a model, risk assessment model, that started off
at retail, that took concentrations of Listeria monocytogenes at retail,
carried it all the way through to consumption. And then, based on that,
could predict a per annum or per serving list of listeriosis. That was
already in place when we got handed the risk management questions, all
right? It had been out there for public review and public comment. It's
a good \-- it's an excellent piece of work. We wanted to build upon that
model. So what you'll see coming up is that what we did is we extended a
model that goes back into the food processing plant, okay? But then
couples to this model at the retail location. The risk management
questions again, just one more time, because you judge a model on how
effective it is based upon the questions that it needs to answer, okay?
How effective is testing and sanitation on food contact surfaces in
preventing listeriosis? What other interventions might be used that
would have some type of effect? And then, if you do food contact surface
testing, how effective is it in helping you find positive ready-to-eat
product? When we first started sitting down about thinking about LM \--
how LM can get on ready-to-eat product at retail, there were three basic
sources that we could conceptually identify. One would be an inadequate
lethality during processing. All right? In other words, the kill staff
is not complete. Something passes through that cooking step. Another
possible source is direct deposition from a non-food contact surface,
sometimes called an environmental surface like a floor drain getting
sprayed up directly on the product. And then finally, a transfer from a
food contact surface, itself. Risk management is currently asking for
some supporting evidence that lethality is working properly. Antidote 11
says some of the published literature suggests that non-food contact
surface transfers are rare and probably small scale. Our model focuses
and assumes, as a start, that all of the LM comes from transfer from a
food contact surface, okay? That's the source, that's the assumption in
our model to get it started. I'll give you a brief overview of the
model. It's a dynamic model. That means it tracks production of
ready-to-eat product over time. It's in-plant because we want to couple
it at retail to the FDA model, okay? It's Monte Carlo because we do
allow for some uncertainty and variability within the model, okay? And
we predict the output of our model is a prediction of the LM
concentrations at retail, okay? We then couple that. Eric sits down and
transfers the numbers from the one screen to the next screen of the FDA
model as an input, and then we run the FDA model on it, all right? We
take a mass balance approach. Mass balance is a fundamental concept in
environmental engineering. Basically, once a bacteria cell gets added to
the food contact surface, we track where it ends up, okay? Maybe it gets
transferred to the product, okay? Maybe it gets killed during the
sanitation step. But we can track where all the bugs that started at
were added to the food contact surface inside a plant, where they
eventually end up. The model, itself, incorporates food contact surface
testing, product testing, okay? Sanitation, pre and post packaging
interventions and growth inhibitors or product reformulation that would
inhibit growth. To date, we have conducted it on deli meats. Okay. This
is the conceptual model, okay? And if you understand this, a lot of the
details of the code later on, you can pretty much ignore. The model
assumes that there is a Listeria reservoir somewhere in the food
processing plant. Harbored sites, okay? Environmental sources that can
be transferred to a food contact surface. That is there and that is
always there, okay? The model doesn't have to describe it. It just says,
it's present when it needs to. Where the model starts is what \-- during
what we have termed a contamination event. Something that causes the
Listeria to move from that reservoir onto a food contact surface. We
needed three piece of data of information to describe that transfer,
that contamination event. How often, how far apart in time does one of
those start, okay? Are they three months apart, are they three days
apart? How often do they occur? Once they start, how often does it last?
And, finally, while it's going on, how many bacteria are being
transferred from that reservoir to the food contact surface? All right?
So timing, duration and number are the three pieces of information that
we needed to fill in to define what a contamination event is. And I'll
show where \-- what we did with those a little bit later.

Once the \-- and, at this point, we're still talking about Listeria
species. Haven't yet narrowed it down just to Listeria monocytogenes,
okay? But once they're on that food contact surface, we can then go and
test the surface, okay, for Listeria species, okay? Now, formally, I'm
going to say testing a lot. You'll see in the actual model, testing, by
itself, doesn't help. It's the intervention that results from testing
that actually would control listeriosis. So, informally, I might use the
term testing, but any time I do that, in your minds, think testing and
intervention. I'll show you what those interventions are in just another
slide or two, all right? Well, we can test the food contact surface. If
it's positive, we can apply some type of intervention or corrective
action. At that point, we transfer. The meat is \-- the ready-to-eat
product is passed over the food contact surface, okay? Some of the
Listeria will transfer from that surface to the product, itself, okay?
We have a transfer coefficient that describes how much of that happens.
So if the transfer coefficient is 50 percent, there's 100 bugs on the
table or the slicer, okay? Fifty of them are going to end up on the
ready-to-eat product. Fifty of them will remain on the surface, okay?
Because we're tracking all of the bugs that ever got entered. So we have
a transfer coefficient, okay? At this point now we have the Listeria
species concentration on the ready-to-eat product. So many colony
forming units of Listeria species per gram of product. We need to be
able to monitor that in terms of Listeria monocytogenes, not Listeria
species. So we need to have some type of ratio there that takes that
Listeria species number and converts it. Some fraction of that would be
represented as Listeria monocytogenes, okay? So at the end, coming out
of this box now, we have an estimate of the Listeria monocytogenes
concentration on the ready-to-eat product. Reasonably clear?

UNIDENTIFIED SPEAKER:

Question.

DR. GALLAGHER:

Yeah.

UNIDENTIFIED SPEAKER:

In developing the transfer coefficient do we take into account the
material of the food contact surface?

DR. GALLAGHER:

Well, not the type of material. Give me two minutes. There's a slide on
the transfer coefficients. All right. We're going to pick up at that
arrow, leaving this one, okay? At that point, the plant might have
implemented pre and post packaging controls, okay? We couldn't fit that
on the tab in the model so we call it post processing. Steam
pasteurization, okay? Product reformulation that prevents \-- no, that's
the next one. Take stem pasteurization. Maybe in the future, something
like irradiation, okay, could go on there. That will actually kill off.
That will reduce the number of bugs, the concentration of bugs, on the
product. And so that's an actual die off. So, at this point, here we
have the concentration on the product as it's about to leave the food
processing plant. We can then test that product, if we want, for
Listeria monocytogenes. And careful, the food contact surface tester for
the Listeria species, the product tests are for Listeria monocytogenes,
okay? But we can test it, and if it's positive, okay, we don't let that
particular lot pass on into the human food supply, okay? Whether it's
re-cooked, whether it's disposed of, the model doesn't care. But that
concentration is blocked. Finally, then we leave the plant, and there's
a transport to retail, okay? That can take, according to USDA estimates,
anywhere from 10 to 30 days. There's potential for growth occurring
during that transport. And so there's \-- we've got a growth factor in
between the plant and when it actually shows up at retail. And then the
end result of our model then is the Listeria monocytogenes concentration
at retail, okay? That's where the FDA model picks up. Okay, on that one,
okay, \-- all right. The outputs are things you've probably already seen
risk of illness or death on a per annum basis. The slides all today are
all talking about death from Listeria monocytogenes in deli meat, okay?
What we could vary is testing for Listeria species on the food contact
surface, okay? Testing for Listeria monocytogenes in the ready-to-eat
product, as well as pre and post packaging interventions. One of the
other questions that Dr. Engeljohn had mentioned earlier, they did want
some guidance on if they find a positive food contact surface, how does
that improve their odds of finding Listeria monocytogenes in the
product, itself? And so you'll see we'll give you some of those results
in a minute. Okay. All right, so some key data requirements. I'll go
through all the data in a little bit, okay? But these were the four that
seemed to be the key to get hold of before we thought we could actually
go and implement this model. First one I've already talked about,
contamination events. Frequency, duration, levels. How much gets
transferred? The next one are the transfer coefficients. How much of the
bacteria moves from the contact surface into the product as it's passed
over it? We also needed the Listeria monocytogenes to Listeria species
ratios, okay? We had to convert those concentrations within the product,
itself, okay? And then finally, we needed production levels by plant
size, okay? How much does a line produce during a typical shift? I will
go through each of these, then I'll show you the models with some of the
other input datas. For the frequency of contamination event, what we
needed was time series data on food contact surfaces. The data source
that we could find for that was an in-depth verification study that FSIS
had conducted a few years ago. What it gave us was Listeria species
prevalence over time for various food contact surfaces at the same
plant, okay? So it was a time series. And we fit that with a survival
analysis, okay, to come up with what was the best probability
distribution to describe it, okay?

It turned out that it logged normal distribution with a mean time
between contamination events of about 23 days fit the data best, okay?
The standard deviation was about 38 days. Yeah, I'm sorry.

UNIDENTIFIED SPEAKER:

Single plant?

DR. GALLAGHER:

Yeah, single plant for this one. Uh-huh. That was the only plant that we
could find that had almost daily time series that could be provided to
us, okay? All right. So what the model will do then is every time it
needs to, it will generate a time series of when contamination events
occur that are still caustic, okay? On average, they're 23 days apart,
okay, but there is a standard deviation. They're not always exactly 23
days apart. For the duration, we used a table that has been published by
Dr. Tompkin, okay? He has a table in there that talks about how often
plants would find weekly consecutive positives on food contact surfaces
for Listeria species, okay? So a given number of plants would find it
two weeks in a row, but not a third. A few more would find it three
weeks in a row, but not a fourth, all right? And we took that table
again, fit it with a survival analysis. Log normal distribution. Fit
pretty well. So we have an average duration of contamination events just
a little bit longer than a week, okay? So once it starts, it tends to
last about a week. And there's a standard deviation again. That's a
stochastic input to the model. It will vary from lot to lot or from
contamination event to contamination event.

UNIDENTIFIED SPEAKER:

From that data, how do you determine that it was not a different event?

DR. GALLAGHER:

Could two events occur simultaneously? Yes. I'll show you one of the
slides, but basically, it would just overlap. It doesn't double the
amount that's being transferred. But two of them could lap, overlap,
together.

UNIDENTIFIED SPEAKER:

Now the working assumption you have on this data is that if you had a
duration of eight days, then eight days were a single contamination
event?

DR. GALLAGHER:

The model doesn't care, okay? In other words, I could have, just for
example, we've said about 23 days apart, okay? But that's stochastic. So
maybe one time I just have three days apart, okay? But the duration of
the first one was five days. The duration of the second one is also five
days. They overlap, okay? But what that means is a contamination event
is occurring for that total of ten days. That's all the model needs to
know. It doesn't actually count or track the number of contamination
events that occur. Okay, the transfer coefficients. Okay, and this is
how much transfers from a food contact surface to the product. We went
to the published literature, okay? Montville \[ph\] and Chen, both
students of Dr. Shaffner, who you might recognize, found the transfer
coefficients pretty generally log normally distributed, and they \--
they were looking at things like hands, to faucets, lettuce to cutting
boards, not necessarily LM. So their transfer coefficients, themselves,
we did not use. They tend to be fairly small. They did find a standard
deviation pretty consistently across all different media of about one
log unit. And so we did \-- we did end up using that standard deviation.
A paper by Middlet \[ph\] and Carpentier \[ph\] found \-- looked
specifically at LM, as well as some other bacteria. But, in this case,
they have a graph that deals just with Listeria monocytogenes, okay?
They found that after 12 contacts with a surface that they had
pre-plated or pre-contacted with LM, they got between 60 and 100 percent
of the bacteria were transferred to the product, okay? So, basically, we
used their mean transfer coefficient with the standard deviation from
Montville and Chen. All right, so we used a log normal, a mean of almost
100 percent transfer, and a standard deviation of one. And because, lots
of times we're talking we can't transfer more than is all \-- all that
is there, okay? We would truncate it if the simulation was predicting
more than one.

UNIDENTIFIED SPEAKER:

Question.

DR. GALLAGHER:

Yeah. Give me a shout or something because I'm\...

UNIDENTIFIED SPEAKER:

Just a clarification on that. Back to what you said about mass balance.

DR. GALLAGHER:

Um-hum.

UNIDENTIFIED SPEAKER:

Then if you have 100 percent transfer, does the remaining surface is
clean or negative?

DR. GALLAGHER:

No, it goes to zero.

UNIDENTIFIED SPEAKER:

That's what I mean, it goes to zero?

DR. GALLAGHER:

Yeah.

UNIDENTIFIED SPEAKER:

So if you have a contaminated surface at 100 percent, correct, there's
no transfer, there's no subsequent transfer?

DR. GALLAGHER:

No, onto additional product, I don't think, unless now we really \-- if
the contamination event's continuing, okay, we do recontaminate the
surface at the beginning of each line of production. So those \-- those
\-- that level that is transferred is transferred to the surface at the
beginning of each line of production, okay? It is \-- normally, it will
be added to what's already on the food contact surface. That's one
reason why it's an at balance approach. But if a transfer coefficient
was 100 percent, that says left over at the end, or if it was, you know,
90 percent, but sanitation dropped the rest of it down to 100 percent,
the new material that gets added on, the new bacteria that gets added
on, get added to nothing. It's starting clean at the end of that lot.

UNIDENTIFIED SPEAKER:

So, for the model you have a continuing re-inoculation and you have 100
percent model?

DR. GALLAGHER:

Not if \-- no, we had a continuing re-inoculation while the
contamination event's occurring. Now, for a large part of the time,
there's no contamination event occurring, so there's no further
re-inoculation. Yeah?

UNIDENTIFIED SPEAKER:

Does this assume any sort of growth in the bacteria? When you're talking
about the transfer coefficient, but is there any assumption regarding
the growth of the bacteria that have been transferred from the\...

DR. GALLAGHER:

This version, well there is no growth on the food contact surface once
it's been applied there.

UNIDENTIFIED SPEAKER:

Was there a reason that you didn't?

DR. GALLAGHER:

We couldn't find anything that would suggest for that length of time,
and on those particular media, that growth would be significant.

MS. HULEBAK:

Another question?

DR. GALLAGHER:

I'm sorry.

UNIDENTIFIED SPEAKER:

Does this confirm a \-- that coefficient is per food contact, is that
right, or\...

DR. GALLAGHER:

Right, and Art Delta, you'll see in a minute, Art Delta did a lot, so
that's the transfer from the food \-- so each one of these time steps is
one lot. One shift \-- one line's shift production.

UNIDENTIFIED SPEAKER:

And is that what Montville based his data on as well, or is it based on
multiple contacts, or was it single contact?

DR. GALLAGHER:

I think there were multiple contacts in that one, but again, that wasn't
necessarily \-- it was not LM. And a lot of the contacts they were
talking about it was not something would be applicable in a food
processing plant.

UNIDENTIFIED SPEAKER:

And weren't you given some additional data from a University of Georgia
study that related to transfer of LM to \-- from food contact product?

DR. GALLAGHER:

Not the one study. Is that the craft? Let's save that because I'm going
to have to think about it. I'm going to think about that. I'm not sure.

UNIDENTIFIED SPEAKER:

Okay.

DR. GALLAGHER:

But if we were, this is what we used.

UNIDENTIFIED SPEAKER:

Okay.

DR. GALLAGHER:

For the ratio of Listeria species to Listeria monocytogenes, we could
not find any concentration data that would let us define a ratio based
on concentrations, which is really what the model wants. The only
relevant data we could find was based on prevalence. Presence, absence,
positive, negative, all right? There is a table in Tompkin's paper, and
we had some blinded industry data that was provided that told us if they
took a set of samples, what proportion that were positive for Listeria
species, what proportion was also positive for LM, okay? That is
prevalence, okay? We've got this number of 50 percent from various
sources. I'm pretty comfortable with that 50 percent in terms of
prevalence. We had no other data. We just assumed it also applies to
concentration. That is an assumption, okay? But that's based upon \--
it's \-- this is where it comes from. We couldn't find any other data
that would do something better than that. Okay, well I did change that,
played with that number a little bit in the model. I'll show you some
results coming up. Finally, we needed production levels by plant and lot
volumes, okay? The source of this data was the FSIS RTE Survey, okay? We
got a lot size per line, per shift for the different plant sizes. And
the plant size here is defined by the number of employees, okay? Okay,
the large plants tended to produce larger lots, okay? Very small plants
produced the smaller lots, okay? We also got a total amount of
production into the human food supply from each of those, so that the
large and the small plants both produced about 48 percent of the
material that ends up at retail, okay? Small plants, very \-- I'm sorry.
The very small plants only provided about 4 percent, okay? So you'll see
this data coming up in a minute, okay? We had no evidence that very
small plants are more contaminated, so the model makes the assumption
that the food contact surface varies with those mean lot sizes. So you
remember what's happening here. I've got, let me just say 100 bugs on
the food contact service. And let's just say transfer all of them
because I can do 100 in my head even in front of you, all right? All
right, now I've got 100 bugs. I have to get a concentration. I have to
divide it through by the mass of the lot, okay? If it's a large plant,
that's a large lot size. So the concentration that would result would be
small compared to if it's a very small plant. The divisor, the lot mass,
is a lot smaller there, so the concentration is higher. We had no
evidence that that is actually what is occurring, okay? So what we did
is we assumed that the food contact surface area by plant size tracked
that same 19,728, okay? So that on average, we get the same
concentration coming out of each of those plant sizes.

MS. HULEBAK:

A question.

DR. GALLAGHER:

Okay.

UNIDENTIFIED SPEAKER:

You keep referring here that the \-- and I don't understand the problem
though, and I don't think you've increased the food contact surface. Is
there a greater probability of having contamination or by reducing the
contact surface is there a lesser probability? Is there inner
relationships here that you've established per size?

DR. GALLAGHER:

Not in terms of it occurring, okay? But we transfer a given number of
bacteria. The term was inoculate, if you're willing to use that one, to
the food contact surface. We have to transfer \-- translate that into a
concentration on a food contact surface. How many bacteria per square
centimeter? That's what we go test. If that concentration drops below
what we can detect, then we're going to find negatives on \-- we're
going to report negatives on the food contact surface, but there are
still Listeria species there. All right? It doesn't change the
probability of what's getting there, but it does change the
concentration on the surface. Finally, how much happens during
inoculation? How much gets transferred during each lot production while
a contamination event is occurring? I have no idea. I have no clue. And
we've checked all literature, and we asked people. I don't know, and I
have no evidence of what the right number there is. But what we do have
is FDA's distribution of LM at retail. Okay? That's based upon industry
data, FSIS data. Okay, that's a known quantity, okay? There might be
some uncertainty about it, but it's a known quantity. We can fill in
these gaps here by running our model in an enterative basis where I'll
just pick my favorite number, run the model, get a concentration at
retail, compare it to what it should be. And then based on that \-- oh,
it's too low. Well, I'll have to add more bacteria. So I'll run it
again. I'll change that number and run it again. And we would do that in
an enterative process, and we call it a calibration, until our output
under base line assumptions match the FDA distribution at retail. That's
how we filled in this piece. And I will show you that calibration slide
coming up. Again, for that calibration then, the main \-- the \-- what
we calibrated to was the LM concentration at retail. That was what we
would adjust our factors to. We did want to keep in mind that there has
been prevalence data reported, okay? We did not calibrate it to it,
okay, but we didn't want to be completely off the mark in terms of what
our model's predicting for prevalence versus what different literature
or industry or other sources have reported for prevalence. So the
prevalence is not a calibration factor. That's only the concentrations
at retail. But we want to check this, okay? To go back, Levine \[ph\]
using FSIS data found for different types of product, and not just deli
meat, not just ready-to-eat, a range of about a half a percent of the 5
percent prevalence of LM in the product, okay? That, if you look at the
numbers, there appears to be a general decrease with time. And that's
over I'd say the last roughly ten years or so. In about 1999, in trying
to focus mostly on products that are most related to deli, seems to be
in the 1 to 3 percent kind of range, as sliced ham, et cetera. Dr.
Lachanski \[ph\], DARS, conducted a study of looking at hotdogs. Okay,
now we're only going to be talking \-- but just again, we're just
looking for kind of information that will help support or not support,
okay? Found a prevalence in hotdogs of about 1.6 percent, okay? I should
point out that was not a random sample. Those plants volunteered to be
part of the study. And so we have to keep that in mind. The NFPA data
found that combined for both leaving the plant and sliced at the deli
counter, an overall prevalence of about .9 percent in deli meat. And
then, finally, this is some preliminary data. It has not been completed
going through the QAQC process. It's FSIS data for calendar year 2002,
okay? We're looking at the HASA Code 03G. Fully cooked, not shell
stable, under the subcategory sliced, diced and shredded, okay? So this
would be sliced ham, sliced bologna, sliced chicken breast. Some things
that are not deli meat. So this is not a perfect measure either. It
would include things like diced chicken that might be going to chicken
potpies or something, right? So this is not a perfect measure either.
But they found 23 LM positives out of roughly a thousand samples. A
prevalence of about 2.3 percent. So I'm not trying to match the
prevalence exactly, but you get an idea of what the range and what
appropriate numbers, reasonable numbers, might be coming out of the
model.

UNIDENTIFIED SPEAKER:

I have a question on the data?

DR. GALLAGHER:

Which one?

UNIDENTIFIED SPEAKER:

The samples then by FSIS under 03G, those are random samples data? I
believe that's not random.

DR. GALLAGHER:

It's \-- it's not weighted according to production, but those samples
are randomly selected from a sampling frame of plants that produce 03G
product. So it's not an estimate of national product prevalence because
the sample could represent 10,000 or 10,001. But it is randomly selected
each month, which establishments gets that way.

UNIDENTIFIED SPEAKER:

Okay, thank you.

DR. GALLAGHER:

All right. And again, we're not tying the model to these prevalences,
but I want to look at our prevalence and make sure it's not 30 percent
prevalence out of our model. That's \-- our model's wrong if we got
something like that, okay? Half percent, three or four percent. They're
in the ranges of some of the numbers that are getting reported. That's
all I wanted to do with this.

UNIDENTIFIED SPEAKER:

You said you're not tying the model to these prevalences, but you are
tying them to FDA's\...

DR. GALLAGHER:

Yeah.

UNIDENTIFIED SPEAKER:

\...level of retail, which tied to some of these numbers they marked?

DR. GALLAGHER:

But again, FDA, we're getting a \-- there's a difference between
prevalence and levels of concentrations. We're actually fitting the
concentrations that FDA is providing at retail.

UNIDENTIFIED SPEAKER:

Okay.

DR. GALLAGHER:

So not positive/negative. It's the concentrations. We'll see those
coming up. So the FDA data was a big part of that one. All right, model
implementation and baseline data, okay? Basically, we wrote it in visual
basic. It's about 4,000 lines of code. Half of that is the interface
that has the actual number crunching. Based on what we were asked to do,
I'm going to go through each of these screens, and it gets a little dull
even for me, but I'll try and not draw you out too much.

This one is actually just so you can keep track of what kind of run it
is. There's just some text entries there. The model doesn't use it for
anything. I was just \-- the user clicks on those different tabs to
enter the different data sets. You can save and call it back up. The
model's not completely finished. The print button, for example, doesn't
work. We just haven't had time yet. Everything \-- yeah, when they said
I had to put up a source code, I thought cringe. Everything that relates
to the risk assessment part is complete though. And we've \-- I can't
guarantee there's no bugs, but we've looked at it really carefully. This
is the plant data. And this is \-- let me move over because this one's
kind of important. The first part, plant size distribution, can
everybody hear me? Okay, that's the data of the FSIS farm ready to eat
service. They're seeing 48 percent from large, 48 from small. We've
talked about that data. The sanitation data \-- the sanitation data, we
assumed that \-- the model assumes that two lots are produced per day
from each line, and that the third shift is a sanitation shift, okay? We
assume that there's a wipe down in between the two lots, and then
there's a more effective cleaning at the end of it. So we have a
wipe-down efficiency of about 50 percent. At the end of the day, after
the two lots, about 75 percent. This is pretty much sitting around the
table, asking some expert opinion. What do we think an overall
sanitation effectiveness is in the plant? They said about 80, 90
percent. And if you combine these two, we're in that range, okay? We
also allow for, and you'll see in a minute, one of the potential
interventions. If you find a positive food contact surface is to go in
and clean your food contact surface more carefully, okay? So we have an
enhanced sanitation that would represent a response to a positive on a
food contact surface. The model doesn't use that one yet, okay? All
right, here's where the actual testing comes into place. We have a
number of tests. This should say per line per month on the food contact
surface. So what you're seeing right now, the initially proposed 4/2/1,
a four per large plant, two per small plant, one for very small. We call
it testing, but it's really testing and intervention. What are the
interventions that can take place if we find a food contact surface
positive? We allow for two. One is that enhanced cleaning, to go clean
the food contact surface more carefully. The other is to force a test
for the lot. Test the ready-to-eat product for Listeria monocytogenes.
So you've got the option to say, if I find a food contact surface
positive, I will then go test a lot for Listeria monocytogenes. Now,
which lot gets tested? This relates to what we're calling test and hold.
Test and hold would assume that the plant will store the lot that got
produced until they get the result back from the food contact surface
test, in which case they can then go test that particular lot that was
in contact with that positive surface, okay? So when those are checked,
okay, the lot and the food contact surface match up. If they are not
checked, you'll see later on with a screen, how long does it take to get
back a food contact surface result? And we assumed about three days. So,
in that case, the lot that was in contact with the positive surface has
already passed out of the plant.

MS. HULEBAK:

We need to use the mike for a transcript.

DR. GALLAGHER:

Oh, sorry. The lot has already passed out of the plant, and the best
they can do is then go test the lot that's currently being produced,
okay? So a lot that's three days past when you knew that the food
contact surface was positive, all right? So that's another option that
you've got. Test and hold. They'll be paired. If test and hold is not
checked, okay, they're separated by, in our baseline, about three days.
All right. We can also test the ready-to-eat product lots for LM, okay?
One way would be because we found a food contact surface positive.
Another way might be just because we think it's a good idea to test
product, okay, as opposed to food contact surfaces. So we've got the
same kind of number of tests per line per month on product for LM here.
If a positive lot is found, we have the option, and for the real runs it
was checked, then we would dispose of that particular lot. And again,
maybe it's re-cooked, maybe it's really disposed of, but that
concentration doesn't pass into the human food supply. Question?

UNIDENTIFIED SPEAKER:

Yeah. Bottom left corner, positive result action proposed does not get
checked?

DR. GALLAGHER:

Yes, both can be checked at the same time.

UNIDENTIFIED SPEAKER:

I have another question. If we go back to the food contact surface, I
know you were allowing for a time for those results to come in, but if
we are doing food contact surface testing without testing holds, it's
assuming to take a period of time for the results of the LM testing on
the product to be returned, and you're saying you assume that all the
product would be disposed of. If you're not doing test and hold, then
you've got product that was produced that's in the marketplace that
could result in, if that could be positive\...

DR. GALLAGHER:

All right, let me \-- I think I \-- if test and hold applies to do they
test, hold it for a food contact surface test?

UNIDENTIFIED SPEAKER:

That's what I'm asking.

DR. GALLAGHER:

Okay, the assumption for the product testing, all right, so in this
case, if that is off, yes, there is material that can be LM positive
getting into the food chain or food supply because it took three days
before they realized it, okay? The model for the product testing assumes
that if you test that lot, that lot will either be allowed, will come
back positive or negative. And based solely on that, will enter the
human food supply.

UNIDENTIFIED SPEAKER:

I'm not understanding the interaction then between food test and hold
because I thought the test and hold would apply\...

DR. GALLAGHER:

We don't\...

UNIDENTIFIED SPEAKER:

I'm not \-- go ahead.

DR. GALLAGHER:

I'm sorry. We don't have a corresponding test and hold for product. That
assumes if you test that lot, you just hold it. Okay, we do for the food
contact surface testing.

UNIDENTIFIED SPEAKER:

Basically, it's not an option on product then.

DR. GALLAGHER:

It's built in.

UNIDENTIFIED SPEAKER:

It's built in.

UNIDENTIFIED SPEAKER:

All right.

UNIDENTIFIED SPEAKER:

It's an option on food contact surface testing.

DR. GALLAGHER:

Thank you.

UNIDENTIFIED SPEAKER:

Thank you for your clarification.

DR. GALLAGHER:

Thank you. All right, so okay. All right, so are we okay on the plant
data? The contamination data, we \--basically, it's the material we've
talked about already. How \-- what's the timing between contamination
events? The numbers look strange but that's because we're working on a
log scale on most of these. Okay. How long does it last? What are the
transfer coefficients? These are the two boxes that we vary during
calibration. It says, contamination event levels. This is how much gets
transferred, inoculated, to the food contact surface. These are the ones
I didn't have any numbers for, so this is what we \-- during the initial
setup of the model, when we're doing the calibration, these are the two
that I'd come back and change. We also needed some definition of the
tests. If you test \-- if you swipe \-- swab for contact surface, what
area are you swabbing? So we've got some numbers in there. I'll show you
the variation of that one coming up. The composite, if you do get the
model up and running, is not implemented yet. Same thing for how large a
sample do you take in terms of the product, okay? The fault is \-\-- the
fault is 25 grams, which is the acceptable one, but we did vary that one
a little bit just to see the effect. That's all. Post processing, okay,
the model takes a fairly simplistic approach, but it captures just about
everything that we need, okay? We don't have to define what type of post
processing it is, simply how much of the industry by plant size is using
it, and then how effective it is. So, in this case, we've got,
basically, if that's point nine, a one log die off of the bacteria,
okay? We can change those numbers and rerun that and you'll see some of
that variation, okay? But again, this reduces the number of the
concentration of LM. The other one is growth inhibiting packaging or
product reformulation.

UNIDENTIFICATION SPEAKER:

Before you go on to that, could I ask you to go back to the previous
line? Under product testing, what kind of statistical sample plant are
you using in the assumption of infection efficiency?

DR. GALLAGHER:

We are assuming, and it shows up in a little bit later slide, both for
food contact surfaces and for products, that the contamination is
uniformly distributed across those because we couldn't find anything
that said otherwise.

UNIDENTIFIED SPEAKER:

Okay.

DR. GALLAGHER:

Clearly, that has some implications. For, I think, one of the other
speakers coming up, we do a, Eric help me out, a poyson \[ph\] count.
And there's a 75 percent chance that if there's one bacteria you'll find
there's a positive.

UNIDENTIFIED SPEAKER:

Okay.

DR. GALLAGHER:

All right, so post processing and growth inhibiting packaging, again,
the difference here is this is a die off of bacteria. This actually
changes the growth-drawing transport from the plant to retail. And so
one's a reduction in the number. The other's a reduction in the growth
factor.

UNIDENTIFIED SPEAKER:

Well, clarification. You say reduction. You're talking about the
bacterial cycles of activity here?

DR. GALLAGHER:

This, so post processing treatment is bactericidal, yes. It kills the
bacteria. Whether it's depasteurization, whatever you want to apply. The
number, we're doing the math balance. We could \-- we didn't count this
box, but we could. How many of them died at that step? So bacteria count
there is the 75 efficiency of finding one bacteria if it's in there. We
can go over that one a little more if you want. The testing lag for the
food contact surface, that's where we put in the three days. We talked
early on about doing a \-- product testing lag has not been
incorporated. The text box is here. The model doesn't use it. The ratio,
again, what's the ratio of LM to AL@ species? We're assuming about 50
percent of the bacteria are AL@ monocytogenes. Food contact surface
areas, because we need to convert those over to a concentration. And
this growth factor. And Eric will talk a couple slides about that one.
But again, how much growth occurs from plant to retail? And we'll talk
\-- that's \-- more about that one coming up. Okay, that's the end of
the data that you have to put in, okay? Well, one more number, okay? How
many lots do you want to produce, okay? All of the ones that we've been
doing, we've been producing a million lots, okay? The model is fairly
stable at that level of production. The model, at a million, it takes
anywhere from five minutes to an hour to run on a computer, depending on
the kind of machine you've got. You can save all the outputs to a file,
although if you run a million lots, they get pretty big. This is the
actual button you click to go ahead and run it. We needed to do that
calibration step a variety of times, so this is what the screen we were
looking at primarily for calibration. Here is the FDA distribution of LM
at retail in deli meats, okay? That doesn't change no matter what runs
we're doing. That's fixed. That's just for us to see. The model will
then predict, okay, the concentration based upon the data screens that
you entered before, what the concentrations are at retail. And then by
eye we do a little bit of statistics, but primarily, it's an eyeball
fit, okay? How well do these match up? So the first couple of runs, we
weren't close, but we would go back to that contamination data and
change that number for how many are getting added to each lot, okay?
Graphs. Graphs are just for us. We want to see the shape of that kind of
distribution. And it varies. And again, it was just mostly for the
modelers to try and figure out how well it's working and what kind of
interactions we're getting. The output steps. Okay. We can do things
like, you know, test every possible lot that ever gets produced, not do
any interventions, and just see what kind of prevalence is occurring
based upon that. To get those numbers, we've got some of these table
outputs. Okay. How many lots got produced? How many got selected to
actually go to retail? We're producing a million lots per each plant
size, but then based upon the production volume, we'll sub select out of
there that number that give us a million lots total going to retail,
okay? We can say how many lots we tested, and whether we tested them
because we want to do a routine testing of lots, okay? Or because we got
a food contact surface positive and we check that as one of the
interventions. So how many lots got tested? These two sum up to that
one. How many lots failed? How many were LM positive? How many food
contact surfaces were tested, and how many of them failed for Listeria
species positive? And we can also do this as a two-by-two table, a
contingency table, looking at each possible combination of when the food
contact surface was positive, what number of lots were positive versus
what number of lots were negative. And you'll see some of those data
coming back out in the results. One thing I should point out, you'll see
in a minute, under our model, there's a maximal amount of testing you
can do unless you want to start changing what the definition of a test
is by compositing samples from the same lot or food contact surface. The
model assumes each line produces two lots a day. They've got 30 days in
a month. If you take 60 samples per line per month, you're testing every
lot that that line is producing. So you'll see some numbers coming up
here where it says 60 tests per month, okay? That, from our point of
view, is the maximal per \-- that's per line. Is the maximal amount of
testing that this model will evaluate. Let me lead you through this one.
This is an output. It's from an earlier run, so the numbers have changed
quite a bit. But just to show you how the model thinks a little bit, and
this is my last slide. We're looking at, here's our time step, okay?
This is a lot number, but it's a delta time. And it's a third of a day
if there's three shifts per day, okay? Okay, this is the concentration
of Listeria species on the food contact surface at the end of the lot
production. That's what we can go test if we want to. And we have that
concentration for every lot whether we test it or not. We know

what that number is in a modeling simulation. This column, and okay,
this might answer your earlier question. Actually, I cut it out. No,
sorry, it won't. There was another one that talked about contamination
events and durations, and you got a true/false column for is this lot
going to be re-inoculated because we're in the middle of a contamination
event. We can do food contact surface testing. This true here in the
food contact surface testing says I'm going to take a sample of that
food contact surface, of that lot production. That food contact surface,
okay? So Listeria species \-- so that says, go test it. So we pass this
number on to the little testing algorithm. This particular case I'm
highlighting comes back and says we found a positive food contact
service. Listeria species as an above detection level for that
particular surface. Now, what do we do with that, okay? To again,
testing by itself, isn't going to help anybody. The assumption here, we
had told the model with one of those check boxes, go ahead and force a
lot sample if the food contact surface was positive. So based upon this
positive, we couldn't sample the lot just because we're sampling a given
number of lots per month anyway. This didn't happen for this one. But
this true here, lot sampled because of a food contact surface positive,
is triggered because of this finding. So we need then the LM
concentration for the lot. We have that species over there. How much
gets transferred, what the ratio is. We convert that over to Listeria
monocytogenes as a concentration. We can run through some post
processing. We want these two numbers the same because we weren't doing
any post processing at that point. We pass that number to the lot
testing algorithm. That concentration. All right, so this now represents
the concentration of LM uniformly distributed, because that's all we're
doing, in the lot. Is that enough to find it positive? In this case,
that concentration is not. And that comes back as a negative. The
concentration was below what we could detect. So this concentration then
passes on through a group out to retail.

UNIDENTIFIED SPEAKER:

Can we explore that a little bit more also because your assumption is
the organism is homogeneously distributed. Might be a reasonable one if
the site of the contamination was a slicer. However, if you're working
with an impact product where in a conveyer belt, you have a localized
contamination on the surface, the current methodology involves taking
the chunk and grinding it up, and so you get into some real shows here.

DR. GALLAGHER:

I completely agree with you. Unless I make the odds up out of my
favorite numbers, I don't know what values to put in here. So, at this
point, I'm telling you the model assumes uniform, okay? That is
something that, with more data collection, more numbers, more research,
hey, it would be nice to fill that one in a little better. All right.
So, we see, if this had turned out true, okay, and we had disposed of
the product, we'd still know what the concentration was, but that
product, that concentration, would not have passed on to the imminent
food supply. I don't see any results yet, but that's really how the
model works. I can take a minute or two for questions, then Dr. Ebel is
going to come up and talk for a little while.

UNIDENTIFIED SPEAKER:

The sanitation efficiency, the assumption that you made, is 75 percent
efficient on average?

DR. GALLAGHER:

That's right.

UNIDENTIFIED SPEAKER:

Now, what sources or references did you use to come up with that number?
What's the basis of that number?

DR. GALLAGHER:

As I said, that was primarily sitting down with FSIS personnel and
asking them what they thought a reasonable efficiency would be.

UNIDENTIFIED SPEAKER:

Is that the number that can be\...

DR. GALLAGHER:

Changed?

UNIDENTIFIED SPEAKER:

\...varied in\...

DR. GALLAGHER:

Any of these text boxes, any of those little things with the whites, we
can change and rerun the model. Okay, but that's not one I had looked
at. You'll see some sensitivities coming up. That's not one we changed.
But, yes, we can rerun that at different levels.

UNIDENTIFIFED SPEAKER:

That's my next question. You haven't looked at it?

DR. GALLAGHER:

Not at that one.

UNIDENTIFIED SPEAKER:

Okay.

DR. GALLAGHER:

Any other questions?

UNIDENTIFIED SPEAKER:

Is this just one application to the groups of plants, individual plants
or industry as a whole?

DR. GALLAGHER:

I would say industry as a whole. What we're distinguishing plants by
plant size, the number of employees. But we don't have a lot of data for
what's actually occurring to give us variability across plants. That's
not out in the literature very much. So I would say for the industry as
a whole. But distribution of LM at retail is based upon, primarily, our
best guess for retail as a whole.

UNIDENTIFIED SPEAKER:

And the numbers that you're counting for the food contact surfaces at
the area\...

DR. GALLAGHER:

Yes.

UNIDENTIFIED SPEAKER:

\...those are pretty large. Are we assuming that every food contact
surface on that line is positive uniformly?

DR. GALLAGHER:

We had one box for food contact surface. That will come up again in the
slide later on. But just like a lot is uniformly distributed, okay, the
food contact surface is uniformly distributed, and we only have one
surface. It's a generic food contact surface. What I have done, and
you'll see in a little bit, if we change that, if you think that's too
large, I've got some other numbers you can see. I did play with that
one.

UNIDENTIFIED SPEAKER:

Well, we can make it \-- well, estimated surface of a slicer and you put
that number in and say, here's what would happen if the slicer found it?

DR. GALLAGHER:

I wouldn't \-- let me think about that one. I wouldn't go quite that far
because what happened, again, what happens to the other surfaces? And we
think of this food contact surface as kind of the representation of all
the surfaces that it comes in contact with. But right, we decided early
on this wasn't dated to a separate \-- you know, it goes from a slicer
to a table to a roller. We couldn't find any kind of data and gave up on
that pretty quickly.

UNIDENTIFIED SPEAKER:

But they may not go from any more than just the one food contact surface
for their product, and that's the only surface in an isolated product?

DR. GALLAGHER:

But this model assumes there is only one food contact surface overall.

UNIDENTIFIED SPEAKER:

Just to help me understand something a little bit. I think the
prevalence, you said, was led to the development of a concentration?

DR. GALLAGHER:

Um\...

UNIDENTIFIED SPEAKER:

The prevalence statement was used to help generate the concentration?

DR. GALLAGHER:

I'm going to \-- not from our point of view. No, we used the existing
FDA concentration data to match to. And you'll see that slide coming up.
We've got the calibration slide there. The FDA, if you look at their
report, I mean they did a lot of their numbers were below detection.

UNIDENTIFIED SPEAKER:

Yes.

DR. GALLAGHER:

And I don't want to speak for the FDA, so Dr. Buchanan, do you remember?

DR. BUCHANAN:

I don't.

DR. GALLAGHER:

That's okay. I saw you look now. But you know how the FDA model handled
the negative \-- you know, negative prevalences in the data that was
reported?

DR. BUCHANAN:

We handled it as a distribution and went down beyond the lower limit
of\...

DR. GALLAGHER:

Yeah, and I believe you'll see that, and we did the same thing.

DR. BUCHANAN:

Well, everything was assumed to be positive, even the negatives.

DR. GALLAGHER:

So there is some concentration. It's just below what we can measure\...

DR. BUCHANAN:

Well\...

DR. GALLAGHER:

\...with positive. Okay.

UNIDENTIFIED SPEAKER:

So my follow-up question then to that is, or a point of clarification
is, if you \-- sampling brands and so forth that you talk about in here
as in testing and intervention, aren't they limited to a certain extent
by the prevalence of the organism or the concentration in terms of
methodology? And then if that's true \-- I know you want to say
something. Do you want to go ahead before I finish?

DR. GALLAGHER:

No, go ahead.

UNIDENTIFIED SPEAKER:

No, go ahead. I mean \-- so, is that tied into this, this model, where I
guess, you know, when I look at the numbers it's saying it looks like
they're 4/2/1, I don't see how that kind of sampling regime tied into
the prevalence data that was given, really supports taking any sort of
positive action that could be reflected in the model.

DR. GALLAGHER:

The model does not use prevalence at any point unless you, as the user,
want to see what that number is. The model tracks concentrations
throughout. If you ask for a prevalence, it then goes and tests and said
this is positive or negative based upon the way we've defined what a
test is. You know, 25 grams, there's a 75 percent chance of finding if
there's one bacteria there. So we can convert from concentration to
prevalence when we need to. There's nothing in the model that depends
upon or tracks prevalence in and of itself.

UNIDENTIFIED SPEAKER:

All right, then maybe you could help clarify the 75 percent sampling
efficiency of being one cell.

DR. GALLAGHER:

If I \-- let's say I swab a slicer. Can I walk? All right, I'll stand
here. No, it will take too long. If \-- sorry. I teach and I like to
wander around and like talk to the particular student. This is hard. If
we swab an area, and let's say on that swab, one bacteria shows up on
the swab, and we test that, okay? Our model, right now, assumes that we
can change it, that there's a 75 percent chance that you will find that
bacteria and call that swab positive. There's a 25 percent chance that
that \-- there's only one, that it would come back as a negative, okay?
If there's two, it would be a 25 percent chance for the one there. If
there's a 25 percent chance for the second one. So, you know, as the
number of bacteria, the actual number now, not a concentration, in the
swab or in the 25 grams of product, as that gets larger, the chance of
us saying that it's positive becomes greater. Good enough?

UNIDENTIFIED SPEAKER:

Where did the initial number come from?

DR. GALLAGHER:

Seventy-five percent? Well, Eric had had some experience with the ecoli
model. Seventy-five might be a little bit high, but we talked about it.
Expert.

DR. EBEL:

Those findings are published. You can get them. All those calculations
have been worked out in great detail.

DR. GALLAGHER:

Okay. Other questions?

DR. MACZKA:

Before we break, we want Eric to present three or four slides.

DR. EBEL:

Well, hi. I'm Eric Ebel, and I've enjoyed Dan's presentation
tremendously, but dreading the fact that I need to get up here and say
something. And I do want to mention that, as of today, I am the father
of three teenage boys, and I hope you can appreciate that this isn't
good news to me. But I am missing my now 13-year-old son's birthday,
which is too bad. Now, do I just hit enter on this to get going?

DR. GALLAGHER:

Hit the space bar.

DR. EBEL:

Space bar, okay. So we're skipping over these guys to get to here. And
what we want to talk about here is how we handle growth from processing
to retail, and to start with, we'll mention that the FDA model assumes
about a 1.9 log of growth on average between processing and retail. But
we do note this incongruity in the reported prevalences between the FSI
sampling and processing and the NFPA results at retail, which were two
major components in the estimation algorithm used to estimate the retail
concentration distribution in the FDA model. And we'll, hopefully, make
that a little clearer as we go through the rest of this set of slides
here. But I do want to point that out, that FDA model handles this
incongruity because they have uncertainty in the retail concentration
distribution. They actually have 300 different estimates of what that
retail concentration distribution might be. And so some of this
incongruity is just lost in the noise of the uncertainty there. But
because our model is calibrated to a singular estimate, in other words,
we use the average estimate across those 300 distributions, we use an
average to represent the retail concentration distribution that we
wanted to calibrate our model to. We had to deal with this incongruity a
little more overtly than FDA's model had done. So in order to do that,
we've examined this. And this is just a simple analysis really to make a
point. And we can start with the notion that the retail distribution in
log space is basically the sum of the production distribution and the
growth distribution. And if we assume that the \-- that distributions
are all normal, these mathematics work out well. In fact, the FDA model
distribution for retail contamination is a normal distribution. The idea
being that contamination is logged, normally distributed in arithmetic
terms. So we make the assumption that those other two components of that
retail distribution are also normally distributed, and it allows us to
do some things. We can solve for the production distribution parameters
for different assumed growth distributions. And then we want to examine
the implied sample prevalence levels, assuming a positive threshold for
detection of one LM organism in a 25 gram sample. So that's our
approach. And we use this now to examine three different cases of what
growth might be in order to help us out. So, to start with, if we look
at this bottom graph here, this is the retail concentration
distribution. Again, the average from the FDA model. The parameters are
\-- had a mean of minus 8 and a standard deviation of 3-1/2. That will
stay constant in all three of these examples. And the green line we've
put in here on the right shows the threshold of detection at minus 1.4
logs, which is equivalent to 1 in 25 or 0.4 grams, cells per gram. So
that's going to be our target. Now, the growth distribution in this case
that we're doing here, we're assuming that the distribution has a mean
of 1.9 logs, and a standard deviation of 1.4. And the consequence of
that kind of a depiction of growth results in our production
distribution, which is in the upper left-hand corner. And in this case,
at that threshold of detection, we would be finding 4.3 percent of
samples on average positive if we randomly sampled from this
distribution. Well, that didn't make us feel good to know that. And, in
fact, we empirically demonstrated this in the model as we considered
modeling variability and growth, and we were finding very low
prevalences at processing in our sampling in our model. So our next step
was to consider instead a scaler value for a growth multiplier, which
is, in fact, what FDA did for their growth multipliers, was use a 1.9
log increase between production and retail. And in this case, again, we
still have our three percent prevalence at retail, where we're inching
up a little bit here to towards at production getting a prevalence
almost of one percent. But again, most of our FSIS results were
suggesting somewhere between one and 3 percent, so we wanted to do
better. And that leads us to our third case, where again, we use a
scaler multiplier of one log, and in this case, then, as you'll see in
the upper left-hand corner, our gray line is showing about 1-1/2 percent
of those, of samples randomly taken from that distribution would be
found positive, which is reasonably in the middle of the one to 3
percent prevalence that we observe from our sampling and FSIS. So the
consequence of all this discussion was to say instead of using a 1.9 log
multiplier, we use a 1 log multiplier in our model for adjusting for
growth between processing and retail. Now, I'll just quickly go over
this slide to illustrate some of the mass balance considerations. In the
top here we've lot the Listeria species that are added. And, basically,
what we've got here is a series of time or lots. And, essentially, the
contamination is beginning at where the blue bar first starts on the
left there. And we have 22 lots represented in this, in this graphic,
and so that represents, essentially, 11 days of contamination that
occurred in the model. And you can see the contamination concentration
that's added in the top graph. This is the number of CFUs per centimeter
squared of food contact surface area that are added into the model for a
particular lot on each particular lot that we're modeling. What's
interesting then is in the bottom graph we see a pretty good tracking of
those Listeria monocytogenes per gram of RTE product. In other words,
the top and the bottom graph \-- graphs will track fairly well. But what
we see in the middle graphic here is the number of LM species, or the
concentration of AL@ species per centimeters squared after a lot has
been produced. And, in that case, you see some of the blue dots are
missing for some of these lots because, essentially, what's happened is
that all the Listeria has been transferred onto the product and none
remains on the food contact surface area. So this just gives a sense of
how \-- how the model is tracking and keeping track of the bacterial
loads on the food contact surface and the RTE product. And I think, at
that point, we probably can call it a morning.

MS. HULEBAK:

Let's just have any questions.

DR. EBEL:

Okay.

DR. MACZKA:

Any questions to Eric?

UNIDENTIFIED SPEAKER:

Just a quick clarification please was on that last line. In some of
those, some of those areas was \-- did you have Listeria monocytogenes
even though there was no Listeria species on your dots?

DR. EBEL:

Okay, on the middle, in the middle on here you're talking about?

UNIDENTIFIED SPEAKER:

Yeah, I think. I can't really tell on this. Do you have situations where
you have Listeria monocytogenes there even though there is no Listeria
species left on the contact surface?

DR. EBEL:

Well, at the end, that's right. But remember that, essentially, the
fraction of the AL@ species that's outlined is a random draw, but it's
going to be about 50 percent on average of the AL@ species, is going to
be LM. So, in fact, what is being transferred is the AL@ species at the
top graph. That's the amount that's added. And then some fraction of
that is the LM that \-- of the part that is transferred. There's 50
percent of that that's LM.

UNIDENTIFIED SPEAKER:

What's the middle? What's the middle?

DR. EBEL:

This is actually after, at the end of the lot. In other words, after the
product has passed over the food contact surface, how much AL@ species
is remaining. That's why it can often times drop to zero because the AL@
species have been transferred to the product.

UNIDENTIFIED SPEAKER:

Okay. Then it \-- and then the next little mark that's subsequent to
that, is that \-- that's not at a level where the blue dot was because
you're saying there's additional contamination?

DR. EBEL:

Exactly. It's added to that. If there's some remaining from the previous
lot, we have addition added.

DR. MACZKA:

Any other questions before we break for lunch? One more?

UNIDENTIFIED SPEAKER:

Has this data been reconciled with contamination ratio or the transfer
of organisms from the food contact to product? Was this actually
reconciled with that? Was it included?

DR. EBEL:

Yes.

UNIDENTIFIED SPEAKER:

And the assumption of it being one or near one, we should say, in every
case, there should be no Listeria species in there on the food contact
surface because you have 100 percent transfer, but this doesn't show
that?

DR. EBEL:

Right. And, in fact, it isn't 100 percent all the time. Or in arithmetic
space our mean is about 70 percent, and yet we have it's a random draw
every lot, and it can range anywhere from zero to 100 percent. So if
there is no \-- no provision for 100 percent transfer all the time.

UNIDENTIFIED SPEAKER:

Small sided data, we've got millions of data points, it should then
match up with that number?

DR. EBEL:

Yeah. The \-- well, if we were to calculate the percent of AL@ species
transferred from the food contact surface to the product, on average, it
should be bang on with the distribution we put in, which is, again,
about a mean of 70 percent bearing between zero and one. And, certainly,
truncated up there at 100 percent. I don't \-- I don't want to imply
because the distribution isn't truncated normal, it stacks up there at
100 percent, but it's \-- it is a mean of 70 percent.

DR. MACZKA:

Another question here, Eric?

UNIDENTIFIED SPEAKER:

Eric, if I'm hearing this transfer from surface to product is it only
one organism? If you have a mixed population of Listeria, only one of
them may be transferred into the species? The reason I ask, if there's a
problem with the detection methods, and overgrowth of inocula\...

DR. EBEL:

What we are actually modeling is a transfer. The initial contamination
of that is an addition per centimeter square to food contact surface. We
then multiply that times the total food contact surface to calculate the
total number of organisms that exist on that food contact surface, and
then some fraction of that total number transfers to the ready-to-eat
food. At that point we say what fraction of those AL@ species that got
onto that ready-to-eat food were LM. And it's just a proportional
examination. I don't know if that's where you're going with it or if
that's what your question is.

UNIDENTIFIED SPEAKER:

Well, there have been past demonstrations by a lot of studies of the
enrichment process that if you have a Listeria inocula and Listeria
monocytogenes in the enrichment grow up at the same time there is a
distinct possibility that the inocula will outgrow the monocytogenes and
you will wind up with a false negative.

DR. EBEL:

Okay. Well, and to whatever extent that's reflected in our likelihood of
culturing one organism, you know, we need to get some feedback on that,
but that relates then to the likelihood of detection and, you know, we'd
be interested in input there.

DR. MACZKA:

One more question?

UNIDENTIFIED SPEAKER:

This may relate more to the FDA part of the risk assessment, but is the
model the same as contamination events that occurred in the production
process versus those that may occur in the retail?

DR. EBEL:

No, it doesn't. It assumes, essentially, all the contamination is
occurring at the processing level.

DR. MACZKA:

Okay, we're going to break for lunch, but we're going to start up again
at one o'clock.

\*\*\*

\[Lunch recess\]

\*\*\*

DR. EBEL:

All right. While I trust everybody enjoyed their lunchtime break, but
it's time to get back into the discussion on our modeling here, so we're
going to talk next about model evaluation, and we want to address how
well does the model calibrate to the FDA Risk Ranging Model input. Our
output needs to become that model's input. So we want to see how well we
calibrate there. And then we'll want to look a little bit at the
behavior of our public health predictions from our model, and then we'll
look at the stability of the in-plant model. So to start with we'll look
at this question of comparing the FDA model to our in-plant model
predictions. And again, realize that we calibrated to an average
distribution from the FDA model and that point shown here for both of
the \-- both the model outputs match up very well to begin with. Now,
what we're looking at here is the percentile along the AX@ axis there,
and along the AY@ axis we're looking at the concentration of LM per gram
in RTE foods. And the beauty of what we were able to do here is that the
FDA model actually only looks at the 80th percentile on up, and the
exposure assessment part of that model. So that made our job easier
because we didn't have to match the entire distribution. But it made our
job harder because we needed to match just that extreme tail. And you
can see the number of data points that we actually match up on. And
those are all outputs then from our model. Those are all the percentiles
we keep track of. We're looking at concentration at each one of these
percentiles. And again, this graphic shows we do a fairly good job of
matching up by simply changing the incoming concentration of AL@ species
added to the food contact surface area. That's the only input that we
changed to make this calibration work. So we feel like we were pretty
good in terms of matching up in a baseline sense with the FDA
distribution.

Now, as we began considering marrying our in-plant model to the FDA
model, we needed to ask some questions. The first question we asked was
does not considering uncertainty in the retail contamination
distribution make a big difference. So this graphic here, the two bars
to the left are an examination of that question. The first bar is,
basically, the FDA model predictions for elderly, number of deaths in
elderly per year, and the uncertainty about that. And this is the
outcome then of the FDA model's predictions. This is our updated FDA
model. And, as you can see, it extends from below 50 to almost 300
deaths, but as a most likely at the median of about 230 deaths per year
predicted by that model. The next bar over is simply that model modified
by only using a singular, a single distribution, to represent the retail
distribution. And then that single distribution is actually the average
distribution that we used to calibrate our model to. And you can see the
effect of doing that in terms of the final public health predictions is
fairly minor. Very \-- there is a shrinking in the final predicted
uncertainty about the number of deaths per year, but it's not that
substantial. And most of it happens down at the right-hand tail. There
is a slight shifting up at the median, but that is something that we
felt we could live with. The next two bars there then represent two
runs, two replicates, of our baseline model, where we take our
percentile values and plug those directly into the FDA model and run
them through. Those become our baseline predictions for numbers of
illnesses. So the conclusion was that, first of all, uncertainty in that
retail distribution doesn't affect our predictions in the rest of the
model that greatly. Second of all, we felt like our model predictions
were fairly robust. Different model runs from the in-plant model were
giving very nearly exactly the same predictions coming out of the FDA
model for numbers of deaths. And I'm using the term FDA model for just
brevity. We recognize still that that FDA model was an FDA/FSIS Risk
Ranking model. And we're very proud of it. Now, the last \-- I guess
then to make the point again, our predictions were robust and
consistent.

MS. KAUSE:

I just want to clarify a little bit. For the clarification, and when
we're referring to the FDA model, we're referring to the exposure
assessment pathway for deli meats from retail to table as well as the
updated dose response curve, not the entire risk ranking.

DR. EBEL:

Okay, thank you, Janell. One other point then is to make it clear that
we won't be considering uncertainty in subsequent presentation of
results in this presentation, but recognize that there was uncertainty
in every prediction of every scenario that we run. This is just to give
you an idea of the magnitude of that uncertainty in the baseline case.
Getting to the question of stability, we've shown in the previous slide
that two runs of the baseline model gave us essentially the same
predicted uncertainty distribution in numbers of deaths for elderly. We
can also directly look at the stability of different percentiles of our
predictions coming out of the in-plant model. And that's what's done
here. From 20 different runs of a particular scenario, the 4/2/1
sampling scenario, each simulation consisting of a million iterations
each, and you can see that we are fairly stable at the 80th, 99th and
99.99th percentile of our predictions coming out of the in-plant model.
So we felt pretty good about that. If there's any variability in our
results it seems to be pretty much at the further right-hand tail, that
99.99 percentile, where we're seeing just a little bit more variability
from simulation to simulation, but still pretty good. Okay, then we can
talk about model results. I first want to describe the scenarios that we
ran through the model, and then I'll show some of the in-plant model
outputs. Then we want to look at this question of the likelihood of an
RTE product testing positive, given that the food contact surface area
was positive. And then we want to consider the efficiency of food
contact surface testing or a test and hold strategy versus a non test
and hold strategy. So we'll have a little bit of an analysis of that.

These are the scenarios that generally we ran, and we focused primarily
on the elderly as we go through the rest of this model. But we do
predict for intermediate age and neonates as well for some of these
scenarios. But this is a good listing then of the scenarios that we run.
And to start with, to recognize that our baseline model consisted of no
testing, no interventions, no processing control actions or growth
inhibitors. Then our food contact surface testing levels are explained
as these sets of triplets. So a 4/2/1 means four tests per line per
month for in large plants, two tests per line per month in small plants,
and one test per month per line per month in very small plants. So that
triplet organization is consistent from scenario to scenario. And, as
you can see, we run several different levels of frequency of intensity
of testing. In all these runs, in general, test and hold is turned on.
We dispose of any product if it tests positive. We also have enhanced
cleaning turned on so if the food contact surface area tests positive,
that enhanced cleaning efficiency is also in action there. Then the next
scenario there is 60/60/60. Lot testing is an scenario where we don't
test the food contact surface. We strictly test the RTE product. But as
the 60/60/60 should signify to you, we test all lots. We take a sample
from every lot. And, of course, then we dispose of the product. We then
also run a scenario where we assume 100 percent of plants have an
operation processing intervention that is 90 to 95 percent effective.
And then we run another scenario where, again, we assume 100 percent
participation by the industry of a growth inhibitor of some sort that's
90 to 95 percent effective. So this is a set of results. And because
you're going to see this, this kind of presentation of the results
several more times, I think it's important to get clear on what it's
representing. The bars are telling us different percentiles for these
scenarios. So along the AX@ axis there you've got a list of different
scenarios that are being run. And the bars, themselves, then represent
the 80th, the 99th or the 99.99th percentile, and then, of course, you
can read off the AY@ axis, the concentration associated with each of
those percentiles. And the concentration is LM in RTE product, and at
retail. So the first set of results then is a representation of the FDA
model's retail concentration, and then our baseline right next door to
it, to show how they compare. And, generally, they match up pretty well.
The 80th percentile were a little bit lower than the FDA. And then a
general trend we see across the testing scenarios is that as we increase
intensity of testing, or the frequency of testing, we see that at least
at the 80th percentile, generally a decline in that concentration,
implying that we're removing contamination from the retail distribution
and, essentially, shifting that distribution to the left. Now, as we get
out to the 60/60/60 and the 60/60/60 lot \-- okay, I'm sorry. Go ahead.

UNIDENTIFIED SPEAKER:

You said there was a difference in this\...

DR. MACZKA:

Excuse me. We've been asked \-- I'm sorry. We've been asked if you could
speak into the mike for recording purposes.

UNIDENTIFIED SPEAKER:

Is there a statistical difference between the significance? Is it \-- is
there a statistical difference between those? I don't \-- it doesn't
visually then look like there's any differences.

DR. EBEL:

Well, I'll agree that the difference is very subtle, and in many\...

UNIDENTIFIED SPEAKER:

But did you run statistics to see if there was, in fact\...

DR. EBEL:

Well, I guess I'm not sure what \-- what would be the point of
statistics. What we're saying is this is the outcome of the state of the
world. And we've changed the state of the world from simulation to
simulation. So I don't know what the statistic would be that I would
calculate. I will agree that the differences in these distributions is
somewhat subtle and especially at the lower levels. But what we \-- I'm
sorry, go ahead.

MS. HULEBAK:

I guess the point would be can you really call it a trend? Can you
repeat my\...

DR. MACZKA:

Can you call it a trend?

DR. EBEL:

Oh, can you call it a trend? Well, we certainly feel like we can. The
point is that if we take an action to any sort of biologic system and
influence it, and that biologic system represents variability, and we
influence it, it \-- I don't know what else we can do other than say it
does appear to influence it lower or higher. And that's essentially what
we're seeing here, is that we are shifting the distribution, again,
subtly at some of the lower testing frequencies more dramatically as we
get to the more \-- to the higher test frequencies. And it gets us to
this point I was starting with, which is a 60/60/60 scenario, either
where we test food contact surface areas and then follow up with testing
RTE product if the food contact surface tests positive, or strictly
testing the RTE product. Both distributions are essentially the same in
terms of that final result, but neither of them are illustrating that we
are eliminating all contamination with that level of intensity, which is
\-- that's essentially 100 percent testing as we've set up the testing
to be done. In contrast then, as we look at processing interventions,
what we signify here is PP, or growth inhibiting packaging, or other
sorts of growth inhibitor scenarios, or a combination of those two. We
see that we get dramatically lower concentrations. Well, I'll leave
dramatically out. We get a lower concentrations in those scenarios than
we do even at the highest level of intensity of testing. And, you know,
I don't \-- I don't think it's wrong to be somewhat confused by these
things, but this is the in-plant model income. As we propagate this \--
these outputs through the rest of it, out to predicted numbers of deaths
per year, we'll get a better sense and have a little bit stronger number
to look at. And so we don't want to argue that these are real easy to
follow, but that's why we wanted to spend some time on this first one
because we're going to be looking at these some more.

DR. MACZKA:

Question? Question in the back? Can you hold on? And if you can identify
yourself too, we might as well do that.

MS. CHUNG:

Yuong Chung \[ph\] from NFPA. You mentioned the gross inhibition was
that a maximum inhibition of one lot?

DR. EBEL:

Yeah, one to probably 1.3. It was a range of 90 to 95 percent.

MS. CHUNG:

Okay, because you mentioned that the gross in the model is 6 to 1 lot,
so is that your link to the effect of your gross inhibition in the deli
issue?

DR. EBEL:

That's \-- yeah, that's incorporated into the \-- into the model.

MS. CHUNG:

So that means that the maximum inhibition in that deli side was one lot,
right? Or no?

DR. EBEL:

Yeah. Yes, absolutely.

DR. MACZKA:

Do we have another question over there?

UNIDENTIFIED SPEAKER:

It does look, in deed, like the growth inhibition is directly
proportional to what you would expect. Is that true also of
post-processing? I assume \-- it looks like there was a one log
reduction in incidents at the end\...

DR. EBEL:

Yeah.

UNIDENTIFIED SPEAKER:

\...which seems to correspond to your 90 percent reduction in
post-process \-- by post-processing intervention?

DR. EBEL:

If I just push the up arrow\...

UNIDENTIFIED SPEAKER:

Yeah, if you push it up to 99.9 due\...

DR. EBEL:

I wanted to get that part up.

UNIDENTIFIED SPEAKER:

There you go.

DR. EBEL:

I can't change the results. I can change the slide, but I can't\...

UNIDENTIFIED SPEAKER:

No, it would be interesting if you could change the results, but \--
anyway, we just wanted to \-- I just wanted to confirm that the 99 \--
if you went up to 99 or 99.9, that that bar would probably feed straight
through proportionately, right?

DR. EBEL:

Yeah, we'll actually show that in some sensitivity\...

UNIDENTIFIED SPEAKER:

Okay, great. Thanks.

DR. EBEL:

Dan wanted to make a point here that \-- so if Dan wants to make a
point, why don't I let him make it?

DR. GALLAGHER:

Let me just point out \-- let me just point out a difference between
what testing can do versus what post processing can do. If you look at
when we go to maximal testing, either at food contact surface or the
lots, see the big drop in the 99.99 percentiles between those two, okay?
Testing is finding the higher concentrations and preventing them from
going on further in the human \-- or into retail. What post processing
will do is drop all three bars by however many log units of death we've
got implied, okay? So notice here, much lower green bars. The extreme
concentrations. But for the post processing, in particular, when you
combine post processing and some type of inhibition, see what that's
done to the 80th percentile? Now, remember, the FDA model does allow for
growth from retail through to consumption. So any of these
concentrations are allowed to grow back up higher as defined by the
growth factors in the FDA model, all right? So when you get to look at
the actual results in terms of public health, keep these kinds of
differences in mind. Tests that can capture the high concentrations and
stop them. Post processing, okay, will reduce all of the concentrations.

DR. EBEL:

I'm particularly happy that you showed me where the pointer was. I
thought that was a TV changer or something. I don't know. Okay, so now
we're ready to go on and ask the question, basically, what's the
likelihood that RTE product test positive given that a food contact
surface area test positive. And so this is this contingency analysis
that we promised to show you. And, essentially, to get these sets of
results, we turned on both food contact surface testing and RTE product
testing at full bore. So were basically testing all lots using both
methods, and then looking at the overlap. How frequently, when you get a
food contact surface positive, do you also find that the RTE product is
positive as well. So we see that, overall, we're getting about 13.7
percent of our food contact surface area samples as being positive, and
overall, about 2.2 percent of all lots have positive results for RTE
product. Now, that 2.2 percent, you might want to remember or recall
that we presented those sampling results from 2002 that said 2.3
percent, so pretty darn close there. If we look at the fraction of RTE
product positives amongst those that were food contact surface
positives, we get about 15.7 percent. So we would conclude from that
that relative to a random sampling of ready to eat product lots, where
we would find 2.2 percent of the samples positive, we're doing seven
times better by focusing on those that were food contact surface
positive. Seem reasonable?

Okay, this is an analysis we did to evaluate the effect of test and hold
versus not testing and holding. So we tried different levels of
intensity of testing, but it's paired between a scenario where we had
test and hold turned on, the T and H, or test and hold turned off, the
no T and H. And what's interesting here is that we don't see too much
difference between them at the lower frequencies of testing. The 4/2/1
strategy you'd be challenged to find any real difference between them.
But we do see, as the intensity or the frequency of testing increases,
that the test and hold results in a lower concentration than the not
testing and holding, which intuitively makes sense, since in a situation
where not testing and holding were called, that means that we're
sampling an RTE product that was produced three days after we found that
food contact surface positive. So they're not temporally very well
related. And so it's not surprising that we wouldn't find full contact
\-- or I'm sorry, RTE product to be positive as often and, therefore, be
able to remove it. But we were interested in why we couldn't make this
sort of show up at all, frequencies of testing. And so we wanted to go a
little bit deeper into our analysis there. And what we did was run
either a four, four tests per line per month, or a 60 test per line per
month, and just crank through the results. And what we \-- what we found
is that regardless of whether we turned test and hold on or off, we're
going to find about 13 to 14 percent of the food contact surface area
samples positive. And again, when test and hold is turned on, we find
between 15 and 16 percent of those positive food contact surfaces have
positive RTE results. But in contrast, if test and hold is turned off,
we only find 4 to 5 percent of those positive food contact surface area
samples having a positive RTE result. Now, 4 to 5 percent is still
better than just a random sample of lots where we would find about 2.2
percent. But it's not nearly as good as the 15 and 16 percent. Now, the
reason we tend to see less visual evidence of a reduction in
contamination for the low frequency of testing is really just a numbers
game. We're talking at the lower test frequency of only removing about
.04 percent or .14 percent of all the lots due to finding them positive
through testing. In contrast, if we get up to the intense level of
testing at 60, where we're testing all of the lots, we're removing about
2 percent in a case where test and hold is turned on, whereas we're
moving about a half a percent if test and hold is turned off. That makes
for a more dramatic difference in the total number of lots that are
being removed because they're positive for RTE testing results. Okay,
I'm going to have \-- I'm sorry, go ahead.

UNIDENTIFIED SPEAKER:

Why did you not look at scenarios that would be indicative of industry
practice, whereas you would have both growth inhibitors and/or other
interventions as well as aggressive test and hold sample? That
combination wasn't one of the scenarios, was it?

DR. EBEL:

Combination of processing and growth inhibitors?

UNIDENTIFIED SPEAKER:

Right.

DR. EBEL:

Yeah. Well, it's actually one of the scenarios.

UNIDENTIFIED SPEAKER:

Environmental testing. Just environmental testing, test and hold, and
intervention.

DR. EBEL:

In combination, yeah, we do. We will get to that. We present those
results in part of our public health outcomes, but we're not presenting
them here\...

UNIDENTIFIED SPEAKER:

Okay.

DR. EBEL:

\...right now. So we will get to that. I'm going to have Dan come up now
and talk some about the sensitivity analysis.

DR. GALLAGHER:

Sensitivity analysis, in a simple sense, is when we change one of the
input values and run the model again and just see what difference it
makes in the output predictions. We do a sensitivity analysis for a
variety of reasons. One is to simply \-- does the model respond the way
we expect it to, so it becomes a debugging check. Another one is what
kind of trends to we see? One of the nice things about a sensitivity
analysis, I can put in values that I know won't fit the real world, but
I can look at what the model responds to get a better visual assessment
of trends than we could over just a realistic, real world change. The
other, I can see how different types of parameters respond, because the
model will respond differently. So we haven't done a lot of these.
There's still a little bit of a time crunch going on. I want to show, I
think, four different variables that we varied because they all have
something slightly different to say about how the model works. The first
one, okay the baseline model as the standard lab protocol looks at \--
looks at 25 grams of an RTE product sampled for Listeria monocytogenes.
We wanted to know, could we make that any better. What would happen if
we either composited multiple samples or simply took a larger sample?
Now again, I caution you, these results assume that the contamination is
evenly distributed throughout the product. That's one of the factors in
the model right now. What you see here, okay, along the AX@ axis, I'm
varying the size. And there's an example where I don't anyone in the
real world is going to sample almost a kilogram of product, okay? But we
can run it through the model to see what the trend is. What you
consistently see, again, the bars up here are the concentrations at
retail. And consistently, across all three quintiles that we're looking
at, percentiles that we're looking at, the more we sample, the larger
the sample is, okay, the better we can decrease the resulting
concentrations at retail. A brief discussion with some of the lab
people, and conceptually, they could be running 100 gram samples, okay?
Now, it's changing an established protocol. This isn't something you
easily do. But this looks like something that would have a potential
benefit. Like I'm not allowed to talk about cost, but it seems like that
would not have much cost involved because we're not changing the number
of tests. Just defining what a test if. That doesn't seem to have any
level off at any point. So you have to let the lab decide what it can
handle as the largest sample mass. Compare that to looking at the food
contact surface area that is sampled. Okay, the area that is swabbed
from the table or the roller. Okay, again, I'm looking then across here,
and it's a very great range. Much greater than we would ever look at in
practice. But what you see is, early on, I get drops, okay? But this one
levels off, okay? Larger and larger areas don't make much difference.
Okay? I'm not changing the retail concentration at all. Now again, I
want to emphasize the caveat we're assuming uniformly distributed
contamination, okay? Those results would change if there's some
variability in where that contamination is. But somewhere around the 10
to 100 to maybe 1,000, going to a larger area doesn't make any
difference in the retail concentrations. And just to look at that same
analysis a little bit differently, this top graph, okay, is the number
of food contact surfaces out of a million that were found to be positive
as we vary the area that gets swabbed or tested. The larger the area,
the greater the number of food contact surfaces that we find positive.
The red one is the number of corresponding ready-to-eat product lots
that we find positive. And what we see here is, again, the same thing.
Once you get out to about 100 square centimeters for our model, you have
found all 2.2 percent of the ready-to-eat product lots that are
contaminated, okay? This will trigger more testing with larger swabbed
areas, will trigger more testing, but not result in a decrease in the
retail concentrations. So a difference, at least over the ranges that we
looked at, between a sample mass for LM and a swabbed area for food
contact surfaces. The base model actually pointed it out. Our model is
at a thousand. And we do have some references for that. I can look them
up if you'd like.

The other one, to bring up \-- a gentleman had a question a few minutes
ago. What would happen if we looked at different post-processing levels?
Pre and post-packaging intervention levels, okay? This looks at a
variety of them. Now again, there's really two variables that are going
into that. How much of the industry is participating in the post \-- pre
and post-packaging intervention? And then how effective it is. All
right. But the question that came up earlier, could we run one, that is
\-- results in a two-log reduction? And the answer is yes, and you're
right, it's basically proportional, okay? And so that retail
concentration with 100 percent of the industry participating is about
one log below the 100 percent of the industry participating, but where
it's only a one log reduction. But remember, that is a two variable
change, okay? How much did industry participate? How effective it is
when they do participate. And, finally, I want to talk about this one a
little bit because I think we need to keep in mind what that calibration
step does to this type of model. What I've done here, and this very
preliminary. And, you know, I was working on this over the weekend. I
looked at different LM to AL@ specie ratios. Remember, that was one of
the factors we used to convert the concentration of AL@ species in the
product, okay, from Listeria species, okay, as we transferred it off the
food contact surface to Listeria monocytogenes, and then we would then
test for, okay? And our baseline assumed about 50 percent of the
bacteria were AL@ mono. I dropped that down to about 5 percent and rose
\-- raised it to about 95 percent just to get an extreme. And I don't
think the data that we have would support that high a ratio. I should
say that. But one of the things that we've got to do is, if you do
something like this, unlike the earlier examples, here we're changing
what our baseline prediction would be. So if I am going to drop \-- if I
am going to say fewer of the bacteria that get on the food contact
surface are LM, okay, I still have to match the FDA distribution of LM
at retail. That's a fixed, that's measured, that's given, that's our
best estimate of the real world. So if I'm going to do that, that means
these are \-- I've got to add a lot more Listeria species to that food
contact surface, right? These are in log units, so they are amounting to
10 to minus 6. If I drop it down to a 5 percent ratio, I've got to add
ten times more, ten to the minus 6, to get back to matching the FDA
output distribution, all right? So some of these factors, as we change
them, we still have to get back to our real-world measurement. That
means we have to go through a recalibration step. Keep that in mind for
some of the new data that might come in. It's not a simple if this
number drops, I know what's going to happen. Think it through the
calibration process and what that implies. All right. Now, the reason I
say this is preliminary, it takes us about two days to do a nice
calibration and get the nice graph that you saw. These were done in
about 15 minutes. So they're close, but I can get better. That's why I'm
not showing the concentrations, okay? What it does show is that for
lower ratios, okay, as I would expect, I'm adding a lot more bacteria, I
get a much higher food contact surface prevalence. All right, there's
ten times as many bugs there as there were before, okay? At higher
ratios, I get a lower prevalence on the food contact surface. So 18
percent of the food contact surface tests positive when I've got a ratio
of 5 percent, 12 percent, when it's 95 percent. The lot prevalence is
relatively constant. That's because we're matching the FDA distribution.
So that's tying us back to having a relatively constant lot prevalence
no matter what that ratio is. The benefit, that contingent benefit,
before we were saying it was about \-- if we knew that the food contact
surface tested positive, how much more likely were we to find a positive
lot that we would go and test, compared to just random? We said it was a
factor of roughly about 7 for our baseline case. That drops if that LM
ratio is lower, okay? At a 5 percent ratio, it's down to a factor of 5.
If it's a 95 percent case, it rises by a factor of 8-1/2. But I do want
you to take away the fact that some of these changes in the data require
a recalibration to make sure we match that known distribution at retail.

UNIDENTIFIED SPEAKER:

Question?

DR. GALLAGHER:

Yes?

UNIDENTIFIED SPEAKER:

Okay, I think I understand what you're doing, but I was wondering if
you've taken a gut check with this and compared it to the Tompkin paper.
And the first line of data came from Tompkin, correct?

DR. GALLAGHER:

The middle column.

UNIDENTIFIED SPEAKER:

The mean LM Listeria species ratio\...

DR. GALLAGHER:

That\...

UNIDENTIFIED SPEAKER:

\...the 5 to 95 came from the Tompkin?

DR. GALLAGHER:

No. No, no, no.

UNIDENTIFIED SPEAKER:

No?

DR. GALLAGHER:

The Tompkin paper is that .52 on average. The Tompkin paper said about
50 percent of the prevalence. And remember, this was one of our issues
before.

UNIDENTIFIED SPEAKER:

Right.

DR. GALLAGHER:

Fifty percent of the prevalence that I would find LM, okay? But it's
this middle column that's our baseline.

UNIDENTIFIED SPEAKER:

All right. Didn't Tompkin also report a range\...

DR. GALLAGHER:

Yes.

UNIDENTIFIED SPEAKER:

\...of about 5 to 95 percent? Or am I wrong?

DR. GALLAGHER:

In terms of prevalence. Actually, we have some that are 100 percent, but
yes. Okay.

UNIDENTIFIED SPEAKER:

Close to that.

DR. EBEL:

GALLAGHER.

UNIDENTIFIED SPEAKER:

Okay. I guess my question is then did you do a gut check with this
overall FCS prevalence? I would think, and I don't remember
specifically, but I would think that that's in the Tompkin paper. I was
wondering if there was some relationship between your estimates and what
he actually reported, and I don't know the answer, but just curios.

DR. GALLAGHER:

I don't know that he reported that. He might have. I don't recall off
the top of my head. We can go back to look at that.

DR. EBEL:

Yeah, actually, I can't answer that either, other than if we get to a
gut check, we have heard anecdotally that these percentages of food
contact surface areas being positive are in the realm of reasonable, so
although I wouldn't say that the upper or lower bounds shown here are
necessarily, you know, the extremes that one could see, but that are
central tendency on fraction of samples that are food contact surface
samples that are positive is consistent with what some other people are
telling us they're seeing.

DR. GALLAGHER:

Eric\...

DR. MACZKA:

There's another question.

UNIDENTIFIED SPEAKER:

Dr. Gallagher, could you clarify something for me?

DR. GALLAGHER:

Please.

UNIDENTIFIED SPEAKER:

I think \-- I thought I heard earlier that when you went back and
calibrated with the FDA/FSIS Risk Ranking, that you just focused in on a
\-- either the mean or the median concentration, and that you did not
consider the entire distribution of concentrations that were related.
Was that in a different application? Is that why I'm a little bit
confused? Because here you're \-- it seems like that might explain some
of the distribution here. But maybe you can clarify that for me. Thank
you.

DR. GALLAGHER:

Let me try. What FDA has are \-- this is our calibration plot again.
They actually have not a single point at those percentiles. They have a
distribution at each percentile, because they would run 300 iterations
and give me \-- or provide an 80th percentile as a distribution output.
We took the median of those to calibrate to because our model doesn't
have a separate \-- does not distinguish uncertainty and variability, as
the FDA does. So when we match their median estimates, that's what you
see here. Now, what I was trying to point out before is if I change some
of the baseline inputs, like that ratio, if I just drop that down to a 5
percent, okay, that would change my red predictions under the baseline
case. It would not be anywhere close to this. And I want to bring that
back up to match it. So what the variable I am least certain about is
how much gets added during a contamination event. So I then go back and
recalibrate that one to bring it back up to the FDA data.

UNIDENTIFIED SPEAKER:

Okay. All right. Sorry. That's why I guess we shouldn't be too surprised
if these things match up because, obviously, that's what you do,
calibrate models.

DR. GALLAGHER:

Wait, wait, yes. Hold on.

UNIDENTIFIED SPEAKER:

If you don't have any\...

DR. GALLAGHER:

Let me point out\...

UNIDENTIFIED SPEAKER:

I understand the significance\...

DR. GALLAGHER:

\...Keep in mind that normally, when we do calibrations, say as an
environmental model, I'm looking at comparing an average to an average,
and if I'm anywhere close, I see people within a factor of ten, say,
they're doing pretty good. Note what we're saying. If you go out to the
99.99th percentile we're still matching that tail. So I think this is
\-- I think this is better than a good calibration. I think this is a
great calibration.

UNIDENTIFIED SPEAKER:

You said you're matching it to FDA data, or is that data or is that
predicted? Is that from your predictive model? That's from their
predictive model or from actual hard data samples?

DR. EBEL:

No, they're starting distribution. For retail contamination, it is
estimated from the available.

UNIDENTIFIED SPEAKER:

May I?

DR. EBEL:

Sure.

UNIDENTIFIED SPEAKER:

It's not from data, but it\...

DR. EBEL:

Well, it's estimated from the available data, so they based it on
consideration of all the sampling evidence that's \-- that, you know,
they did for all food groups. But in the end, it represents, in our
case, a single distribution. That's our best estimate of what the retail
distribution is.

DR. GALLAGHER:

They have to estimate it because they have different data sources. For
example, the NFDA data is involved in that. The FSIS data is involved in
that. They need a way to integrate those different data sources to come
up with an estimate of final distribution, okay? It is not modeled in
the same sense that our data is modeled in terms of a prediction.

DR. MACZKA:

Did you have another question?

UNIDENTIFIED SPEAKER:

Thank you. Can I just ask a basic question for why your model is
predicated on, as I understand it, a single estimate, whereas the FDA
model is predicated on these range of estimates? I mean it seems like
there was a decision made.

DR. GALLAGHER:

There was. There's a distinction between uncertainty and variability,
all right? You know, standard risk ranking. Ideally, you would like to
consider them both separately. FDA did that. Such models are more data
insensitive just by their vary nature, okay? We've looked at much of the
available data and the in-plant model. There's just not that much data
available. We used everything we could find, but there's just not that
much out there. You can sometimes fill in the gaps with things like
expert elicitation.

DR. MACZKA:

Elicitation.

DR. GALLAGHER:

Thank you. Okay. But the problem is the timeframe that we needed to get
some answers back, all right? So, in this case, we decided to look at a
combined uncertainty variability model. If you want another example,
what's being used in the USDA right now, the ESE model that was done.
You know, several years' worth of effort coming out of the Harvard risk
growth used to a similar approach to what ours. They did not explicitly
separate that, separate out that difference. All right, let me just
summarize the sensitivity, then we're back to Eric. So if we increase
the mass that we use to sample the product, okay, we can decrease the
concentrations of LM at retail. If we increase the food contact surface
area, from very small ones to little bit larger ones, we get a benefit.
But over the range that we seem to be sampling, it's relatively flat,
okay? There's not a driving force to make that area larger. Again, the
caveat of both those, we're assuming uniform mix. The testing gives you
a benefit no matter what the LM to AL@ specie ratio is, okay? I've got a
benefit of a factor of about 5 at fairly low ends, okay, up to a factor
of about 8 at the upper end. But there's still always an observable
benefit. Post processing, with industry, significant industry
participation, leads to a drastic drop at retail. All right, Eric's
going to actually present some of the public health defense.

DR. EBEL:

Okay, so after propagating these in-plant scenarios through the FDA Risk
Ranking Model, we're going to predict the number of deaths attributable
to deli meats per year for each of three different age classes that FDA
model as Bob Buchanan explained really consists of an exposure
assessment part and a dose response part, which the dose response part
is actually three different models for the three different age
categories, and that those response model, which is a very elaborate
model, accounts for other sorts of variabilities. Besides the
variability in dose, it accounts for effects of different or
variabilities in virulence of LM strains. It also accounts for some host
factor variability in terms of susceptibility. Now, just to generate
these results through the FDA model, just to give you a perspective on
the time, it takes about, for us anyway, it's about eight hours to
generate an output from the exposure assessment part of the model. And
then to get a baseline dose response analysis takes another eight hours
of run time for that part of the model. And again, that's then times
three if we're evaluating three different age categories. So that, I
mean, it's a very intense model to propagate these outputs from the
in-plant model through. Now, overall, this is the elderly, the predicted
number of deaths per year for elderly. The baseline number, total number
of elderly deaths predicted is about 307 to 310. So you can see that
deli meats in our baseline case represent 250 deaths of those 310 deaths
per year. And then those are just due to deli meats. And then as we
start to propagate these different testing scenarios through, we see
that we are dropping the number of deaths, not precipitously, but
subtly, and then finally to the point where we get the 60/60/60 testing
that we do is really the maximum benefit we can get out of intense
testing, and we're seeing about a 50 percent decline then at the median
for numbers of deaths. And again, this is all results based on the
medians of those predictions.

Then as we examine some of the processing and growth inhibitor
interventions, we see that we actually, in this case, ran two forms of
the post-processing or the processing interventions. One where it was 90
to 95 percent effective, and the other where it was 99 percent
effective. And you can see that, of the 99 percent effective, processing
intervention gives us a much dramatic reduction in the numbers of
illnesses, where the only other one that matches up with it is where we
do a combination of the processing intervention at 95 percent
effectiveness and the growth inhibiting packaging. So those are the only
two scenarios here where we actually see the numbers of deaths drop
below 100. Any questions on this?

Now, this is the same sort of thing, although we propagated fewer of the
scenarios through the intermediate age category. If you recall, that
actually encompasses everybody who's over 30 days old up to everybody
who's 60 years old. The total number of deaths in this age category was
67, and as you can see, about 55 of those 67 deaths are attributable to
deli meats in the baseline. And again, we see the same sorts of patterns
in terms of reductions in predicted deaths as we increase the intensity
of testing. But again, those processing and growth inhibiting scenarios
give us as least as good a reduction in total numbers of deaths, and the
combination is the best. And then we are looking at the predicted
neonatal deaths, and the estimated total is 16 deaths per year, and 14
of those are attributable to deli meats. And again, the patters are,
essentially, the same.

DR. MACZKA:

We have a question.

DR. EBEL:

I'm sorry.

UNIDENTIFIED SPEAKER:

I'm sorry. Maybe I'm getting a head of myself, but I still don't see the
scenario combining environmental testing and inhibitors. Are you getting
there?

DR. EBEL:

I'm getting there.

UNIDENTIFIED SPEAKER:

Okay. I'm sorry.

DR. EBEL:

And this is just a summary now of everything we've looked at so far,
just kind of taking the difference between the baseline and those
different scenarios, again, at the median, and calculating what would be
the lives saved for various scenarios. And we're going to talk a little
bit about some of the tradeoffs, but we can actually get a sense of it
right here as we look at the effectiveness of growth inhibiting in
combination with processing interventions. We see there isn't
necessarily a synergistic sort of behavior, an additive predictive
effect of those two. There is some trading off that's going on. In this
case, what we've got is processing is reducing the number of bugs that
are actually available to grow in the growth inhibitor part. So we're
not getting exactly an additive effect of those two scenarios being run
independently. When we run them together, we do see some averaging out
of their effects because of that. But they certainly \-- and all those
scenarios that we've run represent the most effective in terms of
reduction in total numbers of deaths. We're going to see the same thing
here. And, finally, we've gotten to the question of test and control
combination scenarios. And, in this case, what we've done is examine
combinations of different testing strategies along with the
interventions of either processing or growth inhibition. And one thing
we learned here then was that, again, processing has some effect in
terms of our ability to find positives. So the \-- we don't see an
additive effect here either between the \-- just the effectiveness of
testing and the effectiveness of a processing intervention. And if we
think about it long enough, we realize that processing, because it's
reducing contamination levels, is reducing our efficiency in being able
to find positive lots. So that's why we don't see an additive sort of
relationship between those two things. But, as you see, as we combine
processing or growth inhibiting with testing, we do substantially reduce
the number of predicted deaths in the elderly sub population per year,
and that our testing of every lot there over on the right, we nearly
reduce the number of deaths to zero when we do a combination of testing
and processing and growth inhibiting. I want to wait for the gentleman
who wanted this to be comfortable with it.

UNIDENTIFIED SPEAKER:

Actually, I wanted to ask the same question again because I want to
understand the point. For your gross inhibitor, here is the inhibition
still 90 to 95 percent?

DR. EBEL:

It's effectively 90 percent.

UNIDENTIFIED SPEAKER:

Yeah. So if there is an inhibition strategy that gives you full lot
reduction instead of you know about two lots, what would you expect as
to the effectiveness in terms of reducing illness?

DR. EBEL:

Yeah. Well, I think we would want to propagate that through the model to
give you the best answer, but naturally, we expect that the more
effective a processing intervention is, the more reduction we're going
to see. But these interactions are something that we feel we're better
able to predict using the model.

UNIDENTIFIED SPEAKER:

And what was the basis for choosing the 90 to 95 percent inhibition?

DR. EBEL:

Inhibition or processing effectiveness?

UNIDENTIFIED SPEAKER:

The growth inhibition.

DR. EBEL:

I think we experts sitting around \-- I call it expert. We sat around
with the folks here in FSIS and talked about what did we think was the
effectiveness. We didn't really have anything to report on that.

DR. GALLAGHER:

Remember, these are, in effect, sensitivity analyses as well. We're not
trying to match the real world in this part of it. We're trying to see
if we did this action, what would the result be. So we picked any range
of numbers for it.

DR. EBEL:

We actually, and I have seen some actual data on effectiveness of
various processing interventions, which suggests many even effectiveness
levels better than the two logs that is the maximum we model here. But
we haven't seen much on growth inhibition in the literature in terms of
its effectiveness.

DR. MACZKA:

Question?

UNIDENTIFIED SPEAKER:

Is your 60/60/60 \-- your 60/60/60 presented here, just for
clarification, is that finished product testing, is it environmental
testing, either/or?

DR. EBEL:

It's actually food contact surface testing.

UNIDENTIFIED SPEAKER:

This happens to be food contact surface testing?

DR. EBEL:

Right.

UNIDENTIFIED SPEAKER:

Thank you.

DR. GALLAGHER:

In some of the figures, if it says 60/60/60 lot, that represents
ready-to-eat product testing.

DR. EBEL:

Well, we actually just have a couple more slides to go here, but these
are some limitations that we want to keep in mind, and we've talked
about them several times, but this model does only consider full contact
surfaces as the source of Listeria species and LM in product. And we are
talking about a generic food contact surface. And we're assuming that
AL@ species is evenly distributed across the food contact surface as
well as the product, and that we are operating on a lot basis. So we're
not looking at within lot variabilities at this point.

DR. MACZKA:

Can we have a \-- I'm sorry, I didn't realize you weren't finished.

DR. EBEL:

Well, actually, we are sort of finished because I think you \-- we all
have seen this already once, but this is our summary of the findings,
what actually\...

DR. MACZKA:

You might run through them again.

DR. EBEL:

Okay. So to start with, we did find that food contact surfaces that were
found positive for AL@ species greatly increased the likelihood of
finding RTE product lots positive. As we mentioned before, it's seven
times better than just a random sample if we use test and hold, and it's
still two times better if we don't use test and hold. We also found the
frequency of contamination of the food contact surface area with
Listeria species appears to encompass a broad timeframe and the duration
lasts about a week, although it's variable. The proposed minimal
frequency of testing, the 4/2/1 strategy, does result in a small
reduction in the levels of LM in deli meats at retail. And that
increased testing and sanitation leads to proportionately lower risks of
listeriosis. Furthermore, then the combinations of interventions appear
to be much more effective than any single intervention alone in reducing
the risk of death and/or illness for Listeria in ready-to-eat products.

DR. MACZKA:

There's a question in the back. Question all the way back there.

DR. GALLAGHER:

Actually, we're done with the talk. Why don't they start coming to the
mikes?

DR. MACZKA:

Okay, as it was pointed out, we're really done, so if you want to come
to the mike in the center of the floor, identify yourself, we can take
about, I guess, about five or ten minutes of questions, and then we'll
turn the podium over to Dr. Hulebak. Will you please come up to the
mike, identify yourself.

MS. HULEBAK:

Take a break.

DR. MACZKA:

Yeah, we'll take a break. Sorry.

UNIDENTIFIED SPEAKER:

Just I have a question about the second bullet. Is it an assumption or
is it a finding? I thought that was one of your basic assumptions from
Tompkin's\...

DR. GALLAGHER:

It's a data input parameter based upon the data we could find published
and available to the IDV. So it's a finding.

UNIDENTIFIED SPEAKER:

Oh, yeah, I thought that there was something in that paper that said
it's 8.8 days average duration of contamination.

DR. GALLAGHER:

That's correct, so that is a finding out of Tompkin's paper. The time
between is a finding out of the IDV data.

UNIDENTIFIED SPEAKER:

You may find out that doesn't hold because that was using previous
methodologies that would take three to five days for the results, or
even up to a week for the results of Listeria testing to appear. At this
point, we can have results in two days. And many plants take action, and
by subsequent sampling, you find that the samples are negative or the
sites are negative. So you may want to consider that, that point.

MS. KAUSE:

If you have additional data, we'd be more than happy if you'd submit it
to the docket.

UNIDENTIFIED SPEAKER:

I wanted to ask about your statements on sanitation. Am I \-- during the
day, all I heard you say about sanitation, the data you used was
conversations with in-house people, so you're basing those two
statements not on any type of studies, but just on conversation. Is that
\-- would I be correct in\...

DR. GALLAGHER:

The level of sanitation that we picked to use for the baseline was based
on the discussion with the FSIS personnel. The food contact surface
testing, you've seen plenty of those results, but in all cases, the
testing \-- here again, testing has to have an intervention to have an
effect. Part of the \-- one of the interventions is increase sanitation.

UNIDENTIFIED SPEAKER:

You've made a statement, increased frequency of, let's leave the testing
part off, sanitation leads to proportionately lower risk of listeriosis.

DR. GALLAGHER:

I don't want to leave the testing off.

UNIDENTIFIED SPEAKER:

Well, then, okay, but the slash needs to be removed then. You mean
testing and?

MS. KAUSE:

Sanitation, right.

DR. GALLAGHER:

It can be written that way if you want.

MS. KAUSE:

Another question here?

MR. STEWART:

Skip Stewart, Marketing Institute. On your last slide, you referenced a
lot. You were talking about lots, I think. Could you define, help
clarify what is considered a lot, for example, in relation to your
various testing scenarios? Is that the lot defined over the course of a
month then?

DR. GALLAGHER:

A lot is the volume produced by one line during one shift, so those
masses that we saw earlier from the FSIS survey data, I forget the exact
numbers, but that many thousands of pounds constitute one lot, and we
assume that a plant, every size plant, will produce two lots a day.

MR. STEWART:

Okay, that's helped a little, but then how does the sampling and testing
scenarios relate to production lots, or do they not relate?

DR. GALLAGHER:

They are \-- well, yeah, so the \-- say the 4/2/1 strategy, that means
that we're going to sample four lots in a given month, and we've done it
on a systematic basis, so we're, essentially, testing four evenly spaced
lots in kind in a given month for a given line. And again, I'll \-- a
lot being one shift of production. It would be only 1/2 of one day that
we would have, if it happened to be a sampling day, it would be only one
of the two lots that we would have sampled.

UNIDENTIFIED SPEAKER:

Okay, so I guess this is a follow-up clarification. Does that then imply
that\...

MS. KAUSE:

Come to the mike.

UNIDENTIFIED SPEAKER:

\...does that imply \-- I mean, what I'm trying to understand, based on
the risk assessment that you presented, was \-- it sounds like then you
can't really draw a relation between what you presented in terms of
sampling and testing scenarios to the lot because the lot is only, for
example, in a 60 \-- in a 60 sampling and testing lot scenario, in that
case you'd have one sample for one lot. In a 4/2/1, to your point, you'd
only have one \-- one test for what, every 15? So there is \-- you can't
\-- you can't really use that to define the lot. I mean does it really
reflect back on the lot or it does it \-- I'm trying to understand how
the sampling and testing scheme that appears to encompass a month
relates to a lot of production that has interventions, you know, a
cleanup, or whatever, at the end of the day. All right, what's the
relationship there, I guess?

DR. GALLAGHER:

Our time step, the way the model \-- the model will calculate the
concentration of AL@ mono or AL@ species for a given lot, okay? And then
it moves on to the next lot. So our delta time or the delta step within
the model is as each lot gets passed through it. So then that's why we
have a maximum of 60 tests that are possible because 60 lots are
produced per line. So \-- and we can assume that we will test any given
number of lots that are requested.

MS. KAUSE:

I believe you're next.

UNIDENTIFIED SPEAKER:

I'm trying to understand the relationship where you made your basic
assumption that there was a pretty constant level of organisms available
to contaminate the surface. Was that one of your first assumptions? Am I
correct that there was a\...

DR. GALLAGHER:

Well, we don't necessarily say the concentration in that reservoir is
constant. We don't know what it is. As long as it's enough to provide
the re-inoculation during a contamination event.

UNIDENTIFIED SPEAKER:

I'm trying to relate it to what I really feel is going on in the
industry today, where we do see a dramatic reduction of growth niches
within a processing plant. I think as we all continue to learn from the
testing by taking action and actually eliminating growth niches, that
available contaminant is continuing to be less and less with time. And I
don't see how your model is recognizing the fact that that is really
going on within the industry.

MS. KAUSE:

Again, do you have data that you could submit to the docket? Because,
basically, we work with the available data. And it sounds like you're
discussing that you do have some data that could be made available.

UNIDENTIFIED SPEAKER:

I think we're speaking from the experience of what we're doing with
workshops and working with various \-- various producer companies, as
we've been doing this now for a couple years, and the thing that we're
seeing the companies do and the stories that they're telling are just
quite dramatic. The fact that suddenly they learn how to eliminate these
growth niches. It's amazing that some of their problems just don't seem
to be coming back over and over again, so I think you're looking at a
very dynamic population here, and I caution you against taking a slice
of time at this point.

MS. KAUSE:

Thank you. Next?

MR. GILLIS:

Kevin Gillis, Rode \[ph\] Inc. I just wanted to comment on your one,
two, three, your fifth bullet point. Just from the standpoint of the
summaries, because in a busy world, these tend to be the things people
read, and maybe \-- maybe 500 pages a day, et cetera, will not be. Just
as a point of a suggestion, in the final bullet point, because we talk
about a combination of interventions, in fact, you don't have 50 or a
hundred interventions. We can just say, if what you're saying is you
need, or you would, from the data, it would suggest that a kill step of
some sort, bactericidal effect of some sort, plus the inventory
packaging would be quite a bit clearer, I would suggest, and maybe
easier to understand that final bullet.

MR. HUFFMAN:

Randy Huffman, American Meat Institute. Back to the question, I think
maybe the first one that was asked I believe by Jenny regarding the
transfer coefficient. The data that was used was the paper by, I can't
remember.

DR. MACZKA:

Carpentier?

MR. HUFFMAN:

Carpentier, yeah. I read that this morning. And that, that was fresh
beef, I think, 5 centimeters square pieces of fresh beef, and they're
allowed to sit on a surface at on a film approximately 30 seconds, and
that transfers, the one that's used, but I'm aware of data that was
submitted to the Agency that I think would probably be \-- well, first
of all, it's using ready-to-eat products. That's a significant
difference. But I'd just like for some discussion and comment on why
that data was not used in the\...

DR. GALLAGHER:

We looked at that data fairly carefully. The problem was it was it was
presence/absence data. What was done was, and help me out if I misstate
anything here, but a food contact surface was inoculated with LM, okay?
Different ready-to-eat product was passed over it, and within a lot, so
that was one question, but then that product was then tested,
presence/absence for Listeria monocytogenes. It's difficult to impute a
transfer coefficient based solely on preferences \-- prevalences, when I
can have actual concentrations, which is what the Carpentier paper did.
Now, I could tie it to actual numerical numbers of bacteria transferred.
I can't do that just knowing that, yes, some bacteria, number of
bacteria transferred. I don't know how many. So we did look at that
data, and it was a nice project, but it just did not help out with that
particular number that we were trying to get at right there.

DR. MACZKA:

We're going to have to break at this point, but again, there's another
time in the agenda when we can answer questions. We're going to set up
for the panel discussion now, so you'll have an opportunity\...

\*\*\*

\[Recess\]

\*\*\*

MS. HULEBAK:

The afternoon session is \-- allows us for \-- allows discussion of the
risk assessment by three panelists. They are Jenny Scott, Senior
Director of Food Safety Program at National Food Processors Association,
Charlotte Christin, Senior Food Safety Attorney at the Center for
Science in the Public Interest, and Sophia Kathariou, who is an
Associate Professor, Food Science, at North Carolina State University.
Dr. Kathariou is not here because she's at home in bed with a wicked
case of the flu. But she's present in virtual form. So she sent along
three multiple questions, and I will read them, and at that point, I
will be Dr. Kathariou. So let me first begin by introducing our
panelists. I'll begin with Jenny Scott. I mentioned she's Senior
Director for Food Safety Programs at NFPA here in Washington, D.C. At
NFPA, she's responsible for providing guidance and expertise on issues
and policies related to microbial food safety enhancement as well as
technical assistance in crisis management for NFPA member problems. She
has an A.B. Degree in Biology from Wellsley, an M.S. in Bacteriology
from the University of Wisconsin, and a Master's of Science in Food
Science from the University of Maryland. Ms. Scott has recently been
appointed to the National Advisory Committee on Microbiological Criteria
for Foods. Charlotte Christin is a Senior Food Safety Attorney with the
Center for Science in the Public Interest, where she is responsible for
CSPI's legislative agenda. She has an L.L.M. from George Washington
University, and is licensed to practice law in California and Virginia.
She's active in the Conference for Food Protection and the Food and Drug
Law Institutes Austern \[ph\] Writing Competition.

Sophia Kathariou, as I said, is an Associate Professor for Food Science
at North Carolina State University, and participates in, is a member in
the National Alliance for Food Safety. We have about 45 minutes for this
panel discussion, and the way it will go is that each one of the
panelists, with the exception of Dr. Kathariou, will make an opening,
brief opening statement and comments about what they have heard today.
As I said, I will, at certain points in the course of the next 45
minutes, raise each one of Dr. Kathariou's points. So we can begin with
\-- we'll begin with Charlotte.

MS. CHRISTIN:

Thank you. As I was telling a few people earlier, as the discussion went
along today, I was crossing out paragraph after paragraph of my speech
because my questions were answered, which is a compliment to the
gentleman who explained the risk model to us. And let me also take a
moment to commend the Agency for the hard work that was put into this
document. I can't even begin to imagine the number of hours and the
amount of effort that went into it, and although I have comments and
questions regarding the model, that doesn't in any way reflect on any
perception that there wasn't an effort, a good effort that went into it
and an excellent product that resulted.

Again, my name is Charlotte Christin. I'm a Senior Food Safety Attorney
at the Center for Science in the Public Interest. CSPI is a non-profit
advocacy and education organization focused on food safety, nutrition
and alcohol issues, and supported principally by 800,000 subscribers to
its Nutrition Action Health Letter. Four years ago, Listeria
monocytogenes tainted deli meat from Sarah Lee's Bilmar plant sickened
100 people, killing \-- 100 people, killing 5 adults, and causing 6
miscarriages or stillbirths. In response to the Sarah Lee outbreak, the
Food Safety and Inspection Service vowed to develop aggressive
strategies to cut the rate of listeriosis illnesses and deaths from
ready-to-eat meat and poultry products in half by 2005. Yet just a few
months ago, we were in the midst of two large recalls as another large
listeriosis outbreak from contaminated deli meat was linked to at least
53 illnesses, including 8 deaths and 3 miscarriages or stillbirths. The
recent outbreak and recalls serve as a harsh reminder as we meet today
to discuss this second Draft Risk Assessment on ready-to-eat foods. That
delays in risk management decision making can be devastating to
consumers, as well as to industry and government. In deed, the National
Advisory Committee on Microbiological Criteria for Foods has admonished
the consideration of risk may not necessitate in all situations, an
in-depth, quantitative risk assessment, which requires extensive
resources and time, particularly if it would delay timely protection of
public health. This pathogen's high fatality and hospitalization rates,
its ability to grow under refrigeration, and the lack of information on
infectious dose all demand a prompt public health response. Therefore,
CSPI strongly urges FSIS not to allow today's discussion of the new risk
assessment model to deter or delay the promulgation of the final
Listeria testing regulations. Earlier today, Dr. Engeljohn explained why
non-contact food surface sampling was not addressed in the management
questions. But the failure to include non-food contact sampling as part
of a comprehensive environmental sampling plan limits the model's output
on the effectiveness of sampling. Though the proposed rule would not
mandate non-food contact sampling, the model considered other
interventions, such as the use of growth inhibitors, even though they
were not being made mandatory as part of the Agency's proposal. The need
for and role of non-food contact sampling for Listeria species in
ready-to-eat meat and poultry plants is well established. Therefore,
CSPI urges FSIS to revise its management questions to direct the
effectiveness of an environmental sampling program, including both food
contact and non-food contact sampling. In constructing the in-plant
model, FSIS has limited the model's ability to provide accurate answers
to the management questions. FSIS's decision to nearly halve the growth
rate between processing and retail raises several concerns. First, there
is a significant lack of transparency in this decision making. FSIS
changed the growth rate based on data that were not publicly available.
Nor did the Agency adequately explain its decision to depart from the
studies it used to establish the growth rate in the earlier, relative
risk ranking. This lack of transparency contravenes Codex's general
principles of microbiological risk assessments. Codex further instructs
that data are to be used to reduce uncertainty and increase the
reliability of the risk estimate. But the use of the NFPA data has only
increased the uncertainty associated with FSIS's in-plant model. For
example, what do the raw data show regarding the prevalence and
concentration of Listeria monocytogenes in manufacturer-packaged,
ready-to-eat deli meats. These data are critical in that the in-plant
model is designed to evaluate the risks from contamination occurring at
federally inspected processors. FSIS's decision to cut the growth rate
has a profound impact on the model's output and the estimated illness
reduction rates derived there from. CSPI urges FSIS to revisit its
decision to adjust the Listeria monocytogenes growth rate during
distribution, and if additional market basket sampling of ready-to-eat
deli meats is needed to update the model, the Agency should discuss
those data needs in the risk assessment report. The risk assessment also
provides important new information that argues for FSIS to increase the
mandatory Listeria testing that plants would be required to perform. The
risk assessment's modeling determined that the proposed 4/2/1 scenario
would allow a small reduction in the levels of product contamination at
retail, but an increased frequency of food contact surface testing and
sanitation leads to a proportionately lower risk of listeriosis. This
finding offers a scientific basis for FSIS to strengthen the mandatory
industry testing program in the final rule. CSPI has recommended that
FSIS require all establishments, regardless of size, to increase the
frequency of food contact surface testing beyond the 4/2/1 scenario. We
urge the Agency to use the risk model to help determine the appropriate
amount and frequency of testing needed to meet the Department's goal of
halving the rate of listeriosis illnesses and deaths by 2005. In
conclusion, this summer's outbreak and recalls made clear that Listeria
monocytogenes contamination of ready-to-eat deli meats remains a
significant public health threat. The comments that FSIS receives on
this risk assessment will be useful in future iterations of the model,
but should not prevent the Agency from finalizing its Listeria
monocytogenes regulations with increased industry testing requirements.
CSPI urges FSIS to take that step without further delay before yet
another listeriosis outbreak claims more lives. Thank you.

MS. HULEBAK:

Thank you, Ms. Christin. Let's turn to Jenny Scott now.

MS. SCOTT:

I don't really have a formal presentation here. What I did yesterday was
go through this draft model that we were presented, and jotted down some
of the areas where the industry has some concerns. And I want to talk a
little bit about that, primarily, and also talk about some of the things
that we heard today that have raised additional concerns. But before I
start, I do want to say that I think that the risk assessors have done a
phenomenal job. They really didn't have a lot to work from. They had a
very complex situation to model. And I really think they've done an
exceptional job in doing that. And kudos to you all.

Keep in mind that what was put out there, and is titled the Draft Risk
Assessment, is really just the in-plant model. It is not a risk
assessment per se. It links to the FDA/FSIS risk ranking, and together,
they form a more complete risk assessment. We have not had the luxury of
reviewing all that and looking at it in detail and examining it. We've
just had a taste of it here today. But what I've seen, I'm definitely
impressed. That's not to say that there aren't some issues that I think
need to be addressed before this model actually gets used for policy
making. I will point out that Dr. Murano said this morning that accuracy
is essential. And that's one of the reasons why I think we need to make
some revisions to the model and run some additional scenarios. A lot of
the things that I think are a problem luckily are things that we can
just input a change in the model and do another run. It's not like you
have to change the whole base model or anything. I'll also remind you
that Bob Buchanan said this morning that Codex, the National Advisory
Committee on Microbiological Criteria, FSIS and the International
Commission on Microbiological Specifications for Foods, have all
indicated that risk assessments need to be transparent, need to have
scientific and stakeholder input, and they need to be peer reviewed.
There's been an attempt here, and I think a very good one, to put some
transparency into what has been done. I think we'll probably have more
questions and need to work with the risk assessors to gain a full
understanding of it. I think that there has been limited opportunity for
stakeholder input here, and obviously, it has not undergone peer review.
And I think that that is essential to do so.

Looking at some of the problem areas with respect to this, I have
highlighted three and I've got a three-page document of these. First of
all, I think that the assumption that Listeria species are distributed
evenly across a food contact surface, and evenly distributed within
product is clearly inaccurate, an inaccurate assumption. And I think the
modelers recognize that. They just used something that they could work
with and went forward from there. I think it would be possible, given
more time, to model different approaches here. And it will be very
complex. But I do urge the Agency not to stop at this point. Another
one. The model assumes unrealistic percentages for sanitation efficiency
and for pre and post-packaging interventions. Very clearly, the 75
percent efficiency of sanitation is just not realistic. If that was all
we could get out of our sanitation programs, we would have spoilage out
the whazoo \[sic\]. And we don't see that. And I think there probably
are studies out there in the literature that we can use that would give
some better indication of what realistic numbers might be. And the
industry experience is that it probably is in the neighborhood of 99 to
99.9 percent efficiency in most plants that have a well-designed
sanitation program. And I think that the sanitation and chemical
suppliers out there are really pushing the envelope in getting new
programs in place to deal with Listeria. I think that the intervention
efficiency of 90 to 95 percent is also inaccurate, particularly if we're
looking at a post-packaging heat treatment of product. Those processes
are designed to inactivate the levels of Listeria that are likely to be
there from recontamination of product. And while we can't really say,
necessarily, that they're 100 percent, because given the statistics and
everything, you'd never get to 100 percent. They really are 99.9
percent, and possibly even higher, depending on the intervention. If we
kill the organism through heat, through irradiation, or whatever
intervention we use, and it's not there, there's not going to be a risk
to the consumer. And I think we have to consider that. A third thing
that I think is problematic here. The model starts out by assuming that
Listeria monocytogenes contamination comes from a reservoir. What we in
the industry refer to as a niche or a harborage. That really is not the
norm out there. It does present a scenario that is the highest risk for
listeriosis if the strain is virulent, and so we recognize it as the
worst-case situation and it needs to be modeled. But I think as we look
at the overall impact of what we're learning from, what this model can
give us to its iterations, we have to consider that most of the time,
when we find Listeria species in the environment and on food contact
surfaces, it is a transient event. These are sporadic positives. We do
our sanitation. They are gone. And so there's a whole big part of the
equation that is not considered here. The model may be fairly accurate
in reflecting what happens when a plant has a harborage site, and we do
have these contamination events that we have to find the niche before we
can fix the problem. So I really think that the key to Listeria control
doesn't come down to necessarily more testing of product, more testing
of food contact surfaces, but to having an overall program in place that
aggressively looks for Listeria in the environment and looks for the
establishment in niches, so we don't have this kind of transfer that is
an ongoing problem. Another issue with the model I see is that the model
does assume that all of the retail level contamination originates in the
plant, and I don't think that that's the case. So I think we could
probably come up with some statistics based on our data from
manufacture-packaged and retail-packaged product to make some
assumptions there about that, the proportion of that contamination that
may arise back at the plant. And I also think that there's a real
problem with this .75 efficiency in finding one colony forming unit.
There are statistical tables out there by I.C.M.S.F., and I think if we
look at a lot that has 2 percent contamination, which is a fairly low
level. Not as low as I expect in plants that have good Listeria control
programs. If you only tested three samples, you have a 94 percent chance
of missing a positive lot, even at the 2 percent level. If you're to
test 60 samples, you still have a 30 percent chance of missing that. And
I think that those types of scientific sampling tables need to be taken
into consideration with this. It's probably going to have the opposite
effect from, well, what we would be looking at here, but in terms of
Listeria control. But I do think we have to recognize that the product
testing really is not the way to go. And I think that you've shown that
you increase your chances of finding a product if you have started from
food contact surface positive. But I still think you're not going to
find it with the frequency you think you'll find in there. I won't go
through most of these other points. We will submit written comments on
this. I think industry's concern

here is that if we use this model with inaccurate or misleading
assumptions, that we're going to derive some erroneous conclusions, and
use those to develop policy. And I think, for example, the finding that
a food contact surface positive for Listeria species really increases
the likelihood of finding ready-to-eat product lots that are positive
for LM, it's not really consistent with industry experience. It may be
somewhat true when you have this harborage situation, which I think is
what this model really deals with, but it's probably not consistent with
what we're finding day to day in the plant. I think one thing that's a
real interest in this, a lot of people have criticized the Agency for
not moving forward with their proposed rules as 4/2/1 testing. And, if
nothing else, this model shows us that going ahead with that rule really
would have had very little impact on public health. So I applaud the
Agency for going ahead and doing this risk assessment and trying to get
more science behind what they do propose, and I urge the Agency to make
revisions in the model, to do a peer review of the model, get the best
science in the model you can, and then make policy decisions from that.
Thank you.

MS. HULEBAK:

Thank you, Jenny. If I could have my mike live, thank you, I will now
become Sophia and read to you all her three points, which are compound.
My main comment is that the actual data on which several of the key
assumptions were based were quite limited. Prevalence data for the
bacteria on food contact surfaces are especially limited and derived
from basically only one source. Tompkin, 2002. Furthermore, food contact
surface prevalence data alone are not sufficient for an adequate
evaluation of the potential public health issue that may be involved.
But I'm going to interrupt and just comment as Karen here. And I'm going
to read these statements from Sophia and suggest that, perhaps, to get
conversation going among the panel, as you listen to me, panelists, you
might be thinking about what you might say in response to Sophia's
observations, okay? Okay, to continue, specifically, without strain
typing, it is impossible to attempt an adequate correlation of
prevalence at a specific food contact surface, and contamination in
ready-to-eat meat and poultry products from the same plant.
Contamination of food contact surfaces may be of transient or persistent
nature, and these two fundamentally different types cannot be
differentiated by prevalence information alone. Another observation I
have concerns the need for caution when it comes to the wish to produce
a generic model that would be applicable to all operations producing
ready-to-eat meat and poultry at any given time in their operation. The
epidemiology of food-borne listeriosis indicates that ready-to-eat meat
and poultry implicated in outbreaks are not randomly selected for
multiple processing plants. Instead, they tend to involve periodic
clustered availability to the consumers of multiple contaminated
products, commonly originating from a single plant. The available
information suggests that the implicated plants might have experienced
periodic probations in the operating procedures associated with the
contamination events, becoming reflected in contaminated ready-to-eat
product. USDA, FSIS and CDC data may be able to reveal the extent to
which such period probations may or may not be adequately predictable
based on the results from the periodic inspection of a facility, the
frequency of CCPs that the inspection results may suggest need further
attention, and the frequency of Listeria positive product samples
identified in the period three to six months prior to the implication
and the outbreak. It would seem a concerted focus by frequent testing of
food contact surfaces and product on selected plants would contribute
much more to an eventual decrease in listeriosis cases than equal focus
to all operations, with frequency of sampling being determined only by
size of the plant. I would like to ask the experts on the model's
construction and development whether the model could incorporate
attributes such as the individual track record of a plant to identify
the frequency of testing that would be sufficient for adequate detection
of contamination. A final note on contamination event and reservoirs.
The term contamination event may be confusing to many, especially
because other sources besides food contact surface transfer of bacteria
could contribute to product contamination, e.g. air and workers. Please
also note that footnote number 17 on page 11 does not involve transfer
of Listeria to food contact surfaces, and thus defines contamination
event fundamentally different from the definition on page 8. The
potential contribution of a reservoir to contamination of food contact
surfaces needs to be more adequately discussed. Can floor drains
contribute significantly to food contact surface contamination, or is
the microbial community in them reflective of selected strain types that
preferentially colonize floor drains? Floor drains may harbor transient
populations coming into the plant from raw product, workers, et cetera.
Strains found in floor drains may or may not represent reservoirs
relevant to food contact services and ready-to-eat product
contamination, and their relevance cannot be adequately evaluated
without additional data utilizing strain sub-typing of floor drain
isolates obtained from different plants. And then, finally, what is
meant by harbored sites, and could some known examples be included?

Comments? That was a lot to take in. It's a very good analysis, and
actually, I think Sophia has raised some points that I had on my list
that hadn't covered, and a couple things that I had covered.

I think with respect to strain typing and the correlation of prevalence
with positive, I think she's absolutely right, that I didn't bring the
data up here. I have some data with me. And it's from seafood plants,
not meat and poultry plants. But it's very clear, from the work we've
been doing in seafood plants, that we can find a lot of Listeria
species. We can find Listeria monocytogenes. And it's only when we do
the typing that we're able to figure out exactly where the source of
contamination is. And we can see a lot of Listeria species on food
contact surfaces that we never see Listeria monocytogenes show up in the
finished product, or on food contact surfaces. So I think that that's
something that needs to be addressed in this risk assessment.

I too had concerns about a generic model applicable to all ready-to-eat
operations. We've taken the worst case product, I think, in ready-to-eat
deli meats, because they aren't further cooked, and because, in many of
them, the organism can grow. On the other hand, many of these are
products which will be able to add some inhibitors. I do think we have
to recognize that we can't add inhibitors to everything. We can't
totally eliminate the risk from all these products unless maybe we want
to radiate everything. I don't feel that, you know, we want to do that.
And we've never taken an approach where we've taken the \-- an
intervention that can absolutely guarantee safety. We could can
everything and it would be safe too, but I don't think it's something
that consumer preferences are going to agree to. One thing that Sophia
did mention here, I think we need to set the record straight with
respect to the role of air in causing contamination. It's something that
we hear a lot about, and I think Bruce Tompkin very clearly indicated,
in his article, that with respect to room air, it's not something that
we see as a source of contamination. So I don't think we would need to
take that into account with respect to modeling. But I think the model
should take into account environmental contamination of food contact
surfaces, as Charlotte had indicated earlier.

MS. CHRISTIN:

Thanks. I, too, appreciate Sophia's comments, and I agree with her on
the issue related to the Tompkin data for determination of the duration
of a contamination event. Certainly, Bruce has presented some really
important information. However, it's important to bear in mind that
those data came from ConAgra plants, a company that is renowned in its
aggressiveness to pursue Listeria. A company that treats all Listeria
species as Listeria monocytogenes. So I don't think that we can use
those data without adjustment to assume that that would be the duration
of a contamination event in the majority of plants. I also wondered, in
looking at the risk assessment, why the Cornell data that were presented
at the meeting on the proposed rule were not used, because one of the
things that Cornell presented related to issues of persistence in the
environment. And I think that, actually, Bruce had that in his \-- in
his article as well. But, I mean, certainly, there are instances where
we've seen a harborage for years. So I think that while again, that,
too, may not be the average plant, I think that to use ConAgra data as
the barometer for the duration of a contamination event in the average
plant is a fallacy, and I think that that's something that the Agency
needs to revisit. And on the point of a worst case model, I do
understand the point you're making. However, we need to bear in mind
that FSIS is a public health agency. Its mission is to protect consumers
from meat and poultry products that are not wholesome, that are
adulterated. So in determining the approach in this model and the policy
decisions that flow from this model, you need to bear in mind that this
is, again, intended to protect consumers from unsafe product. And the
importance is to err on the side of caution. Yes, Jenny, we would have a
problem if you all started to radiate everything, particularly if you
didn't label it. But thank you for giving us the option. And, Jenny, I
understand your point about the role of error in the contaminated
product. However, we have seen, with outbreaks, that there can be
condensation that drips directly onto product, directly onto the food
contact surfaces. I understand there were limitations in the model and
that you couldn't address. However, again, when we think of creating a
risk model that is sufficiently protective of public health, we need to
bear in mind that we have, unfortunately, seen cases that relate to
direct contamination from the environment.

MS SCOTT:

I want to comment on this issue of the Tompkin data with relation to
duration of an event. I have a little bit of a concern about only using
the Tompkin data, not because it's the Tompkin data coming from ConAgra
plants that are well managed, but because I believe that additional data
were submitted to the Agency that weren't used here. And they actually
showed that the \-- there are much lower levels there. For example, the
Tompkin data, I think, suggested that 4.9 percent of the time you would
find a positive three times when you went in and did your sampling and
did an intervention, went back looking for it. But I think some more
current data have indicated that in some plants that could be as low as
maybe .12 percent. So I think that there are other data out there that
need to be considered in here. Also, while Sophia didn't mention it
particularly, there was only one data set used to look at the frequency
of a contamination event. That was based on an IDV. Clearly, a plant
that was having some problems. And one thing I noted about that is they
used the data from the IDV with respect to frequency, but they noted
that the data from this plant related to duration did not agree with
other data, which I'm assuming was the Tompkin data and, therefore, they
didn't use that in the duration. They went to the Tompkin data, which
begs the question of, well, how much confidence can we have in the
frequency data as well? If the duration data didn't mesh actual
experience, do the frequency data also? Any other comments?

MS. CHRISTIN:

I also wanted to ask Jenny, will the NFPA data, the raw data that would
answer the question I raised as far as what percentage of the product
sampled was sliced in the plant versus what was sliced at retail? Will
those data be made available?

MS. SCOTT:

Actually, the galley proofs of our survey on ready-to-eat were submitted
to the docket yesterday.

MS. CHRISTIN:

Great.

MS. SCOTT:

Keep in mind that with the deli meat, we started that project with USDA
funding, CSREES funding, and that project did not initially involve
looking at that parameter. And then when we got funding to look at other
products, and we were collecting that, we went back and decided to
collect those data for luncheon meats. So we don't have a complete set
for the luncheon meats. It's also, I think you have to look at the
limitations too. It's subjective. The laboratory got the samples, and
they made an evaluation of whether it was prepackaged, or whether it was
packaged in the deli. And you can do that to a certain extent based on,
you know, you've got a sealed package, it was manufactured. Or if it
kind of came wrapped in butcher paper, then you say, well this is deli
product. And so those are what we have.

MS. CHRISTIN:

Well, having \-- I mean I appreciate that, but the thing is, I think
that this would be very useful information, the raw data, for the Agency
to have, because I mean as was repeatedly made clear today, we're really
trying to assess what's going on in the plants. And, unfortunately, so
often, the Agency had to work backward from the FDA model. So if the raw
data were available, I think that would be useful information to inform
and really increase the circuitry of the risk assessment.

MS. SCOTT:

And the raw data are also being submitted to the Gifsan \[ph\] Risk
Analysis Clearing House, and will be available on the internet, probably
within a month or so.

MS. CHRISTIN:

Great.

MS. SCOTT:

So anyone will be able to take the data and do their own analysis. I
found this model very interesting. I'm very glad you made it as user
friendly as you did. It looks like if I were to get this model on my
computer, that I could just go in and play with some numbers and play
what if. And I think that that is great. I think it would be very useful
for the risk assessors to sit down with some folks from industry and
let's play what ifs together, and model some real scenarios for what we
know is going on in the plants today, and look at the impact, and make
some changes in those assumptions with respect to the efficiency or
efficacy of sanitation and post-packaging treatments and the like. One
of the concerns I have is the findings the way they're presented now
might suggest that you can just increase product testing dramatically,
doing a test of every lot, and have an efficiency of finding Listeria,
getting it out of the marketplace, that is \-- is much better than doing
the testing food contact surfaces, taking interventions when you find
positives. And I think that that is very misleading in the model. I
really don't think that that is the case. I think that a good aggressive
program that monitors the environment, takes action based on any
positive, is going to do more to prevent food contact surface
contamination and finished product contamination then, and you'll have a
higher confidence in the product going out of your plant than you can
get from any amount of product testing.

MS. CHRISTIN:

I also wanted to say, with regard to the FDA and the updates to the
FDA/FSIS relative risk ranking, it would be very nice in considering the
in-plant model that you all have developed to be able to put them
side-by-side and figure out how this flows through. And I understand
earlier today the comment was made that FDA would try to have this
publicly available by June or July. One suggestion might be that you put
it out during the process of peer review, making clear that the document
is being peer reviewed. However, I think it's important that this
information be made available more quickly so that these two models
working together can be evaluated together. It's also very difficult to
tell from the risk assessment exactly how the FDA model was updated. And
I understand that Dr. Buchanan did explain some broad updates today. But
again, it would be really important to be able to look at the details of
that. So if \-- I would urge FDA to try to get its model out there in
the public domain and consider comments on the model and the review of
the model after it's been released.

MS. SCOTT:

I think I'd like to make a comment here with respect to the application
of this model's all ready-to-eat products and the issue of product risk.
I think, and we've been saying all along that there are products out
there that pose a much \-- much less risk than other products, and that
policies need to be tailored based on that. And I think that this risk
assessment, even though, you know, we have some problems with the
assumption, clearly showed that that is the case. If you have a
post-packaging intervention, even if we assume that it was wholly 90
percent efficiency, that was much, you know, had a very significant
impact on reducing the risk. Likewise, putting growth inhibitors in
products has a significant impact on the risk. And so I think combining
growth inhibitors with some management of Listeria in the environmental,
monitoring to ensure that Listeria's under control, you're going to have
a very effective program that's going to reduce the risk from those
products. And then we might need a different type of program in place
for those products that have no kind of barrier to growth that maybe can
have a long shelf life out there, and where Listeria contamination is
likely to occur.

MS. CHRISTIN:

I think that one of the things we need to bear in mind is we have all of
these tweaks to the model that we're asking the Agency to make, is that
consumers still are at risk from ready-to-eat products because, while we
have a new directive that imposes certain duties on the Agency, we still
are only left with a proposed rule with regard to the obligations
imposed on industry. There is an indirect effect on industry practices
as a result of the directive; however, the proposed rule on the
regulations are the instructions to the industry about what needs to be
done. So while we can offer all these criticisms, critiques,
suggestions, we can talk about getting new data and new research done,
we need to bear in mind that consumers continue to face the risk of
illness and death. And so we shouldn't allow this sort of exercise to
continue to put public health at risk.

MS. SCOTT:

And I think I need to comment here that I think the directive has done a
very positive thing. There's a lot of data being shared with the Agency
that were not being shared before. I think a lot of those data can be
used in this risk assessment, so we need to take advantage of that. And
I think that, certainly, even though it is just a directive, it
certainly has the same impact that having a regulation is, because there
are plants that are being shut down as a result of actions related to
this directive, or non-actions, not taking corrective actions. And so I
don't think we need to be overly concerned that just because a rule
isn't out, the industry isn't out there focusing on Listeria and keeping
Listeria contamination out of products and trying to do the best they
can.

MS. CHRISTIN:

Well, certainly, I did not mean to imply that industry was not trying to
do the best. I think that there are some firms, and we saw this with the
Wampler case, where the firm was doing testing and didn't know what to
do with the results. I mean there are those who need the sort of more
direct \-- more direct regulations and guidance to help them gauge their
response to the regulations. And the point you raise about the data that
will be generated by the FSIS directive, in keep \-- when we keep in
mind that the risk assessment needs to be transparent, I assume then
what you mean is that the data that the industry is going to share with
FSIS, you're willing to use to put in a public docket in order that FSIS
can revise its modeling?

MS. SCOTT:

But, clearly, the data would not be identified by plants, and I don't
think industry has any problem with sharing data as long as it's not
tied to specific plants. And I think that just having this directive
come forward also got industry to share more data with the Agency. A lot
of people walked in and said, look, here are the programs we have in
place. Look at what we can accomplish with this program. And keep this
in mind as you move forward.

MS. CHRISTIN:

I absolutely agree. I mean, that is a value of the directive, that FSIS
will finally be able to see some of the data that industry was gathering
and, in fact, the Department knew they were gathering; however, they
didn't have access to. So I agree with you. The more data that can be
shared, it just \-- we need to be cognizant that when we talk about data
being used to inform a risk assessment, again, we go back to the Codex
principles and all the other organizations you listed, which demand
transparency in the risk assessment modeling and decision making.

MS. HULEBAK:

I think we'll bring this part of this panel discussion to a close.
That's been \-- I think we've had an excellent point, counterpoint.
Thank you, both of you.

MS. CHRISTIN:

Thank you..

MS. HULEBAK:

And thanks to Sophia, of course. Now we'll take a brief break. Don't
leave your seats. We're going to bring the modelers and risk assessment
experts back up here. Carol Maczka will then moderate the final session,
which is an opportunity for questions from the floor of a technical
nature to solicit answers of a technical nature from the experts. Thank
you.

\*\*\*

\[Recess\]

\*\*\*

DR. MACZKA:

Thank you. Now we wanted to address one of the issues we heard, which
was the transparency issue. Janell?

MS. KAUSE:

Thanks, Carol. Charlotte raised a number of really good points about
transparency. I'd like to let the people here today know that there's a
more complete document now in the docket, as well as industry data
that's been added to the docket. So I was looking at some of the FPA
data and other datas that are out there, and is available at this time.

DR. MACZKA:

Thanks. If you would have any questions. Let's \-- do you want to
approach the microphone and the aisle and just identify yourself, and
this is for the recorder in the back.

MS. SCOTT:

Jenny Scott, NFPA. Do you plan on making the actual model available for
people to use so we can play around with like FDA made it available on
CD?

DR. GALLAGHER:

Let me get that one. The source code was put up so people can start to
look at that. On the to-do list is to send out a compiled version of it.
It's on the list with a lot of other things on the list, so yes, but.

DR. MACZKA:

Okay, we've got someone else moseying down here.

MR. GILLIS:

Kevin Gillis, Rode Inc. Quick point of clarification. When Ms. Scott was
talking about the types of interventions that we could have, she talked
about heat, of course, and we understand that. But just for
clarification, she said, and we can add growth inhibitors. Is that what
you're talking about when you talk about these post-process steps that
actually have in the model, as an output, the positive impact on public
health? Are you talking about an inhibition of the growth, and that is
maintenance of number, or are you talking about diminuation \[sic\] of
the number.

DR. MACZKA:

Janell, do you want to take that one?

MS. KAUSE:

I'm sorry, I was a little distracted by the noise in the background.

DR. MACZKA:

Okay. Okay, Eric wants to.

DR. EBEL:

Want to is a strong term. Yeah, it is \-- that's what we are intending
to be modeling, is an inhibition of the growth. So if we're monitoring a
log growth between retail \-- or between production and retail, this is
\-- this is affecting that in the model.

MR. GILLIS:

Right. There are\...

DR. EBEL:

It's\...

MR. GILLIS:

\...two ways that you identified to do that. One was a reduction in the
count before it goes to retail, and one was the growth inhibitory
package.

DR. EBEL:

Right. And\...

MR. GILLIS:

And we didn't want to have a confusion here about, really, what
technology we were talking about has the positive health benefit. Which
is that?

DR. EBEL:

Well, they both. Actually, they both show very similar effectiveness
side-by-side. I mean if they're independently modeled, then they\...

MR. GILLIS:

And if they put \-- if you were to put the two together, then you'd
get\...

DR. EBEL:

You'd get an additional effect, right.

MR. GILLIS:

Thanks.

DR. MACZKA:

Jenny Scott.

MS. SCOTT:

Thank you. My voice has changed. I think just a little further under
that, I think that when you presented this model you talked about GIP,
growth inhibitory packaging.

MS. KAUSE:

Right.

MS. SCOTT:

Are we not really primarily talking about growth inhibitors in the
product, itself? And I don't want people to be misled to think that all
we're talking about is inhibitors in the packaging material that goes
around the product, which could be another intervention, of course, but
that's not the only thing that's being modeled there.

MS. KAUSE:

That's correct, Jenny.

MS. BECKMAN:

My name is Ann Beckman. I have a question on behalf of the American
Frozen Food Institute. And the question really has to do with the scope
and the presentation of the document, and the risk assessment, as I
understand that is focused on and, in fact, limited to deli type
products, but the phrase ready-to-eat is used throughout the documents
in somewhat general terms. And FSIS has defined ready-to-eat products to
include, really rather broadly, a lot of frozen products, or separate,
many different types of frozen products, many of which have been found
to present a much lower risk with respect to LM growth. So the question
is can that \-- can the scope of the document be made a little bit more
clear. I know it's obvious if you read through the document, you can
tell that it is limited, that the risk assessment was limited to deli
meats. But could that scope be made a little bit more clear, perhaps, in
the title or in the initial introduction to the document?

DR. MACZKA:

Good comment. Yeah, we'll take that. Other questions? Yes.

MR. BARNHART:

Sorry. Don Barnhart from Barnhart's Foods. My question is about the
model, itself. On the plant data section of the model, in the sanitation
section you have a 50 percent reduction of Listeria species between
shift, 75 percent at the end of the shift, 95 percent then if there's an
event that causes some type of super sanitation or whatever. My question
is that those numbers, as I understand where you got them, that's been
well spoken. I don't understand how those numbers affect the model,
itself, because the samples in the model then are just on total number,
and you don't actually say in the samples, if you took 30 samples, let's
say, out of the month of 60 possible, you don't say whether one's on
first or one on second shift, so you don't know which sanitation event
took place prior to the sample. So I don't see how that percentage
number plays into a model.

DR. GALLAGHER:

Those sanitations between lots are always applied. The food contact
surface is then tested at the end of a lot production, which lot depends
upon the number of lots that are being tested that month and whether you
pick a systematic or random kind of sampling. But you look at the
concentration that's remaining on the food contact surface after the
lot's been produced, after sanitation has gone through, and you test
that to see if it's positive.

MR. BARNHART:

Okay.

DR. MACZAK:

Yes?

MS. JOHNSON:

Alice Johnson from the National Turkey Federation. I just have an
overall question about how the Agency intends to look at the model. I
want to piggyback on what Jenny said and commend the Agency for the work
they've done on the risk assessment, and I think this process has been
very good for explanation. In the past, when the Agency has done risk
assessments, they've subjected them to peer review. Is there any intent
on the part of the Agency to have this model reviewed?

DR. MACZAK:

Janell?

MS. KAUSE:

Yes, we consider the public meeting here part of the review process, as
well as the comment, and the Agency is going to give further thought to
that. Thank you.

MS. JOHNSON:

Thank you.

DR. MACZAK:

Jenny?

MS. SCOTT:

I want to go back to the sanitation question again. When does this
wipe-down take place where you get the 50 percent reduction, as opposed
to the .75?

DR. GALLAGHER:

After the first lot being produced that day.

MS. SCOTT:

Okay. So it really is the .75 was the third shift?

DR. GALLAGHER:

Yes.

MS. SCOTT:

Okay, that's all I have.

DR. MACZAK:

More questions? Anybody? Here we go. It looks like we have a few more
questions coming. Go ahead.

MR. HUFFMAN:

Randy Huffman, American Meat Institute. I just want to revisit a topic
that's come up a couple of times, and Jenny mentioned it in her
comments. I understand there's a paucity of data on contamination on
surface, product contact surfaces, and you worked with what you had, but
it appears that you make the assumption that contamination is evenly
distributed on product contact surfaces, using expert opinion, I'm
assuming. And I guess I'd just like to \-- and then maybe I'm not right
there, but if you could talk a little bit about why you didn't try to
model uneven distribution of contamination. I know it would be more
complex and more difficult, but certainly, that's reality, and that we
all know, any of us that work in \-- with this organism on a daily
basis. We know that it's not evenly distributed. We don't need a
published paper to tell us that. So what would it take to do that, a
model for that?

DR. GALLAGHER:

If you would be able to provide some data that gives us an idea of that
variability, we'd be happy to put it in the model.

DR. MACZAK:

And even if you don't have the actual data, if we can conduct an expert
elicitation, that's an acceptable means of actually trying to acquire
information when you don't have actual data. We would be willing to do
something like that to get a handle on it.

MS. CHUNG:

Yuong Chung from the NRPA. I was wondering, since you have a very nice
user interface in the model, when the model might be available for, you
know, experimentation or like a person like me who are interested?

DR. EBEL:

Okay, I guess I'll answer that. We already had a discussion about that,
and the answer is when Dan gets it done.

DR. MACZAK:

He is on a fast track though.

DR. GALLAGHER:

It is on the list. You have the source code available to start looking
at that, but the compiled version, we tried this weekend. There's some
bugs in the third-party add on, then and my hard CD ROM crashed. We're
working on it.

DR. MACZAK:

Another question?

MR. STEWART:

Skip Stewart, American Meat Institute. Janell, I think you commented
that more data is now available in support of this for the public. Do
you know if that includes the FSIS Listeria data for 2000, 2001 and
2002? I know we referenced 2002 in your presentation, I think, but is
that data now available for\...

MS. KAUSE:

I don't believe the 2002 is yet. In fact, that data was just
preliminary, as it said on the slide.

DR. MACZAK:

Loren, do you want to add to that?

MR. LANGE:

Well, we will, at the summit we've said, we would work expeditiously to
get that data public, and I think, you know, we should even make an
attempt, even if we have to label it preliminary. I think that data
should be our results from 2001 or 2002 should be public as soon as
possible, so we will take that under consideration how to, you know,
whether to sort of \-- what's on the web site was referred to, but being
et al was sort of edited, checked, and checked, and rechecked, and
audited, and then put in a publication. I think the general food
protection event went on our web site. We haven't been through that
level of detail and sort of rechecking to make sure everything's
accurate, but we will certainly consider other alternatives than \-- of
getting it up, because it\'s probably 99.9 percent accurate that we have
now.

DR. GALLAGHER:

And just as a follow up, I think for at least the data that I apparently
\-- I think I heard Loren say that the data that was shown here for the
draft risk assessment was all random data, and, you know, so if that
random data could be supplied that was used for this versus I understand
that the overall database probably contains a lot of targeted testing
that's done by FSIS on particular food products when there's a question.
So to be able to separate those two things out would be helpful for
those of us who are trying to better understand the draft risk
assessment.

MS. KAUSE:

Okay, thank you. The other thing I would like to get back to is that
2002 data was not actually used in the model. It was used as a point of
comparison to see if the model was hitting prevalences in numbers that
we're about in the ballpark.

DR. MACZKA:

I should mention that we have Sherry Dennis at the podium. She's a risk
coordinator at FDA, sitting in for Bob Buchanan. So if you do have any
FDA questions, she can take them on. Selected. Any other questions?
Jenny? We have Jenny. You might want to just stand up, stay up there.

MS. SCOTT:

Since you've given me the opportunity, Sherry, would you tell us exactly
where the data come from for deli meats that were used in the risk
assessment? I know some of it was our data, but what about the rest of
it?

MS. DENNIS:

Oh, boy, you're asking \-- is this on? Is it on?

DR. MACZKA:

Yes.

MS. DENNIS:

Gosh, this is \-- sounds like a quiz, to ask me from memory. Oh, isn't
this nice? My friend, my friends at FSIS. Most of the data were the data
that were used in the draft risk assessment, but there's some new data
that are included, and that's primarily the NFPA data, and then you \--
and some of the new data from FSIS. The rest of it, and I could read
through the list, but I think this is the information that will be
available, as Janell mentioned, in the docket today, and so this is \--
but it's mostly the list of information from the old \-- the draft risk
assessment plus these two new data sets.

DR. MACZKA:

We have someone coming with a question.

MR. CORRIGAN:

Thank you. Phillip Corrigan, Australian Federal Government. First of
all, I think I'll just congratulate the total effort that you're making
here to get a handle on this bug, and Australians had some very serious
outbreaks of Listeria as well. But my question, how much international
collaboration has there been in the work that you've done, if any, or
how much, if there has been some? And the second question is how
portable is your model to a setting in another country, or has it been
very much focused on the national scene here?

MS. KAUSE:

I'll take that. There hasn't been much international collaboration
because this risk assessment was put on an extremely fast track to be
developed in light of some of the outbreaks last fall, and in terms of
portability, as soon as the \-- the in-plant of the model, because
remember, it's a two-part model. It's the in-plant model, the dynamic
part, but by exercise as well as what's married up with the exposure
assessment part and the dose response code from the FDA model. At least
the in-plant model, we hope once it's compiled, will be very user
friendly, and we'll look into how we can get that out to the public. So
that one should be pretty easy to take and use.

DR. MACZKA:

Any other questions? Well, thank you for coming. Oh, no, we're not quite
finished yet. Loren. Loren's going to join us up here and tell us about
next steps.

MR. LANGE:

Let's see if I've got right on the microphone here. I'll add one
statement before I move to my closing remarks about the data, as we do
publish from the 2000, 2001 and 2002 data. For those of you that are
familiar with the testing programs FSIS ran, there used to be, and
what's on the web site, there were nine different product categories, if
you've looked at that. It's the small diameter sausage, large diameter
sausage, sliced ham and cooked poultry products. When the version of
10240.2 was put into effect late in 2000, we did move away from those
product categories. So the new data we will put up will be different,
and we started sampling in plants by pulling out of the PBIS system,
Performance Based Inspection System, the data whether the plant's at the
03G, has a process like fully cooked, nutshell, stable, and then we
sampled according to those, the four processes that actually had
ready-to-eat products. And then when the samples got to the lab, we did
a little bit what NFPA was talking about. We had our people, you know,
look at the product and put it into a category like is it sliced, diced
and shredded. So that was the data that Dan presented, the 2.3 percent,
which was our sort of highest category in 2002, is a lab determination
based on an examination of the product. So when we put it up and get
this data out to the public, it will be in different categories, so you
can't trace those nine product categories through time. And that was one
of just one of the factors we had to give up when we moved to what we
thought would be a more meaningful set of product categories.

I just want to begin by thanking all our speakers and panelists today. I
think it's been a good day. Thank Dr. Hulebak as sort of hosting the
meeting. It's always good to acknowledge the boss. A special thanks to
the risk assessment staff here. Carol Maczka and Janell Kause, Eric Ebel
and Dan Gallagher, who have supported us from Virginia Tech. And I want
to just mention, obviously, Moshe, who's been carrying that microphone
around, but the planning staff that has, you know, facilitated this
meeting. Out in the hall, I know Sheila Johnson was out there, Mary
Harris, Ida Gambrill, and certainly any of the rest of the communication
staff that helped put on this public meeting. And then, of course, thank
you for everybody that got here, considering the weather this morning.

I have about three or four just quick comments I'd like to make in
closing. People have mentioned we've asked for a public comment on the
risk assessment through March 14, 2003. Comments should be submitted to
the docket room and the Federal Register notice, you know, announcing
the meeting, has the address and comment on this risk assessment, just
like on any other docket from the Agency. And, of course, if anyone is
submitting data, sooner's better than later. We would certainly like to
see, if someone has some data that we can fit into this model, as soon
as we can get it, the sooner we can sort of decide how we can use it or
if we can use it. So we'd just appreciate, if you have something that's
ready and available, not to wait until the 14th. You know, we set the
deadline. I mean, I think there's a lot of people understand that a risk
assessment model is a living tool. It's \-- this is the current version
of the risk assessment model. In 2010, we'll have a different, hopefully
improved, more expanded version, and we'll probably still, you know, be
developing more sophisticated and better models even in 2020 and beyond.
But we have set the deadline of the comments because we do have to plan
internally, and we've sort of set a timeframe for when we want to get,
you know, information to the risk managers. We've shared what's
available now with the risk management staff within the Office of Policy
in the Agency, and we do want to get them additional runs and additional
data and other information as soon as possible. So although there's a
timeline in commenting on the model, we certainly don't want to imply
that that's the end of the model and that's the end of trying to conduct
risk assessment for Listeria. As to sort of the next steps for OPHS and
for the risk assessment staff, there is, on the 14th, we did put on the
web site a draft Listeria risk assessment document. There is an expanded
document that covers everything that's presented today. I think that's
correct. That is actually available in the docket room as of today,
along with the deli meat data from NFPA, and I think some\...

DR. MACZKA:

AMI.

MR. LANGE:

\...AMI data. So they are all in the FSIS docket room at this point. And
I think we decided yesterday, we're going to try to get that expanded
and updated draft risk assessment with the date of February 26 up on the
web site as soon as possible. I think everybody's heard that we'll
probably get two types of comments. Some will be comments that can
change one of the inputs into the existing model. Other comments may
suggest a redesign of the \-- some of the basic assumptions and \--
behind the model, such as this, the generic food contact surface, such
as the uniform distribution on food contact surface and stuff. And,
obviously, you know, those would take longer. And just because we intend
to sort of give input to the risk managers from our existing model
doesn't mean that we won't continue to work on, well, further
improvements, or as what I would refer as the next generation of the
risk assessment model. We heard the term transparency a lot today.
Certainly, it was our goal, and being here is to be as transparent with
respect to this model as possible. I guess one could almost say that,
you know, for the presentation here for a model developer, they were
baring the innermost secrets of their souls. I mean this is we were \--
we were certainly \-- we were trying to hide nothing. So \-- and we
tried to explain it as best as possible. And if someone has comments and
suggestions on how models can be explained and presented to the public,
and in a forum like this of how best it is, we certainly would
appreciate comments on that, because we did the best we could, I think,
to try to explain the details of the model. There's some \-- it's
because I'm from a past history, an old modeler or something Jenny said
that did that little hair up on my back, that term. Inaccurate or
misleading assumptions. No one starts out developing a model and says,
ah, we're going to use this inaccurate assumption. You know, we're
planning to do this misleading model. No, modelers don't work that way.
I guess I'll refer that we refer to all this as sort of our best
available, although they may be limited assumptions and, you know, open
to debate and affect the outcome. No one does start out with a \-- so I,
you know, I just noticed that, that when the comments come in, that I
don't think anyone means that these are inaccurate or misleading
assumptions. They were just the assumptions we had to use to develop the
model. And I, to close, you know, it sort of reminded me of this past.
My first seven years in the federal government. I was building a form of
risk model very similar to this for the Department of Defense. We had
programs in Fortran programming and Simscript and something called
basic, so the world's changed a lot. And I don't remember anything about
how I would do it, or how I did this 30 \-- 30 years ago. But I do
remember one thing. It was difficult, and it was challenging. So, I
mean, I, personally, other people have, but I, personally, want to
congratulate Eric, Dan and Janell for the work they have done in
developing this model, because I think, you know, they've really done
just an outstanding effort. So congratulations. And I'd like to just
then close and say good-bye with one comment. Is I do remember one other
thing. Besides it being difficult is that I do remember that the task of
trying to simulate reality and build a model that does reflect reality
does really provide an excellent sort of opening into what are research
needs, what are data needs, and what are the important questions to be
asked. I mean I'm a very strong supporter in us trying to, you know,
develop models and develop improved models because it is a way that you,
you know, approach and attack a problem like this, and I think it's very
important. So thank you for all coming, and with that, I guess the
meeting is adjourned.

CERTIFICATE OF REPORTER, TRANSCRIBER AND PROOFREADER

IN RE: FSIS DRAFT LISTERIA RISK ASSESSMENT TECHNICAL MEETING

HELD AT: WASHINGTON, D.C.

DATE: FEBRUARY 26, 2003

We, the undersigned, do hereby certify that the foregoing pages,
numbered 1 through 249, inclusive, are the true, accurate and complete
transcript prepared from the reporting by the reporter in attendance at
the above identified hearing, in accordance with applicable provisions
of the current USDA contract, and have verified the accuracy of the
transcript by (1) comparing the typewritten transcript against the
reporting or recording accomplished at the hearings, and (2) comparing
the final proofed typewritten transcript against the reporting or
recording accomplished at the hearing.

Date:

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Janet R. Smeltz, Transcriber

York Stenographic Services, Inc.

Date:

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Sarah Mowrer, Proofreader

York Stenographic Services, Inc.

Date:

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_

Charles Brown, Reporter

York Stenographic Services, Inc.