USDA F 

Peer Review Comments and Responses to 

a Draft Risk Assessment of the Public 

Health Effect of an FSIS Catfish 

Inspection Program 

Risk Assessment Division 

Office of Public Health Science 

Food Safety and Inspection Service 

United States Department of Agriculture 

March 2011 
 

 

 

 

 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Table of contents 

1 Introduction................................................................................................................. 2 

2 Reviewers.................................................................................................................... 2 

3 Evaluation Criteria...................................................................................................... 5 

4 Reviewer Number 1 .................................................................................................... 7 

5 Reviewer Number 2 .................................................................................................. 14 

6 Reviewer Number 3 .................................................................................................. 31 

7 Reviewer Number 4 .................................................................................................. 42 

8 Reviewer Number 5 .................................................................................................. 59 

1 INTRODUCTION 

This document provides responses to comments provided by five independent peer 

reviewers to the risk assessment “Draft Quantitative Estimation of the Public Health 

Impact of an FSIS Catfish Inspection Program” dated April 30th, 2009. The risk 

assessment estimates the likelihood of foodborne illness and death caused by Salmonella 

and various chemical compounds in or on catfish. This peer review was conducted in a 

manner consistent with Office of Management and Budget (OMB) Peer Review 

Guidelines (Final Information Quality Bulletin for Peer Review, December 15, 2004 

http://www.whitehouse.gov/omb/memoranda/fy2005/m05-03.pdf). 

Reviewers’ comments are provided verbatim1. Following each paragraph of general 

comments and specific comments, we provide a response. Responses are in Arial font 

and indented. All comments and responses are numbered but are not otherwise labeled. 

The 2009 FSIS risk assessment for catfish was revised in response to these comments and 

updated version of this risk assessment has been made available for public comment: 

http://www.fsis.usda.gov/PDF/Catfish_Risk_Assess_Dec2010.pdf. 

2 REVIEWERS 

An independent, external peer review of the risk assessment was conducted under 

contract with ICF International (ICF) in July 2009. Under the OMB peer review 

guidelines, Agency information submitted for formal peer review is confidential and not 

to be distributed. Five scientific/technical experts (the primary disciplines/types of 

expertise needed for review were modeler/engineer, toxicologist, food scientist, and 

microbiologist) were needed to provide an independent review of the risk assessment. 

ICF identified and chose potential reviewers. The names of the reviewers were withheld 

until all five reviews were submitted to FSIS. 

Below we present a brief biographical sketch of the reviewers’ relevant experience at the 

time of the review. The numerical order of reviews below is unrelated to the alphabetical 

listing of names. 

1 Minor formatting changes were made for consistency of presentation. 

2
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Douglas John Crawford-Brown, Ph.D. 

Dr. Crawford-Brown is Emeritus Professor in Environmental Sciences and Policy and 

Emeritus Director of the Institute for the Environment, at the University of North 

Carolina at Chapel Hill. He has also directed the Summer Programme in International 

Energy Policy and Environmental Assessment in Cambridge, UK, for the past 15 years. 

He moved permanently to Cambridge, UK, in 2008 and is now Director of Sustainability 

for Pell Frischmann, an engineering consultancy in London; a Principal in EnviroTech, a 

business-for-business venture supporting growth of cleantech SMEs; and operates 

Crawford-Brown Energy and Environmental Consultancy. He received his degrees in 

physics (BS; MS) and nuclear science (PhD) from the Georgia Institute of Technology. 

His activities focus on sustainability in the public and private sectors, modeling human 

health risks from environmental contaminants, modeling alternative policies to tackle 

environmental problems, assessing the quality of scientific information, and developing 

tools of risk assessment. He is the author of 140+ scientific articles and 5 books on these 

subjects. He has served on the USEPA’s National Drinking Water Advisory Committee, 

Clean Air Scientific Advisory Committee, Endocrine Disruptor Screening and Testing 

Advisory Committee and National Pollution Prevention and Toxics Advisory Committee; 

on the American Water Works Association’s Technical Advisory Workgroup; on the 

EC’s Panel of Scientific Experts on Health and Risk Assessment; on the Legislative 

Global Climate Change Commission; and on an array of environmental workgroups in 

the UK, Austria, Mexico, France, Germany and Thailand. 

Robert L. Goble, Ph.D. 

Dr. Goble is Research Professor in Environmental Science and Policy (ES&P) and 

Adjunct Professor of Physics at Clark University. He is a member of the George Perkins 

Marsh Institute, Clark’s interdisciplinary research center concerned with humanenvironment interactions, and was its Director from 2006-2008 . Professor Goble 

received a B.A. with honors in physics from Swarthmore College in 1962 and a Ph.D. in 

Physics from the University of Wisconsin in 1967; he worked for nine years thereafter in 

theoretical high energy particle physics at Minnesota, Yale, Utah, and Montana State. 

Beginning in 1974 he turned his interests increasingly to technology assessment and 

hazards, which brought him to Clark University in 1976. Dr. Goble’s current research 

focuses on developing a risk, vulnerability and uncertainty perspective on environmental 

exposures and health. This has included studies of implications of high uncertainty 

supported by NSF and WHO and studies of the implications of inter-individual variability 

among people for assessing exposures and dose response relations supported by the US 

EPA, the Department of Energy, and by the state of California. Much of this work has 

been performed with his colleague Dale Hattis at the Marsh Institute and includes studies 

of the age dependence of exposures and risk. 

Dr. Goble is part of a group at Clark developing community based participatory research 

on health and environmental issues within Worcester; this group is also planning 

exposure assessments in the National Children’s study in Worcester County. Dr. Goble 

studied risks of extreme sea-level rise for the Stockholm Institute and has studied 

regional approaches to sustainability for the German state of Baden-Wurttemberg. He 

was lead author of the exposure volume in EPA’s Critical Assessment Document for 

Acid Deposition. He has worked on projects concerned with the interpretation of fish 

3
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

consumption data (for the Sierra Club), with vulnerability to mercury exposure (for 

EPA), and with seafood risk screening (for the Rhode Island Department of 

Environmental Protection). Dr. Goble has also worked on several projects relating to 

environmental justice and to cooperation between scientists and non-scientists in risk 

assessment; these have been supported by the National Institutes for Health and the NSF. 

Dr. Goble has served on advisory panels for the United Nations, for the National 

Academy of Sciences, and assisted local community supervision of major health studies. 

As part of his research Dr. Goble has been a principal mentor of many MA and several 

Ph.D. students in Physics and in the Environmental Science & Policy Program. 

Dariush Mozaffarian, Ph.D. 

Dr. Dariush Mozaffarian is a cardiologist and epidemiologist; Co-Director of the Program 

in Cardiovascular Epidemiology (www.hsph.harvard.edu/research/cvdepi/); Assistant 

Professor in the Division of Cardiovascular Medicine, Brigham and Women’s Hospital 

and Harvard Medical School; and Assistant Professor in the Department of Epidemiology 

at the Harvard School of Public Health. His research focuses on the effects of lifestyle, 

particularly dietary habits, on cardiovascular health and disease. 

Dr. Mozaffarian has authored numerous scientific publications and research studies 

relating to lifestyle and cardiovascular health, including papers on trans fatty acids; fish 

and omega-3 fatty acids, contaminants, and human health; and the Mediterranean diet. 

He has served on several national and international committees and advisory boards, 

including the American Heart Association Epidemiology and Prevention Leadership 

Committee, Nutrition Committee, Statistics Committee, 2020 Goals Committee, and 

Trans Fat Initiative Committee; the United Nations Food and Agriculture Organization / 

World Health Organization Expert Consultation on Fats and Fatty Acids in Human 

Nutrition; the Pan American Health Organization Task Force on Trans Fat Free 

Americas; the Canadian Health Measures Survey Expert Advisory Committee; the U.S. 

Department of Agriculture Seafood Education Project Advisory Group; the Gates 

Foundation / World Health Organization Global Burden of Diseases Nutrition Expert 

Group (Chair); and the Reviewer Group for the U.S. Food and Drug Administration 

Nutrition Methylmercury Risk Assessment. 

A Fellow of the American College of Cardiology and Fellow of the American Heart 

Association, Dr. Mozaffarian received a B.S. in biological sciences from Stanford 

University (with Honors, with Distinction, Phi Beta Kappa), an M.D. from Columbia 

University (Alpha Omega Alpha), an M.P.H. from the University of Washington, and a 

Doctorate in Epidemiology from the Harvard School of Public Health. He is boardcertified in Internal Medicine and Cardiovascular Medicine. 

Charles R. Santerre, Ph.D. 

Dr. Charlie Santerre is a Professor of Food Toxicology in the Department of Foods and 

Nutrition at Purdue University. Prior to this, he served as an Operations Manager of 

Chemistry at a private food testing laboratory in Columbus, OH and as an Adjunct 

Associate Professor in the Environmental Sciences Program at Ohio State University and 

4
 
 

 

 

 

 

 

 

 

 

 

 

3 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

as an Assistant Professor in the Environmental Health Science Program and the Institute 

of Ecology at the University of Georgia. His research involves food toxicology and 

nutrition. He was the National Spokesperson for the Institute of Food Technologists and 

has served as Chairperson for the Toxicology and Safety Evaluation Division, and as the 

Director of the Food Toxicology Center of the National Alliance for Food Safety. He is 

currently a Scientific Advisor for the American Council on Science and Health, a 

Scientific Expert for the International Food Information Council, and a full member of 

the Society of Toxicology. He received a B.S. degree in Human Nutrition and a Ph.D. 

degree in Environmental Toxicology and Food Science, both from Michigan State 

University. 

Harlee S. Strauss, Ph.D. 

Dr. Harlee S. Strauss is an expert in human health risk assessment and toxicology with 

experience in both community and workplace settings. She has over 20 years of 

consulting experience and is currently the president of H. Strauss Associates, Inc., a 

consulting firm she founded in 1988. Dr. Strauss has taught toxicology to undergraduate 

and graduate students, most recently at Clark University in Worcester, MA. 

Dr. Strauss has personally conducted over 60 site specific human health risk assessments 

several of which had major exposure pathways related to marine or freshwater 

environments. She has evaluated the toxicity of a range of chemicals including 

chlorinated and non-chlorinated solvents, petroleum hydrocarbons, polycyclic aromatic 

hydrocarbons, polychlorinated biphenyls, nitrogen-containing compounds, metals, 

pesticides, and gases such as hydrogen sulfide. These evaluations were conducted for 

various purposes including risk assessment, setting or reviewing exposure or clean-up 

standards, and the investigation of possible relationships between exposure and reported 

health effects in workers or community residents. 

Dr. Strauss has participated in numerous multi-disciplinary projects that applied technical 

knowledge to challenging problems. She has also been a peer reviewer of toxicology and 

risk assessment reports, including the Hudson River, the Proposed Bioaccumulation 

Testing Evaluation Framework for Determining the Suitability of Dredged Material to be 

Placed at the Historic Area Remediation Site (HARS) in New York, Drake Chemical 

Incinerator, various screening assessments under the Canadian Environmental Protection 

Act, and an ATSDR toxicological profile. 

EVALUATION CRITERIA 

Reviewers were asked to respond to the following set of evaluation criteria to facilitate 

the organization and presentation of their comments. These “evaluation criteria” 

constitute the FSIS “charge to peer reviewers” (as defined in OMB’s Peer Review 

Guideline, December 2004). 

1.	 Comment on the model documentation. 

1) Is the report clearly written? 

2) Does it completely cover all aspects of the analyses? 

5
 
 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

3)	 Does it follow a logical structure and layout? 

4)	 Where data are included/excluded, does the documentation 

provide adequate justification? 

5)	 If the answer to any of the above questions is No, the reviewer 

should provide an alternate outline and/or approach for 

documenting adequately and clearly this risk assessment. 

The major assumptions of the model are as follows; please comment according to 

charge #2 below. 

A. Input distribution for each hazard in the analysis. 

B. Consumption distribution in the US population 

1. Distribution of catfish consumption frequency of each age-sex category. 

2. Distribution of the amount of catfish consumed in each age-sex 

category. 

2. Comment on the validity of the model. 

a. Are these assumptions logical and adequately justified? 

b. Is the selection of data appropriate? 

c. Are model outputs reasonable? 

d. Is the evaluation of uncertainties sound? 

3. Comment on the risk assessment model. As a general matter, comment on the 

hazards identified in the assessment? Have we failed to identify 

important hazards? In addition: 

a.	 Chemical Hazards 

i.	 Are the methods utilized to identify the most significant 

potential hazards associated with consumption of 

catfish valid? 

ii.	 Are the approaches utilized to estimate exposure and 

level of concern resulting from the consumption of 

catfish valid? 

iii.	 Are the approaches utilized to estimate the prevalence 

of adverse effects (acute and chronic) resulting from 

hazard exposure valid? 

b.	 Microbial Hazard 

i.	 Are the methods utilized to identify the most significant 

potential hazards associated with consumption of 

catfish valid? 

ii.	 Are the approaches utilized to estimate exposure and 

dose-response resulting from the consumption of catfish 

valid? 

iii.	 Are the approaches utilized to estimate the prevalence 

of adverse effects (acute and chronic) resulting from 

hazard exposure valid? 

6
 
 

 

 

4 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

REVIEWER NUMBER 1 

Comment 1 

1.	 Comment on the model documentation 

1) Is the report clearly written? yes 

2) Does it completely cover all aspects of the analyses? as well as it can 

3) Does it follow a logical structure and layout? yes 

4) Where data are included/excluded, does the documentation provide adequate 

justification? no (see comments below) 

5)	 If the answer to any of the above questions is No, the reviewer should provide 

an alternate outline and/or approach for documenting adequately and clearly 

this risk assessment. (see comments below) 

The major assumptions of the model are as follows; please comment according to charge 

#2 below. 

A. Input distribution for each hazard in the analysis. 

B. Consumption distribution in the US population 

1. Distribution of catfish consumption frequency of each age-sex category. 

2. Distribution of the amount of catfish consumed in each age-sex category. 

2. Comment on the validity of the model. 

a. Are these assumptions logical and adequately justified? (see comments below) 

b. Is the selection of data appropriate? (see comments below) 

c. Are model outputs reasonable? (see comments below) 

d. Is the evaluation of uncertainties sound? (see comments below) 

4.	 Comment on the risk assessment model. As a general matter, comment on the hazards 

identified in the assessment? Have we failed to identify important hazards? In 

addition: 

a.	 Chemical Hazards 

i.	 Are the methods utilized to identify the most significant potential 

hazards associated with consumption of catfish valid? (see comments 

below) 

ii.	 Are the approaches utilized to estimate exposure and level of concern 

resulting from the consumption of catfish valid? (see comments 

below) 

iii.	 Are the approaches utilized to estimate the prevalence of adverse 

effects (acute and chronic) resulting from hazard exposure valid? (see 

comments below) 

b.	 Microbial Hazard 

i.	 Are the methods utilized to identify the most significant potential 

hazards associated with consumption of catfish valid? 

ii.	 Are the approaches utilized to estimate exposure and dose-response 

resulting from the consumption of catfish valid? 

7
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

iii.	 Are the approaches utilized to estimate the prevalence of adverse 

effects (acute and chronic) resulting from hazard exposure valid? 

In addition, I was specifically asked to address the following: 

1.	 Answers to charge questions and any additional comments 

2.	 A description of the procedures used to arrive at your recommendations (Please 

make clear and substantiate the methods and considerations upon which your 

recommendations are based.) 

3.	 A summary of your findings; and 

4.	 A list of sources relied upon (references). 

Since I am a food scientist/toxicologist, I will focus on the chemical safety portion of the 

Draft. 

Overall Rating: The risk assessment is well written and all aspects are clearly defined and 

explained. The length of the document is appropriate. 

Concerns: Since the purpose of this risk assessment was to determine the positive impact 

of a continuous ‘catfish’ inspection system on catfish producers/processors in regards to 

public health, I will address the practical aspects of the risk assessment. 

This type of approach may work well for pathogens which are likely to be present in the 

gut of all catfish but I believe that there are significant problems in assessing 

contaminants in the same manner. I would like to first address domestically-grown 

catfish since this is where I am most experienced. Let me start with a discussion of some 

compounds that are not intentionally added to catfish feed or production ponds. 

Mercury – This is a non-point source environmental pollutant that is found in all fish. It 

primarily enters through the diet and binds to proteins in the muscle. It also concentrates 

in organs, like the kidneys, brain and liver, but consumers do not generally consume 

these parts of the catfish. Mercury accumulates in large marine fish species and is not 

reduced by cooking. Our studies have shown that the average mercury concentration in 

domestically-produced catfish fillets was ~8 ppb with a maximum of 89 ppb. The FDA 

regulatory limit for fish is 1,000 ppb. The EPA’s Reference Dose (RfD = 0.1 μg/Kg body 

weight/d) is significantly more conservative but it would allow a 60 Kg pregnant or 

nursing woman to eat up to 6 μg Hg/d or 750 g (1.6 lb) of catfish per day. While the is 

intense scientific debate about the findings from studies conducted in the Seychelle 

Islands vs. Farao Islands, the National Research Council (2000) found adequate evidence 

for development and implementation of the EPA’s RfD. Thus, for sensitive populations, 

it is best to use the EPA’s RfD instead of the ATSDR’s MRL in order to be more 

protective of the fetus and nursing infant. The risks to the fetus and nursing infant are 

believed to far outweigh the risks to the general population. So, this should be considered 

when developing a risk assessment. Regardless of one’s position of the safety limit, 

mercury concentrations in catfish are already well below regulatory limits for commercial 

seafood and it would be difficult to further lower these concentrations. Thus, the benefits 

from lowering mercury residues would have a marginal impact on risk and an inspection 

8
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

system which monitors mercury in domestic catfish would provide no benefits. To 

counter the potential adverse IQ effects that have been associated with elevated intakes of 

mercury, it might be better to require the inclusion of fish oil which contains long chain 

omega-3 fatty acids (EPA, eicospentanoic acid; DHA, docosahexanoic acid) into the 

catfish feed. The beneficial impact to public health would outweigh the predicted 

beneficial impact of an inspection system. Unfortunately, increasing fish oil in catfish 

feed would also increase PCB concentrations. 

Response 1 

The initial draft risk assessment’s predictions regarding adverse human health 

outcomes associated with mercury (and other chemical compounds) were 

determined to be insufficient to warrant inclusion in the cost-benefit calculations 

of the regulatory impact analysis. In particular, there is no empirical data that 

measures the level of effectiveness of an FSIS inspection program to reduce 

chemical residues on food. The revised risk assessment now focuses only on the 

microbial hazard Salmonella, with which estimates program effectiveness can be 

based on FSIS empirical data on poultry models. The appendix of the revised 

risk assessment includes the hazard identification and exposure assessment 

sections for the chemical hazards. Including these sections provide a qualitative 

assessment of these hazards. 

Comment 2 

Lead – I am concerned that data regarding lead concentrations in catfish tissue may be 

dated. Since the elimination of leaded gasoline, the concentrations of lead in all food 

products have significantly declined. The data reported in our catfish study are now 15 

years old. Thus, lead is likely to be less of a public health concern today in regards to 

exposure from food. 

Response 2 

The data the reviewer is referring to are still the most recent quantitative 

data for lead in catfish. As such, these data were chosen for use in the 

original version of the catfish risk assessment. However, because the 

revised risk assessment focuses on Salmonella, the effects of lead (and 

other chemical compounds) are no longer considered. 

Comment 3 

Diuron – Unlike pathogens and mercury, this pesticide behaves like a point-source 

pollutant and is more likely to show up only in catfish grown near rice-production areas, 

like the Mississippi delta. Since it is a not a volatile or semi-volatile compound, it must 

be measured using a single residue method. I am not aware of any diuron residue data for 

catfish but I would bet it is somewhat limited. This would be one example where testing 

for diuron in catfish that were produced in Georgia, for instance, would not provide 

meaningful results. To assess exposure to diuron, one needs residue data and also data 

involving the consumption of the contaminated catfish. So, the risk to the entire 

population might suggest that this contaminant should not be included in a risk 

9
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

assessment. However, the risk to a specific population (i.e., those that eat catfish 

produced in this region of the country) may suggest that diuron be included in the 

assessment. My intention here is not to advocate that diuron be included but rather to 

point out a limitation of this risk assessment. 

Response 3 

Diuron and other herbicides were considered medium priority hazards 

following the hazard identification stage of the original version of the risk 

assessment. Because the revised risk assessment focuses on 

Salmonella, the effects of diuron (and other chemical compounds) are no 

longer considered. 

Comment 4 

PCBs/Dioxins/Furans – Dioxins are highly toxic compounds that have found their way 

into domestic catfish from feed. Once they were detected and the source identified, it was 

straight forward to remove them from feed and eliminate the problem. Unfortunately, a 

congener analysis is expensive and can cost $1,400 per sample. In addition, the assay 

may take several weeks to complete which make it impractical for use in an inspection 

system. I am not aware of a rapid assay for dioxins in fish tissue; however, USGS has 

developed a sample preparation method that uses a commercially-available ELISA 

(enzyme-linked immunosorbent assay) kit to measure PCBs as Aroclor. We have evolved 

their method to make it useful for screening recreationally-caught fish when developing 

fish consumption advisories. The assay can be completed in a matter of hours but 

confirmation with a GC/MS method would be recommended. While PCBs are still 

percolating from landfills and Superfund sites, their presence in the environment or 

around production ponds can be determined and the need for further testing minimized. 

More typically, PCBs and dioxins enter catfish through contaminated feed and here is 

where it would be best to monitor for PCBs/dioxins/furans. If PCB concentrations in 

catfish are below the federal tolerance of 2 ppm, then it is questionable whether an 

inspection program will be useful. If a limit were established for PCB concentrations in 

feed, then the concentration in catfish could be lowered. 

Response 4 

As mentioned (Response 3) the national residue sampling program will 

determine the chemical compounds to target each year. Testing methods 

will also be determined using this interagency process. 

The Food and Drug Administration has the statutory authority to set tolerances 

for chemicals in feed. 

Comment 5 

DDT/Organochlorines – Even though many of these pesticides were banned in the U.S. 

during the mid-1970s, residues can still be detected in many meat, poultry and seafood 

products. The regulatory limit for DDT-like compounds is 5 ppm and very few domestic 

catfish even come close to this concentration. Our catfish study found mean DDT in 

those samples that had detectable amounts (55%) to be 43 ppb. There are some hot-spots 

of chlordane around the country. Here again, for domestic catfish, measuring pesticides 

10
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

that have long since been banned may not be the best use of resources and may provide 

little benefit to public health. 

Response 5 

The risk below regulatory limits is not zero. The sum of low level risks across an 

entire population may be significant. Because the revised risk assessment 

focuses on Salmonella, the effects of DDT (and other chemical compounds) are 

no longer considered. 

Comment 6 

For the previously mentioned compounds, a continuous inspection program would be of 

little value since these compounds are present at concentrations that are well below 

regulatory limits. It might be helpful to occasionally monitor some of these compounds in 

feed since the diet is the primary route of exposure. One other metal that was not 

mentioned in the Draft is chromium which may be found as an environmental pollutant 

near wood treating facilities that can be in close proximity to production ponds. Another 

class of compounds which is being detected in fish collected across the country is the 

brominated flame retardants (polybrominated biphenyl ethers). These compounds are 

increasing being detected in wild fish but concentrations are relatively low. A couple of 

the commercial PBDE formulations have been banned or voluntarily withdrawn by the 

manufacturer. 

Response 6 

As mentioned (Response 1) the national residue sampling program will 

determine the chemical compounds to target each year. It is unlikely that 

FSIS inspection programs would generate sufficient benefits to warrant 

inclusion of this compound in the risk assessment. The Food and Drug 

Administration has the statute authority to set tolerances for chemicals in 

feed. 

Comment 7 

I recommend that environmental pollutants (i.e., mercury, PCBs, lead, DDT) which are 

not intentionally added and which are below regulatory limits be eliminated from the risk 

assessment. I also suggest that some context be provided for use of cancer as an endpoint. 

It is difficult to compare illnesses from a foodborne pathogen to lifetime risk to cancer. 

Response 7 

We agree with the reviewer that comparison of numbers of illnesses to 

numbers of cancers is difficult. Yet, economic analysis is intended to 

provide such comparisons. Using our original risk assessment predictions, 

the economic benefits of foregone cancer cases were not considered 

substantial. Therefore, compounds associated with cancer risk are not 

considered further in the revised risk assessment. 

11
 
 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 8 

Let me discuss some of the chemicals that may be intentionally added to feed or 

production ponds. In our study (and also from FDA’s monitoring program), the pesticide 

chlorpyrifos, was detected in catfish samples from MS and TX during the first year of our 

study. There was concern that the pesticide was entering ponds from aerial drift during 

application to adjacent row crops and it was subsequently banned from use in these fish 

production areas. Since catfish are more tolerant to this pesticide than many other fish 

species, some have surmised that this pesticide was being intentionally added to ponds to 

kill scrap fish just prior to harvest (i.e., a harvest aid). Our research never attempted to 

determine whether chlorpyrifos was entering through intentional or unintentional means. 

However, we did develop a rapid method using a commercial ELISA kit to measure this 

compound in fish tissue. It was our belief that if producers were aware that a screening 

method was being used, any possibility of improper use would disappear. To our 

knowledge, this kit has never been used by processors. Following the controversy 

involving chlorpyrifos, I have heard comments that 2 other compounds (diazinon and 

malathion) had replaced chlorpyrifos and were being used (illegally) as a harvest aid. I 

have no evidence that would support or refute this contention. It may be prudent to 

monitor these pesticides to discourage improper use during catfish production. 

Response 8 

This comment pertains to the national residue sampling program. As 

already mentioned (Response 1) this program will determine which 

chemicals to include in a residue surveillance program. Chemicals 

selected for testing within the National Residue Program scheduled 

sampling program are ranked using several factors including regulatory 

concern, violation data, etc. 

Comment 9 

Drugs – One drug (emamectin) is being used in salmon production by every foreign 

salmon producing country. This chemical, which is added to feed and used to treat for sea 

lice, is being legally used in the U.S. under an IND but it has not yet been approved for 

general use. While it appears that approval is imminent, other countries (Canada, Chile, 

Norway) have disregarded U.S. policies by using this drug in salmon that are exported to 

the U.S. FDA-CVM is not currently testing for emamectin in imported or domestic 

salmon. If producers follow an approved withdrawal time, the residues of certain 

antibiotics can be at or below detection limits at harvest. For the current risk assessment, 

we should consider whether enforcing residue restrictions would benefit consumers even 

if there are indirect benefits. To illustrate, it is thought that excessive use of ‘agricultural 

grade’ emamectin in Chile has led to resistance by sea lice and has increased the spread 

of infectious salmon anemia. As a result of ISA spread, many production areas have been 

shut down. The local affect is that domestic consumers may find it to be increasingly 

difficulty to purchase farmed salmon. Since Chile provides about 60% of the salmon 

consumed in the U.S., any reduction in their exports to the U.S. may reduce our dietary 

intake of omega-3 fatty acids and thereby, increase the risk of sudden cardiac deaths. So 

in this scenario, properly enforcing regulatory limits can improve public health. 

12
 
 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 9 

It is unclear from this comment whether emamectin is used in the catfish 

industry. We agree with the comment “properly enforcing regulatory limits 

can improve public health”. The national residue sampling plan (see 

Response 1) will determine the compounds to target each year. 

Comment 10 

Listeria monocytogenes – Since there is a growing effort to encourage pregnant women 

to eat fish in order to have healthy babies, I would suggest that the risk assessment 

looking more closely at the risks from this pathogen which can cause spontaneous 

abortion and can grow at refrigeration temperatures. Seafood, including catfish, can be 

improperly cooked by consumers. 

Response 10 

Hazard identification determined that Listeria monocytogenes was a 

medium priority microbial hazard. This and other foodborne agents 

associated with the consumption of catfish are discussed in a recently 

published scientific paper (McCoy, E, Morrison, J., Cook, V., Johnston, J., 

Eblen, D., and Guo, C. (2011). Foodborne Agents Associated with the 

Consumption of Aquaculture Catfish. Journal of Food Protection 74(3): 

500-516(17)). Catfish are rarely produced in a ready-to-eat (RTE) form 

and human Listeria monocytogenes foodborne illnesses are most 

commonly associated with RTE products. Furthermore, the estimated total 

number of annual Listeria monocytogenes illnesses among the U.S. 

population is about 1,662 cases (Scallan, E., Hoekstra, R. M., Angulo, F. 

J., Tauxe, R., Widdowson, M., Roy, S. L., Jones, J. L., and Griffin, P. M. 

(2011). Foodborne Illness Acquired in the United States – Major 

Pathogens. Emerging and Infectious Diseases 17(1): 7-15). There is no 

evidence to suggest that catfish are more likely than other meat products 

to be contaminated with Listeria monocytogenes. Because the total 

servings of catfish consumed annually are substantially smaller than other 

meat products, it is unlikely that many Listeria monocytogenes illnesses 

would be attributed to catfish in our baseline scenario. If there are few 

baseline cases associated with catfish, then the cases avoided by the 

FSIS inspection program will be less significant. 

Comment 11 

One common difficulty that is encountered when developing a risk assessment regarding 

chemical contaminants is the lack of comprehensive studies that measure residue 

concentrations. It is often difficult for investigators to obtain research funding that would 

allow collection of food residue data. This is a limitation that poses a significant negative 

effect on the quality of this assessment. An accurate risk assessment requires adequate 

data to assess exposure and properly estimate human risk from this exposure. For 

chemicals that have not been approved for use in aquaculture production and processing 

(i.e., melamine, malachite green), there are significant gaps in residue data and there are 

13
 
 

 

 

 

5 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

also inadequate estimates of human risk. Ideally, a risk assessment will lead to a risk 

management plan which should include: routine testing for known and unexpected 

chemicals in feed, fish and water; an effort to develop analytical methods; and an 

educational component to help producers and processors make better decisions. 

Response 11 

The focus is on predicting the effectiveness of a continuous inspection 

system in reducing human exposures/illnesses to microbial and/or 

chemical hazards. We strongly agree with the comment that 

comprehensive studies are most useful for risk management. 

Comment 12 

In conclusion, due to the: lack of sufficient or current residue data; lack of consumption 

and exposure data that pertains specifically to sensitive populations (pregnant or nursing 

women) or consumers that are regionally exposed to a selected contaminant; lack of 

human studies that quantify the risk from to exposure to selected hazards; and lack of 

differentiation between background pollutants and illegal/intentional additives, this risk 

assessment cannot adequately determine the benefits of a continuous inspection program. 

I would therefore have to disagree with the conclusion as stated, “Our analyses indicate 

that the implementation of an FSIS inspection based program will have a beneficial 

public health impact by decreasing the number of such adverse effects experienced by US 

consumers.” I do believe that the limitations as stated should not reflect negatively on the 

authors that developed this excellent document. 

Response 12 

The revised assessment, which focuses on Salmonella, better validates the 

output estimates with more detailed discussion of the uncertainty surrounding 

the effectiveness of the FSIS inspection program. The revised risk 

assessment no longer considers the effects of chemical compounds. 

REVIEWER NUMBER 2 

Comment 13 

2.	 Comment on the model documentation 

1) Is the report clearly written? 

This report appears to have been written by folks with expertise in modeling, but little 

understanding of the underlying subjects of toxicology and the factors that determine 

chemical exposure-dose from fish consumption. It is well organized, but it is very short 

on explanation and detailed documentation of datasets actually used. For example, the 

written report does not provide the datasets for chemical contamination used in the 

model. While the data may be in the Excel spreadsheets sent to reviewers, these were 

clearly not made for public consumption and not adequately labeled, described and/or 

documented for outside review. For other parameters, such as toxicity, meal size and 

cooking loss, the report does not describe any of the extensive datasets available, but 

14
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

instead relies on one paper or website as the basis of the evaluation without regard to the 

many other papers in the literature that may show alternate relationships or values for the 

necessary parameters for the modeling. The report gives virtually no documentation for 

the reasons underlying the selection of distributions, only the final result. Curve-fitting 

(hazard concentrations, pareto distribution) is not a good reason to use a distribution if 

other distributions have an underlying physical or biological basis. 

Response 13 

The revised risk assessment is focused on predictions related to 

Salmonella. This update provides a more detailed description on the 

Salmonella model inputs than the original version (which necessarily was 

brief when fully discussing all 11 hazards). The remaining 10 hazards are 

included in the appendix of the revised risk assessment. 

Comment 14 

The report lacks perspective of what is important and how the different parts of the 

assessment connect with each other. For example, the report makes extensive use of 

detailed datasets (e.g. cardiovascular effects of lead) without considering whether the 

other parts of the analysis (i.e. the exposure assessment) are adequately robust to support 

that detailed an evaluation (I don’t believe it is). In addition, it ignores the fact that other 

chemical hazards, such as the chlorpyifos, an organophosphate pesticide, have 

cardiovascular effects that could contribute to the risk, but don’t have the detailed age/sex 

breakdown. The report doesn’t consider the toxic endpoints that have been associated 

with PCBs and DDT in wild caught fish (neurodevelopmental effects, reproductive 

effects – both perhaps ultimately stemming from endocrine disruptor activity), even 

though these chemicals are included in the identified hazards. The report neglects issues 

of ethnicity, which is an important determinant of exposure in terms of parts of fish 

consumed, method of preparation, and perhaps frequency of fish consumption. 

Response 14 

As indicated by the reviewer, there are numerous adverse effects 

associated with exposure to environmental chemicals. However, the 

nature of the adverse effects estimated in this risk assessment was limited 

to effects to which dose vs. response functions could be ascertained. 

Catfish consumption data were derived from 1996-2006 NHANES 

surveys. It appears that ethnicity is not included in these data sets. The 

assessment is based on the consumption of fillet muscle which constitutes 

the majority of catfish consumed in the US. Method of preparation 

(breaded vs. non-breaded, cooking style) was included in the risk 

assessment. 

Comment 15 

15
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

The report goes to great effort to develop a probabilistic model of adverse outcomes, but 

does not present the ranges of results obtained with the exception of one graph for 

Salmonella illness avoided and one graph of cancer avoided. Instead, the results are 

summarized as point estimates in Table 9 with no sense of the distribution or even a 

comment on where the number falls in the distribution or what the distribution looks like. 

If the probabilistic approach is pursued, there should be graphs of distributions of the risk 

results for the individual scenarios as well as the risk avoided. 

Response 15 

The graphs presented for Salmonella and cancer were included for 

illustration of the annual variability in cases. As a default we assumed our 

predictions (which were based on probabilistic models) were rates from a 

Poisson process. This assumption was explained in the report. Providing 

graphs for all outcomes was considered, but we decided that two 

illustrative graphs would be sufficient. If the comment refers to 

uncertainty, then we provide the range of uncertainty in Table 11. 

The revised risk assessment is focused on Salmonella, and the results are 

presented in greater detail. 

Comment 16 

2) Does it completely cover all aspects of the analyses? 

Almost all aspects of the analyses should explained in more detail, as discussed in the 

responses below. 

3) Does it follow a logical structure and layout? Yes. 

4) Where data are included/excluded, does the documentation provide adequate 

justification? No. See response under hazard identification. 

5)	 If the answer to any of the above questions is No, the reviewer should provide 

an alternate outline and/or approach for documenting adequately and clearly 

this risk assessment. See subsequent comments. 

The major assumptions of the model are as follows; please comment according to charge 

#2 below. 

A.	 Input distribution for each hazard in the analysis. 

Hazard concentration in fish: The Pareto distribution is used for the hazard concentration. 

A log-normal distribution is used for this distribution for most environmental datasets 

with which I am familiar. There should be some discussion of why a log normal 

distribution was not utilized and the difference in outcome had it been used instead of the 

Pareto distribution (sensitivity analysis). There is no discussion of the uncertainty of the 

choice of distribution anywhere in the report. Moreover, the report does not provide the 

underlying data of the hazard concentrations in fish. The data for the inorganic (metal) 

hazards appear to come from Santerre et al. 2001 which reports the mean, maximum and 

standard deviation of the detects along with the limit of detection and the number and 

16
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

percent of detects. Normal and lognormal distributions could be determined from these 

data. It is not the case, as stated in the text, that the Pareto distribution optimally 

described the dataset as it appears that the authors of this report did not use the original 

dataset, only the summary of the dataset provided in the cited published paper. 

Response 16 

The justification for the Pareto distribution was informal; it was a 

conservative choice (because it places more probability in the right-hand 

tail than a lognormal distribution) and was straightforward to adapt to 

expert opinion about the true distribution of contamination among catfish. 

Given the variable (and sometimes very limited) data available, a heuristic 

approach was considered simpler to apply across multiple hazards versus 

a formal analytic approach applied to each hazard. 

The revised risk assessment is focused on Salmonella. Its descriptions of 

inputs are more detailed than the original risk assessment. 

Comment 17 

A. Consumption distribution in the US population 

I think the methodology used to determine catfish consumption (frequency, meal size, 

number of consumers) is weak, or at a minimum needs far more clarification. The 

authors seemed to equate catfish production data with catfish consumption data. 

However, there is obviously some factor of wastage, which is likely well known in the 

industry. In addition, it was not specified whether catfish production was whole fish or 

fillets. Since catfish are sold in many forms including whole eviscerated fish, the 

production data is likely more than the weight of fillets. 

Response 17 

We agree with the comment. The updated risk assessment uses data that 

better represent the amount of catfish consumed in the United States each 

year [U.S. Centers for Disease Control and Prevention (CDC). National 

Health and Nutrition Examination Survey, retrieved from 

http://www.cdc.gov/nchs/nhanes/nhanes_questionnaires.htm; and U.S. 

Department of Agriculture, National Agricultural Statistics Service (NASS), 

2009, Catfish Processing. Retrieved from 

http://usda.mannlib.cornell.edu/usda/nass/CatfProc//2000s/2009/CatfProc02-18-2009.pdf]. One notable change in the consumption calculations is 

the use of the processed weight of catfish instead of the catfish weight 

prior to processing. 

Comment 18 

The use of NHANES reports of catfish consumption is more likely an underestimate than 

an overestimate of the number of catfish consumers. Many folks who eat not-self-caught 

17
 
 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

fish have no idea what kind of fish they are consuming. This is particularly true for fried 

fish when they are not the preparers. Who would know if the fish and chips you ordered 

at a restaurant was catfish? 

Response 18 

Queries were made to university, industry, and government organizations 

regarding sources of catfish consumption data. Through this process, the 

1996-2006 NHANES data were identified to be the most extensive catfish 

consumption data. Estimates of total U.S. catfish consumption were 

based on National Marine Fisheries Service and National Agricultural 

Statistics Service data. 

Comment 19 

It is not clear to me why the authors did not include catfish consumption estimates from 

the USDA Continuing Survey of Food Intake by Individuals (CSFII). For example, the 

EPA Exposure Factors Handbook (1997) cites CSFII 1989-1991 for mean per capita 

intake of catfish, as consumed, of 1 g/person/day and uncooked of 1.38 g/person/day. 

The mean and distribution of values for catfish consumers is the most relevant data, and 

perhaps could be determined from this dataset. Subsequent EPA analyses have total 

finfish values that can serve as upper bounds of estimates, although farm raised finfish 

seem to be included in the survey questions (c.f., EPA 2007, 

http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=132173) and would be a better 

upper bound value. Yet another option for determining catfish consumption would be the 

use of industry data. There should also be a discussion as to whether or not farmed catfish 

consumption is increasing or decreasing (I believe it is increasing), and make some 

allowance for this in the assessment. 

Response 19 

We are familiar with the Continuing Survey of Food Intakes by Individuals 

(CSFII) consumption surveys and have used them in many past 

quantitative risk assessments as part of our exposure assessment. Since 

2002, the CSFII and NHANES consumption data have been integrated 

into a more current national consumption survey. The 1996-2006 

NHANES data sets currently used in this risk assessment represents the 

integrated data of the two consumption surveys. 

Comment 20 

1. Distribution of catfish consumption frequency of each age-sex category. 

I don’t understand why there are so many age/sex categories. The consumption data 

aren’t good enough to support this many categories, or at least it is not documented in the 

report. Calculations based on these estimates are only as reliable as the least reliable data, 

which is the great uncertainty of the consumption data. 

18
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 20 

All age/sex categories contain approximately the same number of survey 

respondents. Segregating the consumption data into these groups was 

aimed at minimizing the probability of neglecting consumers whose 

consumption patterns deviate significantly from the mean. 

Comment 21 

1. Distribution of the amount of catfish consumed in each age-sex category. 

This is not well explained and documented. 

2. Comment on the validity of the model. 

a. Are these assumptions logical and adequately justified? 

The model assumes that only catfish fillets, or meat derived from the fillet, are consumed 

by the US population. I don’t believe this is correct based on the products described in 

this report: http://srac.tamu.edu/tmppdfs/453908­

184fs.pdf?CFID=453908&CFTOKEN=47605949&jsessionid=90305fea38ece0c3b0102c 

642a6b51653753 

Consumption of fish parts other than fillet (parts that have a higher fat content) could lead 

to higher concentrations of organic contaminants, such as PCBs and DDT, although it 

may not make a difference to the inorganic hazards. The fillet-only assumption could 

lead to a substantial underestimate of exposure for certain subpopulations, as there are 

ethnic preferences to parts of fish consumed and preparation methods. For wild caught 

catfish, Weintraub and Birnbaum (2008) suggest that parts of fish consumed partially 

accounts for the higher PCB body burdens of non-Hispanic Blacks. 

http://digitalcommons.unl.edu/cgi/viewcontent.cgi?article=1021&context=usepapapers 

Response 21 

The great majority of catfish residue data were generated from the 

analysis of the fillet tissues. To estimate risk associated with consumption 

of other catfish tissues would necessitate the acquisition of additional 

residue data for these tissues, which was unavailable at the time that this 

risk assessment was conducted. 

Wild caught catfish falls outside of the scope of this risk assessment and 

was therefore not considered. As discussed elsewhere, the chemical 

residue analysis is no longer a focus of the revised risk assessment. 

Comment 22 

Equation 3: 

The reason for the inclusion of the term I(h,y), the indicator of whether or not a catfish 

fillet is contaminated by a particular hazard, is not clear to me. If the concentration 

distribution X(h,y) includes the non-detected values, then the I(h,y) term should not be 

19
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

necessary. Its inclusion should lead to an underestimate of predicted risk. If the I(h,y) 

term is needed because of the choice of a Pareto distribution means that only detected 

concentrations are included in the X term, then that should be stated. However, as noted 

previously, there should be a sensitivity analysis showing the difference in outcome of 

this input distribution/model vs. the more commonly used log-normal distribution for the 

concentration term. 

The cooking term Ci should include a term for cooking type. Additional preparation 

methods should be included, including soups which have no loss of chemical 

contaminants (hazard) as a result of preparation. 

Response 22 

The revised risk assessment focuses solely on Salmonella. That pathogen 

does not rely on a Pareto distribution assumption. 

Comment 23 

Toxicity evaluation 

I am extremely uncomfortable with the approach used to determine toxicity throughout 

this report. There is an attempt to incorporate probabilistic approaches to the assessing 

toxicity. However, these approaches are not approved by the USEPA nor are they widely 

used or agreed upon in the scientific community. There is not nearly enough detail 

provided in this assessment to pioneer new methods of toxicity evaluation and I have no 

confidence in the reliability of the outcome. 

Response 23 

Probabilistic risk assessment approaches are widely used throughout the 

international risk assessment community, especially in the area of food 

safety. The approaches used in this assessment received favorable 

response from reviewers in EPA’s office of pesticide programs. 

A search of Monte Carlo on the EPA website generated many references 

to EPA’s support of this technique. For example: 

A Monte-Carlo assessment provides a probabilistic or statistical assessment of dietary 

risk, using more refined information than is used in the Dietary Risk Evaluation System 

calculation. With the advent of Monte-Carlo and probabilistic techniques, it becomes 

possible to estimate more accurately the complete distribution of exposures to the entire 

population. 

Rather than the crude "high end,"single- point estimates, Monte-Carlo provides better, 

more accurate information on the range and probability of possible exposure and their 

associated risk values. (http://www.epa.gov/oppfead1/trac/trac3.htm) 

The intended goal of the two draft white papers, Using Probabilistic 

Methods to Enhance the Role of Risk Analysis in Decision-Making, With 

Case Study Examples, and Using Probabilistic Methods to Enhance the 

Role of Risk Analysis in Decision Making - Managers' Summary is to 

describe potential and actual uses of probabilistic tools in the risk decision 

process, and to encourage their further implementation in human, 

20
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

ecological and environmental risk analysis and related decision making at 

EPA. The enhanced use of probabilistic analyses to characterize 

uncertainty in assessments would not only reflect external scientific advice 

on how to further advance EPA risk assessment science, but will also help 

to address specific challenges faced by managers and improve 

confidence in Agency decisions 

(http://www.epa.gov/osa/raf/prawhitepaper/index.htm). 

Further information on EPA’s endorsement of probabilistic approaches for 

risk assessment can be viewed at: 

Guiding Principles for Monte Carlo Analysis 

(http://www.epa.gov/NCEA/pdfs/montcarl.pdf)
 

Using Probabilistic Methods to Enhance the Role of Risk Analysis in 

Decision Making with Case Study Examples 

(http://www.epa.gov/osa/raf/prawhitepaper/pdf/pra_wp_draft_0609.pdf) 

Comment 24 

For the cancer risk assessment, the model equations/values in the spreadsheet did not 

match the derivation of the EPA toxicity factor. The cancer slope factors developed by 

EPA are calculated based on an assumed 70 year lifetime. Since this fixed value is 

incorporated into the value of slope factor, it is not appropriate to use probabilistic 

methods to determine a lifetime, which on average was 82 (according to the spreadsheet). 

This has the impact of reducing the cancer risk by a factor of 70/82 based only on 

incompatible uses of point and probabilistic methods. 

Response 24 

The reviewer’s comment is logical, however, the revised risk assessment 

focuses only on the potential effects of Salmonella contamination. 

Comment 25 

For PCBs, a slope factor of 2 (mg/kg-d)-1 is most often used in risk assessments 

involving food chain consumption. A value of 1 (mg/kg-d)-1 was used in this 

assessment, with no discussion of how it would change the result. The text does not give 

any basis for assessing this independently as it doesn’t give the cancer risk by chemical 

hazard, only aggregated numbers. 

Response 25 

The slope used in this risk assessment was obtained from the USEPA 

IRIS website. The use of a slope of 2 (as suggested by the reviewer) 

would lead to a reduced estimate of PCB associated carcinogenicity. 

Since the use of a slope value of 1 did not lead to a significant estimate of 

21
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

PCB related cancers, using a slope of 2 would likely lead to the same 

result. 

Note: the revised risk assessment focuses only on potential adverse 

effects associated with Salmonella. 

Comment 26 

For the non-cancer probability, the authors base their approach on a paper published over 

a decade ago by Price et al. The highly reputable authors of this paper provide some 

examples of a proposed methodology. However, it is controversial among at least some 

regulators, and it has not been used in the EPA regulatory community. Moreover, the 

parameters within the paper, for example the form of the uncertainty factors and the 

parameters used to describe the distributions, have not been validated. Indeed, the work 

of Dale Hattis and his colleagues on the amount of variability among humans for 

individual endpoints should play a huge role in any development of uncertainty factor 

distribution used for regulatory and other purposes. This variability is far greater than that 

used in this modeling exercise. None of this uncertainty was even mentioned in the 

document under review. 

Response 26 

FSIS will revisit this issue should it further consider risk estimates for noncancer endpoints for chemical hazards. The revised risk assessment 

focuses on Salmonella, and the effects of chemical compounds are no 

longer considered. 

The revised catfish risk assessment uses the World Health Organization 

Salmonella dose-response model. 

Comment 27 

The method for the estimation of the blood concentration of mercury appears to be 

deterministic, based on the text. If it was probabilistic, the range of values and the 

distribution utilized should be stated in the text. If it was deterministic, then there should 

be some kind of bounding estimates as there will be substantial variability in all of the 

parameters. 

For lead, the authors should consider using the EPA lead model to estimate the additional 

lead burden in adults and children. This model incorporates background and 

pharmacokinetic data and will give a more reliable estimate of blood lead concentrations. 

However, before it is used, the exposure parameters need to be reassessed including meal 

size and cooking loss (see below). 

Response 27 

FSIS will revisit this issue should it further consider risk estimates for noncancer endpoints for chemical hazards. The revised risk assessment 

focuses on Salmonella, and the effects of chemical compounds are no 

longer considered.. 

22
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 28 

b. Is the selection of data appropriate? 

Use of animal LD50s 

Acute toxicity mortality evaluations were based on animal LD50 studies with the 

application of uncertainty factors. However, there are very good human data on the acute 

toxicity for several of the identified chemical hazards, and these should be used instead of 

animal LD50s. For example, the acute toxicity of chlorpyrifos (an organophosphate) has 

been reviewed as part of the EPA reregistration process. A review panel summary of 

acute effects can be found at 

http://docserver.ingentaconnect.com/deliver/connect/tandf/10937404/v2n4/s2.pdf?expires 

=1246894648&id=51079748&titleid=333&accname=Guest+User&checksum=02D0155 

4EF2788963EDC7682ADEEED7A 

Response 28 

We agree that human data are preferable to animal data as inputs for 

human risk assessments. However, the revised risk assessment focuses 

on Salmonella and does not use animal LD50s. 

Comment 29 

Meal size: 

I believe the distribution used to estimate meal size is too narrow. The triangle 

distribution used (for Salmonella) is limited to a 7-11 oz fillet, referenced to a particular 

website. However, other websites show the availability of fillets sizes of 2-3 oz, 3-5 oz, 

5-7 oz etc. The recipes on the Epicurious website call for 5-7 oz fillets. A USDA 

publication described 3.5 oz servings, http://srac.tamu.edu/tmppdfs/453908­

501fs.pdf?CFID=453908&CFTOKEN=47605949&jsessionid=90305fea38ece0c3b0102c 

642a6b5165375. The 3.5 oz value seems to be the mean serving size used for the catfish 

consumption in section 4.3 for calculating the number of meals/year for a catfish 

consumer. The inconsistency in meal size should be resolved or at least explained. The 

methodology that underlies the determination of meal size needs to be more clearly 

spelled out. 

An alternate method for meal size is to look at the many surveys and the industry 

literature that describe meal size. In general, the meal sizes reflect uncooked fish. I don’t 

understand the rationale for reducing meal size because of breading. People won’t 

choose smaller fillets. It seems to be an arbitrary correction. 

Response 29 

This has been corrected in the revised risk assessment. An excerpt from 

the revised report, 

The mean serving size determined from the 8-year dataset analysis 

was 122.28 grams per eating occasion. Given the low frequency of 

catfish consumption, this analysis assumed the quantity consumed 

in one day represented a single catfish serving. The serving size 

23
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

random variable ranges from 5 grams to over 500 grams (1st and 

99th percentiles)(Figure 5). . This random variable is modeled as an 

empiric distribution because attempts to fit the data to parametric 

distributions did not demonstrate adequate goodness of fit. 

The estimates for fraction baked and fraction breaded were taken 

from both the one and two day datasets as independent estimates 

using the SUDAAN CROSSTAB procedure and averaged. Six 

catfish food codes were used to ascertain the fraction baked 

(versus fried) and the fraction breaded as proportions of the 

weighted US population catfish consumer estimates. The 

0.79 Bread f

and 

0.24 Baked f

population-adjusted estimates were .

Breading is assumed to represent between 20% and 30% of total 

serving weight. Therefore, serving size for breaded servings is 

) to adjust for the amount of catfish 

in such servings. 

Comment 30 

Cooking method and Cooking loss (chemical):
 

The model only allows for baking and frying. However, there are recipes for other
 

preparations, including use in soups (SRAC publication above) for which there would be 

zero cooking loss. This should be included in the model.
 

Response 30 

Frying and baking constituted virtually all (>99%) of the survey 

respondents and were therefore the methods considered in this risk 

assessment 

Comment 31 

The sources of the cooking loss values and the rationale underlying the use of a normal 

distribution (rather than, for example, a log-normal distribution or a triangle distribution) 

are not clear to me, nor are they described in the document. I looked at the underlying 

reference for cooking loss from arsenic, cadmium, mercury and lead. They reported the 

results in raw and cooked, both wet weight, thus any cooking loss (gain) could be just due 

to moisture differences. In addition, the text clearly stated that none of the losses were 

statistically significant. Thus, I believe it is inappropriate to incorporate any cooking loss 

for metals. The cooking loss for lead needs particularly careful attention as the 

sensitivity analysis indicated that the final output for IQ was quite sensitive to cooking 

loss. Presumably this would also be the case for the cardiovascular effects that are only 

based on lead, although those results are not presented in the report. 

Response 31 

Since serving size estimates are based on consumption of cooked 

products, cooking-associated changes in hazard concentrations (even if 

multiplied by a randomly selected value between 0.7 and 0.8 (i.e., 

Bread effect Uniform _ (0.7,0.8) 24
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

such changes are associated with cooking-induced moisture changes) 

should be incorporated into the model. 

Sensitivity analysis suggests moderate elasticity for IQ effect (arsenic, 

cadmium, lead, mercury), none for CV effects, and extremely inelastic 

effects for cancer (DDT, PCB, and metals). Therefore, for most catfish 

associated chemical hazards, the cooking losses appear to be minimal. 

Additionally, the revised risk assessment focuses only on the public health 

effects from Salmonella contamination. 

Comment 32 

The referenced cooking-loss data for PCBs (Bayen 2005) came from experiments using 

salmon. This paper also reported on DDTs. Salmon is a fatty fish while catfish are not. 

One would expect greater PCB cooking losses from salmon than from catfish as the loss 

is associated with the loss of fat (this is stated in the Bayen 2005 article as well as many 

others). More details regarding cooking methods and cooking losses should be addressed 

in both the text and in an uncertainty analysis. Cooking loss for PCBs for both point and 

Monte Carlo analyses were described in detail in the EPA Risk Assessment for the 

Housatonic River, Rest of River, fish consumption: 

http://www.epa.gov/NE/ge/thesite/restofriver/reports/hhra_219190/219190_HHRA_Vol4 

_FW.pdf 

This report also takes into account additional cooking methods and describes its method 

to combine estimates for various cooking methods into exposure point concentrations (as 

consumed) for the risk characterization. Some of the data and methodology used in this 

report might be helpful to the authors of the catfish assessment. 

Response 32 

Catfish-associated cooking effects data were used whenever such data 

were available. In the absence of such data, data from other types of fish 

were used as model inputs. 

Comment 33 

The range of cooking loss for nitrofurans is overstated based on the paper cited (Cooper 

& Kennedy 2007). The cooking loss data is based on pig muscle and liver. The data on 

pig muscle is most appropriate to use as the fat content is likely more closely that of 

catfish flesh. As expected, there is greater loss from cooking liver with its higher fat 

content. 

Response 33 

The revised risk assessment focuses on Salmonella effects. 

Comment 34 

Cd bioavailability/absorption from fish: 

25
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Based on my reading of the Cd absorption literature (c.f., ATSDR), the use of 13% 

bioavailability is several-fold too high. ATSDR estimated 3-6% absorption for 

individuals with adequate iron stores. At a minimum, this should be a distribution, 

perhaps triangular with a mean of 4-5%. However, it is not clear to me why cadmium is 

included as a hazard as it had a lower screening score than others, was present in 2% of 

the samples, and likely doesn’t contribute to the risk. 

Response 34 

The revised risk assessment focuses only on Salmonella effects. 

Comment 35 

c. Are model outputs reasonable? 

The points addressed previously lead to both overestimates and underestimates of the 

underlying risk, although I think it is more likely that the risks are underestimated than 

overestimated. In my opinion, the absolute risk values are unreliable including the 

number of adverse consequences avoided, although the relative risks (outcomes as 

percent differences between baseline and FSIS inspection) may be ok. 

Response 35 

FSIS acknowledges that there is uncertainty associated with the risk 

assessment’s best estimates of adverse effects. This uncertainty has been 

addressed in the uncertainty analyses. 

Comment 36 

d. Is the evaluation of uncertainties sound? 

The evaluation of uncertainty is extremely limited. In terms of parameter uncertainty, it 

focuses only on the single datasets selected for use, not all the data for the parameters 

available. It does not include any model uncertainty, including the uncertainty associated 

with the selection of the functional form with which to model the data. It does not deal 

with any of the toxicological uncertainty, and the limited amount of toxicity data 

incorporated. It does not deal in a serious way with interactive effects of these 

compounds although data are available in this regard. 

Response 36 

Given the limited data available for most hazards, it seems that 

uncertainty analysis beyond (i.e., more detailed than) that considered in 

the original report is unnecessary. We have established both very 

conservative and very non-conservative input values to the model and 

generated extreme predictions on either side of our default predictions. 

Model uncertainty (i.e., different choices for concentration distributions 

based on the same limited data) would generate much more limited 

ranges in our predictions relative to the results we obtained. 

26
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 37 

The uncertainty (bounding) analysis is based on the model inputs found to be most 

influential in the sensitivity analysis. Meal size is not included in this, nor does it appear 

to be part of the sensitivity analysis. This omission is surprising to me as meal size should 

be proportional to dose, and thus more influential than, say cooking loss (for chemical 

hazards). 

Response 37 

The inputs for size of individual meals were based on much better data 

than many other model inputs. The dose consumed is the product of initial 

contamination, serving size and any effects of growth/cooking. It was 

assumed that our uncertainty about the random variables for initial 

contamination and growth/cooking effect was greater than our uncertainty 

about the random variable for serving size. 

The revised risk assessment focuses on Salmonella. Serving sizes were 

updated based on the most current data available and the output of the 

revised model. Sensitivity analysis with respect to size of serving in that 

revised assessment. 

Comment 38 

5.	 Comment on the risk assessment model. As a general matter, comment on the 

hazards identified in the assessment? Have we failed to identify important 

hazards? In addition: 

a.	 Chemical Hazards 

i.	 Are the methods utilized to identify the most significant potential 

hazards associated with consumption of catfish valid? 

I am not familiar with the equations used for priority ranking score and the underlying 

rationale, and there is insufficient information presented to make any judgment about 

them. Perhaps these could be explained in more depth in an appendix. In general, the 

factors included seem appropriate. However, I don’t understand the rationale for the 

selection of the high priority hazards based on the toxicity scores. Why is the high 

priority cutoff between 10 and 13 for the organic compounds (organophosphates in, 

diuron out) while it goes down to 4.6 (cadmium in) for the inorganic compounds? 

Response 38 

This approach was used to include a variety of potential chemical hazards 

in the risk assessment. If one class of compounds appeared to be 

associated with significant risk to catfish consumers, we would likely then 

evaluate other compounds with similar chemical properties and use 

patterns. However, the revised risk assessment focuses only on 

Salmonella, and the effects of chemical compounds are no longer 

considered. 

27
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 39 

i.	 Are the approaches utilized to estimate exposure and level of 

concern resulting from the consumption of catfish valid? 

Exposure evaluation for acute effects: The only exposure outcome evaluated is the 

annual per serving exposures. Furthermore, there is no allowance for more than one 

catfish meal/day. This exposure evaluation cannot be used to calculate acute effects; 

those where the exposure period is a day to a week. I am sure there are cases where 

individuals consume more than one catfish meal in a 24 hour period or have more than 

one serving per meal. People eat leftovers or go to fish fries. These are the conditions 

under which to evaluate whether or not acute effects will occur. 

There is a mistake on p. 44 in describing the conversions to match input for the dose 

response function: mg/Kgbw)33 is not meaningful. 

Response 39 

Analysis of the 1996-2006 NHANES consumption data suggest that 

consumption of multiple catfish meals in a single day is rare. 

The metric in question refers to mg of the hazard per kilogram bodyweight 

of the catfish consumers. However, the revised risk assessment focuses 

on the microbial hazard Salmonella where the bodyweight of consumers 

does not enter the calculations. 

Comment 40 

i.	 Are the approaches utilized to estimate the prevalence of 

adverse effects (acute and chronic) resulting from hazard 

exposure valid? 

The selection of non-cancer hazard endpoints seems arbitrary. While it is reasonable to 

select various cardiovascular endpoints, the intense scrutiny of them and not many other 

adverse outcomes seems disproportionate. The chemicals included in the risk assessment 

are associated with many other hazards, including kidney disease, immune system 

dysfunction, and nervous system effects (other than IQ). PCBs and DDTs are endocrine 

disruptors associated with neurodevelopmental and reproductive effects; these effects can 

be observed at low doses associated with environmental (food consumption) exposure. 

Response 40 

As noted by the reviewer, the chemicals included in this risk assessment 

may be associated with a variety of adverse effects. As the purpose of this 

risk assessment is to estimate the magnitude of adverse effects 

associated with catfish consumption, we limited our estimates to 

chemicals and adverse effects for which dose vs. response functions were 

28
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

available or could be derived. However, the revised risk assessment 

focuses on Salmonella, and the effects of chemical compounds are no 

longer considered. 

Comment 41 

There should be some discussion of why a Poisson (normal) distribution was used to 

model adverse effects. My experience with environmental variables and toxicity data is 

they are more likely to be log-normally distributed (hence the use of log-dose with the 

probit function when determining acute effects such as LD50). 

Response 41 

It is not clear what this comment refers to. If it is the choice of Poisson for 

modeling variability (randomness) of annual total illnesses, then our 

rationale is that we chose this distribution for simplicity. Such an 

assumption (that annual human illness counts are described by a Poisson 

distribution) is a reasonable default when it is not possible to empirically 

estimate variability. The model predicts an (average) annual rate of human 

illnesses. Because the Poisson has a variance equal to its mean, we can 

use our prediction of the model as the mean and variance of annual 

illnesses. 

The lognormal variability referred to in this comment is more appropriate 

when describing the variability in exposures across servings. This 

variability is integrated through a dose-response function so that a single 

probability of illness (given exposure) results (e.g., Equation 11 in the 

original risk assessment). In other words, the lognormal (or otherwise 

skewed) exposure distribution does not translate to the variability in total 

illnesses predicted. 

Comment 42 

a.	 Microbial Hazard 

ii.	 Are the methods utilized to identify the most significant potential 

hazards associated with consumption of catfish valid? 

Food safety/microbial hazards are not an area of expertise for me. I could not determine, 

from the information provided in the report and Appendix II, how the determination was 

made to include Salmonella in the risk assessment, but not the intermediate priority 

microorganisms such as Listeria monocytogeneses, C botulinum and the fecal bacteria. It 

seemed the criterion was an epidemiologically confirmed outbreak (for Salmonella). 

However, Listeria and others have been detected in catfish, including processed fillets 

and E. coli strikes me as a plausible and hazardous contaminant. 

Response 42 

We have addressed concerns about Listeria monocytogenes in our 

response to Comment 10. The same reasoning holds for C. botulinum and 

29
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

fecal bacteria, which are also discussed in the paper cited in the response 

to comment 10. 

Comment 43 

i.	 Are the approaches utilized to estimate exposure and doseresponse resulting from the consumption of catfish valid? 

As stated earlier, I think the data used for serving size was too limited, and this is linearly 

related to Salmonella exposure. I am not familiar with the dose-response function, but it 

appears to be widely used and accepted. 

Response 43 

The 1996-2006 NHANES consumption data constitute the most complete 

catfish consumption data identified by the Agency. 

Comment 44 

i.	 Are the approaches utilized to estimate the prevalence of 

adverse effects (acute and chronic) resulting from hazard 

exposure valid? 

The presentation of the health effects in the various scenarios is confusing – a small table 

summarizing the effects would help. I understand from the numbers about the reduction 

in hospitalizations with the FSIS inspection, but I cannot reproduce the estimates of 

deaths. There may be a calculation error or a typo regarding the predicted number of 

deaths. 

Response 44 

The revised risk assessment will only predict total salmonellosis cases 

(and cases avoided). Outcomes of cases will be addressed in a benefitcost analysis. 

30
 
6 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

REVIEWER NUMBER 3 

Comment 45 

1. Model documentation: 

a. Is the report clearly written? 

The report is generally superbly written, with one major exception, and some minor 

exceptions, as detailed below in (e). Particular strengths include very clear text and 

excellent use of tables. 

b. Does it completely cover all aspects of the analysis? 

Yes, with one major exception, as detailed below in (e). 

c. Does it follow a logical structure and layout? 

Yes, extremely so - my congratulations to the authors on this aspect. 

d. Where data are included/excluded, does the documentation provide adequate 

justification? 

Yes, documentation is generally appropriate with the exception of the description of the 

potential CVD effects of lead exposure (p. 49-50). First, the authors should clarify 

whether gender-specific RR’s were utilized, as were presented in the Fewtrell report. 

Second, and more importantly, the authors should specify that these RR’s are based on 

largely cross-sectional observational associations between blood lead levels and blood 

pressure levels, and then secondary extrapolation of these blood pressure differences to 

prediction of future CVD endpoints. It should also be specified that a relationship 

between lead exposure and actual CVD endpoints has not yet been established in 

prospective studies. 

Finally, the counterfactual must be clearly specified, both here and elsewhere (see (e) 

below): these estimates are not for the effect on CVD risk of current catfish 

consumption, including current lead contamination, compared to no consumption, but 

rather for the effect of current catfish consumption, including current lead contamination, 

compared with equivalent catfish consumption that is wholly lead-free. The reason for 

this specification should also be detailed: i.e., that because catfish consumption also 

provides beneficial nutrients, particularly omega-3 fatty acids, that will reduce CVD risk 

(and also lower blood pressure), these estimates should not be construed to be the effects 

of current catfish consumption per se, which is likely reducing overall CVD risk in the 

US, even given the current lead content, compared to no catfish consumption at all. 

31
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 45 

While we can appreciate the points raised in this comment as pertinent to 

an analysis of consumer behavior and public policy with respect to 

encouraging/discouraging catfish consumption, the counterfactual benefits 

of catfish consumption were not relevant to our goal of estimating the 

public health benefits of a rule concerning catfish inspection. 

Comment 46 

e.	 Alternate outline and/or approach for documenting adequately and 

clearly this risk assessment. 

The model is clearly written and presented, with one major, and extremely important, 

exception: 

**Throughout the document, the authors state that the baseline model estimates the 

adverse health effects of current catfish consumption based on current contaminant 

levels, describing for example, the “human health impact due to catfish consumption” (p. 

15), “health effects associated with respect to the consumption of contaminated catfish (p. 

15), “probability of an adverse effect associated with the consumption of one catfish 

meal” (p. 19), “estimation of the mean probability of each adverse public health effect per 

catfish meal” (p. 20), “hazards due to catfish consumption” (Figure 2); “the annual rate of 

adverse effect r in the US population” (p. 52), “estimates of catfish-consumptionassociated hypertensive disease” (p. 58), “catfish-consumption-associated adverse 

effects” (p. 59), results of Tables AIV.3, AIV.4, and AIV.7, etc., etc. 

However, for two of the major outcomes assessed (IQ, CVD), the accuracy of the 

baseline and post-FSIS estimates are strongly dependent on the choice of the 

counterfactual, that is not clearly specified in the document. For both IQ and CVD, the 

models do not estimate the adverse health effects of current catfish consumption, based 

on current contaminant levels, compared to no such consumption of catfish. Rather, the 

models estimate the adverse health effects of current consumption of catfish, based on 

current contaminant levels, compared to the same level of consumption of catfish 

containing zero contaminants. 

Response 46 

Our response to Comment 45 partly addresses this comment. Since only 

the predicted number of annual salmonellosis cases prevented is used in 

the benefit-cost analysis, the revised catfish risk assessment focuses 

solely on the adverse public health outcome associated with the 

consumption of Salmonella-contaminated catfish. 

Comment 47 

These counterfactuals are very different. For the former, the model must consider the 

entire effect of the catfish meal, that would include the hazards but also the beneficial 

32
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

nutrients in catfish. For the latter, only the hazards may be considered (given the 

assumption that the beneficial nutrients are unchanged by removal of the hazards). As 

this risk assessment considers only the hazards, clearly the latter counterfactual is the one 

appropriate to this document – this should be specified and discussed in detail. 

Unfortunately, the authors do not specify the counterfactual, so the accuracy (and 

interpretation) of the estimates can be brought into question. Indeed, based on most of 

the language in the document in which the authors describe what they are estimating 

(e.g., examples above**), the document strongly implies that they are quantifying the 

former estimation (i.e., the effect of currently contaminated catfish consumption 

compared to no such consumption). Notably, this interpretation of the counterfactual 

would render both the IQ and CVD estimates markedly incorrect. 

Catfish contain omega-3 fatty acids that will both improve IQ and reduce CVD risk. 

Thus, to estimate the true “human health impact due to catfish consumption”, or 

“estimation of the mean probability of each adverse public health effect per catfish meal”, 

or any of the other descriptors above, for both IQ and CVD outcomes, both the risks and 

the benefits must be considered if the counterfactual were the effect of a current catfish 

meal compared to no such consumption. To perform this estimation correctly, this would 

require modeling the benefits of the average omega-3 contents of catfish on both IQ and 

CVD risk, and then summing of the risks from contaminants and benefits from omega-3 

fatty acids. Valid data, of much better quality than that used to model the risks, are 

available to perform such assessments of benefits. If this were done, it is highly likely 

that the overall estimated effects of current baseline catfish consumption, including 

current contaminant levels, on both IQ and CVD in the US will be beneficial, not 

harmful. 

Of course, these absolute benefits of catfish consumption would increase even further if 

fewer contaminants were present in the catfish post-FSIS, in that the benefits of the 

omega-3 fatty acids on IQ and CVD would be even less offset by the harms of the 

contaminants. Thus, whereas the overall absolute estimates for IQ and CVD effects of 

both baseline and post-FSIS catfish consumption, compared to no consumption, would be 

different if benefits of omega-3 fatty acids were modeled appropriately, the net estimated 

change (“delta”) comparing baseline vs. post-FSIS would not be different. This is 

because the overall consumption of omega-3 fatty acids should be the same in the 

baseline vs. post-FSIS models, so that the corrections to both models for the benefits 

would offset each other in the calculation of the difference. 

Because the main intent of this document appears to be the estimation of the relative 

effect of the FSIS program, rather than of the absolute effects of current baseline or postFSIS catfish consumption per se, it would be much simpler to retain the current estimates 

and simply specify the counterfactual explicitly throughout the document. That is, the 

appropriate description of both the baseline and post-FSIS models is the effect of catfish 

consumption on outcomes, compared to the same level of consumption of wholly 

contaminant-free catfish. 

33
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

If this counterfactual were clearly specified throughout the document, each time any 

estimates of the “health effects” of a catfish meal were mentioned, then the document 

would be clear and correct. These issues should also be clearly described in the 

Introduction, discussing the benefits of omega-3 fatty acids on IQ and CVD and 

explaining why the baseline and post-FSIS assessment cannot be construed as estimates 

of the health effects of catfish consumption per se, but rather of the health effects of 

catfish consumption with baseline or post-FSIS contaminant levels compared to the same 

level of consumption of catfish with zero contaminants. The Introduction and 

Conclusions should also specify that the actual health effect on IQ and CVD of current 

catfish consumption per se, compared with no consumption, is likely beneficial, and that 

the post-FSIS model approximates how much more this benefit would increase. 

For clarify, it can also be noted that for the Salmonella and cancer outcomes, the choice 

between these two counterfactuals (no consumption vs. equivalent consumption of 

wholly contaminant-free catfish) will not have a major effect on the baseline or post-FSIS 

estimates, as there is little evidence that omega-3 fatty acids (or other non-contaminant 

beneficial factors) in catfish have any appreciable benefits on risk of Salmonella or 

cancer. 

Response 47 

The estimates resulting from this risk assessment do not represent the 

total health effects associated with catfish consumption. The objective of 

this risk assessment is to estimate the impact of an FSIS inspection 

program on adverse effects resulting from the ingestion of hazards that 

are potentially associated with catfish. The risk assessment determines 

the change (increase) in adverse effects resulting from hazards that are 

co-ingested with catfish. These effects are relative to the level of these 

adverse effects that are experienced in a population which consumes 

catfish containing no residues of the hazard of interest, not to the 

population that consumes no catfish. The beneficial effects associated 

with the consumption of nutritious compounds in catfish contribute to the 

background levels for such effects. Adverse effects resulting from the 

ingestion of hazards are assumed to be independent of potentially 

beneficial substances that are natural constituents of catfish. The risk 

assessment assumes that implementation of an FSIS regulatory program 

will not impact the amount of catfish consumed in the US. Therefore, the 

background levels of these adverse effects and health effects are 

constant. 

Comment 48 

Other points for improving clarity: 

Table 9: Impact of each scenario should also be shown per 100,000 persons per year, to 

clarify the size of the effects that are being considered here. This point is briefly made in 

the summary (p. 78), but this will be lost in the full report and many people will focus on 

the Tables alone. Alternatively, the percentages of estimated overall cancer, CVDs, and 

34
 
 

 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

total IQ points that these estimates represent could be added to the Table (see comment 

related to page 59, below). 

Response 48 

This approach will add clarity and will be included in future reports as 

appropriate. 

Comment 49 

Page 57: The statement that it “appears extremely unlikely that catfish-consumptionassociated prenatal mercury exposure would result in an intelligence-deficit-associated 

disease (e.g. mild mental retardation)” is still far too generous to the potential for any 

such effects. It would be more appropriate to state that “On an individual basis, the 

likelihood that catfish-consumption-associated prenatal mercury exposure would result in 

an intelligence-deficit-associated disease (e.g. mild mental retardation) is negligible or 

virtually zero. This is even more likely the case if the likely benefits on IQ from catfishconsumption-associated prenatal omega-3 fatty acid exposure, that were not considered 

herein, were to be taken into account.” Similar corrections are needed for the statement 

on catfish-consumption-associated prenatal lead exposure: “On an individual basis, the 

likelihood that catfish-consumption-associated prenatal lead exposure would result in any 

measurable decrease in an individual’s intelligence or an intelligence-deficit-associated 

disease (e.g. mild mental retardation) is negligible or virtually zero. This is even more 

likely the case if the likely benefits on IQ from catfish-consumption-associated prenatal 

omega-3 fatty acid exposure, that were not considered herein, were to be taken into 

account.” 

Response 49 

The effects of chemical hazards are not considered in the revised risk 

assessment. 

In developing this risk assessment, we estimated the public health impact 

as the difference between the current regulatory situation and an FSIS 

style program. It is likely that the beneficial effects of catfish consumption 

referred to by the reviewer would be constant under both scenarios. As 

such, these beneficial aspects will not affect the public health impacts 

associated with the risk assessment. 

Comment 50 

Page 59: The reporting of the percentage of estimated overall cancer in the U.S. reflected 

by these estimates is a very good use of data to present the findings clearly. Similar 

numbers (i.e., percentage of overall incidences) should be included at the end of the 

previous section for the estimates of hypertensive, cerebrovascular, ischemic heart, and 

other cardiac diseases. All this data (i.e., for both cancers and CVDs) should also be 

added to Table 9, e.g., in a new column or as a footnote. 

Response 50 

35
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

This approach will add clarity and will be included in future reports as 

appropriate. 

Comment 51 

2. Validity of the model: 

A. Input distribution for each hazard in the analysis 

a. Are these assumptions logical and adequately justified? 

It seems a reasonable (although unproven) assumption that the FSIS impact on 

Salmonella prevalence in catfish will be similar to the historical FSIS impact on 

Salmonella prevalence in beef and poultry (page 22). 

However, the assumption that the FSIS impact on each of the other hazards in catfish will 

be proportional to this same metric (the historical FSIS impact on Salmonella prevalence 

in beef and poultry) (page 22) is bizarre, without any presented justifications for this 

seemingly radical assumption. What evidence exists that the impact of FSIS regulation 

on any of the other contaminants will be even remotely similar to this metric? I cannot 

think of any. A wide range of diverse processes influence levels of DDT, PCBs, 

organophosphates, lead, and mercury in foods. PCB levels, for example, are decreasing 

steadily over time in all foods - what evidence is there that the FSIS program would have 

any meaningful impact on this slope of decline? Similarly, what evidence is there to 

support the assumption of proportional (to the stated metric above) declines in lead or 

mercury levels post-FSIS? This assumption seems to be a dramatic stretch, with major 

implications for the results, and must be justified with appropriate evidence or altered. 

Response 51 

We lack empiric evidence about the effectiveness of a proposed program 

for catfish, and must make our case by analogy. If our assumption about 

program effectiveness was optimistic, then our predictions about adverse 

outcomes avoided by the program were also optimistic. We think this 

comment supports our decision to focus the revised risk assessment on 

Salmonella. This revised risk assessment only references FSIS 

experience with reducing Salmonella occurrences among poultry. 

Comment 52 

b. Is the selection of data appropriate? 

Yes. 

c. Are model outputs reasonable? 

Yes. 

d. Is the evaluation of uncertainties sound? 

36
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Generally yes, and the sensitivity analyses are generally excellent and a major strength of 

this report. Two important revisions are needed: 

1. The dose-response functions are the most uncertain input for most of the compounds, 

particularly for each of the compounds that already include uncertainty factors in their 

estimation of effects. Based on this, an increase in slope of 50% is reasonable, but a 

decrease in slope of only 50% is far too small. The slopes of each response should be 

shifted downward by at least as much as the average uncertainty factor (likely several 

fold) used in their estimation. 

2. Sensitivity to changes in the impact of the FSIS regulation (e.g., ranging from 25% to 

double that of the historical effect on Salmonella in beef/poultry) should be added. 

Response 52 

The sensitivity analysis is intended to highlight the model inputs that are 

most influential on the model’s output. The elasticity measure is a 

convenient technique for examining sensitivity. Nevertheless, this 

approach to sensitivity analysis may be limited because it only examines 

local effects (i.e., effects based on small changes to the inputs) rather than 

global effects that measure the influence of large changes to the inputs. 

This comment seems more relevant to a scenario analysis in which 

(multiple) model inputs are adjusted to plausible alternative values and the 

effect on the model output is noted. 

We appreciate the need for additional analysis of the sensitivity of our 

predictions to different assumptions about the effectiveness of FSIS 

inspection. The revised risk assessment examines the effectiveness of 

FSIS inspection in greater detail than was presented in the original risk 

assessment report. 

Comment 53 

B. Distribution of catfish consumption frequency and amount of catfish consumed 

in each age-sex category. 

a. Are these assumptions logical and adequately justified? 

Generally, yes. Two errors are noted. First, whereas 24-hour recall data provide 

unbiased estimates of true population (or subgroup) mean consumption levels, the 

population (or subgroup) distributions of consumption are overestimated due to withinperson error, reflected for example in broader variances, SDs, or percentiles. Thus, the 

variances and distribution percentiles of consumption (page 40) should be corrected for 

this well-known error. This can be accomplished directly in NHANES by evaluating the 

data from the more recent cycles that contain two 24-hr recalls per person. Using this 

data, the within-person vs. between-person variance in consumption can be determined; 

only the latter should contribute to the population (or subgroup) variances. 

37
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

For similar reasons, the statement that the NHANES data “likely represents regular 

catfish consumers and not occasional consumers” (page 41) is incorrect. Each 30-day 

response period is a random sample of each individual’s consumption over the year 

(ignoring seasonal effects, that are difficult to capture and control in NHANES and thus 

can be assume to be random), and thus provides an unbiased estimate of the mean U.S. 

population consumption over 30 days, that therefore includes both regular and occasional 

consumers. As above, the SD or variance of this mean consumption will be 

overestimated, but the mean is unbiased and includes all (regular and occasional) 

consumers. The discrepancy with the production data (page 41) is therefore due to other 

reasons, such as waste, other inaccuracies with the NHANES data, inaccuracies with the 

production data, etc. 

Page 41, last sentence: “The estimated number of U.S. catfish meals consumed per year 

was used for calculating...” Based on the previous sentences, “The estimated number” to 

which this statement refers is unclear. Does this refer to the estimated number from 

NHANES data, or the estimated number from production data? It should be (and I 

assume it is) the former, but this should be clarified here. 

Response 53 

The estimated number of U.S. catfish meals consumed per year was 

calculated from the 1996-2006 NHANES data variable recording the 

responses of 2,979 subjects that reported the number of times catfish was 

consumed over a 30-day period. 

Comment 54 

b. Is the selection of data appropriate? 

Yes. 

c. Are model outputs reasonable? 

Yes, except as noted above. 

d. Is the evaluation of uncertainties sound? 

Yes. 

3.Risk assessment model.
 

General comments on the hazards identified in the assessment. Have the authors 

failed to identify important hazards?
 

All appropriate relevant hazards are identified, and the methods are sound, objective, and 

clearly described. 

a. Chemical hazards 

i. Are the methods utilized to identify the most significant potential hazards 

associated with consumption of catfish valid? 

Yes. 

ii. Are the approaches utilized to estimate exposure and level of concern resulting 

from the consumption of catfish valid? 

Generally yes. Strengths are broad use of published findings to estimate contaminant 

levels; methods to adjust for storage, preparation, and cooking; separate estimates for 

38
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

domestic vs. imported catfish; standardization to age and gender of consumers; and use of 

NHANES to provide nationally representative estimates of mean levels of consumption. 

Distributions of catfish consumption require correction for within-person vs. betweenperson variance, as described above. 

The methods used and their assumptions (and the counterfactual) should be specified for 

estimated effects of lead on CVD, as described above (see 1.d.) 

iii. Are the approaches utilized to estimate the prevalence of adverse effects (acute 

and chronic) resulting from hazard exposure valid? 

Generally yes. 

The use of uncertainty factors requires further discussion and consideration. It must be 

clarified, for example, that all the estimates that utilize uncertainty factors (page 45-46) 

represent the upper bound of actual risk, and that the true risk may be considerably 

smaller. For example, interspecies variation could also result in lower risk in humans 

compared with animal studies, but we never use uncertainty factors <1.0 to account for 

this. Moreover, whereas use of uncertainty factors to account for interspecies variation is 

reasonable (so that the estimates represent the upper bound of actual risk), the use of 

uncertainty factors to account for intraspecies (human to human) variability is not valid 

for the estimation of total population effects. Accounting for intraspecies variation is 

reasonable when the goal is to define a tolerable level of exposure to protect sensitive 

subgroups in the population. However, when the aim is to estimate total population 

effects (as in this analysis), only the average overall effect on the population is relevant, 

that implicitly already includes some persons at higher and others at lower vulnerability. 

Thus, when estimating population effects, use of uncertainty factors to account for 

intraspecies variation “stacks the deck”: the overall population effects are incorrectly 

estimated by basing the population hazard on the hazard in the most sensitive subgroups, 

which is clearly invalid. The range and values of the uncertainty factors should be 

correspondingly reduced to remove for any accounting for intraspecies variation. 

Response 54 

The use of intraspecies uncertainty factor is consistent with the widely 

held assumption that human-to-human responses are more variable than 

the rodent-to-rodent responses observed in homogenous laboratory 

animals. 

Comment 55 

Clarification is also needed for the following: 

1. Are the cumulative probabilities truly “summed” across all relevant hazards? 

Probabilities should not be summed but multiplied. For example, two hazards that each 

increase the probability of an adverse outcome by 5% will together increase the 

39
 
 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

probability by 9.75% (1 - 0.95*0.95), not 10%. These differences will not be major for 

low probabilities, but there is no reason not to compute the total effects correctly. 

Response 55 

It is unclear to what the commenter is referring, but, it may represent 

confusion between the intersection and union of two events. We are not 

determining the probability of acquiring disease X and disease Y (which is 

an intersection of probabilities and calculated as described in this 

comment). We are determining the probability of acquiring disease X or 

disease Y in a total probability sense. This latter probability is a union of 

two events and reflects the sum of the two probabilities (minus any 

overlap in occurrence of diseases X and Y together – which we assume to 

be zero given the generally low probabilities). 

Comment 56 

2. For the estimates of mercury and lead effects on IQ that “were based on the estimated 

yearly consumption of catfish” (page 47), what estimated yearly consumption of catfish 

was used: estimates from the whole U.S. population, or only from women of childbearing age? The latter should be used and specified. 

Response 56 

For this purpose the risk assessment used only women of child-bearing 

age. 

Comment 57 

3. Table 9: Why is the IQ impact of [S]iluriformes consumption more than 10-fold 

higher than the IQ impact of [I]ctaluridae consumption, when the corresponding 

comparative impact of [S]iluriformes vs. [I]ctaluridae consumption is only approximately 

2-fold for every other outcome? This could only be the case if average mercury levels in 

[S]iluriformes were far higher (approximately 5-fold) than in [I]ctaluridae - is this the 

case? If not, why do these estimates differ so dramatically? 

Response 57 

The revised risk assessment focuses on Salmonella, and the effects of 

chemical compounds are no longer considered. 

Comment 58 

Of note, a clear strength of the analysis is the lack of estimation of CVD effects 

attributable to mercury. While this topic is hyped in the media, the present data for CVD 

effects of mercury are limited and highly contradictory, and the authors appropriately 

have not included these presently unestablished effects in their models. 

b. Microbial hazard 

i. Are the methods utilized to identify the most significant potential hazards 

associated with consumption of catfish valid? 

40
 
 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Yes. 

ii. Are the approaches utilized to estimate exposure and dose-response resulting 

from the consumption of catfish valid? 

Yes, well-done using standard and accepted methods. 

iii. Are the approaches utilized to estimate the prevalence of adverse effects (acute 

and chronic) resulting from hazard exposure valid? 

Yes. 

Response 58 

No response needed. 

41
 
 

 

 

 

7 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

REVIEWER NUMBER 4 

Comment 59 

A. Is the report clearly written? 

Some aspects are quite clear. There is clarity in the description of the issue; the regulatory 

framework; the interpretation of results. 

The description of the methodology, however, is not at all clear – or at least is not clear to 

someone trying to reproduce the results. I realize you can’t have too much mathematical 

detail in the report (this could be placed into an appendix), but where the report really 

fails is in conveying the calculational steps and the ways in which these are related to the 

particular issues of probability I mention below. 

Response 59 

The revised risk assessment focuses on Salmonella and describes more 

fully the mathematics of the model. 

Comment 60 

I comment first on the Executive Summary. This section is quite well written in regards 

to ease of understanding. The most important results are included, although they are 

almost impossible to interpret as to public health significance because the size of the 

consuming population is not given. For example, a decrease of 1670 points in IQ would 

be alarming for a population of 100 people, but of no real consequence if spread 

uniformly across a population of a million people (and Table 6 later suggests it is in the 

millions). The way around this problem is to take the approach the USEPA used in their 

most recent assessment of the risks from Pb in air (i.e. the Pb NAAQS Risk and Exposure 

Assessment, 2008). They estimated the number of people with an IQ loss at different 

levels (1 point, 2 points....10 points, etc). They established a level of IQ loss they felt to 

be clinically significant (that level may differ between the two organizations), and only 

then reported the probability of, and number of people with, an IQ decrement larger than 

the threshold level. 

The Executive Summary should also at least briefly specify the inspection program 

considered. What are the catfish being inspected FOR (i.e. what is being measured)? Is 

the proposal (given later in the report, but not here in the Executive Summary) examining 

the fish for each of the compounds for which risks are calculated; or some subset? At 

what point in the food production chain will the inspections occur? What evidence is 

there that the inspections will result in the kinds of detection rates needed to significantly 

reduce the risks? Again, some of these questions are addressed later in the document, but 

they are important enough to at least mention in the Executive Summary. All inspection 

programs have failures, and the rate of failure of the system must be characterized and 

included in the risk assessment, even if the data and past experience simply indicate that 

inspection programs are sufficiently reliable to consider the failure of the program (to 

catch significant cases of contamination of the fish) to be so unlikely as to not affect the 

risk estimates. I doubt, however, that this is the case. 

42
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 60 

The revised risk assessment includes an expansive qualitative description 

of how the model considers the effectiveness of FSIS inspection. 

Additional characterization of uncertainty about this effect was included in 

the updated risk assessment. 

Comment 61 

Page 15 provides the only hint as to why this program of inspection is being considered. 

A reader may be wondering why the program is required – what is it in past experience 

that causes catfish in particular to be of regulatory concern? This paragraph, however, 

doesn’t explain the situation very well; it should be improved. It mentions that there were 

contaminants “in and/or around catfish farms”. The latter half, “around catfish farms” 

doesn’t seem to me very compelling. What matters is whether the catfish come into 

contact with the contaminant, and what the bioaccumulation factor is. This information is 

never specified. To the authors’ credit, the issue is clarified later to some extent by 

making it clear that the prioritization process looked only at the level of contaminant in 

tissue. 

Response 61 

The issue of bioaccumulation is no longer relevant since the updated 

quantitative risk assessment focuses Salmonella contamination. 

Comment 62 

Nor is the claim that there were “7 possible catfish-associated outbreaks” compelling. 

Outbreaks of what? I presume some sort of microbial disorder, but as a reader I am not 

provided the information to determine this. Still, the approach of establishing priorities, 

using the methodology described later, to narrow a large potential list down to 11 is a 

good one and follows the practice of the field. 

Response 62 

For Salmonella, the number of human outbreaks potentially involving 

catfish is one, which occurred in 1991. This outbreak is now explicitly 

considered in an analysis that attempts to validate the model’s prediction 

of annual cases of salmonellosis. This one outbreak is used in the context 

of all foodborne outbreaks (where food vehicles were identified) to 

determine the fraction of illnesses attributable to catfish. 

Comment 63 

On page 16, the reason for considering Siluriformes rather than solely Ictaluridae is 

provided. I can understand the reasoning here, but was also wondering whether it is either 

(1) not possible to separate Ictaluridae from others in the Siluriformes category at the 

market, and inspect only the former, and/or (2) likely that if Ictaluridae poses a risk, so do 

the other members of Siluriformes. If either of these holds, the case for monitoring all 

Siluriformes seems good, but it would be a waste of resources to examine all of the 

43
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

members simply because some parts of the market incorrectly label some fish as catfish. I 

imagine that there may be some reason why catfish are of particular interest (such as 

having high fat content that can bioaccumulate contaminants), and so one wants an 

inspection program that focuses on fish with this same characteristic, not just fish that 

happen to also fall into Siluriformes or Ictaluridae. 

Response 63 

USDA has yet to determine the definition of catfish: this decision is 

pending and is beyond the scope of the risk assessment. 

Comment 64 

B. Does it completely cover all aspects of the analyses? 

All aspects are at least mentioned, but it is very difficult, as I say above, to understand the 

sampling process being used. I will try here to explain where the confusion lies. I will 

assume one wants to know the number of occurrences of a disease in the population due 

to catfish consumption. From Table 6, I would assume there some 2.5 billion servings in 

a year. I could imagine a process in which I sample a concentration at random (from 

different distributions depending on whether this is a pre or post FSIS program, and 

depending on whether the serving is assumed to be imported or domestic). Then I would 

multiply by sample size to get total intake (from an age distribution). Then I would divide 

by body mass (from an age-sex category’s distribution). Then I would use these results to 

calculate an ADRI for that particular event. Then I would calculate a hazard quotient. I 

would repeat this process many times, and keep track of both how many events I had 

modeled. I would then determine the number of these events that had an HQ above 1 and 

divide this by the total number of events to get the fraction of events producing an effect 

(for a non-cancer effect here). This fraction would be multiplied by the 2.5 billion 

servings or events to get the total number of adverse effects. Some variant of this is what 

underlies the calculations in the report. For cancer and IQ loss, one uses the doseresponse function rather than an HQ calculation, but the approach is otherwise the same. 

This is what I expected to find in the report, but I was left wondering what approach 

really was used, because the authors never give a clear description of the sampling 

process or the population to which it is being applied. 

Response 64 

We attempted to estimate the magnitude of adverse effects in the 

population of U.S. catfish consumers. The approach outlined (i.e., the 

“hazard quotient” approach) does not permit the estimation of the 

magnitude of the adverse effects. 

Comment 65 

C. Does it follow a logical structure and layout? 

Yes, it is logical by way or organization of material, but not by way of describing the 

process of reasoning through the steps of calculation. This aspect of the paper needs to be 

greatly improved, with an ideal improvement being a flow diagram for the Monte Carlo 

approach. 

44
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 65 

A flow diagram has been included in the revision (See Figure 1 of report). 

Comment 66 

Section 2.5 describes the overall model structure. It has a problem in that it immediately 

approaches the issue of the Monte Carlo methodology and the calculation of probabilities 

(or more properly, frequencies, as the analysis performed is not an uncertainty analysis 

but rather an inter-subject variability analysis – a distinction the authors need to make 

clear). My primary concern is that the section doesn’t explain how the models are 

structured. What the assessment did was to (1) develop a set of models based on mean 

behavior, exposure and dose-response; (2) to develop inter-subject variability 

distributions for key parameters in these models; (3) propagate this variability through the 

suite of models using the Monte Carlo method; and (4) generate probability density 

functions from the output. Before the reader is told anything about the Monte Carlo 

aspect, they should be told the nature of the “mean value” model, as this is the core 

model that will be wrapped inside the Monte Carlo methodology. 

Response 66 

This comment seems to imply that a “mean” value can be propagated 

through a model that includes a non-linear dose-response. For 

Salmonella, this approach will not work because the mean risk of illness is 

not equal to the risk of the mean dose. For chemical hazards with a doseresponse that is strictly linear in dose, a mean value prediction could be 

generated. Butthe revised risk assessment only pertains to Salmonella, so 

a “mean” value approach will not work. Nevertheless, we provide a 

detailed description of the solution algorithm in the revised risk 

assessment report. 

Comment 67 

The last paragraph on Page 20 is very confusing. It is not at all clear, from this one 

paragraph, how the assessors have modeled exposure. No information is given about the 

size of the population for which the various numbers apply. “Less than two” catfish 

meals are consumed per day, but by whom? Is this the average for the entire US 

population, or just the 19 million or so consumers of catfish? But then a later sentence 

mentions that mean probability was multiplied by the number of catfish consumers per 

year. If this is not the entire population of the U.S., there seems to me a disjoint between 

the calculation of average number of meals per person (presumably across the entire US 

population, although perhaps the authors mean across just the consumers) and the size of 

the population ultimately used to calculate incidence. And am I to assume that the entire 

consumption in a month, which the assessors state will be in the form of only one or two 

meals in the month, is spread uniformly (for purposes of calculation) across all of the 

days of a month? If so, that is fine, as it is standard risk assessment practice in calculating 

Average Daily Rate of Intake (ADRI) to use in a hazard quotient. But then much of the 

document reads as if the total intake from a given dining event counts as just one “event” 

45
 
 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

and is not spread over a period of time to get an ADRI value (remembering that ADRI is 

an average value over a long period of time, not an instantaneous one for a given day). 

The calculation is not well described so the reader can’t be sure (the reader should not 

have to go back into the code to figure out this crucial element of the assessment model). 

Overall, this is a particularly poorly developed part of the report and requires 

improvement. 

Response 67 

The effects estimated in the risk assessment are limited to the catfish consuming 

portion of the US population. 

We understand the reviewer’s confusion. The task of generically explaining a risk 

assessment that considers both chemical and microbial hazards is daunting 

because the approaches can be very different. The revised risk assessment 

focuses on Salmonella, so the explanation of modeling is more specific. Modeling 

microbial hazards is focused on acute exposures, so these models determine the 

per-serving risk of illness. Furthermore, dose-response relationships for microbial 

hazards are not dependent on body weight, blood volumes/concentrations or 

other physiologic metrics. 

Comment 68 

In Section 2.5.1, I cannot be sure how the inspection program operates – in ways that 

influence the probability of a contaminated catfish reaching the table. The ratio of 

positive to negative detects is good information, but if the inspection program finds a 

contaminated fish, some number of other fish will need to be culled. The reader isn’t told 

how this is done. Is the entire “catch” from which that contaminated fish was taken 

culled? Is it all catfish from that water body? If it is a Siluriformes but not an Ictaluridae, 

are all the Siluriformes members from that same catch or water body culled? It would be 

nice to provide this information and explain how this approach relates to the ways in 

which beef and poultry are culled (which is where the pre-and-post ratio of 

concentrations are obtained). 

Response 68 

The details of the proposed catfish inspection system are beyond the scope of 

the risk assessment. 

Comment 69 

And while I understand that the only ratio that could be developed was for beef and 

poultry, I see no reason to believe this same ratio will apply to Salmonella contamination 

in catfish. If the ratio were shown to be the same for beef as for poultry, that might make 

me a bit more comfortable, since it would show that the ratio is roughly similar for two 

very different species (and hence might also be similar for a third species – catfish). Or 

an argument could be made that the PROCEDURE is the same, and hence should be 

expected to yield similar results. I have a harder time accepting that, as the procedure 

surely has a lower limit of detection, and the levels of contamination in fish will be quite 

different from those in beef or poultry. And the culling procedures might differ. More of 

an explanation needs to be given here. 

46
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 69 

The revised risk assessment includes an expansive qualitative description 

of how the model considers the effectiveness of FSIS inspection. In 

general, FSIS believes the effectiveness of its catfish inspection program 

will be similar to what its experience with poultry. Furthermore, the 

emphasis on sanitation in FSIS inspection programs will continue for 

catfish. The revised risk assessment explicitly recognizes the uncertainty 

we have about the effectiveness of an FSIS inspection program for catfish. 

There are two sources of uncertainty considered; the peak level of 

effectiveness achieved and the rate at which that effectiveness is 

achieved. Multiple possibilities for both these factors are considered. 

Comment 70 

In this same section, I am not sure how the modeling treats the other contaminants. For 

Salmonella, the assumption is that the mean concentration with the program in place is 

the ratio of concentrations in beef and poultry after and before an inspection program was 

put in place. But this does not mean the other 10 contaminants will be similarly affected. 

That depends entirely on how much correlation there is between these contaminants and 

Salmonella. I would guess there is little, and note also that Salmonella contamination is 

given by an index I later in Equation 3 (I will return to that in a moment). But the point 

here is that if the other contaminants are assumed to be reduced by the same degree as the 

Salmonella, there must be evidence of full correlation; and if they are not reduced at all, 

there must be evidence of a full lack of correlation. Neither assumption is mentioned or 

explained. 

Response 70 

The revised risk assessment focuses on Salmonella and the effects of the 

other 10 contaminants are no longer considered. 

Comment 71 

Equation 3 is somewhat confusing. X is the concentration of the contaminant at the point 

of production. It is not stated that this concentration is only for contaminated fish, and so 

I must assume it applies to ALL fish at the point of production. But then Equation 3 has 

the index I(h,y) which is 1 only if the fish is contaminated. Surely, the original point of 

production had a mixture of fish, some contaminated and some not, and so the fraction 

contaminated is hidden inside X. I don’t, therefore, understand why I(h,y) is in Equation 

3, unless it is being used as a variable that allows for the eventual effect of the inspection 

program, which presumably lowers the value of I(h,y) to some degree. The inspection 

program, however, could also be reflected in X if desired. This section needs to be 

written more clearly, as the reader will be particularly interested in how the inspection 

program’s effects appear in the Equation. 

Response 71 

The revised risk assessment focuses on Salmonella and the effects of 

other contaminants are no longer considered. 

47
 
 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 72 

The description to Equation 4 causes me to return to an issue I raised earlier: how several 

intakes, or catfish meals, in a month are treated. From the earlier report sections, I was 

assuming the assessors had taken total monthly intake and spread it uniformly across the 

days of the month to obtain an ADRI value for use in an HQ calculation. From the 

paragraph under Equation 4, however, the situation seems quite different. If indeed 

“servings” or “events” are being considered, that sounds as if an individual serving might 

contain the full intake from that serving rather than having that intake spread over a 

month to calculate an ADRI. By this point in the document, I was becoming increasingly 

confused as to how the calculations were being performed. 

For example, consider a person who eats catfish twice a month, with a concentration of 1 

gram of contaminant per kilogram of fish, and 1 kilogram of fish per serving. Under the 

ASDRI approach, the daily intake would be 2 x 1 x 1 / 30 (with the 30 being days per 

month). This is the value that would be used in the HQ calculation, and it would apply 

ON AVERAGE to all 30 days. In the “servings” approach, the person would have two 

servings in a month, each at 1 x 1. This value (of 1 x 1) would be used in the HQ 

calculation. Equation 4 appears to imply that the latter approach is being used. I can 

support such an approach (it is in my mind the better approach), but the writing in the 

report is not clear and so I can’t be sure that is what is happening. 

Response 72 

The revised risk assessment focuses on Salmonella and the effects of the 

other 10 contaminants are no longer considered. 

Comment 73 

In Section 4.1, I generally support the methodology used. The authors have done a good 

job of explaining where they obtained the data; which distributions they chose to apply; 

and the characteristics of those distributions. I would simply ask that a sentence be added 

to each row of Tables 1 to 3 explaining WHY each particular distributional form was 

selected. This often is done by reference to a goodness-of fit criterion. In addition, an 

important feature of such distributions is often their truncation at high and low values. 

They tend to fit towards the middle of the data but not at the extremes. Perhaps I missed 

it, but I don’t see that discussed here. This can be quite a significant issue when the risks 

are high only in the extreme tails. 

Response 73 

Our response to Comment 16 is applicable to this comment. 

Comment 74 

On page 40, the authors report a process that results in the same number of individuals in 

each age-sex category. This would usually worry me, because there is an age distribution 

in the US that is not at all flat. I am assuming, however, that this description really just 

means that the same number of individuals are sampled within each age-sex category, 

48
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

and that the results are kept separate for each age-sex category. If later the sampling 

includes a randomly sampling of age-sex characteristics from the overall population, it 

would NOT be correct to have the same number of individuals in each age-sex category. 

The probability of an individual of a given age-sex being selected should be proportional 

to the fraction of the US population in that category (or if only consumers are considered, 

the fraction of consumers in that age-sex category). 

Response 74 

Age-sex categories were developed so that each group contained 

approximately the same number of respondents. 

Comment 75 

Equation 11 is not described clearly. The natural question to ask is what is meant by a 

“probability” here? Is it the probability that a randomly sampled individual will have 

exhibited one or more of the effects during a lifetime of consumption (I believe the 

answer is no)? Is it the probability that one randomly sampled exposure event (with a 

given age, sex, concentration, etc) will result in an adverse effect? I presume the answer 

lies in Equation 13, where these probabilities are multiplied by number of servings to 

obtain annual occurrences of effect per year. This suggests the authors are calculating 

probability of effect per consumption event (and further, given an earlier comment of 

mine, that they are NOT averaging intake over the days of a month, but rather calculating 

actual intake on a day when catfish are consumed). The authors need to state clearly what 

the probability is referring to so the reader can properly interpret the results. And I find it 

odd that for non-continuous effects, there is no description in the text as to how hazard 

quotients were developed and interpreted. This aspect of the report needs additional 

work. 

Response 75 

Equations 11-13 are necessarily generic for the risk assessment. For 

acute or non-cumulative effects (e.g., salmonellosis), the probability of 

illness is the calculated mean across all servings. For cumulative effects, 

the calculation was somewhat more complicated. We understand the 

reviewer’s request for greater detail of the calculation in the report. 

Nevertheless, the revised risk assessment focuses on Salmonella and the 

calculations in Equations 11-13 accurately reflect the approach used for 

this hazard. 

Comment 76 

On Page 56, we come back to a problem throughout the document: defining the 

population. The authors state on this page that the IQ loss is spread across the “entire US 

population”. I had been assuming, however, that the Monte Carlo sampling was being 

performed across only people who consumed catfish (some 19 million people). The 

calculations can be done either way and the number of effects will be the same (since if 

the entire US population is used, most of the random samplings of individuals will have 

no catfish consumption); the authors just need to be clear which is which. 

49
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

On Page 57, the authors address a concern I raised earlier: that a 1700 or so point IQ loss 

is not large for several million people. They state, correctly, that it is unlikely any 

clinically important effects of IQ loss will be experienced from catfish consumption. This 

statement is not reflected in the Executive Summary, but should be. 

Response 76 

Indeed it would be more appropriate to have said “ “across the entire 

population of U.S. catfish consumers;” However, this change was not 

included in the revised report which focuses on salmonellosis associated 

with the consumption of catfish. 

Comment 77 

D. Where data are included/excluded, does the documentation provide adequate 

justification? Yes, the authors have done a good job of describing the data. Section 3.1 

does a good job of explaining the initial identification process for potential contaminants. 

I see no need for improvement here. 

Sections 3.2 and 3.3 also are good explanations of the prioritization process. While this is 

always a contentious issue in hazard identification, the methods used here are as good as 

any others, and consistent with past practice at the organization, and so I support them 

here. In addition, the results obtained are reasonable based on my own past experience 

with fish (not catfish in particular, but rather fish more generally). 

On Page 42, mention is made of breading causing a “20 to30%” decrease in the size of 

the catfish meal. This is too wide of a range. Did the authors use a Monte Carlo sampling 

between these extremes, and that is why a range is given? Or was a single value used 

throughout (and if so, which value was it – 20 or 30%, or a mean of 25%, or some other)? 

Response 77 

Monte Carlo sampling was used to sample between the extremes. The 

amount of breading was sampled from a uniform distribution ranging from 

20 to 30%. This range of breading was derived from the Southern Region 

Aquaculture Center publication No. 185, “Processed Catfish” (November 

1989). 

Comment 78 

On page 42, a method is mentioned for assigning imported vs. domestic concentrations. I 

assume the authors mean that the concentration selected was from a two-step process: (1) 

whether it is a domestic or imported fish is determined through a random number 

generator based on the fraction of fish that fall into each of the two categories and (2) 

once the category is determined, there is a random sampling of concentrations from 

within that category. This aspect of the study isn’t explained well here and simply needs 

clarification (the computer code executed the step properly). 

50
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 78 

The reviewer’s assumptions are correct: the concentration selected was 

from a two-step process: (1) whether it is a domestic or imported fish is 

determined via Monte Carlo sampling from a binomial distribution based 

on the fraction of fish that fall into each of the two categories, and (2) once 

the category is determined, there is a Monte Carlo selection from the 

hazard concentration distribution for each hazard. 

In addition, the revised model is written in a programming language that 

should afford interested reviewers the opportunity to directly review the 

coding and processing flow of the model. 

Comment 79 

Equation 6 concerns me for two reasons. First, the assessment has a combination of 

cancer and non-cancer effects. The former are usually estimated by probabilities, but the 

latter as hazard quotients. One can’t sum them in any meaningful way (and so I doubt this 

is what the authors meant to convey). Second, summing hazard quotients to obtain what 

is in essence a hazard index requires the contaminants have a common or shared mode of 

action, or at least act on the same organ/tissue. That is not the case here. So, I am not sure 

what is meant by Equation 6, unless it is to be applied only to each specific category of 

effect (such as IQ loss, or to cancer), rather than across categories of effect. I could 

support the former, but not the latter. The report needs to clarify this issue. 

Response 79 

We agree that more clarity could have been provided. However, these 

equations do not appear in the revised risk assessment, which focuses on 

Salmonella only. 

Comment 80 

I agree with the approach in Equation 7, although the risk summary must eventually 

consider the case of an individual consuming catfish multiple times each year (22 times 

on average if Table 6 is to be used). But the results in the report suggest this was applied 

correctly. 

D. Are the assumptions logical and adequately justified? 

Once one can find the assumptions in the document, they do appear to be adequately 

justified. As mentioned above, however, it is quite hard to follow the logic used in the 

calculations, and so it also is hard to determine whether they are completely appropriate 

to the tasks. When I was able to determine this, however, the assumptions were 

appropriate. Where I could not perform this check is in the assumptions on the Monte 

Carlo sampling, since I could not find a clear explication of this procedure in the text. 

Response 80 

51
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

The updated risk assessment provides greater detail regarding the model 

solution algorithm presented below. 

Mathematically, the average exposure dose of Salmonella2 consumed in a random 

contaminated catfish serving is modeled as; 

Equation 2 

D X S G C

where 

D

is one instance of an average exposure dose of Salmonella consumed, 

X

is 

one instance of an average Salmonella concentration per gram of a contaminated catfish 

carcass, 

S

is one instance of a catfish serving size (in grams consumed), 

G

is one 

instance of the growth of Salmonella on a catfish carcass (to account for handling and 

storage between processing and consumption), and 

C

is one instance of the expected 

reduction of Salmonella in a serving catfish caused by the effects of cooking. The inputs 

to this calculation ( 

X S G C , , ,

) are random variables. The inputs for catfish serving size 

and the effect of cooking, however, are somewhat complicated. 

The risk assessment model uses Monte Carlo techniques to convolve the random 

variables ( 

X S G C , , ,

) that predict exposures for each of the four exposure classes and 

complete the numeric integration step described in Equation 3. The model is currently 

developed in the R software package (http://www.r-project.org/ Version 2.9.1), but is 

equivalently solvable in any software that supports Monte Carlo simulation. Each 

simulation of the model comprises three million iterations. Each model iteration 

represents a different contaminated serving of catfish across all four exposure pathways. 

Comment 81 

E. Is the selection of data appropriate? 

Yes, I am comfortable with this aspect of the analysis, including how data were assigned 

distributions for the sampling process. This comment applies to the data on 

concentration; ingestion amount; body mass; and the various health effects coefficients 

and benchmarks. 

The treatment of IQ loss in Section 5.3 is fine, and agrees largely with a similar analysis 

performed by the EPA in looking at the NAAQS (National Ambient Air Quality 

Standard) for Pb. And the authors are correct in noting that this is a continuous variable. 

The treatment of heart-related disease is also the same as in the larger EPA analysis, and 

is acceptable. 

Equation 10 is a reasonable approximation of the dose-response characteristics of 

Salmonella, and so I have confidence in the data employed to develop this equation. 

2 Average dose of Salmonella is modeled because the beta-Poisson dose-response relationship is based on 

an average number of organisms in a serving. For example, if a value for average Salmonella dose of 0.2 

CFU is used in the beta-Poisson, the function determines the probability that a serving will contain one or 

more CFU’s (based on Poisson probabilities), as well as the probability that each integer unit Salmonella 

dose will result in illness (based on beta probabilities). Ignoring this aspect may lead to incorrectly 

including a Poisson function to determine integer Salmonella doses consumed in the exposure assessment; 

this would essentially ‘double-count’ the Poisson effect once the beta-Poisson relationship was included. 

52
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

F. Are model outputs reasonable? 

This is a strange question, because the “reasonableness” is not to be found in outputs but 

rather in the process producing these outputs. In general, there was nothing about most of 

the outputs that drew my eye to them and made me think the results were so far out of 

line with prior expectations as to be unreasonable (if that is what the question means). 

The one exception is the upper bound estimates of risk, which I find so far out of line 

with actual incidence of disease that I call into question whether that result is reasonable 

(with the error probably lying in the comments I made in Section 1 on this issue). 

Response 81 

We agree that the upper bounds of our uncertainty analysis are 

‘unreasonable’; these are reported as extreme predictions that represent 

the model’s predictions when multiple inputs are simultaneously set at 

their extreme values. As we explain, we expect the “true” values for cases 

avoided to lie somewhere in between our extreme predictions. 

A major driver for the range of illnesses avoided is the effectiveness of 

FSIS inspection. The revised risk assessment considers this effectiveness 

in more detail. 

Comment 82 

I have never developed information such as that in Tables 4 and 5, and so I can’t speak to 

the reliability of these estimates. However, the assumption that heat lability is important 

is correct, as is the issue raised about concentrating a contaminant due to loss of moisture 

content (which depends on the style of cooking). I can also say that the Salmonella 

growth curves are appropriate, although my past experience is with poultry and dairy, not 

fish. Still, I can’t see why the same equations/methods shouldn’t apply for growth and deactivation. In poultry and dairy, however, one also must specify the temperature at which 

the product is stored, worked with, etc. That isn’t done here, although the report needs at 

least some mention of where these factors appear inside (or are hidden inside) the growth 

equations. 

I don’t understand Table 6. I understand what it is intended to convey, but the numbers in 

it don’t add up. I would presume that the number of catfish consumers (sixth column) 

times the mean number of servings per year (fifth column) would equal the annual 

servings (fourth columns). But they don’t – not by a wide margin of 7. So I can’t follow 

that table, or how it reflects in later calculations (output). I suppose the servings per year 

might be the average over the entire US population (I very much doubt it), in which case 

the table makes more sense. 

Response 82 

Temperature is included in the estimates based on the cooking style for 

the portion. 

53
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

The reviewer’s assumptions on the columns in table 6 are not valid 

because the distributions associated with this calculation (serving size, 

servings per month, etc.) are not normally distributed. 

Comment 83 

On Page 55, I note that a program aimed at Ictaluridae reduces the occurrence of disease 

better than one focused on the entire category of Siluriformes. This seems counterintuitive, unless the authors are assuming a fixed number of inspections available (hence 

more resources are going to sampling non-Ictaluridae fish in the Siluriformes program, 

leaving the population more exposed to Ictaluridae exposure where the risk is 

concentrated). Whatever the reason, it needs to be explained. 

Response 83 

The comment is incorrect. Predicted reductions for Siluriformes are always 

larger than (or equal to) reductions for Ictaluridae (see Table 9). 

Comment 84 

H. Is the evaluation of uncertainties sound? I suppose so given that I am not convinced 

the uncertainty analysis plays any role in decisions to be taken here. I do, however, 

struggle to understand what one would do with the result where all assumptions are at 

their lower bound or all are at their upper bound. In order to be useful, there would need 

to be some explanation as to how likely either of these results is, something lacking in the 

current report. I agree, however, with decision not to attempt a fully nested uncertaintyvariability assessment, as this would have been beyond available resources (but not 

beyond what might be expected in regulatory assessments where the goal is to understand 

actual risks, rather than to determine whether an inspection program is justified). 

Response 84 

As explained in the report, a probabilistic interpretation of the bounds of 

uncertainty that we estimate in the report is not possible. These can 

simply be interpreted as best/worst-case predictions. We agree with the 

commenter that more detailed uncertainty analysis was unnecessary for 

this risk assessment. So the initial approach will be applied to the revised 

document. Uncertainty analysis is intended to suggest how wrong the 

default predictions might be if (plausible) changes are made to the model 

inputs. The boundaries suggested by the original analysis were quite wide; 

the revised risk assessment considers predictions using public health 

surveillance as an alternative perspective regarding the plausible bounds 

of uncertainty. 

Another purpose of uncertainty analysis is to identify data gaps that might 

be addressed in the future. In the revised risk assessment, we highlight 

the model input uncertainty. Because the model’s predictions pertain to 

the future, however, the true number of illnesses and effectiveness of 

54
 
 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

FSIS inspection cannot be known prior to implementation of FSIS 

inspection. 

Comment 85 

I am similarly not sure what to do with the results in Section 6.2. A sensitivity analysis is 

often useful, but usually to help define how the uncertainty analysis will be performed. 

And then the uncertainty analysis is performed because one either wants to allocate 

resources to research (to reduce residual uncertainty) or to place the results in some sort 

of decision framework. Neither seems applicable here. I suppose Section 6.3, and its 

uncertainty analysis, is here to provide some insights into the reliability of the results in 

previous sections. Fair enough. But the authors don’t give the reader insights into why the 

uncertainty analysis is being performed or what they intend the reader to do with the 

information. This could be improved. 

Response 85 

The sensitivity analysis was indeed completed to allow a more focused 

uncertainty analysis (that only considered the effects of the more sensitive 

model inputs). The uncertainty analysis might also be named a scenario 

analysis; it intends to explain how much higher or lower the model’s 

outputs might be for combinations of extreme input settings in the model. 

While the worst/best-case predictions can not be interpreted 

probabilistically, they are still useful in a decision framework. Absent any 

discussion of uncertainty, a decision simply hinges on a point estimate for 

cases avoided. This uncertainty analysis provides some idea about the 

boundaries of our predictions. These boundaries can, at least, inform 

decision-makers about the benefits that could accrue from the program. 

The revised risk assessment provides a more detailed assessment of 

uncertainty. Nevertheless, the fact that it is impossible to be certain of the 

effectiveness of an FSIS inspection program prior to implementation, 

some uncertainty in the model predictions is unavoidable. This 

uncertainty cannot be informed by empiric evidence (either about human 

illness occurrence or specific effectiveness of various FSIS inspection 

procedures) because catfish-associated illnesses represent but a small 

share of human salmonellosis cases and an FSIS program for catfish is 

yet to be implemented. Therefore, improvements in certainty about 

program effectiveness will have to wait until data are collected upon 

implementation of the inspection program. 

Comment 86 

I agree, however, with the decision not to apply what the authors call a “second-order” 

uncertainty analysis, by which they mean a nested uncertainty-variability analysis. Such 

an analysis probably is infeasible given resources and data, and I am not convinced it 

would add anything to decisions. The lower and upper bound analysis used here is 

adequate. However, I believe the authors have used either ALL lower bound estimates, or 

55
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

ALL upper bound estimates. I suppose there is some merit in doing this, but it would be 

highly unlikely that all of the parameters would be at their upper bound. A better 

approach would have been to sample randomly after assigning lower, mean and upper 

bound estimates of a parameter some likelihood (e.g. 10% for each of the lower and 

upper bounds, and 80% for the mean). I am also assuming that when the upper bound 

values were used, they were used for BOTH the before and after an inspection program 

calculations. If so, one could look at the predicted incidence of disease in the “before 

program” calculations and see whether they bear any resemblance to the incidence we 

currently see in catfish-consuming populations. I would bet they don’t (greatly overpredicting that incidence). For example, are there REALLY almost a million cases of 

Salmonella-associated illnesses as shown in the final column of Table 11 (which must be 

assumed, otherwise how could the inspection program reduce the incidence by close to 

700,000 cases)? I doubt it or no one would be eating catfish! 

Response 86 

Based on comments from public health surveillance experts (e.g., CDC), 

this assumption is on track with respect to the upper bound of uncertainty. 

It seems reasonable to expect that catfish-associated Salmonella illnesses 

are, even at the extreme, fewer than what can be attributed to more 

common food sources of salmonellosis (e.g., chicken or beef). The revised 

risk assessment informs its upper bound estimates about human illnesses 

with this reasoning. 

Comment 87 

I realize the following is not strictly part of the uncertainty analysis, but it is a point about 

uncertainty I need to raise, and this is as good a place as any to include it. On page 45, I 

completely disagree with the description of the application of uncertainty factors (UFs). 

UFs do NOT reflect the “increased susceptibility” of humans. In fact, humans are 

sometimes more susceptible, sometimes less. UFs reflect this fact by reflecting the 

UNCERTAINTY as to whether humans will be more or less susceptible and/or sensitive 

for a particular compound where one can’t be sure on the basis of other evidence. They 

are a way of building in a margin of safety, NOT a way of making the risk estimates 

better reflect the actual risk to humans. This paragraph needs to be re-written to reflect 

scientific understanding of the nature of UFs in regulatory decisions. 

Because of this issue, I am concerned about the description of sampling from the UF 

distribution given on Page 46. There IS no distribution of UF values; the UF is developed 

from a distribution of susceptibility or sensitivity ratios for humans over rats. If any 

sampling is done, it needs to be between 0.1 and 10, not between 1 and 10, which seems 

to suggest that humans can be only MORE susceptible or sensitive than rats. This part of 

the assessment needs to be rethought. 

Response 87 

We would argue that our extrapolation of animal toxicity results to humans 

assumes that human susceptibility to chemicals was appropriate; 

however, as the revised risk assessment focuses on Salmonella such 

toxicity calculations are no longer considered. 

56
 
 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 88 

The use of Equation 8 within this caveat is fine. There are many different distributional 

forms that could be used, and this one is as good as any other, as well as being employed 

often in risk assessments. 

I am not sure what the figures on Pages 60 and 61 are intended to convey, or more 

properly why that information informs decisions. I suppose it is interesting to me 

scientifically, but I don’t believe it says anything about the effect of the inspection 

program. 

Response 88 

Figures 3 and 4 illustrate the predicted annual variability in illnesses or 

cases avoided for Salmonella and cancer, respectively. The point of these 

figures is simply to illustrate that our predicted annual rates have an 

element of randomness in the real world. If the annual rate is constant, 

the realized number of illnesses/cases from year to year could follow a 

Poisson distribution. Such variability might be important in predicting the 

benefits accruing to the program as a function of time. For salmonellosis, 

the variability is minor relative to the average; for cancers, this variability is 

relatively substantive and could be included in a time-series analysis of 

costs and benefits. In any case, the revised risk assessment focuses on 

Salmonella and will not include this source of randomness in its 

predictions. 

Comment 89 

I. The more important hazards have been identified. I don’t believe that adding in 

more hazards would change any central conclusions. It would increase the estimated 

incidence/risk, but my sense is that the current assessment already provides enough 

evidence of the desirability of an inspection program, and that increasing the number of 

hazards considered would not change this. This applies to both the chemical and 

microbial hazards examined. As to methods, I have already commented on these. 

A very minor point, but Table 8 spells Acute as Accute. 

Response 89 

Upon careful consideration the risk assessment was revised to focus on 

Salmonella. The effects of other contaminants are no longer considered. 

Comment 90 

I also feel I should point out what I believe to be the correct methodology that should be 

used. This will not require significant changes to the EXCEL code I reviewed, but it does 

57
 
 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

require significant changes to how the methodology is described in the document. I 

propose: 

1.	 A PDF of existing concentrations for a given contaminant is developed based on existing 

data (this is already executed correctly in the code, and described in the document). 

2.	 A random selection is made from this PDF to obtain a concentration. This will be the 

“non inspection” value. 

3. The result of 2 is multiplied by the post/pre inspection program ratio to obtain the 

expected concentration after the inspection program is in place. Ideally, this ratio would 

be developed based solely on data from the beef and poultry programs that apply to 

samples ABOVE A THRESHOLD LEVEL (since only these samples would be affected 

by an inspection program), and this ratio then applied only to catfish samples that are 

above the chosen threshold level, but I realize this may be asking too much of the 

existing data. 

4. For EACH age/sex group considered, the concentrations in 2 and 3 are converted to 

total intake of contaminant in that serving (two results, one for pre and one for post 

inspection program; and then N sets of these two results – one set for each age-sex 

category). 

5. For that contaminant, the concentrations are used to calculate the various risk metrics 

(cancer risk, IQ loss, HQ, etc) and the result stored. 

6. The above 5 steps are repeated for a large sample population, perhaps 10,000 or more 

to gain stability. 

7. For each contaminant, risk metric and age-sex category, calculate the MEAN value 

for probabilities and IQ loss, or the probability that an HQ value exceeds 1 (the first two 

are true means, the latter one is a frequency count). 

8. The values of step 7 are multiplied by the total number of servings consumed each 

year to obtain the total number of effects in a year. 

Response 90 

This approach does not consider risk associated with exposure below 

regulatory levels or levels of concern. Risks associated with such 

exposures is likely greater than zero and should be included in the overall 

estimates. 

58
 
 

 

 

 

8 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

REVIEWER NUMBER 5 

Comment 91 

General Observations and responses to the charge to peer reviewers 

On the goals, accomplishments, and limitations of the assessment 

First I might note that I am to some extent an outsider. My interests and experience have 

been in trying to make risk analysis a more effective and useful tool in managing hazards, 

especially in situations of considerable uncertainty; I am not, however, experienced in the 

details of food inspection. This limitation might be helpful, however, in giving a 

perspective on the transparency and balance in the presentation of the assessment. Some 

things that are obvious to those “in the know” are not so obvious to the rest of us. And 

very occasionally these raise significant issues that if left alone might receive inadequate 

attention. 

The immediate goal of the analysis and presentation is to provide a robust assessment of 

anticipated benefits of a catfish inspection program that can be compared with the costs 

of the program to inform a decision (in this case a legislative mandate) to proceed to 

establish such a program. This goal could be interpreted rather narrowly: to identify and 

quantify a sufficient set of benefits to inform the decision. It could also be interpreted 

much more broadly: to develop a planning document that could help guide the realization 

of those benefits. While I have some quite significant technical reservations, I believe 

that the assessment and its documentation are largely successful in accomplishing the 

narrow mission. That is there are clearly identified anticipatable benefits whose 

quantitative assessment is believable and can be compared with anticipated costs. The 

pressures of legal mandates and time inevitably drive the agency toward maintaining a 

narrow focus and it would not be reasonable for me to expect a much broader planning 

document. However, a great deal of work has gone into this assessment and much 

valuable information has been compiled and built into the assessment model; it would be 

a pity to have that experience simply filed away and only used in so far as it is part of the 

experience of the creators of the assessment. 

I believe that the addition of a modest amount of qualitative discussion along with some 

improvements in the documentation of the model would not be an undue burden and 

would contribute substantially to the future applicability of the modeling effort. And note 

the quote from the Charge to Reviewers under my specific comments to p. 10. 

Response 91 

The suggestion is appreciated and a qualitative discussion has been 

incorporated into the revised report as it pertains to Salmonella. 

59
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 92 

An interesting historical analogy is with the risk assessments for nuclear power plant 

accidents. These landmarks in the early days of risk assessment were originally 

developed to inform the debate on liability for nuclear accidents (associated with the 

renewal of the Price-Anderson act); however, we have learned subsequently that their 

most important contribution has been to provide guidance for substantial improvements 

in nuclear plant safety. 

For the case at hand, the important observation is that there are many potential benefits to 

an inspection program that are not easily or appropriately quantifiable and these include 

opportunities to better address many issues that arise in considering uncertainties in the 

quantification effort. A qualitative discussion might consider the opportunities provided 

by obtaining better knowledge of the distribution of contaminants, better evaluation of 

the effectiveness of the inspection system, identifying opportunities to make and test 

improvements in the inspection program, and better preparation for new contamination 

concerns as they arise. In addition a qualitative discussion could pay attention to some of 

the indirect benefits of an inspection system: avoidance or mitigation of public anxiety 

and economic losses from contamination incidents (the social amplification of risk); 

improving public health by encouraging more consumption of fish – particularly fish that 

is relatively low in contaminants like mercury; etc. 

Response 92 

The revised risk assessment includes substantial qualitative discussion 

regarding program effectiveness. Nevertheless, we anticipate the greatest 

opportunity will occur once a program is in place and data are generated 

from it. We expect the risk assessment approach will be used to refine and 

improve the inspection program in the future. 

Comment 93 

The identification of opportunities for improving the benefits of an inspection program 

over time should be a (modest) part of the discussion in the scope of the assessment 

(section 2.2) and in setting the stage on risk management (section 2.3) and the risk 

assessment (section 2.4). It should also be a modest part of the discussion of the 

prediction of regulatory effects (section 6.1), sensitivity analysis (section 6.2), and 

uncertainty analysis (section 6.3), and it should be part of the summary (section 7). 

Further below I point out some specific pages where such observations would fit. 

Response 93 

The revised risk assessment includes more discussion about an 

inspection program. Nevertheless, a risk assessment should focus its 

analysis on its intended purpose. In this case, the risk assessment 

primarily informs Agency rule-making and is used to assess predicted 

benefits of a yet-to-be-implemented FSIS inspection program. 

60
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 94 

In addition there is room for improvement of the documentation within the crystal ball 

spread sheet model. More thorough and consistent documentation of the data being used, 

the data sources (and alternatives) and the assumptions made in the various steps in the 

actual spread sheets, would make the model easier to expand and update. 

Response 94 

We have addressed the need for improved documentation with respect to 

Salmonella in the revised risk assessment report. 

Comment 95 

An important aspect of the assessment is that it explicitly compares management 

alternatives. This desirable characteristic is why we should be concerned with the 

ongoing use of the assessment and the practical implementation of management policies. 

In that context the use of Figure 1 to describe the organization of the assessment is 

perhaps not best as management (and the research needed to evaluate and improve 

management actions) is pushed too much to the side. A good alternative that considers 

this concern but maintains much of the original red book spirit is given in the recent NAS 

report Science and Decisions: Advancing Risk Assessment (NRC 2009) as both Figure S1 (from the report summary), and 8-1; the assessment authors may enjoy reading chapter 

8 as well. There is a similar problem with Figure 2 in the assessment. It gives the 

structure for the baseline analysis, but obscures the role that inspections can play and a 

little bit of editing of that figure might be helpful (or at least some clarifying discussion). 

As the assessment notes, and I will comment on later, there really are 4 options 

considered because the possibility of inspecting or not inspecting imported catfish is 

evaluated within the model. There is only very limited discussion of the relative merits 

and difficulties of including imported catfish in the text and that should be remedied. 

Response 95 

The factors mentioned by this reviewer will be considered by the 

appropriate FSIS policy makers. 

Comment 96 

Technical reservations and a comment on hazard identification 

I mentioned that I had some reservations concerning the model in its present form. Some 

of these I note as specific comments directed at the discussion on particular pages of the 

document, but there are three recurring issues that I want to mention up front along with a 

general comment on the selection of hazards for analysis. 

61
 
 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Over-reliance on and lack of critical discussion of the Pareto distribution I try 

not to be very fussy about the choice of distributions for Monte Carlo analyses, 

particularly when the effect of the distribution is largely an interpolation from 

a fit to data. And the fact that there is not much of a mechanistic justification 

for its use (compared to a log normal distribution for instance) does not seem 

so serious since at the high end of values the arguments for log-normality 

become weak and power laws often make a good fit. However, there are 

several problems with the use of Pareto distributions here and they merit, at a 

minimum, a careful critical examination. From my perspective, the most 

significant issues are with low values. The structure assumed in the model is 

that there is generally a finite probability of zero contamination, then zero 

probability for contamination up to some level (perhaps a detection limit, but 

this is not clear from either the text or the spreadsheet, nor is it consistent 

between the domestic and international distributions), and then a finite 

declining probability for contamination above the particular level. In many 

cases, most of the probability resides quite close to the specified level, while 

the expected values (the means) reside well higher on the tail of the 

distribution. (Indeed for some of the distributions, the means are infinite 

implying a large (infinite?) contribution to the expected health effects if there 

were sufficiently many trials; that can be viewed mostly as a mathematical 

peculiarity, but it also is a warning.) The problem at the low levels is that this 

picture of a gap is not realistic (especially for common contaminants like lead 

and arsenic), and there are practical management issues that depend on what 

happens on both sides of the detection limit. Do the catfish farmers in 

responding to the implementation of an inspection system only alter the 

likelihood of detection (increase a probability of zero lead, not that that is 

practical) or do they effectively move a distribution of contamination to a 

lower level? If the latter there are likely to be further benefits from the 

lowering of undetected lead levels. Another way to think about this 

management question is to consider the potential value of improving detection 

limits. 

Response 96 

The comment is well taken. Yet, the effect of some other distributional 

form is unlikely to change the general conclusions of the model; the 

predicted effects of FSIS inspection on hazards other than Salmonella are 

relatively minor. The revised risk assessment focuses solely on 

Salmonella. The focus on Salmonella does not rely on a Pareto 

distribution assumption. 

Comment 97 

Imperfect characterization of inter-individual variability including variability 

in background levels of exposure For some of the harmful effects considered, 

62
 
 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

cancers and IQ deficiencies (for the assumed dose-response models) the per 

meal analysis and simulation works fine; the risks are additive and distributed 

across the population (or identified population subgroups). But there are nonlinearities in the dose response for chronic diseases such as cardiovascular 

diseases and extreme non linearity for acute toxicity (though this may still not 

give many or any cases). There is interindividual variability in the sensitivity 

to such exposures and that is tied to a person, not a meal, and there is also 

interindividual variability in background levels of lead in blood for instance. 

(some of that variability is age dependent – older people are likely to have 

significantly higher blood lead concentrations because of accumulations in 

their bones). Distributions reflecting variability in both sensitivity and 

background should be considered along with the possibility that a particular 

individual can consume a relatively high number of catfish meals. For acute 

toxicity, the possibility of concern is a sensitive individual who eats several 

servings worth of highly contaminated catfish in a relatively short period of 

time. This possibility might well not be of practical significance, but the 

analysis to show that is obscured by a per meal analysis that confounds the 

Price et al. uncertainties in extrapolation with the inter-individual variability in 

sensitivity and that should be clarified. Incidentally, there is a good discussion 

of alternative approaches to dose response in Chapter 5 of the NAS Science 

and Decisions report mentioned above. 

Response 97 

In conducting risk assessments for rulemaking, as we are doing in this 

case, it is appropriate for us to concentrate on the overall probability of 

illness in the population. Benefits are computed across the totality of 

cases and severity is assumed to be independent of the probability of 

illness (e.g., the severity of Salmonella illness is assumed independent of 

the dose that caused the illness). Therefore, the risk assessment 

essentially predicts the “average” probability of illness per serving, and 

then multiplies this probability by the number of servings (per annum) to 

determine the number of illnesses (per annum). The difference in illnesses 

between the baseline and post-policy scenarios determines the number of 

cases avoided. A benefit-per-case-avoided is assigned to determine total 

benefits accruing to the policy. 

Comment 98 

Combining in a Monte Carlo simulation stochastic responses, probability 

distributions to characterize variability and probability distributions to 

characterize uncertainty can obscure the identification of opportunities for 

improving the management of hazards. Here too the NAS Science and 

Decisions is a good resource – chapter 4 provides an abbreviated update of the 

discussion in the 1994 NAS report Science and Judgment in Risk Assessment, 

though the latter is also worth consulting. From a management perspective the 

practical concern is that new information can reduce uncertainty and enable 

63
 
 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

better decisions and better focusing of effort, while new information will not 

reduce variability, but might delineate it better and suggest ways of targeting 

management actions. Thus, generally speaking, it is cleaner not to include 

uncertainty distributions in Monte Carlo simulations unless you are 

specifically using the simulation to reflect the uncertainty. Since I believe the 

authors of the assessment have taken an appropriate approach by 

characterizing uncertainties qualitatively with the presentation of results from 

sensitivity analyses, including uncertainty distributions in the primary (best 

estimate) simulations serves only to create an additional blurring of the 

findings. Some examples are the distribution of effectiveness of FSIS 

programs (it is better to learn about effectiveness), and uncertainties in 

parameters describing dose response. 

Response 98 

We agree with this comment. The inclusion of uncertainty about 

effectiveness and dose-response parameters was inadvertent. The 

revised risk assessment considers uncertainty about effectiveness, but 

makes different predictions for specific levels of effectiveness. 

Nevertheless, the uncertainty included in the original risk assessment’s 

default predictions did not substantially influence the conclusions because 

that uncertainty distribution was averaged across all servings. 

Comment 99 

The selection of hazards The prioritization approach to select hazards for 

analysis seems appropriate and done with care. I want to mention that 

usefulness for future management does add some further considerations. For 

instance, I would have liked to have seen at least one more microbial hazard, 

because a comparison with the issues pertaining with Salmonella might be 

informative. And public concerns matter: I was surprised not to see Dioxin in 

the list, given that there has been a catfish incident involving dioxin. But most 

important is the qualitative aspect which should be stressed in the final 

discussion. At the beginning of section 3, the authors properly observe that the 

knowledge base (and aquaculture practice) may change: one of the beneficial 

opportunities provided by inspections is the capability to keep track of and 

evaluate such changes. 

Response 99 

The revised risk assessment focuses on Salmonella and the effects of 

other contaminants, including dioxin, are no longer considered. We lack 

human illness data associated with catfish consumption that would 

support the inclusion of additional microbial hazards to the high priority 

group. 

64
 
 

 

 

 

 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 100 

Comments on model documentation 

1)	 Is the report clearly written? 

The report is quite well written and easy to follow (a great help to the 

reviewer and future readers) – there are places noted below where 

explanations assume too much prior knowledge 

2)	 Does it completely cover all aspects of the analyses? 

There some significant gaps as discussed above and identified in the specific 

page comments below. 

3)	 Does it follow a logical structure and layout? 

The structure and layout are logical and clear – however as discussed above 

(and in specifics below) the structure has been too much drawn from the old 

red book (NRC 1983) and downplays the importance of critical analysis of the 

effects and effectiveness of inspection. 

Response 100 

The revised risk assessment will provide more detail regarding the 

anticipated public health impact of FSIS’ inspection program based on 

how effective this program might be and the timeframe in which it takes 

FSIS to achieve this “effectiveness” (i.e.., to achieve a percent reduction in 

Salmonella prevalence on catfish and corresponding reduction in annual 

cases of salmonellosis in the U.S. each year). 

Comment 101 

6) Where data are included/excluded, does the documentation provide 

adequate justification? 

Most choices are justified – there is room for some improvement as noted 

below. 

Comment on the validity of the model 

a.	 Are these assumptions logical and adequately justified? 

As noted above, I have some significant reservations about the assumptions in 

the model and these should be addressed. These include the per meal 

approach for some of the hazards as discussed above. Some further issues and 

additional detail are mentioned with specific page references below 

b. Is the selection of data appropriate?
 

Yes, with some reservations below.
 

c. Are model outputs reasonable? 

The basic findings seem plausible. Some aspects should be checked: 

for instance the chronic disease and acute toxicity findings which depend on 

interindividual variability and the implications of using Pareto distributions. In 

these cases my expectation is that there has been an underestimate of potential 

benefits 

65
 
 

 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Response 101 

See response to q 16. The revised risk assessment focuses solely on 

Salmonella. That pathogen does not rely on a Pareto distribution 

assumption. 

Comment 102 

d. Is the evaluation of uncertainties sound? 

The basic approach using sensitivities is sound, provided it is coupled with a 

more complete qualitative discussion. As noted above, some uncertainty is put 

into the Monte Carlo and that is confusing. And there are some important gaps in 

the discussion as noted below. 

Response 102 

We have included an expanded qualitative discussion of uncertainty in the 

revised risk assessment. 

Comment 103 

Comment on the risk assessment model 

a. Chemical Hazards 

I discussed identification above: there are some minor points mentioned 

below. The per-meal approach only is valid for some outcomes. 

b. Microbial Hazard 

As noted above I would have liked to see one other microbial hazard for 

comparative purposes even though the others were only assigned medium 

priority. Also given the large impact of Salmonella, it might well be that 

medium priority hazards are significant. The per meal approach seems 

appropriate for microbial hazard 

Response 103 

We lack additional human illness data associated with catfish consumption 

to support the inclusion of microbial hazards other than Salmonella. See 

McCoy et al (2011) JFP 74(3)500-516) for a discussion of foodborne agents 

associated with the consumption of aquaculture catfish. 

Comment 104 

Specific Observations 

Executive Summary 

p.10 It might be worth quoting the beginning of the Charge to Reviewers especially if 

the future use of this assessment is contemplated. 

66
 
 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

The Food Safety and Inspection Service (FSIS) Office of Public Health Science (OPHS) 

addresses food hazards through prevention and control activities. Risk assessments are 

used to evaluate intervention strategies to reduce foodborne risks and to guide, support, 

and enhance the agency’s overall decision-making process, risk-management policies, 

outreach efforts, data collection initiatives, and research priorities. 

p. 11,12 The executive summary should clarify that there are really four scenarios 

discussed and, preferably, should reflect some further discussion from a revised text that 

explains why all are considered and what the implications are. 

p.13 There should be some reflection of the (revised and extended) summary 

Response 104 

We appreciate these comments and have revised the executive summary 

to be more informative. The purpose of the risk assessment is to provide a 

foundation for policy development. As such, during the risk assessment 

process, minimal assumptions about the implications of the various policy 

options under consideration are included in reports such as this. 

Comment 105 

Introduction 

p.14 Here again you could look toward the future with the Charge to Reviewer’s quote 

p.15 Under Scope need more discussion of what the key issues are, what is and what 

isn’t in the assessment, and, something, about how an inspection program creates 

opportunities for reducing hazards and these can change and be improved over time. 

p. 16,17 Under Risk Management need some discussion of what management can 

accomplish and how it gets put into an assessment model; also some of the qualitative 

discussion of opportunities for adapting management by gathering new information and 

adjusting policies accordingly. And there must be more discussion of the merits and 

justification for considering both imports and domestic inspections. 

Response 105 

The reviewer’s comments have been taken into consideration in the 

drafting of the revised risk assessment report. 

Comment 106 

p.17 Under Risk Assessment, as I noted above, I recommend a different Fig. 1 and that 

you say something about the modern perspective on risk assessments that it is useful to 

consider management options. In addition there should be a bit more discussion of the 

purposes of prioritization (making the assessment more tractable, and, perhaps, guiding 

the testing program) 

Response 106 

We have made the suggested changes in the revised report. 

Comment 107 

67
 
 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

p.18 Here you could mention interindividual variability and variability in background 

and either here or in Model overview a mention of uncertainty characterization 

Response 107 

This variability information has been included in revised report. 

Comment 108 

p.19 Under Model overview I’ve noted that Fig 2 could be edited or, at least, the 

discussion could draw attention to the importance of inspection effects and effectiveness 

on the model. 

Response 108 

We appreciate the reviewer’s comment and we have made the suggested 

changes in the revised report. 

Comment 109 

p.19,20 Some refinement in the statements about per meal analysis is needed. 

Response 109 

This was addressed in the response to Comment 67. 

Comment 110 

p.22 In Model inputs, continuing from the discussion of Figure 2 some discussion of 

the input on regulatory analysis would be helpful. (incidentally it was not easy for me – 

an outsider effect to track down the ref to the change from 1994 to 2008) I think you 

have a good case for assuming that you could do as well for Salmonella in catfish as good 

practice for the other products, but I’d like some discussion of trends and whether the 

expected level of contamination affects your expectations. Also I think it is a much more 

open question whether the same sort of results apply to chemicals. And finally there is 

the question about whether for chemicals the inspections increase the probability of zero 

contamination or reduce the amount of high contamination. 

Response 110 

We agree with this comment. The revised risk assessment includes more 

discussion of program effectiveness, and as previously mentioned, 

focuses on Salmonella only (see also Response 1). 

Comment 111 

P22,23 For model outputs, I do wonder why chronic effects of Arsenic were not 

considered along with lead. 

Response 111 

68
 
 

 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

As previously mentioned (Response 14), we were limited to effects and 

chemicals for which dose-response functions can be derived. The revised 

risk assessment focuses on Salmonella only. 

Comment 112 

Hazard Identification 

p.23 I like the opening paragraph; it represents a good introduction to the kind of 

adaptive approach to management that we can expect of food inspection and that is now 

called for in contemporary risk assessment. 

p.24 As noted above, I wonder why Dioxins aren’t included given that there was a 

significant episode with catfish and dioxin. 

Response 112 

For reasons presented elsewhere (e.g. response 113), the revised risk 

assessment focuses on Salmonella only 

Comment 113 

p.25-27 Regarding prioritization of chemicals – I’d like a bit more qualitative 

discussion of the prioritization scheme (right now it seems too “insidery”). What were 

the ingredients and how was it done. Perhaps a box showing an example or two would 

work. And the list could be made into a more compact Table. 

p.28 And the same goes for microbial prioritization 

Response 113 

The initial draft risk assessment predicted – across all of the hazards considered 

– that Salmonella illnesses were the most frequent adverse human health 

occurrences. The revised risk assessment focuses exclusively on public health 

risk from Salmonella associated with the consumption of catfish. 

Comment 114 

Exposure Assessment 

p.30 Here we get into the issues of per meal exposure and Equation 3 raises the issue 

posed by the use of Pareto distributions – is prevalence some amount of contamination or 

only contamination above a level of detection? 

Response 114 

Prevalence is contamination above level of detection. This has been 

clarified in the updated report. 

Comment 115 

p.31 The Pareto fits require more (and better) justification 

Response 115 

Justification was provided in the original report. Also, see Response 16 

and Response 96. 

69
 
 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

Comment 116 

p.32 The nitrofurans discussion is another example of opportunities for adaptation 

Response 116 

Nitrofurans are not part of the revised risk assessment, which focuses on 

Salmonella. 

Comment 117 

p.33 I think more discussion is needed of the inorganic fraction of Arsenic (I wonder if 

that one paper (Li et al) really covers the range of possibilities); I assume the use of the 

triangular distribution is intended to reflect variability in the ratio; but perhaps more 

uncertainty should be considered at the end. 

Response 117 

Use of the distribution attempts to address the variability in the ratio of 

organic: inorganic arsenic. Again, this is not part of the revised risk 

assessment, which focuses on Salmonella. 

Comment 118 

p.34 Among the Pareto distribution mysteries is why the minimum level differs 

between imports and domestic contamination. And the infinite means are disturbing and 

merit a comment if you don’t reanalyze them. 

Response 118 

This is a reasonable comment; however the revised risk assessment 

focuses solely on Salmonella. In our estimates for Salmonella, we do not rely on 

a Pareto distribution assumption. 

Comment 119 

p.39 Regarding catfish consumption, more discussion is needed of how you balance 

NHANES numbers with NMFS numbers – there appears to be an inconsistency between 

the last three items in the table – and you should just say which #s you use for what. 

What would be interesting also is some discussion of trends – I suspect there has been an 

increase in catfish production/eating and it would be worth thinking about whether that is 

enough to affect the design of a program going forward. 

Response 119 

Table 6 has been revised to reflect changes in the revised draft risk 

assessment. The NMFS data was replaced with 2008 NASS catfish 

production data, because the NMFS data were out of date. In the original 

table 6 annual catfish servings were estimated by dividing the annual 

NMFS catfish production by the NHANES estimated mean serving size. 

70
 
 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

The idealized mean number of servings per year in the next column were 

estimated by dividing the annual number of catfish servings by the 

maximum estimated number of US catfish consumers from NHANES data. 

The last column in the table representing the actual number of catfish 

consumers used in the risk model is the estimated percent of annual 

catfish consumers multiplied by the total expected US population. 

When we examined the 8 years of NHANES data for catfish consumers 

using the single 24-hour recall variable, which was consistent over the 8 

years we were able to show that the percentage of catfish consumers 

evaluated at the four two-year survey release dates showed an increasing 

trend from 1999 through 2004. The percentages of the total 249 subjects 

were in order: 19.3%, 30.5%, 32.9%, and 17.3%. The trend was not 

significant because the last two-year period showed a decrease. The next 

release of data for 2007-2008 should be available soon and should 

provide new information on the catfish consumption trend. 

Comment 120 

p.40,41 Also the NHANES data suggest that there are high frequency catfish 

consumers the large proportion who have eaten a catfish meal in the last day. This 

should be accounted for as variability in consumption. 

Response 120 

This consumption variability is addressed in the uncertainty analyses. 

Comment 121 

p.43 The use of Table 7 merits reconsideration. At issue is whether for body weight 

you want to assume that catfish eaters differ from the rest of the population. You could 

make an argument for that, based on these data, but the samples, once sub-divided aren’t 

very large. A reasonable compromise might be to use those means but to use 

standardized standard deviations 

Response 121 

We chose to use the bodyweight distributions for the 1996-2006 NHANES 

respondents. 

Comment 122 

Hazard Characterization (Dose-Response) 

p.44 equation 6 is ok, provided that we have dealt with the possibility of multiple 

servings for acute hazards and variabilities for chronic disease 

p45,46 Under acute mortality need to rewrite to clarify the difference between 

uncertainty factors and variability in sensitivity 

Response 122 

71
 
 

 

 

 

 

 

 

 

 

Peer Review Comments and Responses to an Updated Risk Assessment of the Effect of an FSIS Catfish 

Inspection Program 

This comment is confusing – it may have stemmed from terminology 

differences used in the risk assessment and the toxicology fields. We 

believe the diction used in the report is consistent with standard risk 

assessment terminology. 

Comment 123 

p.49,50 Needs rewriting to consider non-linearity and interindividual variability in 

sensitivity and background levels 

Response 123 

Quantitative chemical analysis is no longer part of the updated report. 

Comment 124 

Risk Characterization 

p.51-53 Here the text needs to take account of the previous reservations on nonlinearities and distributions 

Response 124 

Clarification of these matters is provided in the revised risk assessment. 

Comment 125 

p.55 The discussion of imports and domestic effects should be clarified. 

Response 125 

Clarification of these matters is provided in the revised risk assessment. 

Comment 126 

p.73 I agree with the justifications for doing uncertainty by sensitivity. But I would 

have liked also a bit more qualitative discussion of model uncertainty, the uncertainty in 

choice of data sets, and the issue of the uniformity of inspection effect. 

Response 126 

Additional quantitative discussion has been added to the revised risk 

assessment. 

Comment 127 

Summary 

p.78 It would be desirable to have a more comprehensive conclusion including 

qualitative opportunities for improving the knowledge base, improving effectiveness of 

inspection programs, and more effective use of risk approaches in evaluation and 

adaptation. 

Response 127 

The revised risk assessment includes more discussion about use of a risk 

assessment model to support future decisions as data are generated by 

the inspection program. 

72