DRAFT – Do not cite, circulate or quote

DRAFT - Do not cite, circulate or distribute.

1

2

3

4

5

6

7

8

9

10

11 MICROBIAL 

12 RISK ASSESSMENT GUIDELINE

13

14 PATHOGENIC MICROORGANISMS 

15 WITH FOCUS ON FOOD AND WATER

16

17

18 Prepared by the Interagency Microbiological

19 Risk Assessment Guideline Workgroup

20

21

22

23

24 DRAFT

25 Version 5.3

26

27 June 20, 2011

28
Page ii Microbial Risk Assessment Guideline

1 DISCLAIMER

2

3 This guideline document represents the current thinking of the workgroup on the 

4 topics addressed. It is not a regulation and does not confer any rights for or on any person 

5 and does not operate to bind USDA, EPA, any other federal agency, or the public. It is 

6 within the discretion of each federal agency to determine whether or how to utilize this 

7 guideline.

8

9 This draft guideline is distributed solely for the purpose of pre-dissemination peer 

10 and public review under applicable information quality guidelines. It has not been formally 

11 disseminated by USDA or EPA. It does not represent and should not be construed to 

12 represent any agency determination or policy.

13

14 Mention of trade names or commercial products does not constitute endorsement 

15 or recommendation for use.

16

17
Microbial Risk Assessment Guideline Page iii

Draft – Do not cite, circulate, or quote

1 TABLE OF CONTENTS

2 Disclaimer .......................................................................................................................... ii

3 Interagency Workgroup Members................................................................................ vii

4 Preface............................................................................................................................. viii

5 1. Introduction.................................................................................................................. 1

6 1.1 What is Human Health Risk Assessment?....................................................... 1

7 1.2 What Led to the Decision to Create this MRA Guideline?............................ 2

8 1.3 What are the Benefits of Developing a Common MRA Guideline?.............. 4

9 1.4 When Can I Apply this MRA Guideline?........................................................ 4

10 1.5 What are the Fundamental Differences between Microbes and Chemicals?5

11 1.6 What is the Relationship of Infectious Disease as Applied to Human Health 

12 in a MRA?........................................................................................................... 8

13 1.7 What are the Benefits of Iterative MRA?........................................................ 9

14 1.8 How Does This Guideline Fit in with My Agency’s Current MRA 

15 Approaches and Practices? ............................................................................... 9

16 1.9 How is this MRA Guidance Related to Other MRA 

17 Frameworks/Guidelines that are Currently Available?............................... 10

18 1.10 What are Major Principles for MRA?........................................................... 11

19 2. Planning and Scoping ................................................................................................ 15

20 2.1 What is Planning and Scoping?...................................................................... 15

21 2.1.1 What is Problem Formulation?............................................................ 16

22 2.2 What are the Benefits of Planning and Scoping?.......................................... 16

23 2.3 What do I Consider When Deciding to Initiate a MRA? ............................. 17

24 2.4 Who Can be Involved with Planning and Scoping? ..................................... 19

25 2.5 How Can the MRA be Used? .......................................................................... 20

26 2.5.1 What “Depth” Can I go into in the Risk Assessment?....................... 21

27 2.5.2 What Are Examples of Types of MRA?.............................................. 22

28 2.5.3 What are Threat and Vulnerability Assessments for High29 Consequence Biological Events? .......................................................... 23

30 2.6 What is Discussed During Planning and Scoping and What Products 

31 Emerge? ............................................................................................................ 25

32 2.6.1 What are Risk Management Questions and What is the Charge? ... 29

33 2.6.2 What is a Risk Profile? ......................................................................... 30

34 2.6.3 What is a Conceptual Model? .............................................................. 30

35 2.6.4 What is an Analysis Plan? .................................................................... 31

36 2.6.5 How are Data Gaps Identified and Addressed in the Context of 

37 Planning and Scoping?.......................................................................... 31

38 2.6.6 How do I Consider Information Quality Including Data Quality? .. 35

39 2.6.7 What is a Value-of-Information Analysis? ......................................... 38

40 2.6.8 What is a Communications Plan?........................................................ 39

41 2.7 What Types of Decisions within Risk Assessment are Science Policy?....... 39

42 2.8 What is My Role as a Risk Assessor in Risk Assessment? ........................... 41

43 3. Hazard Identification and Hazard Characterization ............................................. 43

44 3.1 How do I Define the Hazard? ......................................................................... 43

45 3.2 What are Hazard Identification and Hazard Characterization? ................ 44

46 3.3 What Hazard Characteristics Can I Consider?............................................ 45
Page iv Microbial Risk Assessment Guideline

1 3.4 How do Microbial Hazards Cause Adverse Outcomes? .............................. 46

2 3.4.1 What does Virulence and/or Pathogenicity Mean in the Context of 

3 Causing an Adverse Outcome? ............................................................ 47

4 3.5 What are the Mechanisms that May Lead to the Development of New 

5 Pathogens or Pathogens with New Traits? .................................................... 47

6 3.6 What are the Major Categories of Microorganisms?................................... 49

7 3.7 What Methodological Approaches are Used to Identify and Quantify 

8 Microorganisms?.............................................................................................. 52

9 3.8 Are there any Special Concerns Regarding Microbial Detection Methods?

10 ............................................................................................................................ 53

11 3.9 What Host Factors Can I Take into Consideration?.................................... 56

12 3.10 How does Life Stage Affect Sensitivity to Infection and Disease 

13 Manifestation?.................................................................................................. 58

14 3.11 What Environmental Factors Can I Take into Consideration? .................. 59

15 4. Dose-Response Assessment ....................................................................................... 61

16 4.1 What is Dose-Response Modeling and What are Some General 

17 Considerations for Dose-Response Modeling?.............................................. 61

18 4.1.1 How do I Choose Between Modeling a Discrete Dose Versus an 

19 Average Dose?........................................................................................ 62

20 4.1.2 What is the One-Hit Model and Why is it the Preferred Model?..... 63

21 4.1.3 What Important Factors Can I Consider in Dose-Response 

22 Assessment? ........................................................................................... 65

23 4.1.4 How Can I Model the Spread of Disease in the Population?............. 70

24 4.1.5 What Can I Address for Each Model to Improve Transparency? ... 71

25 4.2 What is Current Practice in Quantitative Dose-Response Modeling for 

26 Microbial Illness?............................................................................................. 72

27 4.2.1 What Models Can I Use for Microbial Dose-Response Assessment?72

28 4.2.2 What is the Output of a Dose-Response Assessment?........................ 79

29 4.2.3 How do I Fit Models to Existing Dose-Response Data?..................... 80

30 4.2.4 How Can I Evaluate Uncertainty in Dose-Response?........................ 82

31 4.2.5 What is Variability in Dose-Response? ............................................... 82

32 4.2.6 How Can I Account for Life Stages and Subpopulations in Dose33 Response Models?.................................................................................. 83

34 4.2.7 Can I Use Uncertainty, Modifying, or Adjustment Factors in a 

35 Microbial Dose-Response Assessment? ............................................... 84

36 4.2.8 What Modeling Methods are on the Horizon? ................................... 85

37 5. Exposure Assessment................................................................................................. 86

38 5.1 What are General Concepts in Exposure Assessment?................................ 86

39 5.1.1 What is an Exposure Assessment?....................................................... 86

40 5.1.2 What is Exposure?................................................................................. 87

41 5.1.3 What are Sources, Pathways, and Routes of Exposure? ................... 87

42 5.1.4 What Environmental Factors Can I Take into Consideration?........ 90

43 5.1.5 What is an Exposure Scenario? ........................................................... 91

44 5.1.6 What are Qualitative and Quantitative Exposure Assessments? ..... 91

45 5.1.7 What is Variability in Exposure Assessment? .................................... 92

46 5.1.8 What is Uncertainty in Exposure Assessment? .................................. 92
Microbial Risk Assessment Guideline Page v

Draft – Do not cite, circulate, or quote

1 5.1.9 What is a Deterministic Exposure Assessment?................................. 93

2 5.1.10What is a Stochastic Exposure Assessment?....................................... 94

3 5.1.11What is Monte Carlo Analysis? ........................................................... 94

4 5.1.12How does Exposure Assessment Fit with the Other Components of 

5 Risk Assessment?................................................................................... 96

6 5.1.13Do Different Exposure Scenarios Always Generate Different 

7 Microbial Doses? ................................................................................... 97

8 5.2 How do I Develop an Exposure Assessment? ................................................ 99

9 5.2.1 What is the Purpose of the Risk Assessment? .................................... 99

10 5.2.2 Which Scenarios Can I Consider? ..................................................... 100

11 5.2.3 What are the Exposed Populations I Could Consider? ................... 105

12 5.2.4 What are Common Approaches to Exposure Modeling I Can Use?

13 ............................................................................................................... 106

14 5.2.5 How is Scenario Analysis Used in Exposure Assessment? .............. 109

15 5.2.6 What is the Role of Predictive Microbiology in Exposure 

16 Assessment? ......................................................................................... 114

17 5.2.7 How Can I Address Secondary Transmission of Disease in the 

18 Population? .......................................................................................... 117

19 5.2.8 What Data Can I Use in an Exposure Assessment? ......................... 118

20 5.2.9 How do I Use Data in an Exposure Assessment? ............................. 120

21 5.3 How do I Analyze a Model’s Results?.......................................................... 121

22 5.3.1 How do I Report Exposure in an Exposure Assessment?................ 122

23 5.3.2 How do I Determine a Change in Exposure and Subsequent Risk?123

24 5.3.3 What is Sensitivity Analysis?.............................................................. 124

25 5.3.4 What is an Uncertainty Analysis?...................................................... 125

26 5.4 What Can I Put Into an Exposure Assessment Report? ............................ 127

27 5.5 What are Possible Future Developments in Exposure Assessment?......... 128

28 6. Risk Characterization.............................................................................................. 130

29 6.1 What is Risk Characterization? ................................................................... 130

30 6.2 What are the Elements in a Risk Characterization? .................................. 132

31 6.3 How Do I Prepare a Risk Characterization? .............................................. 135

32 6.4 Are All Risk Characterizations Quantitative and What Do I Do When 

33 Quantitative Data are Unavailable for Some Elements of the Risk 

34 Characterization? .......................................................................................... 137

35 6.5 Are There Different Forms of Risk Characterization? When Do I Apply 

36 Them?.............................................................................................................. 137

37 6.5.1 When is a Static Model Appropriate?................................................. 139

38 6.5.2 When is a Dynamic Model Appropriate? ........................................... 140

39 6.6 How are Sensitivity and Uncertainty Analyses Related to Risk 

40 Characterization? .......................................................................................... 142

41 6.7 How are Quality of Life Measures Important in MRA?............................ 144

42 6.8 How Can a Risk Assessment be Validated? ................................................ 146

43 7. Risk Management .................................................................................................... 148

44 7.1 What is Risk Management? .......................................................................... 148

45 7.2 When and How Can Risk Managers be Involved in Risk Assessments?.. 150
Page vi Microbial Risk Assessment Guideline

1 7.3 How are Risk Management Options a Useful Component to Include in a 

2 Risk Assessment? ........................................................................................... 152

3 7.4 What are Some Other Inputs into Risk Management Decisions About 

4 Controlling or Accepting Risks?................................................................... 152

5 7.5 What are Some Operational Risk Management Tools and Approaches? 155

6 7.6 What is Risk Management for the Intentional use of Regulated 

7 Microorganisms?............................................................................................ 156

8 8. Risk Communication ............................................................................................... 157

9 8.1 What is Risk Communication?..................................................................... 157

10 8.2 What are the Benefits of Risk Communication?......................................... 157

11 8.3 Who are the Stakeholders of MRAs?........................................................... 158

12 8.4 With Whom Can I Communicate?............................................................... 159

13 8.5 When Can the Process of Risk Communication Begin? ............................ 159

14 8.6 Can I Communicate in Writing, Orally, or Both?...................................... 159

15 8.7 Who Decides What to Communicate? ......................................................... 160

16 8.8 What Information Can be Communicated?................................................ 160

17 8.9 How is the Communication Process a Continuous Dialog? ....................... 161

18 8.10 How In-Depth Can I Communicate? ........................................................... 162

19 8.11 What Can I Do if the Message is not “Getting Through?”........................ 162

20 8.12 How Can I Communicate Risk Successfully? ............................................. 163

21 8.13 How Can I Handle Media and Congressional Office Requests? ............... 164

22 8.14 When Can Risk Communication End?........................................................ 164

23 9. Glossary .................................................................................................................... 166

24 10. Abbreviations......................................................................................................... 175

25 11. References............................................................................................................... 178

26 Appendix A Example Assumptions................................................................................. 1

27 Appendix B Hazard Identification Questions................................................................ 1

28

29

30
Microbial Risk Assessment Guideline Page vii

Draft – Do not cite, circulate, or quote

1 INTERAGENCY WORKGROUP MEMBERS

2

3 Kerry Dearfield, Co-Chair USDA/FSIS

4 Steve Schaub, Co-Chair EPA/OW (retired)

5 Nicholas Ashbolt, Co-Chair EPA/ORD

6 Irwin Baumel EPA/ORD

7 Michael Broder EPA/ORD

8 Uday Dessai USDA/FSIS

9 Moshe Dreyfuss USDA/FSIS

10 Eric Ebel USDA/FSIS

11 Brendlyn Faison EPA/OW

12 Julie Fitzpatrick EPA/ORD

13 Joel Gagliardi EPA/OPP

14 Frank Hearl CDC/NIOSH

15 Abdel Kadry EPA/ORD

16 Janell Kause USDA/FSIS

17 Barbara Klieforth EPA/OSA

18 Ken Martinez CDC/NIOSH

19 Robert McDowell USDA/APHIS

20 Stephen Morse CDC/NCEZID

21 Tonya Nichols EPA/ORD

22 Mark Ott NASA

23 Duane Pierson NASA

24 William Schneider EPA/OPP

25 Carl Schroeder USDA/FSIS

26 Mark Segal EPA/OPPT

27 Sean Shadomy CDC/NCEZID

28 Jeff Swartout EPA/ORD

29 Sarah Taft EPA/ORD

30 Brandolyn Thran AIPH

31 Elizabeth (Betsy) Weirich CDC/NCEZID

32

33 Other contributors (not currently on workgroup): Bonnie Gaborek (USACHPPM); Myra 

34 Gardner (USDA/FSIS); Alecia Naugle (USDA/FSIS); Geoff Patton (EPA/ORD); Gary 

35 Bangs (EPA/RAF); Deborah McKean (EPA/ORD); Gregory Stewart (State); Parmesh 

36 Saini (USDA/FSIS); Gregg Claycamp (FDA/CDER)

37

38 Contractor support: Audrey Ichida (ICF International), Jeff Soller (Soller 

39 Environmental), Margaret Coleman (ICF International), and Heather Simpson (ICF 

40 International).

41

42

43

44
Page viii Microbial Risk Assessment Guideline

1 PREFACE

2

3 This Microbial Risk Assessment (MRA) Guideline is written to be a resource for 

4 microbial risk assessors at the U.S. Department of Agriculture‘s Food Safety and 

5 Inspection Service and the U.S. Environmental Protection Agency and their agents, 

6 contractors, and the general risk assessment community with which these agencies 

7 interact. Other Federal agencies expressed interest in the development of this guideline 

8 and provided experts to participate in this interagency effort (see list of participants). 

9

10 In recognition of the needs and mandates of the participating Agencies and the 

11 various statutory authorities that may apply to microbial risk assessment (MRA), this 

12 Guideline emphasizes the need for a flexible template for conducting microbial risk 

13 assessment. It provides general, broad fundamental risk assessment principles 

14 specifically for microbial risks, but is not intended to be prescriptive nor is it intended to 

15 supplant the internal practices or policies of any Federal agency. Users have the 

16 flexibility to adapt pertinent sections to relevant statutory authorities and purposes if 

17 needed. On the individual level, it is written for persons with some knowledge of

18 microbiology and basic understanding of risk assessment principles but some basics may 

19 be presented at the introduction to a topic. The intention is that this Guideline can be 

20 periodically updated, particularly as more information becomes available. 

21

22 This Guideline primarily focuses on infectious diseases associated with the 

23 gastrointestinal tract and fecal or oral transmission of the causal agents mainly in food 

24 and water, but clearly has application to other scenarios. 

25

26 Many human pathogens found in food, water, and the environment cause acute 

27 diseases that have short incubation periods, symptoms typically lasting several days to a 

28 week, and usually non-lethal effects with complete recovery from the illness. However, 

29 some pathogens associated with the gastrointestinal tract may cause more serious 

30 diseases or sequelae such as reactive arthritis, cancer, Guillain-Barré syndrome, and 

31 juvenile onset diabetes which may have long-term implications. Further, there are 

32 indigenous water-based pathogens, such as Legionella spp. and Mycobacterium spp., that 

33 grow in biofilm environments and their inhalation via aerosols may cause pneumonia, 

34 which can be fatal. Applying risk assessment approaches associated with MRA 

35 procedures discussed in this guideline should help risk assessors characterize the 

36 common exposure sources, causative agents, associated symptoms, contributing 

37 immunity factors, and other common threads contributing to chronic illness.

38

39 The Guideline does not specifically include biological warfare agents, airborne 

40 microbial hazards, or agriculturally or industrially important microorganisms, 

41 oligonucleotides, prions, preformed microbial toxins, and other submicrobial entities. 

42 These agents have many unknowns associated with their sources, modes of ―infection‖ 

43 and disease, transmissibility, and survivability. Nonetheless, information in this 

44 guideline may provide information to risk assessors addressing these issues. 

45
Microbial Risk Assessment Guideline Page 1

Draft – Do not cite, circulate, or quote

1 1. INTRODUCTION

2

3

4 Microbial risk assessments (MRA) are a valuable tool for Federal Agencies 

5 tasked with understanding, reducing, and preventing risks presented by hazardous

6 microorganisms, whether natural or anthropogenic, intentional or unintended. Increasing 

7 globalization has compounded these risks, with the broadening and often rapid 

8 distribution of illnesses. Clear and credible risk assessment methods are proving ever 

9 more necessary for agencies to address both current and future risks associated with 

10 contamination of air, water, soil and food by bacteria, fungi, protozoa, viruses, and their 

11 toxins.

12

13 This guideline is intended to layout an overarching approach to conducting 

14 microbial risk assessment and to introduce the users to tools and methods available to 

15 them. Additionally, it is expected to promote consistency and improve transparency in 

16 the way -microbial risk assessments are conducted. Nonetheless this Guideline is not 

17 intended to replace existing guidelines that are in use by Agencies. The decision to apply 

18 methods and approaches in this guideline, either totally or in part, is left to the discretion 

19 of the individual department or agency. This document is offered to provide information 

20 that may be useful for microbial risk assessors.

21

22

23 1.1 What is Human Health Risk Assessment?

24

25 Risk assessment is widely recognized as a systematic way to prepare, organize, 

26 and analyze information to help make regulatory decisions, establish programs, and 

27 prioritize research and development efforts. In Risk Assessment in the Federal 

28 Government; Managing the Process, published by the National Research Council of the 

29 National Academies of Science (NRC, 1983; hereafter referred to as the ―NRC 1983 

30 report‖), risk assessment is defined as ―the qualitative or quantitative characterization of 

31 the potential health effects of particular substances on individuals or populations,‖

32 structured to include a hazard identification (HI), dose-response assessment, exposure 

33 assessment, and risk characterization. Similarly, the Codex Alimentarius Commission, 

34 established by the United Nations Food and Agriculture Organization (FAO) and World 

35 Health Organization (WHO) and recognized by the World Trade Organization as the 

36 relevant organization for international food safety standards and guidelines, defined risk 

37 assessment as ―a scientifically based process consisting of the following steps: (i) hazard 

38 identification, (ii) hazard characterization, (iii) exposure assessment, and (iv) risk 

39 characterization‖ in Principles and Guidelines for the Conduct of Microbiological Risk 

40 Assessment (Codex, 1999; hereafter referred to as the Codex framework).

41

42 The risk assessment process is used to facilitate the application of science to 

43 policy decisions. Risk assessment at the Federal level informs the risk management 

44 decision making process and risk communication through organized scientific analyses of 

45 data related to a specified hazard. Risk assessment can also evaluate potential or 

46 proposed risk management strategies‘ impact on public health. Essentially, a MRA is the 
Page 2 Microbial Risk Assessment Guideline

1 formal, scientifically based process to estimate the likelihood (probability) of exposure to 

2 a microbial hazard and the resulting public health (and/or environmental) impact from 

3 this exposure. Risk assessment not only includes the likelihood of exposure and the 

4 impact of that exposure, but also includes steps for planning, scoping, and hazard 

5 identification and characterization (see Section 1.4). This Guideline will only focus on 

6 MRA conducted for public health purposes. 

7

8 Risk assessment, risk management, and risk communication are the three 

9 components of a risk analysis. Figure 1.1 is a representation of how these terms are 

10 related. The three activities are shown as partially overlapping because they depend upon 

11 each other and draw from each other. This Guideline addresses risk management and risk 

12 communication from the perspective of what a risk assessor should know about those 

13 disciplines and provides transparency to decision makers and stakeholders regarding how 

14 a MRA is conducted. Therefore, the risk communication and risk management chapters 

15 in this Guideline provide only a small portion of what those disciplines contribute to risk 

16 analysis as a whole.

Risk Analysis

Risk 

Assessment

Risk 

Management

Risk 

Communication

17

18 Figure 1.1 Risk Assessment, Management, and Communication in Risk Analysis 

19

20 1.2 What Led to the Decision to Create this MRA Guideline?

21

22 Government Agencies have conducted formal risk assessments for chemicals in 

23 food, water, and the natural environment for decades. These assessments had their 

24 origins in support of or in response to the establishment of a number of laws and 

25 regulations that required the federal government to control chemical contaminants in food 

26 and environmental media. The NRC 1983 report helped unify the risk assessment 

27 processes for chemicals in foods and the environment and provided a framework that

28 federal agencies, their clients, and the risk assessment community in general could apply 

29 in conducting risk assessments. Since then, the NRC 1983 report has been used by 

30 regulatory agencies as guidance in conducting risk assessments. 
Microbial Risk Assessment Guideline Page 3

Draft – Do not cite, circulate, or quote

1

2 The need for a common chemical risk assessment approach was established in the 

3 1980s, before there was a similar requirement for a MRA approach. At that time, MRA 

4 limitations included lack of data, tools, and methods, such as limited dose-response 

5 models, poor quantification of microbial occurrence, limited analytical methods 

6 (sensitivity, specificity, precision, and accuracy), and poor understanding of human 

7 immunological responses. Since the 1990s, the use of MRAs has gained significant 

8 credibility in the federal regulatory community, as new information on the identification 

9 of infectious microbial pathogens, their occurrence, potential for human exposure, dose10 response, and attributable health effects became increasingly available. A number of 

11 credible mathematical models, protocols, and other tools have become available that 

12 allow MRAs to be conducted even with substantial variability, uncertainty, and lack of 

13 specific data – issues that previously would have impeded preparation of quantitative risk 

14 assessments. Indeed methods are now available to specifically capture and report 

15 variability and uncertainty associated with data use in MRA. During the 1990s, it also 

16 became apparent that the NRC 1983 report had some shortcomings for conducting MRAs

17 since chemicals are different from microorganisms in a number of ways (see Section 1.5). 

18 The NRC 1983 report provides a very useful foundation for chemical and microbial risk 

19 assessment, but does not provide the level of detail that is required to conduct a 

20 quantitative microbial risk assessment. Chemical risk assessors developed more detailed 

21 guidelines to expand on the NRC 1983 report. The detailed chemical risk assessment 

22 protocols clearly would not work for microorganisms, so it became necessary for MRA to 

23 consider microbial aspects that are not considered in the NRC 1983 report. While 

24 Agencies conducting MRAs had for a time continued to generally rely on the NRC 1983 

25 report, they have individually made adjustments to adapt it for MRA. For example, the 

26 FAO of the United Nations and the WHO use the international Codex framework, which 

27 follows the same overall structure as the NRC 1983 report, to conduct a MRA of Listeria

28 monocytogenes in ready-to-eat foods (FAO/WHO, 2004) and the Environmental 

29 Protection Agency (EPA) used the framework in the NRC 1983 report to evaluate the 

30 public health impact of drinking water regulations for Cryptosporidium oocysts (EPA, 

31 2006a).

32

33 The workgroup that created this Guideline did so with the intent of leveraging 

34 limited resources, improving efficiencies, promoting joint interaction, exploiting their 

35 individual experiences, and improving consistency in the way the microbial risk 

36 assessments are done and thereby fostering greater transparency. This Guideline does not 

37 contain every tool that can be used in MRA, but it does include the most widely used 

38 tools. This Guideline also is considerably longer and more detailed than the MRA 

39 frameworks that precede it (Codex, 1999; ILSI, 2000; Codex, 2007a, 2007b). EPA 

40 Office of Water‘s Protocol for Microbial Risk Assessment and the EPA Science Advisory 

41 Board‘s review of that document were also considered during the development of this 

42 document (EPA, 2009a).

43

44
Page 4 Microbial Risk Assessment Guideline

1 1.3 What are the Benefits of Developing a Common MRA Guideline? 

2

3 Agencies that need to be concerned about pathogens often have similar 

4 requirements to protect the health of potentially exposed people. For example, a number 

5 of pathogens of concern originate in the gastrointestinal tract of humans and animals and 

6 can potentially contaminate food, surface water, or drinking water. The agencies that 

7 regulate food and environmental contaminants recognize that the ultimate sources of 

8 pathogens are the same for different media. Because the health effects and dose-response 

9 relationships are similar among different matrices for many of the pathogens, it is useful 

10 to have common principles and approaches to assess risks across media and exposure 

11 settings.

12

13 This Guideline facilitates systematic and transparent consideration of all relevant 

14 factors that impact the risk assessment and facilitates reproducible risk evaluation. 

15 Agencies assessing a similar medium or pathogen can more readily compare and contrast 

16 the details and assumptions of their assessment to another Agencies‘ assessment. This is 

17 not to say that each risk assessment will be completely cross-comparable with other risk 

18 assessments, since there are a number of specific data sources and Agency requirements 

19 that require different inputs and applications. The difference in requirements is why this

20 Guideline is designed to be modular and able to provide flexibility for each Agency‘s 

21 specific requirements. On an international scale, there is also a need to have common 

22 approaches to MRA to effectively satisfy international trade agreements and public health 

23 protection for importation of food and beverage products and assess international risks 

24 from emerging pathogens around the world. 

25

26 This Guideline is the result of the collaborations of microbial risk assessors from 

27 a number of Federal Agencies (USDA/FSIS, USDA/APHIS, EPA/OW, EPA/ORD, 

28 EPA/OPP, EPA/OPPT, EPA/OSA, FDA/CDER, CDC/NIOSH, CDC/NCEZID, NASA, 

29 USACHPPM) who have dedicated their time and expertise to prepare a general guideline 

30 that all microbial risk assessors in the Government, their contractors, and the risk 

31 assessment community in general can apply for conducting MRAs of infectious agents or 

32 disease-causing microorganisms. The workgroup that developed this Guideline agreed 

33 that this document must be general enough so that any Agency, that chooses to do so, can

34 use it effectively. By being based on general and modular guidelines, each specific risk 

35 assessment component can then be supplemented (or modified) to suit the particular 

36 legislative, legal, or exposure factor needs of an agency. 

37

38 The Office of Management and Budget‘s (OMB) Good Guidance Practices has 

39 been followed as this Guideline was developed (OMB, 2007a).

40

41 1.4 When Can I Apply this MRA Guideline?

42

43 This guideline is applicable to a wide array of scenarios but focuses on 

44 microorganisms that are capable of causing infection and disease in the human 

45 population. Specifically it is applicable for assessing risk associated with ingestion of 

46 foodborne pathogens (in raw and processed foods), and water-based or waterborne
Microbial Risk Assessment Guideline Page 5

Draft – Do not cite, circulate, or quote

1 pathogens (in drinking water, recreational water, wastewater, etc.). I could also be 

2 applied to assessing risk of human pathogens in soil and solid wastes or transported 

3 through the air, and human exposure to biological warfare agents. The Guideline may 

4 also apply for other common forms of exposure including inhalation and dermal exposure 

5 pathways. At present, oligonucleotides, prions, preformed microbial toxins, and other 

6 submicrobial entities are not covered under the Guideline owing to a wide array of 

7 unknowns associated with those agents.

8

9 This Guideline should help you account for differences between the general 

10 population and different life stages and sensitive populations. It is flexible enough that 

11 risks to individuals, subpopulations, and the general population can be dealt with using 

12 either available static or dynamic susceptibility models. The approaches and tools 

13 provided by this Guideline should be appropriate for typical human health related 

14 concerns. The Guideline does not include criteria to specifically identify sensitive groups 

15 or life stages, because those groupings are specific to each risk assessment, based on the

16 Agencies‘ unique health protection concern and public health goal.

17

18 The intent of the Guideline is to provide the necessary elements for successfully 

19 conducting a full risk assessment, quantitative or qualitative, including:

20

21 a) Planning and scoping including problem formulation,

22

23 b) Hazard identification (HI),

24

25 c) Hazard characterization (HC),

26

27 d) Dose-response assessment,

28

29 e) Exposure assessment, and

30

31 f) Risk characterization.

32

33 This Guideline incorporates these elements and follows the Codex definition for 

34 the risk assessment process (see Section 1.1), except that the qualitative aspects of hazard 

35 characterization are presented with hazard identification, instead of with dose-response

36 (Codex, 1999). It is intended that the Guideline will also assist risk assessors to meet the 

37 needs and expectations of risk managers, and provide appropriate and effective risk 

38 communication with managers, stakeholders, and the public.

39

40 1.5 What are Some Fundamental Differences between Microbes and Chemicals?

41

42 The differences between chemical hazards and human susceptibility factors

43 considered in chemical risk assessments and microbial-host factors in MRAs can be few 

44 or many depending on the microbial antagonist, host resistance factors, and exposure 

45 considerations. While these two types of risk assessments are conceptually similar, there 

46 are typically enough differences between chemicals and microorganisms that having an 
Page 6 Microbial Risk Assessment Guideline

1 approach specifically detailing unique microbial considerations is very useful. Even 

2 though the uniqueness of each chemical is considered individually in chemical risk 

3 assessment, some significant differences in MRAs to be considered include:

4

5 a) Microbial growth and death – Pathogens increase and decrease in number in 

6 the environment and host. Different species, and even different strains within 

7 a pathogenic species, grow and die in unique patterns. In contrast, although 

8 chemicals can bioaccumulate, bioconcentrate and undergo transformations, 

9 they do not multiply in the environment or hosts. Both chemicals and 

10 pathogens can decrease due to environmental factors; chemicals can be 

11 transformed or degraded, and pathogens can die or become unculturable but 

12 may remain infectious. Some decrease in chemicals may be reversible (e.g., 

13 sequestered toxicants may remobilize). However, microbial toxins can remain 

14 after the organism dies and some enterotoxins are heat stable and resistant to 

15 degradation.

16

17 b) Host immunity and susceptibility – Although body weight, age, and 

18 metabolic capacity differences are considered in chemical risk assessment, 

19 genetic and acquired differences in susceptibility are not considered in 

20 chemical risk assessment in the same manner as in MRA. Whereas 

21 uncertainty factors or assessment using data from known sensitive populations 

22 may be used to consider these host differences in chemical risk assessment, in 

23 MRA immune status can be included in a dynamic model. The immune 

24 system may be considered in chemical risk assessment if a chemical causes a 

25 hypersensitivity reaction. Infection and resulting illness due to pathogens is, 

26 in some cases, highly dependent on the immune status of the individual, which 

27 can fluctuate based on time since last exposure, presence of concurrent 

28 infections, and a number of other factors (e.g., life stage, nutrition, genetics).

29

30 c) Diversity of health endpoints – The same dose of a pathogen may result in a 

31 broad range of health outcomes or endpoints depending on the characteristics 

32 of the host and the exposure scenario. Endpoints for the same dose could 

33 include: no effect, asymptomatic infection, mild or severe symptoms, or 

34 mortality for the most susceptible. The percentage of potential hosts in each 

35 category fluctuates over time.

36

37 d) Genetic diversity and evolution of microbial strains – Microorganisms are 

38 genetically diverse and allelic ratios in a population can change significantly 

39 within a few generations. In addition, microbial genomes can evolve quickly 

40 (within days or weeks) through mutation or vertical gene transfer (within a 

41 species) or horizontal gene transfer (between different species, families, and 

42 higher taxonomic differences). Strains of the same species (e.g., 

43 Cryptosporidium parvum, E. coli) can have multiple genotypes, potentially 

44 with different virulence for human hosts. Some pathogens (e.g., Helicobacter 

45 pylori, many viruses) behave like quasi-species, which are fluctuating 

46 populations of genetically distinct variants that co-exist within a single host 
Microbial Risk Assessment Guideline Page 7

Draft – Do not cite, circulate, or quote

1 (Boerlijst et al., 1996; Covacci and Rappuoli, 1998). Microbes represent a 

2 ―moving target,‖ because the distribution of strains and virulence factors can 

3 fluctuate rapidly in a given medium. Although there is great diversity among 

4 chemicals, particularly considering mixtures, the mechanism of change is not 

5 evolution.

6

7 e) Potential for secondary transmission – Microbial infections can be 

8 transmitted between individuals, and even to some animals. With the 

9 exception of the mother-fetus relationship, chemicals in tissues of exposed 

10 individuals are not known to transmit to other individuals. Chemicals that are 

11 on an exposed individual‘s clothing or skin can be transferred to household 

12 and other inanimate objects (fomites), but that transfer results in dilution of 

13 the chemical, as opposed to pathogen secondary transmission, which can 

14 amplify the pathogen. Some microbes can remain viable for days, weeks, or 

15 months on surfaces, which increases the potential for transmission. For some 

16 pathogens humans can become asymptomatic chronic carriers and can infect 

17 others and contaminate food and water sources with displaying symptoms 

18 themselves for prolonged periods.

19

20 f) Heterogeneous spatial and temporal distribution in the environment –

21 Pathogens are typically heterogeneously distributed in environmental 

22 matrices. Pathogen growth may lead to clustered distributions, and pathogens 

23 may physically clump together or may be embedded in or attached to organic 

24 and inorganic particulate debris, making concentration determinations 

25 difficult. Although concentration in pipe scale and biofilms is also a problem 

26 for chemical contaminants, some pathogens can grow and/or be protected in 

27 these environments. Also, many types of pathogens occur only episodically

28 and typically can be found only during short-lived disease outbreaks (i.e., 

29 epidemics) in a community. Seasonal and event-related (wet weather) spikes 

30 are common.

31

32 g) Single exposure health outcome – Chemical risk assessments are conducted

33 for acute exposures that cause immediate health outcomes and for chronic 

34 exposures with long-term health effects. For chronic exposure to chemicals,

35 the risk may be from daily exposure over a 70-year lifespan, whereas for 

36 pathogens the risk may be from a single exposure with health effects

37 noticeable within days or weeks. Some pathogens may have sequelae, which 

38 are health outcomes that appear much later than the original symptoms. Some 

39 sequelae are chronic (long lasting). Unlike the long-term exposures often

40 considered for chemicals, longer-term risks due to pathogen exposure are not 

41 typically considered for MRA.

42

43 h) Wide range of microbial response to interventions – Many risk assessments 

44 address risks to human health associated with media that have been subjected 

45 to some sort of treatment (such as wastewater treatment for water or strict 

46 processing of foods). Microorganisms respond with wide variability to
Page 8 Microbial Risk Assessment Guideline

1 environmental and treatment factors. For example, response to drinking water 

2 treatment needs to be taken into account when comparing microbial levels in 

3 ambient water and treated drinking water. However, in the Clean Water Act 

4 304(a) ambient water quality criteria, EPA makes a policy assumption that 

5 drinking water treatment has no effect on chemical concentration when they 

determine what levels to set for ambient water (EPA, 2000c).16

7

8 i) Detection method sensitivity – Microbial detection methods may not be 

9 sensitive enough to detect pathogens at a level of regulatory concern. This is 

10 not necessarily the case for all pathogens or all media, but does apply to some 

11 combinations of organisms and media. Theoretically, a single pathogenic 

12 organism can cause infection and lead to illness. Analytical methods for 

13 detecting low levels of pathogens (e.g., one organism in 1000 liters of water) 

14 are not sufficiently developed to be reliable. In short, the human body is a 

15 more sensitive detector of pathogens than many laboratory methods. In 

16 addition, the active or viable-but-not-culturable (VBNC) state is not detectable 

17 by traditional culture-based laboratory methods (see Section 3.8)

18

19 j) Population, community, and ecosystem-level dynamics – Microbial 

20 pathogens have complex interactions with other members of their species, 

21 other species, and the abiotic environment. For example, pathogens compete 

22 with non-pathogens for resources, and many non-viral human pathogens have 

23 animal hosts that can greatly complicate the ecological dynamics of pathogen 

24 occurrence.

25

26 k) Routes of exposure – Many routes of exposure are similar for chemicals and 

27 microorganisms; however, there are some potentially important differences. 

28 Dermal exposure may be important for some chemicals but not necessarily 

29 with microbial exposure since unbroken skin is a natural barrier for entry. 

30 However, dermal contamination with pathogens can lead to oral exposure via 

31 transfer to consumed food or water. Other aspects may include consideration 

32 of direct person-to-person or person-to-environment-to-person routes.

33

34 These factors are significant considerations in MRA, but the approach for dealing with 

35 each one may be unique to a particular risk assessment scenario.

36

37 1.6 What is the Relationship of Infectious Disease to Human Health as Applied

38 in a MRA?

39

40 Understanding the relationships and interactions between a microbial pathogen, 

41 its host, and the exposure to the pathogen in the environment is key to determining the 

42 potential health impact a pathogen will have on an individual or population. The 

43 epidemiological triangle (disease triad) illustrates the inter-relationship between the host, 

44 pathogen, and environment components (Figure 1.2). For each of the components, there 

45 are several key factors that may be considered. In many cases, a comprehensive 

 

1 Except for disinfection byproducts
Microbial Risk Assessment Guideline Page 9

Draft – Do not cite, circulate, or quote

1 quantitative treatment of all the factors to conduct a MRA is not possible because of data 

2 limitations. However, the factors can be at least considered qualitatively. The various 

3 points of the disease triad will be discussed throughout this Guideline.

4 Figure 1.2 The Epidemiological Triangle

5

6

7 1.7 What are the Benefits of Iterative MRA?

8

9 There is general recognition by risk assessors that an iterative approach is 

10 necessary when conducting a MRA, so that there is opportunity to modify the risk 

11 assessment based upon changes in data availability and evolving agency policies

12 (Presidential/Congressional Committee [P/CC], 1997). Often, the lack of data, new data 

13 or interpretations, or uncertainty or variability in information will require you to revisit 

14 the original charge or premise for conducting a risk assessment. As revisions are 

15 conducted, new information can be incorporated, especially in areas found to be the most 

16 important to assessing risk. This ensures that the risk manager is provided with the most

17 accurate interpretation of risks, so that appropriate management decisions can be made. 

18 Risk communication strategies can also be developed in parallel with risk assessment 

19 iterations. You should be prepared to be involved in the risk communication process as 

20 needed (see Chapter 8).

21

22 1.8 How Does This Guideline Fit in with My Agency’s Current MRA 

23 Approaches and Practices?

24

25 This Guideline is not intended to supplant your Agency‘s prerogative in 

26 establishing the approach that you may follow for preparing Agency-wide or even Office27 wide MRA procedures. The goal of this MRA Guideline is to provide an overarching 

28 framework and an approach to conducting MRAs that are flexible and broadly defined, so 

29 that any Agency can utilize them in preparing more tailored approaches to MRA. This 

30 Guideline is not intended to preferentially take the place of your Agency‘s own risk 

31 assessment approach, protocol, or guideline, but should help promote consistency in 

32 MRA among agencies. Each Agency has its own unique regulatory requirements, legal 

33 requirements, policies, and conventions that must be considered in their risk assessments. 

34 Ideally, this interagency Guideline is intended to provide the general framework that can 

35 be incorporated into the more specific MRA guidance that your Agency prepares. 

36 However, it is up to each Agency to make the determination of how or whether they 

37 utilize this guideline, consistent with international and national scientific guidance (e.g., 

38 Codex Alimentarius and the United States‘ National Academies).

39

host

environment pathogen
Page 10 Microbial Risk Assessment Guideline

1 1.9 How is this MRA Guidance Related to Other MRA Frameworks/Guidelines

2 that are Currently Available?

3

4 There are two major sets of frameworks that influenced the development of this 

5 Guideline. The first set of frameworks includes the NRC 1983 report (NRC, 1983) and 

6 the NRC Science and Decisions: Advancing Risk Assessment (NRC, 2009), which are 

7 related because the NRC 2009 framework is an enhancement of the NRC 1983 report. 

8 The NRC frameworks are geared to chemical risk assessment, but have applicability to 

9 MRA. They are also broadly applicable for many different exposure media. The second 

10 set of frameworks is the Codex Alimentarius Commission (Codex), Principles and 

11 Guidelines for the Conduct of Microbiological Risk Assessment (Codex, 1999) and Codex 

12 Proposed Draft Principles and Guidelines for the Conduct of Microbial Risk 

13 Management (MRM) (Codex, 2007a). The Codex framework is specific to the food 

14 media, but is tailored to microbial hazards. This Guideline is broadly applicable to many 

15 media, but is tailored to microbial hazards to humans, so both sets of frameworks were 

16 important for the development of this Guideline. These two sets of frameworks were not 

17 the only sources consulted for development of this Guideline.

18

19 U.S. government agencies (e.g., EPA, USDA, FDA, DHS, and DOD), as well as 

20 government organizations from other countries (e.g., Canada, European Union, New 

21 Zealand) and international Agencies (e.g., WHO, FAO/CODEX, and the Organization for 

22 Economic Cooperation and Development [OECD] – for example, see FAO/WHO, 2006),

23 have prepared various levels of guidance to support MRA applications. An EPA24 sponsored study, Foundations and Frameworks for Human Microbial Risk Assessment

25 (Parkin, 2008), presented an extensive search and evaluation of frameworks available for 

26 use in conducting MRA. The study determined that there were four general categories of 

27 frameworks that have been applied in MRA: 

28

29 a) The 1983 NRC report

30

31 b) A modified NRC 1983 approach without an explicit problem formulation (or 

32 planning and scoping) step

33

34 c) A modified NRC with a problem formulation (or planning and scoping) step

35

36 d) The International Life Science Institute (ILSI) approach (in association with the 

37 EPA‘s Office of Water) developed for water-based media (ILSI, 2000) 

38

39 While most microbial risk assessors recognize shortcomings with utilization of a 

40 uniform, exclusive application of the NRC 1983 approach, most have not explicitly 

41 attempted to take a completely fresh look at approaches to conduct MRA except for the 

42 ILSI (2000) approach which was loosely based upon EPA‘s draft Ecological Risk 

43 Assessment Guidelines. 

44

45 EPA, FDA, USDA, DOD, and DHS have utilized their own unique approaches to 

46 conducting risk assessments, but it is important to keep track of other federally mandated 
Microbial Risk Assessment Guideline Page 11

Draft – Do not cite, circulate, or quote

1 requirements that may apply to MRA. For example, when relevant, Executive Orders 

2 and OMB memorandums apply to MRA (EPA, 2002b; OMB, 2007b; Presidential 

3 Memorandum, 2009; OSTP, 2010).

4

5 1.10 What are Major Principles for MRA?

6

7 The principles in Text Box 1.1 are adaptable to microbial hazards in media other 

8 than food. However, different agencies may have different practices regarding how risk 

9 management and risk assessment activities are separated and the principles are framed in 

10 reference to food safety risks. Some agencies have more integrated management and

11 assessment. In cases where the risk management and risk assessment functions are more 

12 integrated, transparency about how and where risk management is integrated in risk 

13 assessment is crucial. The NRC report Science and Decisions: Advancing Risk 

14 Assessment proposed that EPA should adopt:

15

16 ―... a framework for risk-based decision making. The framework 

17 consists of three phases: I, enhanced problem formulation and 

18 scoping, in which the available risk-management options are 

19 identified; II, planning and assessment, in which risk-assessment 

20 tools are used to determine risks under existing conditions and 

21 under potential risk-management options; and III, risk 

22 management, in which risk and nonrisk information is integrated to 

23 inform choices among options.‖ (NRC, 2009)

24

25 The 2009 NRC framework also has a functional integration of risk assessment and 

26 risk management within a risk-based decision framework.

27

Text Box 1.1 General Principles of MRA

The ten general principles of MRA provided by the international Codex framework for 

microbiological hazards in food are below, as stated in Codex (2007b). The original 

numbering from the document is maintained. Principles 1-19 are general risk analysis 

principles and policy principles and are not included in this excerpt.

20. Each risk assessment should be fit for its intended purpose.

21. The scope and purpose of the risk assessment being carried out should be clearly 

stated and in accordance with risk assessment policy. The output form and possible 

alternative outputs of the risk assessment should be defined.

22. Experts involved in risk assessment including government officials and experts from 

outside government should be objective in their scientific work and not be subject to any 

conflict of interest that may compromise the integrity of the assessment. Information on 

the identities of these experts, their individual expertise and their professional experience 

should be publicly available, subject to national considerations. These experts should be 

selected in a transparent manner on the basis of their expertise and their independence 

with regard to the interests involved, including disclosure of conflicts of interest in 
Page 12 Microbial Risk Assessment Guideline

connection with risk assessment.

23. Risk assessment should incorporate the four steps of risk assessment, i.e. hazard 

identification, hazard characterization, exposure assessment and risk characterization.

24. Risk assessment should be based on scientific data most relevant to the national 

context. It should use available quantitative information to the greatest extent possible. 

Risk assessment may also take into account qualitative information.

25. Risk assessment should take into account relevant production, storage and handling 

practices used throughout the food chain including traditional practices, methods of 

analysis, sampling and inspection and the prevalence of specific adverse health effects.

26. Constraints, uncertainties and assumptions having an impact on the risk assessment 

should be explicitly considered at each step in the risk assessment and documented in a 

transparent manner. Expression of uncertainty or variability in risk estimates may be 

qualitative or quantitative, but should be quantified to the extent that is scientifically 

achievable.

27. Risk assessments should be based on realistic exposure scenarios, with consideration 

of different situations being defined by risk assessment policy. They should include 

consideration of susceptible and high-risk population groups. Acute, chronic (including 

long-term), cumulative and/or combined adverse health effects should be taken into 

account in carrying out risk assessment, where relevant.

28. The report of the risk assessment should indicate any constraints, uncertainties, 

assumptions and their impact on the risk assessment. Minority opinions should also be 

recorded. The responsibility for resolving the impact of uncertainty on the risk 

management decision lies with the risk manager, not the risk assessors.

29. The conclusion of the risk assessment including a risk estimate, if available, should 

be presented in a readily understandable and useful form to risk managers and made 

available to other risk assessors and interested parties so that they can review the 

assessment.

1

2

3 An overarching principle for MRA in this Guideline is to provide a systematic 

4 approach to the consideration of all information, models, and other tools that allow a 

5 suitable examination of the relationship of a microbial pathogen and human exposure to 

6 specific health endpoints and levels of effect. The process may be iterative, with

7 increasing quality of data to reduce uncertainties with each iteration (see section 2.5.1). 

8 Details of this systematic approach will be covered in later chapters. The other major 

9 principles you can consider in conducting risk assessments are ―transparency, clarity, 

10 consistency, and reasonableness‖ (TCCR) (EPA, 2000a).

11

12 a) Transparency is the foremost principle of the four because when followed it 

13 should help satisfy the other principles, in that it will lead to clarity and allow the 

14 evaluation of consistency and reasonableness. To provide transparency, the 

15 methods used and assumptions should be clear and understandable to the intended 

16 recipients of the assessment. From this, the audience for the risk assessment 
Microbial Risk Assessment Guideline Page 13

Draft – Do not cite, circulate, or quote

1 should be able to assess the adequacy of the data and methods used to provide the 

2 information. It also means that conclusions drawn from the risk assessment and 

3 associated research that are science-based are separate from subsequent policy 

4 judgments and risk management decisions, and the use of methods, models, 

5 assumptions and defaults are clear.

6

7 b) Clarity means the manner in which the risk assessment is presented, such as how

8 it is written and the appropriate use of illustrations or graphics to portray the 

9 findings. A main goal of this Guideline is to enable production of documents 

10 available for public scrutiny that are easily comprehendible and in simple 

11 language.

12

13 c) Consistency provides a context for the reader that can be readily understood for 

14 comparison to other similar documents and whether it provides harmony with an 

15 Agency‘s policy, procedural guidance, and scientific rationale. Where there are 

16 differences or inconsistencies, an explanation of why different approaches or 

17 interpretations were used and why the conclusions differ can be provided.

18

19 d) Reasonableness provides the audience with knowledge about the extent that 

20 professional judgments and assumptions are well founded. This can also address 

21 plausibility and comparisons against other similar types of risk assessment 

22 findings.

23

24 While these are the major underlying principles, other factors are also important,

25 such as the assessment of data quality, data analysis, and peer review. Most Agencies 

26 have addressed their needs for these in separate Agency guidelines and all have adopted 

27 OMB Information Quality guidelines relevant to these issues (OMB, 2002).

28

29 The Office of Science and Technology Policy (OSTP), directs that, ―When 

30 scientific or technological information is considered in policy decisions, the information 

31 should be subject to well established scientific processes, including peer review where 

32 appropriate, and each agency should appropriately and accurately reflect that information 

33 in complying with and applying relevant statutory standards‖ (Presidential Memorandum, 

34 2009; OSTP, 2010).

35

36 In 2007, OSTP and OMB reissued the 1995 Principles for Risk Analysis (OMB, 

37 2007b). The six principles pertaining to risk assessment include:

38

39 1. Agencies should employ the best reasonably obtainable scientific information to 

40 assess risks to health, safety, and the environment.

41 2. Characterizations of risks and of changes in the nature or magnitude of risks 

42 should be both qualitative and quantitative, consistent with available data. The 

43 characterizations should be broad enough to inform the range of policies to reduce 

44 risks.
Page 14 Microbial Risk Assessment Guideline

1 3. Judgments used in developing a risk assessment, such as assumptions, defaults, 

2 and uncertainties, should be stated explicitly. The rationale for these judgments 

3 and their influence on the risk assessment should be articulated.

4 4. Risk assessments should encompass all appropriate hazards (e.g., acute and 

5 chronic risks, including cancer and non-cancer risks, to human health and the 

6 environment). In addition to considering the full population at risk, attention 

7 should be directed to subpopulations that may be particularly susceptible to such 

8 risks and/or may be more highly exposed.

9 5. Peer review of risk assessments can ensure that the highest professional standards 

10 are maintained. Therefore, agencies should develop policies to maximize its use.

11 6. Agencies should strive to adopt consistent approaches to evaluating the risks 

12 posed by hazardous agents or events.

13

14 These principles have been incorporated throughout this Guideline.

15
Microbial Risk Assessment Guideline Page 15

Draft – Do not cite, circulate, or quote

1 2. PLANNING AND SCOPING

2

3 Planning and scoping will help ensure that a risk assessment is relevant and well 

4 done. The NRC 2009 framework recommended ―increased attention to the design of risk 

5 assessment in its formative stages [and] that planning and scoping and problem 

6 formulation, as articulated in EPA guidance documents (EPA, 1998a, 2003), should be 

7 formalized and implemented in EPA risk assessments‖ (NRC, 2009). Rigorous 

8 preparation is needed at the start of the risk assessment process, to facilitate 

9 communication during and following the risk assessment and to ensure that all issues are 

10 sufficiently vetted, all participants are clear on the objectives and goals, and managers are 

11 clear about the commitment of personnel and other resources (EPA, 1992; NRC, 1996). 

12

13 For more detailed descriptions of the usefulness and implementation of the 

14 planning and scoping process, the FDA, USDA, and EPA sources provide general 

15 information on how to proceed (FDA, 2002; USDA, 2003; EPA, 2000a, 2002a; NRC 

16 2009). This chapter provides an overview of this first step in the risk assessment process. 

17 Several interesting case studies from ecological risk assessment are presented in EPA‘s 

18 Lessons Learned on Planning and Scoping for Environmental Risk Assessments (EPA, 

19 2002a).

20

21 2.1 What is Planning and Scoping?

22

23 Planning and scoping can be viewed as a process that defines the purpose and 

24 scope of a risk assessment and focuses the issues and approach(es) involved in 

25 performing the assessment. A clearly articulated purpose and scope provides a sound 

26 foundation for later judging the success of the risk assessment and for an effective risk 

27 characterization. In a sense, the planning and scoping process lays out a ―road map‖ for 

28 how the risk assessment will be accomplished. 

29

30 EPA considers Planning and Scoping steps to be as follows (based on EPA, 

31 2003a, 2004b):

32

33 a) Defining the purpose of the assessment

34 b) Defining the scope of analysis and products needed

35 c) Agreeing on participants, roles and responsibilities

36 d) Agreeing on depth of assessment and analytical approach (for example, will the 

37 risk assessment include static or dynamic modeling)

38 e) Agreeing on resources available and schedule

39 f) Formulating the problem (see Section 2.1.1)

40 g) Developing the conceptual model (see Section 2.6.3)
Page 16 Microbial Risk Assessment Guideline

1 h) Constructing the analysis plan (see Section 2.6.4)

2 See Section 2.3 for an overview of how CFSAN does planning and scoping for major risk 

3 assessments.

4 2.1.1 What is Problem Formulation?

5

6 The problem formulation exercise is a discussion and analysis activity that 

7 involves all relevant parties including the risk manager, risk assessment team, risk 

8 communication specialist and, when appropriate, relevant stakeholders and interested 

9 parties. The outputs of the problem formulation are:

10

11 a) A definition of the valued entity and endpoint: what is the entity that should be 

12 protected and what are the undesirable effects that you are trying to avoid. For 

13 MRA this may involve a policy determination of what the ―valued entity‖ is (e.g., 

14 general population, young children, pregnant women, immunologically 

15 compromised) and what is considered to be an appropriate level of protection

16 (ALOP) against infection or disease.

17

18 b) A Conceptual Model that lays out the anticipated exposure scenarios of the 

19 microorganisms from the source to the receptor. With MRA this may be the 

20 movement of enteric pathogens from some source (e.g., treatment works, manure 

21 application to a field, critical point in the food processing system) to the 

22 subpopulation of concern and highlights various hazardous events that may lead 

23 to increased risk and where risk management may be most effective.

24

25 c) An Analysis Plan which provides a road map for addressing the problem. In 

26 short, it is analogous to an experimental design. In the analysis plan risk 

27 hypotheses generated earlier are examined and discussed; the relationships 

28 between pathways and valued entities are further examined; and the level of 

29 precision and data quality are considered in light of available information.

30

31 2.2 What are the Benefits of Planning and Scoping?

32

33 The planning and scoping process helps everyone involved in the risk assessment 

34 understand how the risk assessment fits into the overall decision making process. 

35 Planning and scoping promotes:

36

37 a) initial planning to identify appropriate timelines and necessary resources, thereby 

38 improving efficiency;

39

40 b) agreement among principle parties regarding the goals, commitment, time-frame 

41 and resources, by setting realistic expectations;

42

43 c) the prospect of less unanticipated controversy, since all interested parties 

44 contribute, areas of disagreement can be dealt with upfront, not left as a surprise 

45 at the end;
Microbial Risk Assessment Guideline Page 17

Draft – Do not cite, circulate, or quote

1

2 d) identification of and participation by those from many disciplines (e.g., 

3 microbiologists, toxicologists, economists, lawyers) to help in the process thereby 

4 ensuring that each risk assessment and characterization is useful for the intended 

5 audience(s);

6

7 e) an understanding of the degree of complexity needed in the risk assessment to 

8 adequately inform the decision at hand; and

9

10 f) better informed decisions.

11

12 2.3 What do I Consider When Deciding to Initiate a MRA?

13

14 Unless your agency is required to conduct a particular risk assessment, managers 

15 need to decide whether a risk assessment is appropriate, feasible and will actually be 

16 performed. This decision is commonly made during planning and scoping. Deciding not 

17 to initiate a risk assessment may be an acceptable outcome of planning and scoping. The 

18 agency may decide that a decision can be reached without conducting a risk assessment 

19 or other priorities or risk assessments may take precedence. There are several criteria 

20 you can consider for identifying a candidate risk assessment:

21

22 a) Characteristics and importance of the hazard(s) of concern

23

24 b) Magnitude (presence, prevalence, concentration of hazards) and severity (impact 

25 on public health) of the risk

26

27 c) Urgency of the situation

28

29 d) Subpopulations of concern

30

31 e) Other factors associated with specific hazards (such as water treatment processes, 

32 food processing, cooking, cross contamination)

33

34 f) Availability of resources (time, money, staff)

35

36 A risk assessment may be required to comply with regulatory analysis requirements 

37 (OMB, 2003). The Office of Information and Regulatory Affairs has provided a checklist 

38 to assist agencies in producing regulatory impact analyses (RIAs), as required for 

39 economically significant rules by Executive Order 12866 and OMB Circular A-4 (E.O. 

40 12866, 1993; OMB, 2010; OMB, 2003). Appropriate assessments of risk may be 

41 necessary to address international trade agreements (e.g., World Trade Organization 

42 Sanitary and Phytosanitary Agreement). Ensuring that each assessment of risk is fit for 

43 its intended purpose and is based on scientific data most relevant to the national context 

44 ensures that the effort and scope of the assessment of risk are appropriate for the risk 

45 management questions being raised so that practical risk management options can be 

46 formulated. 
Page 18 Microbial Risk Assessment Guideline

1

2 Other specific examples of when a risk assessment may be appropriate include:

3

4 a) Review of the reliability or utility of a standard

5

6 b) Cases where the current standard is inconsistent with other government policies, 

7 guidelines, or thresholds

8

9 c) Cases where an agency has been petitioned for a regulatory action

10

11 d) Establishment of standards for regulatory action

12

13 e) Evaluation of the public health implications of different tolerable risk levels

14

15 f) Cases where a data gap analysis is desired

16

17 g) Cases where the hazard is a serious health issue, emerging pathogen and/or public 

18 health concern

19

20 h) The exposure system is complex.

21

22 FDA‘s Center for Food Safety and Applied Nutrition‘s (CFSAN) Initiation and 

23 Conduct of All “Major” Risk Assessments within a Risk Analysis Framework outlines 

24 how CFSAN selects, conducts, and communicates food safety risk assessments (FDA, 

25 2002). CFSAN‘s process for identifying and selecting major risk assessments is divided

26 into four phases (FDA, 2002):

27

28 a) Phase 1: Concept Generation – Collect ideas and maintain a list of potential risk 

29 management questions for which a risk assessment would assist with policy 

30 decisions. Develop justification for candidate risk assessments, including, 

31 purpose of assessment, scope of problem, importance to the Center, and how the 

32 result will be used by the Center.

33

34 b) Phase 2: Problem Identification – The candidate risk assessment and supporting 

35 information (justification) are reviewed to determine whether the assessment 

36 meets the Center‘s needs. This phase results in one of three recommended 

37 actions, conduct data feasibility study, not required for regulatory decision, or 

38 more information needed to make decision.

39

40 c) Phase 3: Data Feasibility (Evaluation and Recommendation) – Information is 

41 collected and reviewed to determine availability of data needed to answer risk 

42 assessment question(s). This phase results in one of four recommended actions, 

43 conduct quantitative risk assessment, conduct qualitative risk assessment, more 

44 research needed, or modify question and conduct alternative assessment.

45
Microbial Risk Assessment Guideline Page 19

Draft – Do not cite, circulate, or quote

1 d) Phase 4: Disposition (Selection) – Using the results from the data feasibility 

2 determination as an aid, risk assessment(s) to be conducted are selected. Decision 

3 is based on technical merit, resource availability, the Center‘s priority needs, and 

4 other legitimate factors.

5

6 2.4 Who Can be Involved with Planning and Scoping?

7

8 All interested parties could be involved with planning and scoping, but, from a 

9 practical standpoint, bringing everyone together at one time and discussing all aspects of 

10 the effort at once may be difficult and therefore would probably include more than a 

11 single session. Major participants include relevant risk managers, risk assessors, and 

12 other members of the ―team‖ working on the decision that needs to be made. In addition 

13 to the appropriate microbiologists, infectious disease experts and/or individuals trained in 

14 infection control and other members such as economists, lawyers, engineers, policy 

15 makers, and communicators should be included. A risk assessment dialogue among the 

16 risk managers, risk assessors, risk communicators, economists, and other technical 

17 experts should develop the broad dimensions and elements of the risk assessment, the 

18 management questions and goals for the assessment, a tentative budget and schedule, and 

19 an approach for conducting the risk assessment. The risk assessor, risk manager, and 

20 planning team interactions may vary depending on the Agency. Generally, they meet to 

21 evaluate and select the kind of risk information, exposure scenarios, and assessment 

22 issues to be covered. Risk assessors should be very involved in planning and scoping at 

23 every stage, because risk managers will have technical and data related questions that 

24 may require your input. As a risk assessor, you will want to weigh in on whether the 

25 tools and data available can answer the questions being posed. Planning and scoping 

26 would not be successful without the technical input of risk assessors and the management 

27 perspectives from risk managers. In addition, the risk manager should consider stating

28 explicitly any reasons to limit the technical scope of the assessment, and you should 

29 consider including details on resource limitations, data availability and quality, and 

30 methods availability. In other words, you can be very clear and transparent about what 

31 you plan to include in the risk assessment and what you plan to exclude from the risk 

32 assessment and why. Further elaboration on the role of risk assessors is found in Section 

33 2.8.

34

35 Agencies with a hierarchy of risk managers will have a hierarchy of risk 

36 management activities. Planning and scoping can outline the management hierarchy with 

37 a degree of detail that is appropriate for planning deliverables and milestones. The roles 

38 are not exclusive to each level and in practice would likely overlap considerably. Risk 

39 managers at all levels are responsible for ensuring appropriate communication with 

40 managers above and below their level. It is important to identify the team leader and 

41 his/her basic responsibilities at the outset to avoid misunderstandings as the process 

42 develops. Risk managers are generally the decision makers in their organization. The

43 top manager is the ultimate decision maker for his/her organization and is accountable for 

44 both the risk characterization process and products in his/her office. However, every risk 

45 manager may make decisions appropriate for their level in the management hierarchy. 
Page 20 Microbial Risk Assessment Guideline

1 You should be prepared to communicate with many levels of managers particularly if you 

2 are working on a high profile risk assessment.

3

4 The importance of stakeholder involvement during the planning and scoping 

5 process depends upon the nature of the problem, their interest, and ability to contribute. 

6 Stakeholders can be identified early in the planning and scoping of the risk assessment. 

7 How stakeholders can be involved most effectively needs to be decided on a case-by-case 

8 basis. Public involvement, early and often, leads to a much clearer risk assessment

9 product. This involvement also allows for flexibility and buy-in for future decision 

10 making if the need arises to deviate from the original plan.

11

12 Affected parties can share their points of view about the risk and how it could be 

13 managed. Their input is particularly helpful in determining what should be included in 

14 the assessment, how they might be affected or exposed to the risk, and what additional 

15 data or exposure scenarios should be developed. 

16

17 You should determine if your agency has any stakeholder involvement guidance 

18 and integrate those practices into planning and scoping. Stakeholder involvement 

19 practices may be well developed and formalized or haphazard and inconsistent. Building

20 the necessary relationships with stakeholders to maintain dialogue takes considerable 

21 effort, but this should not deter you and risk managers from engaging in this important 

22 activity. Risk communication specialists should be engaged in developing stakeholder 

23 involvement plans. The National Academies of Science report, Public Participation in 

24 Environmental Assessment and Decision Making, is a resource for considerations when 

25 engaging stakeholders (NRC, 2008).

26

27 2.5 How Can the MRA be Used?

28

29 Although risk assessments conducted by different agencies are not used for the 

30 same purposes, all agencies perform risk assessments with one or more of the following 

31 goals in mind (adapted from U.S. Army Center for Health Promotion and Preventive 

32 Medicine [USACHPPM], 2009):

33

34 a) To mitigate, e.g., to mitigate adverse effects or risk from a specific event 

35

36 b) To confirm, e.g., to determine if regulations, policies, standards, criteria, and/or 

37 goals are adequate 

38

39 c) To decide whether and/or how to regulate, e.g., as needed to establish regulations, 

40 policies, standards, criteria, and/or goals 

41

42 d) To investigate, e.g., to determine research or other requirements that would 

43 enhance predictive and/or risk ranking capabilities, or facilitate completion of 

44 screening or feasibility assessments 

45
Microbial Risk Assessment Guideline Page 21

Draft – Do not cite, circulate, or quote

1 Depending on the risk assessment‘s purpose, a particular assessment approach 

2 may be employed. The appropriate risk assessment approach for a specific risk 

3 management problem or decision depends on the question(s) that need to be answered

4 and the availability of data. For example, if adequate data are available, a quantitative 

5 risk assessment is possible; if fewer data are available, a qualitative assessment or a data 

6 gap analysis (discussed in Sections 2.6.5 and 6.4) may be more appropriate.

7

8 2.5.1 What “Depth” Can I go into in the Risk Assessment?

9

10 A major consideration for the risk assessment approach is how much detail or 

11 ―depth‖ you need to incorporate to address the risk management question(s) or decision. 

12 Due to various management needs, the risk assessment approach is not necessarily a one13 size-fits-all approach. This guidance is intended to provide flexible methods for 

14 supporting different types of assessments (screening, safety) and outputs (qualitative or 

15 quantitative), as described in the American Society for Microbiology (ASM) Press book 

16 (Schaffner, 2008). Guidance from Codex Alimentarius (1999) and World Organization 

17 for Animal Health (OIE, 1999) described qualitative and quantitative outputs as equally 

18 valid (Wooldridge, 2008). Woodridge (2008) provides detailed discussion of qualitative 

19 and quantitative assessments, and other risk analysts (Dennis et al., 2008) discuss both 

20 estimates of risk and safety, mortality for listeriosis and allowable ('safe') levels of Vibrio

21 in seafood, respectively.

22

23 Be aware that the terms referring to the different types of assessments are 

24 specifically defined in different contexts; care should be used when ―naming‖ or referring 

25 to types of assessments. For example, this Guideline is specific to the selection and 

26 conduct of risk assessment, not a safety assessment. One difference is that risk 

27 assessment estimates the likelihood and/or frequency of adverse health outcomes 

28 resulting from an exposure and, in some cases, sources of risk and quantitative reductions 

29 in risk based on various interventions. While much of this guideline is relevant to a 

30 safety assessment, a safety assessment may estimate the likelihood and/or frequency of 

31 exceeding a specified threshold of concern (e.g., predetermined regulatory limits or 

32 standards), or provide a determination of what is ―safe‖ based on the conventions of the 

33 standard-setting procedure. For example, Codex standards traditionally specify 

34 maximum limits for additives and residues in foods based on the concept of ―no 

35 appreciable risk‖ and for contaminants in foods based on the concept of ―as low as 

36 reasonably achievable‖ (FAO/WHO, 1997). Risk-related terms in some statutes have 

37 formal definitions.

38

39 There are many cases where you will likely need to perform a screening risk 

40 assessment versus a fully developed one. This is usually done when a very time critical 

41 decision is needed (e.g., quick mitigation action is required after an event; imminent 

42 exposure to a microbial hazard is identified). Screening risk assessments often provide a 

43 conservative, health-protective, estimate of possible risk that is based on the more 

44 readily available data. It is likely that you will resort to default assumptions to bridge 

45 data gaps that you cannot wait for research to fill. It is important to be transparent about 

46 the amount of uncertainty in your screening estimate and discuss whether the uncertainty 
Page 22 Microbial Risk Assessment Guideline

1 causes underestimates or overestimates of risk based on the assumptions applied. This 

2 may be followed up by a more detailed assessment that will need to be conducted or risk 

3 managers will need to take action(s). Thus, you also may be asked to return to the 

4 assessment and refine and recalculate your estimates based on further gathering of data 

5 with more time allotted (e.g., quick sampling assays, use of surrogate data, expert 

6 elicitation). Alternatively, there may be a risk management decision to conduct a full risk 

7 assessment.

8

9 A comprehensive (―major‖) risk assessment requires a substantial commitment of 

10 resources. Thus, this depth of risk assessment is not appropriate when risk managers do 

11 not need this level of sophistication to make a decision. Circumstances that may not

12 warrant a quantitative risk assessment would include, for example, a risk that is well 

13 described by definitive data, a problem that is relatively simple, or an issue that is not of 

14 regulatory concern. However, a comprehensive risk assessment is a powerful tool to help 

15 risk managers evaluate and interpret information when the data describing a hazard are 

16 incomplete, the exposure system is complex, or the issue is of high regulatory or 

17 stakeholder concern. Details for a major risk assessment are presented in this guideline.

18

19 Risk assessments can be either qualitative or quantitative in their description of 

20 the likelihood of adverse health effects, depending on the extent of the data and 

21 knowledge available, the existence of models or other tools for quantitative predictions, 

22 the complexity of the problem, the scope and nature of the question(s) posed by the risk 

23 managers, and the time available to conduct the assessment. In quantitative assessments, 

24 the risk is expressed as a mathematical statement of the probability of illness or death 

25 after exposure to a specific hazard, and it represents the cumulative probabilities of 

26 certain events happening and the uncertainty associated with those events. Conversely, 

27 qualitative risk assessments use verbal descriptors of risk and severity as well as 

28 uncertainty, and often involve the aggregation of assumptions.

29

30 2.5.2 What Are Examples of Types of MRA?

31

32 Risk assessments can take various forms depending on the agencies‘ needs. 

33 Examples of risk assessment types that you are likely to use include risk ranking, product 

34 pathway, risk-risk, geographic, and for sustainability assessments:

35

36 a) Risk ranking – Risk ranking assessments compare the relative risk among 

37 several hazards. For example, this type of assessment might involve a single 

38 pathogen associated with multiple foods, a single food that has multiple 

39 pathogens, or multiple pathogens and multiple foods. Risk ranking assessments 

40 can help establish regulatory program priorities and identify critical research 

41 needs. The Food and Drug Administration/U.S. Department of Agriculture 

42 (FDA/USDA) Listeria monocytogenes risk assessment is an example of a risk 

43 ranking assessment (FDA/USDA/CDC, 2003).

44

45 b) Product pathway analyses – In product pathway assessments, the factors that 

46 influence the risk associated with specific vehicle/hazard pairs are examined. For 
Microbial Risk Assessment Guideline Page 23

Draft – Do not cite, circulate, or quote

1 food, it ideally starts at the farm and ends with consumption. This type of 

2 assessment technique helps identify the key factors that affect exposure including 

3 the impact of potential mitigation or intervention strategies on the predicted risk. 

4 The FDA Vibrio parahaemolyticus risk assessment is an example of a product 

5 pathway analysis (FDA, 2005).

6

7 c) Risk-risk – In risk-risk assessments, a trade off of one risk for another is 

8 considered, i.e., reducing the risk of one hazard increases the risk of another. An 

9 example of this would be a determination of the impact on public health by 

10 treating drinking water with a chemical (risk to chlorine exposure) versus the 

11 impact of exposure to pathogenic organisms in water that is not treated.

12

13 d) Geographic – In a geographic risk assessment, the factors that either limit or 

14 allow the risk to occur in a given region are examined. The risk of introduction of 

15 disease agents through water, air, food animals, or animal products in the U.S.

16 (e.g., intentionally as in a bioterrorism act or unintentionally) can be examined. 

17 For example, the risk of introduction of bovine spongiform encephalopathy (BSE) 

18 into the U.S cattle herds and the subsequent risk of variant Creutzfeldt-Jacob 

19 Disease (vCJD) in humans by the transmission from cattle through meats and 

20 animal product pathways might be examined using a geographical approach.

21

22 e) MRA within sustainability assessments – Using a systems-level assessment 

23 over the life-time of its technical components, sustainability assessments attempt 

24 to account for human health, ecosystem health, and economic considerations. The 

25 human health aspects include chemical and MRAs. The difference here is to 

26 include the MRA over the expected life-time of the technical system and via all 

27 exposure pathways of pathogens to humans (e.g., drinking water, reuse waters, 

28 aerosols pathways, recreational exposures, and contaminated soils/foods).

29

30 2.5.3 What are Threat and Vulnerability Assessments for High31 Consequence Biological Events?

32

33 This Guidance is primarily for risk assessment and does not go into detail 

34 regarding the related disciplines of threat and vulnerability assessments. Nevertheless, it 

35 is important to understand these specialized tools for evaluating the susceptibility of 

36 systems and facilities to potential threats, such as adversarial actions (e.g., vandalism, 

37 insider sabotage, or terrorist attack), natural disasters, and other emergencies. Therefore, 

38 threat and vulnerability assessments are described briefly below.

39

40 Similar to a risk assessment, threat and vulnerability assessments identify threats 

41 and characterize the nature, probability, and magnitude of adverse effects. However, 

42 vulnerability assessments not only consider the risk to the infrastructure itself, but also 

43 consequences to the surrounding community and environment, such as economic losses 

44 and human health impacts from consuming contaminated food. Vulnerability 

45 assessments also usually identify corrective actions that can reduce the risk or lessen the 

46 severity of potential consequences. Threat and vulnerability assessments are often used 
Page 24 Microbial Risk Assessment Guideline

1 as tools to prioritize which threats or vulnerable operations need to be addressed first. 

2 They are not actually a ―risk assessment‖ per se, although sometimes the results can be 

3 similar to those produced in a classic microbial risk assessment, and the results can help 

4 to inform risk management decisions.

5

6 Since the anthrax attacks of 2001, there has been growing awareness of the 

7 potential for the deliberate use by terrorists of biological agents that pose a threat to the 

8 health and welfare of the exposed subpopulation and to the economy. It may be 

9 necessary to assess the risks associated with intentional contamination of the food or 

10 water supply with biological agents, or release of biological agents as an aerosol into 

11 highly populated indoor or outdoor public areas.

12

13 Threats and vulnerabilities can be characterized sequentially or as part of a single 

14 assessment. Initially, the nature and probability of potential threats are determined. 

15 Threat assessments of intentional contamination include information on the availability 

16 and ease of production of agents, as well as the sophistication and capabilities of the 

17 likely perpetrator. Because of a lack of data and the typically random nature of terrorist 

18 events, the probability of threats can usually not be quantified statistically.

19

20 A vulnerability assessment helps you consider the potential for impact of 

21 significant loss of operational and/or functional capability from a deliberate attack or an 

22 anticipated natural disaster, as well as the vulnerability of the facility/location itself to the 

23 identified threats. For example, projected impact of loss can be viewed as the degree to 

24 which the operation and critical infrastructure of a metropolitan drinking water 

25 distribution system may be impaired by a successful attack involving a biological threat 

26 or a natural disaster resulting in extensive biological contamination of primary source 

27 waters feeding the distribution system. Consequences might also include adverse health 

28 impacts to customers of the water system and economic losses to the community.

29

30 Each scenario is unique and represents a different combination of factors affecting 

31 the probability and the consequences of an intentional release of a biological threat agent. 

32 The health-related factors might include the toxicity of the agent; the concentration of the 

33 agent; fate of the agent in during processing, storage, distribution, and preparation of a 

34 food product; quantity of contaminated food or water consumed; contagiousness; and the 

35 availability and effectiveness of countermeasures, such as recalls and medical treatment. 

36 The economic and public health impacts from a biological attack could be devastating 

37 and catastrophic. Contaminated facilities may be out of commission for extended periods 

38 of time for decontamination, and may never be able to return to operation or profitability. 

39 The public may also lose confidence in the safety of the food and water supply.

40

41 An effective vulnerability assessment provides a prioritized plan for mitigation 

42 measures, such as security upgrades, modifications of operational procedures, and/or 

43 policy changes, and provides a framework for developing risk reduction options and 

44 associated costs. 

45
Microbial Risk Assessment Guideline Page 25

Draft – Do not cite, circulate, or quote

Example Products of 

Planning and Scoping

Statement of Concern 

Statement of Purpose and Objectives

Background Section

Scope

Scenarios

Literature Review

Data Inventory

Tools and Methods Inventory

Risk management questions or Charge

Risk Profile

Conceptual Model 

Value-of-information analysis

Communication Plan

Analysis Plan 

Work Plan

Data quality objectives

1 A preemptive targeting prioritization tool you can use for a vulnerability 

2 assessment is the CARVER plus Shock method (FDA, 2007), which has been adapted for 

3 use in the food sector. This tool can be used to assess the vulnerabilities within a system 

4 or infrastructure to an attack. It allows you to think like an attacker by identifying the 

5 most attractive targets for attack. By conducting such a vulnerability assessment and 

6 determining the most vulnerable points in your infrastructure, you can then focus your 

7 resources on protecting your most vulnerable points.

8

9 CARVER is an acronym for the following six attributes used to evaluate the 

10 attractiveness of a target for attack (FDA, 2007):

11

12 a) Criticality – measure of public health and economic impacts of an attack

13

14 b) Accessibility – ability to physically access and egress from target

15

16 c) Recuperability – ability of system to recover from an attack

17

18 d) Vulnerability – ease of accomplishing attack

19

20 e) Effect – amount of direct loss from an attack as measured by loss in production

21

22 f) Recognizability – ease of identifying target

23

24 In addition, the modified CARVER tool evaluates a seventh attribute, the 

25 combined health, economic, and psychological impacts of an attack, also called the 

26 Shock attributes of a target (FDA, 2007).

27

28 2.6 What is Discussed During Planning 

29 and Scoping and What Products 

30 Emerge?

31

32 Principal outputs from planning and scoping 

33 can include various products that are 

34 appropriate to the management objectives 

35 and the plan for analysis of the risk. Not all 

36 of these examples are necessary for every 

37 occasion and certain ones will be more 

38 appropriate for your particular problem. 

39 Further, some outputs can be contained in 

40 other products (e.g., the risk profile

41 document can contain many of the listed 

42 products). You need to discuss what outputs 

43 will be generated during the planning and 

44 scoping discussions. In general, many of 

45 these products are good candidates for peer 

46 review. Peer review early in the risk 
Page 26 Microbial Risk Assessment Guideline

1 assessment process can provide timely insights, corrections to assumptions, and 

2 directions on proper ways to proceed during the risk assessment.

3

4 During planning and scoping, participants can engage in a dialogue to answer the 

5 following questions and commit the outcome of those discussions to paper to facilitate 

6 mutual understanding (paper products underlined): 

7

8 a) What is the motivation for the risk assessment? Drafting a Statement of 

9 Concern is a good way to reach a common understanding of what broad issue the 

10 risk assessment will address. Describe in simple terms what hazard is being 

11 addressed and how it is thought to relate to human health for an exposure 

12 scenario. Include any other driving factors for the risk assessment, such as a food 

13 safety issues, regulatory requirements, public concern, or new scientific findings. 

14 b) What are the management goals, issues, questions, and policies that need to 

15 be addressed? A Statement of Purpose and Objectives is a concise paragraph 

16 that addresses the management goals. The management questions are framed as 

17 questions and are written down. The management questions should be designed 

18 so that if answered, the risk manager has the information needed to inform the 

19 decisions that need to be made. 

20 c) What is the context of the risk assessment? Risk assessments are done in 

21 historical and social contexts. It is helpful to summarize in a Background Section 

22 any previous risk assessments that addressed the same or similar hazards. In 

23 particular, if your agency has conducted a previous risk assessment on the hazard, 

24 then you can summarize the relationship between the current and previous risk 

25 assessment. The context of a risk assessment may include new mandates,

26 regulatory requirements, policy developments, technical advancements, risk 

27 assessment method and tool advancements, and new or enhanced data sets.

28 d) What is the scope and coverage of the risk assessment? The Scope outlines the 

29 scenarios the risk assessment will cover. Answering the scoping questions below 

30 can help ensure that you have the information regarding scope that is necessary to 

31 conduct the MRA.

32 1) Which infectious disease hazard is being addressed (pathogen strain[s], or 

33 indicator[s], taxon [genus, species, strain/biovar] )? Define the hazard.

34 2) Which human subpopulations will be included in the risk assessment (e.g., 

35 general population, life stages, or geographically defined subpopulations)? 

36 Describe which subpopulations are explicitly included in the risk 

37 assessment model, which will be accounted for implicitly, and which 

38 subpopulations may be excluded by the risk assessment model (e.g., most 

39 extreme behaviors).

40 3) What health outcomes or endpoints are addressed by the risk assessment, 

41 and how is the health outcome measured? Clearly defining the health 
Microbial Risk Assessment Guideline Page 27

Draft – Do not cite, circulate, or quote

1 endpoint is important for transparency and also focuses the scope of the 

2 risk assessment (e.g., infection, disease symptom/s, mortality).

3 4) What unit and routes of exposure are relevant and why? Determine the 

4 time-span of exposure relevant to the decision.

5 5) For risk assessments designed to derive nominally or presumptive ―safe‖ 

6 levels of microorganisms (i.e. levels below a threshold of regulatory 

7 concern), what level of protection will be provided, and what is the 

8 technical or policy justification for that level? Transparency in public 

9 health objectives is important.

10 6) What specific exposure scenarios should be modeled? List specific 

11 scenarios the risk managers would like to model (varying the inputs), 

12 including desired spatial and temporal features.

13 e) What type of risk assessment is needed to address the risk management 

14 question(s)? Section 2.5.2 describes different types of risk assessments, 

15 including quantitative, qualitative, risk ranking, product pathway, risk-risk, 

16 geographic, and systems-level. 

17 f) What is the state of the current knowledge? The planning and scoping can

18 include an overview of current knowledge and can be used to outline the topics 

19 that will be reviewed in more depth in the rest of the risk assessment. A 

20 Literature Review will likely help with understanding the current state of the 

21 science.

22 g) What and where are the available data? In addition to identifying the available 

23 data, a Data Inventory can address data relevance, how the data will be used, data 

24 accessibility considerations, and initial data quality evaluations. An inventory 

25 may also provide notice to the public of the data currently available to the agency 

26 in a call for data. See Section 2.6.6 for further discussion of data quality.

27 h) How do I know what questions the risk assessment needs to answer? The 

28 Risk management Questions or Charge are usually written down and discussed 

29 iteratively between the risk assessment team and risk managers so there is a 

30 common understanding of the questions.

31 i) What are the information/data needs of other members of the “team?” The 

32 risk assessment may be part of a larger project, such as an economic analysis. 

33 There may be economic, social, or legal analyses that need to be coordinated with 

34 the risk assessment. 

35 j) How will you model the risk? A Tools and Methods Inventory should include 

36 statistical methods for estimating model inputs and tools for addressing 

37 uncertainty and variability and should make an initial determination of which 

38 methods and tools are likely to be most useful.
Page 28 Microbial Risk Assessment Guideline

1 k) What are possible risk assessment or risk management options? Particularly 

2 for risk assessments that are needed to evaluate intervention strategies or needed 

3 to support regulatory determinations, different options should be presented as 

4 different scenarios for the risk assessment. Those Scenarios can be clearly stated 

5 during problem formulation and may evolve during risk assessment iterations.

6 l) What are the logistical considerations for conducting the risk assessment? 

7 Logistical considerations can go into the Analysis Plan (see Section 2.6.4) or a 

8 Work Plan document. These include:

9 1) resources available to do the assessment, including funding and staff time;

10 2) participants in the process and their roles;

11 3) plans for coordinating across offices, with other agencies and with 

12 stakeholders; and

13 4) scheduling (e.g., milestones, deliverable due dates, quality audits, 

14 meetings), including provisions for timely and adequate internal, 

15 independent external peer review, and if required, interagency review as 

16 per EO 12866.

17 m) How will planning activities and results be communicated to senior managers 

18 and to the public? Chapter 8 discusses risk communication within the context of 

19 risk assessment.

20 n) What are the legal considerations and constraints that may shape the 

21 ultimate decision and supporting risk assessment? The technical office within 

22 the agency that is conducting the risk assessment can engage the agency‘s legal 

23 department, according to normal practices within that agency. Legal context can 

24 be discussed in the Background Section.

25 o) How and at what iterations will the risk assessment be peer reviewed? OMB 

26 has published Information Quality Bulletin for Peer Review, which provides 

27 general peer review guidance and sets minimal expectations for the review of 

28 scientific information (OMB, 2004). Most agencies have agency specific peer 

29 review guidance that complies with the OMB guidance. You should follow your 

30 agency‘s peer review guidance. For example, EPA‘s Peer Review Handbook

31 (EPA, 2000d, 2006c) provides guidance on selection of peer reviewers that 

32 includes where to find peer reviewers, what mix of expertise may be important, 

33 representing diversity of disciplines, and limiting conflicts of interest. According 

34 to the National Committee on Radiation Programs (NCRP, 1996), an expert has 

35 the following characteristics:

36

37 1) training and experience in the subject area resulting in superior knowledge 

38 in the field;

39

40 2) access to relevant information;
Microbial Risk Assessment Guideline Page 29

Draft – Do not cite, circulate, or quote

1

2 3) an ability to process and effectively use the information; and

3

4 4) is recognized by his or her peers or those conducting the study as qualified 

5 to provide judgments about assumptions, models, and model parameters at 

6 the level of detail required.

7

8 In summary, the planning and scoping discussion may include a preliminary 

9 characterization of exposure and effects, as well as examination of scientific data and 

10 data needs, policy and regulatory issues, and scenario-specific factors to define the 

11 feasibility, scope, and objectives for the risk assessment. The level of detail and the 

12 information that will be needed to complete the assessment also are determined. Just as 

13 important, planning and scoping helps set the boundaries of the problem(s) addressed and 

14 the scope of the MRA.

15

16 For perspective, risk managers (decision makers) naturally desire more 

17 information, less uncertainty, and more in depth interpretation, when the impact of their 

18 decisions increases. They also want to know the financial and social implications of 

19 possible decisions. The risk assessment may not be the appropriate support analysis to 

20 address these issues, but if done well, risk assessment can be a critical input. These are 

21 aspects that may be discussed during planning and scoping. 

22

23 2.6.1 What are Risk Management Questions and What is the Charge?

24

25 A brief description of management questions is presented in Section 2.4. 

26 Generating specific risk management questions for a risk assessment helps formulate a 

27 clear, focused charge that identifies the technical and scientific issues on which you need 

28 to address and suggestions for conducting the risk assessment. Formulating these 

29 questions usually requires significant interaction between risk assessors and risk 

30 managers, as well as dialogue with appropriate other parties (e.g., those with relevant 

31 information about the potential hazard). You want to be sure that the questions you 

32 generate will focus the risk assessment to provide the appropriate analyses that can 

33 properly inform the risk management decision at a level of detail appropriate for the 

34 issue. The resulting risk assessment can be designed to address and answer as best as 

35 possible the risk management questions posed. 

36

37 The charge focuses the assessment by presenting specific questions and concerns 

38 surrounding such issues as the comprehensiveness of the data, information, and literature, 

39 the soundness of the methods proposed, the scientific support for the assumptions that 

40 may be employed, and the sensitivity of the results to possible alternative assumptions. 

41 As a general rule, time is well-spent preparing a good set of questions or a charge, as they 

42 are crucial for an effective risk assessment and ultimate decision. In this context the 

43 charge is the set of questions and does not imply a formal charge.

44

45
Page 30 Microbial Risk Assessment Guideline

1 2.6.2 What is a Risk Profile?

2

3 Codex defines a risk profile for food safety as: a description of a food safety 

4 problem and its context that presents in a concise form, the current state of knowledge 

5 related to a food safety issue, describes potential microbiological risk management 

6 options that have been identified to date, when any, and the food safety policy context 

7 that will influence further possible actions... Consideration of the information given in 

8 the risk profile may result in a range of initial decisions, such as commissioning a 

9 microbiological risk assessment, gathering more information or developing risk 

10 knowledge at the level of the risk manager, implementing an immediate and/or temporary 

decision (Codex Alimentarius Commission, 2007).211 A typical risk profile includes a 

12 description of the situation, product, or commodity involved; information on pathways by 

13 which consumers are exposed to the microorganism; possible risks associated with that 

14 exposure; consumer perceptions of the risks; and the distribution of possible risks among 

15 different segments of the population. For a list of information that Codex recommends 

16 including in a risk profile see Codex (2007).

17

18 A risk profile assists in identifying the risk management questions that need to be 

19 addressed. The risk profile should be clearly and thoroughly documented, so that risk 

20 managers can use it to decide on further action in relation to a specific health issue. If 

21 links are made between risk profiles for other risk assessments, risk profiles can provide 

22 the basis for qualitative ranking of problems for subsequent risk management. 

23

24 Notably, risk profiles can also be used as decision tools that do not lead to risk 

assessment. For examples see the New Zealand Food Safety Authority.325

26

27 2.6.3 What is a Conceptual Model?

28

29 A conceptual model is a written or visual representation you construct of 

predicted relationships between the hazard and exposed subpopulations.430 It is based on 

31 your problem formulation and working hypotheses, supported by preliminary data and 

32 information, and used to organize the conduct of a MRA.

33

34 The conceptual model depicts the movement of the hazardous agent to the host. 

35 Other tiers of conceptual models may identify variables and data needed to conduct the 

36 MRA. A conceptual model (e.g., a source-pathway-receptor model; ―farm-to-fork‖ 

37 model) can be developed early in the planning and scoping process, to the level necessary 

38 to address the risk assessment‘s purpose. For example, in some cases, a pathogen may be 

39 available in multiple media and cause different diseases depending on the route of 

 

2

―Risk profile‖ is sometimes used to refer to summary information at the end of each chapter in 

frameworks based on the ecological risk assessment framework. 

3

http://www.foodsafety.govt.nz/science/risk-profiles/

4 The conceptual model in planning and scoping differs from conceptual models that are used to map how 

parameters are related in modeling software. The term conceptual model is used in both contexts in this 

Guideline.
Microbial Risk Assessment Guideline Page 31

Draft – Do not cite, circulate, or quote

1 exposure. In those cases, during planning and scoping, you can clearly identify the 

2 medium, pathway of exposure, and route of exposure to be assessed.

3

4 Developing a sound and useful conceptual model may require several iterations. 

5 With the conceptual model, you can describe or visualize the relationships among the 

6 assessment and measurement endpoints, the data required, and the methodologies that 

7 will be used to analyze the data. An overall high-level conceptual model as well as more 

8 detailed conceptual models that cover just dose-response or exposure assessment 

9 components may be useful (EPA, 1998a).

10

11 2.6.4 What is an Analysis Plan?

12

13 The analysis plan lays out the approach to be taken by the risk assessment team. 

14 It shows how data sources and information will be used and integrated in the assessment 

15 and how measurement endpoints (e.g., fecal shedding) and uncertainties are related to the 

16 assessment endpoints (e.g., morbidity and mortality). As a product of planning and 

17 scoping, the analysis plan can act as a bridge to the risk assessment. The analysis plan is 

18 the implementation strategy for performing the risk assessment and addressing the 

19 decision needs. It documents the agreements made during the planning and scoping 

20 process and provides details on how the risk assessment will proceed. This provides 

21 transparency to the whole process. In addition, the analysis plan provides measures 

22 against which the final risk assessment and its risk characterization can be evaluated. As 

23 the risk assessment proceeds, the analysis plan may need to be revised to ensure that the 

24 risk assessment still meets the decision needs (EPA, 1998a). If a separate work plan for 

25 logistics is not developed, then staffing, scheduling, and resource details can also go in 

26 the analysis plan.

27

28 2.6.5 How are Data Gaps Identified and Addressed in the Context of 

29 Planning and Scoping?

30

31 Subsequent chapters in this Guideline address many of the tools used to address 

32 data gaps. Incomplete information and data gaps are a significant challenge throughout 

33 the risk assessment discipline. Much of the ―art‖ in risk assessment involves the 

34 judgments regarding incomplete data and data gaps. In addition to missing data, there are 

35 commonly questions about the degree to which available data are representative of the 

36 actual conditions being assessed.

37

38 The extent to which a data gap exists is ultimately a matter of scientific judgment 

39 within the context of what is an acceptable level of confidence. Different assessors and 

40 managers may have different comfort levels for making decisions based on the same data.

41 In some cases there may be differing options about how representative the data are and 

42 whether the data adequately fit in the risk assessment scenario. The quality of existing 

43 data is also considered when determining if a data gap exists (Section 2.6.6). To reduce 

44 the heterogeneity of comfort levels, many different systematic schemes for evaluating 

45 data quality, completeness, and applicability have been developed. Statistical approaches 

46 are standard for evaluating data quantity, but still require a judgment about the 
Page 32 Microbial Risk Assessment Guideline

1 appropriate confidence level for decision making. If the risk assessors and risk managers 

2 on your team cannot agree on what constitutes a significant data gap, the team may need 

3 to take a step back and first agree on criteria for evaluating data. With some data gaps, 

4 risk managers look to risk assessors to tell them whether the data are sufficient, while risk 

5 assessors may claim that a policy decision needs to be made by the risk managers 

6 regarding setting the threshold for sufficiency. These types of situations can stall a risk 

7 assessment, but ultimately the group has to reach agreement on the judgment call or 

8 policy decision. If the existence of a data gap is not obvious and agreed upon easily, the 

9 evidence used to support the identification of the data gap should be clearly documented. 

10

11 Stakeholders may have strong opinions about data gaps. Some may demand a 

12 decision in the absence of data, while others may interpret the goal of science-based 

13 decision making to mean that more complete data must be available to make a decision. 

14 Robust planning and scoping should be able to predict which data gaps have the potential 

15 to cause the most debate. 

16

17 Every parameter in the risk assessment will have some level of incomplete 

18 information. Ranking the importance of the data gaps can help you focus resources on 

19 the most critical data gaps that, if filled, could influence the risk assessment results the 

20 most. Whether a data gap ―matters‖ to the risk assessment results can be evaluated by 

21 conducting sensitivity analysis (Sections 5.3.3 and 6.7) or value-of-information (VOI) 

22 analysis (Section 2.6.7). Once a data gap is identified and determined to be important it 

23 can be a matter of scientific judgment or a policy decision that determines how the data 

24 gap will be addressed in the risk assessment. You can fill the information needs in the 

25 near term using existing data, in the midterm by conducting tests with currently available 

26 test methods to provide data on the topic of interest, or over the long term to develop 

27 better, more realistic understandings of exposure and effects and to construct more 

28 realistic test methods to evaluate pathogens of concern. In cases where an aspect of risk 

29 is likely to be important but insufficient data are available, highlight the deficiency or use 

30 judgment or assumed values to approximate the missing data (see Section 6.4 for further 

31 discussion). Such judgments and approximations should be clearly described and the 

32 implications explained in the risk characterization.

33

34 OSTP states in their scientific integrity memo that, ―The accurate presentation of 

35 scientific and technological information is critical to informed decision making by the 

36 public and policymakers. Agencies should communicate scientific and technological 

37 findings by including a clear explication of underlying assumptions; accurate 

38 contextualization of uncertainties; and a description of the probabilities associated with 

39 both optimistic and pessimistic projections, including best-case and worst-case scenarios 

40 where appropriate‖ (OSTP, 2010).

41

42 Depending upon the circumstances, the utility of a risk assessment may be 

43 compromised if important policy decisions are put on hold while waiting for more 

44 research results. Risk assessors and risk managers need to balance the need to obtain 

45 more data/information against the need to make a timely decision. The data gaps 
Microbial Risk Assessment Guideline Page 33

Draft – Do not cite, circulate, or quote

1 identified in planning and scoping may be very useful to establish a research program 

2 and/or agenda to address current data gaps.

3

4 Expert opinion or judgment is a common source of information used in exposure 

5 assessments. If no other empiric evidence is available, expert judgment may offer the 

6 best available science to inform a model. Alternatively, when data are completely absent 

7 and the availability of expert opinion or judgment is questionable, it is possible to avoid 

8 the need for such data by model simplification (Cox, 2006; Vose, 2008). Such an 

9 approach is particularly worthwhile when empiric evidence is available to inform the 

10 probability distributions of process outputs subsequent to the process that is missing data. 

11 Such  ̳downstream‘ data (i.e., between the missing element and the ultimate exposure 

12 distribution) actually reflect the likelihood of microbe levels given the processes 

13 (modeled or missing) that occurred prior to the process. Therefore, processes for which 

14 data are missing may effectively be skipped over if data are available downstream.

15 

16 Methods for eliciting expert judgments have been suggested (Kaplan, 2000). 

17 Techniques for resolving conflicting opinions among experts focus on having experts cite 

18 the experiences that inform them. In general, a diverse group of experts is preferred 

19 when eliciting input to the exposure assessment.

20

21 Scrutiny and analysis of large amounts of complex data is another use of technical 

22 experts in exposure assessments. Convening an expert panel to review and assess 

23 available evidence about a model‘s variables and parameters is often an effective 

24 technique for endowing the exposure assessment with credible depictions of inputs and 

25 their attendant uncertainties (ECSCC, 2003).

26

27 Ouchi (2004) and Morgan and Henrion (1990) provide summaries of methods and 

28 citations for primary literature in the field of expert elicitation. Some of the methods 

29 summarized by Ouchi (2004) and Morgan and Henrion (1990) include the following:

30

31 a) Behavioral approaches:

32 1) Face-to-face interaction

33 2) Delphi method

34 3) Nominal group technique

35 b) Mathematical approaches (for probabilistic risk analysis):

36 1) Non-Bayesian axiomatic models (opinion pools, performance-based weight 

37 model)

38 2) Bayesian models (additive error and multiplicative error models, stochastic 

39 dependence)

40 3) Psychological scaling models (Thurstone model, Bradley-Terry model, 

41 negative exponential lifetime model)

42 It should be noted that Morgan and Henrion (1990) observed that ―because the 

43 public decision maker must often informally factor in a number of other considerations, it 
Page 34 Microbial Risk Assessment Guideline

1 is rarely of great practical consequence that a more formal treatment in the combining or 

2 weighting of alternative expert views is not possible.‖

3

4 An example of how EPA has used expert judgment is the Office of Air Quality 

5 Planning and Standards (OAQPS) report An Expert Judgment Assessment of the 

6 Concentration-Response Relationship Between PM2.5 Exposure and Mortality (EPA, 

7 2004a). Based on an NRC report recommendation, OAQPS used expert judgment to 

8 develop probability distributions for key sources of uncertainty regarding the mortality 

9 effects of ambient fine (>2.5 microns) particulate matter (PM2.5) exposure. EPA‘s 

Environmental Benefits Mapping and Analysis program (BenMAP)510 includes the 

11 exposure-response functions derived through expert judgment assessment as options for 

12 risk modeling.

13

14 EPA‘s Science Policy Council‘s Draft Expert Elicitation Task Force findings 

15 include (EPA 2009c):

16

17 a) Past experience with expert elicitation at EPA indicates that it can provide useful, 

18 credible results. NAS has highlighted these past efforts as exemplary and 

19 recommended that EPA continue in the direction established by these precedents. 

20 b) The use of expert elicitation is appropriate for some situations; but, not for others. 

21 Factors favoring the use of expert elicitation include: inadequate information to 

22 inform a decision, lack of scientific consensus, and the need to characterize 

23 uncertainty. Factors favoring alternatives to expert elicitation include theoretical 

24 and practical limitations. Typically, an expert elicitation requires a significant 

25 investment of resources and time to provide credible results.

26 c) Expert elicitation can work well when a scientific problem has a body of 

27 knowledge; but, lacks a consensus interpretation. For this case, expert beliefs 

28 about the value and meaning of data can provide valuable assessments and 

29 insights. This may be the case for an emerging scientific challenge or one that 

30 depends on uncertain future events. However, when a problem has abundant 

31 relevant empirical data and relative consensus exists in the scientific community, 

32 there is probably little need to conduct an expert elicitation. At the other end of 

33 the spectrum, if data are inadequate for the experts to develop judgments, an 

34 expert elicitation may not be worthwhile.

35 d) Given that EPA uses other more familiar approaches to characterize uncertainty, 

36 the application and acceptance of expert elicitation at EPA will likely grow with 

37 experience. If early expert elicitation efforts are well designed and implemented, 

38 this will promote the credibility and endorsement of expert elicitation within the 

39 Agency and by external stakeholders.

40 e) The nature of the regulatory process (i.e., legal, political, financial, technical, and 

41 procedural considerations) will influence whether and how to conduct an expert 

42 elicitation and how to communicate and use results. Within the regulatory 

43 process, EPA can use expert elicitation to encourage transparency, credibility, 

 

5

http://www.epa.gov/ttnecas1/benmodels.html 
Microbial Risk Assessment Guideline Page 35

Draft – Do not cite, circulate, or quote

1 objectivity (unbiased and balanced), rigor (control of heuristics and biases), and 

2 relevance to the problem of concern.

3 Hoffmann et al. (2007) used expert elicitation to attribute illnesses associated with 

4 one of eleven major foodborne pathogens to the consumption of one of eleven categories 

5 of food. They used responses from a large panel to create and analyze four uncertainty 

6 measures: (1) agreement among experts; (2) expert agreement with prior estimate; (3) 

7 mean individual expert uncertainty; and (4) variability in experts‘ individual uncertainty. 

8 Hoffman and colleague‘s framework shows how these measures when viewed together 

9 can provide greater insight into the state of knowledge available to support decisions, 

10 than could individual measures. They used statistical analysis to assess the quality of 

11 both expert judgment data and external data. Hoffmann and colleague‘s suggest that 

12 analysis of multiple uncertainty measures is likely to be particularly useful to decision 

13 makers when external validity checks that rely on conventional scientific methods or 

14 further data collection is infeasible or costly.

15

16 2.6.6 How do I Consider Information Quality Including Data Quality?

17

18 Section 515 of the Treasury and General Government Appropriations Act for 

19 Fiscal Year 2001 (Public Law 106-554, also known as the ―Data Quality Act‖ or 

20 ―Information Quality Act‖) directed OMB to issue government-wide guidelines that 

21 ―provide policy and procedural guidance to Federal agencies for ensuring and 

22 maximizing the quality, objectivity, utility, and integrity of information (including 

23 statistical information) disseminated by Federal agencies.‖ Federal agencies responded 

24 to the OMB guidelines (OMB, 2002) by developing Agency specific guidelines. For 

25 example, EPA published Guidelines for Ensuring and Maximizing the Quality, 

26 Objectivity, Utility, and Integrity, of Information Disseminated by the Environmental 

27 Protection Agency (hereafter known as EPA Information Quality (IQ) Guidelines; EPA, 

2002b). EPA‘s IQ Guidelines628 include the following adaptation of the quality principles 

29 found in the Safe Drinking Water Act (SDWA) Amendments of 1996 (EPA, 2002b):

30

31 a) The substance of the information is accurate, reliable and unbiased. This involves 

32 the use of: 

33 1) the best available science and supporting studies conducted in accordance 

34 with sound and objective scientific practices, including, when available, 

35 peer reviewed science and supporting studies; and 

36 2) data collected by accepted methods or best available methods (if the 

37 reliability of the method and the nature of the decision justify the use of 

38 the data).

39 b) The presentation of information on human health, safety, or environmental risks, 

40 consistent with the purpose of the information, is comprehensive, informative, 

41 and understandable. In a document made available to the public, EPA specifies:

 

6 These principles should be adopted or adapted in all Federal agency IQ Guidelines for assessments related 

to evaluations and public heath (OMB, 2002).
Page 36 Microbial Risk Assessment Guideline

1 1) each subpopulation addressed by any estimate of applicable human health 

2 risk or each risk assessment endpoint, including subpopulations if 

3 applicable, addressed by any estimate of applicable ecological risk;

4 2) the expected risk or central estimate of human health risk for the specific 

5 subpopulations affected or the ecological assessment endpoints, including 

6 subpopulations if applicable; 

7 3) each appropriate upper-bound or lower-bound estimate of risk; 

8 4) each significant uncertainty identified in the process of the assessment of 

9 risk and studies that would assist in resolving the uncertainty; and 

10 5) peer-reviewed studies known to the administrator that support, are directly 

11 relevant to, or fail to support any estimate of risk and the methodology 

12 used to reconcile inconsistencies in the scientific data.

13 Discussion of data quality is an important part of planning and scoping. You can 

14 evaluate data quality within the context of your agency‘s data quality guidelines and 

15 work with managers to make decisions about what types of data should or should not be 

16 included based on data quality and scope. Data quality requirements may differ 

17 depending on the planned use of the risk assessment. If data are excluded, it is always 

18 important to note the exclusion and reason for the exclusion. Methods for evaluating data 

19 quality are important tools for producing a risk assessment that has both scientific value 

20 and credibility with stakeholders. 

21

22 In some cases development of Data Quality Objectives may be required by 

23 Agency policy. Data Quality Objectives facilitate transparent documentation of the 

24 justification of why data were included or excluded (IRAC, 2000). The following list 

25 describes several important characteristics that may be helpful to evaluate the usefulness 

26 of data sets for risk assessment, such as submitter information, data source, methods, and 

27 confidentiality (adapted from IRAC, 2000):

28

29 a) General information: Complete name and correspondence address of principal 

30 investigator, purpose of study, and availability of raw data 

31 b) Source of data: Funding source/affiliation of principal investigator or data 

32 collectors, who collected/produced the data, and for numerical data, provide 

33 numerator and denominator study design: type of study, sample size, sampling 

34 frame/sample selection, and how sample relates to the population (is the sample 

35 from a particular country, region or producer?) 

36 c) Data collection: Method of data collection/compilation, age of data, 

37 country/region of origin, time frame for collection (seasonality), and conditions of 

38 collection (field versus laboratory data) 

39 d) Microbial methods: Testing methods (which tests were run), sensitivity and 

40 specificity of test(s), techniques used, precision of measurement, definition of 

41 units being used, species of animals used, if any, and specific organism tested or 

42 studied 
Microbial Risk Assessment Guideline Page 37

Draft – Do not cite, circulate, or quote

1 e) Evaluation of information: Consistency with regard to findings of other 

2 researchers, publications that cite the data, peer review of the data, investigator's 

3 evaluation of data, and investigator's recommended limitations of data 

4 f) Protections for sharing raw data: Confidentiality for human subjects (blinded 

5 data)

6

7 There are general criteria for evaluating data to decide if it should be included in a 

8 risk assessment. Basic questions to evaluate data include the following (adapted from 

9 EPA, 1998a):

10

11 a) Are the study‘s objectives relevant to the risk assessment? The most relevant data 

12 for risk assessment are those that focus on the (1) organism of interest; (2) 

13 population at risk; and (3) circumstances of exposure (e.g., vehicle, level, 

14 timescale, and route).

15 b) Are the variables and conditions the study represents comparable with those 

16 important for the risk assessment?

17 c) Is the study design adequate to meet its objectives?

18 d) Was the study conducted properly?

19 e) Were there associations between observable data and the outcomes (health or 

20 otherwise) of interest?

21 f) Are factors that could increase or attenuate risk (risk factors) controlled for in the 

22 data?

23 g) How are variability and uncertainty treated in the study report?

24 h) Are the data sufficiently robust to be used to support a causal effect between 

25 exposure and infection or illness?

26 i) Does the study meet agency requirements regarding ethics, such as having passed 

27 internal review boards or complying with Agency regulations regarding research? 

For example, EPA‘s Protections for Subjects in Human Research Rule728 requires 

29 that all pregnant women, all nursing women, and all children are excluded from 

30 all studies involving intentional exposure that are intended for submission under 

31 pesticide laws. Additional information on the conduct and use of observational 

32 studies in EPA‘s risk assessments are addressed in Scientific and Ethical 

33 Approaches for Observational Exposure Studies (EPA, 2008).

34 EPA also has general assessment factors, including soundness, applicability and 

35 utility, clarity and completeness, uncertainty and variability, and evaluation and review, 

 

7

http://www.epa.gov/oppfead1/guidance/human-test.htm
Page 38 Microbial Risk Assessment Guideline

1 which are covered in more detail in EPA‘s Science Policy Council Assessment Factors 

2 (EPA, 2003b).

3

4 Since data are never complete and are rarely collected specifically for risk 

5 assessment; many types of data are considered for inclusion in risk assessment. Good 

6 quality data include complete datasets, relevant data, and peer-reviewed data that are 

7 considered high quality by experts in the field and agree with other data sets in terms of 

8 comparison of methods and development of tests. Complete datasets would include 

9 information on all the characteristics listed above. Relevant data may depend on the risk 

10 question that is being addressed. Some characteristics of relevant data include age of 

11 data, region or country of origin, purpose of study, and species involved. 

12

13 2.6.7 What is a Value-of-Information Analysis?

14

15 During the planning and scoping discussions, a question you might ask is whether 

16 to wait for additional research, or is there enough confidence in readily available 

17 information to make a decision (Section 2.4.1). The aim of a value-of-information (VOI) 

18 analysis for the decision maker will be in its ability to determine when no more 

19 information about the risk of microbial pathogens is economically beneficial to making a 

20 decision (Disney and Peters, 2003). For example, VOI is the amount a decision maker 

21 would be willing to pay for information prior to making a decision. The maximum VOI is 

22 for complete information. Consequently, the VOI analysis can contribute to the 

23 identification of cost-effective strategies for reducing a risk to appropriate levels. VOI 

24 analysis provides a way to quantify the value of actions taken to reduce the risk 

25 associated with a decision (Hirshliefer and Riley, 1992).

26

27 Furthermore, decision makers could take a variety of actions to increase their 

28 confidence in the estimates of the probability of human illness, before making a final 

29 decision. VOI analysis could be used to identify optimal ―confidence-increasing 

30 strategies‖ for reducing uncertainty in quantitative risk analyses estimates of the 

31 probability of an occurrence (Disney and Peters, 2003). 

32

33 VOI analysis provides a set of methods for optimizing efforts and resources to 

34 gather, to process, and to apply information to help decision-makers achieve their 

35 objectives (NRC, p). NRC provides a schematic of the application of VOI analysis to 

36 assess the impacts of additional studies in a specific decision context. Information 

37 opportunities that address uncertainties in the baseline model are considered with respect 

38 to the changes they would have on the decision-maker‘s preferred decision option and the 

39 associated change in net benefits. The analysis may also consider any direct costs (for 

40 example, financial) and indirect costs (for example, the health or economic impacts of 

41 delayed decision-making) associated with the information opportunity. The valuation of 

42 information is ultimately driven by the decision-maker‘s values with respect to the 

43 distribution of risks and costs, including any costs associated with delayed decisions 

44 (NRC, 2009).

45

46
Microbial Risk Assessment Guideline Page 39

Draft – Do not cite, circulate, or quote

1 2.6.8 What is a Communications Plan?

2

3 Risk communication (Chapter 8) can be initiated as soon as the risk assessment 

4 process begins and can be incorporated throughout the process; risk communication 

5 should not be an afterthought. During the planning and scoping discussions, the risk 

6 communication specialists need to work with risk assessors, risk managers, and 

7 appropriate stakeholders to develop a communication plan or strategy to be used in all 

8 dealings with the public (essentially all stakeholders). In the plan you should consider:

9

10 a) Clearly identify stakeholders and collaborators (e.g., in multiple partnering 

11 projects).

12

13 b) Set forth specific communication goals (e.g., what will be your proposed 

14 message(s)).

15

16 c) Establish who will be the audiences for whom you will communicate throughout 

17 the risk assessment process. The audience can range from technically 

18 sophisticated risk assessors and knowledgeable risk managers to an educated lay 

19 audience with limited knowledge of risk assessment.

20

21 d) Provide for the development of the content of any communication tailored toward 

22 your projected audiences.

23

24 e) Develop proposed outreach avenues to your audiences (e.g., fact sheets, press 

25 releases, newsletters, notices, open meetings, briefings, emails, websites). 

26

27 You will most likely rely upon communication specialists, including your 

28 agency‘s public affairs office, to get the message across. It is advisable that these 

29 specialists be involved with the risk assessment effort from planning and scoping onward. 

30 A principal spokesperson can be identified (possibly the risk assessor), and you can

31 decide which information channel(s) should be used. Your agency may have guidance 

32 for risk communication plans or stakeholder involvement. For further information

33 regarding risk communication from an assessor‘s point of view, refer to Chapter 8.

34

35 2.7 What Types of Decisions within Risk Assessment are Science Policy?

36

37 This section discusses decisions made within the context of design and data usage 

38 for risk assessment. It does not address the types of policy or regulatory decision making 

39 that occurs after the results of the risk assessment have been considered. Science policy 

40 decisions are usually differentiated from scientific judgment calls. Whereas risk 

41 assessors can make decisions based on scientific judgment, if a decision goes beyond 

42 what would reasonably be considered firmly supported by science, then policy comes 

43 into play. Once policy is involved, then risk managers need to become engaged in the 

44 decision-making. Failing to distinguish between policy decisions and scientific judgment 

45 in a risk assessment is a serious threat to the scientific credibility of the assessment. It is 

46 important to note that:
Page 40 Microbial Risk Assessment Guideline

1 a) The utilization of science policy in the risk assessment process is not meant to 

2 ―bury‖ or ―hide‖ risk management decisions within the risk assessment (i.e., any 

3 science policy position or choice used in the risk assessment process does not 

4 direct the risk assessment itself toward a specific risk management decision).

5 b) To be transparent, policy choices need to be stated explicitly in the risk 

6 assessment.

7 c) Although science policy is utilized in the risk assessment process, it is important 

8 to recognize that the policy positions themselves are developed outside the risk 

9 assessment.

10 d) Scientific data should support science policy positions and risk assessors and risk 

11 managers should ensure that the risk assessment proceeds in a way that best 

12 serves the informational needs for subsequent decision(s) based on the risk 

13 assessment. 

14 Science policy positions and choices are by necessity utilized during the risk 

15 assessment process. You will likely use science policy positions in two major ways in 

16 the risk assessment.

17

18 First, there are some basic, fundamental science policy positions that frame the 

19 risk assessment process to ensure that the risk assessments produced are appropriate 

20 for a particular decision. These scoping ―boundaries‖ for the risk assessment are 

21 articulated during the planning and scoping process and ultimately explained clearly in 

22 the risk characterization (i.e., what will be addressed in the risk assessment for 

23 decision purposes, but also just as importantly, what will not be addressed and why 

24 [e.g., not pertinent to the decision needed]). These science policy positions not only 

25 shape the risk assessment process, but are usually a factor in the decision making

26 process outside the risk assessment.

27

28 Second, the use of default assumptions in a risk assessment is a science policy 

29 choice often invoked when there is a lack of data. These science policy choices are 

30 more specific than the framing science policies mentioned above. Given the nature of 

31 uncertainty and data gaps, default assumptions (sometimes simply called defaults) are 

32 often used to address these uncertainties when data are unavailable or otherwise not 

33 suitable for use. A default assumption is the option chosen on the basis of a science 

34 policy choice that appears to be the best choice in the absence of data to the contrary 

35 (NRC, 1983). The NRC, in its review of risk assessment practices in Science and 

36 Judgment in Risk Assessment (NRC, 1994), acknowledged that default assumptions are 

37 needed to address uncertainty arising from a lack of data and other information. The 

38 report also stated that Agencies should have principles for choosing default options. 

39 When pathogen-specific data are unavailable (i.e., when there are data gaps) or 

40 insufficient to estimate parameters or resolve paradigms, a default can be used in order 
Microbial Risk Assessment Guideline Page 41

Draft – Do not cite, circulate, or quote

1 to continue with the risk assessment. This is a science policy choice, generally agreed 

2 upon during the planning and scoping discussions, when data gaps are identified (see 

3 Section 2.4.1 for information on data gaps). During the risk assessment itself, a default 

4 is used only when essential data are lacking. Point estimates can also be considered 

5 defaults, when the distribution of the parameter only adds unnecessary complexity given 

6 the needs of the risk assessment. For example, drinking water consumption is often 

7 modeled probabilistically for MRAs with a median of 1L per day. The consumption 

value of 2 L/day per person is often used for chemicals and represents the 90th 8 percentile 

9 of the 1994 to 1996 and 1998 Continuing Survey of Food Intake by Individuals 

10 community drinking water consumption data. As illustrated in this example, the choices 

11 you make need to be well within the range of plausible outcomes and often at specific 

12 percentiles (for variability) within that range of observation. The use of 1L versus 

13 2L/day is not related to differences in microbial versus chemical risk assessment.

14 The default assumptions are not pathogen-specific per se, but are relevant to the 

15 data gap in the risk assessment. Defaults are based on published studies, empirical 

16 observations, extrapolation from related observations, and/or scientific theory. A 

17 representative list of the areas where assumptions are commonly made in MRAs is 

18 presented in Appendix A.

19 2.8 What is My Role as a Risk Assessor in Risk Assessment?

20

21 People who perform the risk assessment, in whole or for the most part, are the risk 

22 assessors. One often overlooked responsibility as a risk assessor is to communicate your 

23 key risk findings and conclusions and your confidence in them. Most of these 

24 responsibilities are noted in other sections of this Guideline. Specific responsibilities 

25 often include:

26

27 a) Collect, compile, and document appropriate data

28 b) Let the risk manager know whether the key data used for the assessment are 

29 considered experimental, state-of-the art or generally accepted scientific 

30 knowledge

31 c) Describe quantitative risk estimates in plain English; the use of tables and 

32 graphics may be helpful as a supplement

33 d) Describe the uncertainties inherent in the risk assessment and the default positions 

34 used to address these uncertainties or gaps in the assessment

35 e) Refer the reader to an Agency risk assessment guideline or other easily obtainable 

36 reference that explains terminology (e.g., how a microbial criterion was 

37 developed)

38 f) Determine what types and models are appropriate for the particular risk 

39 assessment

40 g) Input data and models into software 

41 h) Run modules
Page 42 Microbial Risk Assessment Guideline

1 i) Conduct sensitivity analyses

2 j) Identify the subpopulations or systems addressed and provide an explanation for 

3 the selection of these as the ―valued entities.‖

4 k) Describe the level of confidence associated with the conclusions and assumptions 

5 and defaults

6 l) Summarize and identify the key pieces of information critical to your evaluation

7 m) Put this risk assessment into a context with other similar risks that are available to 

8 you and describe how the risk estimated for this pathogen or agent compares to 

9 others regulated by your agency

10 n) Describe how the strengths and weaknesses of the assessment compare with other 

11 assessments prepared in the past

12 o) Describe the rationale and basis for the conclusions drawn by those outside your 

13 agency about this pathogen or agent

14 p) If their conclusions differ from yours, let the manager know whether theirs is a 

15 reasonable alternative; can their conclusions reasonably be derived from the data 

16 set

17 q) Inform the risk manager of the strengths and weaknesses of their evaluations 

18 compared to yours

19 r) If you have developed specific assessments for one or more risk management 

20 options, let the risk manager know what changes in risk would occur under these 

21 various candidate risk management options

22 s) Highlight key or critical issues and, if needed, identify a hierarchy of issues based 

23 on importance to the assessment or those that could influence decisions makers.

24 t) Oversee and manage contractor support on the risk assessment, including review 

25 of documentation and calculations for mathematical errors

26 u) Keep the decision maker informed of the status of your risk assessment and risk 

27 characterization

28 v) Archive the risk assessment and associated materials in a manner consistent with 

29 your organization‘s archiving procedures 

30
Microbial Risk Assessment Guideline Page 43

Draft – Do not cite, circulate, or quote

1 3. HAZARD IDENTIFICATION AND HAZARD 

2 CHARACTERIZATION

3

4 Hazard identification and hazard characterization (HI/HC) are key components of 

risk assessment.85 In HI, the suspect hazard (i.e., the microorganism/adverse effect) is 

6 identified and defined in the context of epidemiological, surveillance, clinical, microbial

7 (agent specific), and environmental (including meteorological and geographical) 

8 information available to the risk assessor. The HC focuses on a particular 

9 microorganism(s) and potential or known mechanisms of host-pathogen interaction, 

10 virulence, and pathogenicity. As discussed in Section 1.6, the epidemiological triangle is 

11 a useful framework for conceptualizing HC. Meteorological and geographic conditions 

12 impact the persistence and transmissibility of microbial agents in the environment and 

13 influence the level of potential exposure. This chapter presents information that is basic 

14 to HI/HC. A list of questions that may be posed during HI/HC is presented in Appendix 

15 B.

16

17 3.1 How do I Define the Hazard?

18

19 The term hazard broadly refers to the subject of an assessment. It can be 

20 interpreted in a number of ways. It may be defined as the stressor agent capable of 

21 causing an adverse effect on the exposed individual or be defined as the adverse effect 

22 itself. The subject of the risk assessment (hazard of interest) is a policy decision driven 

23 by the existing statutes, regulations, or consistency with agency processes. You should 

24 be aware that the term hazard may refer to an agent that is causal or associated with 

25 adverse outcome, the mechanism and metabolic products leading to the outcome, or 

26 simply the subject of an assessment wherein the likelihood for adverse outcomes is being 

27 explored. The adverse effects in humans that result from exposure to microbial agents or 

28 metabolites under favorable host (healthy or susceptible due to certain life-stages/pre29 existing conditions) and environmental conditions may occur soon after exposure or arise 

30 at a substantially later time (sequelae). This guideline focuses on pathogenic infectious 

31 disease hazards.

32

33 Terms such as ―agent‖ and ―stressor‖ may sometimes be used synonymously with 

hazard.934 The microbial Thesaurus developed by EPA (EPA, 2007b) differentiates the 

35 terms as follows: The term ― ̳stressor‘ is used in ecological risk assessment and includes 

36 but is not limited to the connotation that the adverse response can be the result of a lack 

37 of something – such as a habitat – which would be called a  ̳stressor‘. The term  ̳agent‘

38 does not have this connotation.  ̳Agent‘ is used to denote a causative entity that actually 

 

8 As noted previously, in the 1983 NRC Report, Hazard Characterization is the qualitative part of doseresponse assessment. In this Guideline, because of the qualitative nature of Hazard Characterization it is 

considered with Hazard Identification, which is also qualitative. Risk assessors can still organize the risk 

assessment documentation under the headings provided by NRC or WHO. 

9 Epidemiologists may use the term ―agent‖ slightly differently. Agent - a factor (e.g., a microorganism or 

chemical substance) or form of energy whose presence, excessive presence, or in the case of deficiency 

diseases, relative absence is essential for the occurrence of a disease or other adverse health outcome. 

http://www.cdc.gov/excite/library/glossary.htm
Page 44 Microbial Risk Assessment Guideline

1 physically exists as part of the environment and can be used in either ecological or human 

2 health risk assessment.  ̳Hazard‘ is used primarily in human health risk assessment, 

3 although  ̳hazards‘ are not limited to  ̳agents.‘ For example, the number of days spent in 

4 a hospital may be a hazard that correlates with risk of nosocomial infection.

5

6 In the context of MRA for frank pathogens, the term hazard represents the 

7 pathogen‘s potential to generally cause adverse effects in normally healthy humans, while 

8 in the case of opportunistic microorganisms, the term hazard refers to potential to cause 

9 adverse human outcomes under certain environmental and host conditions, most often 

10 when the host is compromised. Thus, hazard may be used to denote the subject of the 

11 assessment that needs further identification and characterization.

12

13 If the hazard is a pathogenic microorganism, then identification of the 

14 microorganism is an important aspect of the hazard description. Microorganisms can be 

15 categorized (defined) based on the methods used to detect them. Nucleic acid-based 

16 assays may result in a different categorization than culture- based assays. As mentioned 

17 previously, quasi-species are particularly hard to define (Section 1.5). It is important to 

18 note that in some situations even small deoxyribonucleic acid (DNA) sequence changes

19 may elicit significantly different adverse outcomes in humans (Battista and Earl, 2004). 

20 But in the case of ―zero tolerance‖ organisms, this distinction does not matter. For 

21 examples of the nuances of microbial nomenclature, refer to Section 3.8. 

22

23 In general, when performing assessments on individual strains or isolates, 

24 identification rather than taxonomy or nomenclature becomes the issue. In this case, 

25 identification refers to the placement of an isolate within an existing taxon, or 

26 determining that it does not match any existing taxon, (i.e., proper labeling of the isolate), 

27 as opposed to the creation of a new group or groups to which the isolate is related, that 

28 share attributes with the isolate (i.e., classification of the isolate). The purpose of 

29 identification is to ensure that you know which taxon or subtaxon you are evaluating.

30

31 Microbial identification is often anything but a trivial exercise. Unless an isolate 

32 has been the subject of a taxonomic study, or is one of the cultures used to establish a 

33 commercial identification method database that is kept current, it is often difficult to 

34 ensure that an isolate belongs unequivocally to a specific taxon. In MRA, to develop 

35 accurate exposure assessment and resulting risk characterization, you should carefully 

36 evaluate the definition of the hazard and pay close attention to how different data sets 

37 used in the risk assessment define the hazard.

38

39 3.2 What are Hazard Identification and Hazard Characterization?

40

41 Hazard identification and hazard characterization provide a qualitative

42 examination of the hazard identified. The quantitative relationship between hazard and 

43 effect is examined in the dose response assessment (see Chapter 4). Hazard identification 

44 provides the framework for gathering relevant information to construct a realistic 

45 scenario focusing on the likely microbial hazards present, so that they can be 

46 appropriately considered for the MRA. In HI, information related to the epidemiological, 
Microbial Risk Assessment Guideline Page 45

Draft – Do not cite, circulate, or quote

1 surveillance, clinical, and microbial aspects of the hazard is reviewed as a critical part of 

2 the assessment. Hazard characterization helps you describe the mechanisms involved in 

3 causing harm and the microorganism‘s ability or potential to cause harmful effects.

4

5 You should be aware that the extent of available data for HI, HC, and exposure 

6 assessment vary greatly in MRAs. Coverage may vary depending on the type of the 

7 assessment (e.g., qualitative, quantitative, retrospective, prospective) and agency policy 

8 or convention. You should consider the extent of coverage and potential overlaps in the 

9 problem formulation step during planning and scoping. In this Guideline, HC and some 

10 elements of exposure assessment are included in the HI/HC chapter. 

11

12 If your Agency uses the WHO/FAO framework (Codex 1999) the preferred 

13 chapter heading for topics discussed in this Guideline (Chapter 3) is Hazard Identification 

14 and the qualitative descriptions related to dose-response and quantitative aspects of dose15 response are in a chapter titled Hazard Characterization.

16

17 3.3 What Hazard Characteristics Can I Consider?

18

19 There are several hazard characteristics that you can consider when assessing a 

20 microorganism or its by-products (adapted from USACHPPM, 2009 and EPA 2009a):

21

22 a) Infectivity – the ability of a pathogen to enter, survive, and 

23 multiply (infect) in a host

24 b) Invasiveness – the ability to degrade and migrate through 

25 the extracellular matrix 

26 c) Virulence – the ability of the pathogen to defeat the host 

27 defenses, increase the severity and longevity of the symptoms

28 d) Pathogenicity – the ability to cause a disease state. It is 

29 the cumulative effect of virulence and invasiveness (see the glossary for a 

30 discussion of pathogenicity versus virulence)

31 e) Host range – which hosts a pathogen can infect. Some 

32 pathogens have very specific host ranges; therefore the disease is limited to one 

33 host. Other pathogens have wide host ranges, and they can cause disease in many 

34 species

35 f) Horizontal gene transfer - the movement (transfer) of 

36 genetic material (e.g., DNA) from one organism taxon to another that, with 

37 maintenance and expression of that genetic material, may lead to antibiotic 

38 resistance traits or acquisition of other virulence factors

39 g) Genetic drift – changes in the frequency of alleles 

40 in a population due to random sampling

41 h) Replication – the ability for a microorganism to multiply 

42 within the environment or the host

43 i) Persistence – the ability of the microorganism to survive in 
Page 46 Microbial Risk Assessment Guideline

1 the environment or the host

2 j) Transmissibility – the ability of a microorganism to 

3 survive, replicate, and pass through animate or inanimate matrices and stay 

4 infective

5 k) Opportunistic Pathogens – the ability of a usually 

6 innocuous microorganism to cause an adverse health effect in a susceptible host 

7 l) Taxonomy and Strain – definition of the hazard

8 with respect to traditional biological classification. Taxonomy and strain variation 

9 have a potentially large impact on risk assessment. The difference in dose10 response range between isolates (and strains) can be orders of magnitude. Some 

11 strains may not be infective for humans. In addition, the ratio of different strains 

12 in the environment can fluctuate

13 m) Resistance to control or treatment processes – the ability 

14 of a pathogen to survive treatments, such as chlorination. If the risk assessment is 

15 for a performance target then the treatment and control processes may be of 

16 central importance

17 Genomics, proteomics, and metabonomics may all be important for hazard 

18 characterization as well as host characterization (EPA, 2006b).

19

20 3.4 How do Microbial Hazards Cause Adverse Outcomes?

21

22 The relationship between a host and a microorganism is complex with dynamic 

23 interplay; there are a number of mechanisms by which a pathogen can induce an illness. 

24 In general, these include either direct invasion of the host cells and colonization of a 

25 specific tissue or organ, causing necrosis or other direct damage or triggering host 

26 responses that are self damaging, or through the production of toxic by-products that can 

27 elicit an adverse effect through toxicological modes of action. Note that toxins may be 

28 produced by microbes in the environment or directly in the host. The conditions under 

29 which microbes produce toxins and how host exposure occurs is important to consider. 

30 For example hazardous algae blooms may result in oral or dermal exposure to toxins 

31 produced by algae in the environment, whereas, pathogenic E. coli produces toxins 

32 during infection which can continue to cause damage to the host even after the microbes 

33 die.

34

35 Among pathogenic organisms, there are several common patterns or themes in the 

36 cascade of events that dictate the progression of disease process. The first essential step 

37 in the establishment of a disease is the ability of the pathogen to adhere to a tissue. This 

38 step, while prompted by the pathogen, is often the result of a host-microorganism 

39 interaction that is host specific. The second step is the invasion/penetration of the host‘s 

40 epithelium, whether the skin, lining of the lungs, or the lining of the gastrointestinal tract. 

41 The pathogens are able to invade the host cell and establish a niche where they can 

42 multiply. The success of a pathogen to initiate and cause disease is limited by its ability 

43 to counteract effectively the host defense mechanism and be able to multiply to a level

44 that elicits a symptomatic response. The ability of the microorganism to effectively 
Microbial Risk Assessment Guideline Page 47

Draft – Do not cite, circulate, or quote

1 defeat or evade the host defense response determines the latency period, intensity, and 

2 persistence of the disease state.

3

4 Modeling mechanisms of infection (e.g. how molecular and cellular host and 

5 pathogen factors interact) may someday be applicable to MRA; however, currently the 

6 science is not developed enough for the pathogens of concern and it is unclear how much 

7 value this feature would add, given the large uncertainties in other areas of MRA.

8

9 3.4.1 What does Virulence and/or Pathogenicity Mean in the Context of 

10 Causing an Adverse Outcome?

11

12 Pathogenic microorganisms have virulence factors with specific modes of action 

for entry, colonization, and adverse health effects.

10 13 The first step in assessing 

14 pathogenicity is to collect the microbial evidence for the adverse health effects associated 

15 with the agent of concern. To cause disease, pathogens must overcome various host 

16 defense systems, and their ability to do this is indicative of the virulence of the 

17 microorganism. Examples of some types of microbial virulence factors include:

18

19 a) Factors that help the microorganism persist in the environment

20

21 b) Factors that help the microorganism evade the host immune system

22

23 c) Expression of surface proteins or polysaccharides that help bind the organism to a 

24 specific site in the host

25

26 d) Production of toxins

27

28 In causing a disease, not only do the pathogenicity and virulence potentials of the 

29 microbial agent play a role, but also the degree of susceptibility of the host and the 

30 influence of environmental factors on exposure influence the final outcome. 

31 Understanding the interactions between a microbial pathogen, the host, and the 

32 environment is key in determining the potential health impact a pathogen will have on an 

33 individual (or a population). The classic epidemiological triangle (disease triad) 

34 illustrates the inter-relationship between the host, pathogen, and environment (See Figure 

35 1.2). 

36

37 3.5 What are the Mechanisms that May Lead to the Development of New 

38 Pathogens or Pathogens with New Traits?

39

40 In the microbial world, there exist mechanisms that consistently produce newer 

41 strains of pathogens or to have existing pathogens acquire more virulent traits from other 

42 microorganisms. Such mechanisms can result in the horizontal transfer of genes within 

43 and between viral and bacterial strains. While horizontal transfer of genes often results in 

 

10 Pathogenicity is the quality or state of being pathogenic, the potential ability to produce disease. 

Virulence is the disease producing power of an organism, the degree of pathogenicity within a group or 

species.
Page 48 Microbial Risk Assessment Guideline

1 reductions in fitness, on some occasions the transfer results in more potent viruses and 

2 other pathogens. Recent advances in whole genome nucleotide sequence analysis 

3 demonstrate that viral, bacterial and protozoan pathogen evolution includes horizontal 

4 gene transfer of virulence factors between different species and high taxa. Thus an 

5 understanding of the role of horizontal gene transfer between different pathogens is 

6 essential for the evaluation of the possible introduction of new microbial hazards. This 

7 may be as a result of an unintentional or deliberate environmental release of natural or 

genetically modified microorganisms.11 8

9

10 Acquisition of new traits comes about by the transfer of genetic traits vertically or 

11 horizontally among microbial species. All living organisms have at least one natural 

12 mechanism for genetic transfer. Techniques of biotechnology take advantage of these 

13 mechanisms to precisely transfer desired characteristics or remove undesirable ones in 

14 genetically modified bacteria. In prokaryotes, these mechanisms include: 1) conjugation, 

15 through which portions of genetic materials are exchanged between two related cells in 

16 physical contact; 2) transduction which occurs through infection by a virus intermediate, 

17 a bacteriophage; and 3) transformation in which there is direct uptake and incorporation 

18 of extracellular DNA. Facilitating these transfers are genes for mobilization of DNA 

19 from one genomic compartment to another, as from a large replicon (chromosome) to a 

20 smaller one (plasmid). These are often found in insertion elements and transposons.

21

22 It is commonly recognized that mobile genetic elements have contributed to rapid 

23 changes in virulence potential by the acquisition of new traits that increase their survival 

24 and adaptation in human hosts and in adverse environmental conditions. When 

25 mobilizing genomic elements (phages, plasmids, insertion elements, or transposons) 

26 acquire such functional gene segments, selection can segregate these into self27 transmissible units, called  ̳genomic islands‘. When specific traits or virulence factors 

28 that contribute to pathogenicity are included within these units, they are called 

29 pathogenicity islands (Knapp et al., 1986; Schmidt et al., 2004). The advent of whole 

30 genome sequencing and other advances in molecular biology has allowed development of 

31 criteria for recognizing pathogenicity islands in microorganisms of interest (Guzman et 

al., 2008; Yoon et al., 2007; Dobrindt et al., 2004).12 32 In some cases, where the suspect 

33 microorganism is known to be related to ones that have been sequenced, the use of 

34 sequence analyses can be employed to look for components of pathogenicity islands or 

virulence factors.13 35 As indicated in section 3.4.1, bacterial pathogenicity determinants 

36 are generally grouped as virulence factors or mechanisms which includes antibiotic 

resistance, pore-forming toxins, superantigens (Schmidt et al., 2004)14 37 and even quorum 

38 sensing (Lerat and Moran, 2004). Thus the knowledge of mechanisms that have the 

39 potential to result in microorganisms with new pathogenic traits may be of critical 

40 importance in conducting certain types of risk assessments.

41

42

 

11 Genetically modified microorganisms are a potential topic for a future volume of this Guideline.

12 http://www.gem.re.kr/paidb/about_paidb.php

13 http://www.ncbi.nlm.nih.gov/PMGifs/Genomes/micr.html; http://www.genomesonline.org/; 

http://www.tigr.org; http://www.sanger.ac.uk/

14 http://www.gem.re.kr/paidb/about_paidb.php
Microbial Risk Assessment Guideline Page 49

Draft – Do not cite, circulate, or quote

1 3.6 What are the Major Categories of Microorganisms?

2

3 The major microbial categories that cause adverse outcomes to humans include 

4 bacteria, fungi, viruses, protozoan parasites, algae, and indeterminate types (Table 3.1). 

5 Helminthes (tapeworms, roundworms) are also considered to be hazardous organisms, 

6 particularly if direct exposure to feces is possible. Although helminthes are multicellular 

7 parasites and not microorganisms, they are sometimes considered in conjunction with 

8 pathogens because infectious stages are too small to be easily detected by the naked eye. 

9

10 There is a vast array of microorganisms and associated literature on pathogenic

11 genera, species, subspecies, strain and subtypes, which is outside the scope of this 

12 document. There is vast literature available on taxonomic characterization of microbial 

13 agents. However, depending on the specific requirement of an assessment, it is 

14 recommended that an assessor consult relevant literature and subject matter experts as 

15 needed. Under some circumstances the hazard may not be identifiable however the 

16 human health effects may be distinct. These are hazardous agents of indeterminate type 

17 but may still be clinically well defined enough to facilitate risk assessment approaches.

18

19 Table 3.1 presents some of the major categories of microbial hazards in the 

20 context of MRAs (Labbe and Garcia, 2001; Peter, 1998; Alexopoulos et al., 1996). The 

21 broad categorization of microbial organisms should help you understand how an agent in 

22 a given category causes disease in humans. The placement of hazardous organisms into 

23 broad categories is particularly important in retrospective assessments to narrow the 

24 focus of investigation based on documented history for the category in question.

25

26 Batz et al. (2004) constructed a comprehensive list of pathogens for the 

27 Foodborne Illness Risk Ranking Model (FIRRM) analytical software tool using data 

28 generated by various federal agencies. This includes estimates of the incidence of 

29 foodborne illness by CDC as reported by Mead et al. (1999) and cost-of-illness studies by 

30 the USDA‘s Economic Research Service, with additional data reported by FDA, USDA 

31 FSIS, and Center for Science in the Public Interest. An additional list of foodborne 

32 pathogenic organisms and toxins compiled by FDA commonly called Bad Bug Book may 

33 also be useful (CFSAN, 2006). A review of waterborne pathogens is provided by Craun 

34 et al. (2010).

35

EPA‘s Water Quality Criteria program,15 36 which addresses microbial 

37 contamination of the nation's waters under the Safe Drinking Water Act (SDWA) and the 

38 Clean Water Acts (CWA), provides information on microbial methods, Health 

39 Advisories, Regulatory Support and Criteria Documents. Health Advisories serve as 

40 informal technical guidance to assist federal, state, and local officials responsible for 

41 protecting public health when emergency spills or contamination situations occur. 

42 Criteria documents and guidance for drinking water contaminants provide information so 

43 preliminary decisions can be made as to whether the contaminant is a significant health 

44 threat via drinking water exposure and whether sufficient data exists to perform 

45 quantitative risk assessments. 

 

15 http://www.epa.gov/waterscience/criteria/humanhealth/microbial/
Page 50 Microbial Risk Assessment Guideline

1

EPA‘s Candidate Contaminant List of microbial organisms (CCL3)16 2 is a list of 

3 twelve microbial agents currently not subject to any regulation based on a contaminant's 

4 potential to occur in public water systems and the potential for public health concern. The 

5 list of chemical contaminants includes cyanotoxins produced and released by

6 cyanobacteria ("blue-green algae"). 

7

 

16 http://www.epa.gov/ogwdw000/ccl/ccl3.html#microbial
Microbial Risk Assessment Guideline Page 51

Draft – Do not cite, circulate, or quote

1 Table 3.1 Major Categories of Foodborne and Waterborne Microorganisms 

2 * The vehicle exposure/pathway may be important as the agent is indeterminate 

Category Features

Examples Morphological Physiological Genetic Pathogenicity

Adaptation 

Mechanisms

Bacteria Francisella

tularensis, 

Brucella suis, 

Escherichia coli

O157:H7, 

Salmonella,

Campylobacter,

Listeria,

Legionella

Cyanobacteria, 

Vibrio

Single-celled 

prokaryotes

Metabolically 

diverse, invasive, 

produce 

intra/extracellular 

toxins 

No nucleus, 

double stranded 

DNA, presence 

of extra 

chromosomal 

DNA/Plasmids, 

mutate 

frequently, 

horizontal gene 

transfer 

mechanisms

Some species form 

spores to withstand 

adverse conditions.

Mutation and gene 

transfer, pathogenicity 

islands and other 

genetic 

traits/mechanisms lead 

to frequent strain 

variation, acquisition of 

enhanced virulence 

traits and adaptation to 

new environments , 

toxin production

Viruses Noroviruses, 

Adenoviruses,

Hepatitis A

Acellular, most 

are enveloped 

with geometric 

structures

Metabolically 

inactive, 

obligatory 

parasitic, host 

dependent

Single or double 

stranded RNA or 

DNA, mutate 

rapidly in host

Frequent genetic drift, 

shift and other genetic 

mechanisms may lead 

to changes in antigenic 

properties, host 

survival/adaptation and 

result in more virulent 

variants/strains

Protozoa Toxoplasma 

gondii, 

Giardia spp.,

Cryptosporidium 

hominis, 

Cryptosporidium 

parvum, 

Naegleria spp.

Single celled 

Eukaryotes of 

the Protista, 

display different 

morphologic 

structures and 

stages of 

infectivity 

Host dependent 

parasites 

Nucleus present, 

not known to 

mutate as 

frequently as 

bacteria and 

viruses

Cyst and spore formed 

to withstand adverse 

conditions. Relatively 

stable genome, 

however, 

mutation and gene 

transfer may lead to 

strain variation, 

enhanced virulence and 

adaptation to new 

environment 

Fungi Aspergillus 

fumigatus, 

Penicillium, 

Candida, 

Aspergillus flavus

Eukaryote

Mostly 

multicellular and 

filamentous, 

pathogenic fungi 

are mostly 

unicellular (e.g., 

yeasts)

Metabolically 

diverse, invasive, 

produce 

mycotoxins 

Nucleus present, 

sometimes 

presence of extra 

chromosomal 

DNA/Plasmids

Spores

Algae

Chlorophyta, 

Rhodophyta

Dinoflagellata

Pfiesteria 

piscicid, ―red 

tide‖

Gambierdiscus 

toxicus 

(Ciguatera)

Single-celled 

photosynthetic 

organisms, often 

dinoflagellates, 

Eukaryotes 

Metabolically 

diverse highly 

complex life 

cycle, a few toxin 

producing

Nucleus present Three typical forms are 

classified as amoeboid, 

flagellated and encysted 

varieties

Indeterminate

agent*

Can vary, 

unknown

Can vary, 

unknown

Can vary Can vary, 

unknown

Can vary, unknown
Page 52 Microbial Risk Assessment Guideline

1

2 3.7 What Methodological Approaches can be Used to Identify and Quantify 

3 Microorganisms?

4

5 You should be familiar with laboratory approaches for identifying and quantifying 

6 the microorganism(s) of concern. For any datasets that required laboratory methods that 

7 were used in risk assessment, carefully review issues related to sensitivity, specificity, 

8 limit of detection, sampling method, and sample size. Extensive reviews of microbial

9 methods are available (CFSAN, 2001; USDA 2008; AOAC International 2007).

10

11 Identifying an unknown microorganism is a two-step process, requiring methods 

12 to characterize the traits of an organism and approaches to interpret the characterization 

13 data. Methods to generate characterization data range from traditional culture-based 

14 phenotypic and biochemical tests to the more recently developed molecular techniques. 

15 Approaches can reflect the evolutionary inheritance of traits (e.g., lineal decent), the 

16 intrinsic properties of the organism regardless of how they were acquired, or a 

17 combination of both.

18

19 Methods used for identification and quantification are often related to similar ones 

20 used for classification of microbes. However, the different purposes of classification and 

21 identification require separate considerations, even when the same technology is utilized 

22 for both. Under the controlled conditions of a classification study a particular 

23 methodology may be exquisite in its ability to distinguish among selected related isolates, 

24 but that same method may be only able to provide an approximate identification when 

25 one encounters an isolate outside the lab without the benefit of closely related isolates 

26 available for comparison.

27

28 While traditional culturing methods for identification and quantification of 

29 microbes are still the mainstay for fecal indicators, it is clear that this approach by itself 

30 does not allow for complete evaluation of microbial organisms beyond the genus and 

31 species level, and does not detect strains that may be active/infectious but non-culturable. 

32 The classical culture-based approach relies on culture of the organism in question, 

33 isolation in pure culture, and a study of the morphological, biochemical, physiological 

34 and other traits. However, not all microorganisms are culturable (see viable-but-not35 culturable discussion in Section 3.8). 

36

37 More sophisticated methods have the ability to discriminate subtypes and also 

38 capture information on pathogenicity determinants of interest at the genotypic and 

39 phenotypic levels, either with isolated cultures or even in mixed enrichments. Some 

40 molecular methods can be used even in the screening steps early on, to capture the 

41 virulence potential of a specific microorganism or multiple microorganisms (profiles) 

42 without the isolation steps that are generally less sensitive and more time-consuming. 

43

44
Microbial Risk Assessment Guideline Page 53

Draft – Do not cite, circulate, or quote

1 3.8 Are there any Special Concerns Regarding Microbial Detection Methods?

2

3 Purpose of risk assessment and choice of the method(s)

4

5 Discrimination down to the smallest organizational level (i.e., strain) is not 

6 necessarily the objective of all identifications done for risk assessments. You should 

7 determine what information is critical for completing the assessment before pursuing a 

8 specific level of identification. If the pathogenic potential is associated with all members 

9 of some higher order taxa, then identification to that higher taxonomic level may be all 

10 that is required for broad-based assessments. For example, knowledge that an isolate is a 

11 member of a species complex with many shared characteristics may be sufficient to 

12 permit assessment of potential for pathogenic effects, such as the Mycobacterium avium

13 Complex (MAC). However, if evaluation of a specific incident is the purpose of an 

14 assessment, it is important to note that even the lowest levels of taxon (SNP variant) are 

15 known to elicit significant adverse outcomes. For examples, an enterohemorrhagic type 

16 of Escherichia coli known as serotype O157:H7, can cause serious hemolytic uremic 

17 syndrome and even death. It is known that the degree of adverse outcomes seen with E. 

18 coli O157:H7 infections vary distinctly among different clades (group of SNP subtypes) 

19 (Manning et al., 2008). Thus, in retrospective assessments you should pay attention to 

20 detailed information within a taxonomic subunit.

21

22 When doing assessments for broad categories of microorganisms, genus-level 

23 identifications may be all that is needed. Culturing techniques that involve phenotypic 

24 analyses may suffice. However, for this level of identification, 16S rDNA analyses have 

25 become commonplace and are usually deemed adequate for prokaryotes. Morphological 

26 features are traditionally used for fungi and protozoa, but biochemical and molecular 

27 methods are beginning to be essential to avoid miss-identifications.

28

29 Culture related issues

30

31 Many non-spore-forming bacteria exposed to environmental stress conditions may 

32 decline in number and not be detectable by traditional culture based laboratory methods, 

33 depending on the level of detection for each method and the number of replicate cultures 

34 employed. To ensure that microorganisms in the viable-but-nonculturable (VBNC) state 

35 are not missed, you should be aware of relevant methods of direct identification such as 

molecular methodologies17 36 for detection and direct microscopy. For microorganisms 

37 present at low levels, specific enrichment cultures can be employed that allow growth of 

38 very low population numbers to levels high enough for traditional culture techniques. A

39 VNBC state has been described where metabolic characteristics are quantifiable though 

40 cells cannot be grown on traditional culture media, most notably in the genus Vibrio, 

41 under conditions when water temperatures drop below 10°C (Smith and Oliver, 2006; 

 

17 There are methods to isolate the DNA from environmental samples that do not require a pre-enrichment 

culture, such as length heterogeneity-polymerase chain reaction (LH-PCR). Methods such as LH-PCR 

require as little as 10 ng of DNA in the PCR reaction to amplify 16S rRNA sequences (Bisson, et al., 

2007). In addition, PCR has been used to detect antibiotic resistance genes in bacteria found in deli-meats 

(Li and Wang, 2010). 
Page 54 Microbial Risk Assessment Guideline

1 Fischer-Le Saux et al., 2002; Rowman, 2004; Huq et al., 2000). Others have shown that 

2 under such stress conditions very small numbers of surviving cells are able to grow to 

3 higher levels using dead cell materials remaining in the stressed culture, therefore giving 

4 the appearance that the cell state had changed from nonculturable back to culturable 

5 (Bogosian et al., 1998). Molecular methods generally detect living and non-living 

6 pathogens, unless specific methods have been developed that target aspects of their 

7 growth viability (Sen and Ashbolt, 2011). However, some important pathogens, such as 

8 Norovirus, cannot be reliably grown in cell culture, so there is currently no way to check 

9 on the infectivity of environmentally recovered virions of Norovirus.

10

11 Similarly in the case of parasites and viruses that are nonculturable and do not 

12 replicate outside the host, serological or advanced molecular methods have been routinely 

13 utilized in identification and reporting. Other methods of detection may also be needed 

14 where pre-formed metabolites/toxins may be present but the suspect pathogen cannot be 

15 recovered, for example, emetic toxin for Bacillus cereus or endotoxins in aerosols.

16

17 Amplification of organisms by growing in culture can also exert selective 

18 pressure, which can skew identification of microbes from environmental or clinical 

19 samples. You should be aware of any methods related considerations that might impact 

20 your judgment of the relevance or quality of a particular study. 

21

22 Level of discriminating power

23

24 The two major subtyping approaches commonly used are based on our ability to 

25 discriminate phenotypic or genotypic traits. The phenotypic approach includes 

26 serotyping, phage typing, multilocus enzyme electrophoresis, and esterase typing. The 

27 genetic subtyping approach encompasses pulsed field gel electrophoresis (PFGE), 

28 ribotyping, PCR-based subtyping techniques (e.g., random amplification of polymorphic 

29 DNA [RAPD]), amplified fragment length polymorphism (AFLP), PCR-restriction 

30 fragment length polymorphism (PCR-RFLP), repetitive element PCR (REP-PCR), DNA 

31 sequencing-based subtyping techniques (e.g., multilocus sequence typing [MLST]) (Liu, 

32 2006), multilocus genotype typing (MLGT), and single nucleotide polymorphisms 

33 (SNPs). Some of these molecular based subtyping techniques provide not only powerful 

34 discriminating capabilities to further identify a unit of potential hazard in question, but 

35 also to relate microbial traits to public health outcomes and perhaps most importantly, 

36 enables subtype-based surveillance to detect outbreaks.

37

38 Depending on the pathogen/unit of hazard of interest, pay close attention to the 

39 discriminating power of the methods used and level of details needed in risk assessment 

40 based on the risk management needs. For example, there are about 3000 serotypes within 

41 the species Salmonella enterica with differing levels of human pathogenicity. Subtyping 

42 strategies include serotyping and further determination of the virulence traits important to 

43 public health. For example, subtyping S. enterica serotype Typhimurium definitive type 

44 104 (DT104) involves identifying isolates through serotyping, followed by antimicrobial 

45 susceptibility testing to identify the R-type ACSSuT (ampicillin, chloramphenicol, 

46 streptomycin, sulfonamides, and tetracycline) antibiotic resistance pattern. S. enterica 
Microbial Risk Assessment Guideline Page 55

Draft – Do not cite, circulate, or quote

1 serotype Typhimurium isolates with R-type ACSSuT are subsequently phage typed to 

2 specifically identify DT104 (Akkina et al., 1999).

3

4 In the case of the bacterial human pathogen Listeria monocytogenes, which is 

5 made up of a range of strains/genotypes with varying degree of pathogenicity, some 

6 strains may be highly pathogenic and sometimes deadly; others may be relatively less 

7 virulent and cause little harm in the host. The ability to differentiate strains of L.

8 monocytogenes is particularly important for microbial hazard identification and for 

9 tracking transmission of pathogenic strains within a given environment via food 

10 processing environments. The purpose of subtyping is to link human infections that may 

11 be related, detecting unusual clusters of human disease, and determining the source of 

12 exposure thorough epidemiological investigation, as well as finding and controlling the 

13 source of contamination. The application of molecular techniques has facilitated the 

14 identification and characterization of major virulence-associated genes and proteins in L. 

15 monocytogenes. Various DNA fragment-based typing methods have been used to 

16 differentiate L. monocytogenes strains at the subspecies level to include epidemic clones, 

17 genotypes, lineage types, and serotypes. 

18

19 Differences in methodological approaches and choice of method(s)

20

21 Phenotypic methods include techniques that directly or indirectly detect, measure, 

22 or characterize features of a microorganism resulting from the observable expression of 

23 its (total) genetic constitution. Phenotypic characteristics of bacteria include 

24 morphological, physiological, and biochemical features. Methods for characterizing 

25 phenotype require growth of the microorganism in pure culture under appropriate 

26 conditions. Chemotaxonomic methods examine phenotype by using quantitative analysis 

27 of the organism‘s chemical constituents. Genotypic methods directly compare nucleic 

28 acid sequences, rather than rely on gene expression.

29

30 Approaches to data interpretation can be determinative, numeric taxonomic, or 

31 phylogenetic. Sometimes neither genotypic nor phenotypic methods alone suffice for 

32 either classification or identification of some bacteria, but it may be possible to combine 

33 these methods using polyphasic taxonomy in which data from phenotypic, 

34 chemotaxonomic (e.g., cell-envelope lipids, electrophoresis patterns), and molecular 

35 methods (e.g., ribosomal deoxyribonucleic acid [rDNA], DNA gyrase, subunit B [gyrB]) 

36 are combined. When results from these methods all agree, you can usually rely on the 

37 outcome. However, conclusive data may not be always available for a microorganism of 

38 interest.

39

40 Other considerations for identification

41

42 The proper use of methodology needed to identify (classify) the subject of an 

43 assessment depends on the level of detail required, whether prospective or retrospective, 

44 and cannot be over emphasized. When the hazard units are whole taxa, you need to 

45 consider the historical perspective and the current state of taxonomy and nomenclature 

46 for that unit to ensure that relevant data and information are compiled. Scientific and 
Page 56 Microbial Risk Assessment Guideline

1 technological advances make the field of microbial classification dynamic in that the 

2 taxonomic and other ways of classification are continually improving our understanding 

3 of the unit of infectious structure (e.g., genus, species, subtypes, SNPs), their risk 

4 potential, and mechanisms of pathogenicity. Phylogenetic analyses of microorganisms 

5 have resulted in frequent re-assignment of microorganisms into different genera or 

6 species, so synonyms for the microorganism may need to be tracked. Additionally, the 

7 evolving nature of microorganisms enables them to acquire newer traits for 

8 pathogenicity, host range, specificity, adaptability, and survivability outside of and within 

9 the host that add to the complexity of HI.

10

11 You should consider defining the unit of hazard in the context of the evolving 

12 taxonomic information and risk management needs. For example, the term Burkholderia 

13 cepacia had been applied both to a single species and groups of strains, termed 

14 genomovars (Mahenthiralingam, 2000; Vandamme et al., 1997, 2000). These groups of 

15 strains have subsequently acquired species status, with independent names being 

16 established. The latter have been called, and remain, the Burkholderia cepacia complex 

17 (Bcc). Burkholderia cepacia is now construed as a single species within the Bcc, but 

18 isolates of other species in the complex have, at earlier times, been called B. cepacia, 

19 with attendant literature using only that epithet. However, at times the unit of hazard 

20 may not be identifiable as a structural taxonomic unit. You should become aware of such 

21 issues to be able to refine the scope of the unit of hazard in question.

22

23 When performing assessments with a hazard unit that is an individual strain or 

24 isolate, identification rather than taxonomy or nomenclature becomes the issue. In this 

25 case identification refers to the verification of the labeling of an isolate, as well as the 

26 placement of an isolate within an existing taxon, or determining that it does not match 

27 any existing taxon. The purpose of identification is to be sure that you know which 

28 organism/unit of hazard applies to a data set. For example, if occurrence data are for a 

29 set of isolates and dose-response data are for only one of the isolates, then the uncertainty 

30 this introduces needs to be discussed. 

31

32 In rare situations, the unit of hazard (e.g., genus, species, subtypes) may not be 

33 identifiable. However, defined host symptomatology may lead to an underlying suspect 

34 agent (Soller et al., 2010). Under such circumstances, the vehicle of transmission of the 

35 suspect agent may be considered as the unit of hazard. Overall, the level of details in 

36 microbial HI/HC depends on the risk assessment characteristics and risk management 

37 needs (e.g., issues, goals).

38

39 3.9 What Host Factors Can I Take into Consideration?

40

41 You can consider the following factors when evaluating potential health effects 

42 due to exposure to a pathogen. Following each factor is a short description of how the 

43 factor may influence the health outcome. There may be little or no data for these factors, 

44 so their consideration may be limited to a qualitative discussion (adapted from 

45 USACHPPM, 2009; EPA 2009a).

46
Microbial Risk Assessment Guideline Page 57

Draft – Do not cite, circulate, or quote

1 a) Age/Life Stage – Life stage refers to a distinguishable time frame in an 

2 individual's life characterized by unique and relatively stable behavioral and/or 

3 physiological characteristics that are associated with development and growth. 

4 Children and elderly are usually considered more susceptible due to immaturity or 

5 other potential weaknesses in their immune systems. Behaviors that affect 

6 pathogen exposure patterns may also be related to age. For example, children 

7 may also experience greater exposure (therefore larger doses) due to their 

8 behaviors (e.g., close-proximity playing, hand-to-mouth tendencies). 

9

10 b) Pregnancy – Women who are pregnant and fetuses are generally considered to be 

11 a sensitive and perhaps more susceptible life stage. For example, Hepatitis E, 

12 which is a self-limiting disease for most people, can cause up to 20% mortality in 

13 women in the third trimester of pregnancy (Jameel, 1999). The underlying reason 

14 for increased susceptibility is due to the influence of pregnancy on the immune 

15 system. Pregnancy can also change exposure patterns, for example, water 

16 consumption in pregnant women is higher than the in the general population.

17

18 c) Immune Status – The immune system plays an important role in clearing 

19 pathogens from the human body, which influences potential health effects. 

20 Previous exposure may confer limited and/or short term protective immunity 

21 (Frost et al., 2005) or long lasting immunity (especially to viruses). The converse 

22 of this may also be true, that is, when individuals or subpopulations that have not 

23 previously been exposed to particular pathogens, infection and illness rates can be 

24 higher than would otherwise be anticipated. ―Traveler‘s diarrhea‖ is an observed 

25 phenomenon that exemplifies this type of situation. In addition, individuals with 

26 compromised immune systems who come into contact with pathogens may react 

27 very differently from individuals with intact immune systems. Definitions of 

28 subpopulations or life stages included in the risk assessment should include the 

29 criteria used to classify individuals as immunocompromised and may need to be 

30 limited to specific identifiable types of immune defects. Note that children, 

31 newborns, the elderly, and pregnant women are also immunologically different 

32 from healthy adults.

33

34 d) Natural Microbiota – The competition that is provided by the presence of natural 

35 microbiota influences the impact a pathogenic organism will exert on a host. 

36 Prior treatment with antimicrobials that alter gut flora is a recognized risk factor 

37 for infection with C. difficile and Salmonella.

38

39 e) Nutrition – The nutritional state of the host affects the immune system. 

40 Malnourished individuals tend to have weaker immune defenses than well

41 nourished individuals.

42

f) Clearance Mechanisms18 43 – The human body has clearance systems to remove 

44 foreign particles from tissues. For example, nasal and oral clearance systems can 

45 remove airborne hazards and intestinal clearance mechanisms can help remove 

 

18 These may also be referred to as innate immunity.
Page 58 Microbial Risk Assessment Guideline

1 gastrointestinal pathogens. Intestinal hypermotility may represent a host defense 

2 mechanism against Giardia (Anderson et al., 2006). Gastric acidity is a barrier 

3 that is a primary factor affecting the outcome of infections from food and 

4 waterborne pathogens. Certain behaviors, for example smoking, may affect the 

5 functionality of clearance mechanisms (e.g., mucociliary escalator); therefore, the 

6 functionality of clearance mechanisms of various host subpopulations should be 

7 considered. 

8

9 g) Genetic Factors – The expression of certain genetic factors may increase an 

10 individual‘s sensitivity to particular pathogens. Therefore, if genetic factors are 

11 known for the organism being assessed, expression level differences could be 

12 considered when characterizing the pathogen and performing dose-response 

13 modeling. Some genetic factors that influence pathogen dynamics may be linked 

14 to race, which could be considered in the characterization of a hazard and dose15 response modeling.

16

17 h) Preexisting Conditions – Preexisting conditions may affect a host‘s response to a 

18 pathogen, therefore, preexisting conditions should be considered, if possible. 

19 Physical and emotional stressors may also influence host susceptibility.

20

21 i) Carrier Status (Persistence in Population) – The possibility of some humans to 

22 serve as ―carriers‖ for pathogens needs to be considered when estimating the 

23 potential spread of pathogens, especially when the a carrier may interact with 

24 hosts who are considered susceptible.

25

26 j) Treatment Efficacy – Whether or not effective treatment is available may be 

27 important to risk assessment. Treatment efficacy can be a major determinant of 

28 mortality.

29

30 k) Social and Behavioral Traits – Social and behavioral traits primarily affect 

31 exposure patterns. For example, a relatively small proportion of the population is 

32 responsible for consuming the majority of raw and partially cooked shellfish 

33 (FDA, 2005; see age and behavior above). Social and behavioral traits may also 

34 be associated with cultural and racial identities and may be important for specific 

35 consideration in a risk assessment.

36

37 3.10 How does Life Stage Affect Sensitivity to Infection and Disease 

38 Manifestation?

39

40 Sensitivity to infection is based on both exposure to a pathogen and the integrity 

41 of the immune system. Early life stages have a combination of factors that increase both 

42 the possibility of infection and intensity and duration of the disease. Young children 

43 spend the first two years or more close to the ground whether crawling or playing. 

44 Reliance on hands to move around, hand-to-mouth activity, and eating with hands as 

45 opposed to using utensils all raise the possibility for exposure and ingestion of pathogenic 

46 microorganisms compared with more mature individuals.
Microbial Risk Assessment Guideline Page 59

Draft – Do not cite, circulate, or quote

1

2 A second issue with children is the immature development of their immune 

3 systems. Infants have not had the exposure to the wide range of microbial stressors 

4 needed to afford protection. Newborn infants have passively acquired immunity from 

5 their mothers, which dissipates over the ensuing months. This provides some protection 

6 for newborn children, allowing them to develop the array of acquired immunity needed 

7 for full protection. Nonetheless, young children tend to be more susceptible than older 

8 individuals.

9

10 A significant consideration is the intensity and persistence of the disease in 

11 newborn infants. In the absence of prior experience with a specific pathogen, the body 

12 requires more time to develop and process its immune response. Therefore, a pathogen 

13 can elicit more pronounced and longer lasting effects from infection resulting in more 

14 adverse outcomes in children compared with adults. In developing countries, where 

15 treatments are limited, children succumb to gastrointestinal infections at a higher rate 

16 than in countries where timely medical intervention is widely available.

17

18 EPA‘s Risk Assessment Forum published Guidance on Selecting Age Groups for 

19 Monitoring and Assessing Childhood Exposures to Environmental Contaminants (EPA, 

20 2005). Based on physiological and behavioral milestones EPA recommends that children 

21 be grouped by the following ages:

22

23 a) Less than 12 months old: birth to <1 month, 1 to <3 months, 3 to <6 months, and 

24 6 to <12 months.

25

26 b) Greater than 12 months old: 1 to <2 years, 2 to <3 years, 3 to <6 years, 6 to <11 

27 years, 11 to <16 years, and 16 to <21 years.

28

29 In addition to children, the elderly may also be more 

30 susceptible to infections than healthy adults. Risk assessments designed to include the 

31 whole population should include discussion of the elderly, even if data specific to this 

32 subgroup are not available. 

33

34 3.11 What Environmental Factors Can I Take into Consideration?

35

36 From the point of origin until it reaches the host, a given microorganism interacts 

37 with the environment at various levels that influence its survivability and virulence. 

38 Hence, at the point when the microorganism finally encounters the host, the ability to 

39 cause adverse outcomes depends on the complex interactions (microorganism-matrix, 

40 matrix-matrix, microorganism-carriers/vectors) that the microorganism experienced 

41 during transport. For example, the survivability, virulence, transmission and successful 

42 development of the diseases/adverse outcome for a given microorganism may be 

43 influenced by the run-off from animal husbandry or from wild animal habitats into water 

44 bodies that can reach the target host directly through exposure to water or indirectly 

45 through zoonotic transmission routes or contaminated foods. Thus, you should consider 

46 the potential role of the non-host environmental conditions in influencing changes in the 
Page 60 Microbial Risk Assessment Guideline

1 properties of a microorganism to render it potentially hazardous. These may include 

2 matrices (air, water, soil, food), fomites, vectors, and carriers. As the physical, chemical, 

3 and biological traits of the matrix can influence the microorganism‘s virulence and 

4 survivability, a thorough characterization of the matrix becomes critical in a given risk 

5 assessment. Environmental factors can be considered with the epidemiological triad in 

6 mind (Figure 1.2), which also provides context for segueing into exposure assessment 

7 (see Chapter 5).

8

9

10
Microbial Risk Assessment Guideline Page 61

Draft – Do not cite, circulate, or quote

1 4. DOSE-RESPONSE ASSESSMENT

2

3 The goal of the ―dose-response assessment‖ in MRA is to establish the 

4 relationship between the dose of a pathogen that individuals or subpopulations are 

5 exposed to and the probability of adverse health effects (e.g., infection, illness, death) to 

6 individuals or subpopulations. Qualitative evaluation (hazard characterization) of a 

7 pathogen is also included in the conclusions drawn with regard to potential health 

8 impacts, particularly if data for a quantitative MRA are not available. From the estimated 

9 quantitative relationship (dose-response model), the probability of potential adverse 

10 health effects of a given severity can be estimated from a given exposure to a pathogen. 

11 The exposure assessment and dose-response assessment are combined in the risk 

12 characterization step where risk due to a particular exposure for a defined subpopulation 

13 and a defined hazard is described. It is important to note that because the dose-response 

14 assessment is used in the context of an exposure assessment, it is critical that the dose 

units used for quantitative estimates are comparable.19 15

16

17 It should be noted up front that there is no general accepted guidance regarding 

18 when the available microbial dose response data are sufficiently representative for a 

19 particular scenario, as the factors associated with each scenario are likely to be unique to 

20 each case.

21

22 The information in this chapter is organized into two sections: 1) general 

23 considerations for dose-response modeling (Section 4.1); and 2) current practice in dose24 response modeling methods (Section 4.2). The section on ―General Considerations for 

25 Dose-Response Modeling‖ discusses the technical application of using statistical models 

26 to estimate health effects due to exposure. The second section focuses on the currently

27 available quantitative dose-response models and on the development of new alternative 

28 models.

29

30 4.1 What is Dose-Response Modeling and What are Some General 

31 Considerations for Dose-Response Modeling?

32

33 Dose-response modeling is the process of using mathematical relationships to 

34 describe the probability of an adverse health effect (e.g., infection, illness) occurring in 

35 an individual or the frequency of an adverse health effect in a subpopulation when that 

36 individual or subpopulation is exposed to a specific dose of pathogenic microorganisms. 

37 The dose level may be measured in terms of a discrete number of organisms (e.g., 

38 oocysts), colony forming units (cfu), plaque forming units (pfu), or by molecular methods 

39 that may estimate gene copies or cell equivalents. A cfu represents one to several viable 

40 bacteria cells while a pfu represents one to several infectious viral particles. 

41 Alternatively, a dose level may be specified by an average administered or ingested dose

42 or represented by some other measure (e.g., median infectious dose [ID50] units). The 

43 most common practice of dose-response modeling has been the fitting of limited data sets 

 

19 For example, the number of organisms ingested in a serving for a human volunteer trial might be 

converted to number of organisms in a daily exposure. The units might go from event based in the doseresponse assessment to a series of daily exposures over a lifetime for the exposure assessment. 
Page 62 Microbial Risk Assessment Guideline

1 that have been derived from experimental trials to statistical models that are often, but not 

2 always, biologically based. More recently, researchers have also been using outbreak 

3 data for dynamic dose-response modeling of disease incidence in subpopulations, and 

4 there is also a movement towards the development of models that incorporate the 

5 inherent complexities found in any biological system. 

6

7 4.1.1 How do I Choose Between Modeling a Discrete Dose Versus an 

8 Average Dose?

9

10 Pathogen doses are inherently discrete. While chemical doses are expressed in 

11 mass units (e.g., mg/kg), pathogen doses are usually expressed as counts of organisms 

12 (e.g., oocysts/liter) or an average dose (e.g., mean cfu per serving). Although a group 

13 exposure may be described in terms of average pathogen concentration, pathogens are 

14 distributed in a particular medium (water, air, food), such that each individual may not 

15 receive exactly the same number of organisms. If the concentration of pathogens is very 

16 low, some individuals could be exposed to zero pathogens, while others could be exposed 

17 to one or more pathogens. In this type of situation, the heterogeneity of the pathogen 

distribution in the matrix can be very important.20 18

19

20 While the average dose of pathogens is continuous and can take any value, the 

21 actual number of organisms that an individual may consume is a discrete quantity. In the 

22 context of a clinical feeding trial, the distribution of pathogens or cfu in the delivery 

23 matrix generally is assumed to be random but homogenous (i.e., with the same mean), 

24 with the probability of exposure to a discrete quantity of organisms or cfu (0, 1, 2, etc.) 

25 given by the Poisson distribution. Because distributional data is preferred in MRA over 

26 point estimates (OMB, 2007b), if you only have data on the average dose, then you can 

27 incorporate some assumption about the distribution (e.g., Poisson, extra-Poisson, 

28 exponential, mixture). Using Cryptosporidium parvum as an example, for an average 

29 dose of 0.5 oocysts, most (60 percent) individuals would consume no oocysts, while 30 

30 percent would consume a single oocyst and 10 percent would consume 2 or more 

31 oocysts. Drinking water exposures are usually low, often below an average dose of 1 ×

10-432 organisms per liter, which essentially means that 1 out of 10,000 individuals would 

ingest a single organism after drinking one liter of water at that concentration.21 33 In 

34 modeling feeding trial data where each dose is not directly counted, it is reasonable to 

35 assume a Poisson distribution, but this may not be a reasonable assumption in modeling 

36 outbreak data.

37

38 For low exposures, the discrete nature of the exposure constrains the maximum 

39 risk of infection to the probability of exposure. That is, in the previous example, if the 

40 pathogen were 100 percent infective, only one individual would become infected. You 

 

20 Pathogens may clump together, particularly if the matrix has other components that aid clumping.

21 In all practicality, doses less than 0.01 represent dilutions of single organisms, with an insignificant 

probability (though still not zero) of exposure to more than one organism. As a result the response is a 

virtually-linear function of dose at very low doses. Any function used for pathogen dose-response must 

follow the same ―rule.‖ Otherwise, a probability of infection greater than the probability of exposure could 

be predicted. 
Microbial Risk Assessment Guideline Page 63

Draft – Do not cite, circulate, or quote

1 should consider using discrete dose-response models in these cases, as the risk is usually 

2 expressed as a unit-pathogen infectivity, which is greatly influenced by the high Poisson 

3 variability in the dose. In contrast, while exposure to pathogens in food is typically very 

4 low, due to the potential for pathogen growth in some foods prior to consumption, the 

5 exposure distribution may be extremely skewed with some frequency of exposures in the 

6 range of millions to billions of organisms per exposure. In these cases, the Poisson 

7 variability around the mean dose is trivial, such that doses in this range can be considered 

8 to be continuous and strict adherence to discrete models is not required. In these cases,

9 you can use quantitative dose-response models based on the average pathogen dose to 

10 estimate the probability of infection or illness (Haas et al., 1999). 

11

12 The Beta-Binomial dose-response model assumes that the exact number of 

13 organisms ingested is known, which is suitable for a feeding trial in which each 

14 administered dose has been enumerated. The exponential and beta-Poisson models 

15 assume that the number of organisms between subjects in a dose group is Poisson 

16 distributed with a fixed mean (FAO/WHO, 2009).

17

18 4.1.2 What is the One-Hit Model and Why is it the Preferred Model?

19

20 One-hit (or no-threshold) dose-response models are generally the most relevant 

21 for microbial dose-response assessment. However, these models may not apply to all 

22 pathogens that cause illness by producing pre-formed toxins in food and they may be 

23 inappropriate for modeling illness and mortality.

24

25 FAO/WHO (2003) observed that despite the traditional concept of a ―minimal 

26 infectious dose‖ in the microbial literature, attempts to define the numerical value of a 

27 threshold level of pathogens that must be ingested in order for the microorganism to 

28 produce infection or disease have typically been unsuccessful. ―An alternative 

29 hypothesis is that, due to the potential for microorganisms to multiply within the host, 

30 infection may result from the survival of a single, viable, infectious pathogenic organism 

31 (―single-hit concept‖). This implies that, no matter how low the dose, there is always, at 

32 least in a mathematical sense, and possibly very small, a non-zero probability of infection 

33 and illness‖ (FAO/WHO, 2003). Similarly, NRC (2003) noted that unlike chemical risk 

34 assessment, ―microbial dose-response assessment for infectious pathogens does not 

35 produce any concept analogous to the no observed adverse effect level (NOAEL), since a 

36 single microbial cell may (under the right circumstances) produce illness.‖ Finally, NRC

37 (2005) observed: ―In assessing risks attributable to exposure to microorganisms, it has 

38 frequently been asserted that there exists a threshold (minimum infectious dose) below 

39 which there is no risk to the population. Such a concept is not consistent with the current 

40 understanding of microbiological risk assessment.... The no-risk concept originated from 

41 the fact that in trials (either animal or human), low doses of microorganisms often 

42 produced no adverse effects in exposed subjects.... [However,] all animal and human 

43 exposure data that have been subjected to dose-response analysis are consistent with 

44 models in which the dose intercept is zero and the value of the conditional dose-response 

45 relationship for one organism is non-zero.‖ Empirically, it is impossible to distinguish 

46 between a very low non-zero risk and a true threshold. Risk assessors must rely on 
Page 64 Microbial Risk Assessment Guideline

1 concept and theory to establish the most relevant dose-response modeling approaches. 

2 Therefore, in the ensuing discussion, the one-hit dose-response models are given 

3 preference over alternative models typically used for chemical dose-response modeling 

4 (e.g., log-normal, log-logistic, Weibull). 

5

6 The one-hit model assumes that one infectious organism has the potential to cause 

7 infection. When an individual organism is ingested, the probability that the organism 

8 defeats the host barriers and initiates an infection may be represented by a unit-infectivity 

9 measure. This measure is termed ―r‖ by convention, as r is the rate constant in the most10 commonly used exponential dose-response model. The exponential distribution is based 

11 on the assumption that each organism is capable of initiating an infection and behaves 

12 independently from other organisms within the host, leading to a binomial probability of 

13 infection. The assumptions of a Poisson pathogen distribution and a binomial probability 

14 of infection lead to a family of models referred to as one-hit models, where the name 

15 relates to the concept that only a single organism is necessary to cause infection. 

16 Different pathogen distributions can be assumed in addition to the Poisson. If there is 

17 reason to believe that the pathogens are not uniformly distributed in the exposure medium 

18 (e.g., clumped), a negative binomial or other skewed distribution can be used (Haas et al., 

19 1995; Teunis et al., 2008b), although the numerical algorithms become more 

20 complicated. From a modeling perspective, the one-hit model means that there is a non21 zero probability that any given organism will survive the host defenses to initiate an 

22 infection, no matter how low the probability may be. Unit-infectivity probabilities can 

range from 1 × 10-10 23 to greater than 0.1 (Teunis et al., 1996; Haas et al., 1999). Although, 

24 conceptually based on biologically plausible mechanisms, most of these models do not 

25 rely on independently validated parameters describing the mechanisms model the 

26 underlying physiological processes explicitly, but rely on fitting curves to empirically 

27 observed data. The potential for applying physiologically-based models is discussed in 

28 section 4.2.8.

29

30 The one-hit concept also applies to the production of illness in that, once an 

31 individual is infected, progression to illness can proceed without additional exposure. 

32 Replication within the host of that single pathogen from the initial exposure would 

33 eventually result in enough pathogens to produce illness symptoms. However, the 

34 standard one-hit dose-response models are typically used for assessing the risk of 

35 infection rather than illness (or mortality). Morbidity and mortality may be expressed in 

36 only a small fraction of infected individuals. Since morbidity is currently most 

37 commonly assumed to be independent of dose (but conditional on infection) it is best to 

38 use a morbidity or mortality ratio applied to the risk of infection (Haas et al., 1995). As 

39 a special case (―quorum sensing‖), illness may not occur until the number of organisms 

40 reaches a critical mass at which time they release the toxins resulting in illness. 

41 However, the increase in the number of organisms is a result of growth within the host 

42 rather than an increase in the exposure dose. The concept of density dependent quorum 

43 sensing is distinct from a threshold for administered dose, because of the possibility, 

44 however small, that a single ingested organism may survive the multiple barriers in the 

45 gut to become established and reproduce (FAO/WHO, 2003). Threshold models have not 

46 been demonstrated to provide significant improvements in fit over the exponential and 
Microbial Risk Assessment Guideline Page 65

Draft – Do not cite, circulate, or quote

1 beta-Poisson models, but their use has been advocated on the basis of analysis of the 

2 infection process and interpretation of epidemiological data.

3

4 4.1.3 What Important Factors Can I Consider in Dose-Response 

5 Assessment?

6

7 Exposure Route

8

9 The route of exposure can have a significant bearing on the dose-response curve. 

10 As noted above, for infection to occur and result in a disease state, the organism must 

11 penetrate the host‘s defenses whose capability varies with tissue type. You need to be 

12 aware that route of exposures can influence both the slope of the dose-response curve as 

13 well as the manifestation. For example, Adenovirus can be highly infectious through an 

14 inhalation route of exposure, and appears to be less infectious through ingestion. 

15 Therefore, matching the route of exposure, with appropriate dose-response information is 

16 important. Discussion of the implications of using the available dose-response dataset for 

17 broader exposure scenarios is helpful.

18

19 In chemical risk assessment, route-to-route extrapolation of internal dose for 

systemic effects can be performed if adequate toxicokinetic data exist.22 20 For pathogens, 

21 although systemic involvement can be an important factor, some pathogens can only 

22 infect specific tissue types. Other pathogens can infect many different tissue types and 

23 have a different dose-response and health endpoint for those different tissues. For 

24 example, Cryptosporidium usually infects the gastrointestinal tissues, but in 

25 immunocompromised patients, it can infect other tissues including lungs and result in a 

26 different set of symptoms (O‘Donoghue, 1995). Usually the most obvious concern is 

27 where the route of exposure leads directly to the most susceptible tissues, such as 

28 diarrhea from ingestion, pneumonia from inhalation, or dermal lesions from skin contact. 

29 However for pathogens that cause illness through release of toxins, the host tissue that is 

30 most susceptible to the toxin may be remote from the physical location of pathogen 

31 replication.

32

33 Exposure Medium

34

35 The nature of the exposure medium can influence the probability that an 

36 individual pathogen will survive host defenses and initiate an infection. For oral 

37 exposure, pathogens can be ingested in food, water, or from direct contact with fomites or 

38 infected individuals. The primary factor in this process is the initial line of defense 

39 against the pathogen—stomach acid and digestive enzymes. If the ingestion medium 

40 serves to raise the pH in the stomach content, the probability of pathogen survival is 

41 enhanced for many organisms, perhaps by orders of magnitude, depending on the extent 

42 of acid buffering. The nature of the food matrix, for example, with respect to food 

43 structure and fat content, can vary considerably in enhancing or limiting survival of 

44 pathogens in the food matrix and in the host gastrointestinal ecosystem (Ross, 2008). In 

 

22 Chemical risk assessors have defined many different aspects of dose, such as potential dose, applied 

dose, absorbed dose, internal dose, and delivered or biologically effective dose. 
Page 66 Microbial Risk Assessment Guideline

1 addition, taking stomach-acid reducers for acid-reflux disease can provide protection for 

2 the pathogen. Ingestion of drinking water, on the other hand, offers no protection from 

3 stomach acid. 

4

5 In the dose-response documentation, you should discuss possible effects of 

6 different matrices relevant to the assessment. You could compare the delivery matrix 

7 used in generating dose-response data from a human feeding study to the matrix being 

8 considered in the exposure scenario. For example, if the dose-response data is from 

9 feeding trials using a water matrix and the exposure scenario in the risk assessment is 

10 juice, then you should clearly describe the difference and elaborate on what the potential 

11 implications of those differences might mean for the risk assessment.

12

13 Pathogen

14

15 During hazard identification the pathogen of concern is defined. In the dose16 response assessment, discussion of whether or not the pathogen of concern matches the 

17 pathogen for which dose-response data is critical. This issue is important because 

18 substantial variability in virulence and infectivity has been shown for closely related 

19 pathogens. For example, the relative infectivity of the Salmonella enterica serotypes 

20 (Coleman and Marks, 2000; Soller et al., 2007) and C. parvum isolates (Haas et al, 1999; 

21 Messner et al, 2001; Teunis et al., 2002; Englehardt and Swartout, 2004) used during 

22 human dose-response challenge studies varied by several orders of magnitude. Although 

23 you may only have dose-response data for one or a few isolates of a species, risk 

24 managers may be interested in the species as a whole (e.g., the C. parvum example 

25 above) or the genus (given that a number of Cryptosporidium spp. may infect 

26 humans)(Xiao, 2010). Isolates only represent a small fraction of the genetic diversity of 

27 the species that is likely to occur in nature. Discussion of the implications of using the 

28 available dose-response dataset for broader exposure scenarios is helpful. There is no 

29 general guidance regarding when the available dose response data are sufficiently 

30 representative for a particular scenario, as the factors associated with each scenario are 

31 likely to be unique to each case.

32

33 Data on the actual pathogen of concern are preferred over data on surrogate 

organisms.

23 34 In cases where data on the pathogen of concern are not available, data on 

35 surrogate organisms can be used if solid biological evidence, such as common virulence 

36 factors, can support that choice. The biological basis for the use of the surrogate must be 

37 clear (FAO/WHO, 2003).

38

39 Haas et al. (1999) provides a method for testing whether differences among 

40 strains are statistically significant. It is also important to determine whether the strains 

41 can be considered a representative sample from the pathogen of concern or an extreme 

42 case. In cases where the strain is representative, you may want to use a mixed model with 

43 random strain effects. In cases where the strain is an extreme example, a bounding 

44 approach can be used (Vose, 2008).

45

 

23 ―Surrogate organism‖ synonyms include ―index pathogen‖ and ―reference pathogen.‖
Microbial Risk Assessment Guideline Page 67

Draft – Do not cite, circulate, or quote

1 Host

2

3 During hazard identification you should discuss host factors that are relevant for 

4 the risk assessment scenario. Given the current state of knowledge, quantitative 

5 microbial dose-response assessment for humans requires some human pathogenicity data. 

6 When using dose-response data from human trials, you should discuss the characteristics 

7 of the subpopulation in the trials compared to the subpopulation defined in the scope of 

8 the risk assessment. You can specifically discuss the types of individuals that are 

9 explicitly represented in the human trials (e.g., healthy adults) and individuals that were 

10 excluded from the trials (e.g., children, elderly, immune compromised, and pregnant 

11 women). Describe the potential implications to the different subpopulations or life 

12 stages, and point out any tools that you used to compensate for the differences, such as 

13 information from epidemiological data from outbreak situations. Also discuss the 

14 likelihood that individuals in the trials may have had some immunity to the pathogen 

15 being evaluated. 

16

17 Although human data are preferable, animal studies and in vitro studies may 

18 provide useful information on host-pathogen interactions (FAO/WHO, 2003). The 

19 strengths and limitations of those data, within the context of your risk assessment 

20 scenario, should be clearly explained. Some pathogens have evolved a narrow host 

21 range, while others have evolved a broad host range. Many important pathogens are 

22 host-specific, but over 60 percent of human pathogens have multiple domestic mammal 

23 hosts (Cleaveland et al., 2001). Among pathogens of wild primates, only 10 percent of 

24 bacteria, 13 percent of viruses, and 28 percent of protozoa are host-specific, and more 

25 than 100 pathogens (including 19 bacteria and 30 viruses) infect both wild primates and 

26 humans (Pedersen et al., 2005). Viruses mutate rapidly and can ―jump‖ to new host 

27 species more easily than bacteria or protozoa. On the other hand, protozoa can be much 

28 less discriminating in their choice of host species, often infecting humans after passing 

29 through wild and domesticated animal species. However, the disease manifestations can 

30 be quite different, implying that different pathogenic mechanisms are in operation. As an 

31 example, C. parvum typically causes diarrhea in humans with no systemic involvement

32 but kills (immunocompromised) mice from generalized systemic distribution with no 

33 diarrheal symptoms. One host cannot be used as a surrogate for another in all respects, 

34 and extrapolation from surrogate host data may require adjustments. Therefore, when 

35 surrogate animal models are used, the biological basis for and limitations of the use of the 

36 surrogate must be clear (FAO/WHO 2003).

37

38 Endpoint

39

40 The endpoints that are typically modeled in MRA are infection, illness, and death. 

41 Infection is the most immediate health effect for direct modeling, as it is the first 

42 manifestation of exposure to a pathogen. However, infection is difficult to assess in 

43 humans if there are no clinical symptoms. Infection is generally equated with 

44 colonization of some tissue, either externally or internally. Animal studies can include 

45 specific tissue analysis of sacrificed groups at intervals following the original inoculation 

46 to analyze the clearance time and/or the level of colonization (EPA, 2007a). However, 
Page 68 Microbial Risk Assessment Guideline

1 colonization per se is difficult to measure in humans, so typical markers (measurement 

2 endpoints) of infection such as shedding of the pathogen in feces or urine, presence of 

3 antibodies in the blood (seroconversion) or clinical symptoms are often used to determine 

4 if infection has occurred. Although (asymptomatic) infection has no direct adverse health 

5 impact, it can be crucial in determining the risk of illness and plays an important role 

6 when estimating the impact of secondary transmission. Illness and mortality are the 

7 primary endpoints of health concern. Illness endpoints could range from gastrointestinal 

8 distress to long-term sequelae. The risks of illness and death generally are estimated 

9 from the risk of infection by applying morbidity or mortality ratios, but can be modeled

10 directly in certain circumstances. For example, the most commonly employed dose 

11 response relationship for the ingestion of Salmonella through a waterborne route of 

12 exposure is for an endpoint of illness, not infection. Define the health endpoint of 

13 concern and specifically relate that definition to the clinical case definition used in any 

14 utilized dose-response studies. If there are any differences between the endpoint 

15 definition for the risk assessment and the case definition for data from trials or outbreak 

16 studies, you should describe the implications of those differences to the interpretation of 

17 the risk assessment results.

18

19 Inconsistencies in illness definitions can introduce uncertainties. For example, the 

20 definitions of diarrhea may differ slightly between different clinical trials or 

21 epidemiological studies and may be based on moisture context of stools. Illness 

22 definition endpoints can be quite complex. For example, EPA‘s bacterial ambient water 

23 quality criteria for recreational waters are based on highly credible gastrointestinal illness

24 as defined as ―any one of the following unmistakable or combinations of symptoms

25 (within 8 to 10 days of swimming): (1) vomiting; (2) diarrhea with fever or a disabling 

26 condition (remained home, remained in bed or sought medical advice because of 

27 symptoms); (3) stomachache or nausea accompanied by a fever‖ (Dufour, 1984). Note 

28 that individuals with only diarrhea are not ―counted‖ as a case with the above definition. 

29 Other studies may include diarrhea alone, without fever as a valid case of gastrointestinal 

30 illness. For example, norovirus does not typically cause fever. You should pay close 

31 attention to the nuances of how endpoints are defined and discuss the impact any 

32 differences may have on the risk assessment.

33

34 How treatment can impact the health outcome should be discussed. Some 

35 infections are treatable and others are not. Treatment efficacy could influence 

36 determining the most relevant health outcome.

37

38 Sources of Data

39

40 Clearly document what sources of data were considered, utilized, and omitted, 

41 and provide justification for those decisions. The most precise human data are the direct 

42 pathogen-challenge studies (clinical feeding trials), in which human volunteer subjects 

43 are fed known doses of a specific ―enteric‖ pathogen and observed for gastro-intestinal 

44 symptoms over a period of time. These data can be modeled directly to obtain dose45 response relationships and parameters for application to specific human exposure 

46 scenarios. For ethical reasons, direct pathogen-challenge studies are typically limited to
Microbial Risk Assessment Guideline Page 69

Draft – Do not cite, circulate, or quote

1 healthy adult subjects challenged with a pathogen that has well-characterized health 

2 outcomes that are no more serious than temporary diarrhea that is self-clearing. Human 

3 pathogen-challenge data has been used in EPA regulations to predict the risk of giardiasis 

4 (EPA, 1989, 1998b, 1999) and cryptosporidiosis (EPA, 2006a). Although data from 

5 clinical feeding trials are carefully collected and documented, there may be limitations to 

6 extrapolating from experimental conditions that need to be addressed (e.g., subjects 

7 limited to healthy adults, use of non-wild-type pathogen strains, dose delivery matrix 

8 buffered to increase the likelihood of pathogen survival).

9

10 Human dose-response information can also be obtained from epidemiological 

11 data (primarily retrospective outbreak analyses). Although such data may be more 

12 representative of the actual host-pathogen-matrix combination of concern, there is usually 

13 considerable uncertainty in the dose estimate, number of exposed or number of 

14 responders, or combinations of those variables. One significant advantage that 

15 epidemiological data have over clinical feeding trials is the potential for evaluating the 

16 relative risk of sensitive populations, as was done for Escherichia coli O157:H7 (Teunis 

17 et al., 2004, 2008a). 

18

19 Animal dose-response data have been used to estimate the human dose-response 

20 curve and to estimate the innate pathogen variability across strains. However, these data 

21 are difficult to translate directly to human dose-response and require critical evaluation 

22 prior to use due to all of the uncertainties associated with interspecies extrapolation. 

23 Epidemiological information also can be used to calibrate (―anchor‖) dose-response 

24 curves derived from animal data with respect to the relevant human response range; this 

25 was done in the FDA/USDA/CDC Listeria monocytogenes risk assessment by shifting 

26 the mouse mortality dose-response curve based on human mortality rates attributed to L.

27 monocytogenes (FDA/USDA/CDC, 2003). In addition, the variability of median lethal 

28 doses (LD50s) in mice across Listeria strains was used as an estimate of the pathogen 

29 virulence variability. The assumption was made that the pathogen virulence variability 

30 was similar to the level of variability likely to be encountered in human exposures and 

31 was useful for dose-response uncertainty analysis. Note that this use of animal data is not 

32 in lieu of human data, but is used to support assumptions about characteristics of human 

33 data, such as potential range of variability. For more information on the use of animal 

34 data in MRA see EPA (2009a).

35

36 Outbreak investigations can often provide valuable information about the 

37 etiologic agent. A pathogen‘s ability to produce an outbreak depends on specific 

38 characteristics such as ability to survive in the environment and rate of growth or die off, 

39 potential to cause disease at a given dose, most likely transmission route, and capacity to 

40 spread through person-to-person contact. Therefore, studying the details of outbreaks can 

41 help risk assessors develop exposure scenarios for specific pathogens, and data from 

42 outbreaks can be an important comparison for dose-response models based on human 

43 feeding trials or animal models. The major limitation to outbreak investigation data is 

44 that investigators often collect a narrow range of information. Their main objective is to 

45 rapidly identify the vehicle and prevent additional infections. Occasionally, data from 

46 concurrent drinking water testing are available, or frozen or unopened suspect food 
Page 70 Microbial Risk Assessment Guideline

1 remains for sample collection and accurate identification and enumeration of pathogens.

2 If actual levels of food or water contamination can be measured or estimated, it may be 

3 possible to estimate the dose-response relationship from outbreak data. An outbreak that 

4 is characterized by a low attack rate in a very large population may provide an 

5 opportunity to define the host-response to very low doses of a pathogen (Teunis et al., 

6 2004).

7

8 Like outbreak data, annual surveillance statistics provide a way to evaluate MRA 

9 models. The initial results of a MRA model can sometimes be cross-checked by 

10 comparison with public health surveillance data. The accuracy of dose-response models 

11 may be assessed by combining them with exposure estimates known to be realistic and 

12 determining if the results approximate the incidence of illness estimated from 

13 surveillance data, taking into account the uncertainty due to under-reporting. Using 

14 annual disease statistics in modeling dose-response and exposure estimates implicitly 

15 includes the entire population and the wide variety of factors that can influence the 

16 response. Another benefit is that surveillance databases may have sufficient detail to 

17 analyze special subpopulations or life stages such as the elderly or the 

18 immunocompromised. If information is available, the surveillance summaries or a series 

19 of reported foodborne or waterborne outbreaks can also identify the etiologic agents 

20 causing disease outbreaks and often the sources of the contamination. Consultation with 

21 public health surveillance experts can provide a ―reality check‖ on the preliminary results 

22 of risk ranking and other quantitative MRAs.

23

24 4.1.4 How Can I Model the Spread of Disease in the Population?

25 

26 In addition to primary transmission via contaminated media (e.g., water or food), 

27 many microbial pathogens can also be transmitted via person-to-person contact and cause 

infection and disease.24 28 This route of microbial exposure, occurring from an infected 

29 person rather than from contaminated media is also referred to as ―secondary 

30 transmission.‖ You can discuss any known features of secondary transmission when 

31 characterizing the microbial hazard, regardless of whether the scope of the risk 

32 assessment includes modeling secondary transmission. The ability with which an 

33 organism can be transmitted from an infected individual to an individual that is 

34 susceptible to infection is an important consideration and should be evaluated during the 

35 exposure assessment if possible.

36

37 Secondary cases (often represented in epidemiological studies by a secondary 

38 attack rate) generally refer to cases that occur among contacts, within the incubation 

39 period of the pathogen, and following exposure to a primary case. In some cases, direct 

40 person-to-person transmission cannot be distinguished from contamination of the 

41 immediate environment (e.g., toddlers sharing toys versus direct physical contact during 

42 play). Depending on the purpose of the assessment, it may be appropriate that the 

 

24 Some situations blur the line between secondary transmission and environmental transmission. For 

example, infections due to a primary case causing an outbreak of cases in a daycare setting is usually 

classified as secondary transmission even though transmission may be from both direct human contacts and 

contaminated objects, food, or water exposures.
Microbial Risk Assessment Guideline Page 71

Draft – Do not cite, circulate, or quote

1 definition of secondary transmission include infections that result from propagation of the 

2 specific exposure of interest, but not encompass distant transmissions (separated by time 

3 and/or space) that may be more appropriately considered to result as a function of person4 to-environment-to-person transmission. Temporal and spatial limitations can be 

5 specifically noted in the definition of secondary transmission. You can discuss the full 

6 range of scenarios that qualify as both primary and secondary transmission. The above 

7 definition of secondary transmission is limited to avoid overlap with pathogen occurrence 

8 in the environment (person-environment-person), although people are, of course, part of 

9 the environment. However, the potential for reintroduction of the pathogen into the 

10 exposure media could also be within the definition of secondary transmission.

11

12 The degree to which the exposed population is susceptible to infection and illness 

13 is an important factor in deciding whether to explicitly model secondary transmission in a 

14 risk assessment. Population susceptibility and immunity change dynamically as a 

15 population is exposed to a pathogen. Moreover, since many infections are asymptomatic, 

16 it is common for pathogens to be transmitted from person to person during asymptomatic 

17 infections or during asymptomatic periods of an infection (before or after symptoms), 

18 when infected individuals interact normally with susceptible individuals. Previous 

19 infection by the same organism often confers temporary or permanent immunity, 

20 although individual immunity may be partial (e.g., resistance to infection may be 

21 overcome when challenged by the same pathogen at a high dose or by a genetically 

22 different strain). If a significant proportion of the population is immune, such as from an 

23 immunization program, secondary spread of disease can be virtually prevented. 

24

25 MRA models can be configured to account for secondary transmission and 

26 immunity in a population through the use of a dynamic model (Anderson and May,

27 1991). These models, which can take several forms (deterministic or stochastic),

28 characterize the dynamic epidemiological status of the population (e.g. susceptible to 

29 infection, symptomatic infection, immune). Static MRA models do not, by their nature 

30 consider secondary transmission, although dose-response parameters derived from static 

31 models may be incorporated into dynamic models. You can indicate if and how 

32 secondary transmission is included in the assessment. Inclusion of secondary 

33 transmission in MRAs often provides non-intuitive results; therefore, if secondary 

34 transmission and other innate characteristics of infectious disease transmission are not 

35 included in the assessment, provide a sound justification for this decision. 

36

37 4.1.5 What Can I Address for Each Model to Improve Transparency?

38

39 To promote transparency in presenting the dose-response assessment, you can

40 address the following points for each model presented:

41

42 a) Assumptions

43 1) Clearly state models‘ key assumptions

44 2) Discuss assumptions inherent when extrapolating to doses lower than 

45 those used in studies. 
Page 72 Microbial Risk Assessment Guideline

1 3) Discuss flexibility in approaches to the dose-response relationship 

2 depending upon the pathogen being considered and the assumption about a 

3 no-threshold effect (i.e., can it be assumed that one organism is sufficient 

4 to produce infection?).

5 b) Applicability of Models

6 1) Discuss the biological rationale for the model

7 2) Discuss the applicability of each model to various exposure situations

8 3) Discuss limitations of models

9 4) Articulate strengths/weaknesses and advantages/disadvantages of the 

10 models

11 c) Results

12 1) Discuss the type of information that the various models are expected to 

13 provide. 

14 2) Discuss the use of likelihood methods to compare how well dose-response 

15 models fit the data

16

17 4.2 What is Current Practice in Quantitative Dose-Response Modeling for 

18 Microbial Illness?

19

20 This section briefly summarizes some common dose-response models and how 

21 those models have been used in previous MRAs. It also discusses the output of dose22 response modeling and evaluating uncertainty and accounting for life stages and 

23 subpopulations. 

24

25 4.2.1 What Models Can I Use for Microbial Dose-Response Assessment?

26

27 Dose-response models are mathematical functions that yield a probability of an 

28 adverse health effect as a function of dose. Numerous dose-response relationships for 

29 microbial endpoints have been published in the peer-reviewed literature. No 

30 comprehensive summary of dose-response models is available for all human pathogens. 

31 However, a thorough summary of dose response models for waterborne pathogens can be 

32 found in Table 4.1. The two most commonly used dose-response models are the 

33 exponential and beta-Poisson. Several alternative models have also been proposed as 

34 alternatives for MRA, including two-parameter models such as log-normal, log-logistic, 

35 and extreme value models (Pinsky, 2000), and three-parameter models, such as Weibull 

36 gamma (Farber et al., 1996), exponential gamma, Weibull exponential, and the shifted 

37 Weibull model (Kodell, 2002). However it should be noted that there is no single 

38 selection criteria for dose-response models that is universally used. Several criteria that 

39 could be used are provided in Section 4.2.3. You should explain the logic behind your 

40 model selection and strengths and limitations of the model selection.

41
Microbial Risk Assessment Guideline Page 73

Draft – Do not cite, circulate, or quote

1 The models discussed in this section estimate risks for exposed individuals; thus, 

they are known as individual risk models.25 2 Population risks (the incidence of disease 

3 among a group of exposed individuals) are generally constructed by combining the 

4 results of individual risk models with estimates of the distribution of doses in the exposed 

5 population (EPA, 2009a). 

6

7

 

25 Note that ―individual‖ risk models may have as their outputs probability distributions of risk that can be 

interpreted to reflect (1) uncertainty in infectivity of the agent tested, and/or (2) variability in individual 

susceptibility among the experimental subjects.
Page 74 Microbial Risk Assessment Guideline

Table 4.1 Overview of Dose-Response Relationships for Waterborne Pathogensa1

2 (Source: EPA 2009a; Adapted from McBride et al., 2002)

Microorganism Model Parametersb Reference(s)

Adenovirus 4 Exponential r = 0.4172c Crabtree et al., 1997

Haas et al., 1999 

Campylobacter 

jejunih,i

Beta-Poisson α = 0.145 β = 7.59 

Haas et al., 1999

Medema et al., 1996

Teunis et al., 1996

Infection: 

Hypergeometric betaPoisson

Illness: Conditional on 

infectiong

α = 0.024 β = 0.011

-9

r = 2.44x108 

Teunis et al., 2005 

Coxsackievirus Exponential r = 0.0145 Haas et al., 1999

Cryptosporidium 

Exponential

r = 0.0042 Haas et al., 1996, 1999

r = 0.077d Okhuysen et al., 1999

r = in the range 0.04 to 0.16 EPA, 2006a

Generalized betaPoisson for Illness

α = 0.060 β = 0.095 Englehardt and Swartout, 

2006

Echovirus 12

Exponential r = 0.0128 Haas et al., 1999

Beta-Poisson

α = 0.401 β = 227.2 Teunis et al., 1996

α = 0.374 β = 186.69 Regli et al., 1991

Rose and Sobsey, 1993

α = 1.3 β = 75 Rose and Gerba, 1991

Entamoeba coli Beta-Poissonα = 0.1008 β = 0.3522 Haas et al., 1999

Escherichia coli 

(pathogenic 

strains)

Beta-Poisson α = 0.1778 β = 1.78x10

6 Haas et al., 1999

E. coli O157:H7

Beta-Poissoneα = 0.248 β = 48.80 Teunis et al., 2008a

Hypergeometric betaPoisson

α = 0.084 β = 1.44 

(children)

α = 0.050 β = 1.001

(adults)

Teunis et al., 2004

Giardia lamblia Exponential r = 0.0199 

Haas et al., 1999

Regli et al., 1991

Rose and Gerba, 1991

Rose et al., 1991

Teunis et al., 1996

Hepatitis A virus Exponential r = 0.5486f Haas et al., 1999

Legionella Exponential r = 0.06 Armstrong and Haas, 

2008

Norovirus

Infection (with 

aggregation): 

Hypergeometric function 

1F1

Illness: Conditional on 

Infectiong

α = 0.040 β = 0.055

= 2.55×10-3 r= 0.086 

Teunis et al., 2008b

Poliovirus I Beta-Poisson α = 0.1097 β = 1524 Regli et al., 1991

Rose and Sobsey, 1993 

α = 15 β = 1000 Rose and Gerba, 1991
Microbial Risk Assessment Guideline Page 75

Draft – Do not cite, circulate, or quote

Microorganism Model Parametersb Reference(s)

Exponential r = 0.009102

Haas et al., 1999

Regli et al., 1991

Rose and Sobsey, 1993

Poliovirus III Beta-Poisson

α = 0.409 β = 0.788 Rose and Sobsey, 1993

α = 0.409 β = 0.788 Regli et al., 1991

α = 0.5 β = 1.14 Rose and Gerba, 1991

Rotavirus

Beta-Poisson

α = 0.26 β = 0.42 Gerba et al., 1996

α = 0.2531 β = 0.4265 

Haas et al., 1999

Regli et al., 1991

Rose and Sobsey, 1993

α = 0.232 β = 0.247 Rose and Gerba, 1991

Hypergeometric betaPoisson α = 0.167 β = 0.191Teunis and Havelaar, 

2000

Salmonella

Beta-Poisson α = 0.33 β = 139.9 Rose and Gerba, 1991

Gompertz log

ln(a) in the range 29 to 50

b = 2.148

Coleman and Marks, 2000

Coleman et al., 2004

Soller et al., 2007

Generalized linear mixed 

models and

fractional polynomials of 

dose

β0 = 0.323 β1 = 5.616 

β2 = -8.462 β3 = -7.782 

d

2

= 0.780

Bollaerts et al., 2008

Salmonella (nontyphoid) Beta-Poisson α = 0.3126 β = 2884 Haas et al., 1999

Salmonella Typhi

Fractional polynomials

β1 = -18.1425 

β2 = 22.5300×10-5 Namata et al., 2008

Beta-Poisson

α = 0.1086 β = 6,097 Haas et al., 1999

α = 0.21 β = 5,531 Rose and Gerba, 1991

Shigella Beta-Poisson α = 0.21 β = 42.86 Haas et al., 1999

Vibrio cholerae Beta-Poisson α = 0.25 β = 16.2 Haas et al., 1999

a 1 These calculations are based on available data that have used particular pathogen strains processed in particular ways. 

2 Where more than one strain of an organism has been studied in clinical trials, a wide range of infectivities can be 

3 discovered. Therefore it must be recognized that these calculations can carry a substantial degree of uncertainty.

b

For the exponential distribution N50= 0.693/r; for the beta-Poisson distribution N50= * (21/ 4 -1). Values are unitless.

c 5 Developed for inhalation exposure to adenovirus 4 aerosols.

d Estimated based on ID50 6 reported for the Texas A&M University (TAMU) isolate.

e 7 Represents a meta-analysis of seven outbreaks and adjusting for heterogeneity. Alpha/beta pairs derived via MCMC 

8 analyses are available from Dr. Teunis. Use of those pairs is preferred to the use of the values shown in this table

f 9 Corresponding dose units are grams of feces.

g Dose-response relation for the conditional probability of illness in infected subjects = 1 – (1+ CV)-r 10 , where and r 

11 are shown in the Table; CV is the dose (concentration × volume).

h 12 An alternate dose-response model is proposed by Brynestad et al. (2008). 

13 i Many of these models have been critiqued in the literature. For example, Coleman et al. (2004) suggest the dose14 response models for Campylobacter identified in this table do not account for strain variability sufficiently and suggest 

15 the need for development of more detailed mechanistic models.

16

17 An overview of Exponential, beta-Poisson, deterministic, and Bayesian 

18 hierarchical models is provided below and a summary of peer reviewed dose-response 

19 models is presented in Table 4.1.

20

21

22
Page 76 Microbial Risk Assessment Guideline

1 The Exponential Model

2

3 The single-hit family of dose-response models was described previously (Section 

4 4.2.1) as the most relevant for microbial dose-response assessment. The simplest of the 

5 single-hit family of models is the exponential model (Equation 1).

6

p = 1− e

−rd

7 (Eq. 4.1)

8

9 Where:

10 p is the cumulative probability of infection in the exposed population

11 d is the average pathogen dose in infectious units (organisms)

12 r is the probability of infection given ingestion of one organism

13

14 For the exponential model, r is a constant for the interaction of any given 

15 pathogen and host species. That is, each infectious particle within each host is assumed 

16 to have the same probability of survival. There is no inter-individual variability in the 

17 host-pathogen interaction: the exponential model assumes that the same probability of 

18 infection applies to every individual in the population. Despite this unrealistic-sounding 

19 assumption, the exponential model provides a good fit for a number of the human 

20 pathogen-challenge data sets (Teunis et al., 1996; Haas et al., 1999). The primary 

21 advantage of the exponential model is its computational simplicity. The primary 

22 disadvantage is that is does not account for inter-individual variability in the population.

23

24 The beta-Poisson Model

25

26 The primary limitation of the exponential model (no variability in r) is 

27 partially26 overcome using the beta-Poisson model, by assigning a distribution to r

28 to represent the variability in the pathogen-host interaction. The most common 

29 distribution applied to r is the beta, giving rise to the beta-Poisson model (more 

30 strictly, the beta-exponential). As r is a probability itself, the assigned distribution 

31 must have a range of 0 to 1. The beta distribution is the most flexible of such 

32 distributions, including shapes similar to Gaussian (normal), triangular, exponential, 

33 power law, uniform, and bimodal. The unit infectivity r-value is the mean of the 

34 distribution, which is readily calculated from the parameters. The two-parameter beta35 Poisson does not have a fixed slope and is more flexible than the exponential. The beta36 Poisson still follows the rules (does not exceed the probability of exposure at low dose),

37 but is more biologically plausible than the exponential and will fit better to data with 

38 higher variances. The beta-Poisson is a function of the confluent hypergeometric 

39 function, which is a sum of an infinite number of terms and has no simple closed40 form mathematical representation (Abromowitz and Stegun, 1964). The beta41 Poisson cumulative probability is given by Equation 2.

42

43 1 – M( , , -d) (Eq. 4.2)

44

 

26 The distribution on r does not distinguish between pathogen and host variability.
Microbial Risk Assessment Guideline Page 77

Draft – Do not cite, circulate, or quote

1 where M is the confluent hypergeometric function (the 1F1 form), and are the beta 

2 distribution parameters, and d is the mean dose in pathogen infectious units. The solution 

3 is estimated numerically.

4

5 The Pareto II distribution, commonly called the beta-Poisson in the literature, was 

6 shown by Furumoto and Mickey (1967) to approximate the exact theoretical beta-Poisson 

7 model, and has found wide usage. The Pareto II distribution function for dose response is 

8 given by Equation 3.

9

10 1 – (1 + d/ ) (Eq. 4.3)

11

12 where d is the dose, and and are parameters corresponding to the beta distribution 

13 parameters for specific ranges, the approximate form being valid for parameter values 

14 >> 1 and << (Teunis and Havelaar, 2000). Outside of this range, the Pareto II can 

15 substantially overestimate the risk, sometimes predicting a probability of infection greater 

16 than the probability of exposure at low doses (Teunis and Havelaar, 2000). The term 

17 ―beta-Poisson‖ will be used, henceforth, with reference to the exact form, while the 

18 analytic approximation will be referred to as the Pareto II. Both forms have been used 

19 for the gastro-enteric infection endpoint (Haas et al., 1996; Teunis and Havelaar, 1999; 

20 Englehardt and Swartout, 2004). Thus, the beta-Poisson is more flexible than the 

21 exponential model while retaining simplicity.

22

23 Accounting for Immunity

24

25 Whatever model you use, the risk of infection applies only to the susceptible 

26 population. In fitting the model to a particular human pathogen-challenge study data set, 

27 it may be possible using available information to take into account the fraction of 

28 immune individuals, sometimes even if the participants were prescreened for presence of 

29 antibodies to the organism. The general fitting algorithm for assessing the fraction of 

30 immune individuals is given in Equation 4.

31

32 (1 – fr) * F(d, θ) (Eq. 4.4)

33

where fr is the fraction of resistant (immune) individuals27 34 , F is the dose-response 

35 function (e.g., exponential, beta-Poisson), d is the dose, and θ is the parameter vector 

36 associated with F. If the subjects were screened for prior exposure, you should offer an 

37 explanation of the (unexpected) finding of a resistant fraction when using this model.

38 Explanations could be based on theoretical considerations or experimental conditions

39 specific to the case.

40

41 Deterministic Models

42

43 The foregoing hit-theory models are stochastic in nature, primarily in that each 

44 host may or may not become infected at any given pathogen dose. Models that are 

 

27 Alternatively, 1 – fr can be replaced by fs, the fraction of susceptibles. 
Page 78 Microbial Risk Assessment Guideline

1 deterministic in nature also have been proposed for microbial dose-response evaluation 

2 (Moon et al., 2005). These models are deterministic because they assume that each host 

3 has a unique tolerance, or threshold dose, above which infection is 100 percent certain, 

4 similar to chemical dose-response assessment. An advantage of these models is that in 

5 general, these models are more flexible than the hit-theory models and will tend to fit 

6 better to many data sets. A disadvantage is that they can result in over prediction of risk 

7 at low doses (dilutions of single organisms), because they do not recognize the discrete 

8 nature of pathogen distribution. Over prediction is likely to happen for dose-response 

9 data sets characterized by high variability, high response at lower doses, or slowly 

10 increasing responses across large dose ranges. Over prediction is particularly prevalent 

11 in modeling uncertainty in bootstrap simulations, for example. In addition, the biological 

12 plausibility of individual (deterministic) host thresholds has not been established for 

13 pathogens, as it has for chemicals. Therefore, at least for low-dose extrapolation (e.g., 

14 determination of unit infectivity), deterministic models are not recommended. However, 

15 for high-dose risk estimation, deterministic models can be useful because of their ability 

16 to fit the response data better than the one-hit models.

17

18 Bayesian Hierarchical Models 

19

20 Bayesian methods to estimate dose-response model parameters and evaluate their 

21 uncertainty are also being used (Englehardt, 2004; Englehardt and Swartout, 2004; 

22 Messner et al., 2001). These methods are particularly useful in cases where data are 

23 available from multiple studies. One-stage or hierarchical models can be fit to the data 

24 using methods that include Markov Chain Monte Carlo Simulation (MCMC) (Gilks et al.,

25 1996; FAO/WHO, 2003; EPA 1997b). A Markov chain is a stochastic model having 

26 discrete states in which the probability of being in any state at any time depends only on 

27 the state at the previous time and on the probability transition matrix. MCMC

28 simulations can be used to generate samples from the joint posterior distribution 

29 (Messner et al., 2001). An advantage to these models, in contrast with traditional 

30 statistical techniques, is that they are able to exploit subjective and related information in 

31 addition to numeric data. 

32

33 A predictive Bayesian dose-response function can be developed as follows. First, 

34 the parametric form of the dose-response function is established by theoretical derivation 

35 and, if possible, empirical confirmation. Then all available knowledge, other than the 

36 theoretical form of the conditional distribution and empirical data already used for that 

37 purpose, is brought to bear upon estimation of the parameters of the distribution. To do 

38 this, the parameters are recognized as uncertain but subject to professional judgment, and 

39 thus, a prior probability distribution is assigned to each parameter. Prior distributions are 

40 then refined with dose-response data, to obtain a posterior distribution. Next, the 

41 predictive Bayesian dose-response function can be found by multiplying the posterior by 

42 the conditional dose-response function, and integrating over the parameter space 

43 (Englehardt, 2004). Bootstrap methods (i.e., repetitive Monte Carlo sampling directly 

44 from the data or from data summary distributions) whether used in a Bayesian or 

45 frequentist framework, may also be used to evaluate parameter uncertainty in dose46 response models (Teunis et al., 1996; Haas et al., 1999; Englehardt and Swartout, 2006). 
Microbial Risk Assessment Guideline Page 79

Draft – Do not cite, circulate, or quote

1 These models can be more complicated than other models described here, are generally 

2 less familiar to scientists and managers, and can be difficult to explain.

3

4 Bayesian hierarchical models have also been used to develop dose-response 

5 relationships for pathogens based on outbreak data rather than feeding study data. For 

6 example, Teunis et al. (2005) analyzed Campylobacter jejuni dose response using 

7 Bayesian methods. Data from both a human volunteer study and an outbreak caused by 

8 drinking raw milk were combined in this analysis. The model incorporated both the 

9 probability of infection and the conditional probability of illness given infection. First, 

10 for the outbreak a certain probability of illness (p0) was assumed for those who were 

11 unexposed to the raw milk but might have become ill due to an alternative route of 

12 transmission. Second, a beta-Poisson model was used to model the probability of 

13 infection given a mean dose (D). Third, a model for the conditional probability of illness, 

14 given that the individual is infected and had mean dose D, was developed. Non15 informative prior distributions for the parameters were defined. The posterior mode 

16 parameter values were calculated by directly maximizing the posterior probability. These 

17 values were used to compute the posterior mode dose-response functions for the 

18 probability of infection and the probability of illness given infection. Uncertainty 

19 intervals for these dose-response functions were computed by using MCMC to simulate 

20 vectors of parameter values. Teunis et al. (2008a) also analyzed data from eight 

21 outbreaks of E. coli O157:H7 using a hierarchical Bayes model and used Bayesian 

22 methods to analyze dose-response functions for the Norwalk virus, based on a volunteer 

23 study (2008b). 

24

25 4.2.2 What is the Output of a Dose-Response Assessment? 

26

27 The output of a dose-response assessment is a value or a set of values for the dose 

28 response parameters. For example, for the exponential dose-response model (described 

29 in section 4.2.1) a single value (―r‖) would be required and for dose-response models 

30 having more than one parameter, a set of parameter values will be required. For many of 

31 the most common dose response functions, the relationship between exposure and risk is 

32 linear at low doses (Haas et al., 1999). For exposures to many organisms at once (such as 

33 in food), the risk of infection needs to be calculated from the mathematical dose-response 

34 function itself. 

35

36 Computing the risk of infection may be necessary to determine the population at 

37 risk for illness, but infection, in itself, is not necessarily adverse. Therefore, information 

38 on the rate of illness, given infection, is needed to perform a meaningful risk assessment. 

39 Although orally ingested pathogens can cause a number of symptoms, some severe 

40 enough to be life-threatening, the only endpoints examined in human studies (for ethical 

41 reasons) are related to gastrointestinal illness, primarily diarrhea. The risk of illness is 

42 estimated in a similar fashion as for infection, except that the dose-response model is

43 constrained such that an illness response is strictly conditional on infection. The 

44 constraint is trivial for the one-parameter exponential model, as the slope is fixed. For 

45 models in which the slope can change (more than one parameter), a higher risk of illness 

46 than infection can be predicted for some data sets if the constraints are not strict. Illness 
Page 80 Microbial Risk Assessment Guideline

1 without infection is not biologically plausible so constraints are necessary to avoid this 

2 implausibility. Applying strict constraints in such models is not a trivial exercise, and is 

3 a topic of ongoing research. The output of an illness dose-response assessment generally 

4 is a morbidity ratio, which is the fraction of those who are infected that become ill. A 

5 common practice is to assume that the risk of illness is constant once an individual 

6 becomes infected, no matter what the dose. In this case, the morbidity ratio is simply the 

7 number of ill individuals divided by the number of infected individuals. Dose-dependent 

8 morbidity ratios, where the conditional (on infection) risk of illness increases with 

9 increasing dose, are more difficult to model, requiring the strictly constrained model 

10 previously discussed (Refer to Teunis et al., 2005). For drinking water risk assessments, 

11 where the exposures are frequently very low, the occasionally large difference between 

12 constant and dose-dependent morbidity ratios can be highly significant. 

13

14 You should be transparent in your discussion of the dose-response output. For 

15 example the dose-response is based on a defined health endpoint, a defined human 

16 population (e.g., the population used in a clinical trial or an epidemiological study), and is 

17 influenced by model selection. Be clear about all the constraining features of the data 

18 and what assumptions were made when those data were used to extrapolate to broader 

19 human populations or health endpoints. For example, the risk managers who will use the 

20 risk assessment results to inform their decision making need to understand the 

21 implications of dose-response data based on healthy adults, when their goal is protection 

22 of the general population that includes all life stages and sensitive populations.

23

24 4.2.3 How do I Fit Models to Existing Dose-Response Data?

25

26 A summary of dose-response relationships published in the peer-reviewed 

27 literature for waterborne exposures is provided in Table 4.1. In many cases, you can

28 simply select the most appropriate dose response relationship from one that has already 

29 been peer reviewed. If an appropriate dose-response relationship for the specific 

30 pathogen/matrix combination is not available, you can derive a relationship in several 

31 ways. If a valid dose-response data set is available, you should be able to fit the data to a 

32 mathematical dose-response model. The dose-response model provides a prediction of 

33 the incidence of the effect in the population, given a specific exposure level or dose. The 

34 dose-dependent probability distribution is defined by a mathematical equation with one 

35 or more parameters. The best values for those parameters are determined by assessing 

36 the likelihood of observing the data, given specific parameter values. The parameter 

37 values that result in the greatest likelihood of the data are chosen as the most 

38 representative ones. The determination of the best parameter values can be done directly 

39 on the data (as in a ―frequentist‖ approach), or by updating a prior judgment as to what 

40 the parameters might be (a ―Bayesian‖ approach). In either case, the resulting fitted 

41 dose-response model is used for predicting human response, either infection or illness, to 

42 pathogen exposure for a specific risk-assessment scenario. 

43

44 The typical dose-response assessment is performed by a direct fit of the functional 

45 form to the data using a ―frequentist‖ approach. That is, the fitted parameters represent 

46 the frequency with which some event has happened previously. The fitted parameters, 
Microbial Risk Assessment Guideline Page 81

Draft – Do not cite, circulate, or quote

1 however, are generally used to predict the future occurrence of that event. Proponents of 

2 Bayesian methods argue that a Bayesian approach is a virtual requirement when trying to 

3 assess probabilities of future outcomes, rather than the frequencies of past events 

4 (Bernardo and Smith, 1994; Berry, 1996; Carlin and Louis, 2001). Bayesian methods 

5 provide a rigorous framework for common-sense interpretation of statistical conclusions. 

6 Frequentist approaches can only place confidence limits on a result that depends on 

7 specific conditions, leading to inferences that might be made in repeated practice. 

8 Bayesian proponents point out that most people erroneously interpret frequentist results 

9 in the Bayesian sense (Gelman et al., 2004). Bayesian methods generally provide much 

10 greater flexibility than do frequentist methods, allowing you to cope with very complex, 

11 data-limited problems. 

12

13 Bayesian methods, however, are not without problems of their own. First, 

14 computation of the more complex integrals generally involves complex numerical 

15 techniques, with which many practitioners will not be familiar. Most analysts will have 

16 to rely on programs, which, in themselves, require some degree of mastery of functional 

17 coding techniques. Second, Bayes theorem requires that a prior relationship between 

18 dose and response be specified, which can be problematic for pathogens. The prior, 

19 which is usually subjective, can vary among investigators. Non-informative priors, 

20 generally uniform distributions over a wide range of parameter values, address the 

21 subjectivity issue but can make strong statements about prior belief of infectivity. An 

22 example of the latter and, perhaps, the only truly non-informative prior, would be the 

23 simplest prior on the r parameter for the exponential distribution (see Section 4.2.1) of a 

24 uniform distribution between 0 and 1 (the full range of r). Although this prior says that 

25 we really don‘t know anything a priori about infectivity, it establishes a prior expected 

26 value of 0.5 for r, which is much greater than any known actual pathogen infectivity. As 

27 the Bayesian posterior is a compromise between the prior and the data, a lot of data 

28 would be required to move the answer towards a less extreme value. 

29

30 Examples of the use of Bayesian techniques in the literature include a number of 

31 analyses of C. parvum (Teunis et al., 1999; Messner et al., 2001; Englehardt and

32 Swartout, 2004, 2006; EPA, 2006a). These analyses are largely hierarchical, assessing 

33 the aggregate infectivity of several isolates of C. parvum treating each one as a distinct 

34 strain. The outputs of most of these analyses consist of distributions of uncertainty for 

35 ―hyperparameters‖ of C. parvum infectivity across strains. Such an analysis was used in 

36 support of the Long Term 2 Enhanced Surface Water Treatment Rule (LT2ESWTR) 

37 promulgated by EPA in 2006 (EPA, 2006a). One of the analyses did not assume that the 

38 C. parvum isolates were different, evaluating the aggregate illness-response data across 

39 all studies (Englehardt and Swartout, 2006). This analysis and one other (Englehardt and

40 Swartout, 2004) used an unconditional Bayesian form, in which the posterior represented 

41 the dose-response function integrated over the entire range of parameter values. This 

42 unconditional or ―predictive‖ form is independent of any pre-selected confidence level 

43 (e.g., 95 percent) and can be considered to be the expected value under uncertainty. All 

44 of the cited C. parvum assessments can be considered to be meta-analyses of sorts, as 

45 they incorporate data from more than one study into a single aggregate output. Meta46 analysis generally allows for a weight-of-evidence approach, where a more objective 
Page 82 Microbial Risk Assessment Guideline

1 review of data may occur. Although none of these analyses did so, in a meta-analysis, 

2 individual data sets can be assigned specific weights (other than unity) prior to 

3 quantitative analysis. In this case, a Bayesian framework offers a rigorous approach in 

4 which to evaluate such weightings. Bayesian model averaging is an technique that 

5 formalizes this process (Hoeting et al., 1999).

6

7 4.2.4 How Can I Evaluate Uncertainty in Dose-Response?

8

9 It is possible to characterize the precision with which the dose response curve has 

10 been determined. The statistical confidence limits primarily reflect variability, with some 

11 (usually unquantifiable) contribution from uncertainty. Therefore the statistical 

12 confidence limits do not represent the full expression of uncertainty. For single hit 

13 models some of the uncertainty can be shown by determining the confidence limits to the 

14 parameters of the dose response curve and also the upper and lower envelope around the 

15 dose response curve (Haas et al., 1999). A likelihood-based approach can be used for this 

16 purpose. This yields an m-dimensional region, where m is the number of parameters in 

17 the dose response model under evaluation (one for the exponential, two for the beta 

18 Poisson, etc.). The computational details of this approach are beyond the scope of this 

19 document, but interested assessors are referred to Chapter 7 of Haas et al. (1999).

20

21 In hierarchical Bayesian approaches, the output of the uncertainty analysis can be 

22 in the form of distributions on the model parameters or ―hyperparameters,‖ such as the 

23 mean of the fitted beta distribution (the average unit infectivity) for the beta-Poisson 

24 dose-response model. A distribution of plausible bounds on the response at any dose can 

25 then be generated to obtain, for example, 95% confidence limits on the response at a 

26 given dose. 

27

28 In the predictive Bayesian models, the parameter uncertainty is integrated with the 

29 response variability to obtain a single dose-response curve without confidence bounds, 

30 but whose shape depends on the amount of information available; the shape becomes 

31 narrower (less uncertain) as the amount of information increases (Englehardt, 2004), 

32 generally resulting in a prediction of lower mean risk than when information is sparse. 

33 Another technique is the parametric bootstrap analysis, in which the result of the initial 

34 dose-response model fit to the data is assumed to be the true response relationship. The 

35 responses at each dose are then regenerated repeatedly based on the respective fitted 

36 response probabilities assuming an underlying response distribution (generally binomial). 

37 A distribution of ―resampled‖ responses is obtained for each dose, from which bootstrap 

38 confidence bounds on the entire dose-response curve can be calculated. 

39

40 4.2.5 What is Variability in Dose-Response?

41

42 It is likely that all parameters in dose-response have some variability. Dose43 response involves the interaction of host and pathogen, so any factors associated with 

44 either the host or the pathogen have variability and the interaction itself is also variable. 

45 The variability in factors presented in Section 4.1.3 may be quantitative or qualitative in 

46 nature. Depending on the needs of the particular risk assessment, you may need to use 
Microbial Risk Assessment Guideline Page 83

Draft – Do not cite, circulate, or quote

1 statistical techniques to characterize variability in:

2

3 a) Dose-response relationships between isolates or strains

4 b) Host immune responses (both immunity and susceptibility)

5 c) Duration of host immune responses

6 d) Host characteristics that influence the dose-response relationship

7 e) Health effects

8 4.2.6 How Can I Account for Life Stages and Subpopulations in Dose9 Response Models?

10

11 Unlike a subpopulation, a life stage is something the whole population passes 

12 through, such as childhood. At any given moment only part of the population is in 

13 childhood, but referring to children as a subpopulation underestimates the importance of 

14 childhood life stages in the larger picture. EPA breaks childhood down into ten age 

15 groups based on behavioral and physiological milestones (EPA, 2005; Section 3.7). 

16 Some life stages, such as children and the elderly, are more sensitive to pathogens 

17 because of host susceptibility and/or behavior patterns when compared to other life 

18 stages, such as adults. In chemical risk assessment, life stages and sensitive populations 

19 are frequently taken into account by the application of uncertainty factors. The use of 

20 uncertainty factors for pathogens, however, is somewhat problematic because of the 

21 discrete nature of exposure. Unlike for chemicals, pathogen dose is usually expressed in 

22 non-reducible organism-level units as an average, such that fractional doses represent 

23 probabilities of exposure to single organisms. Thus, when considering low-dose 

exposures, a dose of 10-424 for a chemical may be in units of milligrams or micrograms, 

representing perhaps 1018 25 molecules, while the same dose for a pathogen would represent 

26 a 1 in 10,000 chance of ingesting a single organism for a given exposure. Depending on 

27 relative infectivity of the pathogen, the indiscriminate application of a fixed uncertainty 

28 factor could result in an impossible risk (risk of infection greater than probability of 

29 exposure) or an arbitrarily high risk inconsistent with the overall exposed population. 

30 Accounting for sensitive life stages and populations for a low-dose exposure scenario 

31 (e.g., drinking water) would require an estimate of the relative infectivity for that 

32 subpopulation compared to the general population, as well as the fraction of that 

33 subpopulation with respect to the entire population. With this information, a new 

34 subpopulation infectivity parameter could be calculated as a weighted average. For 

35 example, simply stating that you believe that a particular life stage is 10 times more 

36 susceptible than the healthy adult population represented by the experimental data is not 

37 adequate, in itself. In this case, if the r-value for the healthy adult population is greater 

38 than 0.1, an impossible risk (> 1) is projected for the life stage. Each case is likely to 

39 require a unique solution, with professional judgment playing a large role. For some 

40 pathogens, the probability of infection or disease for sensitive life stages and populations 

41 may not differ from the normal population, but the severity of the disease outcome may 

42 differ.
Page 84 Microbial Risk Assessment Guideline

1

2 4.2.7 Can I Use Uncertainty, Modifying, or Adjustment Factors in a 

3 Microbial Dose-Response Assessment?

4

5 There are no standard guidelines for the application of uncertainty, modifying, or 

6 adjustment factors in MRA as are in chemical risk assessment. In chemical risk 

7 assessments, uncertainty factors are usually applied as factors of 3 or 10, and are applied 

8 to empirically derived effect levels (e.g., no observed adverse effect level) to 

9 accommodate for a lack of knowledge associated with interspecies extrapolation, high- to 

10 low-dose extrapolation (i.e., effect to no-effect), population variation (i.e., protection of 

11 sensitive populations), and extrapolation across exposure durations (e.g., subchronic to 

12 chronic). The areas of uncertainty potentially most relevant to MRA are the interspecies, 

13 sensitive life stages, and sensitive population extrapolations. As an example, for 

14 interspecies extrapolation, there are sound allometric-scaling principles relating the 

15 uptake and whole-body distribution of chemicals and strongly conserved mechanisms of 

16 toxicity across mammalian species. There are no corresponding agreed upon principles 

17 for cross species pathogenicity relationships. Because many pathogens are highly 

18 species-specific or produce different effects in different species, and immune response 

19 mechanisms can be highly variable across species, use of uncertainty, modifying or 

20 adjustment factors to justify extrapolation is highly suspect. 

21 

22 For chemical risk assessment, all uncertainty factors are applied as a divisor of the 

23 dose to obtain a quasi-threshold exposure level. For MRA, the assessor would have to 

24 pay attention to the absolute value of the computed risks so that implausible or 

25 impossible risks would not arise (e.g., product of general population risk and uncertainty 

26 factor greater than 1). In essence, a variable-magnitude uncertainty factor scheme would 

27 be a likely result, with smaller uncertainty factors for larger average population risks. 

28 Furthermore, to obtain an overall population risk (adjusted for sensitive life stages and 

29 subpopulations), the assessor would have to know the proportion of the population that is 

30 sensitive (for proper weighting of each subpopulation-specific risk). An example of this 

31 process can be found in a human population infection-response assessment for C. 

32 parvum, in which human pathogen-challenge study data were combined probabilistically 

33 with assumptions about the size of sensitive and resistant populations to obtain an 

34 estimate of overall population response (Englehardt and Swartout, 2004). 

35

36 Because of the foregoing considerations, it is not usual practice in MRA to use 

37 uncertainty or modifying factors in a manner similar to their use in chemical risk 

38 assessment. An important distinction between uncertainty factors and adjustment factors 

39 is that uncertainty factors account for unknown distributions of sensitivity and adjustment 

40 factors account for known differences in response. Case-specific adjustment factors can 

41 be employed if strong defensible evidence supports their use. As an example, 

42 epidemiological data was used as a justification for the application of adjustment factors 

43 in the FDA/USDA/CDC risk assessment for L. monocytogenes in ready-to-eat foods. 

44 Scaling factors were used to adjust mouse-derived dose-response curves so they were 

45 applicable to humans (FDA/USDA/CDC, 2003). The size of the scaling factor was 
Microbial Risk Assessment Guideline Page 85

Draft – Do not cite, circulate, or quote

determined by surveillance data reported to FoodNet28 1 for the receptor populations 

2 modeled in the risk assessment. Similarly, in the FDA risk assessment on Vibrio 

3 parahaemolyticus, the dose-response curve was adjusted to reflect CDC‘s illness 

4 estimate. The adjustment factor represents the effect of the apparent differences between 

5 the dose-response observed in human volunteers under controlled conditions versus that 

6 in the general population when exposure is associated with the oyster food matrix.

7

8 4.2.8 What Modeling Methods are on the Horizon?

9

10 Physiologically-Based Dose-Response Models

11

12 New physiologically based models are being developed to begin to capture the 

13 biological complexity associated with dose-response (Blaser and Kirschner, 1999, 2007). 

14 Physiologic models begin with the development of conceptual models that break the 

15 process from exposure to establishment of infection to the expression of illness into 

16 compartments. These compartments serve as ―steps‖ in the process, and the parameters 

17 inside each step are captured in the model. Published research and/or new research 

18 provide or will provide the values for each parameter. This is one of the strengths of the 

19 risk analysis concept, the clear identification of data needs, which can then been 

20 translated into research priorities.

21

 

28 http://www.cdc.gov/FoodNet/
Page 86 Microbial Risk Assessment Guideline

1 5. EXPOSURE ASSESSMENT

2

3 The goal of exposure assessment in MRA is to determine the route, frequency, 

4 duration, and magnitude (amount) of exposure to a microbial hazard in a population. 

5

6 Microbial agents may come from more than one source, may be transmitted via 

7 multiple routes of exposure, and may be spread via secondary transmission. Moreover, 

8 these routes of exposure may be inter-related. Exposure routes relevant to a given 

9 microbial hazard are situation-dependent and influenced by the inherent properties of the 

10 microorganism and its potential host(s). An exposure source can originate from either 

11 natural or anthropogenic events, activities, or locations that generate or release microbial

12 hazards. 

13

14 A number of factors define microbial exposure including the sources and 

15 pathways of exposure, the growth and/or decline in numbers of microorganisms and 

16 variable intake amounts among individuals. Often, some of the necessary data for a 

17 microbial exposure assessment are either lacking (i.e., need to be extrapolated from data 

18 developed for another purpose or limited data that are not representative) or altogether 

19 non-existent. Given that complete data and information are rarely available for microbial

20 exposure assessment, you may need to make simplifying assumptions. Such assumptions 

21 result in uncertainty about exposure estimates. To support better risk management 

22 decisions, you need to objectively characterize uncertainty about exposure assessment 

23 results. 

24

25 This chapter provides general principles and practical guidance for conducting

26 exposure assessments for microbial hazards. Information is organized into five sections: 

27 1) general concepts in exposure assessment (Section 5.1); 2) developing an exposure 

28 assessment (Section 5.2); 3) analyzing results from a model (Section 5.3); 4) 

29 communication, review and validation of model results (Section 5.4); and, 5) future 

30 developments in exposure assessment (Section 5.5). The chapter is also intended to 

31 provide information useful to risk assessors, risk managers, decision-makers, risk 

32 communicators, stakeholders and the general public, and researchers. Other resources 

33 that provide overviews of exposure assessment are those by the WHO/FAO (2008), 

34 European Commission Scientific Steering Committee (ECSSC, 2003), Hass (1999), Cox 

35 (2006), and Vose (2008).

36

37 5.1 What are General Concepts in Exposure Assessment?

38

39 This section includes factors to take into account when conducting an exposure 

40 assessment. It also discusses variability, uncertainty, deterministic and stochastic risk 

41 assessment, and Monte Carlo analysis.

42

43 5.1.1 What is an Exposure Assessment?

44

45 An exposure assessment is the process of estimating or measuring the magnitude, 

46 frequency, and duration of exposure to a microbial hazard(s), along with the number and 
Microbial Risk Assessment Guideline Page 87

Draft – Do not cite, circulate, or quote

1 characteristics of the person or population exposed. You can provide either a qualitative 

2 or quantitative evaluation of exposure, however a quantitative evaluation is always 

3 preferable if the data exist and a quantitative risk assessment is needed. Ideally, an 

4 exposure assessment describes the sources, pathways, routes, and the uncertainties about 

5 exposures.

6

7 5.1.2 What is Exposure?

8

9 Exposure comprises the sources, mode, route, and extent of contact the host has 

10 with the microbial hazard(s) of concern. How often a person is exposed is referred to as 

11 frequency of exposure. How long a person is exposed to a microbial hazard is referred to 

12 as the duration of exposure. 

13

14 MRA is typically only concerned with single event exposures. But reoccurring 

15 exposures can be included, particularly if secondary transmission is being modeled. The 

16 number of microorganisms that correspond to a single exposure is referred to as the dose. 

17 The exposure dose may also be the total number of organisms that constitute a set of 

18 exposures. Simultaneous exposure to multiple hazards is also a concern in MRA.

19

20 5.1.3 What are Sources, Pathways, and Routes of Exposure?

21

22 There are various terms used to discuss the origin, movement or spread, and final 

23 intake of microorganisms by individuals or populations. Generally, the overall 

24 terminology refers to routes of transmission for microorganisms.

25

26 The entity (or entities) that supply microorganisms to a particular exposure route 

27 is the source. The source of microorganisms could be infected food animals, industrial 

28 processes, the environment (water, air, soil), or infected persons. The route of exposure 

29 (or route of intake) is where the microorganism comes into contact with the host. The

30 three common routes of exposure are oral, inhalation, and dermal.

31

32 The physical movement of microorganisms, over time, from their source to the 

33 occurrence of an exposure is referred to as the exposure pathway or route of transmission. 

34 Exposure pathways may be complex; exposures may occur via aerosolization, water, 

35 food, soil, fecal-oral and/or inanimate sources. The mode of transmission can be wind, 

36 flowing water, equipment movement, or vector organism. Not all modes of transmission 

37 are relevant for all exposure pathways. For example, neither inhalation via the nose nor 

38 dermal exposure are highly relevant for foodborne exposures. The number of

39 microorganisms in a particular medium can increase or decrease across time as a function 

40 of changing environmental conditions, throughout the exposure pathway. 

41

42 Elements of Source Evaluation

43

44 While sources of microorganisms may be living or inanimate, the elements of 

45 source evaluation are basically the same, again with the caveat that not all modes of 
Page 88 Microbial Risk Assessment Guideline

1 transmission are relevant for all exposure pathways. In a farm to fork model you can 

2 consider:

3

4 a) How many viable pathogens (or indicators) are present at the source (e.g. infected 

5 chicken, contaminated carcass) at time zero?

6 b) How many are released from the source?

7 c) Over what time period are they released?

8 1) Continuously

9 2) Batchwise

10 d) At what rate are they released?

11 1) Counts/unit time (e.g., cfu, pfu, genomes per minutes, seconds, hours, days)

12 e) What is the form of the release?

13 1) Fomites

14 2) Spray equipment

15 3) Offgases from a fermentor

16 4) Waste water

17 5) Animal slaughter

18 f) To what medium are they released

19 1) Food

20 2) Surface water

21 3) Soil

22 4) Air

23 5) Other surfaces

24

25 The evaluation of movement from a source into an exposure pathway is 

26 sometimes called a release assessment. The purpose of the environmental release 

27 assessment is to identify the sources of potential release, the media of release (air, water, 

28 or land) and the magnitude and frequency of release. The release estimates then serve as 

29 inputs to the assessment of survival and distribution subsequent to release. 

30

31 When there are data available to predict releases, four main steps are used in 

32 constructing the release assessment. 

33

34 a) First, you should collect and synthesize information on how many organisms are 

35 generated at the site of release. For example for land application of biosolids, this 

36 would include the size of each load, the number of loads per unit time, and the 

37 concentration of organisms in a load. 

38
Microbial Risk Assessment Guideline Page 89

Draft – Do not cite, circulate, or quote

1 b) Second, develop a process description to locate the places where releases may 

2 occur. For an industrial process one needs to know where and how the 

3 microorganisms are grown and how they are separated from their growth medium. 

4 The process description should also consider the manner in which 

5 microorganisms may be subjected to circumstances such that they may be 

6 inactivated or destroyed. 

7

8 c) Third, relate each possible point of release to the process involved. In agriculture 

9 this could be a drop spreader or spray nozzle. In an industrial plant it could be 

10 off-gassing (during separation, as from a centrifuge), from equipment during 

11 clean-up, or during product transportation. 

12

13 d) Fourth, for each release source develop quantitative estimates of release which 

14 specify the amount of release, the time frame of release, and the media of release. 

15 If inactivation procedures or engineering controls are applied to the release 

16 source, then their effectiveness will need to be estimated to quantify the amount 

17 released after the control or treatment.

18

19 When intentional or incidental releases occur from inanimate sources, quantitative 

20 estimates can frequently be obtained. Releases to air from sources such as a fermentor‘s 

21 off-gas can be measured as the viable count per site per unit time. You can treat these as 

22 point source releases occurring at approximately rooftop height. A similar approach can 

23 be employed for wastewater releases. Modeling of release modes is usually medium 

24 specific; the output may be useful in further estimating dispersal of the microorganisms 

25 from their source. Incidental releases may be modeled based on empiric evidence 

26 compiled for specific activities, but source evaluations of intentional releases to the 

27 environment are often complex and case specific.

28

29 The dynamic nature of microorganisms is one characteristic that differentiates 

30 microbial exposure assessment from chemical exposure assessment. Predicting changes 

31 in the number of microorganisms along an exposure pathway is often necessary to

32 accurately estimate exposure doses. Environmental conditions that can influence the 

33 growth and decline in the number of microorganisms present in a specific media include, 

34 but are not limited to:

35

36 a) water activity,

37

38 b) pH,

39

40 c) carbon source,

41

42 d) electron acceptor,

43

44 e) sunlight intensity,

45

46 f) temperature,
Page 90 Microbial Risk Assessment Guideline

1

2 g) population density of the microorganisms and/or other microbiota that compete 

3 for nutrients in the media or support pathogen growth e.g. within biofilms on 

4 surfaces, and/or

5

6 h) presence of disinfectants or antimicrobials.

7

8 Depending on the characteristics of the microorganisms, some survive throughout an 

9 exposure pathway while others do not. 

10

11 Many microorganisms have more than one exposure pathway and corresponding 

12 route of intake, often referred to as an exposure route or route of entry. Common

13 exposure routes are:

14

15 a) inhalation (nose and mouth to lungs),

16

17 b) ingestion (oral intake of food, water, soil, and inanimate objects), and

18

19 c) direct (via skin, eyes, ears, inanimate objects, and sexual contact).

20

21 Some microbial exposure assessments are able to characterize exposure via a 

22 primary exposure pathway (e.g., Escherichia coli O157:H7 from infected cattle to an 

23 individual via undercooked ground beef). Nevertheless, microbial and epidemiological 

24 evidence may indicate that some microorganisms spread via cross-contamination 

25 pathways; these pathways are sometimes difficult to characterize. Also, many

26 microorganisms (e.g., noroviruses) are spread from an infected person to another who is 

27 not. This exposure pathway may be poorly characterized because sufficiently detailed 

28 data are often lacking (Zhao, 1998).

29

30 In addition to ingestion routes, chemical risk assessment is well developed for 

31 inhalation and dermal exposure routes. For example EPA has many guidance document 

32 for chemicals such as, Methods for Derivation of Inhalation Reference Concentrations 

33 and Application of Inhalation Dosimetry, the Exposure Factors Handbook, and Risk 

34 Assessment Guidance for Superfund Volume I: Human Health Evaluation Manual (Part 

35 E, Supplemental Guidance for Dermal Risk Assessment) Final (RAGS-E) (EPA, 1994, 

36 1997a, 2004d). Inhalation and direct (e.g., dermal) exposure routes and to what degree 

37 chemical guidance document might be adapted for MRA may be further discussed in 

38 future volumes of this Guideline.

39

40 5.1.4 What Environmental Factors Can I Take into Consideration?

41

42 Environmental factors are considered when the risk assessment is at the stage of 

43 calculating the amount of pathogen that constitutes an exposure. Examples of 

44 environmental factors are provided below with a short description.

45
Microbial Risk Assessment Guideline Page 91

Draft – Do not cite, circulate, or quote

a) Ecological Niche – An ecological niche is an n-dimensional hyperspace29 1 where 

2 the abiotic and biotic elements in the environment allow a species to survive. As 

3 an ecologic niche changes, the amount of pathogen might fluctuate and/or the 

4 likelihood of exposure might increase or decrease.

5 

6 b) Gradients of Concentrations – It is rare that microorganisms are equally 

7 distributed throughout a medium, therefore gradations of concentrations can be 

8 considered in accordance to the media being assessed, and the exposure being 

9 characterized.

10 

11 c) Persistence – Persistence in the environment of an organism (in some form i.e., 

12 spores) can be considered. Those organisms that are not stable in the environment 

13 would pose a different exposure, and subsequent risk, than those that are stable.

14 (See Chapter 3)

15

16 d) Matrix Characteristics – The characteristics of the matrix where the pathogen is 

17 found may determine the amount and state (e.g., dormant, alive, dead) of 

18 microorganism available to a receptor. Things like oxygen content, fat content, 

19 pH, temperature, water content influence the survivability of microorganisms.

20

21 5.1.5 What is an Exposure Scenario?

22

23 An exposure scenario (or hazardous event) is the set of conditions or assumptions 

24 about sources, exposure pathways, amounts or concentrations of microorganisms, and the 

25 characteristics of the exposed individual, population, or subpopulation that constitute one 

26 or more exposures. An exposure assessment may comprise many different scenarios. 

27 Each scenario is the basis for evaluation and quantification of exposure(s) in a given 

28 situation.

29

30 An analysis of ―what if‖ options for mitigation measures, interventions, or policy 

31 changes is often referred to as a scenario analysis. This type of analysis allows you to 

32 evaluate the public or environmental health benefits of various measures that prevent or 

33 mitigate exposures.

34

35 5.1.6 What are Qualitative and Quantitative Exposure Assessments?

36

37 A qualitative exposure assessment is based on data and information which, when 

38 considered along with expert knowledge and identification of attendant uncertainties, 

39 provides a characterization of exposure in descriptive terms (e.g., high, medium, or low). 

40 You will need to conduct qualitative exposure assessment when there is not sufficient 

41 numerical data to develop a quantitative exposure assessment.

42

43 A quantitative exposure assessment provides numerical expressions of exposure. 

44 Such an assessment provides numerical estimates of the likelihood of different microbial

 

29 N-dimensional space that has more than 3 dimensions can be called hyperspace.
Page 92 Microbial Risk Assessment Guideline

1 dose amounts, as well as numerical measures of confidence about its estimates (i.e., 

2 uncertainty). 

3

4 Sometimes multiple quantitative exposure assessments are conducted for a 

5 microbial hazard in order to rank sources, vehicles, and/or pathways of exposure based 

6 on risk. One example of such an approach is a risk assessment of Listeria monocytogenes

7 in ready-to-eat foods (FDA/USDA/CDC, 2003).

8

9 5.1.7 What is Variability in Exposure Assessment?

10

11 Variability describes a range of possible events. It is purely the effect of chance; 

12 it can only be altered by changing the chance of something occurring. Variability in 

13 exposure can be caused by differences in location, activity, and/or behavior of exposed 

14 individuals at a particular point in time. These sources of variation result in differences 

15 in exposure to a microbial hazard(s) in various media. Variability is also caused by 

16 differences in the initial occurrence of microorganisms in various media (e.g., air, soil, 

17 food, and water). Because microorganisms grow and decline within media along the 

18 exposure pathway, there is variability in the amount of microbial hazard per unit of media 

19 intake by an individual or sub-population. You may need to characterize variability in:

20

21 a) The number of microorganisms initially present in the medium

22 b) The environmental conditions in which microorganisms exist

23 c) The processes through which microorganisms move within scenarios

24 d) The dose of microorganisms per unit of intake (e.g., serving of food, inhalation

25 unit, amount of water ingested; spatial and temporal variability)

26 e) The amount of intake (inter-individual variability in exposure)

27 f) Exposures across time (temporal variability)

28 g) Exposures across geographic location (spatial variability)

29 The types of variability considered depend on the type of exposure assessment to be 

30 developed as part of the overall risk assessment.

31

32 5.1.8 What is Uncertainty in Exposure Assessment?

33

34 Uncertainty is imperfect knowledge. You can reduce uncertainty by further study 

35 or by accumulating more information. Uncertainty may reflect imperfect knowledge of 

36 the microbial hazard (e.g., its virulence), environmental pathway/processes, or the human 

37 populations under consideration. Sources of uncertainty fit into two broad categories:

38

39 a) Uncertainty regarding one or more parameters in an exposure assessment 

40 (parameter uncertainty)
Microbial Risk Assessment Guideline Page 93

Draft – Do not cite, circulate, or quote

1

2 b) Uncertainty as a result of incomplete information or scientific theory needed to 

3 fully define the causal bases of exposures (structural model uncertainty)

4

5 Availability and quality of data and information can reduce the amount of 

6 uncertainty in exposure estimates. Objective depictions of uncertainty improve the 

7 transparency of information used by decision-makers in managing risk. The process of 

8 interpreting the influence of uncertainty on the results of an exposure assessment is 

9 referred to as uncertainty analysis.

10

11 5.1.9 What is a Deterministic Exposure Assessment?

12

13 You can conduct a quantitative exposure assessment using ―most likely‖ or 

14 ―conservative‖ values for the variables and uncertain parameters included in the set of 

15 scenarios. However, unless data are not available, the use of single point estimates is not 

16 a preferred approach to inform decision making (OMB, 2007b; EPA, 2002b). These 

17 values are often referred to as point estimates and can result from collapsing the 

18 variability and/or uncertainty about random variables or parameters into singular values. 

19 Depending on how the point estimates are designed, the results may either represent an

average or other extreme exposures (e.g., 95th 20 percentile) among a specified population. 

21

22 The use of point estimates in an exposure assessment is referred to as 

23 deterministic (static) modeling. Point estimates do not account for variability in the 

24 occurrence of the microorganisms at the source, variability in growth and/or decline in 

25 the number of microorganisms through the exposure pathway, or variability in intake 

26 across the population of individuals exposed to the microorganisms. Furthermore, a 

27 deterministic exposure assessment does not explicitly characterize the uncertainty about 

28 exposures. Without explicit characterization of variability and uncertainty, it is possible 

29 that point estimates will substantially over- or under-estimate exposures. If highly 

30 conservative point estimates – thought to be protective of public health – are used, the 

31 deterministic results may be characterized as worst-case estimates. 

32

33 In some cases, deterministic modeling may be used to simplify the modeling of a 

34 highly complex system. For example, extensive modeling of transmission processes 

35 among a population may require simplifying assumptions about contact rates and 

36 transmission coefficients. 

37

38 Another use of deterministic modeling is during your initial analysis of an 

39 exposure assessment model. Propagating simple numbers through the model may help 

40 with error-checking the mathematics of the model. Also, such calculations can provide 

41 early indications of the importance of various model components or pathways. 

42 Nevertheless, conclusions from such screening analyses should be cautiously interpreted 

43 because omission of variability can generate misleading results. 

44

45
Page 94 Microbial Risk Assessment Guideline

1 5.1.10 What is a Stochastic Exposure Assessment?

2

3 In contrast to using point estimates, the use of probability distributions for each 

4 parameter in an exposure assessment is preferred. A probability distribution includes 

5 both a range of values and the likelihood of occurrence for each numerical value. Use of 

6 probability distributions throughout the exposure assessment allows you to represent 

7 variability in exposures of individuals and/or sub-populations. You can also use 

8 probability distributions to characterize the uncertainty in exposure assessments. When 

9 developing a stochastic model, you can use point estimates to verify the mathematical 

10 formula or confirm that the computer code actually performs correctly for trivial cases 

11 (for example, the microbes are not present in the output if the initial concentration is 

12 zero).

13

14 The use of probability distributions in an exposure assessment is referred to as 

15 stochastic modeling. Probability distributions represent the variability and uncertainty 

16 inherent in a system. Stochastic modeling can provide more realistic results by 

17 accurately characterizing the impacts of known sources of variability and uncertainty on 

18 risk estimates. Risk assessments are often concerned with the occurrence of rare events 

19 and stochastic modeling may reveal rare but consequential results (e.g., the occurrence of 

20 an unlikely, but large, population outbreak.)

21

22 Stochastic models often use computer simulations to mathematically combine 

23 multiple probability distributions in an exposure assessment calculation. Monte Carlo 

24 analysis (more detail in next section) is the most widely used probabilistic method to 

25 estimate these combinations. Advanced Monte Carlo modeling techniques can also 

26 quantitatively characterize uncertainty in exposure estimates (Gilks et al., 1996).

27

28 Stochastic modeling is usually more resource-intensive than deterministic 

29 analysis. Defining model inputs as probability distributions can require additional steps in 

30 the planning, review, and communication of the exposure estimates. Use of probability 

31 distributions, however, provides a framework for incorporating more of the available 

32 information into an exposure assessment.

33

34 If data for critical variables and parameters are available, then you should 

35 consider stochastic modeling for the exposure assessment if time permits. Alternatively, 

36 if data and information are insufficient, consideration should be given to the use of other 

37 modeling techniques. For example, the use of interval mathematics or fuzzy mathematics 

38 may provide a more credible assessment of the probabilistic boundaries of exposure than 

39 standard methods using probability distributions (Ferson, 1996).

40

41 5.1.11 What is Monte Carlo Analysis?

42

43 Monte Carlo analysis is a commonly used quantitative technique for exposure 

44 assessments. It involves the random sampling of each of the probability distributions in a 

45 model to estimate the likelihood of the model‘s possible results (Vose, 2008). Each re-
Microbial Risk Assessment Guideline Page 95

Draft – Do not cite, circulate, or quote

1 calculation of the model is termed an iteration, and a set of iterations constitutes a 

2 simulation.

3

4 A cardinal rule of this analytical technique is that every iteration should be 

5 possible in nature (Vose, 2008). If followed, this rule can prevent errors in modeling 

6 logic. For example, a predicted serving cannot contain 2.7 microorganisms, although the 

7 average concentration across some volume or mass may be 2.7 microorganisms. While it 

8 is not possible that any individual would be exposed to exactly 2.7 (viable) organisms, 

9 the simulation can be acceptable if the exposure distribution is defined and sampled 

10 correctly. If a Poisson (discrete) distribution with a mean of 2.7 is used in the risk 

11 modeling, the individual Monte Carlo iterations will all take discrete (integer) values, 

12 maintaining the realism of each iteration and of the simulation as a whole. 

13

14 You may face three common problems when using Monte Carlo methods:

15

16 a) First, correlations and dependencies between variables may be unknown. If 

17 dependent variables are mistakenly assumed to be independent in a Monte Carlo 

18 analysis, the likelihood of common occurrences in the real world may not be 

19 correctly estimated via simulation. If information on the correlations is not 

20 available, then alternative methods (e.g., interval or fuzzy mathematics) may 

21 avoid such mistakes because the probability boundaries calculated by these 

22 methods can include a full range of correlations in their results (Ferson, 1996).

23

24 b) Second, the data necessary to estimate input distributions may be incomplete or 

25 entirely lacking. Although inadequate data are a problem for any exposure 

26 assessment method, Monte Carlo methods are particularly disadvantaged because

27 these methods require explicit definition of the model inputs. Sometimes uniform 

28 or triangular distributions are used when data are sparse.

29

30 c) Third, the mathematical structure of the exposure assessment model may be 

31 questionable. Risk analysts often acknowledge the limitations induced by these 

32 problems and employ sensitivity analysis (or other methods) to assess their 

33 influence on estimated exposures (Law and Keaton, 2000; Vose, 2008). Care 

34 should be taken to make sure that uncertainties related to model specification are 

35 addressed by comparing the quality of fits across different model forms. Where 

36 sufficient data are available, methods such as cross-validation may be used. 

37 Using this approach, an exposure model is estimated using a portion of the data 

38 (usually about 70 percent) and then the model is tested for consistency with the 

39 remaining data set. 

40

41 Examples of Agency guidance on probabilistic risk assessment include, EPA‘s 

42 Guiding Principles for Monte Carlo Analysis (EPA, 1997b) and EPA‘s Using 

43 Probabilistic Methods to Enhance the Role of Risk Analysis in Decision-Making With 

44 Case Study Examples (EPA, 2009b).

45
Page 96 Microbial Risk Assessment Guideline

1 5.1.12 How does Exposure Assessment Fit with the Other Components of 

2 Risk Assessment?

3

4 Fundamentally, risk assessment is a predictive analysis. It intends to ―envision 

5 how the future will turn out if we undertake a course of action...‖ (Kaplan and Garrick, 

6 1981). 

7

8 Predictions are accomplished by answering three questions: 

9

10 a) What can go wrong?

11

12 b) How likely is that to happen?

13

14 c) What are the consequences given that it does happen?

15

16 The set of answers to the first question outlines the mutually exclusive scenarios 

17 to be considered in an exposure assessment. These scenarios are given the symbol si

18 where the index i implies there are potentially many, mutually exclusive, scenarios 

19 wherein something goes wrong (i = 1 to N). Each si that is identified has an associated 

20 probability that it occurs (li), as well as some measurement of its consequence (xi).

21

22 To a large extent, planning and scoping and hazard identification will determine 

23 the scenarios to consider for the exposure assessment. It is imperative that hazard 

24 identification establish the biologic plausibility of causative mechanisms considered in 

25 any exposure assessment.

26

27 Exposure assessment, therefore, is most invested in determining the likelihood of 

28 scenarios and a provisional consequence of the scenarios. In the context of exposure 

29 assessment, the provisional consequence is the magnitude of human exposure (usually 

30 dose) that occurs at the end of the scenario. 

31

32 The output of the exposure assessment is commonly combined with the dose33 response relationship developed in hazard characterization to predict the probability of an 

34 adverse human health outcome. This combination is considered in the risk 

35 characterization chapter (Chapter 6).

36

37 You can order the scenario triplets from smallest to largest microbial dose (i.e., x1

38 ≤ x2 ≤...≤ xN) and similarly align the scenarios and likelihoods for each of those doses. 

39 This organization conveys the likelihood of increasingly larger microbial doses, as well 

40 as the scenarios that are responsible for those doses. Likelihood is a number constrained 

between zero and one. Because the s1, s2,...sN are defined as mutually exclusive30 41

42 scenarios, it is also true that

43

 

30

 Note that each scenario may include exposures through more than one pathway. Each scenario includes 

defined combinations of exposures that are sufficiently different to justify separate consideration. 
Microbial Risk Assessment Guideline Page 97

Draft – Do not cite, circulate, or quote

1

1

N

i

i

l

1 (Eq. 5.1)

2

3 (if all si represent an exhaustive list of the things that can happen, then this sum will 

4 exactly equal one). 

5

6 In some exposure assessments, li will refer to the fraction of all possible 

7 exposures – including exposures with a dose of zero – that derive from scenario si and 

generate a microbial dose of xi 8 . If you know the total exposures for the microbial hazard 

9 that can occur in one year (M), then the frequency of microbial dose xi is F(xi) = M li

10 (i.e., the number of exposures with a dose of xi in one year is the product of total 

11 exposures and likelihood). 

12

13 Given the frequency of microbial doses, the exposure assessment can directly 

14 generate a frequency distribution for human exposures (Figure 5.1). This distribution is 

simply a graphic display of the paired {xi 15 ,F(xi)} values. 

16

0

0.05

0.1

0.15

0.2

X1, S1X3, S3X5, S5X7, S7X9, S9X11, S11X13, S13X15, S15X17, S17X19, S19X21, S21

Dose (X); Scenario (S)

F(X) = Likelihood x M

17

18

19 Figure 5.1 An illustrative exposure distribution developed by ordering scenarios (si) 

20 by the size of dose (xi) and calculating frequency as the product of total exposures 

21 (M) and likelihood of the scenario (li). 

22

23 5.1.13 Do Different Exposure Scenarios Always Generate Different 

24 Microbial Doses?

25

26 Although exposure scenarios may be mutually exclusive (i.e., represent unique 

27 pathways for microbial exposure), it is not necessary for different scenarios to result in 

28 different magnitudes of exposures. For example, you might consider consumption of 

29 water from a single source to which a treatment, with variable effectiveness, is applied. 

30 One scenario might reflect highly effective water treatment and consumption of a single 

31 12 ounce serving that contains no microorganisms. Another scenario might reflect less 

32 effective water treatment and consumption of a serving of 6 ounces that contains no 
Page 98 Microbial Risk Assessment Guideline

1 microorganisms. Although the scenarios are different, the resulting dose (0 

2 microorganisms) is the same. 

3

4 The process of developing an exposure assessment can be complicated and it is 

5 not always understood, before the analysis begins, what scenarios will generate what 

6 doses. It is possible to design the exposure assessment such that scenarios are grouped by 

7 the dose they generate. Alternatively, the scenarios can be described a priori and the 

8 exposure doses they generate subsequently determined analytically. Depending on how 

9 scenarios are defined, it is possible that one exposure scenario may be associated with 

10 multiple doses (and their attendant likelihoods). 

11

12 Consider a scenario wherein a hamburger patty – initially contaminated with 100 

13 E. coli O157:H7 organisms and stored at 60 ̊F for 24 hours, then cooked at 130 ̊F for 4 

14 minutes – is consumed by a healthy 30-year old male. The likelihood of this scenario 

15 might be 0.00089 percent of all servings of ground beef consumed by such a person. The 

16 consequence, from an exposure perspective, might be that 1000 E. coli O157:H7 

17 organisms are ingested. This scenario, when accompanied by all other scenarios for 

18 hamburger patties, is useful because it provides sufficient detail for decision-makers to 

19 appreciate the risk of adverse human health outcomes from consuming hamburger patties 

20 (especially once the risk characterization is completed). Ultimately, the purpose of risk 

21 assessment is to support decisions regarding risk and those decisions typically hinge on 

22 the three elements contained here:

23

24 a) What are the substantial scenarios?

25

26 b) What are the magnitudes of their consequences?

27

28 c) How likely are such scenarios to occur?

29

30 Short contemplation of this example, however, will undoubtedly raise many 

31 questions. How was the scenario identified; surely there are thousands of scenarios at 

32 that level of detail (e.g., what about a hamburger that is cooked at 129 degrees Fahrenheit 

33 [°F] for 2.5 minutes)? How was the likelihood determined? That likelihood seems very 

34 precise; do all servings of hamburger from this scenario have the same likelihood of 

35 occurrence? Why does an exposure of 1,000 microorganisms result from this scenario? 

36 Do all hamburgers handled this way have the same number of microorganisms at 

37 consumption? Are you really this certain about all of this? Are there parts of the 

38 scenario that are more influential on the consequence than other parts? 

39

40 The following sections of this chapter intend to outline general answers to these 

41 questions by discussing the process of developing, analyzing, and reporting an exposure 

42 assessment. These sections address scenario development, calculation of the likelihoods 

43 of scenarios, predictions of exposure doses, uncertainty analysis, and interpretation of 

44 exposure assessment results. Nevertheless, the ideas and concepts in these sections are 

45 not intended to be dogmatic prescriptions for completing an exposure assessment. In 

46 exposure assessment, there are many different approaches that may be valid for solving a 
Microbial Risk Assessment Guideline Page 99

Draft – Do not cite, circulate, or quote

1 problem. Yet, all valid approaches share some fundamental similarities. It is these 

2 similarities that are the focus in the remainder of this chapter.

3

4 The concepts of scenario, likelihood, and consequence are fundamental to 

5 developing an exposure assessment. When analyzing an exposure assessment, the 

6 concepts of variability and uncertainty are crucial. Finally, effective communication and 

7 validation are essential considerations when reporting the findings of an exposure 

8 assessment.

9

10 5.2 How do I Develop an Exposure Assessment?

11

12 As discussed in Chapters 1 and 2, the beginning of the exposure assessment starts 

13 in planning and scoping. This important groundwork lays the foundation for a successful 

14 exposure assessment.

15

16 In general, the exposure assessment should be parsimonious, that is, as simple as 

17 possible while still including the important sources and steps leading to the exposure of 

18 concern. Based on ―problem formulation‖ (see Section 2.1.1), you should make 

19 decisions regarding the approach to exposure assessment (e.g., attribution modeling or 

20 process modeling, empirical modeling of epidemiological data, probabilistic or 

21 deterministic, dynamic or static) and structure of the assessment model (which pathways, 

22 single or multiple models) (Hurd and Kaneene, 1993). 

23

24 Given the complexity of many exposure assessments, the process becomes a 

25 multi-disciplinary collaborative effort. Subject matter experts are regularly consulted and 

26 their judgment incorporated into the process. 

27

28 Conceiving an exposure assessment can be daunting. For this reason, some 

29 structure is needed. It is useful to begin by describing a conceptual model with all the 

30 necessary scenarios, followed by a full mathematical development of the conceptual 

31 model and, finally, by collection and analysis of data necessary to inform the model 

inputs identified in the mathematical model.31 32 This structure theoretically ensures 

33 adequate attention to, and scrutiny of, the exposure assessment in an ordered and efficient 

34 manner.

35

36 Efficient exposure assessment is enhanced when precedents exist and are used. 

37 For example, beginning with already published conceptual models that require minor 

38 modification for a new application avoids unnecessary duplication of effort. Following 

39 reasonable precedents is also how standard methods can evolve.

40

41 5.2.1 What is the Purpose of the Risk Assessment?

42

43 With respect to purpose, most risk assessments can be categorized into two broad 

44 categories – retrospective or prospective. A retrospective purpose applies to microbial

 

31 Note that this conceptual model may be software driven and is different than the conceptual model that is 

developed in planning and scoping which illustrates the broad overview of how risk happens.
Page 100 Microbial Risk Assessment Guideline

1 hazards that are well-established as occurring sporadically or epidemically. A 

2 prospective purpose for a risk assessment applies to potential microbial hazards for which 

3 the adverse human health effects are not established. This categorization scheme is 

4 important because a different series of questions should be considered prior to developing 

5 the exposure assessment depending on the purpose category (see Table 5.1). These 

6 questions are not exhaustive, but the answers will guide the development of an exposure 

7 assessment appropriate for informing specific risk management decisions. In addition, 

8 answering these questions may also identify information requirements and methods for 

9 collecting this information.

10

11 Table 5.1 General Questions Considered Prior to Conducting an Exposure 

12 Assessment

13

Prospective Purpose Retrospective Purpose

Is disease onset only a potential at this point, 

with the luxury of time to provide an answer?

Is disease onset imminent or already occurring, 

thus requiring an immediate answer?

Should the exposure assessment be structured 

as an in-depth analysis using less conservative 

assumptions?

Should the exposure assessment be structured 

more as a screening analysis, using default 

and/or more conservative assumptions?

Should the evaluation focus on both long-term 

and short-term exposures?

Should the evaluation focus only on short-term 

exposures?

Should the analysis focus on both low-level 

and high-level exposures?

Should the analysis focus only on high-level 

exposures?

Should attempts be made to measure (sample) 

or model exposures within the body?

Should concentrations be measured (sampled) 

or modeled in the media of concern?

Should the evaluation consider all potential 

exposure pathways for that particular microbial

agent?

Should the evaluation focus only on those 

exposure pathways of imminent concern?

Should the analysis attempt to consider 

aggregate and/or cumulative exposures to 

multiple microbial agents?

Should the analysis focus only on the microbial

agent causing the adverse health impact and 

only this exposure?

14

15 5.2.2 Which Scenarios Can I Consider?

16

17 Scenario development is the conceptual and creative part of exposure modeling. 

18 It melds considerations of purpose and scope with established or putative causal 

19 pathways. Although standardized frameworks for some microbial exposure assessments 

20 have been suggested (e.g., for food safety applications, see Nauta, 2002), there are few 

21 hard and fast rules for how or what scenarios should be considered in an exposure 

22 assessment. 

23

24 Despite the lack of standard scenarios, those included in a specific exposure 

25 assessment should be clearly communicated and understood. Explicit diagrams (e.g., 

26 conceptual model) can be developed early in a project and these diagrams can include 

27 detailed descriptions of the inputs, parameters, flow and relationships of these 

28 components in the exposure assessment. It is useful to establish meaningful symbols

29 within the diagrams to represent these model components early in the project. Consistent 
Microbial Risk Assessment Guideline Page 101

Draft – Do not cite, circulate, or quote

1 use of symbols will encourage clear communication among risk assessors and risk 

2 managers. 

3

4 Detailed diagrams with consistent symbols can be discussed with risk managers 

5 early and often in an exposure assessment project. This approach encourages open and 

6 clear communication of the modeling approach. Exposure assessment is an iterative and 

7 collaborative process; clear descriptions of the model with constructive feedback from 

8 persons outside the project will facilitate improvements at the conceptual model stage.

9

10 Exposure assessments should be parsimonious (simple as possible, but not 

11 simpler). Although hazard identification may suggest that the chain of causation for 

12 exposure is complex, the exposure assessment should only incorporate the complexity 

13 needed for the purposes of the risk assessment. In other words, risk assessments should 

14 leave avenues unexplored that, although interesting, do not specifically address risk 

15 management options. For example, the exploration of unproven causal relationships is 

16 rarely useful in exposure assessments. 

17

18 Conceptually, an exposure assessment begins by considering the occurrence of 

19 the target microorganism at some place and/or time (i.e., its source). It is plausible that 

20 an exposure assessment could begin by characterizing the distribution of doses (or 

21 concentration of microorganisms) at the time of exposure. Such a beginning would not 

22 include consideration of sources and processes that produced the exposures. 

23 Nevertheless, if the purpose of the risk assessment was investigative and risk managers 

24 strictly sought estimates of the potential adverse human health events that could occur, 

25 then this approach would be satisfactory. Of course, it would only work if data are

26 available to estimate the distribution of doses just prior to exposure. 

27

28 Often exposure assessments begin by considering the occurrence of 

29 microorganisms at a place and time that is somewhat distant and prior to the actual 

30 human exposures. Availability of microbial occurrence data is one justification for where 

31 and when to begin the conceptual model. For example, if the only microbial data 

32 available refers to its occurrence prior to the application of some treatment process, then 

33 the exposure assessment may begin at that place and time. The purpose of the risk 

34 assessment is another justification for where and when to begin the conceptual model. 

35 For example, if the purpose is to set a regulatory performance measure for some 

36 treatment process, then the exposure model will need to begin its considerations of 

37 microbial occurrence at some point prior to the treatment process. 

38

39 Once the beginning of the exposure assessment is determined, you can use 

40 predictive microbiology to determine how microbial occurrence changes before a dose 

41 reaches an exposed human. Processes refer to events or phenomena that influence 

42 microbial occurrence between the beginning and end of the exposure assessment. As 

43 discussed later, generic processes include growth, attenuation, mixing and partitioning. 

44 Which and how many of these processes are included in an exposure assessment depends 

45 on the planning and scoping and hazard identification stages of the project. 

46
Page 102 Microbial Risk Assessment Guideline

1 Many microbial exposure assessments involving food or water will include three 

2 general sequential stages: bulk processing, bulk transport, and consumption (Haas et al., 

3 1999). Including these stages is often necessary to examine the influence of factors (or 

4 covariates) on microbial levels at the point of exposure. This inclusion is especially 

5 important when the purpose of the risk assessment is to predict changes in risk from one 

6 or more changes in these stages. For example, a risk assessment that examines a policy 

7 to require more controlled refrigeration of a food after it is produced – to limit growth of 

8 the target microbe – will include the bulk transport stage in the exposure assessment so 

9 that the effect of the policy relative to current conditions is measurable. 

10

11 To complete scenario development, you should determine how each exposed 

12 population will come in contact with the microorganism of concern. The three well13 recognized routes of initial exposure are inhalation, ingestion, and direct contact. These 

14 routes will influence the mathematical scale of the model. For example, inhalation 

15 exposures will likely depend on concentration measures of microorganisms (e.g., 

16 microorganisms per cubic meter) while ingestion exposures may depend on tracking 

17 actual counts of microorganisms to determine the average number of microorganisms in a 

18 serving. 

19

20 Three examples illustrate different approaches to developing exposure scenarios:

21

22 a) To estimate the risk of viruses in water, an exposure assessment considered the 

23 volume of water consumed and the average concentration of viruses per liter of 

24 water supplying a large city (Haas et al., 1993). In this exposure assessment, no 

25 attention was given to mechanisms potentially responsible for the average virus

26 concentration or to factors that might cause variability in virus concentration 

27 across time or water supplies. 

28

29 For this example, a scenario could represent one possible combination of 

30 concentration (e.g., viruses per milliliter) and water consumed (e.g., milliliters per 

31 person per day). Therefore, the set of scenarios would include all possible 

32 combinations. Alternatively, this analysis is an example as a single scenario (e.g., 

33 water consumed) with variable average dose of virus per day.

34

35 b) To estimate the risk of tuberculosis (TB) transmission on a commercial airliner, 

36 an exposure assessment examined the spatial variability in concentration of this 

37 mycobacterium (Ko et al., 2004). This assessment included air transfer rates 

38 between different cabins in the plane, respiration rates of potentially exposed 

39 individuals, distance from an infectious source, and the rate of expired TB 

40 organisms from an infectious source. Hazard identification had indicated that all 

41 these factors might influence exposures of passengers to TB on a plane. The 

42 exposure assessment focused on the cumulative exposure to infectious TB during 

43 a long (> 8 hours) intercontinental flight. For this example, scenarios reflected 

44 the combinations of cabin location of exposed individuals, location of source 

45 individual and airflow direction.

46 
Microbial Risk Assessment Guideline Page 103

Draft – Do not cite, circulate, or quote

1 c) To estimate the risk of human illness from E. coli O157:H7 in ground beef, an 

2 exposure assessment included causative factors during on-farm production, 

3 slaughter, and storage/preparation stages (Ebel et al., 2003). This complex farm4 to-table exposure assessment examined the influence of season, live animal 

5 prevalence, transport, dehiding, carcass decontamination, carcass chilling, carcass 

6 fabrication, grinding, storage/handling, cooking and consumption on the predicted 

7 exposure per ground beef serving. Hazard identification indicated these factors 

8 might influence exposures. Control of many of these factors was considered by 

9 risk managers.

10

11 For this example, scenarios reflected the combinations of different sources (e.g., 

12 prevalence of different classes of infected cattle presented for slaughter by 

13 season) with effectiveness of decontamination procedures with 

14 times/temperatures of storage and cooking. The unique combinations that 

15 represent individual scenarios were too numerous to count. Furthermore, ground 

16 beef servings were created from combinations of scenarios that produced the beef 

17 that went into a load of ground product. Therefore, the exposure assessment did 

18 not list distinct scenarios but, instead, produced frequency distributions for levels 

19 of E. coli O157:H7 per serving of ground beef for low- and high-prevalence 

20 seasons of the year. 

21

22 An exposure assessment depends on the microbial agent's properties and the 

23 environmental transmission factors relevant to exposures. Microbial agents may stem 

24 from more than one source, may be transmitted via multiple routes of exposure, and may 

25 be spread via secondary or vector transmission. An exposure source can originate from 

26 either natural or anthropogenic events, activities, or locations that generate or release 

27 hazards. Exposure sources can be classified as point sources or non-point sources. 

28 Exposure routes include inhalation (nose and mouth to lungs), ingestion (oral), and direct 

29 contact (skin, eyes, ears, and sexual). Exposure routes are situation-dependent and 

30 medium specific. An exposure pathway encompasses both exposure source and route, 

31 and generally is described by a source and release from a source, an exposure point, and 

32 an exposure route. Table 5.2 illustrates the exposure assessment components and their 

33 relationship to various exposure points.

34
Page 104 Microbial Risk Assessment Guideline

1

2 Table 5.2 Transmission Pathways for Microbial Hazards

Source Release Exposure Point Exposure Route

Natural or 

Anthropogenic

Point or Nonpoint

Natural

Accidental

Intentional

Food

Meat

Poultry

Eggs and Egg 

Products

Fish and shellfish

Produce

Dairy Products

Other Food 

Products

Ingestion

Direct 

Contact

Water

Surface Water

(Drinking)

Ground Water 

(Drinking)

Ingestion

Direct 

Contact

Inhalation

Recreational 

Water

Compost Tea32

Incidental 

Ingestion

Direct 

Contact

Inhalation

Soil

Surface Soil

Subsurface Soil

Sediment

Manure

Biosolids

Air Ambient Air

Indoor Air

Surfaces Porous

Non-porous

Biota

Plants

Animals, 

including 

humans

3

4 The dynamic nature of host-pathogen interactions, unique to infectious disease 

5 risk assessment, can lead to secondary transmission. The strictest definition of secondary 

6 transmission pertains to direct human-to-human contact between a primary case (infected 

7 or ill) and a secondary case that becomes infected or ill from that contact. Broader 

8 definitions include secondary cases that arise from contact with fomites or food or water 

9 contaminated by primary cases. Where secondary transmission includes infection from 

10 pathogens in the environment (e.g., fomites), it would not be considered secondary unless 

 

32 Compost tea is made by soaking or steeping compost in water. The resulting compost tea is used for 

fertilizing plants.
Microbial Risk Assessment Guideline Page 105

Draft – Do not cite, circulate, or quote

1 it occurs in the context of an outbreak where primary cases have already been identified. 

2 In such a case, the term ―secondary transmission‖ is not used in the strictest sense, but is 

3 commonly used by public health professionals in the context of an outbreak.

4

5 5.2.3 What are the Exposed Populations I Could Consider?

6

7 Identifying the exposed individuals or subpopulations of interest is crucial to 

8 determining what data are needed for an exposure assessment. In some cases there may 

9 be statutory requirements or Agency policy that require certain populations to be 

10 considered. The following factors are inherently tied to the exposure scenario:

11

12 a) Exposure Duration and Frequency – Certain individuals/populations may have 

13 a comparatively greater exposure duration or frequency in a given environment. 

14 For some microorganisms, it is possible that exposure to a low concentration for a 

15 long duration may be a concern even if the concentration would not pose any 

16 health risks under short-term exposures.

17

18 b) Exposure Routes – Knowing the characteristics of the exposed population helps 

19 develop the appropriate exposure pathways to consider in the exposure 

20 assessment.

21

22 c) Sensitive Individuals/Populations and Life Stages – Some 

23 individuals/subpopulations may be more susceptible to infection or more likely to 

24 develop severe manifestations of infection. For example, while healthy 

25 individuals may recover from an E. coli infection, it can be deadly to young 

26 children, the elderly, and people with compromised immune systems. Similarly, 

27 chronic smokers may have impaired mucociliary clearance mechanisms, and 

28 therefore, may be more susceptible to respiratory infections.

29

30 Once the exposed individual/subpopulation or life stage has been determined, it is 

31 important to include any scenario-specific conditions in the conceptual model. Sensitive 

32 populations and/or life stages which may have different exposure considerations include: 

33

34 a) Young children (up to 10 different age groups (EPA, 2005)

35

36 b) The elderly

37

38 c) Persons with compromised immune systems

39

40 d) Pregnant women

41

42 e) Chronic smokers

43

44 f) Military personnel (deployed and non-deployed)

45

46 g) Occupationally exposed individuals
Page 106 Microbial Risk Assessment Guideline

1

2 h) Other groups based on behavioral patterns (e.g. subsistence fishing)

3

4 This list is not comprehensive. You should select individuals/subpopulations of 

5 interest based on the purpose of the risk assessment. However, your groupings may be 

6 limited by data availability. Any differences in grouping between the subpopulations 

7 considered in the dose-response and the exposure components of risk assessment should 

8 be carefully explained and the implications of those differences should be discussed.

9

10 5.2.4 What are Common Approaches to Exposure Modeling I Can Use?

11

12 Although there are more elaborate classification schemes (Hurd and Kaneene, 

13 1993), most exposure assessments use an attribution and/or process modeling approach 

14 (Cox, 2006). Process modeling is the more common approach and it generates more 

15 traditional results (i.e., likelihood and dose) from the exposure assessment. Attribution 

16 modeling, in contrast, is used less frequently and does not explicitly estimate the 

17 likelihood of microbial doses. Instead, this approach implicitly synthesizes the exposure 

18 output with hazard characterization to generate links between model inputs and numbers 

19 of human illnesses. Some refer to attribution and/or empirically-based models as  ̳black20 box‘ approaches because causative mechanisms are not explicitly included (ECSSC, 

21 2003). Nevertheless, attribution modeling is empirically based and is often available to 

22 assessors when the problem is rich with surveillance evidence.

23

24 Attribution Modeling 

25

26 Empirical estimates of the number of human illnesses per year caused by the 

27 microorganism are sometimes available. These data might be used, in conjunction with 

28 other information, to back-calculate the fraction of illnesses attributed to various 

29 scenarios. This approach is infrequently used in microbial exposure assessment because 

30 epidemiological surveillance data is prone to data gaps that may not be well characterized 

31 (under reporting of illnesses is significant and varies). Nevertheless, national 

32 surveillance systems combined with thorough investigations of outbreaks may generate 

33 sufficiently valid conclusions about sources and causes of those illnesses. If such data 

34 are available, it is feasible to determine the number of exposures from a target source that 

35 cause human illnesses. Furthermore, the estimated number of illnesses that result from 

36 the target source can be divided by the estimated number of total exposures to determine 

37 the probability of illness per exposure. 

38

39 Ultimately, the number of illnesses from a microorganism that are attributed to a 

40 source represents the total number of illnesses that could be avoided by eliminating that 

41 source. If complete elimination of the source is not feasible, then further analysis might 

42 suggest what fraction of a baseline number of cases could be avoided by a proposed risk 

43 management option that improves the effectiveness of some particular process.

44

45 For illustrative purposes, assume 1,000 cases of illness caused by microorganism 

46 B are detected by public health surveillance each year (Figure 5.2). Analysis of the 
Microbial Risk Assessment Guideline Page 107

Draft – Do not cite, circulate, or quote

1 surveillance system suggests that only one of every 50 cases that occur is detected via 

2 this system. Therefore, the true number of cases is estimated to be 50,000 per year. 

3 Epidemiologic evidence suggests that 20 percent of cases are caused by Source 1 (e.g., 

4 ground beef). Research evidence suggests that 50 percent of source 1 exposures are 

5 directly attributable to process 1 (e.g., under-cooking). Based on these values, you 

6 estimate the number of illnesses attributed to Source 1 is 50,000 0.20=10,000. You 

7 estimate the number of those illnesses attributed to process 1 is 10,000 0.50=5,000.

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23 Figure 5.2 Schematic illustrating direction of inference when using an attribution 

24 approach to exposure assessment.

25

26 Process Modeling

27

28 For many exposure assessments, planning and scoping will suggest that 

29 attribution modeling is inadequate. A dearth of reasonable epidemiologic surveillance 

30 evidence sometimes precludes attribution modeling, as well. In these cases, a process 

31 modeling approach can be used. 

32

33 Generally, process modeling involves the characterization of processes that occur 

34 from the initial source of the microbe to the point of exposure. The influence of 

35 processes associated with growth or attenuation of microbial populations is one of the 

36 purposes of the predictive microbiology field. Research of microbial kinetics and 

37 lethality provides important insight to process modeling. 

38

39 Simulation of the effects of processes on microbial occurrence is commonly 

40 accomplished using Monte Carlo sampling techniques (see Section 5.1.11). 

41 Nevertheless, simple probabilistic models may not require Monte Carlo sampling and 

42 other dynamic modeling methods, such as those based on differential equations, may be 

43 solvable using other mathematical techniques (e.g., Cox, 2006). 

44

45 Regardless of the technique, process modeling is concerned with predicting 

46 exposures given the initial microbial occurrence. Its direction of inference is opposite 

Observed number of adverse

human health outcomes

from microbe B per unit time

Predicted number of adverse

human health outcomes 

from microbe B per unit time

Source 1 Source 2 Source 3 Source n ,.....,

process 1 process 2 ,....., process j

Observed number of adverse

human health outcomes

from microbe B per unit time

Predicted number of adverse

human health outcomes 

from microbe B per unit time

Source 1 Source 2 Source 3 Source n ,.....,

process 1 process 2 ,....., process j
Page 108 Microbial Risk Assessment Guideline

1 that of attribution modeling (Figure 5.2) and calculations tend to follow an input-output 

2 flow from initial microbial levels through each process in the model.

3

4 Process modeling begins with some prediction of microbe occurrence prior to 

5 exposure (Figure 5.3). Processes that occur subsequent to this beginning point influence 

6 the microbial population to grow, maintain, or decline. These processes have been 

7 referred to as transformations and the mathematic effect of these transformations is 

8 sometimes modeled using transformation ratios (Haas et al., 1999). Such ratios simply 

9 describe the level of microorganisms after the process to the level before the process. 

10 You can define 

11

1

amount of microbe after process j, ( )

amount of microbe before process j, ( )

j

j

j

x

x

12 (Eq 5.2)

13

14 and stipulate that this value is reasonably estimated from predictive microbiology. It 

15 might also be invariant to the initial microbe level so that the ratio is constant across all 

16 amounts of the microbe. If you are given an amount of a microbe before the process 

(xj 1), you can calculate the amount of microbe after the process as xj = 17 j xj 1, where 

18 the j subscript could refer to a time step somewhere between the beginning and end of the 

19 model when process j is applied. For example, time step j may be a decontamination step 

20 applied to water, food or something else in which the microorganism occurs. If the 

21 exposure assessment predicts the amount before the decontamination process, then this 

equation predicts the amount after that step. Nevertheless, 22 j is an abstraction of the 

23 complicated interactions that likely exist between microorganisms and their environment. 

24 Sometimes those complex interactions need to be explicitly modeled using more 

25 complicated math and/or simulation. Note that this approach does not incorporate cases 

26 where toxin generation during processing, before exposure is an issue.

27

28 A modular approach to exposure assessment process models has been suggested 

29 for food safety applications (Nauta, 2002). One advantage of subdividing a model is that 

30 complicated components in one module can be summarized into a simpler format before 

31 propagating the output of that module to the next module. For example, a module may 

32 predict microbial growth as a function of storage times and temperatures and other 

33 factors. The resulting prediction may be a continuous distribution describing the 

34 exponential growth of the microbial populations that are input to this module. 

35 Nevertheless, it may only be necessary to consider a small number of discrete levels of 

36 microorganisms in the next module (e.g., by log10 units), so this module‘s continuous 

37 results can be summarized before calculations in the next module are completed.

38

39

40

41

42

43

44
Microbial Risk Assessment Guideline Page 109

Draft – Do not cite, circulate, or quote

1

2

3

process 1

Initial amount 

of microbe B

In Source 1

process 2 ,....., process j

Source 1

exposure

Hazard

characterization

Predicted number of adverse

human health outcomes 

from microbe B per unit time

process 1

Initial amount 

of microbe B

In Source 1

process 2 ,....., process j

Source 1

exposure

Hazard

characterization

Predicted number of adverse

human health outcomes 

from microbe B per unit time

4

5

6 Figure 5.3 Schematic illustrating direction of inference when using a process 

7 modeling approach

8

9 5.2.5 How is Scenario Analysis Used in Exposure Assessment?

10

11 Scenario analysis begins the process of identifying those scenarios that constitute 

12 a risk as well as those that are not a risk. It simplifies the next stage of exposure model 

13 development; quantifying the likelihood of those scenarios that end in an exposure dose. 

14

15 In developing scenarios during the conceptual development phase, you also start 

16 the process of defining the mathematical relationships between steps. Are these additive 

17 or multiplicative? Are there correlations between different inputs? What other factors 

18 influence the relationships among inputs? 

19

20 Compared to scenario development, determining likelihood and dose is usually a 

21 more mathematical exercise. The relationships between model inputs need to be 

22 mathematically described; often statistical methods are employed to quantify these 

23 relationships. Also, the process of converting conceptual relationships into explicit 

24 mathematical relationships commonly involves additional assumptions beyond those 

25 represented in the conceptual model.

26

27 Before data availability or analysis is considered, the mathematical development 

28 needed to determine likelihood and dose should be determined. Once a tentative 

29 mathematical model is completed, its structure can be communicated with outside 

30 reviewers and risk managers. Typically, the mathematics will provoke discussion about 

31 the data needs of the model. A clearly defined mathematical model provides an 

32 opportunity for specialists to review and comment on the course of the project. If 
Page 110 Microbial Risk Assessment Guideline

1 scrutiny of the mathematics determines the need for change, then you can make changes 

2 before a large investment in data acquisition and analysis is completed. Frequent 

3 outreach and feedback is one manifestation of effective exposure assessment.

4

5 An Illustrative Example

6

7 In a very simple illustrative example, you want to predict the exposure risk from 

8 servings containing 10 units of microbe B (Figure 5.4). Two scenarios are considered: 

9 one where the serving is cooked such that all the microorganisms in the serving are 

10 destroyed and another where the serving is not cooked and all the microorganisms 

11 survive. From available data or expert opinion, you determine that the likelihood of 

12 cooking such servings is f, where 0≤ f ≤1. Therefore, the likelihood (l1) of the cooking 

13 scenario (s1) is f which is inseparable from an exposure dose (x1) of zero units of 

14 microorganism B. 

15

10 units of microbe B

in a serving

Cooked

Not cooked

Exposure 1

1

1

1

cooked scenario

0 units of microbe B

s

l f

x

2

2

2

cooked scenario

1

10 units of microbe B

s un

l f

x

Exposure 2 f

1 f

10 units of microbe B

in a serving

Cooked

Not cooked

Exposure 1

1

1

1

cooked scenario

0 units of microbe B

s

l f

x

2

2

2

cooked scenario

1

10 units of microbe B

s un

l f

x

Exposure 2 f

1 f

16

17

18 Figure 5.4 A simple illustrative example of two exposure scenarios resulting from 

19 an initial amount of microorganism B in a serving of food. In this case f is the 

20 probability that a serving will be cooked such that all of microbe B is destroyed. 

21 Conversely, 1-f is the probability that a serving will not be cooked and the total 

22 amount of microorganism B survives to expose a consumer.

23

24 In a slightly more complicated illustrative example, you consider three average 

25 concentrations of microorganism B per serving, three levels of cooking effectiveness, and 

26 three amounts of food or water consumed per serving (Figure 5.5). This example 

27 generates 27 different scenarios for the exposure assessment. 

28

29 Besides the complication of more scenarios, this example also uses a 

30 concentration measure for microorganism levels instead of a fixed number of 

31 microorganisms in a serving. Concentrations are used in many food and water microbial

32 exposure assessments. It is sometimes recommended that modeling microbial

33 concentration be avoided and physical counts of microorganisms should instead be 

34 explicitly tracked (ECSSC, 2003; Nauta, 2005). One justification for such an approach is 

35 that it avoids mass balance mistakes that can occur when processes necessarily change 

36 the microbial numbers. For example, a partitioning of some bulk quantity into subunits 
Microbial Risk Assessment Guideline Page 111

Draft – Do not cite, circulate, or quote

1 necessarily must account for all the organisms that existed in the bulk quantity. Yet, an 

2 average concentration per subunit might result in an incorrect assessment of the exposure 

3 dose per subunit. Nevertheless, for problems without partitioning or mixing processes, 

4 using concentration may be sufficient. 

5

6 You may assume this example pertains to some bulk product in which three 

7 average concentrations (e.g., microorganisms per milliliter or per gram) are possible. 

8 You define each concentration as 1, 2, and 3 and refer to the likelihood of each of 

9 these as f( 1), f( 2), and f( 3). A similar approach is used for cooking effectiveness levels 

10 ( i and g( i)) and consumption amounts ( i and h( i)) to define their values and 

likelihoods33 11 .

12

13

1

Initial

concentration

Cooking

effect.

1

2

3

1

2

3

1

2

3

1

2

3

2

3

Consumption

amount

...

...

1

s

2

s

3

s

4

s

5

s

6

s

7

s

8

s

9

s

Scenario

1 g( ) 2 g( ) 3 g( ) 1 h( ) 2 h( ) 3 h( ) 1 h( ) 2 h( ) 3 h( ) 1 h( ) 2 h( ) 3 h( ) 1

f ( ) 1

Initial

concentration

Cooking

effect.

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

1

2

3

2

3

Consumption

amount

...

...

1

s

2

s

3

s

4

s

5

s

6

s

7

s

8

s

9

s

1

s

2

s

3

s

1

s

2

s

3

s

4

s

5

s

6

s

4

s

5

s

6

s

7

s

8

s

9

s

7

s

8

s

9

s

Scenario

1 g( ) 2 g( ) 3 g( ) 1 h( ) 2 h( ) 3 h( ) 1 h( ) 2 h( ) 3 h( ) 1 h( ) 2 h( ) 3 h( ) 1

f ( )

14

15

16 Figure 5.5 An illustrative example of a slightly more complicated exposure 

17 assessment

18

19 The dose and likelihood for each of the 27 scenarios generated for this exposure 

20 assessment can be calculated directly from the information given. For example, the 

21 average dose for scenario 1 (x1) is the product of concentration ( 1), cooking 

22 effectiveness ( 1) and serving size consumed ( 1). Similarly, the likelihood of scenario 1 

23 (l1) is the joint probability of each of these events occurring; if you know these events are 

24 independent, then l1 (x1, 1, 1, 1) = f ( 1) g( 1) h( 1) . 

25

26 The multiplications for this simple model are common for many process models. 

27 Exposure dose is often the result of multiplicative, input-output, relationships; in this case 

the average dose for a scenario is

i i i i x

28 . The unit of xi is microorganisms per 

 

33 Note that branch likelihoods must sum to one (i.e., 

3

1

( ) 1 ig

) so if we know two of the likelihoods 

then the third is also known (e.g., g( 3) = 1 g( 1) g( 2).
Page 112 Microbial Risk Assessment Guideline

serving; i is microorganisms per volume or mass; I is unitless ratios; and 1 i is volume 

2 or mass per serving. Predictive microbiology sometimes prefers to represent microbe 

3 quantities in log10 units. A logarithmic treatment will convert the multiplicative 

4 calculations of the model to addition or subtraction (e.g., 

log( ) log( ) log( ) log( ) i i i i x

5 ) while the joint likelihood remains the product of the 

6 likelihoods of each variable in the model.

7 

8 Although the results from this example are trivial, it is illustrative to interpret 

9 them. These results explain which scenarios generate the highest doses and the 

10 likelihoods that those doses occur. By summing the likelihoods of scenarios in which 

11 zero organisms occur, the prevalence of non-zero exposures can be determined (i.e., 

1 ( 0) i

i

prevalence f x

12 ). 

13

14 The relative importance of different concentrations, cooking effectiveness levels 

15 and amounts consumed might also be assessed by calculating conditional expected dose 

16 values. For example, the conditional expected dose value for one concentration 

(

E x y |

17 ) is the average dose calculated when only that concentration is considered 

18 but all the values for cooking effectiveness and amounts consumed are still possible. 

19

( ) ( )

|

( )

i j i i

i j

y g h

E x y

f y

20 (Eq 5.3)

21

22 The magnitudes of the conditional expected values suggest the influence of these 

23 inputs on the average exposure dose (Table 5.3). For example, the largest change in 

24 average dose occurs across the possible values for initial concentration, but the smallest 

25 average dose occurs if cooking is completely effective. 

26

27

28
Microbial Risk Assessment Guideline Page 113

Draft – Do not cite, circulate, or quote

1 Table 5.3 Results of a simple exposure assessment

Calculated values Sorted values

i

( )if

i

( )i g i( )i h

scenario

i s

dose, 

i x

likelihood, 

i

l

dose, 

i x

likelihood, 

i

l

1 0.7 0 0.75 10 0.25 1 0 0.131 0 0.750

10 0.2 0.5 0.2 100 0.5 2 0 0.263 5 0.035

50 0.1 1 0.05 150 0.25 3 0 0.131 10 0.009

4 5 0.035 50 0.080

5 50 0.070 75 0.035

Variable Value E x iable y | var 6 75 0.035 100 0.020

i 1 14 7 10 0.009 150 0.009

10 135 8 100 0.018 250 0.005

50 675 9 150 0.009 500 0.021

i 0 0 10 0 0.038 750 0.010

0.5 249 11 0 0.075 1000 0.005

1 462 12 0 0.038 1500 0.003

i 10 12 13 50 0.010 2500 0.010

100 116 14 500 0.020 3750 0.005

150 173 15 750 0.010 5000 0.003

16 100 0.003 7500 0.001

17 1000 0.005

18 1500 0.003

19 0 0.019

20 0 0.038

21 0 0.019

22 250 0.005

23 2500 0.010

24 3750 0.005

25 500 0.001

26 5000 0.003

27 7500 0.001

2 1

3

4 As the previous example illustrates, explicit tree-diagram schematics of exposure 

5 scenarios are daunting when the inputs can assume multiple values and the number of 

6 processes is increased. A full graphic depiction of all 27 scenarios for the previous 

7 example would not fit on a single page. 

8

9 For more complicated exposure models, the inputs are simply treated as random 

10 variables that are mathematically combined. In the simple example, you can represent all 

11 three possible values for initial concentration as where the tilde symbol signifies that 

12 concentration is a random variable. You can similarly define and . 

13

14 Using symbols for the random variables in the model, exposure can be 

mathematically written as 15

x

where the dose delivered to humans is also a 

16 random variable by virtue of the fact that it is a function of random variables. Statistical 

moments of 

x

17 (e.g., its mean and variance) might be predictable, but when the likelihood 

18 distributions for , and involve more than a trivial number of values, other 

techniques (e.g., Monte Carlo simulation) are often used to determine

x

19 . Nevertheless, 

20 the techniques essentially mimic the procedure followed for the simple example; the 
Page 114 Microbial Risk Assessment Guideline

1 possible values for , , and are multiplied together and their joint likelihood is 

2 determined.

3

4 Once an exposure model becomes more complex, the identities of individual 

5 scenarios are difficult to determine. It is common for exposure assessments to focus on 

6 predicting the likelihood of doses without explicitly identifying scenarios. In these 

7 situations, sensitivity analysis is used to sort out the relative influence of model inputs on 

8 the exposure distribution. Nevertheless, it is sometimes crucial to identify the higher risk 

9 scenarios; thinking about the model as a scenario tree is one useful technique for 

10 elucidating those scenarios.

11

12 Random variables can be discrete or continuous. Exposure assessments often use 

13 a mixture of both. Although it is intuitively appealing to consider microbial counts as 

14 discretely distributed random variables, it is not always essential that they be treated as 

15 such. Naturally continuous distributions, like weight measures or measures of 

16 effectiveness, may also be treated as discrete random variables in some models to 

17 simplify their calculations without any loss of information. Like most decisions in 

18 exposure assessment, the choice of distribution is informed by planning and scoping. 

19 Nevertheless, such decisions should be made with an understanding of the biologic 

20 plausibility of the choice.

21

22 Determining the likelihood of doses of microorganisms is the fundamental 

23 objective of most exposure assessments. The key to effective exposure assessment, 

24 therefore, is to explain the mathematical relationships among the random variables that 

25 contribute to exposure. Once you explain and justify the mathematical model, the 

26 process of determining the exposure distribution is relatively straight-forward. 

27 Nevertheless, much of the work of conducting an exposure assessment involves 

28 collection and analysis of available data for the different random variables in the model, 

29 as well as explicit representation of the uncertainty about inputs to the model or the 

30 model itself. 

31

32 5.2.6 What is the Role of Predictive Microbiology in Exposure Assessment?

33

34 The field of predictive microbiology is important to many microbial exposure 

35 assessments. This field is concerned with quantifying the dynamics of microbial

36 populations and these dynamics often depend on environmental and other biologic 

37 factors. Useful discussions on the mathematics and statistics of predictive microbiology 

38 are available (Ross and McMeekin, 1994, 2003; Haas et al., 1999; ECSCC, 2004; Vose, 

39 2008). These references also cite seminal research in this discipline. Within the field of 

40 food safety, the Center of Excellence in Microbiological Modeling provides research and 

products related to pathogen modeling.

34 41

42

43 Functional relationships that describe microbial dynamics are typically of an 

44 input-output form. For example, an exponential growth model is;

 

34 http://ars.usda.gov/Services/docs.htm?docid=8392 
Microbial Risk Assessment Guideline Page 115

Draft – Do not cite, circulate, or quote

1

0

k t N N e t

2

where

Nt

and

N0

3 are the number of microorganisms at times t and zero, respectively, and k

4 is an exponential growth rate constant. Rearrangement of this relationship illustrates 

5 derivation of a transformation ratio for growth;

6

0

Nt k t

e

N

7 (Eq 5.4)

8

9 If the constant, k, in the above equations is a negative value, then the same relationship 

10 can serve to predict attenuation of microorganisms in time unit t.

11

12 The exponential growth rate is only constant for particular environmental 

13 conditions. At a minimum, most exposure assessments will consider environmental 

14 conditions to be variable between scenarios. In such cases, k is some function of 

15 temperature, pH and/or other conditions (i.e., k = f(environmental conditions)). The 

16 reader should refer to the Center of Excellence in Microbiological Modeling or the 

FoodRisk.org35 17 websites for research regarding how various environmental factors – and 

18 microbial strain differences – influence microbial growth and/or attenuation behavior. 

19 Specific guidance for statistical fitting of experimental microbial growth or attenuation 

20 data can be found in these references. 

21

22 Growth or attenuation functions can be deterministic (i.e., one set of parameters 

23 predicts one amount of change) or stochastic (i.e., one set of parameters predicts a 

24 probability distribution for amount of change). In the context of an exposure assessment, 

25 however, the predictions from either a deterministic or stochastic function will be 

26 stochastic because the environmental parameters upon which growth or attenuation 

27 depend are variables. This source of variability relates to human behaviors such as 

28 storage times and temperatures that vary across individuals. 

29

30 Clearly, human behavior can be highly variable; and behaviors such as exposing 

31 raw foods to high temperatures for extended periods of time can dramatically affect the 

32 dose of pathogens ultimately consumed. Although data on microbial growth or 

33 attenuation behavior can be generated readily in experimental laboratories, data on 

34 human food handling behaviors must be collected via well-designed human population 

35 surveys. Such data are available for some commodities, such as ready-to-eat foods (Kosa 

36 et al., 2007; Pouillot et al., 2010), but not necessarily for all food commodities. Although 

37 data regarding refrigeration temperatures may be applicable to most perishable foods, 

38 storage time within the refrigerator may depend on the particular food; this phenomenon 

39 can only be captured via food-specific surveys. In addition, times and temperatures that 

40 foods experience during transport from retail to homes, during food preparation and prior 

41 to (or following partial) consumption are sometimes needed. Actual human behavior 

 

35 http://www.foodrisk.org/resource_types/tools/predictive_micro.cfm
Page 116 Microbial Risk Assessment Guideline

1 data that captures variability in cooking processes is also sometimes important for 

2 estimating the microbial attenuation achieved prior to consumption. 

3

4 You can apply more complex growth and attenuation models in exposure 

5 modeling. For example, the Gompertz equation – or modifications thereof – includes 

6 specific parameters for lag time and asymptotic maximum density (Haas et al., 1999). 

7 Lag time is a characteristic of many growth curves; measured from time zero, it is the 

8 elapsed period before exponential growth begins. The maximum density that a microbe 

9 can attain before competition for nutrients among the microorganisms halts growth is 

10 another modeling characteristic studied by predictive microbiologists.

11

12 Predictive microbiology provides insight and data concerning the behavior of 

13 microorganisms across different environmental conditions. Nevertheless, such insight

14 and data needs to be translated and extrapolated from experimental studies to natural 

15 conditions when applied to exposure assessment. Adjustment of results from controlled 

16 experimental settings to highly variable (and uncertain) natural conditions can be

17 difficult. Therefore, care should be taken when applying predictive microbiology to 

18 exposure assessment.

19

20 Common difficulties for direct application of predictive microbiology to exposure 

21 assessment are: accounting for variable temperatures across time and accounting for the 

22 competitive effects of other microorganisms on the growth characteristics of a target 

23 microbe. Varying temperatures suggest variable transitions between growth, 

24 maintenance and attenuation of microbial populations. Depending on whether growth or 

25 attenuation is a process with memory or is memory-less, the modeling techniques needed 

26 for microbial dynamics will be different (Vose, 2008). The existence of other 

27 microorganisms in media can influence the growth rate or maximum density a target 

28 microbe can achieve, which is termed the Jameson effect (Ross and McMeekin, 2003). 

29

30 ILSI (2010) looked at mechanisms that have an impact on physical distributions, 

31 characteristics of frequency distributions employed to model microbial distributions, and 

32 the impact of both physical and frequency distributions on illness risk and food safety 

33 management criteria. ILSI outlined six mechanisms that can impact the microbial 

34 distribution in a foodstuff: contamination, growth, death, joining, mixing, and 

35 fractionation (ILSI, 2010).

36

37 Although not necessarily a part of predictive microbiology, the processes of 

38 microbial transfer and cross-contamination are also not well researched. There is a need 

39 for further development of modeling approaches for these potentially important 

40 processes. Some default techniques have been suggested for use in food safety 

41 applications (ECSCC, 2003). 

42

43
Microbial Risk Assessment Guideline Page 117

Draft – Do not cite, circulate, or quote

1 5.2.7 How Can I Address Secondary Transmission of Disease in the 

2 Population?

3

4 The approach described heretofore assumes that exposures result directly from the 

5 media of interest (e.g., food or drinking water) and the potential for person-to-person 

6 transmission of disease is not taken into account. Such an approach generally assumes 

7 that multiple or recurring exposures constitute independent events with identical 

8 distributions of contamination (Regli et al., 1991). Furthermore, secondary transmission 

9 and immunity are most often assumed to be negligible in this approach. Nevertheless, 

10 such assumptions may not be valid.

11

12 To more completely assess all possible exposures, it may be necessary to consider 

13 possible secondary transmissions that result from a primary infection. Such an approach

14 commonly requires consideration of a disease transmission model. A variety of models 

15 have been formulated, mathematically analyzed, and applied to infectious diseases 

16 (Hethcote, 2000). Mathematical models of disease transmission have become important 

17 tools that have led to understanding the transmission characteristics of infectious diseases 

18 in communities and better approaches to decreasing the transmission of these diseases

19 (Hethcote, 2000; King et al., 2008; Riley et al., 2003). Modeling infectious disease

20 processes such as person-to-person transmission of infection and immunity requires 

21 dynamic methods where the number of individuals that are assumed to be susceptible to 

22 infection is time-varying and risk is manifest at the population level (Anderson and May, 

23 1991; Hethcote, 1976, 2000).

24

25 Epidemiological disease transmission models stratify a population of potentially 

26 exposed humans into different states according to disease status:

27

28 a) Susceptible

29

30 b) Diseased (infectious and symptomatic)

31

32 c) Immune (partial or complete)

33

34 d) Carrier (infectious but asymptomatic)

35

36 Only a portion of the population is in the susceptible state at any point in time, 

37 and only those individuals in a susceptible state can become infected through exposure to 

38 pathogens. Members of a population may move between model states and model 

39 parameters predict the numbers of people that are in each of the epidemiological states at 

40 any given point in time. Factors affecting the population dynamics include the level and 

41 frequency of exposure, the ability of individuals in infectious states to infect susceptible 

42 individuals, and the temporal processes of the disease (e.g., incubation period, duration of 

43 disease, duration of protective immunity). The rate parameters may be determined 

44 through literature review or through site-specific data. 

45
Page 118 Microbial Risk Assessment Guideline

1 Disease transmission models may also be used to determine the primary exposure. 

2 Such models may be necessary to predict the level and frequency of contaminated media 

3 when little or no empiric evidence is available. For example, assessing exposures that 

4 result from the inadvertent slaughter of a Highly Pathogenic Avian Influenza-infected 

5 U.S. poultry flock requires modeling the epidemic spread of the virus within that flock 

6 (USDA, 2008). This approach is needed because there is no evidence available 

7 concerning the occurrence of this pathogen among U.S. poultry flocks.

8

9 Secondary transmission modeling for airborne microorganisms such as anthrax 

10 and severe acute respiratory syndrome (SARS) virus may be discussed in more detail in a 

11 future volume of this Guideline (Riley et al., 2003; Bartrand et al., 2008).

12

13 5.2.8 What Data Can I Use in an Exposure Assessment?

14

15 Ideally, the data needed for an exposure assessment are determined by its specific 

16 conceptual and mathematic models. If the needs are clearly determined before effort is 

17 expended in collecting and analyzing data, a fuller and more efficient treatment of 

18 relevant data can be accomplished. 

19

20 Exposure assessments data usually stem from either population- or experimental21 based surveys or studies. These data are preferably from published or reviewed research 

22 and are fully relevant and representative of what is needed in the exposure assessment.

23 However, this is not always the case. An extensive discussion of data types and sources 

24 is available (WHO/FAO, 2008).

25

26 The broad categories of data needs for exposure assessment are: microbial, 

27 processes, and exposed humans. Within each of these categories, however, is an array of 

28 data types that may be needed for specific analyses. 

29

30 Data on Microorganisms 

31

32 Data about the occurrence and amount of microorganisms within the medium of 

33 interest (e.g., water, food, air) is important for process modeling. It is desirable to have 

34 occurrence data for multiple points between the beginning of the model and the point of 

35 exposure. 

36

37 Prevalence data provide presence/absence data for the occurrence of a 

38 microorganism in a medium. Such data support estimation of the proportion of some 

population in which the microbe occurs (i.e., 

# 

# 

of units with microbe prevalence

of units in population

39 ) 

40 during some cross-section of time. Observational studies that solely report prevalence 

41 are rarely directly applicable to exposure assessments. Instead, the apparent prevalence 

42 must be adjusted for the probability that units with the microbe would be detected if 

43 actually present (i.e., sensitivity) in order to estimate the true prevalence. Apparent 

44 prevalence is also influenced by the probability that units without the microbe might be 
Microbial Risk Assessment Guideline Page 119

Draft – Do not cite, circulate, or quote

1 incorrectly detected (i.e., 1 – specificity), for example, if there were cross-reactivity in an 

2 immunological assay. 

3

4 Counts of microorganisms in sampling studies are desirable for exposure 

5 assessments. Such data may arise from microbiologic techniques such as direct plating, 

6 observation or most probable number assays (Haas et al., 1999). These data provide an 

7 empirical distribution of the number of samples that contained each count of microbe 

8 observed. Nevertheless, it is important to also know the sensitivity and specificity of the 

9 methods used in count assays to accurately interpret the data.

10

11 A well-designed exposure assessment fully characterizes the microbe (or 

12 microorganisms) on which it is focused. Oftentimes, occurrence data will not be specific 

13 for the target microbe and additional data will be needed to interpret the relevance of the 

14 occurrence data. For example, count data for all Salmonella serotypes on broiler 

15 carcasses would need to be adjusted if the focus of the exposure assessment was 

16 Salmonella enteric Typhimurium. If you knew that the count data were adequately 

17 described by a Poisson distribution and you also estimated that 20% of all Salmonella

18 were Salmonella enteric Typhimurium, then you might simply model the counts of 

19 Salmonella enteric Typhimurium as distributed according to a Poisson (

0.2

) 

20 distribution. Yet, other data may suggest this simple approach does not adequately 

21 account for the clustered occurrence of specific Salmonella serotypes on broiler 

22 carcasses. 

23

24 Process Data

25

26 General processes common to many exposure assessments include growth and 

27 attenuation of the target microbe(s), as well as mixing and partitioning of the medium in 

28 which the microbe occurs (Nauta, 2002). The processes modeled in an exposure 

29 assessment should be informed by the evidence used to construct the conceptual model. 

30

31 Predicted changes in microbial amounts resulting from growth or attenuation 

32 processes may be available from predictive microbiology research. Nevertheless, these 

33 predictions are often functionally dependent on environmental factors. Therefore, you 

34 need data to characterize the variability in parameters such as temperature and time in 

35 order to employ predictive microbiology in an exposure assessment. 

36

37 Partitioning of water, food or air into smaller subunits prior to exposure is a 

38 common problem in exposure assessment that requires industry or ecologic data to solve. 

39 Mass balance considerations may be required to account for recycling or cross40 contamination of microorganisms in some scenarios. Some of these data may come from 

41 industry- or government-sponsored surveillance systems; but sometimes expert 

42 experience will be the only information available. 

43

44
Page 120 Microbial Risk Assessment Guideline

1 Human Characteristics and Behavioral Data

2

3 Demographic and behavior data concerning exposed humans will be specific to 

4 the subpopulations and media considered in the exposure assessment. Much of the data 

5 used to characterize subpopulations will come from routine government surveys. These 

6 surveys provide demographic data by geographic region, age, sex and other factors. The 

7 estimated proportion of the population that represents a susceptible population may be 

8 available from epidemiologic research. Extrapolations from non-representative data may 

9 require substantial modeling and expert judgment to accomplish. 

10

11 Data on human behaviors that influence the exposure assessment will be needed. 

12 Some behaviors, like time and temperature of storage or cooking, are highly variable 

13 among the human population. Some of these behaviors have been the subjects of ongoing research projects.

36 14 These data inform the growth and inactivation processes via 

15 their predictive functional relationship with microbial counts.

16

17 Specific data on some human behaviors that increase the likelihood of exposure to 

18 a particular microbe (e.g., preference for raw meats or seafood, occupational exposure to 

19 microbe rich environments, cohabitation with infected individuals) are sometimes 

20 difficult to locate. Nevertheless, some frequency and contact rates have been 

21 summarized for water and air media (Haas et al., 1999). Similarly, government surveys 

22 can provide data on variability in consumption, inhalation or contact amounts across 

individuals and groups of individuals.

37 23 

24

25 Some well known and frequently used sources of human consumption data 

include CDC‘s National Health and Nutrition Examination Survey (NHANES)38 26 and 

27 USDA‘s Continuing Survey of Food Intakes by Individuals (CSFII), which as of 2002 

28 have been integrated and maintain the name NHANES (Dwyer, 2003). The FoodNet 

Population Survey Atlas of Exposures is also a useful resource.39 29

30

31 5.2.9 How do I Use Data in an Exposure Assessment?

32

33 Data provide evidence about the inputs to the exposure assessment, but data also 

34 influence the magnitude of uncertainty surrounding its results. Weak or absent data are 

35 usually associated with large uncertainties while data that are substantial, relevant and 

36 representative contribute little uncertainty to an exposure assessment.

37

38 Population-based, observational data are commonly used to estimate the 

39 parameters for random variables in exposure assessments. Statistically robust approaches 

40 for selecting appropriate probability distributions, estimating the parameters of those 

41 distributions and comparing alternative distributions are explained elsewhere (Vose, 2008

42 and Haas et al., 1999). Often the process of fitting data to distributions is complicated 

 

36 www.cfsan.fda.gov/~lrd/ab-foodb.html

37 www.ars.usda.gov/main

38 http://www.cdc.gov/nchs/nhanes.htm

39 http://www.cdc.gov/foodnet/surveys/FoodNetExposureAtlas0607_508.pdf
Microbial Risk Assessment Guideline Page 121

Draft – Do not cite, circulate, or quote

1 because the data were generated by imperfect detection systems. Adjustments for 

2 imperfect detection sensitivity and specificity are discussed in those same references. 

3

4 Although there are pros and cons to strict application of either classic statistical 

5 (―frequentist‖) or Bayesian estimation methods, it is often the case that the results of the 

6 two approaches are very similar. Results will tend to differ when the available 

7 dose‐response data are very limited and/or when there is substantial information other 

8 than the numerical dose‐response data that leads to a very informative (i.e., precise) prior.

9

10 Procedures for statistical fitting of data to distributions include appropriate 

11 methods for determining the uncertainty in the estimated distribution parameter(s). This 

12 uncertainty is propagated through the exposure assessment and combined with other 

13 sources of uncertainty to quantify the total uncertainty about the resulting exposure 

14 distribution.

15

16 Data of questionable relevance to the specific exposure assessment require special 

17 consideration. Similarly, data that are not entirely representative of the populations 

18 modeled should be carefully used. It can be argued that data pertaining to one microbe 

19 are also relevant to another. Such  ̳surrogate‘ relationships should only be exploited 

20 when there is an absence of data that is directly relevant. Establishing the credibility of a 

21 surrogate for the target microbe may require a high standard of proof. It is usually 

22 preferred to use directly relevant data and honestly represent its uncertainty than to mix 

23 highly relevant and surrogate data in an attempt to reduce uncertainty.

24

25 Highly representative data are generated from random sampling of all relevant 

26 subpopulations. For example, an exposure assessment pertaining to the United States 

27 would preferably use human behavior data generated from a representative sample of 

28 U.S. persons. Nevertheless, representative data may not be available for some model 

29 inputs. Instead, data from specific regions or other countries may be available. Based on 

30 comparison of other factors it may be concluded that one or more of these other data 

31 sources could be a reasonable substitute. In such cases, you can make an effort to 

32 determine the best substitute and only use its data in the exposure assessment. It is 

33 usually inappropriate to mix multiple sources of less representative data because the 

34 resulting estimates often imply more confidence than is legitimate to claim.

35

36 5.3 How do I Analyze a Model’s Results?

37

38 The general purpose of an exposure assessment is to translate the technical inputs 

39 of a model into a description of the likelihood of exposure doses in some defined part of 

40 the human population. Quantitative assessments estimate numeric values while 

41 qualitative assessments may use ordinal metrics (e.g., high, medium, and low) to signify 

42 the magnitudes of exposures. 

43

44 Quantitative exposure assessment models usually comprise random variables that 

45 consequently generate variability in exposures. Therefore, a common output of the 

46 exposure assessment is the frequency distribution of possible doses that come in contact
Page 122 Microbial Risk Assessment Guideline

1 with the human population of interest. This distribution is an estimate of the actual 

2 variability in exposures that occurs in nature. If the model is calculated based on input 

3 values thought to accurately reflect current conditions, this variability reflects what is 

4 occurring at present. Exposure assessments often refer to such predictions as the baseline 

5 exposure. If the model is calculated based on proposed risk management changes, then 

6 the variability reflects a prediction about the future. 

7

8 It should be noted that the exposure distribution calculated from a model is also 

9 constrained by some unit of time. This time component must be explicit to correctly 

10 interpret and extrapolate the variability predicted by the model. 

11

12 Analysis of the exposure distribution includes determining the sensitivity of this 

13 distribution to changes in the random variables (or other model inputs) used to predict it. 

14 Sensitivity analysis determines which model inputs are the primary drivers or predictors 

15 of substantial doses with relatively high likelihoods of occurring. Conclusions about the 

16 important inputs directly inform risk managers by suggesting what changes in the system 

17 will cause the greatest reduction in exposures. 

18 

19 If all model inputs were perfectly known, then the output of an exposure 

20 assessment might arguably consist of a single exposure distribution. Yet uncertainty 

21 potentially pervades all aspects of an exposure model and the resulting uncertainty about 

22 its predictions can be incorporated into any analysis. Uncertainty analysis is concerned 

23 with determining the influence of the various sources of uncertainty on the predictions of 

24 the exposure assessment. 

25

26 5.3.1 How do I Report Exposure in an Exposure Assessment?

27

28 The natural output of an exposure assessment is an exposure distribution; this 

29 distribution provides likelihood or frequency values for the range of possible doses that 

30 constitute exposures. You should clearly identify the applicable time period and the 

31 exposed human population to which the exposure distribution applies. The most 

32 common format for reporting exposures is tables or graphs.

33

34 An exposure distribution may reflect the possible doses an individual could

35 experience in, for example, one year. The objective of the risk assessment should 

36 determine the type of exposure distribution reported. It could reflect those doses relevant 

37 to a highly sensitive population, life stage, or to the entire population.

38

39 An exposure assessment will convey the variability of doses for the relevant 

40 population per unit time, but it may also include consideration of the variability of the 

41 entire distribution across time. A dynamic exposure assessment that predicts trends in 

42 exposure across time may report the trends based on the statistical expected values of the 

43 individual exposure distributions or it may report the actual distributions for each time 

44 period considered.

45
Microbial Risk Assessment Guideline Page 123

Draft – Do not cite, circulate, or quote

1 It is essential that you communicate the magnitude of uncertainty about the true 

2 exposure distribution. A number of techniques may be used to convey the uncertainty, 

3 but no method is universally applicable and all methods can be computationally intensive

4 (Lammerding and Fazil, 2000; Cullen, 1999). Second-order modeling – in which 

5 variability is derived for one combination of uncertain inputs and the process is repeated 

6 until a full range of plausible combinations is achieved – is a common technique to 

7 accomplish a clear separation of variability from uncertainty (Vose, 2008). Nevertheless, 

8 the complexity of this technique may preclude its widespread application. 

9

10 Multiple exposures to various doses for the same individual can complicate 

11 exposure assessments. It is not uncommon for exposure assessments to assume that 

12 exposure to non-zero doses is an infrequent phenomenon within a population. Therefore, 

13 it is typically assumed that there is a one-to-one correspondence between exposures and 

14 individuals, or at least the same individual is unlikely to be exposed twice within a short 

15 period of time. Nevertheless, in some cases this assumption is not robust, especially 

16 when considering exposures that may be clustered in space and time. Microbial dose17 response models typically describe the likelihood of an adverse outcome as a function of 

18 a single exposure to some dose. If persons are expected to face multiple exposures, then 

19 the process of combining the exposure information with a typical dose-response function 

20 is be different from a standard one-dose one-person approach (Haas et al., 1999).

21

22 5.3.2 How do I Determine a Change in Exposure and Subsequent Risk?

23

24 The purpose of many MRAs is to determine how risk management decisions 

25 might change the risk of an adverse human health outcome. From the perspective of an 

26 exposure assessment, this usually requires calculating the model with and without the 

27 proposed change. For example, risk managers may want to evaluate the effect of some 

28 new mitigation process on the frequency or level of exposure doses. A baseline exposure 

29 prediction is compared to an exposure prediction based on the inclusion of the new 

30 mitigation process. 

31

32 In large exposure models, measuring change in exposure is complicated by the 

33 role of uncertainty in the model‘s predictions. The baseline model predicts an exposure 

34 distribution with its attendant uncertainty. The mitigation model will predict a different 

35 exposure distribution with its own attendant uncertainty. But, the uncertainties between 

36 the two predictions are not independent of each other. It should be clear that the same 

37 things that contribute uncertainty to the baseline predictions usually apply to the 

38 mitigation predictions. Therefore, measuring the change in exposure between the two 

39 predictions is not trivial. Typically, the change you are interested in quantifying is in the 

40 number of adverse human health outcomes. That change cannot be calculated until risk 

41 characterization takes place and the problem of dependent uncertainties is compounded 

42 by the inclusion of dose-response uncertainty.

43

44 A direct method for accurately measuring changes in exposures or adverse human 

45 health outcomes is to model the baseline and mitigation predictions simultaneously using 
Page 124 Microbial Risk Assessment Guideline

1 some parallel modeling structure. This method is computationally daunting, however, 

2 and is not commonly followed. 

3

4 Another approach can provide boundaries for the magnitude of change in 

5 numbers of adverse human health outcomes. In this approach, two uncertainty 

6 distributions about numbers of adverse human health outcomes, generated by separately 

7 calculating the baseline and mitigation predictions, are subtracted from each other 

8 assuming perfect positive correlation and assuming perfect independence. Generic 

9 equations for each approach are:

10

Independence or perfect correlation 

E B M E B E M

11 (Eq 5.5)

12

Independence

Variance B M Variance B Variance M

13 (Eq 5.6)

14

2

Perfect correlation

Variance B M Variance B Variance M Variance B Variance M

15

16 (Eq 5.7)

17

18 These equations provide the expected value and variance of the difference using 

19 the two approaches. It may be appropriate to assume some parametric distribution for 

20 change in adverse human health outcomes; in that case these moments can be used to 

21 determine that distribution‘s parameters. The results assuming independence and perfect 

22 correlation provide boundaries for the more accurate predictions that could be generated 

23 using parallel calculation. In some cases, the bounds may not be sufficiently different to 

24 cause concern but in other cases this analysis may suggest the need to invest in the 

25 development of a parallel model structure.

26

27 5.3.3 What is Sensitivity Analysis?

28

29 Sensitivity analysis examines the relative influence and importance of a model‘s 

30 inputs on its output (see Section 6.7). A well-designed exposure assessment model 

31 should comprise inputs that influence exposures, so the important idea of sensitivity 

32 analysis is measuring the  ̳relative‘ influence. Sensitivity analysis may be completed as 

33 part of an exposure assessment or it may be done as part of risk characterization. 

34

35 There is no universal standard for conducting sensitivity analysis (Frey et al., 

36 2003). In fact, multiple approaches may be legitimately used because each approach may 

37 examine a different type of influence. The typical sensitivity analysis examines the 

magnitude of change in exposures for some change in inputs (i.e.,

output

input

38 ). One 

39 challenge is determining how a change in exposures should be measured. In some cases, 
Microbial Risk Assessment Guideline Page 125

Draft – Do not cite, circulate, or quote

1 change in the average dose value may be satisfactory (e.g., measuring conditional 

2 expected dose). In other cases, the change in the variability of doses may be of interest. 

3 If sensitivity is measured using analysis of variance techniques, then changes in the 

4 average and variance can be assessed together (Frey et al., 2004). Correlation or other 

5 quantitative measures of association (e.g., spider plots, tornado charts) are commonly 

6 available in commercial software packages used for Monte Carlo simulation (Vose, 

7 2008).

8

9 To proceed with sensitivity analysis, it is important that the objective of the 

10 analysis is clear to the analyst. This objective can be informed by the overall purpose of 

11 the risk assessment. Sensitivity analysis for its own sake is rarely rewarding when 

12 applied to complex models and often a frustratingly inefficient use of the analyst‘s 

13 resources. For a focused exposure assessment, with specific purposes and scope, it is 

14 likely that focused analysis on specific components‘ importance and influence (examined 

15 in multiple ways) may be more useful for risk managers.

16

17 The major challenge for sensitivity analysis is that it is difficult to separate 

18 sensitivity from uncertainty in most exposure assessment models. Uncertainty about 

19 model components can result in a very mixed description of the importance of model 

20 components. Theoretically, a model input may be highly influential across part of its 

21 uncertainty range but much less influential across another part. At the least, 

22 acknowledgement of such discrepancies should be communicated to risk managers. 

23 Again, focused sensitivity analysis facilitates deeper analysis of a few components 

24 instead of superficial analysis of many components.

25

26 5.3.4 What is an Uncertainty Analysis?

27

28 The goal of uncertainty analysis is to identify those model inputs whose 

29 uncertainty substantially contributes to the total uncertainty about exposures implied by 

30 the exposure model (see Section 6.7). Uncertainty is a lack of knowledge; therefore, 

31 uncertainty can only be reduced via accumulation of new knowledge. Uncertainty 

32 analysis suggests where to focus future data gathering efforts and/or scientific research. 

33 Like sensitivity analysis, there is no standard method for conducting uncertainty analysis.

34

35 Although the objective of uncertainty analysis differs from sensitivity analysis, 

36 the results of uncertainty analysis are usually not independent of the results of sensitivity 

37 analysis. If uncertainty about an input contributes substantial uncertainty about the 

38 model‘s results, then that input usually is also likely to be identified as highly influential 

39 through sensitivity analysis.

40

41 Random variables and parameters in an exposure model are subject to parameter 

42 uncertainty (Morgan and Henrion, 1990). This source of uncertainty refers to sampling 

43 and measurement errors that are inherent to empirical data. Statistical techniques may be 

44 available to quantify parameter uncertainty.

45
Page 126 Microbial Risk Assessment Guideline

1 Uncertainty about model structure is another source of uncertainty that may be 

2 propagated through an exposure assessment. This source may refer to use of surrogate 

3 variables, model simplifications or alternative specifications of the model processes. An 

4 example of the latter reference might be uncertainty about whether to include a process 

5 (e.g., cross-contamination) in an exposure model. It is often difficult to quantify the 

6 magnitude of uncertainty about model structure. Nevertheless, such uncertainty can 

7 substantially alter exposure predictions from a model. Uncertainty associated with model 

8 specification can be investigated by testing the differences in fit and predictions of 

9 multiple model forms. If the models are of the same general form (e.g., exponential 

10 family), then the effects of including or excluding covariates can be evaluated using 

11 likelihood-based criteria such as the Akaike information criterion. 

12

13 If economic information is combined with uncertainty analysis, a value of 

14 information analysis approach may yield insights relevant to the goal of uncertainty 

15 analysis. Yet, these methods have rarely been employed in microbial exposure 

16 assessment. For a simple dichotomous decision, value of information methods can 

17 evaluate the economic returns of hypothetical new information relative to the choice 

18 made prior to acquiring the new data. These methods highlight an important point about 

19 new information; if additional data will not change a decision, then those data are not 

20 valuable. Therefore, risk managers can acknowledge that their decisions hinge on the 

21 degree of uncertainty about results from a model. Newly acquired data will presumably 

22 reduce that uncertainty, but its value may be naught if risk managers were not influenced 

23 by the magnitude of uncertainty about the original results.

24

25 The techniques for uncertainty analysis are similar to those for sensitivity 

26 analysis. A factorial design for uncertainty analysis enables evaluation of how alternative 

27 values for uncertain model inputs influence the model results. For example, if there are k 

28 uncertain inputs and two realistically extreme values (i.e., high and low) for each input 

are proposed, then the model can be calculated 2

k

29 times to examine the influence of each 

30 input‘s uncertainty on the results (ECSSC, 2003). This approach supports examining 

31 interactions between the inputs. For example, one extreme value of one input may 

32 slightly influence the model‘s results, but when the model is calculated with that extreme 

33 value and certain other extreme values of other inputs, the magnitude of its influence 

34 substantially increases.

35

36 Analysis of uncertainty is daunting if the number of uncertain parameters or 

alternative model structures is large. A simple, univariate alternative to the 2

k

37 factorial 

38 design is to calculate results for each of the two extreme values of each input 

39 independently; this only requires 2k re-calculations of the model (ECSCC, 2003). The 

40 predicted change in results for each input can be graphically displayed to demonstrate 

41 their relative influence on the model‘s results.

42
Microbial Risk Assessment Guideline Page 127

Draft – Do not cite, circulate, or quote

1 5.4 What Can I Put Into an Exposure Assessment Report?

2

3 Communication of exposure assessment results is challenging because the 

4 intended audience is usually diverse. A presentation of these results has to satisfy 

5 technical specialists as well as those who are less specialized in quantitative methods 

6 (ECSCC 2003, WHO/FAO, 2008). Consequently, a balance between technical details 

7 and general conclusions of the analysis is a goal of any communication (see Chapter 8).

8

9 The credibility of an exposure assessment is enhanced through peer-review and 

10 feedback from public outreach. If communication of the exposure assessment is 

11 reasonably transparent and balanced, reviewers can focus their attention on the merits of 

12 the analysis and contribute to improving its accuracy and validity.

13

14 The output of an exposure assessment is usually an exposure distribution. 

15 Thoughtful contemplation of the best formats for communicating this distribution and its 

16 uncertainty should precede any final decision. A number of graphical formats for 

17 presenting risk results are available (Vose, 2008). Tabulated results can be more useful 

18 in some cases and both table and figures may be needed in other cases. A limited number 

19 of moments (e.g., mean, variance, skewness) along with meaningful quantiles of this 

20 output are routinely provided in a report. You can consider different preferences for data 

21 display when selecting the best format. All tables, graphs, and other figures should have 

22 clear, concise narrative text to help the reader understand what is being presented.

23

24 If specific processes included in the exposure assessment are the subjects of risk 

25 management choices, then you should summarize outputs from these processes in the 

26 report. Plots of the central tendencies of microbe counts that illustrate trends across the 

27 breadth of a model can be useful for some readers.

28

29 To clearly communicate the scenarios considered in the exposure assessment, you 

30 can illustrate diagrammatically the conceptual model in the report. A well-annotated 

31 conceptual model will enhance reader‘s understanding of the analytic approach taken. 

32

33 You can list all inputs used in the model and clearly define their reference symbol 

34 and name, describe their purpose in the model, provide the probability distribution name 

35 and parameter values for random variables, and explain how you handled uncertainty for 

36 variables and parameters. You can communicate important inputs by using graphical 

37 depictions of their distributions.

38

39 Include the mathematical structure of the model transparently, but concisely, in 

40 the report. Because it will only be useful to a specialized audience, it is usually 

41 appropriate to place the mathematics in an appendix. Nevertheless, this mathematical 

42 description of the model will provide the greatest opportunity for clearly explaining the 

43 exposure assessment to the specialized audience. This is the audience that can verify the 

44 validity of the model or identify errors in its logic or assumptions. 

45
Page 128 Microbial Risk Assessment Guideline

1 Inclusion of a discussion of important assumptions is necessary. A transparent 

2 treatment of assumptions improves the reader‘s understanding of the analysis; although 

3 some might disagree with assumptions made, it is much preferred that they understand 

4 the reasons for the assumptions. Models are always imperfect depictions of reality and 

5 their results are always conditioned on assumptions. The strength of a model, then, is 

6 based on the strength of the justifications of its assumptions.

7

8 Presentation of the results of sensitivity and uncertainty analyses in tabular or 

9 graphical formats is also helpful and may be required depending on the use of the risk 

10 assessment. It is crucial that such presentations are meaningful to both a specialized and 

11 non-specialized audience. Complex analyses that do not illuminate important 

12 conclusions will create confusion for the reader. 

13

14 Exposure assessments can generate large volumes of analytic output. 

15 Nevertheless, presentation of the exposure assessment requires deliberation by the 

16 analyst; the reader of a report expects that care is taken in what is presented and how it is 

17 presented. Avoid arbitrary decisions about the content of the report.

18

19 5.5 What are Possible Future Developments in Exposure Assessment?

20

21 The discipline of microbial exposure assessment continues to grow and evolve. 

22 Compared to more established analytic fields like economics, epidemiology or statistics, 

23 microbial exposure assessment is still relatively new. Therefore, it is expected that 

24 methods and approaches will continue to improve.

25

26 One technique of increasing interest for exposure assessment is the use of MCMC 

27 simulation (Gelman et al., 2004). The MCMC method is based on Bayes Theorem, but it 

28 is designed to solve problems that would normally be intractable using standard Monte 

29 Carlo approaches. Using MCMC methods, ―prior distributions‖ are specified for model 

30 inputs and the empiric evidence is integrated with the model to determine which 

31 combinations of model inputs best describe—in a probabilistic sense—the empiric 

32 evidence. 

33

34 Many exposure assessments are generated to guide government policy 

35 development. Traditionally, risk assessments are conducted independent of economic 

36 analyses, but risk assessment products are commonly incorporated into economic 

37 analyses conducted to support government policy (Williams and Thompson, 2004). In 

38 the future, it is expected that economic analysis will be integrated into the risk assessment 

39 to provide more accurate and useful policy information. 

40

41 Within exposure assessment models, costs and tradeoffs between alternative 

42 processes could be incorporated so that their effects on the model‘s results were explicitly 

43 calculated. For example, it might be the case that a process with variable effectiveness is 

44 associated with extraordinarily high marginal costs at very high levels of effectiveness, 

45 but low or absent costs at moderate or low levels of effectiveness. To fully appreciate 
Microbial Risk Assessment Guideline Page 129

Draft – Do not cite, circulate, or quote

1 such phenomena, the exposure model needs to incorporate the economic factors. 

2 Otherwise the effects of these economic factors may not be understood by risk managers.

3

4 As mentioned for uncertainty analysis, VOI methods are very useful for 

5 determining economically-efficient future data gathering efforts (Yakota and Thompson, 

6 2003; Disney and Peters, 2003). In the future, it is expected that these methods will 

7 become standard for exposure and risk assessment analyses. 

8

9 Exposure assessment models can serve as templates for the incorporation of better 

10 data when it becomes available. In the future, exposure assessments may be updated with 

11 data collected explicitly for the purpose of exposure assessment. Traditionally, exposure 

12 assessments use data generated for other purposes. Risk assessors then must struggle 

13 with incorporating such data into an exposure assessment. If data are collected for an 

14 exposure assessment, however, it is likely that the data will be better structured for that 

15 purpose. The result of using data that is ―fit for purpose‖ is improved exposure estimates 

16 for risk management decisions.

17

18 MRA deals with many different types of problems; there are a number of 

19 approaches and methods that may provide satisfactory solutions. Nevertheless, exposure 

20 assessment will continue to evolve towards standard approaches and methods that 

21 represent accepted defaults for certain types of problems. Increasing academic attention 

22 and scrutiny of exposure assessment will almost certainly bring greater consistency in 

23 methods. Because the field of MRA is still emerging, it is also critical that risk assessors 

24 search for reasonable precedents in methodology when embarking on a new exposure 

25 assessment project. If precedents are adopted and improved, standardization of methods 

26 and models can eventually occur.

27

28 International groups such as the WHO/FAO and Codex Alimentarius will 

29 continue to provoke thoughtful discussions about the most appropriate exposure 

30 assessment methods and approaches. Such discussions will contribute to improved 

31 standardization of exposure assessments in the future.

32
Page 130 Microbial Risk Assessment Guideline

1 6. RISK CHARACTERIZATION

2

3 As introduced in Section 1.1, risk characterization is one of the four fundamental 

4 components of risk assessment. Risk characterization is the integrating component of the 

5 risk assessment process that ―characterizes‖ or describes and summarizes microbial 

6 health risks. It is the final integrative step of risk assessment. This chapter addresses 

7 what a risk assessor can do to take the pertinent elements from the previous components 

8 of the risk assessment (hazard identification, hazard characterization, dose-response, and 

9 exposure assessment) and integrate them into a coherent, understandable, and informative 

10 conclusion that is useful for decision makers and stakeholders. Further, the risk 

11 characterization discusses scenario, model, parameter, data, and analysis options that risk 

12 managers should understand and consider when interpreting the results of the risk 

13 assessment. The risk characterization, in a sense, brings to full circle the initial planning

14 and scoping for the risk assessment described in Chapter 2. The content of the risk 

15 characterization is intended to reflect the issues and questions laid out during planning 

16 and scoping.

17

18 For further detail and discussion on risk characterization, good references are the 

19 NRC reports (NRC, 1983, 1994, 1996, 2009), EPA‘s Risk Characterization Handbook 

20 (EPA, 2000a), and An Examination of EPA Risk Assessment Principles and Practices

21 (EPA, 2004b). This chapter is not intended to provide the level of detail about risk 

22 characterization provided in the previous references, but rather provides the microbial

23 risk assessor with guidance on what information to include and how to integrate the 

24 information from the previous three chapters.

25

26 6.1 What is Risk Characterization?

27

28 In its most general sense, risk is the possibility (and if estimated, probability) of 

29 suffering harm. For the purposes of MRA, hazard may be causal or associated with 

30 adverse outcome as a representation of intrinsic effects expressed by a microbe. Risk 

31 contains elements of both hazard and exposure. Thus, risk is generally understood to be 

32 the integration of intrinsic effects, represented by Hazard, and the values for Exposure. It 

33 is usually represented by some form of the equation:

34

35 Risk = ƒHazard • ƒExposure (Eq. 6.1)

36

37 Hazard identification allows you to select and focus on specific features of subject 

38 organisms associated with the potential to cause harm. Exposure analysis provides a 

39 description of the routes and an estimate of the degree to which a host may be exposed. 

40 None of these components can stand alone to characterize risk. When combined with 

41 host factors in an evaluation of dose-response, one can get a quantitative Hazard 

42 Characterization of that potential once a host is exposed. Risk characterization takes the 

43 specific identified hazards, examines the probabilities of their existence under specific 

44 exposure scenarios, and combines those probabilities with those of the likelihood that the 

45 agent will encounter the host in sufficient quantity to cause an effect. 

46
Microbial Risk Assessment Guideline Page 131

Draft – Do not cite, circulate, or quote

1 Risk characterization is the final step of the MRA process in which all preceding 

2 data collection and analyses are combined to convey the overall conclusions about 

3 potential risk to humans. During risk characterization, the results of the iterative risk 

4 assessment process are integrated and documented in a descriptive risk characterization 

5 summary. Risk characterization communicates the key findings and the strengths and 

6 weaknesses of the assessment through a conscious and deliberate transparent effort to 

7 bring all the important considerations about risk into an integrated analysis by being 

8 objective, transparent, clear, consistent, and reasonable (OMB, 2007b; EPA 2002b; EPA 

9 2000a). For these reasons, the risk characterization needs to be complete, informative, 

10 and useful for decision-makers. Therefore, this section of the risk assessment needs to be 

11 both sufficiently technical to be scientifically accurate, taking into consideration the 

12 uncertainties and reporting the assumptions but also comprehensible by an educated lay 

13 audience. As noted below this is the component that most directly leads to a 

14 regulatory/management decision and serves as communication tool for stakeholders. 

15

16 Risk characterization describes the ways in which exposure and dose-response 

17 (quantitative) or exposure and hazard assessments (qualitative) are used together to 

18 formulate a statement of risk. Risk characterization can be quantitative, when values are 

19 available for all terms in the risk equation, or it may be semiquantitative, when only some 

20 values are available. In many cases, default values/assumptions based on known 

21 conditions are used in place of measured ones. Further, when the data are not useful to 

22 present a quantitative estimation of risk, then a qualitative description of the risk may be 

23 all that can be presented in a risk characterization, which may be sufficient in certain 

24 cases. Regardless of quantitative versus qualitative, the risk characterization should 

25 address the risk management questions posed in the planning and scoping phase (and any 

26 questions that may have been added or revised during the assessment itself).

27

28 Moreover, risk characterization forms the starting point for formulating risk 

29 management considerations and provides a foundation for (regulatory) decision-making. 

30 It characterizes both quantitative data and qualitative information in technical and non31 technical terms, explaining the extent and weight of evidence, results, and major points of 

32 interpretation and rationale. It also summarizes the strengths and weaknesses of the 

33 evidence, conclusions, uncertainties, variability, potential impact of alternative 

34 assumptions, and discusses scenario, model, parameter, and analysis options that may 

35 deserve further consideration as the results from the assessment are subsequently used for 

36 decision making purposes.

37

38 As an example of how one agency views risk characterization, EPA‘s Policy 

39 Statement on risk characterization is as follows: 

40

41 Each risk assessment prepared in support of decision-making at EPA 

42 should include a risk characterization that follows the principles and 

43 reflects the values outlined in this policy. A risk characterization should 

44 be prepared in a manner that is clear, transparent, reasonable and 

45 consistent with other risk characterizations of similar scope prepared 

46 across programs in the Agency. Further, discussion of risk in all EPA 
Page 132 Microbial Risk Assessment Guideline

1 reports, presentations, decision packages, and other documents should be 

2 substantively consistent with the risk characterization. The nature of the 

3 risk characterization will depend upon the information available, the 

4 regulatory application of the risk information, and the resources 

5 (including time) available. In all cases, however, the assessment should 

6 identify and discuss all the major issues associated with determining the 

7 nature and extent of the risk and provide commentary on any constraints 

8 limiting fuller exposition. (EPA, 2000a)

9

10 6.2 What are the Elements in a Risk Characterization?

11

12 During the risk assessment process, you should have identified areas where policy 

13 options were considered, where management decisions and assumptions were made, and 

14 where uncertainties are important. The point of risk characterization is not to reiterate the 

15 details of each chapter of the risk assessment, but rather to integrate those chapters to 

16 arrive at the risk assessment output (e.g. risk estimate, risk ranking, or other output), 

17 describe the relevant findings, cross-reference the exposure and dose-response 

18 assumptions (e.g., do the age grouping in exposure assessment and dose-response 

19 assessment match), and discuss other salient elements (as described below). 

20

21 Risk characterization consists of two principal steps—risk estimation and risk 

22 description. Risk estimation is the compilation of the types and magnitude of effects 

23 anticipated from exposure to the microbe or medium and can be qualitative or 

24 quantitative depending on the data and methods used. The risk estimation is derived 

25 from the output components of the risk assessment (hazard identification, hazard 

26 characterization, exposure assessment, and dose-response analysis). Specifically, the 

27 results from the characterization of exposure can be expressed as the number of 

28 organisms to which an individual is exposed in a defined amount of time and/or for a 

29 certain consumption rate. Resultant estimates of the potential for adverse human health 

30 effects can be expressed as an individual risk estimate (e.g., 1 per 1000 probability of 

31 illness) or as a population level risk estimate (100 illnesses per year in a region with a 

32 population of 100,000 individuals). As described in further detail below, the risk 

33 estimation can also be modeled to consider time-dependent elements such as secondary 

34 (person-to-person) transmission, host immunity, and multiple routes of exposure (ILSI, 

35 2000). 

36

37 The second component of risk characterization, risk description puts the risk 

38 estimation into context by summarizing the event of interest according to its nature, 

39 severity, and consequences and discussing and quantifying (to the extent possible) (1) the 

40 uncertainties associated with the key components within the risk characterization; (2) the 

41 variability associated with key inputs to the model(s); (3) the confidence in the resulting 

42 risk estimates through a weight of evidence discussion; (4) the limitations of the analysis; 

43 (5) the critical assumptions; and (6) the plausibility of the results. Clearly, your 

44 professional judgment will also be necessary to determine what should be included in the 

45 risk characterization. You can consider the following elements in risk characterization 

46 (adapted from the EPA Risk Characterization Handbook, EPA 2000a):
Microbial Risk Assessment Guideline Page 133

Draft – Do not cite, circulate, or quote

1

2 a) Key information – Consider 1) the studies available and how robust they are; 2)

3 the major risk estimates you calculated, the assumptions and the extrapolations 

4 made during the estimated risk calculation, and the residual uncertainties; 3) the 

5 use of default parameter values, policy choices and risk management decisions 

6 made, if any; 4) whether the key data used for the assessment are considered 

7 experimental, state-of-the art, or generally accepted scientific knowledge; and 5)

8 variability.

9

10 b) Context – Consider 1) how you are addressing the risk management questions; 2)

11 how the estimated risk from this microbial hazard compares to other estimates for 

12 this hazard, if available. Include discussion of regulatory requirements or if there 

13 are regulatory values to consider.

14

15 c) Sensitive Populations – Consider 1) the range of people that may be affected, 

16 including innately susceptible populations (e.g., ethnic groups, gender, 

17 socioeconomic and/or nutritional status, other genetic predisposition) and those 

18 that are highly exposed; 2) a quantitative characterization for each sensitive 

19 population may not be necessary or possible. For example, where there are many 

20 sensitive population groups for a given microbial hazard, it may be sufficient to 

21 estimate risks for the most sensitive group, and then assume that as long as that 

22 group is protected, other groups may be protected adequately. If the quantitative 

23 portion of the risk assessment is strongest for the general population due to data 

24 availability, then some data-based adjustment for sensitive populations may be 

25 considered. Both results can be presented and discussed.

26

27 d) Life Stages – Consider the age groupings evaluated and note any life stages that 

28 may have particular vulnerability due to behaviors or situations that influence 

29 exposure patterns and/or innate susceptibilities. For microbial hazards with only 

30 short term effects the different life stage may be treated as sensitive populations. 

31 However, if any longer term effects (e.g., health endpoints that span a 70 year 

32 life) are of interest then life stages may need to be considered differently than 

33 sensitive populations. For example, everyone in the general population passes 

34 through childhood life stages and exposures to pathogens could vary at different 

35 childhood life stages. Thus, depending on the scope of the risk assessment, 

36 consideration of childhood life stages may be necessary as risk estimates may 

37 vary for the range of important life stages.

38

39 e) Scientific Assumptions – Describe, 1) where key data gaps exist; 2) what the key 

40 assumptions you used during your assessment are and the impact they may have 

41 on the assessment outcome. Also note if there is precedent in other risk 

42 assessments for the approach or assumptions employed, and note the justifications

43 for selection of any default parameter values that are used.

44

45 f) Policy Choices – Describe, 1) if your office has different policies about how to 

46 assess risk (e.g., different uncertainty factors or different levels of regulatory 
Page 134 Microbial Risk Assessment Guideline

1 concern); 2) any policy choices that may bound the scope of the assessment. If 

2 appropriate, include discussion of consistency with other Agency approaches or 

3 decisions.

4

5 g) Variability – Describe 1) how variability arises from true heterogeneity in 

6 characteristics such as dose-response differences within a population, or 

7 differences in microbial levels in the environment; 2) if the values of some 

8 variables used in an assessment can change with time and space, or across the 

9 population whose exposure is being estimated. The discussion of variability 

10 should link to the discussion of assumptions, because variability considerations 

11 may be lost or assumed when values for parameters are selected. This element is 

12 critical and discussed at length in the previous chapters. An inadequate 

13 discussion of variability can result in a loss of transparency or in the worse case a 

14 misleading risk assessment.

15

16 h) Uncertainty – Describe 1) what uncertainties exist in the assessment (e.g., 

17 measurement uncertainty, model uncertainty, uncertainty due to data gaps); 2)

18 how you addressed uncertainty (e.g., uncertainty analysis, sensitivity analysis); 3)

19 what impact reducing scientific uncertainties could have on your assessment; 4)

20 where possible, quantitative uncertainty analyses of the data should be presented; 

21 at a minimum, a qualitative discussion of the important uncertainties should be 

22 provided. This element is critical and discussed at length in the previous chapters 

23 (additional references: Morgan and Henrion, 1990; Frey et al., 2004). As with 

24 variability, an inadequate discussion of uncertainty can result in a loss of 

25 transparency or in the worse case a misleading risk assessment. 

26

27 In practice it is often difficult to separate variability and uncertainty, because 

28 many uncertainties are in the area of characterization of variability (e.g. lack of 

29 data for certain conditions). In cases where there is uncertainty about variability, 

30 be sure to be clear about your use of the two terms.

31

32 i) Bias and Perspective – Consider 1) that in the light of uncertainty and default 

33 choices, your agency may proceed in the direction of more public health 

34 protection compared to less protection. If this is the case, this perspective needs 

35 explicit explanation; 2) it is not always clear where agency/public health bias 

36 enters into your agency risk assessment - to the extent it may make a difference in 

37 the outcome of your assessment, highlight the potential bias so the impact will not 

38 be overlooked or misinterpreted by the risk manager. For example explain the 

implications of selecting a 50th versus 95th 39 percentile in a data set.

40

41 j) Strengths and Weaknesses – Throughout the risk characterization you should 

42 highlight and/or describe 1) major imbalances among the components of the 

43 assessment, e.g., the case for the microbe posing a hazard may be strong, while 

44 the overall assessment of risk is weak because there are no data about whether 

45 there is exposure to the microbe; 2) the data/information that present the strongest 

46 evidence for your conclusions as well as the data that may not be as strong 
Microbial Risk Assessment Guideline Page 135

Draft – Do not cite, circulate, or quote

1 (weakness) and how you incorporated that into your conclusions. You should 

2 also discuss the quality of the data used and how the data quality pertains to 

3 variability and uncertainty.

4

5 k) Key Conclusions – Describe 1) the key points that need to be communicated for 

6 knowledgeable interpretation of the risk assessment; 2) that small subset of key 

7 findings supporting information (strengths and weaknesses, results from 

8 sensitivity/uncertainty analyses) that really makes a difference in the assessment 

9 outcome.

10

11 l) Alternatives Considered – Consider 1) if there are plausible alternatives to the 

12 risk estimated in your assessment and how you dealt with those alternatives (e.g., 

13 alternative models that could be used, different hazard pathways); 2) the 

14 limitations of making comparisons among the alternatives; 3) where appropriate, 

15 how your conclusion about risk compares to other possible risks. If other risks are 

16 compared, the discussion should highlight the limitations of such comparisons as 

17 well as the relevance of the comparisons

18

19 m) Research Needs – Describe the key data needs and/or methodology gaps that 

20 were identified during the course of the risk assessment. 

21

22 Each element described above is important to address in a risk characterization; however, 

23 no single element is necessarily more ―critical‖ than others. As the risk assessor, you 

24 need to be aware of all these elements and address them appropriately in the risk 

25 characterization. For each element you considered describe the data you found, or if you 

26 found no data or information for a particular element, or you used a default assumption

27 that also needs to be stated.

28

29 6.3 How Do I Prepare a Risk Characterization?

30

31 Because the purposes of MRA vary, the mechanisms of risk characterization, 

32 including risk integration, can also be quite varied. Some organizations prefer to attempt 

33 separate hazard and exposure characterizations, and combine the results once all the 

34 pieces are described and quantified. Others find that it is valuable to have feedback from 

35 the major assessment components throughout the assessment timeframe, and thus have 

36 some elements of risk integration running simultaneously with the component 

37 assessments. Hybrids of these approaches are possible, as well. Your selection of an 

38 approach to characterization and integration may be driven by institutional practices, 

39 completeness of data, and/or timing of availability of data.

40

41 It is common to have an iterative process in which a simple scoping 

42 characterization (e.g., risk profile) is generated early in the process. This allows the 

43 assessors to identify data gaps, to recommend additional data generation or gathering, or 

44 to use default assumptions as a starting point to evaluate the potential implications of 

45 scenario being evaluated. Later iterations may adjust the scope of the characterization by 

46 focusing in on the most important hazards or exposures identified in the initial iterations, 
Page 136 Microbial Risk Assessment Guideline

1 or expand as needed when additional hazard or exposure elements are identified or 

2 additional data are provided.

3

4 As discussed in Section 2.7 and Chapter 7, there are many different levels of 

5 decisions that need to be made during risk assessment. As the assessor, you take 

6 responsibility for decisions involving scientific judgment. Other decisions that are 

7 ultimately policy calls, are informed by science, but will most likely be made by risk 

8 managers. Risk managers may want to evaluate policy decisions and make revisions 

9 during different iterations of the risk assessment. Any changes in policy decisions or 

10 scope can be tracked for transparency. It is unlikely that scientific judgments will change 

11 unless more data become available or compelling scientific arguments are made during 

12 internal or external peer review. It is not unusual for risk managers to adjust policy 

13 decisions or refine the scope or questions for the risk assessment during the risk 

14 assessment process. In many ways, this process highlights the importance of the iterative 

15 nature of risk assessment. You should be prepared for these types of changes and provide 

16 clear documentation to help delineate scientific judgment from policy decisions. Very 

17 often something that could be scientific judgment if data were available becomes a policy 

18 decision when data are lacking. Determining the threshold between those two situations 

19 can be unclear.

20

21 Some of the driving forces for this level of variation in performing risk 

22 characterization are available resources, time constraints, legal requirements, and Agency

23 culture. In some cases, a first attempt at a complete risk characterization, in the sense 

24 that all evaluation elements are accounted for to at least a limited extent, is desirable to 

25 provide managers with an initial scoping of likely outcomes before all information has 

26 been fully evaluated. Sometimes, certain factors are given precedence at the outset of an 

27 assessment, such as a particular exposure scenario, and interchange between the hazard 

28 and exposure components can take place throughout the course of an assessment. This is 

29 generally the case when examination of a specific health effect or exposure model is the 

30 driving force behind the risk assessment. In other cases, it may be necessary to isolate 

31 the hazard and exposure components to avoid results from one having an undue influence 

32 on the assessment of the other. This may happen if the type of assessment is intentionally 

33 broad and the desire is to not eliminate scenarios, or emphasize specific effects until a 

34 first pass is made. This can result in an iterative assessment, with feedback among the 

35 components occurring during later iterations.

36

37 In selecting an approach for risk characterization, be careful to ensure that any 

38 simplifying assumptions that are employed are in fact appropriate and clearly identified. 

39 Within that context, and to the extent possible, you should demand higher quality input 

40 data and fewer simplifying assumptions when seeking increased accuracy and precision

41 from your risk assessment. From a modeling perspective, biological ―realism‖ is often 

42 counter-balanced by analytical or computational complexity. The increase in the 

43 complexity of a model structure can increase variability and/or uncertainty due to 

44 increased needs associated with model specification (EPA, 2004c). On the other hand, a 

45 simpler model involves implicit or explicit assumptions that may or may not be realistic 

46 or appropriate for a particular situation. More complex models should be considered or 
Microbial Risk Assessment Guideline Page 137

Draft – Do not cite, circulate, or quote

1 used under conditions in which the added complexity may provide sufficient additional 

2 insight that the additional complexity is warranted (King et al., 2008; Soller and 

3 Eisenberg, 2008). As discussed in Chapter 5, statistical methods, such as use of the 

4 Akaike Information Criterion or similar likelihood-based measures, to judge the desirable 

5 level of complexity in statistical models. Representative MRA model forms are 

6 discussed in Section 6.5.1.

7

8 6.4 Are All Risk Characterizations Quantitative, and What Do I Do When 

9 Quantitative Data are Unavailable for Some Elements of the Risk 

10 Characterization?

11

12 While many assessments are quantitative, risk characterization can also be 

13 qualitative and/or descriptive. It is also possible that some parts of the analysis can be 

14 quantitative, while only approximations and/or defaults are possible for other elements 

15 (this is sometime referred to as a semi-quantitative analysis). 

16

17 Although ―risk‖ is often thought to imply a probability of an adverse health effect, 

18 and therefore thought of as quantitative, some assessments can only be expected to 

19 approximate risk probabilities. In certain cases, the risk characterization can at best be 

20 thought of as a screening exercise, providing only a sense of whether the risk might be 

21 judged high, medium, or low. Sometimes one can obtain values for some of the 

22 components of hazard and exposure characterization, but not all. In these cases, it may 

23 be possible to utilize default values for the missing data elements to provide enough 

24 information to establish limits of risk for the organism and conditions in question. A 

25 bounding analysis may be appropriate. For example, you can do a deterministic analysis 

26 with plausibly conservative values for the unknown parameters and see if the resultant 

27 risk is above the level of concern. If it is not above the level of concern no further data 

28 on these parameters is probably necessary. If the risk is above the level of concern, then 

29 you have found out that gathering more data on the unknown parameter value(s) would 

30 be justified.

31

32 Finally, another type of assessment that can be qualitative is a relative risk 

33 assessment. These types of assessment have been valuable for numerous agencies, 

34 especially for evaluating the potential benefits of management actions (treatments) or 

35 alternatives in conditions where rigorous and quantitative data were not available.

36

37 6.5 Are There Different Forms of Risk Characterization? When Do I Apply 

38 Them?

39

40 Risk characterization should be complete, informative, and useful for decision41 makers. The appropriate level of detail for any particular assessment will be a function of 

42 the goals of the assessment, the questions that the assessment are intended to answer, and 

43 the data that are available to conduct the assessment. Whether or not a particular level of 

44 detail is appropriate for a particular situation will depend on the purpose of the 

45 assessment.
Page 138 Microbial Risk Assessment Guideline

1 When hazards are well defined and specific exposure scenarios allow accurate 

2 exposure and dose-response calculations, a quantitative risk characterization may be in 

3 order. This is generally the case with event-driven retrospective assessments. 

4 Prospective assessments often lack sufficient information and data to make the 

5 appropriate calculations for detailed quantitative assessments. In some cases, default 

6 assumptions and/or parameter values can substitute for measured values so that 

7 calculations can be made, but with considerable uncertainty. Qualitative risk 

8 characterization may be useful, when risk management choices need to be made and a 

9 general sense of risk is all that is required by managers. For example, if default values, 

10 coupled with an understanding of the uncertainty that accompanies their use, can enable 

11 completion of a semi-quantitative assessment that gives enough information for decision 

12 making, then that assessment is more useful than the failure to produce any assessment, 

13 due to the inability to cope with the lack of data for which the default assumptions 

14 substitute. Finally, the appropriate form of risk characterization may change during the 

15 risk assessment, if questions to be answered change or if additional data become 

16 available.

17

18 The availability of data and the appropriate type of risk characterization are 

19 generally related. Assessments of specific events can sometimes result in precise 

20 calculations of exposure and dose-response, assuming the agent is well identified. In 

21 some prospective analyses, estimates of organism concentrations in a particular source 

22 may be easier to estimate than the actual exposure of particular populations. In these 

23 types of cases, default values may need to be substituted for accurate population 

24 estimates. Similarly, dose-response relationships are only available for a limited range of 

25 organisms (refer to dose-response chapter for a list of available dose response 

26 relationships), so quantitative assessments can only be conducted for those microbes that 

27 have known (or derivable) dose-response relationships.

28

29 With respect to quantitative risk characterization, you can employ a variety of 

30 model forms for the assessment of infectious disease transmission and the potential 

31 impact or benefit of intervention efforts/management actions. Particular characteristics 

32 of each model form allow for the capture of different aspects of the disease transmission 

33 system (EPA, 2004b). The two most commonly employed classes of MRA models are 

34 static and dynamic models. An overview of these types of models along with pros and 

35 cons of each is provided by Soller and Eisenberg (2008). Exclusion from the following 

36 discussion does not preclude use of a particular model form; however, justification for 

37 use of a particular model form should be included in the risk description.

38

39 For examples of risk characterization within MRA you can refer to Foodrisk.org 

for a searchable database of MRAs.40 40

41

 

40 http://foodrisk.org/risk_analysis/RA/RAs.cfm
Microbial Risk Assessment Guideline Page 139

Draft – Do not cite, circulate, or quote

1 6.5.1 When is a Static Model Appropriate?

2

3 A static model would be appropriate in those cases where the central question is 

4 concerned with the probability of infection or illness relative to the dose of pathogens

5 acquired from a single exposure. Such models can handle complex details about the 

6 course of events that lead to exposure and infection and can be analyzed by well7 established statistical techniques that require fewer assumptions than do dynamic models

8 (discussed below). Static models are useful for analyzing situations where the effect of 

9 an intervention directed to individuals (e.g., point-of-use remediation) is more important 

10 than the effect on transmission throughout the population; they are not appropriate for 

11 measuring indirect effects at the population level (e.g., the effect of water treatment 

12 interventions on risk due to secondary transmission). 

13

Susceptible

Individual

Infected/

Diseased

Pathogen

from Specific 

Source

Prob(dose)

14

15

16 Figure 6.1 Static Risk Assessment Conceptual Model

17

18

19 Some infectious diseases are not readily transmitted from person to person but are 

20 acquired, to the best of current knowledge, only by consumption of or contact with 

21 contaminated environmental materials (e.g., L. monocytogenes from food, Naeglaria 

22 fowleri infection from water). In other cases, although an agent may have the potential to 

23 be transmissible, the particular situation is such that the person-to-person component is 

24 unknown or thought to be negligible. Understanding the pattern of human infections 

25 from such pathogens or exposure scenarios may be best achieved through the use of static 

26 models (parallel to those used for toxicological risk assessments) (Figure 6.1). 

27

28 These chemical risk assessment-based models are used to estimate risk at an 

29 individual level and typically focus on estimating the probability of infection or disease 

30 to an individual as a result of a single exposure event. With respect to microbial 

31 contaminants in the environment or a particular media (food, water), a fundamental 

32 simplifying assumption of static model-based analysis is that exposure events and 

33 infection/disease are independent; that is, the outcome from one exposure event does not 

34 affect a subsequent exposure, and one individual‘s outcome has no impact on any other 

35 individual‘s outcome. Thus, secondary transmission and immunity are most often 

36 assumed to be negligible or are of similar magnitude and effectively cancel each other 

37 out. (It is generally assumed that secondary transmission would increase the level of 

38 infection/disease in a community relative to a specific exposure to pathogens, and 
Page 140 Microbial Risk Assessment Guideline

1 immunity would decrease the level of infection/disease in a community relative to a 

2 specific exposure to pathogens.)

3

4 6.5.2 When is a Dynamic Model Appropriate?

5

6 Risk managers and regulators are often concerned with risk on a societal or 

7 population scale. Thus, individual risks need to be translated to the level of the exposed 

8 population or some other relevant part of that population. When an infectious agent that 

9 occurs in the environment or a particular media is also contagious, its impact on a 

10 population can be significantly influenced by the interactions between contagious and 

11 susceptible individuals. To assess the full impact of human exposure to pathogens, you

12 should consider addressing risk at the population level in addition to individual risk at the 

13 dose-response level. For a thorough evaluation of risks that are manifest at the 

14 population level, MRA methods should explore the relative importance of secondary 

15 transmission and immunity, and thus capture and integrate the dynamic interplay of hosts, 

16 agents, and environments.

17

18 Dynamic MRA models take two main forms: deterministic or stochastic. 

19 ―Deterministic‖ means that the model output is strictly determined by the starting 

20 conditions and the values of the parameters in the equations that define the system. In 

21 stochastic models, events are treated as stochastic (random) events rather than 

22 deterministic ones. Deterministic dynamic MRA models are suitable for large 

23 populations of individuals randomly interacting with one another. In this form, the 

24 population is divided into epidemiological states such as: (1) susceptible, (2) diseased 

25 (infectious and symptomatic), (3) carrier (infected but asymptomatic), and (4) immune 

26 (partial or complete). Only a portion of the population is in a susceptible state at any 

27 point in time, and only those individuals in a susceptible state can become infected 

28 through exposure to pathogens. The dynamic aspect of the model means that members of 

29 the study population move between epidemiological states at different rates, and thus, the 

30 number of individuals in each state changes over time. A representative conceptual 

31 model for this type of MRA model is presented in Figure 6.2. 

32
Microbial Risk Assessment Guideline Page 141

Draft – Do not cite, circulate, or quote

Susceptible Exposed Carrier

Diseased

Post-Infection

Environmental 

Pathogen 

Sources

Psym

2

sym

Susceptible -Susceptible to infection

Exposed - Infected, pre-symptoms, pre-infectious

Carrier - Infectious, asymptomatic

Diseasesd - Infectious, symptomatic

Post-infection - Protected from infection

 Movement of individuals

 Movement of pathogens

1

2

3 Figure 6.2 Dynamic Risk Assessment Conceptual Model

4 (Source: Soller, 2009; Soller and Eisenberg, 2008)

5

6 Deterministic dynamic MRA models are expressed mathematically as a set of 

7 differential equations. These equations describe the rate of change in the number (or 

8 density) of individuals in a particular state (or compartment) over time and have defined 

9 parameters and starting conditions. Deterministic dynamic MRA models have a number 

10 of limitations. If they are used to model relatively small populations, the assumption of 

11 homogeneous mixing of the individuals in the population can lead to miss-estimation of 

12 disease. These models also require appropriate parameter values for transmission rates 

13 and some of this information can be quite difficult to determine accurately. Lack of 

14 knowledge and data, as well as inherent biological variability, suggest a need for 

15 uncertainty and sensitivity analyses of parameter values. Furthermore, random events 

16 such as local introduction or local die-out of a disease in a neighborhood of a 

17 heterogeneously mixing population are difficult to incorporate into these models (EPA, 

18 2004b).

19

20 In a stochastic form, dynamic models incorporate probabilities at an individual 

21 level and are evaluated by an iterative process (e.g., susceptible person A has a 

22 probability of contacting person B, who has a probability of being infectious). This type 

23 of model also uses states for classifying the epidemiological status of the population and 

24 subpopulations under study, but differs from the deterministic dynamic MRA models in 

25 that the compartments contain discrete individuals rather than the numbers or densities of 

26 persons that are represented by the compartments in deterministic dynamic MRA models.

27

28 In stochastic dynamic MRA models, events are treated as random (stochastic) 

29 events rather than deterministic ones. These models employ distributions of outcomes 

30 rather than the average outcomes as do the deterministic models; a stochastic model will 
Page 142 Microbial Risk Assessment Guideline

1 produce different results each time it is run. Stochastic forms are suitable for small 

2 populations and heterogeneous mixing patterns where stochastic events can have a major 

3 impact. In a small population, chance events, such as an infectious person contacting 

4 only immune persons during the infectious period of illness, may have a substantial 

5 impact on the transmission dynamics of the disease (EPA, 2004b). 

6

7 Based on this information, risk characterization can be thought of as modular, 

8 with different modules requiring specific data for calculations. Some modules require 

9 detailed data, while others may only require default estimates or descriptive information. 

10 Risk management options may obviate the need for precise data on exposure 

11 components, if exposure can be limited by specific actions. Thus, the appropriate module 

12 (and corresponding level of detail) for any particular assessment will be driven by the 

13 goals of the assessment, the questions that the assessment are intended to answer, and the 

14 data that are available to conduct the assessment.

15

16 6.6 How are Sensitivity and Uncertainty Analyses Related to Risk 

17 Characterization?

18

19 The discussion in this section is limited to data sensitivity and uncertainty and 

20 does not include sensitivity and uncertainty in overall decision-making, which may 

21 consider decision-maker judgments and values beyond the risk assessment.

22

23 Although uncertainty and variability are different, in practice it can be difficult to 

24 separate the two, particularly when uncertainty about variability is important (Section 

25 6.2). It may be practical to characterize uncertainty and variability together if you clearly 

26 describe what you are doing. Uncertainty analysis ―is the computation of the total 

27 uncertainty induced in the output by quantified uncertainties in the inputs and models‖ 

28 (Morgan and Henrion, 1990). Uncertainty analysis is a key concern for risk managers 

29 because it provides information about the overall reliability of the risk estimates. 

30 Measures of model ―uncertainty‖ communicate to risk managers the risk assessor‘s best 

31 judgment as to the overall quality of the numerical risk estimates generated by the MRA. 

32 Confidence intervals, ―credible ranges‖ developed through Bayesian analyses, and other 

33 measures of dispersion in risk should be presented clearly, and their meaning 

34 communicated clearly. Similarly, clear graphical or tabular presentations are very useful. 

35 To the extent that intermediate calculations add value and understanding to the results, 

36 they can also be included. Key assumptions related to model selection, input data, and 

37 parameters should be provided and discussed, as well as their implications for the model 

38 results and uncertainty. Any conservative assumptions that are built into the model 

39 should be explained and the impact of using less conservative assumptions should be 

40 discussed.

41

42 It is also important to carefully evaluate the impact of known sources of 

43 variability in model outputs. This is generally done through use of one or more forms of 

44 sensitivity analysis. Sensitivity analysis can also help determine whether more resources 

45 should be put into parameter estimation. Sensitivity analysis ―is the computation of the 

46 effect of changes in input values or assumptions (including boundaries and model 
Microbial Risk Assessment Guideline Page 143

Draft – Do not cite, circulate, or quote

1 functional form) on the outputs‖ (Morgan and Henrion, 1990). Sensitivity analyses 

2 techniques range from simply conducting a small number of additional model runs with 

3 different parameter values to performing a fully probabilistic evaluation of the effects of 

4 variations in parameter values on model outputs. The specific approach that is taken will 

5 depend on the nature of the data and models supporting a given assessment. USDA

6 identified several sensitivity analytical techniques useful for MRA (Frey et al., 2004). 

7 The methods evaluated ranged from simple and intuitive (varying input values across 

8 their observed ranges, scatter plots) to more complex statistical procedures (e.g., 

9 classification and regression tree [CART]). For any given risk assessment, it is likely that

10 one or more of these methods will be useful for sensitivity analysis.

11

12 Although sensitivity analyses are useful for evaluating the effects of the 

13 variability in single parameters on risk estimates, when multiple parameter values vary, 

14 the results of sensitivity analyses should be interpreted cautiously (EPA, 1997a). If the 

15 variations in parameter values are independent of one another, it is easy to overestimate 

16 the impact of varying more than one value because using upper or lower percentile values 

17 for more than one variable can yield point estimates of risk that are overly conservative 

18 or insufficiently protective. If the variability in risk parameters is correlated, the impact 

19 of their variations may not be easy to estimate using sensitivity analysis. In such cases, a 

20 more detailed and comprehensive analysis may be required, usually employing 

21 probabilistic approaches such as Monte Carlo or related simulation techniques. Where 

22 the variability in model parameters can be partitioned into components mainly reflecting 

23 variability and uncertainty, ―two-dimensional‖ Monte Carlo analysis can be employed to 

24 estimate the relative importance of these two components. 

25

26 The EPA Exposures Factors Handbook (EPA, 1997a) provides several approaches 

27 to quantitative uncertainty and sensitivity analysis (Table 6.1):

28

29
Page 144 Microbial Risk Assessment Guideline

1

2 Table 6.1 Approaches to Sensitivity and Uncertainty Analysis Recommended in 

3 EPA’s Exposure Factors Handbook (Source: EPA, 1997a)

4

Approach Description Example

Sensitivity analysis Changing one input variable at a time 

while leaving others constant to examine 

affect on output

Fix each input at lower 

(then upper) bound while 

holding others at nominal 

values (e.g., medians)

Analytical 

uncertainty 

propagation

Examining how uncertainty in individual 

parameters affects the overall uncertainty 

of the exposure assessment

Analytically or numerically 

obtain a partial derivative of 

the exposure equation with 

respect to each input 

parameter

Probabilistic 

uncertainty analysis

Varying each of the input variables over 

various values of their respective 

probability distributions

Assign probability density 

function to each parameter; 

randomly sample values 

from each distribution and 

insert them in the exposure 

equation (Monte Carlo 

simulation)

Classical statistical 

methods

Estimating the population exposure 

distribution directly, based on measured 

values from a representative sample

Compute confidence 

interval estimates for 

various percentiles of the

exposure distribution

5

6 In addition, Morgan and Henrion (1990) discuss in detail the following four techniques 

7 for sensitivity and uncertainty analysis, including:

8

9 a) Deterministic – One-at-a-time analysis of each factor holding all others 

10 constant at nominal values

11 b) Deterministic joint analysis – Changing the value of more than one factor at 

12 a time

13 c) Parametric analysis – Moving one or a few inputs across reasonably selected 

14 ranges such as from low to high values in order to examine the shape of the 

15 response

16 d) Probabilistic analysis – Using correlation, rank correlation, regression, or 

17 other means to examine how much of the uncertainty in conclusions is 

18 attributable to which inputs

19 6.7 How are Quality of Life Measures Important in MRA?

20

21 Quality of life measures are usually included in cost-effectiveness analyses (CEA) 

22 rather than within risk assessment. You should be aware of how your risk assessment 

23 results might be used, such as in a CEA. For example, EPA has used quality-adjusted life 
Microbial Risk Assessment Guideline Page 145

Draft – Do not cite, circulate, or quote

years (QALY) and Morbidity Inclusive Life Years (MILYs)41 1 in the regulatory impact 

2 analysis for the Final Clean Air Interstate Rule (EPA, 2005b, Appendix G) and the 

3 LT2ESWTR. (EPA, 2006a, Appendix U). 

4

5 Quality of life captures the impacts that illness has other than medical costs and 

6 lost work hours. It is particularly relevant for chronic illnesses that cause pain, suffering, 

7 and a sacrifice in lifestyle. One concept, known as QALY, is a method for assigning a 

8 numerical value for quality of life and translating that numerical value to a monetary 

9 measure (WHO, 2001). Duration and severity of illness can also be used to characterize 

10 quality of life, but these are not expressed in monetary units, so would not be utilized in 

11 the same manner as QALYs. Disability adjusted life-years (DALYs) are recommended 

12 in WHO Water Quality: Guidelines, Standards and Health to integrate the effects of a 

13 single agent, compare the health effects of different agents or conditions, and to inform 

14 the debate on acceptable risk (WHO, 2001). WHO expects that ―DALYs will play an 

15 important role in prioritizing risk factors, determining levels of acceptable risk, setting 

16 health targets and appraising effectiveness [of policy or mitigation] through examining 

17 public health outcome.‖ DALYs and QALYs are not calculated in the same manner and 

18 have reversed scales of measure. DALYs measure a health gap, with full health 

19 represented as 0 and full disability (death) as 1.0; QALYs measure health expectancy, 

20 with full health represented as 1.0 and lowest possible health state (death) as 0 (Airoldi 

21 and Morton, 2009; Gold et al., 2002; Rice et al., 2006).

22

23 It is important to note that QALYs and DALYs are not objective measures and 

24 require a descriptive conceptualization of health states. In addition, there can be 

25 significant differences in ranking due to ethnicity, gender, and area of residence (different 

26 cities; urban versus rural). Thus, there is much controversy regarding the validity of 

27 these measures partially because there is no accepted ―gold standard‖ for determining 

28 criterion validity (Gold et al., 2002). 

29

30 Agencies may have different histories of involving economists in risk assessment. 

31 For example, EPA has recognized for many years the need for early involvement of 

32 agency economists in risk assessments. EPA‘s primary concern has been that without 

33 this early involvement outcome measures from the risk assessment may be of a form that 

34 cannot is not useable in cost benefit analysis. In addition, there is increasing recognition 

35 that the risk generation process often involves predictable human response to 

36 environmental factors or firms‘ or individuals‘ responses to economic and social 

37 conditions. Economists are trained to model and estimate these kinds of responses. The 

38 inclusion of economists on risk assessment teams as scientific analysts, not as managers, 

39 could improve the accuracy of risk assessments by allowing risk assessment to 

40 incorporate models of production systems and human behavior that influence risk levels 

41 (Williams and Thompson 2004). 

42

43

 

41 MILY combines QALYs saved from avoided cases of non-fatal morbidity with life years resulting from 

mortality risk reductions (assigned a weight of 1.0).
Page 146 Microbial Risk Assessment Guideline

1 6.8 How Can a Risk Assessment be Validated?

2

3 Validation of an assessment can occur at multiple levels (ECSCC 2003, 

4 WHO/FAO 2008). Validation of the conceptual and mathematical models, the computer 

5 algorithm and the assessment‘s predictions all occur before a risk assessment can be 

6 considered validated. Except for the final level, the validation process is often 

7 accomplished through review by persons outside the risk assessment project.

8

9 Model validation and verification in risk assessment are general terms that are 

10 sometimes used to refer to rigorous data driven evaluation of models, but more often they 

11 are used interchangeably to refer to a less rigorous ―reality check‖ that may have poorly 

12 defined validation criteria. Risk assessors should be aware of the differences between 

13 model validation and verification and whether a model has been validated for 

14 interpolation or extrapolation. Researchers gathering data for the USDA use the more 

15 formal rigorous definitions of verification and validation as follows (Oscar, 2005): 

16 ―Verification... is the successful outcome of the performance evaluation process where 

17 the model predictions were compared with the data used in model development (that is, 

18 dependent data). In contrast, validation... is the successful outcome of the performance 

19 evaluation process where model predictions were compared with data that was not used 

20 in model development (that is, independent data).‖

21

22 The iterative nature of most assessments suggests that the models have been 

23 reviewed several times by risk managers and/or experts at the conceptual and 

24 mathematical development phases of the project. Nevertheless, public and peer review is 

25 usually solicited to examine the results of the assessment. Close scrutiny of the 

26 conceptual and mathematical models – and the computer algorithm – by specialists 

27 knowledgeable in statistics, epidemiology and mathematics will serve to sanction the 

28 validity of the mechanics of an assessment model. 

29

30 The output of an exposure or dose-response assessment (a dose-response 

31 relationship, an exposure distribution) is often not readily measured in nature. 

32 Surveillance data may be available for some outputs of the models and statistical 

33 measures of agreement between the model‘s predictions and empirical observations are 

34 helpful in describing the accuracy of the model. Creative uses of empiric evidence may 

35 serve to support a contention of validity. Nevertheless, most risk assessments cannot

36 meet this burden of proof concerning their validity. Such is the nature of many risk

37 assessment problems; their verification primarily stems from the logic and reasoning built 

38 into the models used to solve them.

39

40 Because validation implies different criteria in different situations, any discussion 

41 of validation should refer to how the validation was performed so that readers may 

42 understand the degree of rigor the validation effort entailed. One method of validating 

43 the risk assessment findings is to compare the outputs to epidemiological data to 

44 determine whether the risk estimates are consistent with reality. The following examples 

45 illustrate MRA model validations:

46
Microbial Risk Assessment Guideline Page 147

Draft – Do not cite, circulate, or quote

1 a) Rotavirus in Drinking Water: To confirm the validity of the output results of 

2 the epidemiologically-based model used in a case study of rotavirus in drinking 

3 water (Soller et al., 1999), a dynamic model was modified using actual data and 

4 best judgment to analyze and simulate a 1981 rotavirus outbreak in the Eagle-Vail 

5 and Avon communities in Colorado (Hopkins et al., 1984). Although a rigorous 

6 direct comparison of the results from the actual outbreak and the rotavirus 

7 simulation could not be conducted due to a lack of specific surveillance data (e.g., 

8 concentration data, secondary spread), a qualitative comparison was made to 

9 assess the plausibility of the output from the model. The overall attack rate for 

10 diarrhea and/or vomiting during the rotavirus epidemic was reported to be 

11 approximately 32% (Hopkins et al., 1984). Using virus detection or serological 

12 methods, it was estimated that a total of approximately 23% of the population 

13 became ill from rotavirus exposure during this event. The results of a 5000 trial 

14 Monte Carlo simulation of the outbreak using the model showed that about half of 

15 the trials resulted in average daily disease prevalence rates ranging from 7.5% and 

16 25%, which compares favorably to the historical estimate of 23%. Thus, it may 

17 be inferred that the output from the model seems plausible and intuitively 

18 consistent with the actual outbreak data.

19

20 b) Cryptosporidium in Drinking Water: Teunis and Havelaar (1999) conducted a 

21 case study of Cryptosporidium in drinking water and discussed the importance of 

22 and opportunities to attempt validation of their calculated estimates of yearly 

23 individual infection risk through comparison with actual epidemiological data on 

24 endemic/epidemic cryptosporidiosis. Their approach also provides a logical and 

25 transparent methodology to integrate quality of life-based approaches into the risk 

26 assessment by expressing all health effects in one single metric—the DALY. 

27 Such an approach has the added advantage of not being disease-specific and lends 

28 itself for risk comparisons (e.g., with chemical risks, for economic evaluations).

29

30 Whether or not formal validation is possible, peer review is an important aspect of 

31 evaluating models (OMB, 2004).

32
Page 148 Microbial Risk Assessment Guideline

1 7. RISK MANAGEMENT

2

3 7.1 What is Risk Management?

4

5 Risk management is a ubiquitous term used in settings as diverse as financial 

6 investing, military planning and public health. Within federal public health agencies, risk 

7 management refers to activities ranging from high-level policy making to routine, 

8 sometimes pro forma, risk control applications in operational risk management. This 

9 chapter provides an overview of the risk management processes likely to be encountered 

10 by microbial risk assessors. 

11

12 The NRC ―Red Book‖ initially defined risk management in very broad terms as 

13 ―the process of evaluating alternative regulatory options and selecting among them.‖ 

14 (NRC, 1983). By 1996, a follow-on NRC committee described the activities of risk 

15 managers:

16

17 Risk managers are supposed to deal with broad social, economic, ethical, 

18 and political issues in choosing from among a set of decision options by 

19 using the results of the risk assessment and their understanding of the 

20 other issues. Making tradeoffs, which may be called risk-benefit, cost21 benefit, or risk-risk evaluations, is part of risk management. (NRC, 1996)

22

23 The NRC reports focused on management processes occurring with a single risk 

24 assessment. During the mid-1990s, the Presidential/Congressional Commission on Risk 

25 Assessment and Risk Management argued that risk management should no longer be 

26 thought of a process that focuses on decisions about managing one risk at a time. Rather, 

27 governmental agencies need to confront the task of managing risks from multiple hazards 

28 and exposures. To this end, the Commission provided a framework for risk management 

29 recommending that the risk management process include steps to (P/CC, 1997):

30

31 a) Formulate the problem in broad context

32 b) Analyze the risks

33 c) Define the options

34 d) Make sound decisions

35 e) Take actions to implement the decisions

36 f) Perform an evaluation of the effectiveness of the actions taken

37 The engineering and systems analysis view of risk management has been 

38 described by Haimes (2004): risk management is the process focused on controlling risks 

39 by addressing:

40

41 a) What can be done and what are the options for controlling risks?
Microbial Risk Assessment Guideline Page 149

Draft – Do not cite, circulate, or quote

1 b) What are the trade-offs in terms of risks, benefits and costs?

2 c) What are the impacts of risk management decisions on future options for risk 

3 management?

4 Federal state and local public health agencies are charged with the responsibility 

5 of preventing, mitigating, or controlling risks to the public‘s health. As a general 

6 concept, it is clear that the ―mission‖ of risk management is accomplished using risk 

7 management processes at several levels. For example, Table 7.1 describes risk 

8 management as a strategic, applied, or operational function in the agency. The strategic 

9 level is concerned with managing the agency‘s portfolio of risks; the applied level—the 

10 primary focus of this Guideline—concerns the risk management processes surrounding 

11 specific risk assessments; and operational risk management deals with risk management 

12 that is guided by standard operating procedures. 

13

14 Table 7.1 Classes of Risk Management in Federal Agencies

Class of Risk Management Description

Strategic Risk Management (and 

Policy Making)

Long-term, broadly based view of the agency‘s entire 

risk portfolio. Interface with the public, industry and 

governmental stakeholders about policy issues such 

as the level of acceptable risk, the risk-based 

decision- making process

Applied Risk Management Charters and collaborates with risk assessors on 

newly identified or emerging risks, new scenarios for 

known risks and risk mitigation scenarios. The 

information gained from the risk assessment is used 

in risk management decisions about controlling the 

risk

Operational Risk Management Implements prescribed administrative, engineering or 

other controls to maintain risk at appropriate levels or 

below. 

15

16 The Codex Draft Principles of and Guidelines for the Conduct of Microbial Risk 

17 Management includes the following 8 principles (Codex, 2007a).

18

19 a) Principle 1: Protection of human health is the primary objective in MRM.

20

21 b) Principle 2: MRM should take into account the whole food chain.

22

23 c) Principle 3: MRM should follow a structured approach

24

25 d) Principle 4: MRM process should be transparent, consistent and fully 

26 documented.

27
Page 150 Microbial Risk Assessment Guideline

1 e) Principle 5: Risk managers should ensure effective consultations with relevant 

2 interested parties.

3

4 f) Principle 6: Risk managers should ensure effective interaction with risk assessors.

5

6 g) Principle 7: Risk managers should take account of risks resulting from regional 

7 differences in hazards in the food chain and regional differences in available risk 

8 management options.

9

10 h) Principle 8: MRM decisions should be subject to monitoring and review and, if 

11 necessary, revision.

12

13 7.2 When and How Can Risk Managers be Involved in Risk Assessments?

14

15 Risk management begins before risk assessment. Sometimes, the recognition of 

16 potential problem and the general hazard identification occurs externally to the agency 

17 and is brought to the attention of risk managers by the public, stakeholders, or other 

18 governmental organizations. Risk managers typically determine the need for a risk 

19 assessment and provide the risk assessment team with the specific risk analysis to be 

20 performed. This often includes setting the analytical boundaries and constraints for the 

21 risk analysis. For example, the risk assessment might be focused on risks from exposures 

22 to the hazard only within the U.S. borders or, perhaps to a particular subpopulation at risk 

23 of illness from exposures to the hazard. The initial problem formulation and discussion 

24 about boundary conditioning are often accomplished interactively with risk assessors who 

25 have the particular knowledge about what can be performed quantitatively and whether or 

26 not a quantitative risk assessment can be accomplished within the project time 

27 constraints.

28

29 Risk managers should work interactively with you (risk assessors) during the 

30 planning and scoping activities to collaborate on defining clear, scientifically defensible 

31 ―risk questions‖ before the analytical components of risk assessment are executed (FDA, 

32 2002). Forming a risk question is analogous to stating a testable hypothesis at the outset 

33 of a basic research project: it is a necessary antecedent to designing an objective and 

34 informative project. Here, risk managers are generally aware of the type of information 

35 needed to answer policy questions, the resources available to mount complex and large36 scale risk assessments, and relevant stakeholder concerns. You will probably rely on risk 

37 managers for a ―big picture‖ perspective of the agency‘s entire portfolio of risk 

38 management activities and how the current risk assessment fits into the agency‘s work 

39 plan.

40

41 Risk managers have valuable insights into the value and potential problems of risk 

42 assessments. Thus, high quality risk management requires risk managers to interface

43 with you at various stages throughout the entire risk assessment process so that they can 

44 help you anticipate problems in the analyses and redirect resources, if necessary, to

45 improve or ensure the quality of information resulting from the risk assessment. 

46 Additionally, risk managers might become aware of new information about the risk in 
Microbial Risk Assessment Guideline Page 151

Draft – Do not cite, circulate, or quote

1 question that might be useful for focusing the risk assessment on a modified risk 

2 question. The frequency of risk assessor and risk manager discussions will likely depend 

3 on the complexity and nature of the risk being investigated.

4

5 The risk managers should explain clearly why the assessment is being performed 

6 and what questions need to be addressed. The risk managers should also advise the 

7 assessors, economists, engineers, and other contributing experts involved in the planning 

8 and scoping of any interested party, affected party, or policy interests to be considered in 

9 the context of the risk issue. These factors may influence the risk management options, 

10 management goals, key participants, data sources, selection of assessment endpoints, or 

11 the schedule for the development of the assessment. The risk manager and appropriate 

12 others should discuss any regulatory basis for the risk assessment and what kind of 

13 information is required to satisfy such requirements.

14

Risk assessment teams usually have a lead risk assessor.42 15 The lead risk assessor 

16 is responsible for ensuring that risk assessments are properly performed and documented 

17 and that the key information from risk characterization is elevated up the management 

18 chain and communicated to senior management. The lead risk assessor should ensure 

19 that the risk characterization integrates other considerations specified in applicable 

20 statutes, Agency and office policies, executive orders, and other factors to make and 

21 justify regulatory decisions. The lead risk assessor‘s specific responsibilities might

22 include:

23

24 a) Ensure that all risk assessment work products produced by or submitted to your 

25 organization are well written and characterized.

26

27 b) Provide advice, guidance, and support for the preparation, conduct, and 

28 completion of an appropriate risk assessment for your decision.

29

30 c) Play a major role in managing and documenting the planning and scoping 

31 process.

32

33 d) Ensure that sufficient funds are designated in the office‘s budget request to 

34 conduct a risk assessment.

35

36 e) Establish a realistic risk assessment schedule.

37

38 f) Ensure that the products prepared by individual risk assessors for their portion of 

39 each risk assessment document are integrated into a complete risk assessment.

40

41 g) Establish systems to maintain records of the risk assessments prepared by risk 

42 assessors under your supervision.

43

 

42 Synonyms for this position include, technical integrator, risk assessment team leader, risk assessment 

team liaison, and risk assessment manager.
Page 152 Microbial Risk Assessment Guideline

1 h) Ensure that the key points from the risk assessment are carried forward in all 

2 deliberations or considerations for decision making.

3

4 i) Review implemented decisions for the degree of implementation, efficacy, and 

5 ongoing relevance.

6

7 7.3 How are Risk Management Options a Useful Component to Include in a Risk 

8 Assessment?

9

10 ―Risk managers use information from risk assessment and economic analysis, 

11 together with information about public values and statutory requirements, to make 

12 decisions about the need for and methods of risk reduction‖ (P/CC, 1997). To

13 accomplish risk management decision-making, a decision among options for risk 

14 management controls requires that the decision alternatives for risk management be 

15 specified. The characterization of options often means that risk assessors are asked to 

16 calculate risks given one or more proposed risk management scenario. Risk managers 

17 use these scenarios, the inputs of benefit-cost assessments and other information to make 

18 decisions about the best option for controlling risks.

19

20 One of the principles of risk management is that the risk management analysis

21 and the proposed risk control strategy should be commensurate with the level of risk. 

22 The reality of this principle in practice is that it also often relates to uncertainty about the 

23 magnitude of risk. Highly uncertain risk estimates sometimes lead risk managers to 

24 expend additional risk assessment resources in an effort to reduce the uncertainties before 

25 decisions about controlling the risks are taken. 

26

27 From the risk manager‘s perspective, risk assessment is only one among several 

28 tools that can be used to inform the risk decision made by decision makers. Decision 

29 making by risk managers can call for benefits-risk assessments, risk-risk analysis, VOI

30 analysis, or trade-off analysis as additional information useful to making decisions about 

31 managing risks. Decision analysis might be used to create a systematic and transparent 

32 decision making process that evaluates the importance of factors ranging from the 

33 objectively scientific to social values. For most scientific endeavors, ―risk-informed‖ or 

34 ―risk-based‖ decision making benefits from a formal decision analysis that provides a 

35 systematic analysis of complex scientific information, concerns of stakeholders, the 

36 constraints on risk management options caused by gaps in data, models or policies, and 

37 need for transparency in governmental decision making.

38

39 7.4 What are Some Other Inputs into Risk Management Decisions About 

40 Controlling or Accepting Risks?

41

42 Decision making about risks often requires balancing results of a risk assessment 

43 with the results of benefit-risk and risk-risk tradeoff analyses; the need for risk 

44 management resources to address other risks in the agency‘s portfolio; and political 

45 pressures from stakeholders in industry, the public, or legislatures. The most important 

46 input from risk assessment into the decision-making is a high-quality risk 
Microbial Risk Assessment Guideline Page 153

Draft – Do not cite, circulate, or quote

1 characterization from which risk managers can evaluate the scientific underpinnings of 

2 risk estimates, including a characterization of uncertainties in the estimates. 

3

4 Risk managers make decisions under uncertainty. The results of benefit-risk and 

5 other analyses prior to risk management decision making often include equivalently 

6 uncertain estimates for the impact of proposed risk controls on the reduction or 

7 elimination of risks. Here, risk managers can employ formal decision analysis to trade8 off decision alternatives, based on both objective and subjective assessments, for a 

9 transparent decision. In other situations, the decision to control or accept risks might be 

10 well-defined by existing regulation or guidance. Ultimately, the goal of risk management 

11 is to achieve an appropriate level of risk. The affirmation of this goal might not occur 

12 until after risk management controls are applied and the risks have been evaluated 

13 iteratively. 

14

15 One of the most difficult aspects of risk management for regulatory functions is 

16 setting an acceptable or tolerable level of risk. Different approaches to setting standards 

17 have different ways of incorporating tolerable risk levels. You should be aware of what 

18 approach has historical precedence in the field that applies to your risk assessment. A 

19 summary of some approaches to acceptability that have been suggested to regulators is 

20 provided below (adapted from Humber and Almeder 1986, Fischhoff et al. 1981, Lave 

21 and Romer 1981, Lowrance 1976):

22

23 a) Reasonableness – This is a commonly cited principle in safety judgments. For 

24 example the Consumer Product Safety Commission is mandated to ―reduce 

25 unreasonable risk of injury.‖

26 b) Custom of usage – The U.S. Food and Drug Administration (FDA) ―generally 

27 recognized as safe‖ (GRAS) determination is for food substances which do not 

28 have to be regulated as additives because among other things they have a history 

29 of usage. Table salt and sugar are examples.

30 c) Prevailing professional practice or professional judgment – Originally 

31 established for physician‘s clinical practice, the principle is also used for local 

32 building standards and toy design. The underlying assumption is that sanction by 

33 custom is safer than untested. In professional judgment professionals rely on 

34 personal experience, accepted professional practice and their clients‘ desires to 

35 judge risks.

36 d) Best available practice, highest practicable protection, and lowest practicable 

37 exposure, best available technology – air and water quality regulations are 

38 associated with these standards, but they still require judgment. Lowest practical 

39 is also known as ―as low as reasonably practical‖ (Vatn 2004).

40 e) Risk benefit (degree of necessity or benefit) – A rough balancing of risks and 

41 benefits is attempted.

42 f) The Delaney Clause or No-risk – named after former New York Congressman 

43 James Delaney who added language to the Federal Food, Drug, and Cosmetic Act 

44 that states ―no [food] additive shall be deemed to be safe if it is found...to induce 
Page 154 Microbial Risk Assessment Guideline

1 cancer in man or animal.‖ It has been criticized for omitting other health 

2 endpoints and because it ignores dose-response relationships. It attempts to lower 

3 risks to zero.

4 g) No observable adverse effect level (NOAEL) – because no adverse effects are 

5 observed at a given level of chemical, that level is deemed acceptable. It is 

6 customary in chemical risk that when the NOAEL is based on animal data, a 

7 safety factor of 100 is applied to arrive at the level for humans (decimal point is 

8 moved to the left to make the appropriate level 100 times lower).

9 h) Cost-effectiveness – Equates the cost of saving lives or preventing adverse 

10 effects across programs.

11 i) Formal benefit-cost analysis (also referred to as cost-benefit analysis) –

12 quantitative framework that includes explicit dollar value of a human life or 

13 human wellbeing. Performance of cost-benefit analysis still only serves to inform 

14 the risk management decision. It cannot determine the decision (Williams and 

15 Thompson 2004).

16 j) Risk-risk – balances various risks against each other.

17 k) Quantitative risk assessment – is able to provide a range within the actual risk 

18 that the acceptable risk can fall. Alternative management assumptions can be 

tested. 43 19

20

21 For chemical risk assessment usually the risk level is stated quantitatively. For 

22 microbial risk level, regulators very often refer to quantified risk reduction, without 

23 actually stating the level of risk associated with the risk reductions or commenting on the 

24 acceptability of the level of risk. Some levels of risk that are customary for microbial 

25 hazards in different media include:

26

27 a) Ambient recreational water – The U.S. and the most stringent European Union 

28 and WHO standards are associated with about one to two percent (1-2%) 

29 increased risk of gastrointestinal illness due to exposure to ambient water during

30 recreational activities (EPA, 2004c). 

31

32 b) Foods – Hazard Analysis Critical Control Point (HACCP) plans are common and 

33 are not linked to acceptable health risk levels, but are designed to ensure 

34 contamination of food is prevented or otherwise mitigated. Food Safety 

35 Objectives and Performance Objectives are stated goals (often numeric in nature) 

36 for public health, processing, transportation, or retail safety (Crouch et al., 2009; 

37 Havelaar, et al., 2004; Rieu et al., 2007). Management options are often linked to 

38 quantified public health outcomes.

39

40 c) Biosolids – Microbial standards are currently based on operational standards 

41 which should provide pathogen levels that are below detection limits. The 

42 standards are not linked to level of health risk.

 

43 Proposed control measures must really reduce risks, not transfer them somewhere else (de Koning 1987).
Microbial Risk Assessment Guideline Page 155

Draft – Do not cite, circulate, or quote

1

2 d) Air – There are no standards or Threshold Limit Values for microbial pollutants 

3 in the United States, but numeric criteria for mold and bacteria levels have been 

4 set in other countries. Indoor levels are compared to outdoor levels.

5

6 e) Vaccinations – Historically accepted or rejected risk levels of different adverse 

7 health outcomes are compared to benefit:risk analysis for new vaccinations (FDA,

8 1999).

9

10 f) Occupational – The General Duty Clause provides employees with workplaces 

11 that are ―free from recognized hazards that are causing or are likely to cause death 

12 or serious physical harm.‖ Target risk levels for quantified health outcomes are 

not discussed (29 CFR 1910.1030).44 13

14

15 7.5 What are Some Operational Risk Management Tools and Approaches?

16

17 A major reason for the development of the public health field was for the 

18 prevention of microbial diseases in human population caused by unsanitary conditions 

19 (e.g., John Snow and the 1854 Cholera outbreak) or the presence of vectors for disease 

20 transmission. The regulatory tools at the disposal of the public health risk manager span 

21 an entire range of options to prevent the occurrence of pathologic organisms (e.g., low22 acid canning regulations) to limiting the means of primary or secondary transmission to 

23 vaccination of the host (e.g., for Bacillus anthracis) to improve host resistance to disease. 

24 Sanitary engineering designs that provide clean drinking water and separate waste water 

25 channels are credited with preventing countless outbreaks of disease (e.g., cholera, 

26 typhoid fever) and saving countless lives in more modern times. Even the education and 

27 outreach programs to promote safe handling of raw foods during preparation are part of 

28 risk management strategies to reduce the incidence of foodborne illness. 

29

30 Generally speaking, operational risk management controls for health hazards are 

31 classified as physical, administrative and management controls (Table 7.2). For the 

32 particular case of microbial hazards, a ―biological‖ classification for the host could be 

33 added for the possibility of immunization against some of the microbial hazards.

34

 

44http://www.osha.gov/pls/oshaweb/owadisp.show_document?p_table=STANDARDS&p_id=10051
Page 156 Microbial Risk Assessment Guideline

1 Table 7.2 Class of Risk Management Controls for Operational-Level Microbial

2 Risk

Class of Operational 

Risk Controls Examples

Physical Barriers and physical factors that eliminate or reduce 

the chance of contamination and growth of pathogens

Pasteurization with heat or ionizing radiation

Packaging

Disposable gloves

Administrative Engineering designs: process controls

Work practices and standards

Sanitary practices by personnel

Training program design

Managerial Training execution

Supervision 

Appropriate skills for the tasks that might lead to 

contamination or favorable growth

Biological (specific 

microbes)

Vaccination

Food animal treatments with benign organisms

3

4 7.6 What is Risk Management for the Intentional Use of Regulated 

5 Microorganisms?

6

7 Microorganisms, whose intentional uses fall under federal laws, including, but not 

8 limited to, the Federal Insecticide, Fungicide, and Rodenticide Act, the Federal Food, 

9 Drug, and Cosmetic Act, the Plant Protection Act, the Endangered Species Act, and the 

10 Invasive Species Act, often have specific risk management options available to the 

11 regulators. These options can involve placing restrictions on their use or, if justified, 

12 denial of permission for their use. Restrictions on their use can vary according to the 

13 specific law and regulations involved. For example, for pesticides, the restrictions can be 

14 used to limit exposure by specifying on the approved labels exactly when, where, and 

15 how much can be applied. Furthermore, protective clothing, including respiratory masks, 

16 may be required to be used. Restrictions can also be set on what levels are acceptable to 

17 appear on food crops, although due to the potential for growth subsequent to application, 

18 microorganisms with any potential for human toxicity and/or pathogenicity are generally 

19 not allowed to be used on food crops. Containment requirements can be placed on 

20 intentional uses of regulated microorganisms for field testing and for industrial uses. 

21 Genetic engineering can be used to reduce risk concerns for intentionally used and 

22 regulated microorganisms, e.g. by deleting toxin genes. In addition, marker genes can be 

23 used to better identify the specific approved uses. Restrictions on the storage and 

24 movement of the regulated microorganisms can also be used to mitigate risk.

25

26
Microbial Risk Assessment Guideline Page 157

Draft – Do not cite, circulate, or quote

1 8. RISK COMMUNICATION

2

3 A risk assessment is only as good as your communication of its output. All of the 

4 time and effort you spend conducting a risk assessment is for naught unless there is clear 

5 communication of risk assessment results to risk managers and other stakeholders. 

6 Effective risk communication requires diligence and has the same level of importance as 

7 any other component of the assessment.

8

9 This Guideline does not provide a detailed treatment or guidance for risk 

10 communication; you can refer to other places for such detail (Sellnow, 2008; Morgan, 

11 2002; Lundgren and McMakin, 1998). Rather, this chapter provides an initial 

12 understanding of how risk communication plays an important role in the risk assessment 

13 process.

14

15 Risk assessors are not expected to carry the load of risk communication. You 

16 need to work with the appropriate communication offices in your Agency (e.g., public 

17 affairs office, Congressional outreach office). These communicators need to be part of 

18 the team during planning and scoping and throughout the remainder of the risk 

19 assessment process. 

20

21 8.1 What is Risk Communication?

22

23 Risk communication describes and exchanges information about risk, including 

24 its form and severity, and what can be done to lessen or avoid it. It includes two equally 

25 important objectives:

26

27 a) Inform risk managers about risk so that they may make informed decisions.

28

29 b) Inform the public about risk so that they understand the nature of the risk and 

30 what is being done or will be done about it.

31

32 Risk communication is the interactive exchange of information and opinions 

33 concerning risks and risk management among risk assessors, risk managers, consumers, 

34 and other interested parties (WHO, 2000).

35

36 8.2 What are the Aspects of Risk Communication?

37

38 Risk communication principles include (OMB 2007b):

39

40 a) Risk communication should involve the open, two-way exchange of information 

41 between professionals, including both policy makers and ―experts‖ in relevant 

42 disciplines, and the public.

43 b) Risk management goals should be stated clearly, and risk assessments and risk 

44 management decisions should be communicated accurately and objectively in a 

45 meaningful manner.
Page 158 Microbial Risk Assessment Guideline

1 To maximize public understanding and participation in risk-related decisions, 

2 agencies should:

3 1) explain the basis for significant assumptions, data, models, and 

4 inferences used or relied upon in the assessment or decision;

5 2) describe the sources, extent and magnitude of significant uncertainties 

6 associated with the assessment or decision;

7 3) make appropriate risk comparisons, taking into account, for example, 

8 public attitudes with respect to voluntary versus involuntary risk; and,

9 4) provide timely, public access to relevant supporting documents and a 

10 reasonable opportunity for public comment.

11

12 The aspects of a good risk communication plan (see Section 2.6.6) include:

13

14 a) Involvement and input of risk managers and stakeholders throughout the risk 

15 assessment process

16 b) Clear risk management questions that are understood by managers and other 

17 assessors

18 c) Awareness by managers and other stakeholders of the strengths and limitations of 

19 the assessment

20

21 Social and personal behaviors are strongly influenced by risk managers‘ 

22 pronouncements, but only if the underlying risk assessment process is transparent to the 

23 public and considered credible. Trust is based on open communication and the credibility 

24 of provided information; thus, the public must be aware of the science behind the risk 

25 assessment. People who write risk communication statements should consider 

26 acknowledging both the power and the limits of the risk assessment process and the data 

27 used in the risk assessment. 

28

29 8.3 Who are the Stakeholders of MRAs?

30

31 Stakeholders are people or organizations that may be affected by the relevant 

32 decision and thus, have an interest in the outcome of the risk assessment. They typically 

33 include people such as those in industry who will be responsible for implementing and 

34 financially affected by new rules and regulations borne of the risk assessment, as well as 

35 the general public. For example, a risk assessment that results in new regulations for 

36 Escherichia coli in ground beef would affect all persons who produce and consume 

37 ground beef. Anyone interested in a risk assessment may be reasonably termed a 

38 stakeholder. Stakeholder groups will self-identify but should also be sought out.

39

40 Risk managers, assessors, and communicators are not typically considered 

41 stakeholders because they should remain unbiased. If the results of the risk assessment

42 could affect them and the outcome was altered because of this knowledge, this would be 

43 considered a conflict of interest.
Microbial Risk Assessment Guideline Page 159

Draft – Do not cite, circulate, or quote

1

2 8.4 With Whom Can I Communicate?

3

4 Even if risk communication specialists are on the team, you as the risk assessor 

5 may serve in a risk communicator role and be responsible for communicating with 

6 stakeholders and developing outreach materials. If this is the case, you can work closely 

7 with appropriate Agency offices to follow protocol. For example, many agencies will 

8 request that communication with members of Congress be done through a specific office. 

9 Communication strategies and materials for different stakeholders should be tailored to 

10 that particular audience. For example, communication with technical experts could have 

11 a high level of technical detail, while communication to the lay public would be less 

12 technical. 

13

14 The language used in communication materials should conform to your Agency‘s 

15 standards, but needs to be absolutely comprehensible by almost all being addressed. 

16 Agency communications, public affairs, and/or outreach offices should be consulted to 

17 assure that the message can be understood. Your agency may also have specific guidance 

18 for coordinating with other federal, state, and local health and environmental agencies.

19

20 8.5 When Can the Process of Risk Communication Begin?

21

22 There is a tendency to view risk communication as a final stage to the risk 

23 analysis process—something that occurs upon completion of the risk assessment and risk 

24 management. It is critically important that risk communication strategies be developed at 

25 the beginning of a risk assessment (during planning and scoping), to inform risk 

26 managers and other stakeholders throughout the risk assessment. The best place to start 

27 the risk communication process is during planning and scoping discussions in which a 

28 risk communication specialist can draft a risk communication plan (i.e., the 

29 communication strategy, including the risk assessor‘s role).

30

31 During development of the risk assessment, the team leader can be proactive in 

32 implementing the risk communication plan, particularly communicating with risk 

33 managers and stakeholders. For example, one way to do this is by announcing that the 

34 Agency plans to conduct a risk assessment. An announcement can be placed in the 

35 Federal Register. Other good venues for announcing risk assessments (and subsequent 

36 activities) include the agency‘s web site, advertising/announcement sections of trade and 

37 professional journals, and at professional meetings.

38

39 8.6 Can I Communicate in Writing, Orally, or Both?

40

41 Both. Oral communication is needed at many points throughout development of 

42 the risk assessment. It is appropriate to keep risk managers and other stakeholders 

43 informed about progress of the risk assessment. For example, you may wish to schedule 

44 a weekly, (or other appropriate frequency) phone call with managers to keep them 

45 updated. Similarly, periodic conference calls or public meetings with stakeholders lessen 

46 the chance that they are caught off guard when the risk assessment is completed.
Page 160 Microbial Risk Assessment Guideline

1

2 Communication in writing is almost always appropriate, especially since a written 

3 record is usually needed. For example, it is good to have the risk assessment questions in 

4 writing. Doing so entails back-and-forth work with risk managers to identify and clearly 

5 articulate the purposes of the risk assessment. By solidifying the questions in writing, 

6 you help ensure that risk managers, risk assessors, and stakeholders clearly understand 

7 what the risk assessment is intended to do. 

8

9 The most current risk communication message should be available in some 

10 written form (e.g., fact sheets), as well as electronically, throughout the process. 

11 Information should be available on your Agency‘s website (although it should also be 

12 accessible to all who do not have computer access). Coordination with local health 

13 authorities in most cases is also important, and they may be able to suggest useful 

14 approaches to communicating with their public.

15

16 Virtually all completed risk assessments include a written report. The TCCR 

17 principles discussed and integrated throughout this Guideline will help with the effective 

18 written communication in the form of the report (Section 1.10). In addition, many risk 

19 assessments are presented publicly as presentations or seminars. 

20

21 8.7 Who Decides What to Communicate?

22

23 The risk manager—in consultation with the risk assessor—is responsible for 

24 deciding what information to communicate. Any formal communication or information 

25 release should be firmly based on the documented findings of the risk assessor. 

26 Therefore, you are responsible for communicating risk to the risk manager for a full 

27 understanding of the potential risks that would then be considered in the decision making 

28 process. Again, risk communicators can assist in the delivery and presentation of the 

29 information to the public.

30

31 MRA is an inexact science, requiring judgment calls and policy decisions on the 

32 part of highly trained, experienced professionals. This message should be included in 

33 any communication of risk, following a statement of known facts and preceding 

34 prescriptive risk reduction measures. Mathematical constructs and underlying 

35 assumptions should be made clear. Although there may be differences of opinion among 

36 risk assessors, the risk manager is ultimately responsible for deciding on a transparent 

37 and clear message on which the audience can evaluate agency actions.

38

39 8.8 What Information Can be Communicated?

40

41 The content of the formal risk assessment (as communicated to the public or to 

42 stakeholders) is determined by the risk manager, but you are responsible for presenting

43 all available data, including those that challenge or do not support points in the 

44 assessment. The information that the risk manager may decide to communicate includes:

45
Microbial Risk Assessment Guideline Page 161

Draft – Do not cite, circulate, or quote

1 a) Data on human disease identified as either historical or experimentally-derived, or 

2 projections generated by modeling

3 b) Underlying uncertainties or data gaps 

4 c) The degree of potential hazard to sensitive populations such as children, the 

5 elderly, and immune compromised people (focusing on susceptibility and 

6 severity)

7 d) Explanation that the different pathways that were deemed relevant were explored 

8 thoroughly

9 e) The possibility of person-to-person transmission (if applicable)

10 f) Any results from animal testing and how these data may be relevant to humans

11 g) The potential for zoonotic transmission between humans and animals (if 

12 applicable)

13 h) Potential actions to reduce exposures (for example, following posted signs 

14 regarding swimming, fishing, or harvesting clams)

15

16 You will have to provide the risk manager with your best professional judgment 

17 at that time as to the degree of risk associated with a specific hazard. This judgment can

18 be accompanied by all relevant data about both the hazard and the target population. 

19 Probably some of the most important pieces of information you need to communicate are 

20 descriptions of the probabilities, uncertainties, and possible sources of biases or error 

21 (assumptions, generalizations) in any interim or final assessment.

22

23 8.9 How is the Communication Process a Continuous Dialog?

24

25 Effective risk communication is a continuous process that requires constant 

26 feedback throughout the risk assessment. This begins at the planning and scoping stage, 

27 when all parties involved (including stakeholders as appropriate) should have a chance to 

28 comment on the need for a risk assessment, the scope of the risk assessment, and the 

29 other factors discussed in planning and scoping. Doing this helps to promote buy-in and

30 helps to ensure respect for the process. Communication of risk from assessors to 

31 managers is constant throughout the process. Communication of risk from risk managers 

32 to the public is usually episodic, but nonetheless, scheduled regularly.

33

34 As an example, interactive communication occurs between risk managers and risk 

35 assessors in developing risk management questions. A risk manager may say s/he wants 

36 a risk assessment to address ―Salmonella.‖ It is then up to you to press for more 

37 specifics. At this point, perhaps the manager refines the question to ―What is the effect of 

38 increased cooking temperatures on illnesses from Salmonella?‖ Then you may indicate

39 that data are only available for a specific serotype, Enteritidis for example, but not all 

40 serotypes. Thus, the manager can work to refine the question to include Enteritidis only. 

41 In the end, iterative communication helps ensure clear and concise questions, which in 

42 turn increases the likelihood of a useful risk assessment. This iterative dialog can be 

43 conducted in person to speed up the process, but written documentation of the 
Page 162 Microbial Risk Assessment Guideline

1 understandings between risk managers and risk assessors is important for clarity and the 

2 orientation of new team members.

3

4 Communication can also be iterative when describing results of the assessment to 

5 managers and other stakeholders. In virtually all cases, there is considerable room for 

6 improvement in risk assessments. Therefore, it is unwise to present results from risk 

7 assessments as if they were written in stone. Instead, the results should be presented as a 

8 best effort, with the idea that feedback from managers and stakeholders will likely 

9 improve the assessment.

10

11 8.10 How In-Depth Can I Communicate?

12

13 Depth of communication depends on the audience. For example, if you are 

14 describing the risk assessment to another risk assessor, perhaps one who will peer review 

15 the assessment, then your communication should be very detailed. If, on the other hand, 

16 you are presenting the results to a high-level risk manager in the space of ten minutes, in17 depth details should be avoided unless specifically sought. When communicating with 

18 stakeholders, it is important to present results in a clear manner without talking down to 

19 the audience. For example, do not say, ―This next part is complicated, so I‘ll put it in 

20 terms you can understand.‖ Instead, start with a broad description of the work and then 

21 proceed to the details as both time and the audience‘s needs dictate. Lastly, it is very 

22 important to write a detailed report of the assessment. That way, even though you may 

23 be unable to give specifics during a talk, briefing, or other venue, you will always be able 

24 to refer your audience to the written report. The detailed report should be sufficiently 

25 detailed that another team of risk assessors with appropriate expertise can replicate the 

26 risk assessment. In addition to a formal detailed report, summary graphics, executive 

27 summary, fact sheets, and other types of overview documents can aid in communication 

28 efforts.

29

30 8.11 What Can I Do if the Message Is Not “Getting Through?”

31

32 It is incumbent upon you as a risk assessor to convey the results of the 

33 assessment. If you cannot do this, then the utility of the assessment is lessened. 

34 Accordingly, when you communicate the results of the assessment (or any aspect of the 

35 assessment for that matter), work to engage your audience. Take time to explore if your 

36 audience understands the points you are trying to communicate. If not, take a step back 

37 and work to clarify the parts of your message that are confusing. Look at this as an 

38 opportunity to exchange information and thoughts with your audience. Keep in mind that 

39 the difficulty may lie in your communication, not in your audience‘s comprehension. It 

40 may help if you communicate through various means. For example, in addition to 

41 speaking, it may help to take out pencil and pad and sketch your message. Be sure to 

42 allow time throughout your communication (be it a formal slide show or an informal 

43 conversation) for the audience to seek clarification. When experiencing problems getting 

44 through to the audience, consult communication specialists within your Agency that can 

45 assist with communication. 

46
Microbial Risk Assessment Guideline Page 163

Draft – Do not cite, circulate, or quote

1 8.12 How Can I Communicate Risk Successfully?

2

3 Successful risk communication requires strategic planning. This planning 

4 requires in turn a very thorough review of the costs and benefits of specific actions (or 

5 inaction), and considers possible outcomes. Successful risk communications consider the 

6 public to be stakeholders, inasmuch as Agency decisions and actions affect them directly. 

7 The stakeholders base their own behaviors on information (however anecdotal) provided 

8 by trusted sources. Strategic risk communication practices may help to develop audience 

9 understanding and ultimately gain stakeholder and public cooperation. 

10

11 Strategic risk communication involves planning how to address stakeholder 

12 questions identified during planning and scoping and later during the assessment process. 

13 The responses to these questions should be straightforward and couched in simple 

14 language (rather than technical jargon). Concepts need to be packaged correctly, i.e., 

15 clearly, truthfully, and respectfully. This last requirement—that relevant concepts be 

16 presented in understandable ways that enable discussion amongst all stakeholders, rather 

17 than as abstruse knowledge suitable for expert analysis only—is critical in establishing 

18 successful communication.

19

20 Risk managers turn to strategic risk communication when they need ―buy-in.‖ 

21 The best way to achieve cooperation of public and stakeholders in, rather than agitation 

22 against, risk-management decisions is by helping the audiences choose among options 

23 that involve cost (risks) and safety (benefits). Usually no one on the team can foresee the 

24 full range of audience responses. The starting relationship between Agency and audience 

25 may be skeptical and/or confrontational, pitting audience experience against expert 

26 analysis. Strategic risk communication helps the audience to gain insight into the 

27 problem (and/or proposed actions) and to establish exactly what aspects of the proposed 

28 action (timing? approach?) are within their control. Ideally, successful risk 

29 communication will form the basis for mutual trust; but at the very least, strategic risk 

30 communication transfers information.

31

32 People respond not only to what is said, but how it is said. In addition, people 

33 response to the way in which actions are carried out. Stakeholders value the qualities of 

34 listening, understanding, and responsiveness on the part of the Agency. Other important 

35 factures include their own perception of risk acceptability, due process (in which 

36 stakeholders are able to participate in judging risks and predicted benefits), their personal 

37 sense of the risk-manager‘s credibility, and, above all, open communication 

38 (transparency). Successful risk communication is developed with full appreciation of the 

39 technical complexity of the situation in question, the controversy about or unavailability 

40 of the requisite science, the sensitivity of the communication environment, the potential 

41 relevance of political realities, and the perceived credibility of parties involved.

42

43 Other preparations can also contribute to successful communication. For example, 

44 practicing presentations before audiences such as co-workers can test the 

45 understandability of the presentation and increase the presenter‘s comfort level. The 

46 length of presentations is also an important consideration, since typically audiences do 
Page 164 Microbial Risk Assessment Guideline

1 not want long presentations without opportunities for to ask questions. Furthermore, 

2 practical information such as what actions can be taken by individuals or communities to 

3 interrupt exposures (e.g., beach closures to limit exposure to fecal contamination) can be 

4 included whenever the risk assessment is presented to the public, even if the actions 

5 suggested are not directly addressed or considered in the risk assessment.

6

7 8.13 How Can I Handle Media and Congressional Office Requests?

8

9 Risk assessors are sometimes approached directly with questions by external news 

10 sources and congressional offices. It is the responsibility of the risk manager and/or your 

11 communications offices (e.g., public affairs office, congressional liaison office), not the 

12 risk assessor, to communicate risk. If you are contacted directly from someone in the 

13 media or a congressional office, inform that person that your communications office is 

14 authorized to answer their questions (e.g., requests for interviews, background 

15 information, policy questions). Provide them with your communications office contact 

16 information (phone number, e-mail address). If you are contacted directly via e-mail, 

17 simply forward the message to your supervisor and communications office. No matter 

18 how you are contacted, inform your supervisor and include as many details about the 

19 request as you can.

20

21 However, there will be occasions that, with appropriate permissions, you can 

22 provide technical information relevant to the risk assessment being queried. Remember 

23 that there is no such thing as ―off the record.‖ Assume everything said is on the record; 

24 even ―background‖ information is still a response. Once you provide information, 

25 nothing really prevents the recipient from including that information in their article or 

26 report. A good reporter for example may not use ―off the record‖ information as directly 

27 attributable to you, but will have a lead to contact others to verify it and then may use or 

28 release that information. You might also consider having a press officer on the phone or 

29 during interviews to make sure that the appropriate information is communicated. You 

30 may need to be aware of press deadlines and might be asked to contribute graphics to 

31 include in articles where appropriate. 

32

33 8.14 When Can Risk Communication End?

34

35 Once a decision has been made, fully implemented, and openly communicated as 

36 per the risk communication plan, subsequent communication efforts may not be as 

37 intensive. However, there will likely be a need to monitor how the implemented decision 

38 is accomplishing its goal(s) or not, and communication will be critical at that time. You 

39 need to be aware that resultant actions based on the risk assessment may need to be re40 evaluated and addressed further in the future (and this can be part of your communication 

41 plan). However, be careful not to mislead the public into expectations of involvement 

42 and re-evaluation that might not be appropriate.

43

44 Risk communication shouldn‘t have an absolute end; it can be an ongoing 

45 process. In another vein, risk managers can take the initiative to incorporate risk 

46 communication into routine functions. For example, the various microbial societies 
Microbial Risk Assessment Guideline Page 165

Draft – Do not cite, circulate, or quote

1 (most notably, in the United States, the ASM) produce informational outreach material 

2 targeting specific age groups and educational levels. These professional societies seek to 

3 expand public understanding of the role of microorganisms in human affairs beyond 

4 disease causation through staff dedicated to the effort. Agency public affairs offices 

5 might fulfill their responsibility to the public by teaming with these societies and by 

6 working with internal environmental-education staff to develop guidelines and 

7 procedures for ongoing risk communication. At least one senior public affairs manager 

8 could be tasked to work with Agency experts to identify and communicate risk specific to 

9 children, seniors, and other sensitive populations. Interaction with public health and 

10 safety agencies (the Centers for Disease Control and Prevention [CDC] or the 

11 Occupational Safety and Health Administration [OSHA]) would be advisable, as would 

12 consultation with academic and industrial clinicians. A database of these and other 

13 external consultants can be developed in anticipation of potential outbreaks.

14

15 The risk communication network just described would be expected to develop 

16 outreach materials continuously, and to establish name recognition for the responsible 

17 Agency via routine (e.g., bimonthly) public education broadcasts or activities. Such 

18 learning opportunities could be easily incorporated into local public school curricula, and 

19 ideally would be organized and distributed by agency offices nationwide. The network, 

20 once in place, can also provide informational materials to concerned individuals (and 

21 Internet blogs) or to news outlets.

22

23
Page 166 Microbial Risk Assessment Guideline

1 9. GLOSSARY

2

3 The term definitions in this glossary are from the EPA Thesaurus of Terms Used in MRA 

4 unless otherwise noted (EPA, 2007b). For the original sources of the definition, see the 

5 Thesaurus.

6

7 acceptable risk

8 This is a risk management term. The acceptability of the risk depends on scientific data, 

9 social, economic, and political factors, and on the perceived benefits arising from 

10 exposure to an agent. Tolerable risk is a synonym.

11

12 analysis of variance

13 This is a statistical technique that isolates and assesses the contribution of categorical 

14 factors to variation in the mean of a continuous outcome variable. The data are divided 

15 into categories based on their values for each of the independent variables, and the 

16 differences between the mean outcome values of these categories are tested for statistical 

17 significance. 

18

19 analysis plan

20 This is a plan that provides all the details of exactly how each part of the risk assessment 

21 will be performed. It usually describes in detail what analyses will be performed, how 

22 they will be performed, who will perform the work, schedules, resources, quality 

23 assurance/quality control requirements, and documentation requirements. 

24

25 appropriate level of protection (ALOP)

26 Codex defines ALOP as the level of protection deemed appropriate by the member 

27 (country) establishing a sanitary or phytosanitary measure to protect human, animal, or 

28 plant life or health within its territory. The term is also used more broadly to refer to risk 

29 levels selected for regulations, rules, and risk assessments.

30

31 CODEX

32 The Codex Alimentarius Commission was created in 1963 by FAO and WHO to develop 

33 food standards, guidelines and related texts such as codes of practice under the Joint 

34 FAO/WHO Food Standards Programme. The main purposes of this Programme are 

35 protecting health of the consumers and ensuring fair trade practices in the food trade, and 

36 promoting coordination of all food standards work undertaken by international 

governmental and non-governmental organizations.

45 37

38

39 conceptual model

40 Ecological risk assessment defines a conceptual model as a written description and/or a 

41 visual representation of actual or predicted relationships between humans or ecological 

42 entities and the chemicals or other stressors to which they may be exposed. ILSI (2000) 

43 states that a conceptual model depicts the purpose, defines the scope and scale, 

 

45 http://www.codexalimentarius.net/web/index_en.jsp
Microbial Risk Assessment Guideline Page 167

Draft – Do not cite, circulate, or quote

1 determines appropriate variables and identifies data needed for risk assessment. It can 

2 also serve as a preliminary or exploratory risk assessment. 

3

4 cost-benefit analysis and cost-effectiveness analysis (CEA)

5 See OMB (2003) for full descriptions of cost-benefit analysis and CEA.

6

7 data objectivity

8 Data objectivity ―focuses on whether the disseminated information is being presented in 

9 an accurate, clear, complete, and unbiased manner, and as a matter of substance, is 

accurate, reliable, and unbiased.‖(OMB, 2002)46 10

11

12 dose-response

13 This is a relationship in which a change in amount, intensity, or duration of exposure to a 

14 pathogen is associated with a change in the manifestation and magnitude of human health 

15 effects. 

16

17 dose-response assessment

18 This is the determination of the relationship between the magnitude of exposure (dose) to 

19 a chemical, biological or physical agent, and the severity and/or frequency of associated 

20 adverse health effects (response). 

21

22 dose-response curve

23 This is a graphical representation of the quantitative relationship between administered, 

24 applied, or internal dose of a chemical or agent, and a specific biological response to that 

25 chemical or agent. 

26

27 dynamic model

28 This considers the individual within a community rather than the isolated individual. 

29 Time-dependent elements such as secondary transmission, host immunity, and animal 

30 reservoirs are included. 

31

32 endpoint

33 For chemical risk assessment an endpoint is an observable or measurable biological event 

34 or chemical concentration (e.g., metabolite concentration in a target tissue) used as an 

35 index of an effect of a chemical exposure. For ecological assessment an endpoint is an 

36 explicit expression of the environmental value that is to be protected, operationally 

37 defined by an ecological entity and its attributes. For example, salmon are valued 

38 ecological entities; reproduction and age class structure are some of their important 

39 attributes. Together ―salmon reproduction and age class structure‖ form an assessment 

40 endpoint. For MRA an endpoint is usually a health effect or infected state. However 

41 indicators or conditions associated with human health could also be endpoints.

42

43

 

46 http://www.whitehouse.gov/omb/fedreg_reproducible/
Page 168 Microbial Risk Assessment Guideline

1 epidemiology triad

2 This is the traditional model of infectious disease causation. It includes three 

3 components: an external agent, a susceptible host, and an environment that brings the 

4 host and agent together, so that disease occurs. 

5

6 exposure

7 Exposure is contact made between a chemical, physical, or biological agent and the outer 

8 boundary of an organism. Exposure comprises the sources, mode, route, and extent of 

9 contact with the microbial hazard(s) of concern. How often a person is exposed is 

10 referred to as frequency of exposure. How long a person is exposed to a microbial hazard 

11 is referred to as the duration of exposure.

12

13 exposure assessment

14 This is the process of estimating or measuring the magnitude, frequency, and duration of 

15 exposure to a microbial hazard(s), along with the number and characteristics of the 

16 person or population exposed. The route of exposure is also considered.

17

18 exposure pathway

19 The exposure pathway is the physical and temporal movement of microorganisms from 

20 their source to the occurrence of an exposure. For chemicals the exposure pathway is the 

21 route a substance takes from its source (where it began) to its end point (where it ends), 

22 and how people can come into contact with (or get exposed to) it. An exposure pathway 

23 has five parts: a source of contamination (such as an abandoned business); an 

24 environmental media and transport mechanism (such as movement through 

25 groundwater); a point of exposure (such as a private well); a route of exposure (eating, 

26 drinking, breathing, or touching), and a receptor population (people potentially or 

27 actually exposed). When all five parts are present, the exposure pathway is characterized 

28 as ―complete‖, that is, capable of contributing to human health risks. 

29

30 frank pathogen

31 A microorganism capable of producing disease in both healthy and compromised 

32 persons.

33

34 hazard

35 The term hazard can be interpreted in a number of ways. It may be defined as the 

36 stressor agent capable of causing an adverse effect on the recipient or the adverse effect 

37 itself. The selection of the definition is a policy decision driven by the existing statutes, 

38 regulations, or consistency with in-house processes. Codex considers a microbiological 

39 hazard is a hazard arising from bacteria, viruses, yeasts, molds and algae, parasitic 

40 protozoa and helminthes, and their toxins or metabolites. 

41

42 hazard identification

43 This is the process of determining if data support the case for a chemical or a microbe 

44 causing adverse health effects in humans and what those effects might be. 

45

46
Microbial Risk Assessment Guideline Page 169

Draft – Do not cite, circulate, or quote

1 health effect

2 This is the clinical manifestation of disease associated with a specific pathogen, including 

3 symptomatic and asymptomatic infections, clinical illness, mortality, and sequelae. 

4

5 health endpoint

6 This is an observable or measurable biological event used as an index to determine when 

7 a deviation in the normal function of the human body occurs. 

8

9 host

10 This is a person or other living animal, including birds and arthropods, that affords 

11 subsistence or lodgment to an infectious agent under natural conditions. In an 

12 epidemiologic context, the host may be a population or a group. 

13

14 host specificity

15 This is the characteristic of a pathogen that renders it capable of infecting one or more 

16 specific hosts. 

17

18 immunocompromised

19 Immunocompromised individuals have a weakened immune system, making them more 

20 susceptible to infections than the general population. 

21

22 incubation period

23 This is the time from the moment of inoculation (exposure) to the development of the 

24 clinical manifestations of a particular infectious disease. 

25

26 indicator

27 An indicator is any biological entity or processes, or community whose characteristics 

28 show the presence of specific environmental conditions.

29

30 infectious dose

31 This is the number of organisms that make individuals ill or carriers. It should be noted 

32 that methods to count microbes may not be counting individual microorganisms. For 

33 example a colony forming unit (cfu) may be a clump of cells that formed one colony on a 

34 plate. An infectious dose is the minimum number of organisms that will result in entry 

35 through the host barriers, survival of the pathogen, and multiplication in the host.

36 Infection may or may not result in symptomatic illness. On a population basis, there is no 

37 discernible minimum infectious dose for pathogens (FAO/WHO 2003). Instead there is a 

38 probability distribution for infection associated with different dose levels reflecting intra39 and inter-individual variability in the pathogen-host relationship.

40

41 infectivity

42 Infectivity describes the ability of a pathogen to enter, survive and multiply (infect) a 

43 host.

44

45
Page 170 Microbial Risk Assessment Guideline

1 microorganism

2 These are viruses, bacteria, yeasts and simple fungi, single-celled algae, protozoa, all are 

3 defined as being organisms that can only be seen with the aid of a microscope. Most are 

4 beneficial but some produce disease (pathogens). Non-pathogenic microorganisms are 

5 critical for recycling energy and nutrients globally, such as in soil, the oceans,

6 composting and sewage secondary treatment. 

7

8 pathogen

9 These are microorganisms (e.g., viruses, bacteria, protozoa and the ova of helminth

10 parasites) that can cause disease in humans, animals and plants. 

11

12 pathogenicity

13 Pathogenicity refers to the ability of an organism to cause disease (i.e., harm the host).

14 This ability represents a genetic component of the pathogen and the overt damage done to 

15 the host is a property of the host-pathogen interactions. Commensals and opportunistic 

16 pathogens lack this inherent ability to cause disease. However, disease is not an 

17 inevitable outcome of the host-pathogen interaction and, furthermore, pathogens can 

18 express a wide range of virulence. Virulence, a term often used interchangeably with 

19 pathogenicity, refers to the degree of pathology caused by the organism. The extent of 

20 the virulence is usually correlated with the ability of the pathogen to multiply within the 

21 host and may be affected by other factors (.i.e, conditional). In summary, an organism 

22 (species or strain) is defined as being pathogenic (or not), and depending upon 

23 conditions, may exhibit different levels of virulence.

24

25 Pathogenicity is the quality or state of being pathogenic, the potential ability to produce 

26 disease. Virulence is the disease producing power of an organism, the degree of 

pathogenicity within a group or species.

47 27

28

29 planning and scoping

30 This is the process that defines the purpose and scope of a risk assessment and focuses 

31 the issues and approach(es) involved in performing the assessment.

32

33 problem formulation

34 In ecological risk assessment, problem formulation is the initial stage of a risk assessment 

35 where the purpose of the assessment is articulated, assessment endpoints and a 

36 conceptual model are developed, and a plan for analyzing and characterizing risk is 

37 determined. 

38 In microbial assessment, problem formulation is a systematic planning step that identifies 

39 the goals, breadth, and focus of the MRA, the regulatory and policy context of the 

40 assessment, and the major factors that will need to be addressed for the assessment. 

41

42 qualitative risk assessment 

43 Qualitative risk assessment uses verbal descriptors of risk and severity as well as 

44 uncertainty, and often involves the aggregation of expert opinions. The results are often 

 

47 For further discussion of these terms visit 

http://scienceblogs.com/effectmeasure/2006/06/pathogenicity_virulence_transm.php
Microbial Risk Assessment Guideline Page 171

Draft – Do not cite, circulate, or quote

1 stated in an estimated range, such as ―there is a moderate to high risk of a certain 

2 outcome occurring.‖ 

3

4 quality-adjusted life year

5 This is a unit of health care outcomes that adjusts gains (or losses) in years of life 

6 subsequent to a health care intervention by the quality of life during those years. QALYs 

7 can provide a common unit for comparing cost-utility across different interventions and 

8 health problems. Other units for measuring health outcomes include DALYs and 

9 healthy-years equivalents (HYEs). 

10

11 quantitative risk assessment

12 In quantitative assessments, the risk is expressed as a mathematical statement of the 

13 chance of illness or death after exposure to a specific hazard, and it represents the 

14 cumulative probabilities of certain events happening and the uncertainty associated with 

15 those events.

16

17 risk analysis

18 A process consisting of three components: risk assessment, risk management and risk 

19 communication.

20

21 risk assessment

22 In the context of human health, risk assessment is a systematic way to prepare and 

23 organize information and help establish programs, R&D, and regulatory priorities; the 

24 qualitative or quantitative characterization of the potential health effects of particular 

25 substances on individuals or populations; a scientifically based process consisting of the 

26 following steps: (i) hazard identification, (ii) hazard characterization, (iii) exposure 

27 assessment, and (4) risk characterization; the formal, scientifically based process to 

28 estimate the likelihood (probability) of exposure to a hazard and the resulting public 

29 health impact from this exposure. The product of the risk assessment is often a statement 

30 regarding the probability that populations or individuals so exposed will be harmed and 

31 to what degree (risk characterization). 

32

33 risk characterization

34 In risk characterization, risk due to a particular exposure for a defined population is 

35 described in coherent, understandable, and informative conclusions about the 

36 microbiological risk to exposed humans in a way that is useful for decision makers as 

37 well as stakeholders in that risk. In all cases, major issues and uncertainty and variability 

38 associated with determining the nature and extent of the risk should be identified and 

39 discussed. The risk characterization should be prepared in a manner that is clear, 

40 transparent, reasonable, and consistent. 

41

42 risk communication

43 This is the exchange of information about health or environmental risks among risk 

44 assessors and managers, the general public, news media, and other stakeholders. WHO 

45 considers risk communication to be the interactive exchange of information and opinions 
Page 172 Microbial Risk Assessment Guideline

1 concerning risks and risk management among risk assessors, risk managers, consumers, 

2 and other interested parties (WHO, 2000).

3

4 risk management

5 In the context of human health, risk management is a decision making process that 

6 accounts for political, social, economic and engineering implications together with risk7 related information in order to develop, analyze and compare management options and 

8 select the appropriate managerial response to a potential chronic health hazard. 

9

10 risk profile

11 Developing a risk profile involves an initial systematic collection of information, which 

12 is evaluated to determine what other actions (including a MRA) and resources may be 

13 needed. The risk profile is an overall summary of the context in which a risk is being 

14 analyzed, including: a description of the risk(s) considered, values threatened by the risk, 

15 social perception of the risk, who benefits from producing the risk, who benefits from 

16 managing the risk, and characteristics of the risk, the risk-producer and the risk-bearer, 

17 which are pertinent to successful management of the risk.

18

19 secondary transmission

20 This is the direct or indirect propagation of a pathogen from an infected person (with or 

21 without clinical illness) to additional people.

22

23 sensitivity analysis

24 Sensitivity analysis examines the relative influence and importance of a model‘s inputs 

25 on its output measuring the  ̳relative‘ influence. It is the process of changing one variable 

26 while leaving the others constant to determine its effect on the output. This procedure 

27 fixes each uncertain quantity at its credible lower and upper bounds (holding all others at 

28 their nominal values, such as medians) and computes the results of each combination of 

29 values. The results help to identify the variables that have the greatest effect on exposure 

30 estimates and help focus further information gathering efforts. 

31

32 sequelae

33 These are abnormal conditions that arise following the acute phase of a disease. For 

34 example, kidney failure may follow acute E. coli O157:H7 infection.

35

36 stakeholder

37 A stakeholder is any organization, governmental entity, or individual that will be 

38 responsible for implementing, or financially affected by, new rules and regulations borne 

39 of the risk assessment, or may be impacted by a given decision based on the risk 

40 assessment.

41

42 subpopulation

43 This is a subset of the target population that has been identified for a specific purpose, 

44 usually requires the ability to estimate an attribute of the subpopulation. 

45

46
Microbial Risk Assessment Guideline Page 173

Draft – Do not cite, circulate, or quote

1 susceptible, sensitive, vulnerable

2 These terms refer to individuals or populations that for varying reasons suffer more 

3 severe consequences than the general population as a result of exposure to a hazard. 

4 Although these terms are used interchangeably by many risk assessors and public health 

5 experts, the Interagency Risk Assessment Consortium Susceptible Populations Workshop 

6 considered the below definitions, but emphasized that when these terms are used, they 

should be defined.48 7

8 Susceptibility is: A capacity leading to higher risk at a given exposure level, due to

9 biological (intrinsic) factors that can modify the effect of a specific exposure

10 Sensitivity is: A capacity for higher risk due to the combined effect of susceptibility

11 (biological factors) and differences in exposure

12 Susceptibility - Includes intrinsic factors only; Characteristic of an individual; Defined by 

13 the host

14 Vulnerability - Includes intrinsic and extrinsic factors; Characteristic of an individual or a 

group; Defined by the host (behavior) and environment49 15

16

17 transparency

18 This is conducting a risk assessment in such a manner that all of the scientific analyses, 

19 uncertainties, assumptions, and science policies which underlie the decisions made 

20 throughout the risk assessment are clearly stated (i.e., made readily apparent). For risk 

21 assessment to be transparent, methods, and assumptions should be clearly stated and 

22 understandable to the intended audience, so that the audience is able to evaluate the 

23 adequacy of the data and methods. 

24

25 uncertainty analysis

26 This is used to estimate the uncertainty associated with model inputs, assumptions, and 

27 structure/form and the process of interpreting the influence of uncertainty on the results

28 of a risk assessment.

29

30 uncertainty factor

31 These are usually applied to accommodate for a lack of knowledge associated with inter32 species extrapolation, high to low dose extrapolation (i.e., effect to no-effect), population 

33 variation (i.e., protection of sensitive subpopulations), and extrapolation across exposure 

34 durations (e.g., subchronic to chronic.) Although uncertainty factors are commonly 

35 applied in chemical risk assessment, much less information is available supporting the 

36 application of uncertainty factors to microbiological risk assessment. 

37

38 uncertainty

39 Uncertainty is imperfect knowledge of the microbiological hazard (e.g., its virulence), 

40 environmental pathway/processes, or the human populations under consideration (from 

41 MRA)

 

48 http://foodrisk.org/IRAC/events/2010-01-10/downloads/Concept_of_Susceptibility-R_Parkin.pdf

49 In this document vulnerability is also used in the context of a ―vulnerability assessment,‖ which is not 

related to the definitions discussed in this set of terms. Refer to Section 2.5.3 for the definition of 

vulnerability with respect to the CARVER method.
Page 174 Microbial Risk Assessment Guideline

1 Uncertainty represents a lack of knowledge about factors affecting risk assessments and 

2 can lead to inaccurate or biased estimates of risk and hazard. Some of the types of 

3 uncertainty include scenario uncertainty, parameter uncertainty, and model uncertainty. 

4

5 variability

6 This refers to the observed differences attributable to true heterogeneity or diversity in a 

7 parameter. Examples include human physiological variation (e.g., natural variation in 

8 body weight, height, breathing rate, drinking water intake rate), weather variability, 

9 variation in soil types and differences in contaminant concentrations in the environment. 

10 Variability is usually not reducible by further measurement of study, but it can be better 

11 characterized.

12

13 virulence

14 This is the degree of intensity of the disease produced by a microorganism as indicated 

15 by its ability to invade the tissues of a host and the ensuing severity of illness. (see 

16 pathogenicity for comparison)

17

18

19

20
Microbial Risk Assessment Guideline Page 175

Draft – Do not cite, circulate, or quote

1 10. ABBREVIATIONS

2

3 ACSSuT Ampicillin, chloramphenicol, streptomycin, sulfonamides, and tetracycline

4 AFLP Amplified fragment length polymorphism

5 AIPH Army Institute of Public Health

6 ALOP Appropriate level of protection 

7 AOAC Association of Analytical Communities 

8 APHIS Animal and Plant Health Inspection Service (U.S. Department of 

9 Agriculture)

10 ASM American Society for Microbiology

11 Bcc Burkholderia cepacia complex

12 BenMAP Environmental Benefits Mapping and Analysis program

13 BSE Bovine Spongiform Encephalopathy

14 °C degrees Celsius

15 CART Classification and regression tree

16 CARVER Criticality, Accessibility, Recuperability, Vulnerability, Effect, and 

17 Recognizability

18 CDC Centers for Disease Control

19 CDER Center for Drug Evaluation and Research (Food and Drug Administration)

20 CEA cost-effectiveness analyses

21 CFSAN Center for Food Safety and Applied Nutrition

22 CFR Code of Federal Regulations

23 cfu colony forming unit

24 DALYs Disability-adjusted life years

25 DHS Department of Homeland Security

26 DNA Deoxyribonucleic Acid

27 DOD Department of Defense

28 DT104 Definitive type 104 (p48)

29 ECSSC European Commission Scientific Steering Committee 

30 EPA Environmental Protection Agency

31 °F degrees Fahrenheit

32 FAO Food and Agriculture Organization

33 FDA Food and Drug Administration 

34 FIRRM Foodborne Illness Risk Ranking Model 

35 FSIS Food Safety and Inspection Service

36 GAO U.S. Government Accountability Office

37 GRAS generally recognized as safe

38 gyrB DNA gyrase, subunit B

39 HACCP Hazard Analysis Critical Control Point

40 HC Hazard Characterization

41 HI Hazard Identification

42 HIV Human immunodeficiency virus

43 HYEs Healthy-years equivalents

44 ID50 Median Infectious dose 

45 ILSI International Life Science Institute

46 LD50 Median lethal dose
Page 176 Microbial Risk Assessment Guideline

1 LT2ESWTR Long Term 2 Enhanced Surface Water Treatment Rule

2 MAC Mycobacterium Avium-Complex 

3 MCMC Markov Chain Monte Carlo Simulation

4 MILYs Morbidity Inclusive Life Years

5 MLGT Multilocus genotype sequencing 

6 MLST Multilocus sequence typing 

7 MRA microbial risk assessment

8 MRM microbial risk management

9 mRNA messenger ribonucleic acid

10 NASA National Aeronautics and Space Administration

11 NCEZID National Center for Emerging and Zoonotic Infectious Diseases

12 NCRP National Committee on Radiation Programs

13 NIOSH National Institute for Occupational Safety and Health (Centers for Disease 

14 Control)

15 NN Neural Networks

16 NOAEL No observable adverse effect level

17 NRC National research council

18 OAQPS Office of Air Quality Planning and Standards

19 OECD Organization for Economic Cooperation and Development

20 OMB Office of Management and Budget

21 OPP Office of Pesticide Programs (US Environmental Protection Agency)

22 OPPT Office of Pollution Prevention and Toxics (US Environmental Protection 

23 Agency)

24 ORD Office of Research and Development (US Environmental Protection 

25 Agency)

26 OSA Office of the Senior Advisor (US Environmental Protection Agency)

27 OW Office of Water (US Environmental Protection Agency)

28 P/CC Presidential/Congressional Commission

29 PCR Polymerase chain reaction

30 PCR-RFLP Polymerase chain reaction-restriction fragment length polymorphism

31 PFGE Pulsed field gel electrophoresis

32 PM2.5 Particulate Matter with a diameter smaller than 2.5 microns

33 QALYs Quality-adjusted life years

34 RAF Risk Assessment Forum (US Environmental Protection Agency)

35 RAPD Random amplification of polymorphic DNA

36 rDNA Ribosomal deoxyribonucleic acid

37 REP-PCR Repetitive element polymerase chain reaction

38 R&D Research and development

39 SARS Severe Acute Respiratory Syndrome

40 SDWA Safe Drinking Water Act

41 SNP Single nucleotide polymorphism

42 TAMU Texas A&M University

43 TB Tuberculosis

44 TCCR Transparency, Clarity, Consistency, and Reasonableness

45 U.S. United States

46 USDA U.S. Department of Agriculture
Microbial Risk Assessment Guideline Page 177

Draft – Do not cite, circulate, or quote

1 VBNC Viable-but-not-culturable 

2 vCJD variant Creutzfeldt-Jacob Disease

3 VOI Value of information

4 WHO World Health Organization

5
Page 178 Microbial Risk Assessment Guideline

1 11. REFERENCES

2

3 Abromowitz, M. and I.A. Stegun. (1964) Handbook of Mathematical Functions with 

4 Formulas, Graphs, and Mathematical Tables. National Bureau of Standards Applied 

5 Mathematics Series – 55. 

6

7 Airoldi, M. and A. Morton. (2009) Adjusting life for quality or disability: stylistic 

8 difference or substantial dispute? Health Economics 18(11):1237-1247.

9

10 Akkina, J.E., Hogue, A.T., Angule, F.J., Johnson, R., Peterson, K.E., Saini, P.K., 

11 Fedorka-Cray, P.J., and W.D. Schlosser. (1999) Epidemiologic aspects, control, and 

12 importance of multiple-drug resistant Salmonella Typhimurium DT104 in the United 

13 States. Journal of American Veterinary Medical Association 214:790-798.

14

15 Alexopoulos, C.J., C.W. Mims, and M. Blackwell. (1996) Introductory Mycology, 4th 

16 Edition. John Wiley & Sons, Inc, Hoboken, N.J.

17

18 Anderson, R.M. and R. May. (1991) Infectious Diseases of Humans: Dynamics and

19 Control. New York: Oxford University Press.

20

21 Anderson, Y.S., Gillin, F.D., and Eckmann, L. (2006) Adaptive Immunity-Dependent 

22 Intestinal Hypermotility Contributes to Host Defense against Giardia spp. Infection and 

23 Immunity 74(4):2473–2476.

24 http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1418922/pdf/1556-05.pdf

25

26 Association of Analytical Communities (AOAC International). (2007) Official Methods 

27 of Analysis. Eighteenth edition. Revision 2. 

28

29 Armstrong, T.W. and C.N. Haas. (2008) Legionnaires‘ disease: Evaluation of a 

30 quantitative microbial risk assessment model. Journal of Water and Health 6(2):149-166.

31

32 Ashbolt, N.J., Schoen, M.E., Soller, J.A. and D.J. Roser. (2010) Predicting pathogen risks 

33 to aid beach management: the real value of quantitative microbial risk assessment 

34 (QMRA). Water Research 44(16) 4692-4703.

35

36 Bartrand, T.A., Weir, M.H., and C.N. Haas. (2008) Dose-response models for inhalation 

37 of Bacillus anthracis spores: Interspecies comparisons. Risk Analysis 28(4):1115-24.

38

39 Battista, J.R. and A.M. Earl. (2004) Mutagenesis and DNA Repair. In R.V. Miller and 

40 M.J. Day, Editors. Microbial Evolution: Gene Establishment, Survival, and Exchange.

41 Washington, DC: ASM Press, pp. 3-11.

42

43 Batz. (2004) Identifying the most significant microbiological foodborne hazards to public

44 health: a new risk-ranking model. Food Safety Research Consortium, Discussion Paper 

45 Series, Number 1, September 2004. 

46 http://www.rff.org/RFF/Documents/FRSC-DP-01.pdf
Microbial Risk Assessment Guideline Page 179

Draft – Do not cite, circulate, or quote

1

2 Bernardo, J.M. and A.F.M. Smith. (1994) Bayesian theory. Statistical Methods and 

3 Applications 3(1):155-160. Also in paperback book (Bayesian Theory - Wiley Series in 

4 Probability and Statistics - April 24, 2000).

5

6 Berry, Donald A. (1996) Statistics: A Bayesian Perspective. Belmont, CA: Duxbury 

7 Press, Wadworth Publ. Co.

8

9 Bisson, I.-A., Marra, P.P., Burtt E.H., Sikaroodi, M, and P. M. Gillevet. (2007) A 

10 molecular comparison of plumage and soil bacteria across biogeographic, ecological, and 

11 taxonomic scales. Microbial Ecology 54:65-81.

12

13 Blaser, M.J. and D. Kirschner. (1999) Dynamics of Helicobacter pylori colonization in 

14 relation to the host response. Proceedings of the National Academy of Sciences

15 96(15):8359-8364.

16

17 Blaser, M.J., and D. Kirschner. (2007) The equilibria that allow bacterial persistence in 

18 human hosts. Nature 449(7164):843-849.

19

20 Boerlijst, M.C., Bonhoeffer, S., and M.A. Nowak. (1996) Viral quasi-species and 

21 recombination. Proceedings: Biological Sciences 263(1376):1577-1584.

22

23 Bollaerts, K., Aerts, M., Faes, C., Grijspeerdt, K., Dewulf, J., and K. Mintiens. (2008)

24 Human salmonellosis: Estimation of dose-illness from outbreak data. Risk Analysis

25 28(2):427-440.

26

27 Bogosian, G., Morris, P.J.L., and J.P. O‘Neil. (1998) A mixed culture recovery method 

28 indicates that enteric bacteria do not enter the viable but nonculturable state.

29 Applied Environmental Microbiology, 64(5):1736-1742.

30

31 Bogosian, B.J., and E.V. Bourneuf. (2001) A matter of bacteria life and death. EMBO 

32 Reports 2(9):770-774.

33

34 Brynestad, S., Braute, L., Luber, P., and E. Bartelt. (2008) Quantitative microbiological 

35 risk assessment of campylobacteriosis cases in the German population due to 

36 consumption of chicken prepared in homes. International Journal of Risk Assessment and 

37 Management 8(3):194-213.

38

39 Carlin, B.P. and T.A. Louis. (2001) Bayes and Empirical Bayes Methods for Data 

40 Analysis. Second edition. New York: Chapman and Hall.

41

42 Center for Food Safety and Applied Nutrition (CFSAN). (2001) Bacteriological 

43 Analytical Manual (BAM). CFSAN, FDA. http://www.cfsan.fda.gov/~ebam/bam44 toc.html

45
Page 180 Microbial Risk Assessment Guideline

1 Center for Food Safety and Applied Nutrition (CFSAN). (2006) The Bad Bug Book:

2 Foodborne Pathogenic Organisms and Natural Toxins Handbook. Washington, DC: FDA. 

3 http://vm.cfsan.fda.gov/~mow/intro.html

4

5 Cleaveland, S., Laurenson, M.K., and L.H. Taylor. (2001) Diseases of humans and their 

6 domestic mammals: pathogen characteristics, host range, and the risk of emergence, 

7 Philosophical Transactions of the Royal Society B 356:991–999.

8

9 Codex (Codex Alimentarius Commission). (1999) Principles and Guidelines for the 

10 Conduct of Microbial Risk Assessment, CAC/GL-30. 

11 http://www.who.int/foodsafety/publications/micro/cac1999/en/

12 www.codexalimentarius.net/download/standards/357/CXG_030e.pdf

13

14 Codex (Codex Alimentarius Commission). (2007a) Principles and Guidelines for the 

15 Conduct of Microbiological Risk Management. CAC/GL 63-2007

16 http://www.codexalimentarius.net/web/more_info.jsp?id_sta=10741

17

18 Codex (Codex Alimentarius Commission). (2007b) Working Principles for Risk Analysis 

19 for Food Safety for Application by Governments. CAC/GL 62-2007.

20 ftp://ftp.fao.org/docrep/fao/010/a1550t/a1550t00.pdf

21

22 Coleman, M. and H. Marks. (2000) Mechanistic modeling of salmonellosis. Quantitative 

23 Microbiology 2:227-247.

24

25 Coleman, M.E., Marks, H.M., Golden, N.J., and H.K. Latimer. (2004) Discerning strain 

26 effects in microbial dose-response data. Journal of Toxicology and Environmental

27 Health, Part A: Current Issues 67:667-685.

28

29 Covacci, A., and R. Rappuoli. (1998) Helicobacter pylori: Molecular evolution of a 

30 bacterial quasi-species. Current Opinion in Microbiology 1(1):96-102.

31

32 Cox, L.A. (2006) Quantitative Health Risk Analysis Methods: Modeling the Human 

33 Health Impacts of Antibiotics Used in Food Animals. New York: Springer Science.

34

35 Crabtree, K.D., Gerba, C.P., Rose, J.B., and C.N. Haas. (1997) Waterborne adenovirus: A

36 risk assessment. Water Science and Technology 35(11-12):1-6.

37

38 Craun, G.F., Brunkard, J.M., Yoder, J.S., Roberts, V.A., Carpenter, J., Wade, T., 

39 Calderon, R.L., Roberts, J.M., Beach, M.J. and S.L. Roy. (2010) Causes of outbreaks 

40 associated with drinking water in the United States from 1971 to 2006. Clinical 

41 Microbiological Reviews 23(3), 507-528.

42

43 Cullen, A.C. (1999) Probabilistic Techniques in Exposure Assessment: A Handbook for 

44 Dealing with Variability and Uncertainty in Models and Inputs. New York: Springer, 352 

45 pp.

46
Microbial Risk Assessment Guideline Page 181

Draft – Do not cite, circulate, or quote

1 de Koning, H.W. (1987) Setting Environmental Standards: Guidelines for Decision2 Making. World Health Organization, Geneva, Switzerland.

3

4 Dennis, S.B., Kause, J., Losikoff, M., Engeljohn, D.L., and Buchanan, R. (2008) Chapter 

5 5. Using Risk Analysis for Microbial Food Safety Regulatory Decision-Making, pp. 137-

6 175 In: Microbial Risk Analysis of Foods, Series Editor: Michael P. Doyle; Editor: 

7 Donald Schaffner. ASM Press, Washington, DC (978-1-55581-461-8).

8

9 Disney, W.T., and M.A. Peters. (2003) Simulation modeling to derive the value of 

10 information for risky animal-disease import decisions. Preventive Veterinary Medicine

11 61:171-184.

12

13 Dobrindt, U., Hochhut, B., Hentschel, U., and J. Hacker. (2004) Genomic islands in 

14 pathogenic and environmental microorganisms. Nature Reviews Microbiology 2:414-424. 

15

16 Dufour, A.P. (1984) Health Effects Criteria for Fresh Recreational Waters. U.S. 

17 Environmental Protection Agency, Cincinnati, OH. EPA-600/1-84-004. 

18 http://www.epa.gov/microbes/frc.pdf

19

20 Dwyer, J., Picciano, M.F, and D.J. Raiten. (2003) Future directions for the integrated 

21 CSFII-NHANES: What we eat in America–NHANES. Journal of Nutrition 133:576S22 581S.

23

24 Ebel, E.D., Schlosser, W.D., Orloski, K., Kause, J., Roberts, T., Narrod, C., Malcolm, S., 

25 Coleman, M., and M. Powell. (2003) A Risk Assessment of Escherichia coli O157:H7 in 

26 Ground Beef. In Torrence, M.E. and Isaacson, R.E. (eds.) Microbial Food Safety in 

27 Animal Agriculture: Current Topics. Ames, Iowa: Iowa State Press.

28

29 Englehardt, J.D. (2004) Predictive Bayesian dose-response assessment for appraising 

30 absolute health risk from available information. Human and Ecological Risk Assessment

31 10:69-78.

32

33 Englehardt, J.D. and J. Swartout. (2004) Predictive population dose-response assessment 

34 for Cryptosporidium parvum: Infection endpoint. Journal of Toxicology and 

35 Environmental Health, Part A: Current Issues 67(8-10):651-666.

36

37 Englehardt, J.D. and J. Swartout. (2006) Predictive Bayesian microbial dose-response 

38 assessment based on suggested self-organization in primary illness response: C. parvum.

39 Risk Analysis 26(2):543-554.

40

41 Environmental Protection Agency (EPA). (1989) National Primary Drinking Water 

42 Regulations: Filtration, Disinfection, Turbidity, Giardia lamblia, Viruses, Legionella, and 

43 Heterotrophic Bacteria. Final Rule. Federal Register 54(124):27486. [Also known as the 

44 surface water treatment rule (SWTR)]

45
Page 182 Microbial Risk Assessment Guideline

1 Environmental Protection Agency (EPA). (1992) Framework for Ecological Risk 

2 Assessment. EPA/630/R-92/001.

3 http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=30759&partner=ORD_NCEA

4 Note: This Framework served as the foundation for, and has been superseded by, EPA's 

5 1998 Ecological Risk Assessment Guidelines.

6

7 Environmental Protection Agency (EPA). (1994) Methods for Derivation of Inhalation 

8 Reference Concentrations and Application of Inhalation Dosimeter. EPA/600/8-90/066F.

9

10 Environmental Protection Agency (EPA). (1997a) Exposure Factors Handbook. Office of 

11 Research and Development, National Center for Environmental Assessment. 

12 Washington, DC. EPA/600/P-95/002Fa.

13

14 Environmental Protection Agency (EPA). (1997b) Guiding Principles for Monte Carlo 

15 Analysis. EPA/630/R-97/001.

16

17 Environmental Protection Agency (EPA). (1998a) Guidelines for Ecological Risk 

18 Assessment. EPA/630/R095/002F. EPA, Washington, DC.

19 http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF

20

21 Environmental Protection Agency (EPA). (1998b) Giardia: Human Heath Criteria 

22 Document. EPA-823-R-002.

23

24 Environmental Protection Agency (EPA). (1999) Giardia: Drinking Water Health 

25 Advisory. EPA-822-R-99-008.

26

27 Environmental Protection Agency (EPA). (2000a) Risk Characterization Handbook. 

28 EPA-100-B-00-002. EPA, Washington, DC. http://www.epa.gov/osa/spc/2riskchr.htm

29

30 Environmental Protection Agency (EPA). (2000b) Estimated Per Capita Water Ingestion 

31 in the United States. Publication No. EPA-822-00-008, 1-208. U.S. Government Printing 

32 Office, Washington, DC.

33 http://www.epa.gov/waterscience/criteria/drinking/percapita/index.html

34

35 Environmental Protection Agency (EPA). (2000c) Methodology for Deriving Ambient

36 Water Quality Criteria for the Protection of Human Health. EPA-822-B-00-004.

37 http://www.epa.gov/waterscience/humanhealth/method/complete.pdf

38

39 Environmental Protection Agency (EPA). (2000d) EPA Science Policy Council Peer 

40 Review Handbook. Second Edition. Washington, DC. EPA-100-B-00-001. 

41

42 Environmental Protection Agency (EPA). (2002a) Lessons Learned on Planning and 

43 Scoping for Environmental Risk Assessments. EPA Science Policy Council, Washington, 

44 DC.

45
Microbial Risk Assessment Guideline Page 183

Draft – Do not cite, circulate, or quote

1 Environmental Protection Agency (EPA). (2002b) Guidelines for Ensuring and 

2 Maximizing the Quality, Objectivity, Utility, and Integrity, of Information Disseminated 

3 by the Environmental Protection Agency. EPA/260R-02-008. 

4 http://www.epa.gov/quality/informationguidelines/documents/EPA_InfoQualityGuideline

5 s.pdf

6

7 Environmental Protection Agency (EPA). (2003a) Framework for Cumulative Risk 

8 Assessment. Office of Research and Development, National Center for Environmental 

9 Assessment, Washington, DC. EPA/630/P-02/001F. 

10 http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=54944

11

12 Environmental Protection Agency (EPA). (2003b) Assessment Factors. Science Policy 

13 Council. EPA 100/B-03/001.

14 http://www.epa.gov/spc/pdfs/assess2.pdf

15

16 Environmental Protection Agency (EPA). (2004a) An Expert Judgment Assessment of 

17 the Concentration-Response Relationship Between PM2.5 Exposure and Mortality. Office 

18 of Air Quality Planning and Standards (OAQPS) Report and Peer Review of the Report.

19 Report: http://www.epa.gov/ttn/ecas/regdata/Benefits/pmexpert.pdf

20 Peer review of the report:

21 http://www.epa.gov/ttn/ecas/regdata/Benefits/memo_7.30.04.pdf

22

23 Environmental Protection Agency (EPA). (2004b) An Examination of EPA Risk 

24 Assessment Principles and Practices. Office of the Science Advisor Staff Paper. 

25 EPA/100/B-04/001. http://www.epa.gov/OSA/pdfs/ratf-final.pdf

26

27 Environmental Protection Agency (EPA). (2004c) Water quality standards for coastal and 

28 great lakes recreation waters final rule. Federal Register 69(220):67217-43.

29

30 Environmental Protection Agency (EPA). (2004d) Risk Assessment Guidance for 

31 Superfund Volume I: Human Health Evaluation Manual (Part E, Supplemental Guidance 

32 for Dermal Risk Assessment) Final (RAGS-E). EPA/540/R/99/005.

33

34 Environmental Protection Agency (EPA). (2005a) Guidance on Selecting Age Groups for 

35 Monitoring and Assessing Childhood Exposures to Environmental Contaminants. 

36 National Center for Environmental Assessment, Washington, DC. EPA/630/P-03/003F.

37 http://cfpub.epa.gov/ncea/CFM/recordisplay.cfm?deid=146583.

38

39 Environmental Protection Agency (EPA). (2005b) Rule to reduce interstate transport of 

40 fine particulate matter and ozone (Clean Air Interstate Rule); revisions to acid rain 

41 program; revisions to the NOX SIP call; final rule. Federal Register 70(91):25162-

42 25405. http://www.epa.gov/air/interstateairquality/rule.html.

43

44 Environmental Protection Agency (EPA). (2006a) National primary drinking water 

45 regulations: Long term 2 enhanced surface water treatment rule - final. Federal Register 

46 71(3):654-785. 
Page 184 Microbial Risk Assessment Guideline

1

2 Environmental Protection Agency (EPA). (2006b) Summary of NCEA Colloquium on 

3 Current Use and Future Needs of Genomics in Ecological and Human Health Risk 

4 Assessment. EPA/600/R-04/039F. 

5 http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=149984.

6

7 Environmental Protection Agency (EPA). (2006c) EPA Science Policy Council Peer 

8 Review Handbook. Third edition EPA/100/B-06/002. Washington, DC.

9

10 Environmental Protection Agency (EPA). (2007a) Pesticides; Data Requirements for 

11 Biochemical and Microbial Pesticides. Federal Register 72 (207):60988-61025. October 

12 26, 2007. 

13

14 Environmental Protection Agency (EPA). (2007b) Thesaurus of Terms Used in Microbial 

15 Risk Assessment. 

16 http://www.epa.gov/waterscience/criteria/humanhealth/microbial/thesaurus/

17

18 Environmental Protection Agency (EPA). (2007c) Compendium of Prior and Current 

19 Microbial Risk Assessment Methods for Use as a Basis for the Selection, Development, 

20 and Testing of a Preliminary Microbial Risk Assessment Framework. EPA/600/R21 07/129. National Homeland Security Research Center. 

22 http://www.epa.gov/NHSRC/pubs/600r07129.pdf.

23

24 Environmental Protection Agency (EPA). (2008) Scientific and Ethical Approaches for 

25 Observational Exposure Studies. EPA 600/R-08/062. 

26 http://www.epa.gov/nerl/sots/SEAOES_doc20080707.pdf

27 Environmental Protection Agency (EPA). (2009a) Protocol for Microbial Risk 

28 Assessment. Draft July 30, 2009. Office of Science and Technology, Office of Water, 

29 Washington, DC. and EPA Science Advisory Board Review of EPA's Microbial Risk 

30 Assessment Protocol (EPA-SAB-10-008)

31 http://yosemite.epa.gov/sab/sabproduct.nsf/7FAA3A556A92CF21852576160064DEC2/$

32 File/Draft+MRA+Protocol+July+30+2009+for+DWC+Sept+21-22+2009+Meeting.pdf

33 http://yosemite.epa.gov/sab/sabproduct.nsf/07322F6BB8E5E80085257746007DC64F/$F

34 ile/EPA-SAB-10-008-unsigned.pdf

35

36 Environmental Protection Agency (EPA). (2009b) Using Probabilistic Methods to 

37 Enhance the Role of Risk Analysis in Decision-Making with Case Study Examples. 

38 EPA/100/R-09/001

39 http://www.epa.gov/osa/spc/expertelicitation/index.htm

40

41 Environmental Protection Agency (EPA). (2009c) DRAFT Expert Elicitation Task Force 

42 White Paper. January 6, 2009 External Review Draft.

43 http://www.epa.gov/osa/spc/expertelicitation/index.htm

44

45 Environmental Protection Agency (EPA). (2010) Integrated Risk Information System 

46 Glossary. Accessed October 1, 2010
Microbial Risk Assessment Guideline Page 185

Draft – Do not cite, circulate, or quote

1 http://www.epa.gov/ncea/iris/help_gloss.htm

2

3 European Commission, Scientific Steer Committee (ECSSC). (2003) Risk Assessment of 

4 Food Borne Bacterial Pathogens: Quantitative Methodology Relevant for Human 

5 Exposure. Final report. SSC Task Force Report on Harmonization of Risk Assessment 

6 Procedures.

7

8 Executive Order 12866. (1993) Executive Order 12866 of September 30, 1993, 

9 Regulatory Planning and Review. Federal Register 58 (190).

10 http://www.reginfo.gov/public/jsp/Utilities/EO_12866.pdf

11

12 Farber, J.M., Ross, W.H., and J. Harwig. (1996) Health risk assessment of Listeria 

13 monocytogenes in Canada. International Journal of Food Microbiology 30(1-2):145-156.

14

15 Food and Agriculture Organization/World Health Organization (FAO/WHO). (1997)

16 Risk Management and Food Safety. Report of a Joint FAO/WHO Consultation Rome, 

17 Italy, 27 to 31 January 1997, FAO Food and Nutrition Paper Number 65. 

18 ftp://ftp.fao.org/docrep/fao/w4982e/w4982e00.pdf

19

20 Food and Agriculture Organization/World Health Organization (FAO/WHO). (2003) 

21 Hazard Characterization for Pathogens in Food and Water, Guidelines. Microbiological 

22 Risk Assessment Series 3. 

23 http://www.fao.org/docrep/006/y4666e/y4666e00.htm

24 http://www.cepis.org.pe/bvsacg/e/cd-cagua/guias/c.referencias/07.pathogen.pdf

25

26 Food and Agriculture Organization/World Health Organization (FAO/WHO). (2004) 

27 Risk Assessment of Listeria monocytogenes in Ready-to-Eat Foods.

28 ftp://ftp.fao.org/es/esn/jemra/mra4_en.pdf

29

30 Food and Agriculture Organization/World Health Organization (FAO/WHO). (2006) 

31 Food Safety Risk Analysis: A Guide for National Food Safety Authorities. FAO: Food 

32 and Nutrition Paper 87. ftp://ftp.fao.org/docrep/fao/009/a0822e/a0822e00.pdf

33

34 Food and Agriculture Organization/World Health Organization (FAO/WHO). (2009) 

35 Risk Characterization of Microbiological Hazards in Food: Guidelines. Microbiological 

36 Risk Assessment Series17.

37 ftp://ftp.fao.org/ag/agn/jemra/MRA17_05.10.09_f.pdf

38

39 Food and Drug Administration (FDA). (1999) A Defined-Risks Approach to the 

40 Regulatory Assessment of the Use of Neoplastic Cells as Substrates for Viral Vaccine 

41 Manufacture. Developed by Andrew M. Lewis Jr., Philip Krause, and Keith Peden for the 

42 Cell Substrate-Adventitious Agent Working/Interest Group. 

43 http://www.fda.gov/cber/minutes/brief091499.pdf

44

45 Food and Drug Administration (FDA). (2002) Initiation and Conduct of All ―Major‖ Risk 

46 Assessments within a Risk Analysis Framework: A Report by the CFSAN Risk Analysis 
Page 186 Microbial Risk Assessment Guideline

1 Working Group. Center for Food Safety and Applied Nutrition (CFSAN).

2 http://www.cfsan.fda.gov/~dms/rafw-toc.html

3

4 Food and Drug Administration (FDA). (2005) Quantitative Risk Assessment on the 

5 Public Health Impact of Pathogenic Vibrio parahaemolyticus in Raw Oysters.

6 http://www.cfsan.fda.gov/~dms/vpra-toc.html

7

8 Food and Drug Administration (FDA). (2007) An Overview of the CARVER Plus Shock 

9 Method for Food Sector Vulnerability Assessments.

10 http://www.fsis.usda.gov/PDF/Carver.pdf

11

12 Food and Drug Administration/U.S. Department of Agriculture/Centers for Disease 

13 Control and Prevention (FDA/USDA/CDC). (2003) Quantitative Assessment of Relative 

14 Risk to Public Health from Foodborne Listeria monocytogenes Among Selected 

15 Categories of Ready-to-Eat Foods. FDA Center for Food Safety and Applied Nutrition 

16 (CFSAN), USDA Food Safety and Inspection Service, and Centers for Disease Control 

17 and Prevention, Washington, DC. http://www.foodsafety.gov/~dms/lmr2-toc.html.

18

19 Ferson, S. (1996) What Monte Carlo methods cannot do. Human and Ecological Risk 

20 Assessment 2:990-1007.

21

22 Fischer-Le Saux, M., Hervio-Heath, D., Loaec, S., Colwell, R.R., and M. Pommepuy. 

23 (2002) Detection of cytotoxin-hemolysin mRNA in nonculturable populations of 

24 environmental and clinical Vibrio vulnificus strains in artificial seawater. Applied and 

25 Environmental Microbiology 68:5641-5646.

26

27 Fischhoff, B., Lichtenstein, S., Slovic, P., Derby, S.L., and R.L. Keene. (1981) 

28 Acceptable Risk. New York, NY: Cambridge University Press.

29

30 Frey, H.C., Mokhtari, A., and T. Danish. (2003) Evaluation of Selected Sensitivity 

31 Analysis Methods Based Upon Applications to Two Food Safety Process Risk Models. 

32 Prepared for: Office of Risk Assessment and Cost-Benefit Analysis USDA, Washington, 

33 DC. http://www.ce.ncsu.edu/risk/Phase2Final.pdf

34

35 Frey, H.C., Mokhtari, H., and J. Zheng. (2004) Recommended Practice Regarding 

36 Selection, Application, and Interpretation of Sensitivity Analysis Methods Applied to 

37 Food Safety Process Risk Models. Prepared for the Office of Risk Assessment and Cost38 Benefit Analysis, U.S. Department of Agriculture.

39 http://www.ce.ncsu.edu/risk/Phase3Final.pdf

40

41 Frost, F.J., Roberts, M., Kunde, T.R., Craun, G., Tollestrup, K., Harter, L., and T. Muller. 

42 (2005) How clean must our drinking water be: The importance of protective immunity. 

43 Journal of Infectious Diseases 191:809-814.

44

45 Furumoto, W.A. and R. Mickey. (1967) A mathematical model for the infectivity-dilution

46 curve of tobacco mosaic virus: theoretical considerations. Virology 32(2):216-23.
Microbial Risk Assessment Guideline Page 187

Draft – Do not cite, circulate, or quote

1

2 Government Accountability Office (GAO). (2001) Chemical Risk Assessment: Selected 

3 Federal Agencies‘ Procedures, Assumptions, and Policies. GAO-01-810. Washington, 

4 DC: http://www.gao.gov/new.items/d01810.pdf

5

6 Gelman, A., Carlin, J.B., Stern, H.S., and D.B. Rubin. (2004) Bayesian Data Analysis, 

7 Second Edition. Boca Raton, Florida: Chapman and Hall/CRC.

8

9 Gerba, C.P., Rose, J.B., Haas, C.N., and K.D. Crabtree. (1996) Waterborne rotavirus: a 

10 risk assessment. Water Research 30:2929-2940.

11

12 Gilks, W., Richardson, S., and D.J. Spiegelhalter (eds.). (1996) Markov Chain Monte 

13 Carlo in Practice. London, UK: Chapman and Hall.

14

15 Gold, M.R., Stevenson, D., and D.G. Fryback. (2002) HALYs and QALYs and DALYs, 

16 oh my: similarities and differences in summary measures of population health. Annual 

17 Review Public Health 23:115-134.

18

19 Guzmán, E., Romeu, A., and S. Garcia-Vallve. (2008) Completely sequenced genomes of 

20 pathogenic bacteria: A review. Enfermedades Infecciosas y Microbiologia Clinica 26:88-

21 98. 

22

23 Haas, C.N., Rose, J.B., Gerba, C., and S. Regli. (1993) Risk assessment of virus in 

24 drinking water. Risk Analysis 13(5):545-552.

25

26 Haas, C.N., Crockett, C.S., Rose, J.B., Gerba, C.P., and A.M. Fazil. (1996) Assessing the 

27 risks posed by oocysts in drinking water. Journal of the American Water Works 

28 Association 88(9):131-136.

29

30 Haas, C.N., Rose, J.B., and Gerba, C.P. (1999) Quantitative Microbial Risk Assessment.

31 New York: John Wiley and Sons.

32

33 Haimes, Y.Y. (2004) Risk Modeling, Assessment, and Management. Second edition. 

34 New York: John Wiley & Sons.

35

36 Hethcote, H. (1976) Qualitative analyses of communicable disease models. Mathematical 

37 Biosciences 28:335-356.

38

39 Hethcote, H.W. (2000) The mathematics of infectious diseases. SIAM Review 42:599-

40 653.

41

42 Hoeting, J.A., Madigan, D., Raftery, A.E., and C.T. Volinsky. (1999) Bayesian model 

43 averaging: A tutorial. Statistical Science 14(4):382–417.

44
Page 188 Microbial Risk Assessment Guideline

1 Hopkins, R.S., Gaspard, G.B., Williams, F.P., Karlin, R.J., Cukor, K.G., and N.R. 

2 Blacklow. (1984) A community waterborne gastroenteritis outbreak: evidence for 

3 rotavirus as the agent. American Journal of Public Health 74:263-265.

4

5 Hirshliefer, J. and J. Riley. (1992) The Analytics of Uncertainty and Information. 

6 Cambridge: Cambridge University Press.

7

8 Humber, J.M.; R.F. Almeder. (1986) Quantitative Risk Assessment. Biomedical Ethics 

9 Reviews. Clifton, New Jersey: Humana Press.

10

11 Huq, A., Rivera, I.N.G., and R.R. Colwell. (2000) Epidemiological Significance of 

12 Viable But Nonculturable Microorganisms. In: Colwell, R.R. and Grimes, D.J. (eds.)

13 Nonculturable Microorganisms in the Environment. Washington, DC: ASM Press. 

14

15 Hurd, H.S. and J.B. Kaneene. (1993) The application of simulation models and systems 

16 analysis in epidemiology: A review. Preventive Veterinary Medicine 15:81-99.

17

18 International Life Sciences Institute (ILSI). (2000) Revised Framework for Microbial 

19 Risk Assessment. Washington, DC: ILSI Press. 

20 http://rsi.ilsi.org/Publications/MRAWorkbook.htm

21

22 Interagency Risk Assessment Consortium (IRAC). (2000) Public Meeting on Food Safety 

23 Risk Analysis Clearinghouse Data Quality Objectives. December 5, 2000.

24 http://foodrisk.org/IRAC/events/2000-12-05/index.cfm

25

26 Jameel, S. (1999) Molecular biology and pathogenesis of hepatitis E virus. Expert 

27 Reviews in Molecular Medicine (6 December):1-16.

28 http://www-ermm.cbcu.cam.ac.uk/99001271h.htm

29

30 Kaplan, S. and B.J. Garrick. (1981) On the quantitative definition of risk. Risk Analysis

31 1:11-27.

32

33 Kaplan, S. (2000) Combining probability distributions from experts in risk analysis. 

34 Letter to the editor. Risk Analysis 20(2):155-156.

35

36 King, A.A., Ionides, E.L., Pascual, M., and M.J. Bouma. (2008) Inapparent infections and 

37 cholera dynamics. Nature 454:877-880.

38

39 Knapp, S., Hacker, J., Jarchau, T., and W. Goebel. (1986) Large, unstable inserts in the 

40 chromosome affect virulence properties of uropathogenic Escherichia coli 06 Strain 536. 

41 Journal of Bacteriology 168:22-30.

42

43 Ko, G., Thompson, K.M., and E.A. Nardell. (2004) Estimation of tuberculosis risk on a 

44 commercial airliner. Risk Analysis 26(2):379-388.

45
Microbial Risk Assessment Guideline Page 189

Draft – Do not cite, circulate, or quote

1 Kodell, R.L., Kang, S.-H., and J.J. Chen. (2002) Statistical models of health risk due to 

2 microbial contamination of foods. Environmental and Ecological Statistics 9:259-271.

3

4 Kosa, K.M.; Cates, S.C.; Karns, S.; Godwin, S.L.; and D. Chambers. (2007) Consumer 

5 knowledge and use of open dates: Results of a web-based survey. Journal of Food 

6 Protection 70(5):1213-1219.

7

8 Labbe, R.G. and S. Garcia. (2001) Guide to Foodborne Pathogens. Hoboken, NJ: John 

9 Wiley & Sons.

10

11 Lammerding, A.M., and A. Fazil. (2000) Hazard identification and exposure assessment 

12 for microbial food safety risk assessment. International Journal of Food Microbiology 

13 58(3):147-57.

14

15 Law, A.M., and W.D. Kelton. (2000) Simulation Modeling and Analysis, Third Edition. 

16 New York: McGraw-Hill Companies.

17

18 Lave, L., and T. Romer. (1981) A Survey of Safety Levels in Federal Regulation. Nuclear 

19 Regulatory Commission. NUREG/CR-2226.

20

21 Li, X.J., and H.H. Wang. (2010) Tetracycline resistance associated with commensal 

22 bacteria from representative ready-to-consume deli and restaurant foods. Journal of Food 

23 Protection (73):1841-1848.

24

25 Lerat, E., and N.A. Moran. (2004) The evolutionary history of quorum-sensing systems 

26 in bacteria. Molecular Biology and Evolution 21:903-913.

27

28 Liu, D. (2006) Identification, subtyping, and virulence determination of Listeria 

29 monocytogenes, an important foodborne pathogen. Journal of Medical Microbiology

30 55:645-659.

31

32 Lowrance, W.W. (1976) Of Acceptable Risk: Science and the Determination of Safety. 

33 Los Altos, CA: William Kaufmann, Inc. 

34

35 Lundgren, R. and A. McMakin. (1998) Risk Communication: A Handbook for 

36 Communicating Environmental, Safety, and Health risks. Second edition. Columbus, OH:

37 Battelle Press.

38

39 Lunn, DJ, Thomas, A., Best, and N.D. Spiegelhalter. (2000) WinBUGS - A Bayesian

40 modeling framework: Concepts, structure, and extensibility. Statistics and Computing

41 10(4):325-337.

42

43 Mahenthiralingam, E., J. Bischof, S. K. Byrne, C., Radomski, C., Davies, J.E., Av-Gay, 

44 Y., and P. Vandamme. (2000) DNA-based diagnostic approaches for identification of 

45 Burkholderia cepacia complex, Burkholderia vietnamiensis, Burkholderia multivorans, 
Page 190 Microbial Risk Assessment Guideline

1 Burkholderia stabilis, and Burkholderia cepacia genomovars I and III. Journal of

2 Clinical Microbiology 38(9):3165-3173.

3

4 Manning, S.D., Motiwala, A.S., Springman, A.C., Qi, W., Lacher, D.W., Ouellette, L.M., 

5 Mladonicky, J.M., Somsel, P., Rudrik, J.T., Dietrich, S.E., Zhang, W., Swaminathan, B., 

6 Alland, D., and T.S. Whittam. (2008) Variation in virulence among clades of Escherichia

7 coli O157:H7 associated with disease outbreaks. Proceedings of the National Academies 

8 of Science 105:4868-4873.

9

10 McBride, G.B., Till, D., Ryan, T., Ball, A., Lewis, G., Palmer, S., and P. Weinstein. 

11 (2002) Freshwater Microbiology Research Programme: Pathogen Occurrence and Human 

12 Health Risk Assessment Analysis, Technical Publication, Ministry for the Environment, 

13 Wellington, 93 pp. http://www.mfe.govt.nz/publications/water/freshwater-microbiology14 nov02/freshwater-microbiology-nov02.pdf

15

16 Mead, P.S.., Slutsker, L., Dietz, V., McCaig, L.F., Bresee, J.S., Shapiro, C., Griffin, P.M., 

17 and R.V. Tauxe. (1999) Food-related illness and death in the United States. Emerging 

18 Infectious Diseases 5(5):607-625. http://www.cdc.gov/ncidod/EID/vol5no5/pdf/mead.pdf

19

20 Medema, G.J., Teunis, P.F., Havelaar, A.H., and C.N. Haas. (1996) Assessment of 

21 the dose-response relationship of Campylobacter jejuni. International Journal of Food 

22 Microbiology 30(1-2):101-11.

23

24 Messner, M.J., Chappell, C.L., and P.C. Okhuysen. (2001) Risk assessment for 

25 Cryptosporidium: A hierarchical Bayesian analysis of human dose-response data. Water 

26 Research 35(16):3934-3940.

27

28 Moon, H., Kim, H-J., Chen, J.J., and R.L. Kodell. (2005) Model averaging using the 

29 Kullback Information Criterion in estimating effective doses for microbial infection and 

30 illness. Risk Analysis 25(5):1147-1159.

31

32 Morgan, M.G. and M. Henrion. (1990) Uncertainty: A Guide to Dealing with Uncertainty 

33 in Quantitative Risk and Policy Analysis. Cambridge: Cambridge University Press.

34

35 Morgan, G.M., Fischhoff, B., Bostrom, A., and Atman, C.J. (2002) Risk Communication 

36 A Mental Models Approach. New York: Cambridge University Press.

37

38 National Advisory Committee on Microbiological Criteria for Foods (NACMCF). (1997) 

39 Hazard Analysis and Critical Control Point (HAACP) Principles and Application 

40 Guidelines. Adopted August 14, 1997.

41 http://www.fda.gov/Food/FoodSafety/HazardAnalysisCriticalControlPointsHACCP/HAC

42 CPPrinciplesApplicationGuidelines/default.htm

43

44 Namata, H., Aerts, M., Faes, C., and P. Teunis. (2008) Model averaging in microbial risk 

45 assessment using fractional polynomials. Risk Analysis 28(4):891-905.

46
Microbial Risk Assessment Guideline Page 191

Draft – Do not cite, circulate, or quote

1 Nauta, M.J. (2002) Modeling bacterial growth in quantitative microbiological risk 

2 assessment: Is it possible? International Journal of Food Microbiology 73:297-304.

3

4 Nauta, M.J. (2005) Microbiological risk assessment models for partitioning and mixing 

5 during food handling. International Journal of Food Microbiology 100:311-322.

6

7 National Committee on Radiation Programs (NCRP). (1996) A Guide for Uncertainty 

8 Analysis in Dose and Risk Assessments Related to Environmental Contamination.

9 NCRP, Scientific Committee 64-17, Washington, DC. NCRP Commentary No. 14 [as 

10 cited in EPA, 1997b, page 14].

11

12 Newsome, R., Tran, N., Paoli, G.M., Jaykus, L.A., Tompkin, B., Miliotis, M., Ruthman,

13 T., Hartnett, E., Busta, F.F., Petersen, B., Shank, F., McEntire, J., Hotchkiss, J., Wagner, 

14 M., and D.W. Schaffner. (2009) Development of a risk-ranking framework to evaluate 

15 potential high-threat microorganisms, toxins, and chemicals in food. Journal of Food 

16 Science 74(2):R39-R45.

17

18 National Research Council (NRC). (1983) Risk Assessment in the Federal Government: 

19 Managing the Process. Washington, DC: National Academy Press.

20 http://www.nap.edu/catalog.php?record_id=366

21

22 National Research Council (NRC). (1994) Science and Judgment in Risk Assessment. 

23 Washington, DC: National Academies Press. 

24 http://www.nap.edu/catalog.php?record_id=2125. 

25

26 National Research Council (NRC). (1996) Understanding Risk: Informing Decisions in a 

27 Democratic Society. Washington, DC: National Academies Press.

28 http://www.nap.edu/catalog.php?record_id=5138

29

30 National Research Council (NRC). (2003) Scientific Criteria to Ensure Safe Food. 

31 Washington, DC: National Academies Press. 

32 http://books.nap.edu/catalog.php?record_id=10690

33

34 National Research Council (NRC). (2005) Reopening Public Facilities After a Biological 

35 Attack: A Decision-Making Framework. Washington, DC: National Academies Press.

36 http://www.nap.edu/catalog.php?record_id=11324

37

38 National Research Council (NRC). (2008) Public Participation in Environmental 

39 Assessment and Decision Making. Washington, DC: National Academies Press. 

40 http://www.nap.edu/catalog.php?record_id=12434

41

42 National Research Council (NRC). (2009) Science and Decisions: Advancing Risk 

43 Assessment. Washington, DC: National Academies Press.

44 http://www.nap.edu/catalog.php?record_id=12209

45
Page 192 Microbial Risk Assessment Guideline

1 O‘Donoghue, P. (1995) Cryptosporidium and cryptosporidiosis in man and animals.

2 International journal for Parasitology: 25:2:139-195.

3

4 World Organization for Animal Health (OIE). 1999. Chapter 4. Import risk analysis In: 

5 OIE Animal Health Code. World Organisation for Animal Health, Paris, France.

6

7 Okhuysen, P.C., Chappell, C.L., Crabb, J.H., Sterling, C.R., and H.L. DuPont. (1999) 

8 Virulence of three distinct Cryptosporidium parvum isolates for healthy adults. Journal of 

9 Infectious Diseases 180(4):1275-1281.

10

11 Office of Management and Budget (OMB). (2002) Guidelines for ensuring and 

12 maximizing the quality, objectivity, utility, and integrity of information disseminated by 

13 federal agencies; republication. Federal Register 67 (36)8452-8460.

14 http://www.whitehouse.gov/sites/default/files/omb/fedreg/reproducible2.pdf

15

16 Office of Management and Budget (OMB). (2003) Circular A-4, Regulatory Analysis 

17 (09/17/2003).

18 http://www.whitehouse.gov/sites/default/files/omb/assets/omb/circulars/a004/a-4.pdf

19

20 Office of Management and Budget (OMB). (2004) Revised Information Quality Bulletin 

21 for Peer Review, April 2004. 

22 http://www.whitehouse.gov/omb/inforeg/peer_review041404.pdf

23

24 Office of Management and Budget (OMB). (2007a) Final Bulletin for Agency Good 

25 Guidance Practices. Bulletin No. 07-02. 

26 http://www.whitehouse.gov/omb/memoranda/fy2007/m07-07.pdf. 

27

28 Office of Management and Budget (OMB). (2007b) M-07-24 Memorandum for the 

29 Heads of Executive Departments and Agencies, Subject: Updated Principles for Risk 

30 Analysis, September 19, 2007.

31 http://www.whitehouse.gov/sites/default/files/omb/assets/omb/memoranda/fy2007/m07-

32 24.pdf

33

34 Office of Management and Budget (OMB). (2010) Agency Checklist: Regulatory Impact 

35 Analysis. October 28, 2010.

36 http://www.whitehouse.gov/sites/default/files/omb/inforeg/regpol/RIA_Checklist.pdf

37

38 Oscar, T.P. (2005) Validation of lag time and growth rate models for Salmonella 

39 typhimurium: acceptable prediction zone method. Journal of Food Science 70(2):M129-

40 M137.

41

42 Office of Science and Technology Policy (OSTP). (2010). Memorandum for the Heads of 

43 Executive Departments and Agencies, Subject: Scientific Integrity. December 17, 2010. 

44 http://www.whitehouse.gov/sites/default/files/microsites/ostp/scientific-integrity-memo45 12172010.pdf.

46
Microbial Risk Assessment Guideline Page 193

Draft – Do not cite, circulate, or quote

1 Ouchi, F. (2004) A Literature Review on the Use of Expert Opinion in Probabilistic Risk 

2 Analysis. World Bank Policy Research Working Paper, Report Number WPS3201. 

3 http://econ.worldbank.org

4

5 Parkin, (2008) Foundations and Frameworks for Human Microbial Risk Assessment.

6 http://www.epa.gov/raf/files/epa_mra_fw_comparison_report_0609.pdf

7

8 Pedersen A.B., Altizerb, S., Poss, M. Cunningham, A.A., and C.L. Nunn. (2005) Patterns 

9 of host specificity and transmission among parasites of wild primates. International 

10 Journal for Parasitology 35(6):647-657.

11

12 Peter, J.B. (ed). (1998) Use and Interpretation of Laboratory Tests in Infectious Disease.

13 5th Edition. Santa Monica, CA: Specialty Laboratories.

14

15 Pinsky, P.F. (2000) Assessment of risks from long term exposure to waterborne 

16 pathogens. Environmental and Ecological Statistics 7:155-175.

17

18 P/CC (Presidential/Congressional Commission on Risk Assessment and Risk 

19 Management). (1997) Risk Assessment and Risk Management in Regulatory Decision20 Making, Volume 2. Washington, DC: Government Printing Office. 

21 http://www.riskworld.com/riskcommission/Default.html

22

23 Pouillot, R.; Lubran, M.B; Cates, S. C; S. Dennis. (2010) Estimating parametric 

24 distributions of storage time and temperature of ready-to-eat foods for U.S. households. 

25 Journal of Food Protection 73(2):312-321.

26

27 Presidential Memorandum. (2009) The White House, Office of the Press Secretary, 

28 Memorandum for the Heads of Executive Departments and Agencies, Subject: Scientific 

29 Integrity. March 9, 2009. http://www.whitehouse.gov/the-press-office/memorandum30 heads-executive-departments-and-agencies-3-9-09

31

32 Regli, S., Rose, J.B., Haas, C.N., and C.P. Gerba. (1991) Modeling risks for pathogens in 

33 drinking water. Journal of the American Water Works Association 83(11):76-84.

34

35 Rice, G., Heberling, M.T., Rothermich, M., Wright, J.M., Murphy, P.A., Craun, M.F. and 

36 G.F. Craun. (2006) The role of disease burden measures in future estimates of endemic 

37 waterborne disease. Journal of Water and Health 4 (Suppl 2):187-199.

38

39 Riley, S., Fraser, C., Donnelly, C.A., Ghani, A.C., Abu-Raddad, L.J., Hedley, A.J., 

40 Leung, G.M., Ho, L.M., Lam, T.H., Thach, T.Q., Chau, P., Chan, K.P., Lo, S.V., Leung, 

41 P.Y., Tsang, T., Ho, W., Lee, K.H., Lau, E.M., Ferguson, N.M., and R.M. Anderson. 

42 (2003) Transmission dynamics of the etiological agent of SARS in Hong Kong: Impact 

43 of public health interventions. Science 300:1961-1966.

44

45 Rose. J.B., Haas, C.N., and S. Regli. (1991) Risk assessment and control of waterborne 

46 giardiasis. American Journal of Public Health 81:709-713. 
Page 194 Microbial Risk Assessment Guideline

1

2 Rose, J.B., and M.D. Sobsey. (1993) Quantitative risk assessment for viral contamination 

3 of shellfish and coastal waters. Journal of Food Protection 56(12):1043-1050.

4

5 Rose, J.B., and C.P. Gerba. (1991) Use of risk assessment for development of microbial 

6 standards. Water Science and Technology 24:29-34.

7

8 Ross, T. and McMeekin, T.A. (1994) Predictive microbiology. International Journal 

9 Food Microbiology 23:241-264.

10

11 Ross, T. and T.A. McMeekin. (2003) Modeling microbial growth within food safety risk 

12 assessments. Risk Analysis 23(1):179-197.

13

14 Ross, T. (2008) Chapter 3. Translating Knowledge of Microbial Ecology into Risk 

15 Assessment Models, pp. 51-97 In: Microbial Risk Analysis of Foods, Series Editor: 

16 Michael P. Doyle; Editor: Donald Schaffner. ASM Press, Washington, DC (978-1-

17 55581-461-8).

18

19 Rowman, N.J. (2004) Viable but non-culturable forms of food and waterborne bacteria: 

20 Quo vadis? Trends in Food Science and Technology 15:462-467. 

21 http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VHY-4CVX484-

22 5&_user=5941288&_coverDate=09%2F01%2F2004&_rdoc=1&_fmt=&_orig=search&_

23 sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=5941288&

24 md5=a7b4ed126d6e80d28143b03d4b0ebbca

25

26 Schaffner, D.W. (2008) Microbial Risk Analysis of Food. Series Editor: Michael P.

27 Doyle. ASM Press, Washington, DC 270 p. (978-1-55581-461-8).

28

29 Schmidt, H. and M. Hensel. (2004) Pathogenicity islands in bacterial pathogenesis. 

30 Clinical Microbiology Reviews 17:14-56.

31

32 Schoen, M.E. and N.J. Ashbolt. (2010) Assessing pathogen risk to swimmers at non33 sewage impacted recreational beaches. Environmental Science and Technology

34 44(7):2286-2291.

35

36 Sellnow, T.L., Ulmer, R.R., Seeger, M.W., and R.S. Littlefield. (2008) Effective Risk 

37 Communication A Message-Centered Approach. In Series: Food Microbiology and Food 

38 Safety. New York: Springer Publishing.

39

40 Sen, K. and N.J. Ashbolt. (2011) Environmental Microbiology: Current Technology and 

41 Water Applications. Portland, OR: Caister Academic Press.

42

43 Smith, B. and J.D. Oliver. (2006) In situ and in vitro gene expression by Vibrio vulnificus 

44 during entry into, persistence within, and resuscitation from the viable but nonculturable 

45 state. Applied Environmental Microbiology 72:1445-1451.

46
Microbial Risk Assessment Guideline Page 195

Draft – Do not cite, circulate, or quote

1 Soller, J.A., Eisenberg, J.N., and A.W. Olivieri. (1999) Evaluation of Pathogen Risk 

2 Assessment Framework. Oakland, CA: Eisenberg, Olivieri and Associates.

3

4 Soller, J.A., Seto, E.Y., and A.W. Olivieri. (2007) Application of Microbial Risk 

5 Assessment Techniques to Estimate Risk Due to Exposure to Reclaimed Waters. 

6 WateReuse Foundation, Final Project Report WRF-04-011. 

7

8 Soller, J.A., and J.N. Eisenberg. (2008) An evaluation of parsimony for microbial risk 

9 assessment models. Environmetrics 19:61-78.

10

11 Soller, J.A. (2009) Potential implications of person-to-person transmission of viral 

12 infection to US EPA’s Groundwater Rule. Journal of Water Health 7(2):208-223

13

14 Soller, J.A, Bartrand, T., Ashbolt, N.J., Ravenscroft, J., and T.J. Wade. (2010) Estimating 

15 the primary etiologic agents in recreational freshwaters impacted by human sources of 

16 faecal contamination. Water Research 44:4736 -4747.

17

18 Teunis, P.F., van der Heijden, O.G., van der Giessen, J.W.B., and A.H. Havelaar. (1996) 

19 The Dose-Response Relation in Human Volunteers for Gastro-Intestinal Pathogens. 

20 RIVM (National Institute of Public Health and the Environment) Report No. 284550002.

21

22 Teunis, P.F.M., and A.H. Havelaar. (1999) Cryptosporidium in Drinking Water: 

23 Evaluation of the ILSI/RSI Quantitative Risk Assessment Framework. RVIM Report No. 

24 284 550 006. Bilthoven, The Netherlands.

25

26 Teunis, P.F.M., and A.H. Havelaar. (2000) The beta-Poisson model is not a single hit 

27 model. Risk Analysis 20(4):513-520.

28

29 Teunis, P.F., Chappell, C.L., and P.C. Okhuysen. (2002) Cryptosporidium dose-response 

30 studies: Variation between isolates. Risk Analysis 22(1):175-183.

31

32 Teunis, P., Takumi, K., and K. Shinagawa. (2004) Dose response for infection by

33 Escherichia coli O157:H7 from outbreak data. Risk Analysis 24(2):401-407.

34

35 Teunis, P.F.M., van den Brandhof, W., Nauta, M., Wagenaar, J., van den Kerkhof, H., 

36 and w. Van Pelt. (2005) A reconsideration of the Campylobacter dose-response relation. 

37 Epidemiology and Infection 133:583-592.

38

39 Teunis, P.F.M., Ogden, I.D., and N.J.C. Strachan. (2008a) Hierarchical dose response of 

40 E. coli O157:H7 from human outbreaks incorporating heterogeneity in exposure. 

41 Epidemiology and Infection 136(6):761-770.

42

43 Teunis, P.F.M., Moe, C.L., Liu, P., Miller, S.E., Lindesmith, L., Baric, R.S., Pendu, J.L., 

44 and Calderon, R.L. (2008b) Norwalk virus: How infectious is it? Journal of Medical 

45 Virology 80(8):1468-1476.

46
Page 196 Microbial Risk Assessment Guideline

1 U.S. Army Center for Health Promotion and Preventive Medicine (USACHPPM). (2009) 

2 Technical Guide 316 – Microbial Risk Assessment for Aerosolized Microorganisms. U.S. 

3 Army Center for Health Promotion and Preventive Medicine.

4

5 U.S. Department of Agriculture (USDA). (2003) Risk Analysis at FSIS: Standard 

6 Operating Procedures. Washington, DC: USDA, FSIS. 

7 http://www.fsis.usda.gov/OPPDE/rdad/FRPubs/RASOPs.pdf

8

9 U.S. Department of Agriculture (USDA). (2008) Microbiology Laboratory Guidebook. 

10 http://www.fsis.usda.gov/science/Microbiological_Lab_Guidebook/index.asp 

11

12 Vandamme, P., Holmes, B., Vancanney, M., Coenye, T., Hoste, B., Coopman, R.,

13 Revets, H., Lauwers, S., Gillis, M., Kersters, K., and J.R.W. Govan. (1997) Occurrence 

14 of multiple genomovars of Burkholderia cepacia in cystic fibrosis patients and proposal 

15 of Burkholderia multivorans sp. nov. International Journal of Systematic Bacteriology

16 (47):1188–1200.

17

18 Vandamme, P., Mahenthiralingam, Holmes, B., Coenye, T., Hoste, B., De Vos, P,. 

19 Henry, D., and D.P. Speert. (2000) Identification and population structure of 

20 Burkholderia stabilis sp. nov. (formerly Burkholderia cepacia Genomovar IV). Journal 

21 of Clinical Microbiology 38:1042-1047.

22

23 Vatn, J. (2004) A discussion of the acceptable risk problem. Norwegian University of 

24 Science and Technology. http://www.ntnu.no/ross/reports/acceptable_risk.pdf;

25 http://www.ntnu.no/ross/info/notes.php. Posted on February 18, 2004.

26

27 Vose, D.J. (2008) Risk Analysis: A Quantitative Guide. Third Edition. Chichester, 

28 England: John Wiley & Sons Ltd.

29

30 World Health Organization (WHO). (2000) The Interaction between Assessors and 

31 Managers of Microbiological Hazards in Food. WHO/SDE/PHE/FOS/007

32 http://www.who.int/foodsafety/publications/micro/en/march2000.pdf

33

34 World Health Organization (WHO). (2001) Water Quality: Guidelines, Standards and 

35 Health: Assessment of Risk and Risk Management for Water-Related Infectious Disease. 

36 Fewtrell, L., Bartram, J. (eds.) Published on behalf of IWA Publishing, WHO and 

37 Swedish Institute for Infectious Disease Control. 

38 http://www.who.int/water_sanitation_health/dwq/whoiwa/en/index.html

39

40 World Health Organization (WHO). (2005) Water Safety Plans, Managing Drinking41 Water Quality from Catchment to Consumer. WHO/SDE/WSH/05.06. 

42 http://www.who.int/water_sanitation_health/dwq/wsp0506/en/index.html

43

44 World Health Organization/Food and Agriculture Organization (WHO/FAO). (2008) 

45 Microbiological Risk Assessment Series 7 - Exposure Assessment of Microbiological 

46 Hazards in Food – Guidelines. http://www.fao.org/docrep/010/a0251e/a0251e00.htm
Microbial Risk Assessment Guideline Page 197

Draft – Do not cite, circulate, or quote

1

2 Williams, R.A., and K.M. Thompson. (2004) Integrated analysis: Combining risk and 

3 economic assessments while preserving the separation of powers. Risk Analysis

4 24(6):1613-1623.

5

6 Wooldridge, M. (2008) Chapter 1. Qualitative Risk Assessment, pp. 1-28 In: Microbial 

7 Risk Analysis of Foods, Series Editor: Michael P. Doyle; Editor: Donald Schaffner. 

8 ASM Press, Washington, DC (978-1-55581-461-8.)

9

10 Xiao, L. (2010) Molecular epidemiology of cryptosporidiosis: an update. Exp Parasitol 

11 124:80-89.

12

13 Yakota, F. and K.M. Thompson. (2003) Value of information (VOI) analysis in 

14 environmental health risk management (EHRM): Past, present and future. Risk Analysis

15 24(3):635-650.

16

17 Yoon, S.H., Park, Y.K., Lee, S., Choi, D., Oh, T.K., Hur, C.G., and J.F. Kim. (2007) 

18 Nucleic Acids Research 35:D395-D400.

19

20 Zhao, P., Zhao, T., Doyle, M.P., Rubino, J.R. and J. Meng. (1998) Development of a 

21 model for evaluation of microbial cross-contamination in the kitchen. Journal of Food 

22 Protection 61:960-963.

23

Microbial Risk Assessment Guideline Page A-1

Draft – Do not cite, circulate, or quote

1 Appendix A Example Assumptions

2

3 This appendix contains a list of types of assumptions that you may encounter during 

4 MRA. Many of these assumptions are due to data gaps. As data become available in the 

5 future, these assumptions could be different. Whether any of these assumptions are 

6 justifiable and adequate for a given risk assessment can be decided on a case-by-case 

7 basis. For example the assumptions about the immune status of the population may be 

8 simple (no one has immunity) or complex (based on actual immunity data).

9

10 For all assumptions, the strengths, limitations, and implications of the assumption should 

11 be fully explored and documented in the risk assessment. This level of transparency 

12 helps peer reviewers decide if they agree with the basis of the assumptions and whether 

13 the results of the risk assessment seem credible. 

14

15 Some assumptions are related to the scope of the risk assessment and are less subject to 

16 challenge on a scientific basis. For example the choice to evaluate a single agent instead 

17 of a mixture of agents is a policy decision about scope. In addition most MRA is limited 

18 in scope to estimating risk as a result of a single exposure event. In chemical risk 

19 assessment the exposure duration is often a 70-year life span.

20

21 General Assumptions:

22

23 MRA’s focus on known pathogens that contribute significantly to the human disease 

24 burden or emerging pathogens for which the potential for disease is recognized, but the 

25 disease burden is unknown.

26

27 MRA’s related to food-borne infections typically focus on a food-pathogen pair. 

28 This assumes that a specific pathogen is associated with a specific food. In reality, more 

29 than one pathogen could be associated with a specific food (for example, Salmonella and 

30 Campylobacter in chicken) and also different types of foods could be associated with a 

31 specific pathogen (for example, Salmonella in chicken and pork).

32

33 A given MRA’s scope reflects the regulatory jurisdiction of the sponsoring agency 

34 and the statues behind regulations. For example EPA risk assessments in water media 

35 are divided into ambient water which is regulated by the Clean Water Act and drinking 

36 water which is regulated by the Safe Drinking Water Act. EPA has not to date sponsored 

37 a MRA that examined the risks of one pathogen in all water media. MRA for foods 

38 categories that are regulated by multiple agencies have required interagency 

39 collaborations.

40

41 A mathematical model is assumed to adequately represent complex biological 

42 phenomenon and ecological relationships. 

43

44 Available data are assumed to be representative of the parameter. In practice, this 

45 assumption can lead to an overestimate or underestimate. Although MRA‘s routinely 
Page A-2 Microbial Risk Assessment Guideline

1 incorporate uncertainty (and/or variability) about the parameters of interest, the 

2 underlying assumption is that data included in a risk model is representative of the 

3 pathogen and target population of interest. Many examples illustrate the ubiquitous 

4 nature of this assumption. Data from experiments using animal models are commonly 

5 used rather than data from human infections. Certain related bacteria are considered to 

6 be surrogates for the pathogen of interest. Data based on short-term exposures are 

7 extrapolated to model the chronic effects of prolonged exposures. Public health 

8 surveillance data is considered to be reflective of the actual disease burden experienced 

9 by the human population under surveillance. Surveys of foods reflect the true prevalence 

10 and distribution of pathogens in that food. Surveys that focus on individual‘s food 

11 consumption during a limited time period reflect the long-term consumption patterns of 

12 the individual, and the study population is representative of the population of interest.

13

14 It is assumed that it is appropriate to pool data derived from a variety of sources. 

15 This assumption can result in an overestimate or underestimate of true risk. Estimated 

16 prevalence and cell number distribution of pathogens is generally determined by 

17 combining results available in published literature, government‘s surveillance reports and 

18 industry reports. These data have inherent variability and uncertainty resulting from the 

19 variety of methods used to obtain them. It is assumed that pooling data from multiple 

20 sources (with or without weighting each observation with respect to a quality score) will 

21 provide a valid estimate of the parameter of interest with appropriate limits of 

22 uncertainty.

23

24 Normal distributions or triangle distributions are often assumed for parameters. 

Point estimate based on 50th, 75th, or 95th 25 percentile are sometimes assumed and may not 

26 be correct or appropriate.

27

28 Assumptions concerning the Agent:

29

30 It is generally assumed that a minimum unit of the microbiological hazard is 

31 necessary to cause disease. For example, a risk model could be based on an underlying 

32 assumption that a single cell of a pathogenic bacterium has the potential to cause disease. 

33 Alternatively, it could be assumed that some minimum concentration of a bacterial toxin 

34 is required to cause disease.

35

36 Most models assume that each microbial unit acts independently, and that a single 

37 bacterial cell or viral particle has the potential to cause disease.

38

39 It is usually assumed that the chance of contracting a disease once an individual is 

40 infected is independent of the ingested dose, i.e., once infected, a particular individual 

41 contracts the disease regardless of ingested dose. A higher dose of pathogen would not 

42 cause more severe symptoms.

43

44 It is often assumed that factors intrinsic to the pathogen are the primary 

45 determinant of the agent’s ability to cause infection/disease. Extrinsic factors (related 

46 to both the environment and the potential host) also impact the affect the pathogen‘s 
Microbial Risk Assessment Guideline Page A-3

Draft – Do not cite, circulate, or quote

1 ability to cause infection/disease, and in some cases may be a more important 

2 determinant of the health outcome than the agent itself.

3

4 Certain assumptions must be stated regarding the variability among pathogen 

5 subtypes with respect to multiple characteristics that influence the development 

6 and/or detection of infection and/or disease. Many characteristics intrinsic to a 

7 specific pathogen ultimately influence both the probability that a specific subtype of a 

8 pathogen will result in an infection (or development of disease) and that this 

9 infection/disease will be detected and subsequently reported. These characteristics must 

10 be considered and prioritized according to their relative importance in impacting the 

11 health outcome of interest. In some cases, data is available that permits differentiation 

12 among subtypes with respect to these factors during the risk assessment process. For 

13 example, certain Salmonella serotypes appear to be more pathogenic to humans than 

14 others, and their relative pathogenicity could be defined to vary between serotypes in risk 

15 models. In other cases, these data are not available and it becomes necessary to assume 

16 that all subtypes are equal with respect to the defined characteristic. For example, certain 

17 Salmonella serotypes could be assigned the same relative pathogenicity in risk models 

18 when information that discriminates among the serotypes is unavailable. It is common to 

19 assume all isolates in the scenario are equally pathogenic.

20

21 Certain assumptions must be stated regarding the variability among pathogens and 

22 pathogen subtypes with respect to survival and growth in a variety of matrices. For 

23 food matrices in particular, many characteristics intrinsic to a specific pathogen 

24 ultimately influence the ability of the pathogen to survive and/or grow in that matrix. 

25 These characteristics must be considered and prioritized according to their relative 

26 importance in impacting the role of a given food as a vehicle. Assumptions concerning 

27 these characteristics are often necessary. For example, the pathogen may not be able to 

28 proliferate in the food, but it can survive and cause an adverse effect when consumed by 

29 a susceptible host. Some pathogens can form spores to ensure survival during adverse 

30 environmental condition and other may produce toxins under certain circumstances.

31

32 It is assumed that available growth kinetics models are adequately representative of 

33 all pathogen subtypes and are appropriate across different food matrices/environmental 

34 conditions. This assumption can lead to an overestimate or underestimate of risk.

35

36 Assumptions concerning the Host:

37

38 Certain assumptions are required with respect to the variability of susceptibility of 

39 individuals to the development of infection/disease. For example, it is frequently 

40 assumed that certain groups of people are more susceptible to infection/disease than 

41 others (i.e., young, elderly, pregnant woman, immunocompromised individuals). 

42

43 Target population is assumed not to be vaccinated or immune due to previous 

44 exposures. The probability of infection or illness resulting from exposure is independent 

45 of previous exposures; also, the probability of infection or illness resulting from 
Page A-4 Microbial Risk Assessment Guideline

1 secondary transmission is also independent of previous exposures. This ignores the 

2 possibility of temporary or permanent immunity.

3

4 Assumptions concerning the Environment:

5

6 Risk assessment models typically assume that microbes are homogeneously 

7 distributed throughout the specified matrix. While this may be a reasonable 

8 assumption for certain foods (e.g., milk, juices) and air, and water, it is unlikely to hold 

9 true for most food categories or soil matrices.

10

11 MRA models typically assume that the quantitative levels of contamination (i.e. 

12 microbiological counts) are best represented by a log normal distribution.

13

14 Assumptions are stated concerning the analytical methods used to detect the 

15 presence/quantitative level of the pathogen in a matrix. It is typically assumed that 

16 viable pathogen can be detected (e.g., culture, bioassay, serological test, polymerase 

17 chain reaction [PCR]). It is necessary to indicate the limit of detection, analytical 

18 specificity, epidemiologic sensitivity, and epidemiologic specificity of a described 

19 method when interpreting the results obtained when testing to identify a particular 

20 pathogen in a defined matrix. It is often assumed that all subtypes of a pathogen are 

21 equally likely to be detected using a particular analytical method; however, this may not 

22 be valid. Similarly, certain methods are applied across a variety of matrices. In some 

23 situations, characteristics of the matrix itself may interfere with pathogen detection.

24

25 Assumptions concerning the geographic and temporal (seasonal) distribution of 

26 pathogens are required. For example, it is believed that E. coli O157:H7 exhibits a 

27 seasonal distribution in cattle, ground beef, and ambient waters. You could assume that 

28 this pathogen is more prevalent during May – September than in October through March 

29 when modeling potential exposures or you could assume that prevalence is uniform 

30 throughout the year. You could make similar assumptions concerning the geographic 

31 distribution of a pathogen.

32

33 The complex series of environmental events that impact the survival and/or growth 

34 of microbiological hazards are typically simplified in the context of risk models. 

35 Considerable variability exists with respect to environmental conditions (i.e., time, 

36 temperature, pH) over time. For example, conditions associated with storage, 

37 refrigeration, product formulation, and batching process will vary greatly. Risk models 

38 assume that a single situation (or perhaps a limited number) occurs and that this factor 

39 occurs consistently over time. 

40

41 Assumptions Concerning the Exposure Scenario:

42

43 The exposure time span of interest is usually specified in the scope. Whether the 

44 exposure time span is a specified event (e.g., meal, trip to the beach) or a lifetime it is still 

45 an assumption that must be transparent.

46
Microbial Risk Assessment Guideline Page A-5

Draft – Do not cite, circulate, or quote

1 A specific food is the vehicle of transmission for a given pathogen. In reality, multiple 

2 foods may serve as a vehicle. Further, assumptions regarding the relative importance of 

3 potential exposures (i.e., food-borne; direct contact with animals, wildlife, insect vectors, 

4 or the environment; human-to-human transmission; water-borne exposures, recreational 

5 exposures) are necessary.

6

7 Certain assumptions are required with respect to individuals’ exposures to 

8 pathogens of concern and the variability in exposures among individuals. These 

9 assumptions may lead to an underestimate or overestimate of risk. Consumer behaviors 

10 are diverse (e.g., by region, ethnicity, religion, food preparation, eating practices, 

11 packaging methods, manufacturing production practices, food production practices, local 

12 conditions, sanitation). Also, foods are not going to be consumed with the same 

13 frequency by the same people over an entire year. Factors such as seasonal availability of 

14 certain foods and changing eating habits may be appropriate to consider in a national 

15 scale MRA. It is not feasible or advisable to try to break out every possible behavior that 

16 may influence the exposure scenario. Often an average behavior is assumed to be 

17 representative at a population level. For example the number of servings consumed by 

18 each person or the number of contaminated servings.

19

Microbial Risk Assessment Guideline Page B-1

Draft – Do not cite, circulate, or quote

1 Appendix B Hazard Identification Questions

2

3 This appendix contains examples of specific hazard identification questions that may be 

4 useful for the risk assessor‘s consideration. These are not all the questions risk assessors 

5 might consider.

6

7 General Questions:

8

9 1. What is the hazard in question and what is the specific media or food of concern? 

10 (e.g. Campylobacter spp. in poultry). 

11 2. What are the common routes of exposure associated with the hazard?

12 3. Are there indicator organisms or surrogates that can allow for an indirect 

13 evaluation of this pathogen in the absence of data?

14 4. Which pathogens are of concern to public health? Which are regarded as being of 

15 greatest concern, and why?

16 5. Are any media closely associated with, or often linked to, specific illnesses?

17 6. What agents are present in the media that may cause adverse health effects?

18 7. Prioritize and tabulate all the pathogens in terms of their degree of severity.

19 8. How is the media linked to the illness associated with the pathogen 

20 (epidemiologic evidence? Laboratory evidence?)

21 9. Are there available epidemiological data and microbiological data to associate 

22 what type of pathogen is associated with the media of interest?

23 10. Are there adequate public health data to substantiate the occurrence of pathogenic 

24 microorganisms in the media in question?

25 11. Are there any established standards/guidelines regarding the pathogen of interest? 

26 (assuming there are answers)

27 12. Is there data relevant to support the hazard? scientific literature? databases from 

28 industry, government agencies, international organizations? expert opinion? 

29 clinical studies, epidemiological studies, surveillance studies, outbreaks (domestic 

30 and international)? laboratory animal studies, investigations of the characteristics 

31 of the food-borne pathogen? microbial ecological studies? microorganism‘s 

32 behavior studies? sensitive populations (high risk, elderly, prenatal)? 

33 13. For food media, is there a list of generic processes for the normal exposure of the 

34 food product in question in the food chain? retail, restaurant, HACCP plan 

35 systems?

36

37 Questions concerning the Agent:

38

39 1. What type of pathogen is this organism (Bacteria, Virus, Parasite, Fungus, Prion, 

40 noninfectious toxin)? 

41 2. What are other taxonomic/strain considerations that influence ability to cause 

42 disease? Does the pathogen have particular strains that differ in ability to cause 

43 disease? If so, what is the strain of interest?

44 3. What intrinsic properties influence this agent‘s ability to cause disease?

45 4. What is the ―life cycle‖ of this agent? 
Page B-2 Microbial Risk Assessment Guideline

1 5. Does the pathogen produce a toxin? If so, is it the toxin that presents the hazard, 

2 or the pathogen?

3 6. Are there indicator organisms or surrogate species that can allow for an indirect 

4 evaluation of this agent in the absence of data?

5 7. How is the pathogen identified?

6 8. What factors influence the spatial distribution of this agent (clumping, 

7 aggregation, particles, clustering)?

8 9. Is the pathogen an anaerobic, gram-positive, spore- forming rod that produces a 

9 toxin?

10 10. Is the pathogen a microaerophilic organism?

11 11. What are the biochemical/taxonomic characteristics to identify the pathogen?

12 12. What is the subtyping of the pathogen? 

13 13. What are the survival characteristics? heat resistance? susceptibility to 

14 antimicrobial food additives? acid resistance? sensitivity to disinfectants or 

15 desiccation? sensitivity to radiation? UV? ionizing?

16 14. Are there phenotypic characteristics that influence virulence and/or 

17 pathogenicity?

18 15. Are there genotypic characteristics that influence virulence and/or pathogenicity?

19 16. Are the spores heat resistant? Can it survive in treatment processes?

20 17. Is there a microbiological testing/identification method for the pathogen in human 

21 clinical samples? Is there a specific method for testing for this pathogen? 

22 18. What methods are available for detecting and quantifying the agent? 

23 19. What are the cultural characteristics?

24 20. What are the detection methods to identify the food-borne pathogen?

25 21. What are the sampling and enrichment techniques to identify the food-borne 

26 pathogen? 

27 22. What are the growth characteristics? free-living vs. obligate parasite? growth 

28 requirements? temperature? pH? water activity? oxygen? Relative humidity?

29 23. What is the main pathogenic strain of this hazard, and what other specie(s) within 

30 the identified genus are humans susceptible to? (e.g., Campylobacter jejuni and 

31 coli).

32 24. Are there phenotypic characteristics that influence this agent‘s virulence and/or 

33 pathogenicity? 

34 25. Are there genotypic characteristics that influence this agent‘s virulence and/or 

35 pathogenicity? 

36 26. How does the agent cause pathology and/or disease?

37 27. Does the microorganism produce a toxin while growing in the intestinal tract?

38 28. How infectious/toxigenic is the pathogen?

39 29. What is the incubation period? 

40 30. What is the pathogenic microorganism‘s virulence?

41 31. What is the mechanism of action for infection and illness?

42 32. Is secondary transmission possible?

43 33. What is the target organ? 

44 34. Can the pathogen replicate in the media of concern?

45 35. Is the pathogen's ability to cause disease restricted to specific strains with 

46 identifiable virulence characteristics? 
Microbial Risk Assessment Guideline Page B-3

Draft – Do not cite, circulate, or quote

1 36. What are the virulence-associated characteristics to identify the pathogen? 

2 37. What is the pathogenicity? dis. char. and diagnosis? sequelae? host range? 

3 infectious dose? subpopulations at risk? animal models?

4 38. Describe the contagiousness of the infection (is or is not)

5 39. Identify the degree of severity for each pathogen

6

7 Questions concerning the Host:

8

9 1. Who is the susceptible population and what makes them susceptible? (e.g. 

10 children, adults, pregnant women)

11 2. What types of hosts can be infected by this agent? Is there host specificity? 

12 3. Are any practices/behaviors closely associated with, or often linked to, specific 

13 illnesses?

14 4. Is illness host-specific, i.e. more likely to affect susceptible populations?

15 5. Are there subpopulations at increased risk for this pathogen? 

16 6. What are the demographics associated with this pathogen?

17 7. What are the general socioeconomic trends associated with this pathogen? 

18 8. What is/are the characteristic(s) of the infection caused by the above hazard? (e.g. 

19 gastroenteritis). 

20 9. What is the nature of identified cases of infection? (e.g., sporadic, small/family 

21 related outbreaks).

22 10. What types of pathology and/or diseases are caused by this agent?

23 11. What are the typical symptoms of illness, and how do these differ between 

24 different pathogens?

25 12. What are the symptoms?

26 13. Is the microbe presenting itself as a gastrointestinal symptom? Vomiting, watery 

27 diarrhea? inflammatory diarrhea? 

28 14. Does the pathogen produce all or any of the following symptoms: abdominal 

29 cramps, nausea, vomiting, diarrhea, fever, dehydration?

30 15. Does the microbe usually have no manifestations of gastrointestinal illnesses 

31 associations? neurologic? systemic? hepatitic?

32 16. What is the incubation period within susceptible host, before the onset on an 

33 infection?

34 17. Is the pathogen found in the large intestinal tract only?

35 18. Does it shed in the feces?

36 19. What is the associated morbidity/mortality?

37 20. How might one define immunocompromised?

38 a) Are there specific quantifiable biomarkers of immunocompetency?

39 b) Can one define such biomarkers?

40 21. How would one account for genetic/ethnic/cultural differences within or between 

41 populations?

42 22. What is the definition of biomarkers for malnutrition?

43 23. What are the age groupings? How do they relate to susceptibility? 

44 24. Should age be defined with respect to chronology or physiology?

45 25. What would one do if the definitions [for different biomarkers] overlap?

46 26. What are the definitions for chronic ailments?
Page B-4 Microbial Risk Assessment Guideline

1 27. What are multiple concurrent factors?

2 28. How is immune status defined?

3

4 Questions concerning the Environment:

5

6 1. What are the environmental conditions of the pathogen in question: 

7 contamination, growth, inactivation, elimination, survival, for each part of the 

8 exposure scenario?

9 2. Does the pathogen have any specific survival characteristics that may promote its 

10 survival through the exposure scenario?

11 3. How do various environmental conditions (temperature, nutrients, pH) influence 

12 this agent‘s growth, survival, and/or death?

13 4. Are there certain environmental conditions and/or control processes that this agent 

14 can survive (or develop resistance)?

15 5. Are there seasonal, geographic or climatic factors that affect the occurrence of 

16 this agent?

17 6. What are the time factors involved in identifying the pathogen?

18 7. What are the temperature factors involved in identifying the pathogen?

19 8. What is the geographical range and seasonality of the pathogen?

20 9. Is the organism found in warmer months?

21 10. What is the geographic location?

22 11. What season does it occur?

23 12. If carried by another species, what is the geographic range of that species?

24 13. Is there a reservoir for the pathogen?

25 14. What are the ways in which the pathogen can contaminate the media?

26 15. Is that pathogen zoonotic?

27 16. Where (in what media?) is the pathogen commonly found?

28 17. Is the pathogen endemic?

29 18. Has it been identified in other countries recently?

30 19. What is the reservoir and/or environmental niches for this agent? Are there other 

31 important ecologic factors that influence this agent?

32 20. How can the media become contaminated with this pathogen? Are there any 

33 specific practices/behaviors that could promote survival of this pathogen? 

34 21. For food, did contamination occur during growing, harvesting, processing, 

35 storing, shipping, or final preparation?

36 22. Does the pathogen grow in the media?

37 23. What is the route of contamination and location of the pathogen in the media?

38 24. What is the microbial ecology to identify the pathogen?

39 25. What are the technologies associated with identifying the pathogen? 

40 26. What are the consumer trends associated with identifying the pathogen?

41 27. For food, what is the effect of food processing and preparation on their survival?

42 28. What is the consumption patterns associated with this pathogen?

43 29. What are the marketing and preparation practices associated with this food-borne 

44 pathogen?

45 30. What are the globalization trends associated with this pathogen regarding the

46 media of interest? 
Microbial Risk Assessment Guideline Page B-5

Draft – Do not cite, circulate, or quote

1 31. Is the pathogen's presence in contaminated media the result of an error or 

2 breakdown in normal controls?

3 32. What are the consumer behavior and consumption practices associated with this 

4 pathogen and media?

5 33. For food, identify and list (document) all the associated places (poultry (chicken, 

6 turkey), slaughter house (bovine, swine) , ready to eat, ...), locations/residing 

7 critical control points, and the time that certain pathogens are present.

8 34. What is the principal reservoir of this hazard, and where is it also commonly 

9 found? 

10 35. What are common environmental media and the expected concentration of this 

11 agent? 

12 36. Is the pathogen found in humans or animals or both?

13 37. Where is the source found? humans? domestic wild animals? poultry/eggs? 

14 swine? cattle? rodents? pets such as turtles, chicks, dogs and cats.

15 38. Is the pathogen found in cat, rodent or bird feces?

16 39. Is the food-borne pathogen found in soil? dust? sewage? and intestinal tracts of 

17 animals and/or humans?

18 40. What is the pathogen‘s reservoir in nature? 

19 41. Is the pathogen found in the intestinal tract?

20 42. Identify and list all the potential sources (reservoir) for all of these pathogens.

21

22 Questions concerning Transmission:

23

24 1. What is the mode of transmission? (e.g., direct contact with contaminated 

25 animals/carcasses, ingestion of contaminated food or water).

26 2. Identify all the potential routes/pattern of transmission (direct or indirect ) of 

27 microbial infection (disease) for each pathogen 

28 3. Is the pathogen found in putting something in the mouth that has been 

29 contaminated with the stool of an infected person or animal; direct contact with 

30 the droppings of infected animals?

31 4. Is the pathogen a person-to-person transmission such as in child daycare settings?

32 5. For food media, what are the eating habits associated with the pathogen?

33 6. How does this agent infect a susceptible host? What are the typical routes of 

34 infection and/or portals of entry?

35 7. Is there potential for secondary spread?

36 8. What is the infectious dose?

37

38