Centers for Disease Control

and Prevention

National Institute for Occupational 

Safety and Health

Centers for Disease Control

and Prevention

National Institute for Occupational 

Safety and Health

CURRENT INTELLIGENCE BULLETIN 69 

NIOSH Practices in 

Occupational Risk Assessment

Centers for Disease Control

and Prevention

National Institute for Occupational 

Safety and Health
Minor revisions were made to the frontmatter to indicate actual publication in March 2020. A 

new DOI number was inserted. It is as follows, https://doi.org/10.26616/NIOSHPUB2020106revised032020.
Current Intelligence Bulletin 69

NIOSH Practices in

Occupational Risk Assessment

Robert D. Daniels, Stephen J. Gilbert, Senthilkumar Perumal Kuppusamy, Eileen D. Kuempel, Robert M. Park, 

Sudah P. Pandalai, Randall J. Smith, Matthew W. Wheeler, Christine Whittaker, Paul A. Schulte

 DEPARTMENT OF HEALTH AND HUMAN SERVICES

Centers for Disease Control and Prevention

National Institute for Occupational Safety and Health
ii NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

This document is in the public domain and may be freely copied or 

reprinted.

Disclaimer

Mention of any company or product does not constitute endorsement by the National Institute for Occupational Safety and Health, Centers for Disease Control and Prevention. In 

addition, citations to websites external to NIOSH do not constitute NIOSH endorsement 

of the sponsoring organizations or their programs or products. Furthermore, NIOSH is not 

responsible for the content of these websites. All web addresses referenced in this document 

were accessible as of the publication date.

Get More Information

Find NIOSH products and get answers to workplace safety and health questions:

1-800-CDC-INFO (1-800-232-4636) | TTY: 1-888-232-6348

CDC/NIOSH INFO: cdc.gov/info | cdc.gov/niosh

Monthly NIOSH eNews: cdc.gov/niosh/eNews

Suggested Citation

NIOSH [2020]. Current intelligence bulletin 69: NIOSH practices in occupational risk 

assessment. By Daniels RD, Gilbert SJ, Kuppusamy SP, Kuempel ED, Park RM, Pandalai 

SP, Smith RJ, Wheeler MW, Whittaker C, Schulte PA. Cincinnati, OH: U.S. Department 

of Health and Human Services, Centers for Disease Control and Prevention, National 

Institute for Occupational Safety and Health. DHHS (NIOSH) Publication No. 2020-106 

(revised 03/2020), https://doi.org/10.26616/NIOSHPUB2020106revised032020.

DHHS (NIOSH) Publication No. 2020-106 (revised 03/2020)

DOI: https://doi.org/10.26616/NIOSHPUB2020106revised032020

March 2020
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment iii

Foreword

The Occupational Safety and Health Act of 1970 [29 USC 15], through delegation of functions by the Secretary of Health and Human Services, mandates the National Institute for 

Occupational Safety and Health (NIOSH) to ...“develop criteria dealing with toxic materials and harmful physical agents, which will describe exposure levels that are safe for various 

periods of employment...” [29 USC 669 (a) (3)]. Critical in developing these recommended 

criteria is assessment of the risk of adverse health effects that workers might experience given exposure to occupational hazards. The foundation for making these recommendations is 

quantitative risk assessment (QRA). NIOSH QRA is a science-based process that identifies 

workplace hazards, assesses the response to exposure of those hazards, and characterizes 

the associated health risks to inform risk management decisions.

NIOSH has over 30 years of experience conducting risk assessment. The methods used are 

consistent with National Research Council 1983 and 2009 guidance, which is the dominant 

benchmark in the field. The field of modern risk assessment, established by a broad range 

of legislation, has been highly productive in addressing issues of health, safety, and the 

environment. However, over the last few decades the science behind risk assessment has 

evolved and the utility, impact, and credibility of risk assessments have been challenged. To 

address these issues, it is important to have a transparent process by which risk assessments 

are conducted. This document describes the process that NIOSH uses to conduct QRA.

There is a pressing need for risk assessment, particularly of chemical substances. Over 50 

million U.S. workers are exposed to hazardous chemicals in the course of their work each 

year, either by skin contact or by inhaling vapors, gas, dust, or fumes. Toxic chemicals encountered in the workplace pose a wide range of health hazards to workers. Health effects may include irritation, sensitization, respiratory disease, cancer, and cardiovascular, 

immunological, and reproductive disorders. NIOSH conducts risk assessment when there is 

limited direct information to make a determination of a safe level of exposure for workers in 

order to prevent disease, injury, or death. Given limited information, assumptions in NIOSH 

risk assessment generally favor worker protection. This document and the resultant guidance, such as occupational exposure limits, should provide useful information to all stakeholders who use risk assessment information in the management of occupational hazards.

John Howard, MD

Director, National Institute for Occupational

Safety and Health

Centers for Disease Control and Prevention
iv NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Abstract

Exposure to on-the-job health hazards is a problem faced by workers worldwide. Unlike 

safety hazards that may lead to injury, health hazards can lead to various types of illness. 

For example, exposures to some chemicals used in work processes may cause immediate 

sensory irritation (e.g., stinging or burning eyes, dry throat, cough); in other cases, workplace chemicals may cause cancer in workers many years after exposure. There are millions 

of U.S. workers exposed to chemicals in their work each year. In order to make recommendations for working safely in the presence of chemical hazards, the National Institute for 

Occupational Safety and Health (NIOSH) conducts risk assessments. In simple terms, risk 

assessment is a way of relating a hazard, like a toxic chemical in the air, to potential health 

risks associated with exposure to that hazard. Risk assessment allows NIOSH to make recommendations for controlling exposures in the workplace to reduce health risks. 

This document describes the process and logic NIOSH uses to conduct risk assessments, 

including the following steps:

■ Determining what type of hazard is associated with a chemical or other agent; 

■ Collating the scientific evidence indicating whether the chemical or other agent 

causes illness or injury; 

■ Evaluating the scientific data and determining how much exposure to the chemical or 

other agent would be harmful to workers; and 

■ Carefully considering all relevant evidence to make the best, scientifically supported 

decisions.

NIOSH researchers publish risk assessments in peer-reviewed scientific journals and in 

NIOSH-numbered documents. NIOSH-numbered publications also provide recommendations aimed to improve worker safety and health that stem from risk assessment.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment v

Executive Summary

Introduction

Occupational health risks describe the potential and severity of adverse effects in workers 

from their exposure to workplace hazards. Herein, adverse effects refer to health effects 

that are changes in the morphology, physiology, growth, development, reproduction, or 

life span of an organism, system, or population that result in an impairment of functional 

capacity, an impairment of the capacity to compensate for additional stress, or an increase 

in susceptibility to other influences [IPCS 2004]. Safeguards, derived from a combination 

of scientific assessment, engineering, and best management practices, mitigate these risks. 

Risk assessment is an important tool for informed decision-making on workplace safeguards when there are limited data on risks. Since the 1990s, quantitative risk assessments 

conducted by the National Institute for Occupational Safety and Health (NIOSH) have buttressed recommendations on limiting chemical exposures and some other workplace hazards. The need for these recommendations is critical given estimates that over 50 million 

U.S. workers report frequent exposure to chemicals at work each year [Calvert et al. 2013]*

.

This report describes the underlying science and general approach NIOSH researchers use 

in conducting high quality, scientifically sound quantitative assessments of the risks associated with these workplace hazards. It focuses on chemical risk assessment practices; 

however, some of these practices have benefitted NIOSH assessments of other workplace 

hazards, such as ionizing radiation and noise. The report informs NIOSH risk assessors, 

other scientists, stakeholders, and the public on the NIOSH risk assessment process. It is 

one of many routine exchanges between NIOSH, its stakeholders, and the risk assessment 

community, both home and abroad, which act to ensure best practices in risk assessment 

supporting worker protection.

Risk Assessment Process

NIOSH risk assessments are typically carried out by a multidisciplinary team of epidemiologists, toxicologists, biostatisticians, industrial hygienists, and other exposure scientists 

(e.g., health physicists and chemists), hereafter referred to as risk assessors. NIOSH risk assessments are often conducted in response to investigations prompted by persons who are 

at risk (e.g., affected workers), risk managers (e.g., employers, regulators), or risk assessors, 

alone or in combination, who need information on the probability and severity of potential 

workplace hazards. NIOSH then develops a risk assessment plan containing two key components: (1) a conceptual model that identifies the hazard (sources, stressors, and pathways), 

persons potentially at risk, and apparent adverse effects; and (2) an analysis plan (work plan) 

that outlines the analytic components (i.e., data and methods) and interpretative approaches 

*Correspondence with authors (Calvert et al., 2013), who confirmed that about one third of workers participating in the 

2010 National Health Interview Survey reported either “frequent occupational skin contact with chemicals” in their main 

current job over the past year or “frequent occupational exposure to vapors, gas, dust, or fumes” at their longest-held job.
vi NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

(e.g., risk metrics) to be used. This approach is consistent with recommendations by the 

National Research Council (NRC) of the National Academies [NRC 2009]. Planning helps 

to ensure that the risk assessment applies the best scientific methods available, applies the 

highest-quality evidence, and addresses the needs of the decision-makers (risk managers). 

Initial plans comprise preliminary information that is subject to change as more information becomes available; therefore, plans may evolve throughout the risk assessment process.

NIOSH risk assessments determine the relationship between the occupational exposure 

and adverse effects leading to the development of a reference value, either a recommended 

exposure limit (REL) or a risk management level for carcinogens (RML-CA), to be used to 

guide exposure control. Data permitting, this determination is preferred to be quantitative. 

The quantitative risk assessment uses three major components of the risk assessment process that are completed sequentially: hazard identification, dose-response assessment, and 

risk characterization (Figure 1). Hazard identification is the systematic process for assessing the weight of evidence on whether an agent of interest causes an adverse effect in exposed workers. The findings from hazard identification are characteristic descriptions and 

information on the exposures of interest, any important cofactors (e.g., other risk factors, 

moderating factors, mediating factors, or confounders); modes and mechanisms of action; 

and conditions (e.g., pre-existing diseases) under which changes in exposures change the 

probabilities or timing of adverse effects. These data are prerequisites for conducting the 

dose-response assessment. The term dose-response is broadly defined as the relationship 

between the amount of an agent administered to, taken up by, or absorbed by an organism, 

system, or population and the change developed in that organism, system, or population 

in reaction to the agent. In NIOSH quantitative risk assessment, the dose-response is generally expressed as the conditional probability of the adverse effect in exposed workers at 

different levels of occupational exposure, given assumed levels for other direct causes of 

the adverse effect. In practice, the terms exposure and dose have been expressed in many 

ways over time and are often used interchangeably. By strict interpretation, exposure refers 

to contact between an agent (e.g., hazardous substance) and a target (e.g., human lung tissue), and dose is the amount of the agent administered to, taken up by, or absorbed by the 

target [IPCS 2004]. The dose-response assessment provides estimates of the dose-risk relationship for use in the third component of risk assessment, namely, risk characterization. 

Risk characterization is the qualitative and, wherever possible, quantitative determination 

of the probability of occurrence, including attendant uncertainties, of known and potential 

adverse effects in workers under defined conditions of exposure to an agent [IPCS 2004]. It 

reflects the culmination of the planning, problem formulation, and analysis phases of risk 

assessment to integrate the science of hazard identification and dose-response assessment 

with risk policy (e.g., target risks). NIOSH uses the output from risk characterization (i.e., 

a description of risk estimates and attendant uncertainties) in combination with information on available technology (e.g., analytic feasibility) to establish a sound basis for NIOSH 

recommendations. These recommendations inform decision-makers who are responsible 

for managing workplace risk. Figure 1 shows the components of risk assessment and their 

relationship with risk management.

NIOSH risk assessments follow the traditional NRC risk assessment steps except that 

NIOSH does not use the exposure assessment step [NRC 1983; NRC 2009]. In a risk assessment, exposure assessment is the process of estimating or measuring the magnitude, 

frequency, and duration of exposure to an agent, along with the number and characteristics 

of the population exposed [IPCS 2004; NRC 2009]. The exposure assessment is driven by 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment vii

the needs of environmental risk assessments which require information on the extent of 

human exposure to be able to project population risks and overall disease burdens in risk 

characterization. 

In contrast, NIOSH uses the information from the first two steps of the risk assessment to 

estimate health risks of individuals from the agent at various levels of exposure. This forms 

the risk basis for occupational exposure limits for use in risk management. The NIOSH 

mission, as stated in the Occupational Safety and Health Act, is “...to describe exposure 

levels that are safe for various periods of employment, including but not limited to the exposure levels at which no employee (emphasis added) will suffer impaired health or functional capacities or diminished life expectancy as a result of his work experience” [29 USC 

669 (a)(3)]. Therefore, the extent of exposure described as exposure assessment in the NRC 

formulation and the at-risk population are implicit. 

Similarly defined approaches are used by the Occupational Safety and Health Administration (OSHA) in determining Permissible Exposure Limits (PELs), and in the Integrated 

Risk Information System (IRIS) implemented by the U.S. Environmental Protection Agency (EPA). The EPA’s IRIS comprises two of the four steps (hazard identification and doseresponse assessment) described by the NRC. T﻿he omission of a formal exposure assessment 

step does not preclude the evaluation of exposure data that are pertinent to other risk assessment steps. For example, the quality of exposure data from observational studies is a 

common concern in hazard identification and dose-response assessment. 

Hazard Identification

Hazard identification is typically the lengthiest component of the risk assessment process. 

Identifying hazards requires knowledge of both the agent and the adverse effect. Furthermore, NIOSH risk assessors approach hazard identification in terms of supporting quantification of the dose-risk relationship; therefore, its findings are intended to define the 

Risk Assessment Risk Management

Figure 1. NIOSH risk assessment and risk management processes.
viii NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

population at risk, the agent, the adverse effect(s) of interest, and any cofactors (e.g., effect modifiers, confounders, or other sources of uncertainty) in sufficient detail to conduct 

sound quantitative dose-response analyses. Consistent with systematic review principles, 

the general framework for gathering and evaluating relevant human and animal study data 

consists of five basic steps: (1) define the causal questions of interest and develop criteria for 

study (data) selection; (2) develop literature search protocol and conduct search; (3) review, 

identify, and select relevant information; (4) evaluate and integrate evidence across studies; and (5) synthesize and interpret findings [Rhomberg et al. 2013]. The paths to meeting 

these steps can vary widely with the specific scientific context. In general, risk assessors 

judge the weight of evidence in study evaluation by using multiple factors—such as strength 

of association, consistency, specificity, temporality, biological gradient, plausibility, coherence, experiment, and analogy—as first posited by Sir Austin Bradford Hill [1965]. For 

synthesis and interpretation, risk assessors consider the following factors:

■ The design and conduct of studies providing data for risk assessment:

ȣ Are study results generalizable and relevant to the risk assessment problem? 

ȣ Are results reproducible? 

ȣ Was confirmation or refutation of findings attempted? 

ȣ What factors may jeopardize external validity of the results?

■ The characterization of exposure, dose, and adverse effect: 

ȣ What is the utility of the study data for hazard identification? 

ȣ Will these data be suitable for inclusion in the database for the dose-response 

assessment? 

ȣ What are the sources of measurement error and their potential effects on the doseresponse association?

■ The degree of data certainty and strength of findings in support of hazard 

identification: 

ȣ Have researchers used sound statistical methods? 

ȣ Are results robust under alternative assumptions? 

ȣ Have results been over- or misinterpreted? 

ȣ How likely are findings due to chance, bias, residual confounding, or other sources 

relevant to internal validity of the study? 

To improve efficiency, NIOSH often uses hazard identification by other agencies, such as 

the U.S. National Toxicology Program (NTP), EPA, OSHA, the Mine Safety and Health 

Administration (MSHA), the Agency for Toxic Substances and Disease Registry (ATSDR), 

the European Chemical Agency (ECHA), and the International Agency for Research on 

Cancer (IARC). These agencies have a long history of identifying hazards by using sound 

and transparent methodologies. Information on hazard identification is also available in 

the literature.

Relevant data primarily stem from human epidemiologic and animal toxicologic studies. 

Ideally, the direct estimation of risk from human data is always preferred to using data 

from experimental animal studies, because: (1) data reflecting actual exposures and responses within the population of interest are inherently superior for risk assessment; and 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment ix

(2) the uncertainty in extrapolating data from animal toxicologic studies to predicting human risks can be much larger than that in well-designed epidemiologic studies [HertzPicciotto et al. 1995; Smith 1988; Stayner et al. 1999]. Although some epidemiologic data 

may arise from experimental designs, most information pertinent to risk assessment stems 

from observational studies of working populations (e.g., cohort and case-control studies). 

Human data are not without limitations; therefore, risk assessments tend to rely on a combination of human epidemiologic and animal toxicologic data for hazard identification and 

dose-response analyses. In fact, human studies tend to provide evidence of an association 

between exposure and disease, which can guide the choice of agents, exposure routes, and 

pathological endpoints for examination in toxicological studies that may contribute greatest to quantifying risks.

In environmental risk assessments, exposure assessment is considered a separate step for 

assessing the likelihood of exposure for estimating population risks and/or disease burden. 

In contrast, NIOSH risk assessments, as described herein, estimate the risks to a hypothetical working population from a known exposure. Although exposure probabilities are 

not typically calculated, dose-response analyses include exposure information; therefore, 

NIOSH systematically assesses the availability, magnitude, and validity of exposure data 

used in relevant studies as a part of hazard identification and applies this information, as 

applicable, in the dose-response assessment. 

Dose-response Assessment

The second component of NIOSH risk assessment is the dose-response assessment. Given 

exposure and outcomes of interest, the aim of the dose-response assessment is to obtain 

reliable and valid estimates of the point of departure (PoD) in a cause-and-effect relationship for effects with a response threshold, or the risk at levels of exposure (e.g., the risk per 

unit dose) for non-threshold effects. Here, the PoD refers to a point on the dose-response 

curve that is established from experimental or observational data that corresponds to a level 

of no (or low) effect without substantial extrapolation. These estimates are essential to risk 

characterization. 

NIOSH generally obtains dose-response estimates via statistical models constructed to provide the conditional expectation of the dependent variable (the adverse effect) given one or 

more explanatory variables, but at least including the variable describing the agent exposure 

of interest. Model input data stem from toxicologic and/or epidemiologic investigations 

identified and assessed in hazard identification. Because different model specifications can 

lead to different estimates, a key step in dose-response analysis is model selection. Clearly, it 

is preferable to base model selection on biologic plausibility, although a strong advantage of 

one model among several plausible models is rarely evident. Furthermore, data from most 

studies are imperfect and potentially incomplete. In lieu of available statistical techniques 

and algorithms designed to deal with data imperfections, the risk assessor may have to rely 

on assumptions based on scientific judgment. 

Thus, another important part of the dose-response assessment is sensitivity analysis. In a 

sensitivity analysis, risk assessors quantitatively evaluate plausible alternative risk assessment strategies, defaults, and assumptions for their impact on risk estimates. In addition 

to providing a measure of analysis robustness, sensitivity analyses aid the risk manager by 

providing a range of plausible estimates of the dose-risk relationship.
x NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Risk Characterization

The final step in NIOSH risk assessment is risk characterization. It is the translation of 

information from hazard identification and dose-response assessment into a basis, completely or in part, for recommendations on limiting workplace exposure. The framework of 

NIOSH risk characterization centers on a choice between two distinct approaches, based 

primarily on the evidence supporting the absence or presence of an impairment threshold. 

For effects with a response threshold, NIOSH typically adjusts the PoD in dose-response 

analysis by using factors that account for natural heterogeneity (e.g., interspecies variability or interindividual variability) to arrive at an estimate of a safe dose. Here the term safe 

implies that excess risk at this exposure level is absent or negligible. NIOSH used this approach in its risk assessment of nonmalignant pulmonary effects from exposures to carbon 

nanotubes and nanofibers [NIOSH 2013b]. In contrast, consider a causal agent that is neither necessary nor sufficient to cause disease (e.g., cancer). In this case, cause and effect is 

best described as a relationship between the probability of disease and the dose level that is 

absent of a dose threshold. Cancer from low-dose ionizing radiation is a classic example of 

an effect generally considered having residual risk at any level of dose. In a NIOSH risk assessment of radon exposure and lung cancer in uranium miners [NIOSH 1987], a safe level 

of ionizing radiation exposure was not assured; therefore, residual lung cancer risk under 

select exposure scenarios was estimated with probabilistic means. When effects appear to 

be without a response threshold, NIOSH obtains quantitative estimates of low-dose risk 

by model-based extrapolation of the risk at doses below the observed data. To illustrate, 

probabilistic models have been used by NIOSH to estimate the dose that would cause a lifetime excess cancer risk of 1 in 1000 from occupational exposure to hexavalent chromium 

[NIOSH 2013a] and titanium dioxide [NIOSH 2011]. There are instances in which the risk 

characterization approach is less dependent on a determination of whether the process has 

a threshold. For example, the response threshold may reside far below the observable range 

in dose-response analyses and may vary widely among exposed individuals. Under this 

condition, NIOSH may opt for assessing lifetime risks based on model extrapolation. Similarly, an effect that is generally considered stochastic (e.g., cancer) may be indirectly caused 

by exposure through a precursor effect (e.g., inflammation) residing on the causal pathway 

that has a threshold. If this is the only important pathway present, then a safe level of exposure is more likely (i.e., a level below inflammation). 

An important consideration of risk-based characterization is the selection of a target risk, 

which is a single level of risk broadly considered tolerable, given reasonable and practical 

risk management. There are multiple methods and principles available for establishing risk 

acceptance criteria, and the adopted methods and principles will undoubtedly influence 

the choice of target risk. Thus, risk acceptance (or tolerance) criteria are more likely to be 

unique to the situation at hand rather than be pre-defined [Rodrigues et al. 2014; Vanem 

2012]. Nevertheless, NIOSH has established a target risk level for non-threshold carcinogens of one excess case per 10,000 workers continuously exposed over a 45-year working 

lifetime [NIOSH 2017]. This level is a starting point for initiating a risk management process. The setting of target risk levels for other outcomes is a fundamental component of 

risk management; therefore, actions are primarily the responsibility of the decision-makers 

and not the risk assessor. As such, a detailed discussion on the various risk management 

principles in play for determining these levels is beyond the scope of this report, although 

discussions are available in several published reports [Aven 2016; HSE 2001; Rodrigues et 

al. 2014; Tchiehe and Gauthier 2017; Vanem 2012]. Finally, health risk is but one aspect 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment xi

typically needed to derive a target risk level, given that risk tolerance can depend on the 

combination of individual, societal, economic, and environmental impacts. Although employers in managing risks may consider these other factors, NIOSH quantitative risk assessment is solely focused on characterizing health risks.

Conclusions

The identification and quantification of occupational risk are paramount to worker protection. NIOSH has a long, rich history of systematically assessing workplace hazards and 

communicating recommendations aimed to mitigate associated risks. As such, most recognize NIOSH as a leader in occupational risk assessment and often seek its expertise in this 

field. This report aids others in their understanding of the NIOSH risk assessment process. 

To this end, the report describes the NIOSH approach to addressing hazard identification, 

dose-response analyses, and risk characterization, including demonstrated examples of 

NIOSH risk assessments. 

Above all, the NIOSH approach stresses careful attention to the aims of the risk assessment 

throughout the assessment process. It is important to thoroughly investigate the robustness of key assumptions and provide transparency for both the main analysis and analyses 

of alternative modeling strategies and defaults. Maintaining mindfulness of the intended 

audience is of utmost importance; therefore, NIOSH risk assessors endeavor to follow the 

guiding principles of transparency, clarity, consistency, and reasonableness when conducting risk assessment (Table 1).

Table 1. Guiding principles of NIOSH risk assessments

Principle Description Criteria for risk characterization

Transparency Explicitness is essential 

in the risk assessment 

process.

Use a risk analysis plan.

Describe assessment approach, 

assumptions, extrapolations and use of 

models. 

Describe plausible alternative assumptions. 

Identify data gaps. 

Distinguish science from policy.

Describe uncertainty. 

Describe relative strength of assessment.

Clarity The assessment itself 

is free from obscure 

language and is easy to 

understand.

Be brief and concise.

Use plain English (avoid jargon). 

Avoid technical terms. 

Use simple tables, graphics, and equations.

(Continued)
xii NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Table 1 (Continued). Guiding principles of NIOSH risk assessments

Principle Description Criteria for risk characterization

Consistency The risk assessment 

conclusions harmonize 

with those in other risk 

assessments and with 

other NIOSH actions.

Use this technical report.

Follow NIOSH policies on technical 

writing and peer review.

Place assessment in context with similar 

risk assessments. 

Reasonableness The risk assessment 

uses sound science and 

sensible judgment.

Use review by peers. 

Use best available scientific information. 

Use good judgment. 

Adapted from the EPA Risk Characterization Handbook [Fowle and Dearfield 2000].

Risk assessment science is continuously evolving. Methods currently under development 

may provide additional, powerful tools to assess risks to workers based on very limited data. 

Validation of these new approaches is a critical need. In efforts to stay abreast of the science, 

NIOSH will continue to embrace new methodologies, but it will do so with appropriate caution and deliberate evaluation of new techniques and approaches.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment xiii

Contents

Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii

Abstract. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iv

Executive Summary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Risk Assessment Process. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v

Hazard Identification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii

Dose-response Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix

Risk Characterization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . x

Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi

Acknowledgments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii

1 Background. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1 NIOSH Risk Assessment History. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Purpose and Scope. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Risk Assessment Plan (Problem Formulation). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Hazard Identification. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

4.1 Hill’s Views on Causation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

4.2 Adverse Effects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.2.1 Cancer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4.2.2 Non-cancer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.3 Human Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.3.1 Epidemiologic Study Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

4.3.2 Health Effect Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.3.3 Exposure Methods and Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3.4 Factors Compromising Internal Validity . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

4.4 Laboratory Animal Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

4.4.1 Relevance and Appropriateness of the Animal Model. . . . . . . . . . . . . . . . . . 33

4.4.2 Animal Toxicologic Study Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

5 Dose-Response Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.2 Dose-Response Modeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.2.1 Parametric Dose-Response Modeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.2.2 Model Uncertainty and Model Averaging. . . . . . . . . . . . . . . . . . . . . . . . . . 45

5.2.3 Nonparametric Modeling. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.3 Point of Departure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.3.1 NOAEL/ LOAEL-Based Assessments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
xiv NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

5.3.2 The Benchmark Dose Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

5.4 Selecting a Dose-Response Modeling Method. . . . . . . . . . . . . . . . . . . . . . . . . . . 52

5.5 Laboratory Animal Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

5.5.1 Parallelogram Approach. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.6 Dosimetry Adjustments for Human Equivalent Concentrations. . . . . . . . . . . . 55

5.6.1 Respiratory Tract Regions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.6.2 Particle Exposure. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

5.6.3 Gas and Vapor Exposures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.7 Dose-Response Modeling with Epidemiologic Data. . . . . . . . . . . . . . . . . . . . . . 69

5.7.1 Limited Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

5.7.2 Longitudinal Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.8 Alternative Analysis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.8.1 Choice of Adverse Effect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.8.2 Model Uncertainty Analyses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

5.8.3 Extrapolation Methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

6 Risk Characterization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

6.1 Risk Definitions Common to NIOSH Risk Assessment. . . . . . . . . . . . . . . . . . . 81

6.2 Risk Characterization Framework. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

6.2.1 Carcinogens. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

6.2.2 Non-Carcinogens. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

6.3 Using Risk Assessment as a Basis for RELs or RML-CAs. . . . . . . . . . . . . . . . . . 90

6.3.1 Target Risk Levels. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

6.3.2 Working Lifetime and Persistent Risk. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

6.3.3 Competing Risks in Projecting Lifetime Risk. . . . . . . . . . . . . . . . . . . . . . . 93

6.3.4 Application of Uncertainty Factors (UFs). . . . . . . . . . . . . . . . . . . . . . . . . . 95

6.3.5 Characterizing Multiple Outcomes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97

6.4 Special Considerations When Developing a Short-Term 

Exposure Limit (STEL). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

6.5 Addressing Uncertainty and Variability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

6.6 More on NIOSH RELs and RML-CAs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

7 Conclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

8 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107

Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151

B.1 Validity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

B.2 Selection Bias. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

B.3 Information Bias. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153

B.3.1 Measurement Error and Misclassification. . . . . . . . . . . . . . . . . . . . . . . . . . 154

B.4 Confounding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 159

B.5 Healthy Worker Effects. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160

B.5.1 Healthy Worker Hire Effect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161

B.5.2 Healthy Worker Survivor Effect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment xv

B.5.3 Length of Follow-up. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

B.6 Other Potentially Important Sources of Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

B.6.1 Model-form Misspecification Error. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 164

B.6.2 Publication, Interpretation, and Analysis Biases. . . . . . . . . . . . . . . . . . . . . 164

B.6.3 Effect Modification and Interaction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165

B.6.4 Random Error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166

Appendix C . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

C.1 Occupational Exposure Banding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168

C.2 Emerging Alternatives To Assessing Apical Endpoints. . . . . . . . . . . . . . . . . . . . 169

C.3 Biomarkers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169

C.4 Use of Genetics and Epigenetics in Risk Assessment. . . . . . . . . . . . . . . . . . . . . 170

C.5 Molecular Toxicology and High-Throughput Analysis. . . . . . . . . . . . . . . . . . . . 171

C.6 Quantitative Structure-Activity Relationships . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

C.6 Nanomaterials Risk Assessment. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

C.6.1 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173

C.6.2 Dose Normalization In Vitro and In Vivo. . . . . . . . . . . . . . . . . . . . . . . . . 173

C.6.3 Correlation of In Vitro and In Vivo Responses . . . . . . . . . . . . . . . . . . . . . 173

C.7 Alternative Methods for Nanomaterials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

C.7.1 Comparative Potency Estimation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

C.7.2 Hazard Classification/Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174

C.7.3 Validation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175

C.8 Non-Chemical and Cumulative Risk Assessment. . . . . . . . . . . . . . . . . . . . . . . . 175
This page intentionally left blank.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment xvii

Acknowledgments

This document was authored by a multidisciplinary team within the Division of Science 

Integration (Paul Schulte, PhD, Director). Contributing authors were Robert D. Daniels, 

PhD; Stephen J. Gilbert, MS; Senthilkumar Perumal Kuppusamy, PhD; Eileen D. Kuempel, 

PhD; Robert Park, MS; Sudha P. Pandalai, MD; Randall Smith, MA; Matthew W. Wheeler, 

PhD; Christine Whittaker, PhD; and Paul Schulte. NIOSH acknowledges John A. Bailer, 

PhD; Theresa E. Boots, MS; David A. Dankovic, PhD; Nathan M. Drew, MS; Douglas O. 

Johns, PhD; Mary K. Schubauer-Berigan, PhD; and James T. Wassell, PhD, who provided 

helpful insights following reviews of earlier drafts. NIOSH also knowledges Vanessa B. 

Williams and Seleen S. Collins for editing and production support. 

Special appreciation is expressed to the following individuals for serving as independent, 

external peer reviewers and providing comments that contributed to the development of 

this document: 

Vanessa Beaulac, MS

Air Health Science Division 

Health Canada

Jessica Myers, PhD

Toxicology Division

Texas Commission on Environmental Quality

Valentine Schaeffer, PhD

Advisor to the Director, Directorate of Standards & Guidance

Occupational Safety and Health Administration

Leslie Stayner, PhD

Professor, Epidemiology and 

Biostatistics Division

University of Illinois at Chicago 

School of Public Health
This page intentionally left blank.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 1

1 Background

In their highly cited article, Kaplan and Garrick 

[1981] first posited that an analysis of risk is an effort to answer three questions:

1. What can happen?

2. How likely is it that it will happen?

3. What are the consequences if it does happen? 

Thus, the hazard (the potential source of harm) imposes risk (the likelihood of hazard causing harm) 

that is a function of both opportunity (likelihood) 

and consequence, as generally expressed by the following equation: 

Risk = f (hazard, consequence, opportunity)

Risk is omnipresent and diverse in the human experience; therefore, steps are necessary to manage 

the many kinds of risks in our daily lives, such as 

business risk, social risk, political risk, and occupational risk. To illustrate, the risk function for health 

risk associated with hazardous exposure is: 

Risk = f (target, exposure, toxicity, opportunity)

Where:

■ The target is the biological entity (e.g., an 

organ, an individual, or a population) that is 

exposed,

■ Exposure is contact between the agent (a 

chemical, biological, or physical entity) and 

the target,

■ Toxicity is the deleterious biological effects (i.e., 

the adverse effects) elicited by the agent, and 

■ Opportunity is the likelihood of cause 

and effect.

In this example, the target and the exposure form the 

hazard, while toxicity represents the consequence.

Occupational risk is the potential and severity of 

adverse health effects in workers from their exposure to workplace hazards. In this context, the adverse effect is simply a specified unfavorable change 

in health status of a worker from known exposure. 

Safeguards, carefully derived from scientific assessment and best practices, reduce occupational 

risks. Risk assessment is an important tool for informed decision-making on workplace safeguards 

when there are limited data on risks. For example, 

risk assessment provides the scientific underpinnings to authoritative recommendations, such as 

occupational exposure limits (OELs). Risk assessment conducted by the National Institute for Occupational Safety and Health (NIOSH) provides 

the foundation for Recommended Exposure Limits 

(RELs) and Risk Management Limits for Carcinogens (RML-CAs) for chemicals and other workplace hazards, such as ionizing radiation and noise. 

NIOSH first considered the need to quantify occupational risks when standards for benzene exposure promulgated by the Occupational Safety and 

Health Administration (OSHA) were challenged in 

the 1980s, resulting in the well-cited Supreme Court 

decision “Industrial Union Department, AFL-CIO 

v. American Petroleum Institute, 448 U.S. 607 

[1980],” hereafter referred to as the Benzene Decision. That decision essentially established a need 

to quantify the risk from occupational exposure 

as a basis for exposure limits. In response, NIOSH 

developed a “Risk Assessment Team,” which later 

expanded to a “Risk Assessment Activity.” This 

team of toxicologists, epidemiologists, exposure 

scientists, and statisticians provided quantitative 

risk assessments for radon [Hornung and Meinhardt 1987; NIOSH 1987]; ethylene glycol monomethyl ether, ethylene glycol monoethyl ether, and 

their acetates [NIOSH 1991]; cadmium [Stayner 

et al. 1992a; Stayner et al. 1992b]; 1,3-butadiene 

[Dankovic et al. 1993]; and coal dust [Kuempel et 

al. 1997]. In 1995, the Risk Assessment Activity was 

formally organized. Since that time, NIOSH staff 

have conducted quantitative risk assessments for 

a wide variety of agents, including diesel exhaust 
2 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

[Stayner et al. 1998], 1,3-butadiene [Stayner et al. 

2000], asbestos [Stayner et al. 1997], silica [Park et 

al. 2002; Rice et al. 2001], noise (with and without 

co-exposure to carbon monoxide) [NIOSH 1998], 

titanium dioxide [NIOSH 2011], hexavalent chromium [NIOSH 2013a], carbon nanotubes and 

nanofibers [NIOSH 2013b], diacetyl and 2,3-pentanedione [NIOSH 2016], manganese in welding 

fume [Park and Berg 2018; Park et al. 2009], toluene diisocyanate [Daniels 2018], and metalworking 

fluids [Park 2018] (Table 1-1).

In general, risk assessment is a process in which 

NIOSH characterizes the risk of adverse health effects from workplace hazards by using information 

from hazard identification (exposure and outcome) 

and dose-response assessment. This evaluation 

determines whether an exposed population is at 

greater-than-expected risk of adverse effects, such 

as disease (cancer or non-cancer) or injury. Once 

the hazard is identified, the magnitude and nature 

of its associated risk can be explored further, using 

either qualitative or quantitative approaches. Qualitative risk assessments are descriptive and indicate 

whether an adverse effect is likely or unlikely under 

specified conditions of exposure. Quantitative risk 

assessments provide numerical estimates of risks 

based on mathematical modeling. For example, a 

quantitative risk assessment may relate conditions 

of workplace exposure to an estimate of increased 

lifetime risk of a disease or injury. 

Quantitative risk assessments require (1) data on 

exposures relevant to the adverse effect of interest; 

(2) data on the adverse effect associated with the 

exposure of interest; and (3) a mathematical model 

describing that dose-response relationship. The list 

of completed risk assessments shown in Table 1-1

clearly indicates the preference for directly estimating worker risks using data from human studies compared to experimental animal studies. Risk 

assessments based on epidemiologic, populationbased studies have real-world relevance to workers, 

but they generally suffer from limitations inherent 

to study design and available data. Risk assessments 

based on experimental animal data provide detailed 

information on the dose-response relationships; 

however, there is often concern about the validity 

of extrapolating animal-based risk assessments to 

humans, who generally have much lower and more 

variable exposures. The integration of mechanistic, 

animal, and human data is important for developing a thorough understanding of the risks. 

The risk assessment process has become increasingly complex over the past several decades. In occupational safety and health regulation, the need 

to quantify risk became apparent with the Benzene 

Decision, which established that OSHA could not 

issue a standard without demonstrating a significant risk of material health impairment. The ruling allowed (but did not require) numerical criteria 

to be used to determine whether a risk is “significant.” As a result, risk assessment became standard 

practice in OSHA rulemaking for health standards, 

and quantitative risk assessments are now preferred 

whenever data, modeling techniques, and biological understanding are adequate to support their 

development. NIOSH has adopted many of the 

same risk assessment practices as OSHA in order 

to keep the analyses relevant and meaningful within OSHA’s regulatory context.

1.1 NIOSH Risk Assessment 

History

Historically, NIOSH employed a variety of methods 

to establish recommendations intended to prevent 

adverse effects in workers. NIOSH considered the 

health effects associated with experimental or observed exposure concentrations and applied a safety factor to ensure that even the most susceptible 

individual would be generally protected from a hazard. One major exception was in addressing issues 

of carcinogenicity. When evaluating carcinogens, 

NIOSH typically assumed that no exposure could 

be considered safe. This led to RELs for carcinogens 

that were not numerical, but directed employers to 

keep exposures as low as feasible [Fairchild 1976].

In its first decades, NIOSH was largely uncertain 

about the utility of dose-response modeling, especially for carcinogens. In 1982, NIOSH commented 
NIOSH CIB 69 

• NIOSH Practices in Occupational Risk Assessment

3

Table 1-1. Examples of NIOSH quantitative risk assessments (in chronologic order)

Agent Adverse Effect* Dose-response Assessment† Risk Characterization Reference

Radon Lung cancer Epidemiologic, Cox proportional 

hazards regression

Extrapolation, excess lifetime risk, target risk unspecified

[Hornung and Meinhardt 1987; NIOSH 1987]

EGME, EGEE, EGMEA, EGEEA 

Reproduction, developmental, hematotoxic effects 

Toxicologic, NOAEL and LOAEL assessments

PoD/UF [NIOSH 1991]

Cadmium Lung cancer Epidemiologic, Poisson and CPH 

regression, additive relative rate function

Extrapolation, excess lifetime risk, target risk unspecified

[Park et al. 2012; Stayner et al. 1992a; Stayner et al. 1992b]

1,3-butadiene Leukemia Toxicologic, Weibull time-to-tumor 

regression model, animal to human extrapolation,epidemiologic and toxicologic literature review

Extrapolation, excess lifetime risk, target risk unspecified

[Dankovic et al. 1993][Stayner et al. 2000]

Asbestos Lung cancer, asbestosis Epidemiologic, Poisson regression, 

additive relative rate function (cancer), power function (asbestosis) 

Extrapolation, excess lifetime risk, target risk unspecified

[Stayner et al. 1997]

Coal mine dust Coal workers’ pneumoconiosis, 

progressive massive fibrosis, pulmonary dysfunction 

Epidemiologic, logistic and multiple linear regression

Extrapolation, excess lifetime risk, target risk unspecified

[Kuempel et al. 1997]

Diesel exhaust Lung cancer Toxicologic and epidemiologic 

(review)

Extrapolation, excess lifetime risk, target risk unspecified

[Stayner et al. 1998]

Silica Lung cancer Epidemiologic, Poisson regression, 

additive relative rate function 

Extrapolation, excess lifetime risk, target risk unspecified

[Rice et al. 2001]

(Continued)
4 NIOSH CIB 69

• NIOSH Practices in Occupational Risk Assessment

Table 1-1. Examples of NIOSH quantitative risk assessments (in chronologic order)

Agent Adverse Effect* Dose-response Assessment† Risk Characterization Reference

Silica Non-malignant lung disease Epidemiologic, Poisson regression, 

additive relative rate function 

Extrapolation, excess lifetime risk, target risk unspecified

[Park et al. 2002]

Noise Material hearing impairment Epidemiologic, logistic regression Extrapolation, excess 

lifetime risk with no target risk level specified

[NIOSH 1998; Prince et al. 2003]

Hexavalent chromium Lung cancer Epidemiologic, Poisson regression 

linear ERR model

Extrapolation, excess lifetime risk, 10-3 target 

risk

[NIOSH 2013a; Park et al. 2004]

Titanium dioxide Lung cancer Toxicologic, nonlinear extrapolation, 

BMD model averaging, quantal endpoint 

Extrapolation, excess lifetime risk, 10-3 target risk

[NIOSH 2011]

Carbon nanotubes and nanofibers

Non-malignant adverselung effects

Toxicologic, NOAEL and BMD assessments

PoD/UF [NIOSH 2013b]

Diacetyl and 2,3-pentanedione

Pulmonary dysfunction Epidemiologic, linear extrapolation, 

multiple regression, Poisson regression

Extrapolation, excess lifetime risk, 10-3 target 

risk

[NIOSH 2016; Park and Gilbert 2018]

Toluene diisocyanate Asthma Epidemiologic, Poisson regression 

linear ERR model, and BMD

PoD/UF and extrapolation, excess lifetime risk, 10-3 target risk

[Daniels 2018]

Manganese Neurobehavioral impairment Epidemiologic, BMD Extrapolation, excess 

prevalence, target risk unspecified 

[Park and Berg 2018; Park et al. 2009]

Metalworking fluids Cancer (multiple sites) Epidemiologic, Poisson regression 

linear ERR model

Extrapolation, excess lifetime risk, 10-3 target 

risk

[Park 2018]

*Analyses may have considered multiple adverse effects. The adverse effect shown is the primary effect examined. †The dose-response assessment refers to the primary source supporting final models and/or recommendations on risk-based exposure limits.

Abbreviations: BMD, benchmark dose; EGEE, ethylene glycol monoethyl ether; EGEEA, ethylene glycol monoethyl ether acetate; EGMEA, ethylene glycol monomethyl ether; EGMEA, ethylene glycol monomethyl ether acetate; ERR, excess relative rate; LOAEL, lowest observable adverse effect level; NOAEL, no observable adverse effect level; CPH, Cox proportional hazards; PoD, point of departure; UF, uncertainty factor.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 5

to OSHA on an Advance Notice of Proposed Rulemaking on the Identification, Classification and 

Regulation of Potential Occupational Carcinogens 

[NIOSH 1982]: 

Because our understanding of the mechanism 

of carcinogenicity is incomplete, our use of 

mathematical models to predict its outcome 

must be employed with extreme caution. To select a model or models from among the many 

choices and to have them incorporated into Administration policy will not resolve those issues.

However, just a few years later, NIOSH engaged 

in quantitative risk assessment. As cited in 1986 

NIOSH testimony on OSHA’s Proposed Rule on Occupational Exposure to Benzene [NIOSH 1986a], 

NIOSH drew on the Benzene Decision, which focused on significant risk of material impairment of 

health, and United Steelworkers of America v. Marshall, 647 F. 2d 1189 (D.C. Cir. [1980]), cert, denied 

101 S. Ct. 3148 [1981], hereafter referred to as the 

Lead Decision, to conclude:

These two decisions provided the impetus for 

the inclusion of a quantitative risk assessment 

effort in the standards recommending program of NIOSH [NIOSH 1986a].

The first reference to quantitative risk assessment in 

NIOSH policy statements was in the 1986 NIOSH 

testimony to OSHA recommending 0.1 ppm as a 

permissible exposure limit (PEL) for benzene, based 

largely on findings from a NIOSH risk assessment 

using epidemiologic data [Rinsky et al. 1987]. Risks 

at 0.1 ppm were determined to be around one excess cancer per 1000 workers over a working lifetime. However, this initial risk-based REL was never 

incorporated into an updated Criteria Document 

for benzene, and later documentation referred to 

the limit of quantification (LOQ) of the analytical 

method (also around 0.1 ppm at the time) as the 

basis of the 0.1 ppm REL [NIOSH 1988]. Here, the 

LOQ is the amount or concentration of the analyte 

at which quantitative results can be reported with a 

high degree of confidence, which is based on assayspecific acceptance criteria [NIOSH 1995b].

In 1986, although NIOSH did not conduct its own 

risk assessment for formaldehyde, NIOSH testified 

that the OSHA risk assessment for formaldehyde 

was acceptable [NIOSH 1986b]. This risk assessment used animal bioassay data to estimate the human cancer risk of 3.46 cases per 1000 workers exposed over a working lifetime at the proposed PEL 

of 3 ppm. However, the NIOSH REL of 0.016 ppm 

as an 8-hour time-weighted average (TWA) was 

based on the lowest concentration that was considered quantifiable at the time. 

In 1987, NIOSH published its first Criteria Document to include a quantitative risk assessment: Criteria for a Recommended Standard for Occupational 

Exposure to Radon [NIOSH 1987]. The risk assessment was based on epidemiologic data on excess 

lung cancer in underground uranium miners exposed to radon [Hornung and Meinhardt 1987]. 

The risk assessment found that continuous exposure to radon progeny concentrations of one Working Level Month (WLM) annually over a working 

lifetime corresponded to 5–10 excess lung cancers 

per 1000 miners. The risk from radon exposure 

versus the feasibility of controlling exposures was a 

point of discussion in the document. The REL was 

ultimately based on the limits of control technology at the time; however, NIOSH also communicated risks at this level, which supported additional 

recommendations for continued control technology development.

NIOSH risk assessments during the 1990s largely 

incorporated data from well-designed occupational epidemiologic studies that had become a 

mainstay of the Institute’s field studies program. 

Epidemiological risk assessments of lung cancer in 

humans were conducted for cadmium, chrysotile 

asbestos, and diesel exhaust [Stayner et al. 1998; 

Stayner et al. 1997; Stayner et al. 1992a; Steenland 

et al. 1998]. Cancer as a result of worker exposures 

to ethylene oxide was also examined [Steenland 

et al. 2003]. NIOSH also conducted worker-based 

risk assessments for various lung function measures 

after coal dust exposure and for hearing loss 

after noise exposure [NIOSH 1995a; Prince et al. 

2003]. In addition, although not a complete risk 
6 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

assessment, physiologically based pharmacokinetic 

(PBPK) modeling was used for dose estimation 

in a worker study of 2,3,7,8-Tetrachlorodibenzop-dioxin (TCDD) exposure [Lawson et al. 2004]. 

Animal-based risk assessments were conducted 

to predict human risks in the absence of sufficient 

human data. Toxicologic-based examples include 

assessments of 1,3-butadiene and cancer at various sites in the mouse and rat, and assessments of 

glycol ethers and reproductive effects in the mouse, 

rat, and rabbit [Dankovic et al. 1993; NIOSH 1991; 

Stayner et al. 2000].

In the 2000s and beyond, the need for quantitative 

risk estimates preferentially based on epidemiologic data resulted in risk assessments becoming 

increasingly complex. Advances in risk assessment have included innovations in reconstructing 

past exposures in epidemiological studies, expansion of statistical modeling techniques, increased 

understanding of the role of particle dosimetry 

issues in risk assessment, and exploration of doseresponse modeling for non-cancer health endpoints. Advancements such as new techniques in 

statistical modeling methods to account for survivor bias in human studies, the incorporation of 

genetics and genomics into risk assessment, and 

the potential for using quantitative structure activity relationships for risk assessment pose many 

challenges and opportunities for the future [Buckley et al. 2015; Comber et al. 2003; Schulte et al. 

2015; Weitzel et al. 2011].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 7

2 Purpose and Scope

Quantitative risk assessment is a foundation of authoritative recommendations. NIOSH conducts 

high quality, scientifically sound quantitative assessments of workplace hazards as input for developing its Criteria Documents, including establishing the basis for RELs and alternative forms 

of authoritative recommendations, such as hazard 

banding. This document describes the underlying 

science and general approach used by NIOSH researchers when conducting risk assessments, defined as the (preferably quantitative) determination 

of the relationship between the predicted exposure 

and adverse effects in workers. This information is 

intended for scientists, stakeholders, and the public to improve their understanding of the NIOSH 

risk assessment process. Every risk assessment is 

unique; therefore, situations may arise which require steps that are not specifically addressed in 

this report. Furthermore, discussions on NIOSH 

risk management and risk communication practices that typically follow the completion of its risk 

assessments are beyond the scope of this report. In 

particular, exposure assessment, as described by 

the National Research Council (NRC) [NRC 1983], 

may improve risk management practices by identifying and assessing efficiencies of feasible control 

options. Although important to the NIOSH mission of protecting workers, these activities are outside the scope of this document.

In developing its program, NIOSH benefitted from 

seminal reports by the NRC, which lay the foundation for modern risk assessment [NRC 1983; 

NRC 2009]. The NRC paradigm identified four 

major steps: hazard identification, dose-response 

assessment, exposure assessment, and risk characterization. This basic construct is used throughout 

the risk assessment community; however, some 

authoritative bodies have described the process 

using different terms and groupings of steps. For 

example, the Integrated Risk Information System 

(IRIS) includes the first two steps of the NRC risk 

assessment process. Also, the International Organization for Standardization (ISO) describes risk 

assessment as an overall process comprising three 

steps, namely, risk identification, risk analysis, and 

risk evaluation, which best correspond to hazard 

identification, dose-response assessment, and risk 

characterization of the NRC paradigm [ISO 2018]. 

Similarly, three NRC steps (hazard identification, 

dose response assessment, and risk characterization) are most applicable to NIOSH risk assessment, as shown in Figure 2-1. 

1. Hazard identification is the identification of the 

type and nature of adverse effects that an agent 

has an inherent capacity to cause in an organism, 

system, or population [IPCS 2004]. An adverse 

effect is defined as the specified change in the 

morphology, physiology, growth, development, 

reproduction, or life span of an organism, system, 

or population that results in an impairment of 

functional capacity, an impairment of the capacity to compensate for additional stress, or an increase in susceptibility to other influences [IPCS 

2004]. Hazard identification is the initial stage of 

the risk assessment. The products from hazard 

identification are characteristic descriptions and 

data on the exposure of interest; any important 

cofactors (e.g., other risk factors, moderating factors, mediating factors, confounders, or colliders); 

modes and mechanisms of action; and conditions 

(e.g., pre-existing diseases) under which changes 

in exposures change the probabilities or timing of 

adverse effects. Preferably, these data are suitable 

for quantifying the dose-response relationship. 

Therefore, hazard identification is the necessary 

antecedent to dose-response assessment. 

2. Dose-response assessment is an analysis of the 

dose-response association between exposure 
8 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Dose-Response

Assessment

Hazard

Identification

Risk

Characterization

Available Technology

Target Risk

Risk Assessment Risk Management

Risk

Management

Options

Mode of

Action

Figure 2-1. NIOSH risk assessment and risk management processes.

to the agent and adverse effects. The term doseresponse is broadly defined as the relationship 

between the amount of an agent administered 

to, taken up by, or absorbed by an organism, system, or population and the change developed in 

that organism, system, or population in reaction 

to the agent [IPCS 2004]. In NIOSH risk assessment, the dose-response is typically described as 

the conditional probability of the adverse effect 

(i.e., the response) in exposed workers at different levels of occupational exposure to a hazardous agent, given assumed levels for other direct 

causes of the adverse effect. The meanings of the 

terms exposure and dose have been expressed in 

many different ways over time, and these terms 

are often used interchangeably (see section 

4.3.3.1). Desired products of the dose-response 

assessment are estimates of the risk per unit 

dose having reasonable statistical properties for 

use in quantitative risk characterization. In lieu 

of sufficient data for quantification, the doseresponse may be qualitatively described. 

3. Risk characterization is the qualitative and, wherever 

possible, quantitative determination, including 

attendant uncertainties, of the probability of occurrence of known and potential adverse effects of an 

agent in workers under defined exposure conditions [IPCS 2004]. It reflects the integration of the 

sciences from the two preceding steps (i.e., hazard 

identification and dose-response assessment) with 

additional information necessary to complete the 

basis for the REL or other supported recommendation. Some of this information may be based on risk 

assessment/management policy rather than science; 

therefore, risk characterization is also a component 

of the risk management process. 

NIOSH risk assessments do not include all the steps 

outlined in the NRC paradigm. A NIOSH risk assessment does not include the exposure assessment step. Exposure assessment is the process of 

estimating or measuring the magnitude, frequency, 

and duration of exposure to an agent, along with 

the number and characteristics of the population 

exposed. Traditionally, exposure assessment provides information on sources, pathways, and routes 

of exposure necessary to be used in conjunction 

with dose-response information to project whole 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 9

population risks. For an environmental risk assessment, exposure assessment is critical to consider 

the likelihood and severity of exposure in assessing 

whole population risk. 

In contrast, NIOSH uses the information from the 

first two steps of the risk assessment to estimate 

health risks to the individual from the agent at 

various levels of exposure. This forms the risk basis 

for RELs for use in risk management. The NIOSH 

objective, as stated in Section 20(a)(3) of the Occupational Safety and Health Act of 1970, is “... to describe exposure levels that are safe for various periods of employment, including but not limited to 

the exposure levels at which no employee [emphasis added] will suffer impaired health or functional

capacities or diminished life expectancy as a result 

of his work experience.” Thus, the risks of concern 

to NIOSH cover the range of exposures as experienced by each worker, and are not considered generally as a population risk.

Nevertheless, exposure information is used for 

dose-response analyses; therefore, the quality of 

exposure data ultimately used to describe the 

dose-response relationship must be evaluated as 

a component of hazard identification. Thus, this 

document describes exposure methods and measures (see section 4.3.3), including sources of bias 

(see section 4.3.4), in Chapter 4.0. Additional information on measurement error is available in 

Appendix B. 
This page intentionally left blank.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 11

3 Risk Assessment Plan (Problem Formulation)

There can be many paths to initiating a risk assessment. NIOSH risk assessments have begun after 

concerns were raised by workers, employers, regulators, and lawmakers. Risk assessments can follow 

anecdotal evidence on emerging occupational health 

issues or systematic examination of signs and symptoms of occupational disease. Factors contributing 

to a decision to conduct a risk assessment usually 

include broad information on the availability and 

use of the agent, numbers of workers at risk, exposure pathways, the type and severity of effects, 

general mitigation options, and alternatives to risk 

assessment. Consideration of these factors usually 

evolves over time. For example, early case reports 

of bronchiolitis obliterans in popcorn plant workers [Kreiss et al. 2002] motivated several preliminary 

health assessments that ultimately led to a formal 

quantitative assessment of pulmonary disease risk 

from exposure to diacetyl and 2,3-pentanedione that 

formed the risk basis for a REL [NIOSH 2016]. 

In general, a risk assessment has two distinct initiating stages: (1) planning and scoping and (2) 

problem formulation (Figure 3-1) [NRC 2009]. 

Planning and Scoping Problem Formulation

Hazard Identification

Risk Characterization

What are the

occupational risks?

Are current safeguards 

(e.g., REL) adequate

to protect workers?

Conceptual 

model

Analysis 

plan

Dose-Response

What technical 

assessments are

needed to evaluate 

risk and management 

options? 

What conditions affect 

safeguards and the

level of analysis

required?

What resources are 

available to conduct

the assessment?

Integrate available data

Risk Assessment

Risk Managers

and Stakeholders

Risk Managers and Risk Assessors

Figure 3-1. The interrelationships between planning and scoping, problem formulation, and risk assessment.
12 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Planning and scoping typically involve a dialogue 

between stakeholders and risk managers (with support from risk assessors) on the hazards and potential risk mitigation strategies, including conceptualizing the need, purpose, structure, and content of a 

risk assessment to aid in decision-making. Problem 

formulation occurs from communication between 

risk managers and risk assessors (with support of 

stakeholders) on the technical design of the risk 

assessment, which uses the broad concepts developed in planning and scoping. Although planning 

and scoping provide input into problem formulation and therefore are initiated first, the activities in 

both stages will likely progress concurrently. 

The planning and scoping stage is described as a 

deliberative process that is intended to assist decision-makers in defining a risk-related problem 

[NRC 2009]. Planning and scoping begin with an 

acknowledgment that risk assessment is the appropriate decision-making tool. In practice, planning 

and scoping usually begin with discussions among 

risk managers and stakeholders on a certain health 

risk. These discussions may involve risk assessors 

in a supportive role. In general, the planning and 

scoping stage (1) identifies the occupational health 

concern, (2) assesses whether existing safeguards 

are appropriate or what additional information is 

needed to inform decision-making, and (3) determines the resources needed to conduct the risk assessment. The product is a statement describing the 

specific concerns the risk assessment will address 

and the resources needed to do so.

At the outset of problem formulation, NIOSH 

investigators develop a risk assessment plan that 

contains two critical components: (1) a conceptual model that identifies the hazard (sources, 

stressors, and pathways), persons at risk, and 

potential adverse effects for analysis; and (2) an 

analysis plan (work plan) that outlines the analytic components (i.e., data and methods) and interpretative approaches (e.g., risk metrics) to be 

used [NRC 2009]. The conceptual model guides 

decisions on data needs, and the analysis plan 

matches elements of the conceptual model with a 

proposed analytic approach. The two overarching 

principles in developing the plan are (1) to ensure 

that the risk assessment uses the best scientific 

methods and the highest-quality evidence and 

(2) to address the needs of the decision-makers 

(risk managers). Thus, in the problem formulation stage, it is imperative to be mindful that the 

risk assessment serves both scientific and communicative needs. This is best accomplished by 

including input from both scientists and decision-makers in the design of the risk assessment 

plan, if practical. For complex risk assessments, 

the analysis plans and conceptual models may 

benefit from peer review.

One way to formulate the risk assessment plan is to 

use a series of questions that the risk assessment is 

intended to address, such as this partial list of some 

examples:

■ What agents are involved?

■ Who is potentially at risk?

■ What are the characteristics of the potential 

adverse effects caused by the hazard? 

■ What types of data support or inform the risk 

assessment process? 

■ What dose-response data will be included? 

For example, what criteria determine the 

acceptability of experimental animal data with 

inhalation as route of exposure?

■ How will exposure be expressed (e.g., inhaled 

dose, absorbed dose, or air concentration)? 

What are the reasonable alternative expressions, 

and how would using those alternatives change 

the risk assessment? 

■ How are the health effects defined and measured? 

Are health effects aggregated (e.g., all cancer)? 

If aggregated, how would using alternative 

aggregation strategies alter the risk assessment? 

■ What evidence of a dose-response association 

is available? What types of causal mechanisms 

are likely to be involved? 

■ How should one deal with the background 

(control) incidence of different effects? Are the 

processes producing the observed health effects 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 13

in control animals (or unexposed populations) 

likely to interact with processes by which the 

hazard of interest causes these effects, or should 

the toxic mechanisms be treated as if they were 

independent of background processes?

■ How should one evaluate important sources of 

uncertainty in the risk assessment? Are there 

any reasonable anticipated adjustments to the 

exposure or health effect based on mechanism, 

metabolism, potential confounding factors, 

other exposures, or other factors? What is the 

anticipated impact of uncertainty on the risk 

assessment findings?

■ How will the final risks be expressed, and if 

quantitative analysis is done, what target risk 

levels are used? What is the support for those 

decisions, and are there reasonable alternatives? 

If yes, how would using those alternatives affect 

the risk assessment?

■ What is the timeframe for completing the 

assessment?

The above list of questions may be applicable in 

many risk assessments; however, the list is not 

comprehensive. Each risk assessment will require 

its own set of questions to explore. 

As necessary, the risk assessor refers to the plan 

throughout all aspects of the risk assessment. Appropriate plans include enough detail so that another risk assessor could reproduce the analysis. 

Risk assessment plans are living documents. If circumstances dictate changes are necessary, then risk 

assessors should revise their plans by clearly indicating the changes made, along with a justification 

for the revision. This helps to document decisions 

made throughout the course of the risk assessment. 

The development of a written risk assessment plan 

is a relatively new addition to the NIOSH risk assessment process. Up to this point, NIOSH risk 

assessments published in Criteria Documents or 

Current Intelligence Bulletins have not included a 

formal written plan.
This page intentionally left blank.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 15

4 Hazard Identification

The first step in occupational risk assessment is 

hazard identification, which is the process of characterizing the nature and strength of the evidence 

of causation, hereafter referred to as the weight of 

evidence (WoE), between an agent of interest (e.g., 

benzene) and an adverse effect (e.g., leukemia) in 

exposed workers. In this way, hazard identification 

is pursuing a causal explanation, which is a cognitive goal of identifying disease mechanisms and 

detecting causal factors [Russo and Williamson 

2007]. Ideally, this evidence will serve as input to 

the dose-response assessment to support quantitative risk assessment. Evaluating the WoE generally 

requires a systematic approach to critically assess 

and interpret the body of scientific information 

(i.e., a systematic review). This information stems 

primarily from epidemiologic studies of humans 

and animal toxicology, with mechanistic data providing a reinforcing role. Human data sources are 

preferred for quantifying occupational risks; however, hazard identification has at times relied solely 

on animal data or a combination of human and animal data. When used in combination, either data 

source may take a supportive role in the risk assessment, along with complementary mechanistic data. 

In a recent survey aimed to identify best practices 

in WoE analyses, investigators reviewed approximately 50 WoE frameworks from academia, the 

federal government, international bodies/organizations, consortia, and others (such as consulting 

firms, industry toxicology research groups, and 

military organizations). Although specific methods 

differed, the general framework of an acceptable approach to gathering and evaluating relevant human 

and animal study data consists of five basic steps: 

(1) define the causal questions of interest and develop criteria for study (data) selection; (2) develop 

literature search protocol and conduct search; (3) 

review, identify, and select relevant information; 

(4) evaluate and integrate evidence across studies; 

and (5) synthesize and interpret findings [Rhomberg et al. 2013]. The paths to meeting these steps 

can vary widely with the specific scientific context 

of the risk assessment; therefore, precise methods 

for assessing WoE cannot be prescribed without 

understanding the individual context, although 

general guidelines are provided herein and are also 

available elsewhere [Higgins and Green 2008; NRC 

1983; NRC 2014; NTP 2015a; NTP 2015b; Rhomberg et al. 2013; WHO 2000]. As such, risk assessors strive to develop and describe their approach 

in sufficient detail to ensure a transparent and defendable standard of WoE is met for their evaluation [Weed 2005]. To accomplish this, NIOSH risk 

assessors may develop systematic review protocols 

for evaluating data quality and synthesizing evidential information. As discussed in a recent NIOSHcommissioned report on systematic reviews [Hempel et al. 2016], these protocols incorporate best 

practices to accomplish the following:

■ Assess all human and animal data relevant to 

mode of action, their human relevance, and 

dose-response. 

■ Evaluate the types of data that have been considered.

■ Trace the reasoning by which the data bear on 

the evaluation of the assessment question.

■ Consider alternative modes of action and develop a biological story for each plausible mode 

of action/outcome combination.

■ Consider the relevance, response, and predictivity of the outcomes and use other knowledge 

(e.g., biological pathways) to inform the relevance determinations.

■ Integrate data across all lines of evidence so 

that the interpretation of one will inform the 

interpretation of others (e.g., if the proposed 
16 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

mode of action were true, determine what observable consequences it should have across 

lines of evidence).

■ Clearly present the WoE findings and explore 

ways to measure and communicate different 

magnitudes of WoE and different degrees of 

plausibility of explanations and their risk-assessment consequences.

In conducting the systematic review, risk assessors 

consider the internal and external validity of the 

research, as it relates to the risk assessment. Internal validity is the degree to which study findings 

are free from bias. External validity is the degree 

to which study findings may apply, be generalized, 

or be transported to the population or groups of 

interests (i.e., workers) that did not participate in 

the study. Broadly, risk assessors consider the following factors:

■ The design and conduct of studies providing 

data for risk assessment. Are study results generalizable and relevant to the risk assessment 

problem? Are results reproducible (have they 

been confirmed or refuted)? What factors may 

jeopardize external validity of the results?

■ The characterization of exposure, dose, and 

adverse effect. What is the utility of the study 

data for integration and evaluation across studies? Will these data be suitable for inclusion in 

the database for the dose-response assessment? 

What are the sources of measurement error 

and their potential effects on the dose-response 

association? 

■ The degree of data certainty and strength of 

findings in support of hazard identification. 

Have sound statistical methods been used? Are 

models correctly specified? Are results robust 

under alternative assumptions? Have results 

been misinterpreted? How likely are findings 

due to chance, bias, residual confounding, or 

other sources relevant to internal validity of the 

study? (See Appendix B.) 

In practice, to inform their assessments, NIOSH 

risk assessors have sometimes relied on hazard 

identification by other agencies, such as the U.S. 

National Toxicology Program (NTP), the U.S. 

Environmental Protection Agency (EPA), the 

Occupational Safety and Health Administration 

(OSHA), the Mine Safety and Health Administration (MSHA), the Agency for Toxic Substances and 

Disease Registry (ATSDR), the European Chemicals Agency (ECHA), and the International Agency 

for Research on Cancer (IARC). These agencies 

have a long history of hazard identification using 

sound, transparent methodologies. The NIOSH 

Chemical Carcinogen Policy [2017] provides additional information on the use of available cancer 

hazard assessments. Hazards have also been identified by recent research that has not been reviewed 

and synthesized by these agencies. This occurs most 

often in cases where emerging hazards have been 

identified or when new information on an existing 

hazard becomes available. In all cases, NIOSH risk 

assessors evaluate, integrate, and synthesize the existing evidence to characterize the hazard for doseresponse analyses and risk characterization. This is 

accomplished by using best practices of the many 

frameworks established for hazard identification. 

These practices are discussed in comprehensive reviews [Higgins and Green 2008; NRC 1983; Rhomberg et al. 2013], recent commentaries [Howard et 

al. 2017; Woodruff and Sutton 2014], and technical 

reports [EPA 2018a; NRC 2014; NTP 2015a; NTP 

2015b; WHO 2000]. In addition, tools for conducting and assessing systematic reviews are available 

to NIOSH risk assessors, such as AMSTAR and a 

recent report commissioned by NIOSH as an aid 

for conducting systematic reviews [Hempel et al. 

2016]. Historically, NIOSH risk assessors have utilized thorough literature reviews as the foundation 

for risk assessments. This new report on systematic reviews will serve as an important resource to 

guide future reviews. 

In summary, NIOSH risk assessors are mindful that 

the term hazard is defined as the inherent property 

of an agent (or situation) having the potential to 

cause an adverse effect when an organism, system, 

or population is exposed to that agent. Thus, identifying hazards requires knowledge of both the agent 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 17

and the adverse effect. Furthermore, NIOSH risk 

assessors approach hazard identification in terms 

of supporting the next step in the risk assessment. 

Therefore, data must sufficiently define dimensions 

of the population at risk, the agent, the adverse 

effect(s) of interest, and any cofactors (e.g., effect 

modifiers, confounders, or other sources of uncertainty), knowledge of which is necessary for conducting sound quantitative dose-response analyses. 

4.1 Hill’s Views on Causation

Observed associations are typically evaluated by 

NIOSH against multiple factors to assess WoE. The 

framework used to make an assessment is likely to 

be specific to the problem at hand; however, there 

are numerous WoE frameworks available to the 

risk assessor for planning an approach [Rhomberg 

et al. 2013]. Perhaps the most widely known WoE 

framework was introduced by Sir Austin Bradford 

Hill [1965], who proposed nine heuristic aspects 

of association commonly referred to as the “Bradford Hill criteria.” These aspects comprise strength 

of association, consistency, specificity, temporality, 

biological gradient, plausibility, coherence, experiment, and analogy. However, using these aspects 

to weight data is but one approach; Hill cautioned 

against the use of his views as a set of definitive 

rules on causality and acknowledged that many 

additional factors may be equally if not more important to WoE. Similar concerns have surfaced in 

several contemporary critical assessments of Hill’s 

views [Fedak et al. 2015; Hofler 2005; Howick et al. 

2009; Ioannidis 2016; Phillips and Goodman 2004; 

Thygesen et al. 2005]. Thus, the term guidelines is 

preferred to criteria, as posited by Howick et al. 

[2009]. More information on the formulation of 

Hill’s guidelines, including critical assessments of 

their use in causal inference, is available in the assessments referenced above and in seminal epidemiologic texts [Checkoway et al. 2004; Rothman et 

al. 2008]. The guidelines are briefly described below. 

1. Strength of association refers to the magnitude and statistical precision of the observed 

association, whereby a strong association 

is less likely to be influenced by unmeasured confounders, other sources of bias, or 

chance alone. Thus, this aspect addresses the 

feasibility of statistical inference. A strong 

association is neither necessary nor sufficient for a causal relationship. For example, 

the association between cardiovascular disease and smoking is considered relatively 

weak; however, it is also considered causal. 

Conversely, an effect estimate achieving statistical significance provides little evidence 

of causality without due consideration of 

other aspects, such as underlying statistical 

methods, biologic plausibility, and reproducibility of results. 

2. Consistency refers to the reproducibility 

of similar effects in different populations 

(studies). Generally, evidence from a series 

of studies reporting similar effects is weighted more than findings from a single study. 

Like strength of association, consistency 

also addresses the feasibility of statistical 

inference, because increased homogeneity 

across studies is evidence against poor internal validity. Nevertheless, consistency is 

neither necessary nor sufficient for a causal 

relationship.

3. Specificity, in Hill’s view, is the simple premise that an association is more likely to be 

causal if it is observed between one cause 

and one effect. Of course, specificity is reliant on the definitions of the cause (exposure) and effect (disease). In practice, 

epidemiologic examinations tend to involve complex exposures and multifactorial 

diseases with similar pathways; therefore, 

highly specific agent-disease associations 

are seldom observed. For this reason, many 

consider specificity to be of little importance for causal inference in most settings. 

4. Temporality refers to the general acceptance 

that the cause (exposure) must precede the 

effect (disease) in time. This is the only criterion that is considered necessary for a 

causal relationship. Thus, study designs that 
18 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

firmly maintain the temporal progression 

from cause to effect are far more persuasive 

in causal inference. 

5. Biological gradient refers to the observed 

presence of a dose-risk relationship (i.e., 

dose-response). Typically, this is defined 

as a monotonic trend in disease frequency 

with increasing levels of exposure. Studies 

designed to examine dose-response trends 

are more persuasive for causal inference. 

Nonetheless, the absence of a monotonic 

biologic gradient does not preclude the existence of a causal relationship. This aspect 

of Hill’s guidelines is the focus of NIOSH 

quantitative risk assessment, which is exploited by the dose-response modeling described in Section 5.0. 

6. Plausibility refers to a measure of biologic 

reasonableness for explaining the agentdisease association. The guideline is largely 

a function of the current understanding on 

toxicity and disease etiology. It is important 

to synthesize evidence from a wide array 

of animal and human studies to assess the 

plausibility of an association between contributing causes and complex diseases. Toxicological data from experimental animal 

studies can be particularly useful for assessing biological plausibility. For example, if an 

agent causes toxicity in animals similar to 

that observed in humans, then this is strong 

evidence of biological plausibility. 

7. Coherence is related to plausibility; it implies 

that the interpretation of a causal association agrees with known disease etiology. Of 

course, coherence relies on current knowledge, which is always subject to change. Hill 

stated that the absence of coherent information should not be considered as evidence 

against causation. In contrast, the presence 

of conflicting information is counter to causality. The risk assessor must judge whether 

the conflict is true (thus potentially negating 

a cause-and-effect relationship) or false, because of study errors or misinterpretation. 

8. Experiment refers to evidence of a successful intervention; that is, removing (or reducing) the cause results in the disappearance 

(or attenuation) of the effect. This is sometimes referred to as evidence of manipulative causation. For example, lower lung 

cancer rates have followed patterns of decreased smoking. This observation supports 

the hypothesis that lung cancer is caused 

by smoking. Hill considered this criterion 

as “... the strongest support for the causal 

hypothesis.” However, evidence from interventions is rarely available to risk assessors. 

9. Analogy is related to plausibility; if a causal 

association is apparent with an agent, then 

the standard of evidence is lessened for similar agents by analogy. For example, human 

data on the toxicity of diacetyl are believed 

informative on risks from exposures to the 

chemically similar agent 2,3-pentanedione, 

for which human data are unavailable.

Although Hill’s guidelines are still in wide use 

[Wakeford 2015], there have been many advances 

in science since their introduction. In response, several authors have made efforts to modernize these 

guidelines [Becker et al. 2015; Cox 2018; Fedak et 

al. 2015; Hofler 2005; Howick et al. 2009; Weed 

2018], while others have found their continued use 

in causal inference limited [Ioannidis 2016]. Cox 

[2018] recently updated and strengthened Hill’s 

views, including suggested statistical tools, in the 

context of modern causal discovery principles and 

methods. These methods are aimed toward drawing valid causal conclusions that better inform decision makers on risk management approaches. In 

general, modern causal discovery recognizes the 

interplay in the network of causal mechanisms that 

explain the complex dose-risk relationship typical 

in most diseases. In particular, manipulative causal 

analysis is used to infer disease probabilities under 

changing exposure conditions (i.e., intervention) 

while holding other factors fixed (e.g., how does a 

change in exposure level change the probability of 

the adverse effect?) [Pearl 2010]. Clearly, manipulative causation informs decision analysis supporting 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 19

risk management, which usually assumes that decision variables have values that can be manipulated 

by decision-makers to help choose an effective risk 

mitigation course [Cox 2018]. In contrast, associative causation, which assesses causality on the 

basis of measures of association (e.g., relative risk 

and odds ratios) and WoE criteria, is directed more 

towards identifying potential causes (i.e., affording a causal explanation, as posited by Russo and 

Williamson [2007]) rather than assessing the effectiveness of interventions. Of the two approaches, 

associative causation is found most often in observational data, and causation is often interpreted 

based on Hill’s guidelines or similar qualitative 

WoE criteria. 

Regardless of the actual framework used, risk assessors are encouraged to apply Hill’s guidelines, or 

a derivative of these guidelines, to the data integration step of hazard identification, using them not 

as a checklist or simple measure of WoE but to uncover patterns in the data that support scientific inference on causality [Hempel et al. 2016; Rhomberg 

et al. 2013]. Finally, it should be clear that neither 

approach is sufficient to empirically validate causal 

discovery, given limitations in available data [Cox 

2018; Oates et al. 2016]; therefore, some judgement 

(and accompanying uncertainty) on hazard identification is necessary.

4.2 Adverse Effects

Adverse effects in workers—sometimes referred 

to as the adverse health effect(s), outcome of interest, or simply the response—must be clearly 

defined in the population at risk and comparative 

populations, such as control groups or the general population. NIOSH risk assessors affirm that 

case definitions and ascertainment methods used 

in candidate epidemiologic studies are adequate 

for risk assessment. When considering cancer and 

non-cancer adverse effects or biomarkers for those 

effects, it is important to understand the progression of disease and select a measurable adverse 

effect as early in the process or with the least severity of effect as possible. Ideally, an empirically 

observable endpoint that is clearly a key event or 

precursor to the adverse effect of interest should 

be targeted for risk assessment, but the strength 

of association between the exposure and outcome 

and the potential for confounding are important 

to consider. Furthermore, an agent may involve 

multiple adverse effects. Frequently, risk assessors 

have limited evaluations to the most sensitive effect 

by examining multiple effects separately and then 

choosing the effect offering the greatest risk per 

unit exposure. However, recent efforts have shifted 

toward a more holistic approach of estimating aggregate risks from the combined effects of exposures to one or more agents. 

4.2.1 Cancer

Cancer is a term used to describe over 100 different diseases in which abnormal cells divide without control and can invade nearby tissues [Ruddon 

2007; Schottenfeld and Fraumeni Jr 2006]. Given 

this broad definition, there are many possible characterizations of cancer as an adverse effect used in 

epidemiologic studies, ranging from all cancers 

combined to a precise classification of a primary 

malignant tumor. In dose-response analyses, studies showing specific adverse effects are superior to 

those showing the effects of all cancers combined, 

given varying etiologies among types of cancer; 

however, specificity of the adverse effect may come 

at a cost of statistical imprecision, given the rarity 

of most individual cancers. Moreover, most occupational studies have examined mortality data 

from death certificates, which often lack desired 

cancer specificity. Therefore, human studies have 

infrequently examined specific malignancies (e.g., 

lung adenocarcinoma). Instead, cause-specific cancer endpoints are typically constructed by grouping multiple tumors that share common traits (e.g., 

lung cancer, respiratory cancers, and solid tumors). 

It is important to consider the potential effects of a 

heterogeneous grouping on the dose-response. For 

example, ionizing radiation is a known leukemogen; however, strong evidence of chronic lymphocytic leukemia (CLL) radiogenicity is lacking [Linet 
20 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

et al. 2007; Silver et al. 2007]. This has led some expert committees to conclude that CLL is nonradiogenic [NRC 1990; UNSCEAR 2000]. If CLL is not 

associated with ionizing radiation, then combining 

it with radiogenic leukemias will act to attenuate 

the dose-response between the grouped outcome 

and exposure; therefore, most studies have excluded CLL from the leukemia grouping.†

 Group definitions can vary between studies and even within 

a study because of changes in diagnostic criteria 

over time. When combining data for hazard identification and subsequent dose-response analyses, 

the risk assessor considers the compatibility of the 

adverse effect definition between studies.

4.2.1.1 Carcinogenesis 

The mechanisms of carcinogenesis are rapidly becoming an important aspect of hazard identification. Human carcinogenesis is a multistage process 

that can involve numerous mechanisms causing 

various biological changes leading to tumorigenesis. These mechanisms can vary widely by agent; 

therefore, risk assessors systematically assess 

available mechanistic data to appropriately characterize the dose-risk relationship and evaluate the 

overall carcinogenic hazard of an agent. IARC has 

identified the following characteristics of human 

carcinogens that are useful in a systematic strategy 

of assessing mechanistic data for hazard identification [IARC 2019; Smith et al. 2016]: 

■ The agent acts as an electrophile either directly 

or after metabolic activation. Electrophiles are 

electron-seeking molecules that commonly 

form addition products (adducts) with cellular 

macromolecules including DNA, RNA, lipids, 

and proteins.

■ The agent is genotoxic, i.e., it induces DNA 

damage that may or may not result in mutation. A genotoxic agent that induces mutations 

is termed mutagenic. 

†

The research on CLL radiogenicity is inconsistent; however, in 2011 

after a review of the literature and consultation with subject matter experts, NIOSH recommended that CLL be considered radiogenic for purposes of compensating workers covered under the 

Energy Employee Occupational Illness Compensation Program 

Act (EEOICPA). 

■ The agent alters DNA repair or causes genomic instability. 

■ The agent induces epigenetic alterations. The 

term epigenetic refers to stable changes in 

gene expression and chromatin organization 

that are independent of the DNA sequence that 

can be mitotically inherited over cell divisions.

■ The agent induces oxidative stress, i.e., an imbalance between formation of reactive oxygen 

and/or nitrogen species and their detoxification. 

■ The agent induces chronic inflammation. 

■ The agent is immunosuppressive. Immunosuppression is a reduction in the capacity of the 

immune system to respond effectively to foreign antigens.

■ The agent modulates receptor-mediated effects. 

■ The agent causes immortalization. The agent 

disrupts normal cellular replicative senescence 

to cause unlimited proliferation. 

■ The agent alters cell proliferation, cell death, 

or nutrient supply.

Most carcinogens demonstrate more than one of 

these traits. Assessing mechanistic data requires 

three basic steps: (1) identify relevant information, 

(2) screen and organize mechanistic data, and (3) 

synthesize mechanistic information (e.g., develop 

adverse-outcome pathways). In this way, the IARC 

approach provides a foundation for carcinogen 

classification (i.e., hazard identification); however, 

mechanistic data can also inform choices made in 

risk characterization, such as estimating the response expected at low doses.

4.2.2 Non-cancer

For non-cancer risk assessment, it is important to 

evaluate issues of severity, reversibility, progression 

to more serious conditions, and other pertinent issues. NIOSH has typically conducted quantitative 

risk assessment on non-cancer adverse effects by 

assuming chronic exposure (see Table 1-1). There 

may be instances where the effects after short-term 

or intermediate-length exposure are determined to 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 21

be critically important. In those cases, the risk assessor must evaluate and document the impact of 

the exposure in the context of a shorter-term exposure duration and any longer-term sequelae.

With respect to assessing risk persistence, a key 

question is whether the observed pathophysiologic 

change defined as the endpoint of interest is reversible with cessation of exposure. Unfortunately, 

data are often insufficient to answer this question; 

therefore, irreversibility is typically assumed as a 

worst-case scenario until evidence to the contrary 

becomes available. Two examples of NIOSH risk 

assessments illustrate this issue. First, NIOSH examined Parkinson’s disease–like symptoms and 

manganism resulting from manganese exposure 

in welders [Park et al. 2009]. The NIOSH risk assessment quantified the relationship between 

manganese exposures in confined-space welding 

and cognitive deficits in working memory or verbal IQ. However, the information was insufficient 

to conclude whether the risk of exposure-related 

neurobehavioral deficits persisted after cessation of 

exposure. Without the data, NIOSH conservatively 

assumed that excess risk accrued with exposure 

and persisted afterward. Similarly, a risk assessment of diacetyl exposure and the development of 

bronchiolitis obliterans used multiple definitions of 

pulmonary dysfunction as a case-surrogate for the 

onset of the condition [NIOSH 2016]. Again, risk 

accumulation and persistence were assumed in lieu 

of contrary information. In both risk assessments, 

the assumption of irreversible adverse effects had 

large impact on estimates of lifetime risk per unit 

exposure.

4.3 Human Data 

NIOSH risk assessors prefer the direct estimation 

of risk from human data. In practice, however, risk 

assessments usually must rely on a combination of 

human and animal data for hazard identification 

and dose-response analyses. This is because both 

data sources are imperfect; human data tend to be 

vulnerable to potential biases from confounding factors, and there is large uncertainty in extrapolating 

risk in animals to humans. It is common for human data to be weighted more than animal data 

for hazard identification, but to be less informative on dose-response. In those instances, human 

studies provide evidence of an association between 

exposure and disease, which can guide the choice 

of agents, exposure routes, and pathological endpoints for examination in animal toxicology studies that might contribute greatest to quantifying 

risks.

4.3.1 Epidemiologic Study Design

Figure 4-1 shows a hierarchy of epidemiologic 

study designs, ordered by the potential contribution to WoE. Human data for WoE assessment may 

originate from experimental or observational studies. Regarding the former, study participants are 

intentionally exposed to an agent under controlled 

experimental conditions. In this context, the term 

controlled refers to design parameters intended to 

minimize the effects of factors other than the exposure condition on the measured response [NASEM 

2017]. These studies are sometimes referred to as 

clinical studies, human challenge studies, or controlled human exposure studies. Adherence to a 

strict experimental design is a trait of controlled 

human exposure studies that lessens the potential 

for major biases; therefore, these data tend to be well 

suited to hazard identification and dose-response 

analyses. As explained previously (see Hill’s guidelines), experimental designs with exposure intervention can provide strong support for causal 

inference. However, human experimental data on 

exposures to hazardous agents are sparse for obvious ethical reasons; therefore, observational studies 

tend to be the most important information source 

for directly addressing the dose-risk relationship in 

humans and are the focus of the discussion on epidemiologic study design. 

Observational studies can be further classified as either analytic (e.g., longitudinal and cross-sectional 

studies) or descriptive (e.g., case reports, case series, and ecologic studies), the latter being the least 

informative for risk assessment. Detailed descriptions 
22 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

 

Figure 4-1. Hierarchy of human epidemiologic studies.

of epidemiologic study designs are available in seminal texts [Breslow and Day 1980; Breslow and Day 

1987; Checkoway et al. 2004; Rothman et al. 2008]. 

Of observational research types, longitudinal analytic studies (e.g., cohort or panel studies) are most 

useful with respect to WoE. These studies follow 

the exposure and health status of each individual 

in a study sample or population over time. An important strength of this design is its ability to measure temporal changes in exposure and outcome 

at the individual level. Thus, this study design allows for direct examination of the dose-response. 

Cohort studies are the most common source of 

human data in NIOSH risk assessment. A cohort 

comprises a group of individuals who share some 

defining characteristics, who are followed in time. 

Data can be collected prospectively; however, most 

occupational cohort studies are historical, using 

data that span a time prior to initiation of the study. 

A disadvantage of a cohort study is that it requires 

the recruitment of many participants who must 

be observed over a long period for examining rare 

outcomes (e.g., cancers); therefore, a detailed accounting of individual exposures for everyone in 

the study group may be impractical. 

Measures of association can vary in cohort studies. If comparisons are made between the study 

population and an external referent (e.g., the U.S. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 23

population), common measures of association 

are the standardized mortality ratio (SMR) or 

standardized incidence (morbidity) ratio (SIR) 

[Rothman et al. 2008]. These measures are simply the ratio of the observed number of cases to 

the number of expected cases, where the number 

of expected cases is calculated based on disease 

rates observed in the referent population that are 

standardized by characteristics (e.g., age, race, 

gender, and calendar period) in the study population. Trends in SMRs by categories of exposures can offer crude dose-response information; 

however, because of indirect standardization 

methods, comparisons of SMR are vulnerable to 

bias due to differences (e.g., differences in age, 

gender, and race) in comparison groups. Internal comparisons (comparisons made within the 

study population) offer better dose-response 

data than SMR and SIR analyses. Measures of 

association from internal comparisons include 

trends across standardized rates by exposure categories or risk measures from dose-response regression models. Risk measures can be expressed 

on a relative scale, such as hazard ratios (HRs), 

rate (or risk) ratio (RR), or excess relative risk 

(ERR), or in terms of risk differences, such as attributable risk or excess absolute risk (EAR). 

A case-control (or case-referent) study compares 

exposure among persons with the outcome of interest (i.e., cases) to exposures among persons 

preferably drawn from the same population (controls). Thus, the reduction in study size saves time 

and expense relative to a cohort study. This design 

is particularly useful when examining rare adverse effects. Cases can be enumerated at a point 

in time (prevalent cases) or over a period (incident 

cases). An important consideration is the number 

of matched controls per case. In the absence of a 

dose-response relationship, reasonable asymptotic 

relative efficiency is achieved with few controls 

[Breslow et al. 1983; Goldstein and Langholz 1992; 

Ury 1975]. However, the actual relative efficiency 

decreases as the strength of the exposure–response 

increases and as the skewness of the exposure distribution increases [Bertke et al. 2013]. The standard 

effect measure of the case-control study is the odds 

ratio (OR), which approximates the risk ratio (relative risk) if the disease is rare. 

A special instance of a case-control study is one 

that is nested within a specified cohort. This design retains many of the analytic advantages of the 

large cohort while reducing the number of subjects 

needing exposure estimates. Thus, a nested casecontrol study allows for improvements in exposure 

data that can lead to better information on doseresponse. As in cohort studies, a nested design 

also allows for precise treatment of the timescale; 

therefore, measures of association related to events 

per unit person-time can be estimated by means of 

dose-response regression modeling (e.g., HR, RR, 

and ERR).

A cross-sectional study (e.g., survey) examines the 

frequency or level of an attribute (e.g., exposure 

and/or adverse effect) in a defined population at a 

particular point in time. This design is often used 

to examine the prevalence of nonfatal diseases or 

symptoms that typically do not rapidly lead to employment termination (e.g., mild decreases in lung 

function, changes in blood pressure, pre-clinical 

biomarkers of early effect, and skin irritation). This 

design is a poor choice for examining diseases that 

are rare or periodic. Because cross-sectional studies are based on prevalent cases, this design has 

limited value for examining etiologic relationships. 

Other disadvantages are the lack of information on 

temporal sequence between cause and effect and 

the potential for selection bias from health-related 

employment termination that took place prior to 

recruitment into the study.

Epidemiologic studies that are conducted with observation at the group level (e.g., plant, city, county, 

or nation) instead of the individual level are called 

ecologic or aggregate studies [Rothman et al. 2008]. 

These studies can involve a single cross-sectional 

survey or repeated measures (i.e., time-trend design). Ecologic studies may be a practical alternative to individual-level studies when exposures and 

disease are relatively homogeneous within a population but differ between populations, or when 
24 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

individual exposure estimates are not possible. 

Within-group heterogeneity is a likely condition; 

therefore, extrapolation to the individual level is 

not feasible. Thus, an association at the group level 

does not imply the same association at the individual level. This limitation is known as the ecologic fallacy. Another major limitation is that these 

studies lack the ability for adequate control of confounding. For these reasons, ecologic studies have 

limited value in assessing causal associations. Nevertheless, ecologic studies may provide descriptive 

information on differences in populations that may 

be suggestive of a potential cause-and-effect relationship. This information can be used to support 

findings from analytic studies.

Other descriptive study reports, such as case reports and case series, provide information on 

symptomology, disease history, diagnostic features, 

and outcomes for one or more subjects under observation. The term case report refers to a description of a person with a disease, whereas case series

refers to a series of related case reports that were 

typically collected at a specific practice, clinic, or 

hospital over a defined time. Like cross-sectional 

studies, routine data studies are of limited value 

for examining etiologic relationships but may initiate larger investigations that are better designed 

to inform on causation. In addition, these studies 

are purposed for identifying emerging trends in 

adverse health, high-risk populations, and unrecognized hazards; therefore, they can be an important data source for hazard identification in risk 

assessment. Case reports can be very informative 

on rare diseases when exposures are well defined. 

For example, 4,4 ́-methylenebis(2-chloroaniline) 

(MBOCA) is considered carcinogenic in humans 

(IARC Group 1) because of sufficient evidence in 

experimental animals and strong mechanistic evidence; however, evidence in humans was deemed 

inadequate [IARC 2012]. Liu et al. [2005] reported 

on a 52-year-old non-smoking male patient with 

bladder cancer who was significantly exposed to 

MBOCA while employed as a chemical worker 

for 14 years. Reconstruction of his past exposures 

yielded no other exposures to bladder carcinogens. 

Thus, this case study provided direct evidence of 

MBOCA potentially acting as a human bladder 

carcinogen. 

Data may also be available as summaries, such as 

integrative reviews [Cooper 1982; Jackson 1980], 

meta-analyses [Thacker 1988], and pooled studies 

[Blettner et al. 1999; Checkoway 1991; Friedenreich 1993]. For these study designs, an overarching goal is to reconcile inconsistent results among 

multiple studies to infer generalizations. Research 

integration can be accomplished qualitatively or 

quantitatively, depending on the approach. Given 

a set of relevant published observational studies, an 

integrative review gathers information to provide a 

qualitative assessment of the evidence in narrative 

form. These reviews are timely and inexpensive; 

however, findings are vulnerable to publication bias 

and the subjective judgement of the reviewer. For 

example, risk assessors should be wary of judgements based mostly on a tally of observed positive 

or negative associations [Greenland 1987]. 

A meta-analysis is an extension of the integrative 

review that analytically combines the published information to provide a quantitative synthesis across 

the literature, usually in the form of a summary effect measure. Although they are quantitative, summary effect measures rely on consistency in the 

data, design, and conduct of the studies selected 

for meta-analysis; therefore, methods used to select 

studies and address heterogeneity are an important 

consideration for the risk assessor. Given that these 

studies also rely on published data, there is a potential for publication bias. 

In contrast to meta-analyses, pooled studies are 

the reanalysis of individual data from multiple 

studies [Friedenreich 1993]. Pooling allows for a 

consistent definition of study variables, improved 

examination of confounding and effect modification, and superior statistical modeling of the doseresponse. These studies can include unpublished 

data; therefore, the potential for publication bias 

can be reduced. A potential drawback of pooling 

is that it may disregard characteristics (e.g., differences in unmeasured risk factors) of the individual 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 25

study populations being combined, which may bias 

summary risk estimates or lead to spurious results 

[Bravata and Olkin 2001]. As in meta-analysis, superior pooled studies are designed to select data 

(or studies) using criteria that reduce the potential 

for bias and include methods to assess and account 

for residual heterogeneity. Pooled studies are less 

common because they tend to be more expensive 

and time-consuming and can require considerable 

coordination between study centers. 

When assessing human studies for WoE, NIOSH 

risk assessors often use available checklists of study 

design and analysis criteria that have been widely 

vetted. For example, clinical research has greatly 

improved under the Consolidated Standards of 

Reporting Trials (CONSORT) statement, which 

includes a 22-item checklist and flow diagram 

[Moher et al. 2001]. Similar checklists are available 

for meta-analyses of clinical trials and observational studies [Moher et al. 1999; Stroup et al. 2000]. 

The checklists developed under the Strengthening 

the Reporting of Observational Studies in Epidemiology (STROBE) initiative are useful tools for 

assessing the strengths and weaknesses of standalone observational studies [von Elm et al. 2007]. 

STROBE provides separate checklists for cohort, 

case-control, and cross-sectional studies. Also, for 

longitudinal studies, risk assessors are encouraged 

to use the checklist offered by Tooth et al. [2005]. 

This checklist consists of 33 questions on study design and analysis criteria. This and similar checklists may augment the systemic approach to assessing WoE; however, they should be viewed only as 

potentially useful tools among many. Given that 

each risk assessment is unique, reliance solely on 

published checklists should be avoided. 

In summary, most human data used in NIOSH occupational risk assessments are drawn from observational studies and (preferably) from longitudinal 

studies of working populations. These data include 

information on disease status from registries, death 

certificates, medical records, diagnostic exams, or 

self-reports (i.e., questionnaires). Exposure data 

sources include personal or ambient measurements, 

modeling, constructed proxies (e.g., job-exposure 

matrices), self-reports, or any combination of these. 

Risk assessors must fully understand the nature and 

limitations of the data in studies selected for risk assessment.

4.3.2 Health Effect Data

Mortality is a common endpoint in epidemiologic 

studies. Cause of death information stems primarily from death certificates, which can provide information on multiple causes of death. Typically, 

the underlying cause of death (UCOD) is preferred, given longstanding and well-accepted use 

in public health, although some studies examined 

multiple causes to increase the number of available 

cases, especially for rare outcomes. The UCOD is 

defined as the disease or injury that initiated the 

train of events leading directly to death, or the circumstances of the accident or violence that produced the fatal injury [WHO 1977]. Although the 

unidimensional UCOD is conceptually easy to 

understand, it does not consider other important 

contributors to death that may also be listed on the 

death certificate. This is especially true for complex 

chronic illnesses, which are typically characterized 

by multiple contributing causes. To make better use 

of available information, multiple causes of death 

(MCOD) data have become an appealing alternative in some studies [Chamblee and Evans 1982; 

Israel et al. 1986; Redelings et al. 2007].

For study purposes, death causes are usually translated to codes from the International Classification 

of Diseases (ICD). Coding death certificates is a 

highly specialized and interpretive process that is 

nearly always conducted only by a qualified nosologist. Agreement between nosologists tends to be 

high, but some disagreement and errors in coding 

are unavoidable, as are inaccuracies in the death 

certificates themselves. For example, the UCOD recorded on death certificates have differed upwards 

of 20% to 40% when compared with autopsy conclusions [Cameron and McGoogan 1981; Engel et 

al. 1980; Maudsley and Williams 1996; Sehdev and 

Hutchins 2001]. Coding sequence errors in translating information from the death certificate are 
26 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

also likely. In both cases, the effects of these errors 

may be offset in analyses using MCOD data.

Morbidity data are typically abstracted from either 

disease registries or medical records. Obtaining 

morbidity information is generally more difficult 

than obtaining mortality data, given there are few 

reportable diseases (e.g., cancer) and most U.S. 

disease registries have originated more recently 

than mortality databases. Cancer registries are 

perhaps the most informative, given that nearly all 

states have had registries for acquiring data since 

the early 1990s. Cancer incidence data are generally considered superior to mortality data, because 

of improved diagnostic information in registries 

and greater discovery of highly survivable cancers. 

In addition, incidence data are less susceptible to 

survival effects (e.g., competing risks and healthy 

worker survivor effects) in dose-response analyses, 

although incidence data are more susceptible to 

screening bias. 

The United States lacks a national cancer incidence 

database; studies of U.S. workers require matching 

to multiple state cancer registries. The number of 

cases depends on whether workers leave the covered area during the observation period. Therefore, 

interstate migration can be an important source 

of underascertainment. For example, retirement 

patterns suggest that interstate migration rates for 

retirees are upwards of 5% [Cowper et al. 2000]. 

Localized migration rates can be much higher. In a 

study of cancer among U.S. firefighters in Chicago, 

Philadelphia, and San Francisco, registry data were 

obtained for over 16% of cases via linkage to eight 

other states, with nearly 7% from Florida alone 

[Daniels et al. 2014]. Acquiring data from multiple 

state cancer registries can be onerous, costly, and 

time-consuming. Furthermore, U.S. disease registries may have limitations due to several factors, 

such as poor coverage, underreporting of some diseases (e.g., melanoma), no reporting of others (e.g., 

basal cell carcinoma), duplicate reporting among 

multiple databases (e.g., neighboring state cancer 

registries), varying data acquisition procedures 

across databases, and a relatively short time span 

since registry inception. Although morbidity data 

can be generally superior in analyses, longitudinal 

studies of chronic work-related illnesses, including 

cancer, have primarily relied on mortality data. 

Most researchers have preferred registries to medical charts, when practical (e.g., for a cancer study), 

given that registries are generally less resource intensive and data are less affected by losses due to 

death or follow-up. However, ascertainment from 

registries can be quite poor for some cancers that 

are underreported, such as melanoma treated in 

private clinics [Cockburn et al. 2008] or not reported (e.g., basal and squamous cell carcinomas, 

excluding genital sites). A viable (and potentially 

superior) option to registry linkage is to combine 

self-reported data with medical verification. These 

methods, sometimes referred to as “medical followback” [Schubauer-Berigan et al. 2015] and “active 

follow-up” [Pinsky et al. 2016], generally involve 

four steps: (1) administer questionnaires to identify prospective cases, (2) determine whereabouts 

of relevant medical records for case confirmation, 

(3) obtain consent and obtain access to the records, 

and (4) verify diagnosis and other characteristics. 

These methods also provide for gathering information on important covariates that is not available in 

disease registries. An example of this design is the 

recent NIOSH study of breast cancer incidence in a 

cohort of U.S. flight attendants [Schubauer-Berigan 

et al. 2015]. For this study, researchers verified the 

cases that were first identified by self (or proxy) by 

contacting the physician, hospital, or other health 

care organization in which the cancer diagnosis 

was made and obtaining supporting documentation of the diagnosis.

Data on adverse effects may be self-reported alone 

or stem from expert diagnosis. For example, study 

data may originate from members (or proxies) of 

the population at risk who have reported specific 

symptoms or diagnosed conditions that may be regarded as caused by the exposure of interest. These 

data are vulnerable to errors from inaccurate recall [Atkinson et al. 2016; Howell et al. 2015; Wallace and Kohatsu 2008], which may be attenuated 

somewhat by using medical information to confirm 

a reported diagnosis (e.g., Schubauer-Berigan et al. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 27

[2015]). Expert assessment of signs or diagnoses 

requires uniform application of an adverse effect 

definition by knowledgeable evaluators. Evaluators 

should be blinded to the study subjects’ exposure 

status when assessing health effects to reduce the 

potential for evaluator bias. 

A recent example of using expert assessment in 

a NIOSH risk assessment is found in the criteria 

document supporting the REL for diacetyl and 

2,3-pentanedione [NIOSH 2016]. In that study, 

researchers quantitatively assessed the effects of 

diacetyl and 2,3-pentanedione exposures on pulmonary function, using spirometry data and defined case definitions based on expert assessments 

of forced expiratory volume (FEV) and forced vital 

capacity (FVC). These case definitions were used in 

models quantifying the dose-response relationship 

between diacetyl exposures and changes in pulmonary function. The models also included data gathered by using a medical questionnaire to collect 

self-reported information on respiratory health, 

dermal symptoms, allergies, smoking habits, coexposures, and protective equipment used.

4.3.3 Exposure Methods 

and Measures 

The National Research Council has defined exposure science as the collection and analysis of 

quantitative and qualitative information needed 

to understand the contact between the receptors 

(e.g., workers) and the physical, chemical, or biologic stressor [NRC 2012]. Exposure science plays a 

critical role in (1) systematically assessing the availability, magnitude, and validity of exposure data 

as a part of hazard identification and (2) providing input to dose-response analyses (e.g., explanatory variables in dose-response regression models). 

As such, the evaluation of exposure methods and 

measures supporting NIOSH risk assessment focuses on the methods used in informative studies 

to estimate or measure exposure, and a synthesis 

of exposure information for use in dose-response 

analyses. In particular, the risk assessor must 

evaluate the likelihood of potential bias in the 

dose-response resulting from exposure misclassification or misspecification (see Appendix B).

Ideally, the dose-response relationship between an 

adverse effect and an agent is quantified by using 

complete exposure histories on each subject in the 

affected population. Of course, ideal conditions are 

rarely present in observational studies of working 

populations. Therefore, exposure assessors face 

many challenges such as these: 

■ The reliance on data from previous studies or 

employer information that is suboptimal for 

risk assessment purposes.

■ A lack of sensitive, specific, precise, accurate 

measurements of worker exposures. Exposure values are often derived indirectly from 

employment information (job titles and employment) and other proxy sources (e.g., other 

research, industrial hygiene data, process records, and institutional knowledge). 

■ Incomplete information on exposure or other 

risk factors that could influence effect measures. For example, exposures that occur while 

a worker was employed elsewhere (i.e., outside 

of studied facilities) are rarely known.

■ Temporal and spatial variation in occupational 

characteristics (e.g., tasks, chemical inventories, and engineering controls). These can result in wide-ranging interindividual and intraindividual variation in exposure, differences 

which can add to the uncertainty in exposure 

indices. 

■ Industry settings that involve complex exposures to combinations of hazardous agents 

rather than a single agent of interest. Health 

effects from exposures to an agent may be entangled with effects from other agents. Furthermore, the combined effects of a mixture of 

agents (i.e., cumulative risk) may differ from 

the additive effect of separate exposures to 

these agents.

Because of the evolution of workplace hazard controls, present day exposures to hazardous agents 

tend to be lower than in earlier times, resulting in 
28 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

less evident exposure-related adverse effects. Thus, 

there is increased need for the most informative 

exposure estimates for quantifying a correspondingly smaller attributable risk. As such, the field of 

exposure science is rapidly progressing to meet the 

demand for improved methods for estimating exposures. Many of these methods are summarized 

in several works on occupational epidemiology and 

risk assessment [Checkoway et al. 2004; EPA 1992; 

Nieuwenhuijsen 2010; NRC 1983; White et al. 2008]. 

The quality of exposure information in observational studies is often limited by data availability. When 

considering data for the purpose of occupational 

risk assessment, the risk assessor generally weights 

available information by the hierarchy shown in 

Table 4-1. This order supports a general preference 

of individual exposure estimates over group estimates and quantitative values over exposure classes. 

Thus, exposure information ranges from individual 

exposure estimates derived from personal monitoring (most precise), to exposure status that is dichotomously assigned (least precise). 

4.3.3.1 Exposure Indices

Exposure is typically quantified directly or indirectly in terms of either exposure or dose [NRC 2012; 

White et al. 2008]. The metrics derived are sometimes referred to as exposure indices [Checkoway 

and Rice 1992; Loomis et al. 1999; Nieuwenhuijsen 2010]. The terms dose and exposure have been 

used interchangeably in risk assessment; however, 

a distinction between these terms is generally recognized [IPCS 2004; Kriebel et al. 2007; NRC 2012; 

Paustenbach 2000]. Exposure refers to contact between an agent and a target. Contact takes place 

at an exposure surface over an exposure period. 

Dose is the total amount of an agent administered 

to, taken up by, or absorbed by an organism, system, or population. Thus, dose is also the amount 

of agent that enters a target after crossing an exposure surface [IPCS 2004]. Strict adherence to 

the distinction between dose and exposure relies 

on the choice of target and exposure surface, and 

dose estimation may require a complete accounting of various physiologic and metabolic systems 

that modify the amount deposited into the chosen 

human target. In practice, the choice of exposure or 

dose metrics depends on the aims of the candidate 

study, which may or may not align with the needs 

of the risk assessment. Therefore, NIOSH risk assessors consider how the choice affects the WoE 

provided by the study during hazard identification 

and, if data are selected for dose-response analyses, 

what additional steps (if any) are needed to convert 

the quantity used in the dose-response analysis to 

the quantity needed for a suitable REL. 

Exposure indices can be expressed in many ways 

by using information on three basic dimensions: 

intensity (e.g., concentration, mass), duration (e.g., 

hours, days), and frequency (e.g., times per day) 

Table 4-1. Types of exposure data in occupational epidemiologic studies 

(adapted from Checkoway et al. [2004])

Type of Data Dose Approximation

Quantified personal measurements on all workers Most Precise

Quantified area- or job-specific measurement data

Ordinal ranking jobs and tasks (e.g., JEMs, SEGs, using exposure categories)

Duration of employment in the industry

Ever/never employed in the industry Least Precise

Abbreviations: JEM, job-exposure matrix; SEG, similar exposure group.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 29

[Nieuwenhuijsen 2010; White et al. 2008]. Indices 

may include each dimension separately or in combination, such as assessing ionizing radiation exposure as a time-integrated dose (e.g., lifetime dose 

equivalent measured in sievert [Sv]) or a time-averaged dose (dose equivalent rate measured in Sv 

per hour) [Checkoway and Rice 1992; Kriebel et al. 

2007]. Ideally, the choice of metric is determined 

from adequate information on which metrics (if 

any) best predict risk. In lieu of this information, 

the characteristics of the adverse effect might imply 

the most appropriate choice. For example, exposure 

indices used to examine acute toxicity effects are 

typically based on short-term or instantaneous intensity (e.g., peak airborne concentration), whereas 

cumulative dose (i.e., the time integral of exposure 

intensity) is generally preferred for chronic effects 

in which biologic damage appears proportional to 

the delivered dose quantity (e.g., silica and chronic 

silicosis or ionizing radiation and cancer) [Checkoway and Rice 1992; Rappaport 1991]. As another 

example, an adverse effect may be reversible by 

elimination of toxic agents from the body over 

time. In this instance, a measure of the amount of 

the hazardous agent residing in the body (i.e., body 

burden) might be preferred. 

Body burdens are a metric of internal exposure 

and are typically determined via biomonitoring 

methods that measure the hazardous agent, its 

metabolites, or other reaction products in a biologic matrix (e.g., human tissues, saliva, blood, or 

excreta) [Needham et al. 2007]. Pharmacokinetic 

models can also be used to estimate body burdens. 

Thus, an understanding of the underlying biological mechanism related to the outcome of interest is 

important for index selection. If an understanding 

of the expected response is lacking, then it may be 

necessary to assess the dose-response by examining multiple indices [Blair and Stewart 1992]. In 

this case, the choice of the best index is based on its 

validity and reliability, as well as its utility in subsequent dose-response analyses.

Summary (aggregate) exposure metrics (e.g., average, geometric mean, or peak exposures) are often used to assign group-level exposure indices in 

the absence of individual data. Clearly, the choice 

of summary metrics can have a marked effect on 

results from dose-response analyses. Often, the 

choice is limited to published results that may not 

be best suited for risk characterization. For example, exposure distributions of most occupational 

agents tend to be right-skewed, and geometric 

mean values are used in many studies as a measure 

of central tendency. Although these measures may 

be appropriate for the intended purpose, the choice 

was likely made without consideration of a subsequent use in describing population risk. It has been 

shown that the appropriate group assignment for 

risk characterization is largely dependent on the 

expected shape of the dose-response, regardless 

of the underlying exposure distribution [Crump 

1998; Seixas et al. 1988]. In most situations, the 

anticipated response increases with dose; therefore, the arithmetic mean (i.e., average) provides a 

better approximation for assessing population risk 

[Crump 1998]. 

4.3.3.2 Direct Assessment Methods

Direct methods of assessing exposure refer to obtaining direct measurements of the agent of interest (e.g., airborne concentrations) or biomarkers 

of exposure [Checkoway et al. 2004; Nieuwenhuijsen 2010; NRC 1991]. These data are generally the 

most informative in risk assessment and are preferably collected from measurements at the individual 

level, although group assignment is also possible. 

Information from personal monitoring is likely to 

provide the most accurate estimate of individual 

exposure. Personal monitoring can be conducted 

by using direct reading devices and breathing-zone 

air samples; by using in vivo (e.g., whole body radiation counter) measurements; or by using biomarkers of the agent of interest, its metabolites, or 

its effects (e.g., chromosome aberrations from ionizing radiation exposure) in biologic media such 

as blood, hair, excreta, sputum, sweat, or exhaled 

breath [Nieuwenhuijsen 2010; White et al. 2008]. 

Ideally, exposure data stem from monitoring during tasks that are representative of the occupation 

of each worker and over an adequate period to 
30 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

inform the exposure distribution. For example, 

many workers employed in the nuclear industry 

have worn personal radiation dosimeters in radiation areas throughout their careers, beginning as 

early as the late 1940s. However, personal monitoring of ionizing radiation exposure is the exception; 

limitations in logistics, costs, and technology have 

excluded widespread use of personal monitoring in 

most other industries. 

In most epidemiologic studies, the direct measurement of individual exposure is not practical given 

large numbers of study participants; therefore, a 

common approach is to use group-level measurements [Checkoway et al. 2004]. Group-level measurements pertain to either (1) measurements 

from personal monitoring of a worker or a sample 

of workers who represent a group with similar exposure or (2) ambient measurements in work areas occupied by a group with similar exposure. A 

summary measure of exposure is assigned to each 

member of the similar-exposure group; thus, estimates rely on the underlying assumption of similar 

exposure level and variation among persons within 

the similar-exposure group. Group measurements 

are a source of Berkson error and may result in 

shared error (see Appendix B). The effects of these 

errors on risk estimates should be evaluated in the 

risk assessment whenever practical. 

Ambient (stationary or area) measurements are 

further limited by the required translation to individual exposure [Nieuwenhuijsen 2010]. For 

example, measurements from a fixed air sampler 

placed between the exposure source and the exposed person may tend to overestimate the individual exposure. Furthermore, sampling plans are 

often designed to describe maximum exposures 

for regulatory compliance purposes. Hence, exposure estimates from such sampling plans would be 

susceptible to overestimation. Risk assessors consider the potential for exposure misclassification 

resulting from the design and conduct of ambient 

exposure measurements that are subsequently used 

in risk assessment. Regarding group assignments 

based on personal monitoring, exposure estimates 

are strengthened by increased sample sizes and the 

use of repeat measures that enable an assessment 

of between- and within-worker variability. NIOSH 

risk assessors consider sample size and the availability of repeat measures when assessing the validity of group assigned exposures from personal 

monitoring data. 

4.3.3.3 Indirect Assessment Methods

There often are few historical industrial hygiene 

monitoring data available for most hazards; therefore, indirect methods of exposure estimation are 

commonplace in occupational studies. Exposure 

estimates can be derived indirectly from proxy 

measures, questionnaires, expert judgement, jobexposure matrices (JEMs), statistical models, or 

any combination of these sources. Several comprehensive reviews on data sources, assessment methods, uncertainties, and validation techniques are 

available to risk assessors [Kauppinen 1994; Seixas 

and Checkoway 1995; Stewart et al. 1996; Teschke 

et al. 2002].

4.3.3.3.1 Self-Report or Proxy Respondent Data

When data are obtained directly from individual 

study participants or indirectly from proxy responses to a study interview or questionnaire, assessments are subject to bias from recall that has 

been influenced by case status (i.e., recall bias). 

The literature is abundant with reports examining 

the validity and reliability of exposure estimation 

methods using self- or proxy-reported data [Ahlborg Jr 1990; Baumgarten et al. 1983; Benke et al. 

2001; Bond et al. 1988; Bourbonnais et al. 1988; 

Fritschi et al. 1996; Joffe 1992; Nieuwenhuijsen 

2010; Stewart et al. 1987; Teschke et al. 1994]. 

4.3.3.3.2 Job Exposure Matrix

A JEM is widely used by NIOSH for estimating 

exposure indices, whereby a job, which is defined 

by relevant employment information (e.g., job 

title, task, department, and plant), is systematically linked to an exposure level. Noteworthy early 

JEM examples involve assessments of exposures to 

silica, asbestos, and solvents [Dement et al. 1983; 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 31

Eisen et al. 1984; Gardner et al. 1986; Rice et al. 

1984; Rinsky et al. 1987; Seixas et al. 1997; Stewart 

et al. 1986]. JEMs have been used to identify similar exposure groups, to provide individual qualitative and quantitative exposure estimates, and in 

conjunction with algorithms and statistical models, 

to fill in gaps in exposure information during time 

periods when monitoring data were unavailable 

[Coughlin and Chiazze 1990; Dement et al. 1983; 

Eisen et al. 1984; Hallock et al. 1994; Hornung et al. 

1994; Seixas et al. 1997; Woskie et al. 1988].

In its simplest form, the JEM is a table with rows 

and columns characterizing occupation and exposure, respectively. Thus, each cell represents an estimate of the exposure for individuals linked to an 

occupation. Strata for occupation and exposure are 

optimized to increase estimate precision while reflecting the exposure gradient, which is necessary 

for dose-response analyses. Of course, there is still 

a large potential for exposure misclassification in 

a two-dimensional JEM. This misclassification can 

be reduced by adding dimensions. Contemporary 

JEMs typically describe exposures along four axes, 

comprising strata for the agent, job or task, time, 

and location.

NIOSH risk assessors evaluate the quality of source 

data and methods used in the JEM to reduce exposure misclassification. Given the JEM’s reliance 

on employer-provided information, the completeness, accuracy, and scale of these data are typically 

scrutinized. These data fall into two categories corresponding to the primary dimensions: (1) individual employment information used to establish 

task, time, and location of the worker and (2) process information and plant industrial hygiene data 

used to assess the exposure potential (i.e., agent). 

Worker data often stem from personnel records, 

medical histories, or questionnaires. Exposure data 

often include job descriptions, chemical inventories, monitoring data, and incident and accident 

reports. Information on exposure modifiers is likely found in plant records on engineering controls, 

administrative policies, and personal protective 

equipment use. Employer-provided information 

is rarely complete; therefore, JEMs are sometimes 

augmented by data from other sources (e.g., new 

measurement data, statistical models, and other 

JEMs). For example, industrial hygiene data from 

routine sampling that began in the 1980s may have 

supported exposure estimates for previous decades. 

In this case, the risk assessor assesses the methods 

used to extend estimates to unmonitored periods. 

In some instances, data are available from other 

sources (e.g., measurement data, statistical models, and other JEMs) that can be used to assess the 

quality of the JEM and/or quantify the magnitude 

of potential misclassification. For example, consider a cohort study that estimated exposure by using 

personnel records and ambient air measurements 

but had personal monitoring data available on a 

subset of the study population. These monitoring 

data could then be used as a standard for comparison to study estimates and be a means to calibrate 

the JEM. 

The exposure information in the epidemiologic report is likely to be brief and have limited use for 

assessing data completeness. Fortunately, superior 

JEMs are often documented in separate detailed 

reports that are available in the literature or can be 

found in study records. For example, exposure estimates for a cohort mortality study of berylliumprocessing workers in multiple plants [Schubauer-Berigan et al. 2011] relied on data from three 

separately published JEMs [Chen 2001; Couch et 

al. 2011; Sanderson et al. 2001] for dose-response 

analyses that were subsequently used in quantitative risk assessment for developing permissible 

exposure levels [Schubauer-Berigan et al. 2017]. 

When JEM data are not published, risk assessors 

are encouraged to contact investigators for additional documentation needed to assess the quality 

of the exposure data supporting study findings. 

4.3.3.3.3 Expert Assessment

Employment information and/or self-reported data 

are often used in tandem with expert judgement by 

industrial hygienists, chemists, engineers, and other 

professionals to estimate exposure [Nieuwenhuijsen 

2010; Teschke et al. 2002]. It is generally thought 
32 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

that experts, having a better understanding of exposure mechanisms because of their training, can 

more accurately estimate exposures than can the 

workers themselves. Furthermore, if the experts 

are kept blind to case status, the potential for information bias is reduced. Nevertheless, it may be 

impractical for experts to become suitably familiar with all exposure conditions over the period 

of interest without detailed information from employer records and the affected workforce. Another 

disadvantage of expert judgment is an inherent 

inconsistency among experts, given relatively unstructured opinions about the exposure that have 

developed from varying levels of training and experience. In preferred studies using expert judgement, reliability is typically assessed by comparing 

estimates from two or more experts [Benke et al. 

1997; Kromhout et al. 1987; Ramachandran et al. 

2003; t’ Mannetje et al. 2003; Van Wendel De Joode 

et al. 2005a]. When available, comparisons with 

measurement data are preferred for assessing validity [Benke et al. 1997; Tielemans et al. 1999; Van 

Wendel De Joode et al. 2005b]. 

4.3.4 Factors Compromising 

Internal Validity 

An important consideration not listed among Hill’s 

viewpoints is refutation of alternative explanations 

of study findings. An important source of spurious 

results in research findings is systematic error (i.e., 

bias). Bias is defined as a deviation of the results 

or inferences from the truth, or processes leading 

to that deviation [Gail and Benichou 2000]. Study 

designs are typically evaluated by risk assessors to 

ensure that candidate epidemiologic studies are absent of major systematic errors. Common sources 

of systematic errors can be classified into four general forms: 

■ Selection bias, resulting from procedures used 

to select participants into or out of the study 

or that inherently occurs as part of the normal 

occupational setting (e.g., healthy worker survivor bias effects described below).

■ Information bias, resulting from misclassification of the study participants’ disease or exposure status.

■ Confounding, which is a mixing of the effects 

from the exposure of interest with the effects of 

other measured or unmeasured factors (confounders) on the risk of the adverse effect. Insufficient accounting for confounding factors 

can lead to biased risk estimates.

■ Healthy worker effects, which are a combination of selection and confounding biases resulting from relationships between heath status, employment, and exposure. This source 

of potential bias is restricted to observational 

studies of working populations.

On the basis of these general forms, risk assessors 

must answer the following questions: 

1. Is there any evidence suggesting a potential 

for a strong selection bias that may fully explain the observed findings? 

2. Is there any evidence supporting the potential for a strong information bias that may 

fully explain the observed findings? 

3. Could the findings of the study be attributed to confounding by other risk factors, either because of inadequate control (residual 

confounding) or because of lack of control?

4. Are there plausible alternative explanations 

that may fully account for the observed 

findings? 

5. In lieu of bias, how likely are the study findings to have resulted from chance?

These questions describe the potential for a strong 

bias or imprecise findings that may invalidate risk 

estimates. Ideally, the risk assessment relies on 

study results that are not attributable to bias or 

chance; therefore, a positive response to any of the 

questions above is generally grounds for removal 

of the study from further consideration. The term 

strong is used to describe an unacceptable bias by 

its degree; therefore, it implies that a small distortion of the effect relative to its reported size (i.e., 

a potential weak bias) may not be disqualifying 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 33

for risk assessment purposes. However, the potential for a bias that may only partially explain study 

findings still requires evaluation for proper use 

in risk assessment. To aid in responding to these 

questions, risk assessors are encouraged to review 

Appendix B of this report, which provides specific 

information on potential biases common in human 

risk assessment.

Bias can occur in any stage of the research, including the literature review, study design, data 

collection, analysis, interpretation of results, and 

publication. When reviewing studies for validity, 

risk assessors avoid using a “guilty until proven innocent” approach, whereby one assumes that the 

study design and analysis are inadequate unless 

enough information to the contrary is provided by 

the study authors. Instead, the risk assessor evaluates the potential impact of study limitations and 

omissions on findings for determining WoE [Zaccai 2004]. Although Appendix B provides some 

discussion on specific types of biases that may be 

encountered, for additional information risk assessors may consult several highly cited articles [Arrighi and Hertz-Picciotto 1994; Delgado-Rodríguez 

and Llorca 2004; Greenland et al. 1999; Grimes and 

Schulz 2002; Sackett 1979] and epidemiologic texts 

[Breslow and Day 1980; Breslow and Day 1987; 

Checkoway et al. 2004; Gail and Benichou 2000; 

Rothman et al. 2008].

4.4 Laboratory Animal Data

Data from human studies are often inadequate to 

fulfill hazard identification; therefore, risk assessors use toxicological information from bioassays 

in animals, either alone or in combination with 

information from human studies. As noted by 

Cohen et al. [2004], using animal data as the basis for human risk requires two fundamental assumptions: (1) the findings in the animal study are 

relevant to humans (interspecies extrapolation) 

and (2) the doses used in the animal bioassay are 

relevant for estimating risk at human exposure levels (dose extrapolation). In general, animal studies 

predict human health risks well [Allen et al. 1988; 

Crump et al. 1989; Griffin 1986]. For example, in 

its recent report on tumor site concordance, IARC 

confirmed that the induction of cancer in experimental animals is relevant to the identification of a 

carcinogenic hazard to humans, given that all human carcinogens that have been adequately tested 

in animals have also been shown to cause cancer in 

animals [IARC 2019]. Nevertheless, tumor site concordance is not assured or even routinely observed. 

For example, there are four agents for which there is 

sufficient evidence for breast cancer in humans and 

seven agents for which there is sufficient evidence 

for breast cancer in experimental animals, but only 

one of these agents causes breast cancer in both humans and animals [IARC 2019]. NIOSH recognizes 

that differences exist between species because of 

the unique inherent physiological and biochemical 

mechanisms in each species [Homburger 1987] and 

because exposure conditions in animal models can 

differ greatly from the work environment. Moreover, 

humans lack the homogeneity observed among inbred experimental animals. Collectively, these differences may lead to false-positive or -negative assay 

findings [Ennever and Lave 2003; Ettlin and Prentice 2002; Haseman and Elwell 1996]. 

As in human studies, not all animal studies are 

equally useful. Some studies are limited by virtue 

of their sample size, experimental design, methods, 

and the interpretation of the results by authors. It 

is very important that the toxicity evaluation of 

a substance be based on information from wellconducted studies. Evaluation of the quality and 

reliability of individual animal toxicity studies requires consideration of factors associated with a 

study’s hypothesis, design, methods, execution, 

analysis, and interpretation [Hothorn 2014; Klimisch et al. 1997; Lu and Kacew 2002; NTP 2015a; 

NTP 2015c; Salem and Katz 2014]. 

4.4.1 Relevance and Appropriateness 

of the Animal Model

A relevant and appropriate animal model of human 

disease is one that includes a living organism in 

which normative biology or behavior can be studied, 
34 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

or in which a spontaneous or induced pathological 

process can be investigated, and in which the phenomenon in one or more respects resembles the 

same phenomenon in humans [NRC 1981]. The 

term relevance refers to the comparability of the observations in animals to those in humans. Clearly, a 

preferred model is one in which the phenomenon 

of interest is observed equally in both animals and 

humans. In practice, the degree of direct comparability can be low, which is a limitation in animal 

studies. Limited comparability does not preclude 

the use of animal information in human risk assessment. In fact, animal studies may provide the best 

dose-response information to support human risk 

assessment. The term appropriateness refers to factors that support the choice of animal model for risk 

assessment, depending on the scientific questions 

to be addressed. These factors can include animal 

life span; genetic homogeneity; specific anatomical, 

physiological, or behavioral attributes; the frequency of the effect of interest and its background occurrence; and availability (supply and cost) [Hedrich 

and Bullock 2004; Krinke 2000; NRC 1981]. 

Understanding the mode of action (MoA) of a 

chemical helps to establish the best animal model 

for use in the toxicity testing and risk assessment 

(MoA is further described in Section 4.4.2.2). For 

example, male rats are not a useful model for evaluating the risk of kidney cancer from gasoline exposure in humans [Baetcke et al. 1991]. The MoA for 

kidney cancer in male rats from gasoline exposure 

(and various other chemicals, as described later 

for d-limonene exposure) involves the presence of 

alpha-2u-globulin protein. This protein combines 

with the metabolites of gasoline, which can eventually induce kidney tumors. Humans, female rats, 

and mice do not have this protein. From everything 

known to date, the presence of alpha-2u-globulin 

is necessary for the development of kidney tumors; 

therefore, no excess kidney cancer has been observed in exposed mice or female rats. 

For test compounds that depend on a metabolite 

to produce an adverse effect in the animal (for example, epoxide formation for some carcinogens), 

the most appropriate animal species is often the 

one that has the closest similarity to humans with 

respect to relevant metabolic processes involved 

in toxicity [Bogaards et al. 2000; Martignoni et 

al. 2006; Nilsson et al. 2012; Panchal and Brown 

2011]. Sometimes other factors besides metabolism 

are important in selecting an appropriate animal 

model. For example, an adequate number of test 

animals are needed to ensure that a study has adequate statistical power to detect an adverse effect. 

Therefore, if rhesus monkeys are most metabolically similar to humans, but only small numbers of 

these animals were used in the experiment and the 

toxicologic response was equivocal, then the rhesus 

monkey may not be the best animal model for the 

risk assessment.

Ideally, animal studies used in human risk assessment should be performed in animals of appropriate age (adult versus newborn), of both sexes, and 

with a health status (e.g., pregnant versus nonpregnant) that corresponds to human exposure 

and toxicity. The study should take into consideration the appropriate duration and pattern of exposure (acute versus chronic; single exposure versus 

repeated administration) to simulate human occupational exposure.

Often, a test compound will have data from several animal studies. The information on test animals should include species, strain, sex, age, and 

number of animals per group from any individual 

study. Ideally, an animal model with the most valid 

biological rationale (e.g., similar pharmacokinetic 

profiles) should be selected as the animal model 

most relevant to humans. However, in some cases 

no such closely relevant model exists. In such cases, 

the animal model that is most sensitive (i.e., showing a toxic effect at the lowest administered dose after dosimetric adjustments are performed) is often 

used [Barnes and Dourson 1988].

4.4.2 Animal Toxicologic Study Design

Animals most often used in bioassays are rat, mouse, 

guinea pig, hamster, rabbit, monkey, and dog. Codified U.S. EPA guidelines for conducting animal 

toxicity studies are available [Health Effects Testing 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 35

Guidelines, 40 CFR 798]. International guidelines 

are provided by the Organisation for Economic 

Co-operation and Development (OECD), in the 

OECD Guidelines for the Testing of Chemicals, 

Section 4, Health Effects, Tests 403, 412, 413, and 

452 for inhalation exposure studies [OECD 2009; 

OECD 2018b; OECD 2018c; OECD 2018d] and 

Tests 402, 404, 410, 411, and 429 for dermal toxicity studies [OECD 1981a; OECD 1981b; OECD 

2010; OECD 2015; OECD 2017]. These guidelines 

provide recommendations on physical parameters 

of test substances and testing conditions; on laboratory animals (e.g., species, number, sex, age, and 

condition); and on gross pathology, histopathology, and clinical, biochemical, hematological, ophthalmological, and urinary excretion tests to be included in the study. It is generally recommended to 

conduct toxicity tests for each test compound in at 

least two species, typically rats and mice [Bingham 

et al. 2001; Salem and Katz 2014]. In addition to 

improving consistency between studies, these test 

guidelines can be useful for evaluating WoE among 

studies, where deviations from these guidelines can 

indicate potential weaknesses.

NIOSH considers that for an experimental animal 

study to be useful for human risk assessment, its design should be sufficiently documented to include 

information on study aims and hypotheses tested; 

reasons for selecting the animal model used; species, strain, weight/age, sex, and source of animal 

used; details of each experiment performed, including its design and number of animals used; exposure, including dose, route, schedule, and duration; 

and statistical methods [EPA 1994]. This information may be found in a study protocol or in the 

methods section of the study report. Superior studies provide enough documentation of the methods 

to replicate findings. Combined with a clear and 

thorough presentation of findings, the study design 

information is a valuable resource for judging WoE. 

It is also often helpful if individual animal data are 

made available for additional analysis.

Studies should include enough test animals and 

dosing groups to support statistical analysis. The 

type of study conducted (acute, sub-chronic, or 

chronic) provides insight into the number of animals and dose groups needed. For example, chronic 

animal bioassays (2-year studies in mice and rats) 

typically involve 50 animals per sex per species per 

dose. These types of studies are useful for collecting 

information on carcinogenicity and other health 

effects that require chronic exposure. Sub-chronic 

studies, on the other hand (90-day studies in mice 

and rats) typically use fewer animals per species 

per sex per dose and are typically used to detect 

non-cancer, organ system health endpoints such as 

pulmonary inflammation, liver toxicity, and kidney 

toxicity. The toxicology study guidelines referenced 

in this section provide more detailed information 

on studies with adequate numbers and dose groups.

Datasets from studies adhering to good laboratory 

practices (GLP) [OECD 2005] and to internationally accepted test guidelines (e.g., OECD, EPA, and 

EU) are preferred as candidates for risk assessment. 

In addition, studies with sufficient details on methods, 

analysis, and results that have been peer-reviewed are 

often acceptable [Klimisch et al. 1997]. 

4.4.2.1 Exposure Information 

Exposure conditions play a vital role in the experimental design of animal toxicity studies. Determination of the dose that reached the test animal in 

a study is a complex process. This involves using 

proper methods for the generation, characterization, and delivery of a test compound [EPA 1994]. 

NIOSH has established the following criteria to assess the suitability of animal toxicologic studies for 

risk assessment purposes.

Ideally, the study should clearly define the physicochemical characteristics of the substance used, such 

as purity, stability, pH, partition coefficient, vapor 

pressure, particle size and distribution, breathing 

zone concentration, and vehicle. The concentration 

of the test compound should be reported as means 

and variances. The exposure concentration, type of 

exposure (e.g., vapor or aerosol), administration 

route, exposure schedule, and exposure duration 

should be clearly described. 
36 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

For an inhalation study, the information should 

include a description of the generation and characterization technology used (e.g., chamber design, type, dimensions, uniformity of distribution, 

source of air, generating system, air conditioning, 

and exhaust treatment) [Nelson 1992; Wong 2007]. 

The number of air changes, air flow rate, oxygen 

content, temperature, and relative humidity are 

exposure chamber characteristics that should be 

monitored and reported as means and variances. 

The description of the characterization method(s) 

should also include frequency of measurement, 

calibration of the measurement instrument, frequency of the calibration, and other quality assurance elements [Barrow 1989; Chen and John 2001; 

Moss and Cheng 1989; Wong 1999].

The various inhalation exposure techniques include 

whole body, head only, nose only, intra-tracheal instillation, and oropharyngeal aspiration [Driscoll 

et al. 2000; Phalen 1997; Sahu and Casciano 2009; 

Wong 2007]. Factors such as safe and efficient generation, amount of material, test compound stability, exposure duration, and the measurements 

desired influence the selection of an exposure technique for a study design. For instance, in chronic 

inhalation exposure studies, whole-body exposure 

of laboratory animals in cages is the most common 

method, whereas nose-only exposures are most 

often used for short-duration particle exposures. 

However, it should be noted that several factors 

such as heat, stress, and anesthesia could affect the 

biological patterns of the animal, potentially influencing results [Hughes et al. 1982; Mete et al. 2012; 

Overmyer et al. 2015; Stratmann et al. 2010; Suvrathan et al. 2010]. Due consideration of these issues 

should be included in detail in the data analysis to 

ensure appropriate comparisons.

Test agents may affect lung ventilation, function, 

uptake, clearance mechanisms (e.g., mucociliary clearance), and retention of the dose. Particle 

overloading in the lungs of test animals, especially 

rats and mice—a well-known outcome of particle 

exposures at higher concentrations and/or longer 

duration—should also be evaluated [EPA 1994; 

Morfeld et al. 2015; Morrow 1988; Oberdörster 

1995; Oberdörster 1997; Olin 2000; Pauluhn 2014]. 

Overloading results in aggregated alveolar macrophages (AMs) engorged with phagocytized dust 

particles. These AMs release an array of mediators, 

resulting in various inflammatory responses and 

tissue injury [EPA 1994; Kanj et al. 2005; Laskin 

and Pendino 1995]. Overloading is characterized 

by a dose-dependent decrease in the rate of particle 

clearance from the lungs; thus, first-order clearance 

kinetics would underestimate the retained particle 

dose in the overloaded rat. Overloading can be addressed in quantitative risk assessment by using 

dosimetry models that account for higher-order 

kinetics to estimate the retained particle dose in 

rodents and humans. The nature and severity of rat 

lung responses to particle overload are less certain 

than the lung responses in humans at equivalent 

doses; however, without overloading, rats would 

not reach the equivalent mass lung doses that have 

been observed in workers in dusty jobs [Kuempel 

et al. 2014; Morfeld et al. 2015; Olin 2000; Pauluhn 

2014]. Comparison of the particle burdens in the 

lungs of test animals to the particle burdens expected in the lungs of occupationally exposed humans 

is useful in evaluating whether the experimental 

study in question is relevant to occupational health 

risk assessment. Sections 5.6.2.5 and 5.6.2.7 provide additional information on lung clearance kinetics of particles in rodents and humans.

Animal studies have different durations (acute, subchronic, and chronic) and frequencies of exposure 

(single, intermittent, and continuous). Study variations help to identify the hazard associated with a 

given test compound, but not all of them may be 

appropriate for quantitative risk assessment, depending on the human exposure of concern. 

Appropriate control groups of unexposed (e.g., aironly controls in inhalation studies) and/or vehicleexposed animals should be included in the study. 

The control group(s) should be treated similarly to 

the chemically treated group, except that the control 

group should not receive any of the test compounds 

[Hayes 2008; Salem and Katz 2014]. In addition, 

historical control data can also be used to evaluate the differences between control and treated 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 37

groups. In general, historical control data should 

be submitted from the same laboratory and should 

be from animals of the same age and strain generated during the 5 years preceding the current study 

[OECD 2018d]. 

Most often, animal bioassays expose animals to 

higher doses of a chemical than those common in 

human exposure [Klaassen et al. 2013]. Sometimes 

the animal doses are comparable to occupational 

exposures, but often they are significantly higher 

(see Section 5.6.2.8 for more details). Toxicity observed at high doses may or may not occur at lower 

doses. Therefore, animal studies should always be 

evaluated in the context of dose-response relationships. Doses for vapor exposure are usually provided in units of parts per million (ppm) or milligrams per cubic meter (mg/m3

), and doses for 

airborne particle exposures are usually provided as 

the mass concentration (mg/m3

) and aerodynamic 

particle size (e.g., mass median aerodynamic diameter, MMAD). Number concentration is also 

reported for airborne particles or fibers. The size 

and shape of particles determine the region in the 

respiratory system in which particles are deposited 

(see Section 5.6.2.2 for more details). Care should 

be taken to ensure that particle morphology under 

test conditions reflects human exposure patterns. 

The physicochemical properties of a particle determine whether it will be dissolved in the blood or 

removed by clearance mechanisms.

In dermal exposure, the contact area, absorption, 

concentration of the chemical, contact frequency, 

retention time, and penetration potential contribute to the dermal toxicity [Marquart et al. 2003; 

Poet and McDougal 2002; Schuhmacher-Wolz et 

al. 2003; van Ravenzwaay and Leibold 2004]. 

4.4.2.2 Consideration of Mode of Action and 

Adverse Outcome Pathways

Typically, when conducting risk assessment to inform the development of a REL, NIOSH evaluates 

health effects that may be experienced by humans 

or that may be related to health effects experienced by humans, as evidenced by the results of 

human and animal studies. Once the constellation 

of health effects under consideration has been established, the risk assessor critically evaluates the 

health effects to determine which effect(s) are of interest. In doing so, the risk assessor clearly explains 

the rationale for selection of the health effects and 

their relevance to human health. 

As part of its hazard identification, NIOSH evaluates MoA and adverse outcome pathway (AOP) 

information to determine whether evidence is 

adequate to establish that the events leading to 

adverse effects in animals are unlikely to operate 

in humans. An AOP is a construct portraying the 

sequence of events between a direct molecular initiating event and the adverse effect relevant to risk 

assessment [Ankley et al. 2010]. Whenever information is available to describe the MoA or the AOP, 

NIOSH uses this information to evaluate the doseresponse information. MoA is generally thought 

of as the underlying biochemical interactions that 

lead to the expression of the adverse effect. Full 

information on the MoA is rarely available. MoA 

refers to the general processes and key events that 

are involved in the toxicity of a chemical. MoA 

analysis includes review of physical, chemical, and 

biological information on the substance [Boobis et 

al. 2006; Boobis et al. 2008].

It is sometimes observed that humans have a different sensitivity to a test compound than experimental animals do [Lasagna 1987]. In addition, there 

are cases where the MoA or adverse effect identified in an animal model is not relevant to human 

health. The case of d-limonene exposure causing 

kidney tumors in male rats, but not in female rats 

or either sex of mice, is one example. The male rat 

kidney tumors have been linked to a metabolite 

of d-limonene binding to the protein, alpha-2uglobulin, leading to toxicity, cellular regeneration, 

and tumor formation. Humans have no functionally similar protein; therefore, this MoA does not 

appear to operate in humans, and the male kidney 

tumors do not indicate a human cancer risk. Cohen et al. [2004] summarize this case and others. 

In their report, the authors describe a framework 
38 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

for evaluating the relevance of chemically induced 

animal tumors to humans.

An understanding of the MoA of the toxic agent can 

help define which data are most appropriate for consideration. The EPA has described MoA framework 

initially for cancer risk assessment in the Guidelines 

for Carcinogen Risk Assessment as a sequence of key 

events and processes, starting with interaction of an 

agent with a cell, proceeding through operational 

and anatomical changes, and resulting in cancer 

formation [EPA 2005]. Examples of possible modes 

of carcinogenic action include “...mutagenicity, mitogenesis, inhibition of cell death, cytotoxicity with 

reparative cell proliferation, and immunologic suppression.” Later the MoA framework concept was 

expanded to assess risk for non-cancer endpoints 

[Bogdanffy et al. 2001; Julien et al. 2009; Lochner 

et al. 2005; Seed et al. 2005]. The substance may 

induce adverse effects by more than one MoA in a 

single tissue or at different sites. Therefore, a single 

MoA for an endpoint may or may not apply to all 

other health endpoints.

An expanded MoA framework focuses on the WoE 

establishing the MoA in animals and whether the 

key events identified in animals are plausible in humans, and it considers kinetic and dynamic factors 

to determine whether the MoA is plausible in humans. To use the MoA framework, the risk assessor 

asks the following questions (Figure 4-2):

■ Is the WoE sufficient to establish the MoA in 

animals? The first step in considering the relevance of MoA information to human health 

is having sufficient MoA information on the 

animal species for the health effect of interest. 

The default position is that the health effects 

observed in animals are relevant to humans. 

As stated in Seed et al. [2005], “[W]hen data 

are insufficient to confidently characterize an 

MoA for test animals, the animal tumor data 

are presumed to be relevant to humans and a 

complete risk assessment is necessary.”

■ Are key events in the animal MoA plausible in 

humans? To evaluate whether the MoA is 

relevant to humans, there must be sufficient 

information available regarding the potential 

for the key events identified in the animal MoA 

to operate in humans. For example, a key enzymatic pathway observed in the animal should 

also be present in humans or, if the exact enzymatic pathway is not present, then determine 

if there are pathways that serve a similar or 

identical function. If there is insufficient information in humans to characterize the relevant 

pathways, as described above, then the animal 

data are presumed to be relevant to humans 

and a complete risk assessment is necessary. 

However, if there is clear evidence that the 

relevant pathways do not operate in humans, 

then the risk assessor should assume that the 

observed effects in animals are not relevant for 

humans for this endpoint. Unless it is known 

that all the health effects observed in animals 

derive from a common MoA, this analysis 

needs to be conducted separately for all health 

effects under consideration for risk assessment. 

■ If we take into account kinetic and dynamic factors, 

as well as life stages, then are key events in the animal 

MoA still plausible in humans? This step requires 

quantitative information on the relative kinetic 

and dynamic factors that would influence risk in 

humans and animals, as well as consideration of 

life stages of potential exposure in humans. For 

example, consider the case in which the MoA has 

been identified in animals, involving toxicant metabolism by a specific enzymatic pathway found 

in both animals and humans; however, there is 

a high rate of metabolism by this pathway in the 

rodent that is not evident in humans. In addition, 

humans have a competing enzymatic pathway that 

metabolizes the toxicant much more rapidly. Thorough analysis of the kinetics indicates the potential 

human toxicity via this MoA is, in fact, very low, 

suggesting the conclusion that there is no need to 

conduct a risk assessment for this endpoint. The 

same type of analysis could be conducted when 

considering potential exposures during specific 

life stages, if that is deemed a critical variable. 

AOPs are structured representations of biological events leading to an adverse effect. AOPs are 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 39

 

Figure 4-2. Framework for Mode of Action (MoA) assessment (adapted from Seed et al. [2005])

flexible frameworks that can include linking relationships that are causal, mechanistic, inferential, 

or correlation based [Ankley et al. 2010]. AOPs 

have been established for several chemical groups. 

In an AOP, a structured sequential chain of events 

is constructed with all available scientific information. For example, a chain of events might entail 

a toxicant exposure causing a molecular initiating 

event, which then leads to a series of key events 

causing the adverse outcome of interest. Information on key events in AOPs, the biochemical 

mechanism of action, or the processes involved in 

the presumed MoA could give insight into the toxicity of chemicals, including the characteristics of 

the dose-response (see Appendix C for more information on AOPs).

4.4.2.3 Selecting the Adverse Effect 

of Interest

Ideally, the dose-response relationship is demonstrable. The observed effects should be directly 

related to the magnitude of exposure to the test 

compounds and not influenced by concurrent exposure to other compounds or already existing 

health conditions [Lewis et al. 2002]. In general, 

the recommended list of hematology, clinical biochemistry, and histopathological examinations to 

be evaluated in the laboratory studies are given in 

several guidelines [Crissman et al. 2004; OECD 

2009; OECD 2018b; OECD 2018c; OECD 2018d; 

Weingand et al. 1996]. NIOSH risk assessors refer 

to these guidelines, as applicable, for a better understanding and evaluation of the study.
40 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Preferably, the histopathological examinations 

should be of all tissues for all treated doses and 

control groups and all tissues from animals. However, many published studies, excluding the NTP 

studies, employ histopathological examination only 

for the target endpoints, which although somewhat 

limited in scope still may be useful for risk assessment. In addition, quantitative histopathological 

measures of response (e.g., dichotomous or ordinal 

categories of the occurrence and severity of an adverse response) are necessary to use these data in 

dose-response modeling.

A special consideration is the appropriateness of 

selecting early biomarkers, precursor effects, and 

critical molecular/cellular changes in lieu of the 

adverse effect anticipated in humans. An example 

of this is evaluating inflammatory markers that 

lead to secondary genotoxicity and ultimately cancer after exposure to inhaled particulates. Cancer 

is of direct and relatable interest to human health. 

Moreover, for poorly soluble, low-toxicity particulates, the MoA has been described as an irritation 

response, followed by inflammation and production of reactive oxygen species, which leads to secondary genotoxicity and ultimately an increased 

risk of cancer. When the MoA is well-supported, 

it may make sense to conduct risk assessment on 

early biomarkers or molecular changes. Ideally, the 

causal and quantitative relationship between the 

occurrence of the biomarker or molecular changes 

and the outcome of interest is understood. When 

the relationship is correlative or the quantitative 

aspects are poorly understood, the relevance to 

human experience becomes more tenuous. Validation of the causal pathways and description of the 

quantitative relationship and attendant uncertainties between the outcome of interest and the precursor events or biomarkers are important steps to 

consider in conducting this type of risk assessment. 

Similarly, tests using structurally related compounds or any active metabolite of the compound 

of interest could also be considered for a comparison of results. However, toxicity studies of structural

analogs and metabolites should be carefully considered in light of the MoA, metabolic issues, and 

other factors that may influence differences in response between the alternate chemical and the 

chemical of interest. Demonstrated relevance of the 

findings for analogs or metabolites to the chemical 

of interest is necessary. In addition, the limitations 

and uncertainties associated with using toxicity 

data from alternate chemicals should be thoroughly and explicitly discussed when reaching any risk 

assessment conclusions. When multiple toxicity 

studies are available, the studies should be reviewed 

with reference to the types of effects observed in 

different test species and strains. Consistency of response across species, sex, and/or route of exposure 

increases the WoE that the effect might occur in 

humans. In contrast, an effect observed in only one 

species or sex may need further evaluation. Results 

replicated by independent researchers would have 

increased credibility. 

Once the evidence is evaluated, NIOSH assesses 

the toxicological database for completeness. A 

complete toxicological database includes studies 

that evaluate carcinogenic, genotoxic, reproductive, developmental, and other organ effects (e.g., 

immunotoxic, neurotoxic, nephrotoxic, irritation, 

and sensitization). Ideally, the literature describes 

the dose-response relationship; concordance across 

species, strain, sex, exposure routes, or in multiple 

experiments with respect to adverse effects; effects 

that are biologically plausible and of human relevance; and similar effects with structurally related 

compounds. However, a complete toxicological database is not essential for hazard identification if the 

observed adverse effects are relevant to occupational 

exposures. If there are only limited data on a specific chemical, then all available studies with limited 

information should be critically evaluated to determine the usefulness of the information for risk assessment. If concordance across species/strain/sex is 

not observed, then additional evaluation is needed; 

in the absence of information to the contrary, the 

more sensitive species/strain/sex is often used.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 41

5 Dose-Response Assessment

5.1 Introduction 

The dose-response assessment is the second step 

of NIOSH risk assessment. The aim of the doseresponse assessment is to obtain reliable and valid 

estimates of the point of departure (PoD) in a cause 

and effect relationship between the exposure and 

outcome of interest or the risk at prescribed dose 

levels (e.g., risk per unit dose) that can be used in 

risk characterization. 

The dose represents a quantitative metric d, usually 

derived from some external exposure, and believed 

predictive of an adverse effect. Whenever the relationship between the biologically effective dose and 

another dose metric (e.g., absorbed dose, inhaled 

dose, or exposure concentration) is well described 

by a constant ratio (i.e., a directly proportional relationship over the range of doses under consideration), then these doses are interchangeable and 

their dose-responses are equivalent. For example, if 

the inhalation rate during exposure is constant over 

the range of concentrations, then the inhaled dose 

rate is proportional to the concentration; but if the 

inhalation rate is not constant, then the dose-response based on inhaled dose requires adjustment 

to obtain the corresponding dose-response based 

on exposure concentration. 

As another common example, consider a nonlinear 

rate of metabolic activation. The rate of metabolic 

activation of a toxicant may be best approximated 

as linear at low exposure concentrations, but it 

may become nonlinear at high concentrations. If 

the biologically effective dose is the amount that 

is metabolically activated, then the dose-response 

analysis is usually based on the amount activated 

rather than the exposure concentration, if that has 

been quantitatively described and validated. 

In general, quantitative risk analysis relies on mathematical models of association used to describe 

the conditional probability of the adverse effect 

at different levels of exposure to the agent, given 

levels for other direct causes of the adverse effect. 

Statistical methods used in dose-response modeling are numerous and diverse. The onset of personal computing has led to advancements in this 

area that continue today. For example, at the time 

of this writing, considerable advancements in the 

EPA Benchmark Dose Software suite were ongoing. 

Thus, a detailed description of modeling methods 

preferred for all situations faced in risk assessment 

is beyond the scope of this Bulletin. However, simple modeling practices that may be encountered are 

described in the following sections to broadly illustrate dose-response modeling concepts. In specific 

situations, NIOSH encourages risk assessors to take 

advantage of the substantial technical progress that 

has been made in dose-response modeling techniques by referencing recent literature.

5.2 Dose-Response Modeling 

Dose-response regression modeling provides a basis 

to estimate the expected response as a function of 

dose d and possibly other risk factors, X1, X2, ..., Xc, 

together with assumptions about the variability of 

responses. In animal toxicology studies, the doseresponse is often simplified to expected response = 

f(d) since the other risk factors are controlled by design or by the random assignment to dose levels. As 

an illustration, consider the outline of animal study 

data in Table 5-1. At each dose level d, there are n animals exposed, and the corresponding response, Y, 

is number of animals presenting with the adverse effect of interest. The expected proportion of animals 

with the adverse effect is related to each dose d and 

is equivalent to the probability of the adverse effect 

f(d). The function f(d) is evaluated at any dose between the background response (when d = 0) and 

the maximum observed dose. 
42 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

It is preferable to base model selection on biologic 

plausibility. In practice, however, model specification with a clear advantage based on biology is seldom observed. Instead, a suite of plausible models 

is usually fit to the data. When multiple models of 

a response adequately describe the data, the model 

selected for a risk assessment is generally chosen 

by using criteria that are defined beforehand. For 

example, the Akaike information criterion (AIC) is 

a measure of model fit that is often used to select 

a model from among a group of models [Akaike 

1974]. The selection process should be clearly described in the risk assessment. Because different 

model-selection criteria can lead to different model 

choices, model selection is often an area explored 

in sensitivity analysis. Multiple (alternative) estimates are then reported with a description of how 

each estimate was derived. 

5.2.1 Parametric Dose-Response 

Modeling

The function f(di ; θ), which describes the relationship between dose and the expected response for 

observation i, is often assumed to have a known 

form that depends on a vector of parameters θ

whose unknown values are estimated. This assumption places strong constraints on the shape of 

the dose-response curve, and the data are used to 

estimate θ. Unknown quantities of critical interest 

such as risk associated with a given dose or the dose 

associated with a given risk are estimated based on 

the fitted dose-response f (di;θ). Within the form 

adopted for f(∙) multiple ways to describe the effect of dose may be available, e.g., θ1B1(d) + θ2B2 

(d) + ... + θKBK(d), where the Bk(d) are known 

functions of dose d, and K = dimension of the vector of parameters θ = [θ1,...,θK] and k ∈ {1, ... , K} 

identifies a component of θ. As an illustration, if 

the effect of dose is to be described by θ1d + θ2d 2

then Bk (d) = d k

 and θ3 = θ4 = ... = θK = 0. This suggests that a hierarchy of increasing flexibility may 

be examined, e.g., θ1d followed by θ1d + θ2d2

, etc., 

but this should be done carefully since the inclusion of unnecessary terms decreases the degrees 

of freedom and therefore reduces statistical precision. The omission of a necessary term is likely 

to introduce a statistical bias into the estimation. 

Thus, there is trade-off between a potential for bias 

associated with an overly constrained model of the 

dose-response versus a degradation of precision, 

i.e., increased variance, from an unnecessarily flexible model. This relationship between potential 

bias and increased variance holds in general and is 

referred to as a bias-vs-variance trade-off. Ideally, 

a model is selected that is at the level of complexity at which an increase in bias is equivalent to the 

reduction in variance. In practice, there may not be 

an analytical solution for selecting the “best” model; therefore, resampling-based measures such as 

cross-validation and theoretical measures such as 

AIC and Bayesian information criteria (BIC) may 

be used to support model selection. 

Table 5-1. Illustration of data from an animal bioassay for a dichotomous response.*

Dose (di)

Number of exposed 

animals (ni)

Number observed with 

the response (Yi)

Observed proportion

(Yi / ni)

d0

† n0 Y0 Y0 / n0

d1 n1 Y1 Y1 / n1

... ... ... ...

dD nD YD YD / nD

*For example, cancer incidence of a target organ or tissue.

†

Typically, an unexposed group of controls is used and d0 = 0.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 43

5.2.1.1 Dichotomous Response 

Data Modeling

Many different parametric models have been proposed for data from animal toxicology or human 

epidemiologic studies. For example, in the animal 

toxicology setting, the following model specifications are commonly used in dichotomous doseresponse modeling [EPA 2012b]: 

Equation 5-1. Logistic 

Equation 5-2. Log-logistic , 0 ≤ γ < 1, β ≥ 1

Equation 5-3. Gamma , 0 ≤ γ < 1, α ≥ 1, β ≥ 0 

Equation 5-4. Multistage (degree=2) , 0 ≤ γ < 1, θ1 ≥ 0, θ2 ≥ 0

Equation 5-5. Probit

Equation 5-6. Log-probit , 0 ≤ γ < 1, β ≥ 0.5 

Equation 5-7. Quantal-linear , 0 ≤ γ < 1, β ≥ 0

Equation 5-8. Quantal-quadratic , 0 ≤ γ < 1, β ≥ 0

Equation 5-9. Weibull , 0 ≤ γ< 1, α ≥ 1, β ≥ 0

Equation 5-10. Dichotomous Hill , 0 ≤ γ< 1, 0 ≤ η< 1, β ≥ 1

 

 

 

 

 

 

 

 

 

 
44 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

where f(d) is the probability of adverse response at d,

is the (complete) Gamma function evaluated at x, 

Φ(x) is the cumulative distribution function of a 

standard normal random variable at x (i.e., the integral of a N(0,1) density from –∞ to x), and f(0) 

= γ when d = 0 for models 5-2 to 5-4 and 5-6 to 

5-9. The parameter η of Equation 5-10 represents 

an upper limit on f(d), i.e., f(d)< η for every dose; 

Equation 5-10 may be useful when an upper limit 

on f (d) < η where η < 1 is plausible and it may 

necessitate that η be estimated from the doseresponse data. Some bounds in the above models 

are arbitrarily set to prevent extreme properties 

and attendant computational problems, although 

nonlinear dose-response patterns remain available. Hence, modification of these constraints 

may be necessary when consideration of either 

pattern is unwarranted. Furthermore, although 

the models 5-1 through 5-10 encompass a wide 

variety of curves to represent the dose-response 

and have readily available software for their 

implementation, other parametric forms could 

be considered if necessary. 

Dichotomous outcomes from animal bioassays 

are often modeled under an assumption that the 

sampling variation of the underlying experimental 

process is well represented by a binomial distribution. In some cases, especially where the data are 

pooled from multiple studies or substantial genetic 

variations of the animals are present, this assumption may not be appropriate and extra-binomial 

variability, or over-dispersion, may be observed. 

In these instances, beta-binomial, quasi-likelihood 

methods, or more fully defined models incorporating random effects are preferred.

5.2.1.2 Continuous Response Data Modeling

Continuous data arise when response values come 

from a continuous distribution, for example, precisely 

measured liver weights or pulmonary function tests. 

In these situations, a variety of parametric models 

can predict the mean response. For example, the 

following five parametric models are often considered when modeling continuous data [EPA 2012b]:

Equation 5-11. Linear

Equation 5-12. Linear-Quadratic

Equation 5-13. Power 

Equation 5-14. Hill 

Equation 5-15. Exponential 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 45

The parameters β0, β1, β2, and β3 are specific to the 

given model. It may be necessary to specify bounds 

on these parameters when fitting models. For example, the parameter appearing as the exponent of 

dose d of the Power model must be restricted to be 

greater than zero. 

For continuous data, the choice of the response 

data distribution is important. Some response data 

distributions are right-skewed and are best approximated by a lognormal distribution. In others, 

the distribution of the data is symmetric around 

the mean and a normal distribution is preferred. 

To choose between multiple distributions, the AIC 

can be used to discriminate between different possible distributions. In such cases, it is necessary to 

compute the log-likelihood based on fully specifying the probability density functions.

The above models describe the mean response; 

however, additional modeling assumptions on the 

variance of the response data are generally needed. 

For example, in many situations, the variance may 

be a function of the mean, e.g., it may be a positive 

constant, or it may be proportional to the mean or 

power of the mean such as its square. Model fit is 

examined to assess whether the response mean and 

variance structures are supported. 

5.2.1.3 Parametric Dose-Response Modeling 

including Other Predictors

The extension to include other predictors into the 

function f (di , X1i , X2i , ..., Xci ; θ) to describe the expected response shares much with the dose-only 

modeling described above in that the data are used 

to estimate θ to obtain f (di , X1i , X2i , ..., Xci; θ). However, the models of the data are more complex and 

the vector of parameters θ contains coefficients that 

govern the effect of dose and coefficients for the 

effects of the other variables. In epidemiological 

studies, di, is an exposure metric constructed from 

possibly complex employment histories, and identifying the optimum construct for di may, itself, be 

an important component of the modeling procedure. For human observational studies, predictors 

in the function f (di, X1i, X2i,..., Xci; θ) usually include 

age, sex, and other demographic variables and may 

include interactions or effect modifiers, e.g., doserate effects or other effect modifiers that allow for 

the effect of dose to depend on the other predictors. 

These additional factors can make model selection 

using human data more complex, because confounders, effect modifiers, and complex selection 

processes can be present and often can enter the 

model in many ways. In addition, the differences 

between the study population and the target population should be considered for the risk estimation. 

If interactions are present, then the estimates of 

interest may be made conditional on fixed values 

of X1i , X2i , ..., Xci or averaged over the appropriate 

marginal distribution of X1i , X2i ,..., Xci , or a combination can be used, i.e., fixing some values and 

averaging the others. 

5.2.2 Model Uncertainty and 

Model Averaging

Model averaging takes into account model uncertainty by incorporating results from all models into 

the estimation process through a weighted average 

of the model-specific excess risk estimates. This 

technique has been applied in a general modeling 

context by Raftery [1995], who suggested the use 

of the posterior model probabilities as weights derived from a Bayesian analysis of all models considered. Because a full Bayesian analysis is frequently 

computationally burdensome, Buckland et al. [1997] 

proposed simpler methods, where weights are 

based upon the penalized likelihood functions 

formed from the AIC and BIC [Schwarz 1978]. The 

AIC and the BIC are defined on likelihood functions where the AIC= ‒2 ln(L)+2p and the BIC= ‒2 

ln(L) + ln(n)p, where p is the number of parameters 

in the model, L is the maximized value of the likelihood function of the model, and n is the sample 

size (i.e., the number of observations). 

The NIOSH approach to model averaging is to 

use a model-averaged fit to synthesize risk estimates across multiple fitted parametric models. An estimate of the dose-response function 

 is calculated as a weighted average of K

model-specific dose-response estimates 

for k = 1, ..., K. Formally this is represented as 
46 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

 , where 

is the adverse effect, given the dose d using the kth

model, θk is the estimated parameter vector for the 

kth model, and wk represents the corresponding 

weight for the kth model. Given the model Mk in 

the model space that includes K models, the weight 

wk is:

 

where Ii represents the penalized information criterion described above (e.g., AIC or BIC). Other 

weighting mechanisms exist; for more information on these different strategies, see Morales et al. 

[2006] and Moon et al. [2005].

Recently, NIOSH collaborated with the EPA to 

develop extensions of the EPA’s Benchmark Dose 

Software (BMDS) Version 3.0 that includes model 

averaging [EPA 2018b]. This method is a Bayesian approach, which computes the model weights 

through the Laplace approximation [Raftery et al. 

[1997]. In almost all circumstances, results using 

this approach are qualitatively identical to (i.e., 

lead to the same risk estimates as) the approaches mentioned above. Given the ease of use of this 

software and methodological advances of this approach, NIOSH prefers its use; however, given the 

similarities of previous methods, other model averaging approaches might be used on a case basis. 

More information on EPA benchmark dose tools, 

including BMDS Version 3.0 and later versions, is 

available at https://www.epa.gov/bmds.

5.2.3 Nonparametric Modeling

The use of parametric models to describe the doseresponse relationship may not be necessary. Instead, a semiparametric or nonparametric curve 

can be used that allows for a more flexible approach 

of fitting data to a dose-response curve. The methodologies available to achieve this vary and often 

make the mild assumption of monotonicity with a 

possible smoothness constraint. 

Wheeler and Bailer [2012] describe a Bayesian semiparametric method that uses a flexible spline construction for dose-response analyses. This method 

was shown to be superior to the model averaging 

method of Wheeler and Bailer [2007] in terms of 

its statistical properties. The method is fully Bayesian, which requires attention to the specification of 

prior distributions, but it allows one to include prior 

information on such things as the incidence of the 

response in historical animal controls or in human 

reference populations. Even though this method is 

free of the model selection issues encountered in 

benchmark dose modeling, informed choices must 

still be addressed with this method. Its use requires 

the choice of spline basis functions located at specific knot locations, which should be selected before 

modeling begins. Ultimately, flexibility in the choice 

of these models comes at the expense of statistical 

and computational challenges in fitting such models. 

Other fully semiparametric/nonparametric modeling methodologies have been developed for dichotomous and continuous data [Guha et al. 2013; 

Lin et al. 2015; Piegorsch et al. 2012; Piegorsch et 

al. 2013; Wheeler et al. 2015], some of which overcome the known selection problems of Wheeler 

and Bailer [2007]. These methods are fully nonparametric in that they assume no prior form of the 

dose-response curve except monotonicity. Lin et al. 

[2015] showed that their continuous method would 

converge to the true underlying dose-response 

curve for large samples. The method of Wheeler 

et al. [2015] accounts for uncertainty in the specified response distribution for continuous outcomes 

and the dose-response. This method was shown 

through simulation to produce accurate estimates 

of excess risk, provided studies had adequate numbers of observations. Like model averaging, these 

methods allow for a flexible representation of the 

dose-response curve and are often preferable to a 

single parametric model fit. 

5.3 Point of Departure

The PoD is the point on the dose-response curve 

that is established from experimental or observational data generally corresponding to an estimated 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 47

level of no effect or a low effect level that is without 

significant extrapolation to lower doses. The PoD 

can be used in conjunction with uncertainty factors 

to derive a reference level of exposure or to mark 

the beginning of low-dose extrapolation to dose 

points associated with a target risk level. Extrapolation or the use of uncertainty factors is typically 

necessary when there is instability in model-based 

estimation at very low doses. These PoD concepts 

have their origins and continue to be widely used 

in animal toxicologic studies; therefore, much of 

the discussion on PoD metrics is in the context of 

methods using animal bioassay data. Nevertheless, 

these methods are adaptable to epidemiologic data 

[Bailer et al. 1997; Budtz-Jørgensen et al. 2001; Noble et al. 2009]. 

Three definitions of the PoD are commonly used 

in NIOSH risk assessments: (1) the No Observed 

Adverse Effect Level (NOAEL), (2) the Lowest Observed Adverse Effect Level (LOAEL), and (3) the 

Benchmark Dose (BMD), which are described in the 

following sections. Of these prospective PoDs, only 

the BMD approach requires fitting mathematical 

models to data, as described in the previous section. 

Thus, issues regarding model selection are restricted to the BMD approach. It is also noteworthy that 

the description herein is in the context of a single 

endpoint. Risk characterization may require consideration of multiple endpoints; therefore, additional 

methods may be necessary for comparing, contrasting, or combining information from dose-response 

analyses to describe the occupational risk. Figure 5-1

shows the relationship between the various PoD definitions, using a hypothetical dose-response curve. 

5.3.1 NOAEL/ LOAEL-Based 

Assessments

The NOAEL is defined as the highest dose level at 

which there are no significant increases in the frequency or severity of adverse effects between the exposed population and its appropriate control; some 

effects may be produced at this dose level, but they 

are not considered adverse or precursors of adverse 

effects observed [EPA 2012b]. For example, given 

a rank order series of exposure groups in an experimental study, the NOAEL is the administered 

dose level in the exposure group that immediately 

precedes the first exposure group in which the frequency of the observed adverse effect significantly differs from that in the control (no exposure) 

group. Similarly, the LOAEL is the lowest dose level 

or concentration at which there are significant increases in frequency or severity of adverse effects 

between the exposed population and its appropriate control group [EPA 2012b]. 

Usually, statistical hypothesis tests with a significance level of 5% (one-sided) are used to identify 

NOAELs or LOAELs. As such, problems arise in 

studies with few subjects observed at low exposure 

levels, resulting in insufficient signal-to-noise ratios 

and statistical power. It has been shown in most animal studies that the highest exposure group qualifying as a NOAEL is, on average, equivalent to a 

model-based BMD using a BMR of 10%, which is 

empirical evidence that many NOAELs were associated with an increased response that did not meet 

the standard significance level of 0.05 [Wignall et 

al. 2014]. Other limitations in using a NOAEL/

LOAEL approach are that it does not allow for consideration of the shape of the dose-response curve, 

which would inform estimation at lower levels, 

and that it is constrained to be one of the levels of 

exposure selected in the experiment. In addition, 

the spacing of exposures in an experiment can result in only high doses having sufficient power to 

detect statistically significant differences from the 

background condition [Crump 1984], even though 

biologically significant effects at lower doses may 

have been missed due to limited statistical power or 

sampling error. Hence, basing an interpretation of 

a NOAEL as representing a threshold below which 

effects are null is generally unfounded. Despite these 

limitations, the NOAEL/LOAEL approach may be the 

only alternative for determining a PoD for application of uncertainty factors when data are insufficient 

to model the dose-response adequately. NIOSH has 

used a NOAEL/LOAEL approach in assessing risks 

of occupational exposures to ethylene glycol ethyl 

ethers and associated chemicals [NIOSH 1991].
48 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

5.3.2 The Benchmark Dose Approach

Given the limitations of the NOAEL/LOAEL approach, the risk assessment community has widely 

adopted the BMD approach originally proposed 

by Crump [1984] for determining the PoD when 

observed data are adequate to model the dose-response (see Section 5.5). The BMD is defined as the 

dose or concentration on the dose-response curve 

that produces a predetermined change in the response rate of an adverse effect relative to the background response rate of this effect. This predetermined change is called a benchmark response, or 

BMR. A default level is not specified, because the 

choice of the BMR should be based on biological 

and statistical considerations that are specific to 

each risk assessment. In general, the BMR should 

be biologically reasonable and should be supported 

by the observed data (i.e., the PoD does not require significant extrapolation). For quantal data, 

the BMR is usually in the range of 5%–10% extra 

risk (see Section 6.1) for animal toxicologic data, 

which is the limit of responses typically observed in 

well-conducted animal experiments [EPA 2012b]. 

Much smaller values (e.g., ≤ 1%) are typically supported by quantal human data. Ideally, the BMR for 

continuous data represents the smallest change in 

Figure 5-1. A hypothetical dose-response association from an animal study with five dose groups. The lower 

limit on the benchmark dose or concentration (BMDL/BMCL) is selected as the point of departure (PoD). Other 

potential PoDs are the No Observed Adverse Effect Level (NOAEL) and the Lowest Observed Adverse Effect Level 

(LOAEL). The figure is adapted from EPA [2010].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 49

 

Figure 5-2. The Benchmark Dose (BMD) method, selecting a single parametric model. Adapted from Davis 

et al. [2011].

Abbreviations: AIC, Akaike information criterion; BMR, benchmark response; WoE, weight of evidence on toxicity.

response that is considered biologically significant; 

however, change points greater than this level may 

be necessary because of statistical limitations. Conversely, a statistically significant change may not indicate an adverse effect when low variability in the 

data is observed [Haber et al. 2018]. 

Given a BMR value that is selected a priori, the risk 

assessor fits various dose-response models to the 

observed data. This approach is applicable to dichotomous, ordinal, or continuous response data 

and categorical or continuous exposure data [Chen 

and Chen 2014; Crump 1984; Crump 1995]. For 

continuous response data, the BMR is usually based 

on a central measure of the biological effect (e.g., 

mean organ weight), a measure of its variability (e.g., 

standard error), and the number of observations at 

each dose level [Davis et al. 2011]. Regression models are fit to data that should include at least two 

dose groups above the control and in the low-dose 

range of interest (e.g., in the range of the BMR). The 

resulting curve(s) are used to calculate the BMD and 

its one-sided lower 95% confidence limit (BMDL). 

The BMDL is typically used to define the PoD. This 

process accounts for the variability and uncertainty in the experimental results, but not uncertainty 

in model selection [Davis et al. 2011]. The general 

BMD approach is illustrated in Figure 5-2. 
50 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

This BMD approach is preferred by the EPA [2012b] 

and NIOSH, who have collaborated to develop 

BMDS Version 3.0, which is readily accessible to 

risk assessors worldwide [EPA 2018b]. Specifically, the EPA software allows for the examination 

of a suite of dose-response functions for selection 

of the best single dose-response model using both 

frequentist and Bayesian methods. Additionally, 

BMDS Version 3.0 included Bayesian model averaging techniques to account for model uncertainty. 

Currently, model averaging is available only for dichotomous response models. The remainder of this 

section refers to single models. Model averaging 

was described previously in Section 5.2.2.

Model fitting is achieved by maximum likelihood 

(traditional frequentist modeling approach) or using a Bayesian approach. The adequacy of models is 

judged by goodness of fit (i.e., testing for lack of fit), 

typically using a critical value of 0.1 as a threshold 

for acceptance. Selecting the best-fit model from a 

set of adequately fitting nested models can be accomplished, in part, by likelihood ratio tests. Similarly, selecting a model from a set of models that are 

not hierarchically nested can be achieved by comparing AIC or BIC values. NIOSH risk assessors examine the variability in BMD or BMDL estimates 

across adequately fitting models. Reasonable agreement in estimates among a set of models suggests 

little model dependence; therefore, selection based 

on the lowest AIC value is well supported. Conversely, divergence in model estimates is indicative 

of model dependence. The EPA guidance does not 

explicitly define “divergence”; however, it has been 

suggested by others that BMDL values within a factor of 3 are sufficiently close [Haber et al. 2018]. 

It is important to note that model dependence results from extrapolation; therefore, it is more likely 

to occur when the BMR value is below the observable range. It is prudent to examine adequately fitted models closely to determine if the variability is 

attributable to anomalies in the data. Risk assessors 

may reject models that do not adequately describe 

the low-dose portion of the dose-response relationship, as determined by examining residuals and 

model plots. When the group of adequately fitted 

models is divergent and in lieu of other evidence 

supporting model rejection, a health-protective 

approach is to select the model that provides the 

lowest BMDL estimate [EPA 2012b]. In practice, 

however, selecting the lowest BMDL can be suboptimal in some cases, such as when the BMD confidence intervals are inordinately wide, suggesting 

that the true BMD might be much greater than the 

BMDL. Ultimately, the decision on model selection 

lies solely with the risk assessor, who must consider 

all available information and document the decision. Options for single-model selection include 

summary estimates from multiple models, such as 

in model averaging analyses (Section 5.2.2) or the 

use of semiparametric or nonparametric models 

(Section 5.2.3). 

Ideally, risk assessors directly estimate the doseresponse and its associated uncertainty within the 

range of the observed data. Still, frequently the 

PoD is determined at a higher response rate than 

a response of interest; therefore, extrapolation toward the origin of the dose-response curve may be 

required. For example, a PoD based on a BMR of 

10% excess risk of cancer is likely to require extrapolation to a much lower dose-risk region of interest to support a suitable estimate of lifetime risk. 

In animal toxicology, the common practice of (1) 

setting the BMR at 10% extra risk, (2) using the 

BMDL as the PoD, and (3) linearly extrapolating to 

the risk level of interest is well supported by studies 

showing that the BMD is often in the range of the 

NOAEL [Sand et al. 2011; Wignall et al. 2014]. For 

example, NIOSH typically uses linear extrapolation for cancer risk assessments unless mechanistic 

or mode of action (MoA) data support a different 

approach (Section 6.2.1). In cases in which data 

support a nonlinear dose-response, then low-dose 

extrapolation is accomplished via the selected parametric dose-response curve or by model averaging, semiparametric, or nonparametric methods. 

When extrapolation below the range of the data is 

necessary, risk estimates are of unknown validity. 

Model goodness-of-fit does not address this problem. Mechanistic data or information from biologic 

models (e.g., PBPK models) with well-understood 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 51

low-dose behaviors may be necessary to support 

low-dose risk estimates.

NIOSH used the BMD approach in its risk assessment of occupational exposures to carbon nanotubes and nanofibers [NIOSH 2013b]. The dataset 

was abstracted from short-term and subchronic 

studies of nonmalignant pulmonary responses in 

exposed rats and mice. Both quantal and continuous response data were examined. The BMR was set 

at 10% added risk of early stage adverse lung effects. 

The one-sided 95% BMDL was selected as the PoD. 

Modeling was conducted with the EPA benchmark 

modeling software. Although several models were 

specified, only a multistage (polynomial degree 2) 

model adequately fit quantal response data used in 

this risk assessment. The continuous dose-response 

data were fit with a second order polynomial model 

for all data with three or more dose groups, and a 

linear model for data with two groups. As a departure from traditional methods, model goodness-offit was considered adequate at P > 0.05. The authors 

explained the choice as “... a trade-off in the type 

I or type II error.” Nevertheless, all selected models (i.e., NIOSH 2013b, Tables A-3 to A-5) met P ≥ 

0.10. NIOSH also applied BMD concepts to epidemiologic data in its assessment of the risk associated with occupational diacetyl exposure [NIOSH 

2016]. Multiple BMRs describing pulmonary impairment were derived from continuous data on 

pulmonary function. 

5.3.2.1 Determining the PoD with Model 

Averaging and Semiparametric 

Methods

Linear extrapolation below the PoD is unnecessary 

when using model averaging, semiparametric, or 

nonparametric approaches [Wheeler and Bailer 

2012; Wheeler and Bailer 2007] because the estimation of exposures corresponding to small excess 

risks is model-based. The model-based extrapolation may result in a value similar to a linear extrapolation from a PoD unless substantial evidence 

against the latter is present in the data. For example, when applied to actual data and investigated 

in simulation studies, model-averaging and semiparametric approaches have adequately described 

both the model and statistical uncertainties at excess 

risk levels well below the 5% or 10% level. Wheeler 

and Bailer [2013] found that for dose-responses 

that were low-dose linear, these approaches yielded 

estimates that differed negligibly from a linear extrapolation from the 10% level for target risks as 

low as 0.001%. For nonlinear dose-response relationships, these methodologies were observed to 

provide superior estimates (i.e., BMDLs that maintained nominal coverage but were closer to the 

point estimate) than the PoD linear extrapolation 

while still accurately describing the risk. These two 

methodologies were also observed to be internally 

consistent in producing similar estimates, usually 

within a factor of three, across all excess risk levels 

examined.

Given limited information on the validity of extrapolations below 1% risk using model averaging in 

BMDS Version 3.0 [EPA 2018b], NIOSH recommends comparing the extrapolation to the semiparametric and previous model average approaches. 

If the results are similar, then it is reasonable to 

use the extrapolation using BMDS Version 3.0. If 

they are different by a factor greater than 3, then 

NIOSH computes the BMD at a BMR of 1% or 

above and linearly extrapolates below-risk levels 

less than 1%. 

In addition to examining single parametric models, NIOSH used a model-averaging method to 

summarize risk estimates from linear-quadratic, 

Weibull, and log-probit models in its risk assessment of lung cancer and titanium dioxide (TiO2) 

exposure [NIOSH 2011]. This approach was chosen because the dose-response relationship appeared nonlinear, and the specific models used in 

the three-model-average procedure did not impose low-dose linearity for risk extrapolation. In 

this model, Weibull and log-probit models were 

weighted more heavily than the linear-quadratic, 

which supported a dose-response that was sublinear at low doses (Figure 5-3).
52 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

5.4 Selecting a Dose-Response 

Modeling Method

The estimated dose-response curves from multiple biologically plausible models can differ substantially over a range of doses that can include 

the low dose region. Thus, in addition to biologic 

plausibility, the strength of the data and the statistical methodologies must be assessed to inform 

the choice on approaches to estimating risks and 

quantify relevant uncertainties. Misspecification of 

the model-form can lead to spurious dose-response 

estimates (see Appendix B). Moreover, although 

widely recognized as a fundamental component of 

statistical inference, the model selection process 

itself is seldom integrated into inference [Buckland et al. 1997]. Therefore, even if estimates are 

unbiased, precision (e.g., as indicated by the width 

of confidence intervals) is likely overestimated (e.g., 

confidence intervals are too narrow). In response, 

there are several statistical procedures available 

to assist in model specification and to account for 

model uncertainty (Appendix B). 

One strategy for accounting for model uncertainty 

is to use model averaging methods. These methods 

have been shown to be both robust and flexible. As 

stated previously, NIOSH generally prefers to use 

a Bayesian model averaging approach to address 

model-form misspecification and to account for 

model uncertainty in assessing dose-response (see 

Figure 5-3. BMD models and three-model-average fit to the lung tumor data (without squamous cell keratinizing 

cysts) in male and female rats chronically exposed to fine or ultrafine TiO2 [NIOSH 2011].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 53

Section 5.2.2). Still, every dose-response analysis 

is unique and requires careful consideration of 

the approach used. For example, possible exceptions to using nonparametric, semiparametric, and 

model averaging techniques are when (1) compelling mechanistic or statistical evidence supports a 

specific dose-response function and (2) data limitations require a simpler approach or a more parsimonious model. 

5.5 Laboratory Animal Data

The adequacy of the database to support dose-response analysis based on animal studies is an important consideration in occupational risk assessment. 

Risk assessors evaluate animal studies in the context 

of the risk assessment questions under investigation. 

Studies are identified that may shed light on the research question. For example, a single-dose acute 

toxicity study may not have much relevance for assessing chronic exposures to a chemical, but it may 

be useful for setting an immediately dangerous to 

life and health (IDLH) value or a short-term exposure limit (STEL), depending on the specifics of the 

study. Ideally, risk assessors evaluate all studies that 

may contain relevant information on the dose-response relationship. Each study is evaluated for adequacy of study design and conduct (duration of exposure, dosing regimen, species, numbers and sexes 

of animals, description of experimental conditions), 

health endpoints observed, statistical analyses conducted, and how the data support the conclusions 

of the study. Risk assessors give greater weight to 

studies reporting statistically and/or biologically significant dose-response associations. Alternatively, 

risk assessors may apply statistical methods to make 

joint inferences from multiple heterogeneous datasets [Triantafillou and Tsamardinos 2015; Vesterinen et al. 2014]. Relevance of the endpoint to human health, severity of the health endpoint, and the 

sensitivity of the health endpoint should be considered. Preferably, risk assessors clearly document the 

rationale for including/excluding or grouping studies, dose groups, or health endpoints from analysis 

in the risk assessment. 

For dose-response modeling of dichotomous response data, both the number of animals showing 

the effect in the group and the total number of subjects in the group are necessary, at a minimum. For 

modeling of continuous response data, individual 

animal data are strongly recommended, although 

information on central tendency and variability 

may be sufficient; typically, data on the number 

of subjects, mean of the response, and variability 

measure (e.g., standard deviation, standard error, 

or variance) for each group are adequate to perform the analysis. For dose-response modeling of 

categorical responses, the number of animals examined and the counts for each response category 

of each dose group are generally sufficient [EPA 

2012b; Hertzberg 1989].

When the dose-response analysis is conducted with 

benchmark dose analysis and/or model averaging, 

there are specific requirements for the datasets. In 

general, toxicologic animal studies with more than 

one dose group are required for dose-response 

analysis. Ideally, there are responses in more than 

one dose group that are different from background 

and different from the maximal response. Multiple 

intermediate responses of this type increase confidence that the study contains adequate information 

on the dose-response curve and does not represent 

only background or only maximal responses. It 

may be possible to calculate a BMD and BMDL 

with only a single dose showing a response near the 

BMR [Kavlock et al. 1996]. However, if the studies show responses in more than one dose group 

but all the responses are at the background level, 

are near the maximal response level, or appear as 

a very steep rise of the dose-response curve over 

a small range of doses, then the data may not be 

adequate for regression modeling. Thus, it is preferable to have studies with observed responses sufficient to provide a unique solution to the optimizing procedure. For the dichotomous data models, 

this usually requires at least two dose groups with 

responses intermediate between background and 

maximal. An additional advantage accrues from 

having at least one dose group near the BMR, to 

yield a better estimate of the BMD [EPA 2012b].
54 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Overall, the specific type of toxicity information 

required is dependent upon the question to be 

addressed and the interplay with human data. For 

occupational risk assessment, data from well-conducted chronic bioassays, preferably in more than 

one species (typically one in rats and one in mice), 

a two-generation reproductive study, and a developmental study in mammalian species would provide a reasonable database to reduce the uncertainty and increase the confidence in the risk estimates. 

A well-conducted subchronic study that evaluated 

a comprehensive array of endpoints could also be 

useful, especially in the absence of chronic bioassays, if significant differences in adverse effects are 

not expected from longer exposures. In most cases, 

NIOSH is concerned about chronic exposures to 

hazards, but in some cases, acute or intermediateduration hazards may be of concern. In other cases, 

data needs are endpoint-specific. For example, if 

acute or subchronic data demonstrate neurotoxic, 

immunotoxic, or cardiotoxic effects, then a neurotoxicity, immunotoxicity, or cardiotoxicity battery 

of tests could satisfy the data requirements [EPA 

1994; EPA 2002]. When the typical animal bioassays are not available, data from alternative testing systems such as high-throughput molecular 

toxicity assays and quantitative structure-activity 

relationship (QSAR) models could be used to inform the risk assessment and to fill the data gaps. 

NIOSH has not had extensive experience in using 

these types of data, so each use would be on a caseby-case basis. 

5.5.1 Parallelogram Approach

First introduced by Sobels [1977], the “parallelogram approach” (Figure 5-4) is an argument by 

analogy for inferring missing data when you have 

closely related data that is especially useful for 

cross-species extrapolation. It has been used in 

genotoxicity studies to predict human germ cell 

mutations from measured mouse germ cell mutations, mouse somatic cell mutations, and human 

somatic cell mutations [Anderson et al. 1994]. It 

has been used in PBPK studies to predict human in 

vivo metabolic parameters from measured mouse 

in vitro parameters, mouse in vivo parameters, and 

human in vitro parameters [Kienhuis et al. 2009]. 

NIOSH, in part, used this technique to assess comparative potency of the closely related chemicals 

diacetyl and 2,3-pentanedione, when both human 

toxicity and animal toxicity data were available for 

diacetyl but only animal toxicity data were available for 2,3-pentanedione [NIOSH 2016].

Figure 5-4. An example of the parallelogram approach.

The parallelogram approach is conceptually very 

simple but requires explicit assumptions. For example, to estimate the metabolic constants for a 

substance to use in a PBPK model, one must assume the following:

■ There is a constant and knowable relationship 

between metabolic constants measured in vitro and metabolic constants measured in vivo

within a species.

■ The relationship between in vivo and in vitro

metabolic constants is the same, regardless of 

species.

Therefore, once the ratio between mouse in vitro and 

in vivo metabolic constants has been measured and 

the human in vitro metabolic constant is known, 

the mouse ratio can be applied to the human in vitro constant to estimate the human in vivo metabolic 

constant.

Similarly, for genotoxicity studies, one must assume the following: 

■ There is a constant and knowable ratio between 

somatic mutations and germ cell mutations 

within a species.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 55

■ The relationship between somatic mutations 

and germ cell mutations is the same, regardless 

of species.

Therefore, once the ratio between mouse somatic 

mutations and mouse germ cell mutations has been 

measured and the human somatic mutations have 

been measured, the mouse ratio can be applied to 

the human somatic mutations to estimate the human germ cell mutations.

For the comparative potency example, NIOSH had 

mouse toxicity data and human epidemiology data 

on diacetyl. NIOSH was also interested in a closely 

related (1-carbon different) flavoring chemical, 

2,3-pentanedione. However, there were no human 

data on 2,3-pentanedione toxicity. In this case, 

NIOSH assumed the following:

■ There is a constant and knowable relationship 

between the lung toxicity in mice and the lung 

toxicity in humans for a chemical.

■ Di-alpha-ketones such as diacetyl and 2,3- 

pentanedione are related closely enough that 

they share toxic modes of action.

■ The relationship between lung toxicity and 

hazardous exposure in mice and humans is 

constant for these closely related chemicals.

Although NIOSH did not follow this logic through to 

predict human risk estimates for 2,3-pentanedione, 

the same logical structure applies. NIOSH stopped 

with an assessment that 2,3-pentanedione was in 

a similar range of toxicity as diacetyl and used the 

diacetyl risk assessment to set a recommended 

exposure limit for 2,3-pentanedione. In this case, 

the uncertainties in the method and the sparseness of the data argued for cautious application 

[NIOSH 2016]. 

Applying a parallelogram approach requires that 

the measured values used to construct the ratios 

reflect the same or very closely allied methods and 

data sources. This will not work if the technique 

or type of tissue used for mouse in vitro assays 

is substantially different from the human assays. 

The assumptions regarding which values in the 

parallelogram are similar should be carefully examined. The uncertainty in the parallelogram approach is lessened with cross-species validation. 

For example, similar measured ratios of in vitro

to in vivo metabolic parameters in mice, rats, and 

hamsters strengthens the argument that it is reasonable to extrapolate to humans. Depending on 

the data available, this step is not always possible. 

Finally, the parallelogram approach is a useful tool 

to consider when key data are unavailable, but it 

requires strong assumptions that must be closely 

examined and carefully justified.

5.6 Dosimetry Adjustments 

for Human Equivalent 

Concentrations

Ideally, occupational risk assessment is based on 

the biologically effective dose (e.g., tissue dose) that 

mediates the adverse response observed in workers. 

In practice, enough information to directly estimate 

the dose from exposures in humans is seldom available. Instead, dosimetry methods are commonly 

employed to estimate the dose delivered to target 

sites in laboratory animals, which are then adjusted 

for relevance in humans by accounting for the physiological differences between species [EPA 1994]. 

Herein, dosimetry refers broadly to methods relating exposure metrics to a biologically effective dose 

from the agent(s) of interest. Dosimetry models are 

the foundation of animal-to-human extrapolation.

In general, a critical dose observed in an animal 

study (i.e., an estimate of the biologically effective 

dose in the animal) is extrapolated to humans by 

using dosimetry modeling. This extrapolation is 

referred to as the human-equivalent concentration 

(HEC) for inhalation exposure or dose (HED) for 

other routes of exposure [EPA 1994; FDA 2005]. 

The HEC is an amount of the agent in humans 

that is believed to induce the same magnitude of 

toxic effect as the experimental animal species’ 

concentration or dose. Describing all aspects of 

dosimetry methods that may be used in NIOSH 

risk assessment is beyond the scope of this report. 
56 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

However, basic dosimetry modeling for estimating 

human risks from exposures to particulates, gases, 

and vapors is described in the following sections. 

The regions of the respiratory tract are also briefly 

described. 

5.6.1 Respiratory Tract Regions

The respiratory tract in both humans and laboratory experimental animals is divided into three 

regions based on structure, size, and function: the 

extrathoracic (ET) region extends from nose to 

larynx, the tracheobronchial (TB) region extends 

from trachea to the terminal bronchioles, and the 

pulmonary (PU) region includes the respiratory 

bronchioles, alveolar sacs, alveolar ducts, and alveoli (Figure 5-5). The pulmonary region is where 

gas exchange occurs (i.e., uptake of oxygen and release of carbon dioxide). Diseases of the respiratory tract have been associated with exposure to substances that deposit in each of these regions. These 

regions also correspond to the inhalable, thoracic, 

and respirable particle size fractions for airborne 

sampling [ACGIH 2015].

5.6.2 Particle Exposure

5.6.2.1 Overview

In quantitative risk assessment of inhaled particles, 

the dose from particles deposited or retained in the 

respiratory tract region is estimated by dosimetry 

modeling methods. Models for particulate exposures must account for interspecies differences in 

the factors that determine the deposition, clearance, and retention of particles (spherical or nonspherical) from the respiratory tract [EPA 1994; 

Kuempel et al. 2015]. Dose estimation is one of the 

major sources of uncertainty in a risk assessment 

(e.g., as discussed for carbon nanotubes in NIOSH 

[2013b], Section A.6.3). Use of validated dosimetry 

models reduces the uncertainty in extrapolating 

animal data to humans. 

 

Figure 5-5. Human respiratory tract regions [EPA 1994].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 57

To estimate a human-equivalent internal dose or 

exposure concentration of particles by using animal data, the main dosimetry method options are 

(1) application of uncertainty factors (see Section 

6.3.4), (2) general/categorical adjustments (e.g., 

EPA “Regional Deposited Dose Ratio” in respiratory tract), and (3) substance-specific PBPK models 

(e.g., to account for particle dissolution and translocation beyond the respiratory tract). Application 

of uncertainty factors (UFs) is simpler and requires 

less information but is also associated with greater 

uncertainty. Other methods generally require additional information and involve complex analysis 

but may provide more accurate dose estimates for 

the risk assessment. 

5.6.2.2 Deposition Mechanisms

Particle size is a key factor in estimating the deposited doses in the respiratory tract region. Standard 

definitions of airborne particle size fractions include inhalable, thoracic, and respirable [ACGIH 

2015]. Inhalable particles are those capable of entering the nose or mouth and depositing anywhere 

in the respiratory tract. For example, particles with 

an aerodynamic diameter of 100 μm have an approximately 50% probability of being inhaled and 

deposited in the respiratory tract. The extrathoracic fraction is the mass fraction of inhaled particles 

with low probability of penetrating beyond the larynx. The thoracic fraction refers to the mass fraction of inhaled particles capable of reaching beyond 

the larynx into the thoracic region and depositing 

in the conducting airways. The respirable fraction 

is the mass fraction of inhaled particles that is capable of reaching and depositing in the gas-exchange 

region of the lungs [Brown et al. 2013]. 

Aerodynamic equivalent diameter is defined as 

the diameter of a sphere of standard density of 

one gram per cubic centimeter (1.0 g/cm3

), having the same terminal velocity when settling under 

gravity as the particle under consideration [Hinds 

1999]. Diffusion equivalent diameter is defined as 

the diameter of a sphere with the same thermal or 

Brownian diffusivity as the particle under consideration [Hinds 1999]. For nonspherical particles 

such as fibers, shape and orientation are additional 

factors that can influence deposition [Sturm and 

Hofmann 2009]. Dosimetry models can provide 

dose estimates based on different metrics (e.g., the 

total particle mass, volume, or surface area in specific respiratory tract regions), which can be used 

in extrapolating dose-response relationships across 

species [Asgharian et al. 2018].

Respiratory tract deposition models can take into 

account these particle properties to predict the 

deposited dose in each region. In addition to the 

particle properties, lung morphology can influence 

particle deposition. Differences in airway structure, 

lung volume, and breathing patterns (e.g., nasal only 

or oronasal) have been observed among individuals 

and are also related to age, gender, and race [Schulz 

et al. 2000]. Some deposition models account for interindividual variability in lung morphology [ARA 

2009; ICRP 1994]. Activity level (e.g., resting or exercising) influences the ventilation rate and breathing pattern, thereby affecting particle deposition in 

the respiratory tract. 

At a minimum, data are generally available to estimate the deposited dose of particles in a respiratory 

tract region of humans or animals, given the exposure concentration, duration, and airborne particle 

size estimates. Examples of these basic methods 

and information sources are discussed next. 

5.6.2.3 Ventilation Rates and Activity Levels 

5.6.2.3.1 Humans

NIOSH generally uses the International Commission of Radiological Protection (ICRP) standard reference value for workers for the total air 

intake (volume inhaled), which is 9.6 m3 in an 

8-hour workday [ICRP 1994]. This total air intake is equivalent to an average minute ventilation rate of 20 liters of air per minute (L/min) 

(i.e., 9.6 m3

 = 20 L/min × 480 min × 0.001 m3/L). 

These reference values are based on adult males, 

assuming 5.5 hours of light exercise and 2.5 hours 

of rest/sitting. The adult male minute ventilation 

rates are 25 L/min for light exercise and 9 L/min 

for resting (sitting). 
58 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Thus, the total air intake in an 8-hour workday in 

men is calculated as follows:

Equation 5-16.

9.6 m3

= [(5.5 hours × 60 minutes per hour)

 × 25 L⁄min]+[(2.5 hours × 60 minutes 

per hour) × 9 L⁄min] ÷ [1,000 L⁄m3

 ]

Minute ventilation (VE , in L/min) is calculated as 

the product of the tidal volume (VT , in L) and the 

breathing (respiratory) frequency (ƒ, in min-1): 

Equation 5-17.

VE = VT × ƒ 

Tidal volume is the volume of air inspired or expired in each respiratory cycle, and the respiratory 

frequency is usually described in breaths per minute [EPA 1994]. These respiratory values (tidal volume and breathing frequency) vary by age, gender, 

and activity level [ICRP 1994]. For example, VE of 

25 L/min (as used in Equation 5-16) is calculated 

from VT of 1.25 L and f of 20 min-1 (as shown in Table 

8 of ICRP [1994]). For adult female workers, the 

average air intake is 8.2 m3

 in an 8-hour workday, 

assuming the same activity levels and using the 

gender-specific values for VT and ƒ in ICRP [1994]. 

For dosimetry modeling, these respiratory values 

are used to estimate deposited dose, given the 

exposure scenario. In the Multiple-Path Particle 

Dosimetry (MPPD) model, v.3.04 [ARA 2015], the 

breathing frequency and tidal volume are required 

input values. For example, for a resting adult male 

human, values of VT and f have been reported as 

625 milliliters (ml) and 12 min-1 ICRP [ICRP 1994]. 

For workers, NIOSH [2011] used the values of 1,143 

ml for VT and 17.5 min-1 for f, which are weighted 

averages of the respiratory values that correspond 

to the average male worker reference values of 20 

L/min (VE) and 9.6 m3

 (total volume inhaled) in an 

8-hour workday, as described above.

5.6.2.3.2 Animals

Species-specific ventilation rates are required for 

estimating the deposited dose of airborne particles 

in the respiratory tract of animals. When experimental ventilation rates are not available, speciesspecific average ventilation rates can be calculated 

with the following allometric scaling equation:

Equation 5-18. 

ln(VE) = b0 + b1 × ln (BW)

where VE is the minute ventilation (L/min) as 

described previously; BW is body weight in kilograms (kg), and b0 and b1 are the species-specific 

parameters (e.g., as reported in EPA [1994]). For 

example, estimates of b0 + b1 are -0.578 and 0.821, 

respectively, in rats (in Table 4-6 of EPA [1994]). 

For a 300-g rat, VE is calculated from equation 5-19 

as follows:

Equation 5-19.

0.21 L ⁄ min = exp[–0.578 + 0.821 × ln (0.3)] 

This VE corresponds to the value estimated from 

the breathing parameters for tidal volume of 0.21 

ml and breathing frequency of 102 breaths/min, 

which have been used in estimating lung dose in 

rats with body weight of 300 g [NIOSH 2013b] as 

shown in equation 5-20: 

Equation 5-20.

0.21(L ⁄ (min) = 2.1(ml) × 102 (min-1 ) × 

0.001 (L ⁄ ml)

5.6.2.4 Deposited Dose Calculation

The deposited dose of inhaled particles in the respiratory tract region is a biologically relevant estimate of equivalent dose in humans or animals. 

Equivalent dose metrics are needed to extrapolate 

dose-response relationships and risk estimates 

from animals to assess human risk. 

The deposited lung dose can be estimated as follows:

Equation 5-21.

Deposited lung dose (mg) 

= exposure concentration (mg⁄m3

) × duration in 

hours (hours per day × days per week × weeks 

exposed) × ventilation (L⁄min) × 0.001 m3

⁄ L × 60 minutes per hour × regional deposition fraction
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 59

Exposure concentration and duration would be as 

reported in the animal or human study, or for the 

exposure scenario of interest. Minute ventilation is 

calculated as shown in Section 5.6.2.3. The regional 

deposition fraction of interest is estimated for the 

respiratory tract region associated with the adverse 

effect in the risk assessment. The regional deposition fraction is estimated from the airborne particle diameter, and these values have been measured 

in various particle sizes, including in a study of several small laboratory animals [Raabe et al. 1988]. 

The deposition fraction can also be estimated in 

MPPD [ARA 2015] for several species and strains 

(human, rat [Sprague-Dawley and Long-Evans], 

mouse [BALB/c and B6C3F1], rhesus monkey, pig, 

or rabbit). Airborne particle size (mean ± SD) and 

density are required input values in MPPD. 

For example, to estimate the deposited lung dose 

in a rat subchronic (13-week) inhalation study at 

a pulmonary effect level of 5 mg/m3

, the exposure 

concentration and duration (in Equation 5-21) 

are as reported. The minute ventilation is calculated as shown in equation 5-19, and the pulmonary deposition fraction is estimated in MPPD. 

Typically, the particle size data are also reported. 

For simplicity, assuming particle mass median 

aerodynamic diameter (MMAD) of 1 μm, monodisperse (geometric standard deviation of 1), and 

unit density (1g/cm3

), a rat pulmonary deposition fraction of approximately 0.06 is estimated 

in MPPD [ARA 2015]. The deposited lung dose 

(Equation 5-21) in this example in rats is calculated 

as follows:

Equation 5-22.

1.5 mg = 5 mg⁄m3

 × (6 hours per day × 5 days per 

week × 13 weeks) × (0.21 L⁄min × 

 0.001 m3

 ⁄ L × 60 minutes per hour) × 0.06

If lung doses are not reported in a rodent study, then 

the deposited dose can be estimated by this method. 

The worker-equivalent airborne concentration can 

then be estimated by back-calculating to determine 

the airborne concentration that would result in the 

equivalent pulmonary-deposited dose in humans 

(Figure 5-6). More biologically relevant dose estimates may also take into account the clearance of 

particles by respiratory tract region to estimate the 

retained dose over time, as discussed below.

5.6.2.5 Biokinetic Mechanisms and Models 

of Inhaled Particles

5.6.2.5.1 Clearance, Retention, and Translocation

The biological mechanisms of particle clearance 

depend on the respiratory tract region in which 

the particles deposit and on the physicochemical 

properties of the particles. Particles that deposit in 

the TB region are cleared mainly by mucociliary 

transport, in which particles or other exogenous 

materials are transported toward the mouth, where 

they are swallowed or expectorated. Mucociliary 

transport occurs from the terminal bronchioles to 

the larynx. Insoluble particles that deposit in the 

TB region are generally cleared relatively rapidly, 

with biological half-times on the order of hours. 

Soluble particles may dissolve in the mucus [EPA 

1994]. Particles that deposit in the pulmonary region are cleared primarily by alveolar macrophages 

that phagocytose (engulf) particles, where they are 

dissolved or transported to the TB region for mucociliary clearance [Schlesinger 1985]. Clearance of 

poorly soluble particles can differ across species because of differences in the rates of mucociliary transport in the conducting airways and macrophagemediated clearance from the alveolar region [Miller 

2000; Snipes 1989]. Pulmonary clearance is approximately 10 times slower in humans than in rats, based 

on first-order clearance assumptions [Snipes 1989]. 

Retention is described as the temporal distribution of uncleared particles in the respiratory tract 

[Lioy et al. 1984]. In humans, two distinct phases of 

particle retention occur. The first phase is thought 

to represent mucociliary clearance of particles depositing in the TB region and is complete within 

approximately 24 hours, although a particle size–

dependent slow clearance fraction has also been 

demonstrated [ICRP 1994; Stahlhofen et al. 1989]. 

The second phase, which involves retention halftimes from approximately 30 to several hundred 
60 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Figure 5-6. Dosimetry and risk assessment steps used by NIOSH to develop occupational exposure limits for 

airborne particles extrapolated from animal data (adapted from Kuempel [2011] and Oberdörster [1989]).

days or longer, may represent particle clearance 

within the alveoli (air sacs) and interstitium (connective tissue separating the alveoli) of the pulmonary region.

Particles or fibers retained in the lungs can move 

into the lung interstitial tissue (either alone or inside macrophages). Particle retention in the interstitium increases the risk of fibrosis for poorly soluble particles. Translocation of particles from the 

lungs to the lung-associated tissues and systemic 

organs has also been reported, for particles from 

coal dust to carbon nanotubes [LeFevre et al. 1982; 

Mercer et al. 2013]. The physicochemical properties that influence the clearance or retention of particles from the respiratory tract include the chemical composition, size, surface properties, solubility, 

and shape [Kreyling et al. 2013]. 

5.6.2.5.2 Models of Long-term Particle Retention in Humans

Studies in workers have shown that the long-term 

retention of respirable particles involves the sequestration of some portion of the dust in the lungs, even 

at low exposures that would be below overloading 

in rats [Gregoratto et al. 2010; Kuempel et al. 2001]. 

These independent studies include workers exposed 

to particles from relatively low (radioactive cobalt) 

to high (coal dust) mass concentrations. The human pulmonary clearance and retention models 

that include an interstitial sequestration compartment have been shown to provide better prediction 

of long-term retained lung burdens in humans with 

either low or high dust exposures compared to models with either simple first-order clearance or dosedependent overloading (first-order clearance until 

reaching a critical dose associated with decreasing 

clearance rate) [Gregoratto et al. 2010; Kuempel and 

Tran 2002; Kuempel 2000; Kuempel et al. 2001; Tran 

and Buchanan 2000]. Consistent with these findings, 

a study comparing rat and human particle retention 

patterns in the lungs showed that coal miners retained a greater proportion of particles in the alveolar interstitial tissue, whereas rats retained a greater 

proportion of particles in the alveolar spaces [Nikula et al. 2001]. 

In a dosimetry model of the respiratory tract, the 

ICRP [1994] included three first-order pulmonary 

(alveolar-interstitial, or AI) clearance compartments. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 61

A fixed proportion of respirable particle deposition 

in the alveolar region is assigned to each compartment (i.e., 30%, 60%, and 10% for AI1 , AI2 , and 

AI3 , respectively). The first-order clearance rate 

coefficients are 0.02, 0.001, and 0.0001 day-1, corresponding to retention half-times of 34, 693, and 

6,930 days, respectively. More recently, the ICRP 

published an updated model [Paquet et al. 2015]. 

Regarding long-term particle clearance, the main 

difference between models is a much higher particle retention fraction in the pulmonary region in 

the updated model.

In the studies reported in ICRP [1994], the particle 

lung clearance in humans was quantified for up to 

approximately one year after inhalation. Based on 

those studies, ICRP [1994] estimated that 50% of 

the alveolar deposited particle dose remains at 300 

days following inhalation. Studies published since 

that report show much higher long-term particle 

retention in the lungs of humans. For example, in 

workers accidentally exposed to radioactive particles, “a significant fraction (>10%)” of particles 

was retained in the thorax at 10 years after the acute 

exposure (Paquet et al. [2015], Section A.2.3). An 

interstitial sequestration compartment was found 

to be necessary to adequately fit the particle retention data in coal miners [Kuempel et al. 2001]. This 

model structure is more biologically reasonable, 

including an alveolar compartment with clearance 

to the bronchial region, and an interstitial compartment that clears slowly to the lymph nodes. 

The kinetics of particle clearance in human lungs 

differs from that in rodents. In rodents, first-order 

clearance is faster and the translocation of particles 

to the interstitium occurs primarily after overloading of alveolar clearance. In humans, a fraction of 

the respirable particles that deposit in the alveolar 

region may not be cleared but may enter the interstitium and be retained for a longer duration. These 

findings were confirmed in additional studies in 

humans, including at much lower particle exposures [Gregoratto et al. 2010; Paquet et al. 2015]. 

Gregoratto et al. [2010] applied the Kuempel et al. 

[2001] model to the more recent data in humans, 

as well as to the original data that had been used 

in ICRP [1994]. The parameter values fitted to 

the pooled data resulted in an alveolar clearance 

half-time estimate of approximately 250 days, with 

approximately 33% of the initial alveolar deposited 

mass dose of insoluble particles being sequestered 

in the interstitium [Paquet et al. 2015].

The MPPD human clearance and retention model 

(including v. 1.0 to current v. 3.04) [ARA 2015; 

Price et al. 2002] uses the ICRP [1994] model to 

predict clearance and retention in humans. Consistent with the updated ICRP model, higher worker 

lung burdens were estimated in the interstitial sequestration model [Kuempel et al. 2001] than in 

the MPPD, v. 1.0 [Price et al. 2002], as reported in 

Dankovic et al. [2007]. Thus, the earlier ICRP model (or models using it as their basis) may underpredict the average long-term particle retention in 

humans and therefore may underestimate the risk 

of adverse effects associated with retained particle 

dose in the lungs. 

5.6.2.6 PBPK Models to Estimate Dose

Physiologically-based pharmacokinetic (PBPK) 

models provide mathematical estimates used to 

simulate the distribution of chemicals and metabolites in the body after exposure to a substance over 

time. These models incorporate the use of a series 

of differential equations to account for physiological parameters such as blood flow and volume, and 

metabolic capacity of discrete tissue types (e.g., lung, 

richly perfused tissues, poorly perfused tissues, and 

liver). Many of these parameters were compiled for 

several animal species by Brown et al. [1997]. Other 

factors such as dosing route are also considered for 

the physiological differences in uptake rate, metabolism, and distribution of chemicals that occur when 

exposures are by differing routes. 

Some examples of PBPK models that have been 

used in NIOSH risk assessments include respiratory tract models to estimate the particle deposition 

and clearance kinetics for inhaled particles, such as 

titanium dioxide and carbon nanotubes [NIOSH 

2011; NIOSH 2013b]. PBPK or dosimetry models 

are used to estimate more biologically relevant doses 
62 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

for use in dose-response modeling in quantitative 

risk assessment. These models can be useful for 

temporal extrapolation of dose when the relationship between the external exposure and internal 

dose is nonlinear. This is because PBPK models 

can account for capacity-limited processes in the 

absorption/uptake, distribution, metabolism, and/

or excretion of a toxicant. Capacity limitation may 

be due to saturation of a key process, e.g., involving a receptor or enzyme. An example of an important capacity-limited process that influences 

the clearance kinetics and dose of inhaled particles 

in rodents is the overloading of pulmonary (alveolar macrophage-mediated) clearance of particles, 

which results in a dose-dependent increase in the 

retained particle dose [Bellmann et al. 1991; Bolton 

et al. 1983; Morrow 1988; Stöber et al. 1990]. As 

a result, the biological mechanisms and pathways 

operating at lower, non-overloading doses can differ from those operating at higher doses, when 

defenses of cells or organisms are overwhelmed 

[McClellan 1997; Oberdörster et al. 2005b]. In this 

case, a dosimetry model that does not account for 

changes in clearance kinetics may provide a poor 

estimate of dose and underestimate the retained 

lung dose at higher airborne exposures [Kuempel 

et al. 2015]. 

Additionally, PBPK models provide mathematical 

modeling estimates of aggregate exposure dosing 

(oral, inhalation, dermal) and target tissue dose estimates of both parent chemical exposure and metabolites. This information may reduce uncertainty 

in risk assessment calculations compared to those 

using dose estimates based on absorption only. 

In addition to the use of PBPK models for particle 

dose estimation, these models are also applicable 

to dose estimation for other chemical forms, including gases and vapors (see Section 5.6.3.6). In 

an example involving a liquid, a PBPK model for 

dichloromethane was used to illustrate the steps 

to estimate tissue dose in chemical risk assessments [Andersen et al. 2005]: 1) identify toxic 

and critical effects in animals and humans; 2) 

catalogue MoAs, metabolism, parent compound/

metabolite data; 3) identify potential MoA for the 

critical effect; 4) develop potential relationship 

between tissue dose (associated with toxicity) and 

the response; 5) develop the PBPK model based 

on relevant routes of exposures, exposure concentrations, and species; 6) provide estimates of dose 

metrics during exposures that produce toxicity; 

and 7) use outputs to estimate risk for human exposures of interest with the assumption of a similar 

dose-response relationship between PBPK test species and humans. 

Although risk assessment methods may be advanced using PBPK models, there are important 

limitations to consider. PBPK models often require data intensive inputs on physiological and 

biochemical/metabolic processes that often come 

from separate sources or may be missing or poorly 

characterized [Khalil and Laer 2011]. In addition, 

validation of PBPK model outputs using empirical data can be difficult or impractical given data 

limitations. Lastly, PBPK modeling efforts typically 

require specialized software and expertise that may 

have limited availability in some situations.

Detailed guidance on developing PBPK and/or 

computational fluid-dynamics models is beyond 

the scope of this report. Additional information on 

these topics is available in Andersen et al. [2005] 

and in guidance provided by the U.S. EPA: “Approaches for the Application of Physiologically 

Based Pharmacokinetic (PBPK) Models and Supporting Data in Risk Assessment” [EPA 2006a].

5.6.2.7 Overloading Considerations in 

Rodent Model and Dose Estimation

The effects of particle overloading of lung clearance in rats and mice involve a sequence of events 

including persistent pulmonary inflammation in 

both species, fibrosis primarily in rats, and cancer 

in rats [Baan 2007; Elder et al. 2005; Oberdörster 

1995]. Rats have been shown to be better predictors 

of lung cancer from inhaled particles that are carcinogenic to humans (i.e., classified by IARC as having limited or sufficient evidence), in comparison 

with mice or hamsters, which yield false-negative 

results more often [Mauderly 1997]. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 63

This well-studied rodent phenomenon of particle 

overloading of pulmonary clearance is the basis for 

the risk assessment approach of identifying the 

non-overloading dose in rats as the NOAEL to 

extrapolate to humans [Morrow et al. 1991; Pauluhn 2010]. Although this concept seems reasonable on the basis of the rat data, it may not be adequate to estimate chronic responses in humans 

because of differences in the clearance and retention kinetics (as discussed in Section 5.6.2.8). 

The dose metrics associated with overloading of 

lung clearance include particle mass (unit density 

particles), particle volume (particles with density other than 1.0 g/cm3

), and particle surface area 

(nanoparticles) [Bellmann et al. 1991; Morrow 

1988; Tran et al. 2000]. In contrast to microscale 

particles, nanoscale particles or highly toxic particles have been shown to cause impaired pulmonary clearance at a lower mass or volumetric particle dose than for microscale poorly soluble low 

toxicity (PSLT) particles [Bellmann et al. 1991; 

Oberdörster et al. 1994]. Particle surface area has 

been shown to better describe the decreased clearance and pulmonary responses to nanoscale compared to microscale particles [Tran et al. 2000]. 

Since particle dosimetry models are generally 

based on the particle mass, dose conversion may 

be necessary between the estimation of effect level 

in the rodent study (e.g., surface area dose associated with adverse effect) and the estimation of the 

equivalent dose in humans. 

5.6.2.8 Interspecies Dose Estimation in 

Risk Assessment

Scientific models are generally available to estimate the human-equivalent lung doses of inhaled 

particles from those in rodents [ARA 2015; Paquet 

et al. 2015]. Less well understood are the human 

and rat biological responses to equivalent mass, 

surface area, or volumetric particle lung doses. For 

example, the biological MoA for the development 

of lung tumors in rats exposed to PSLTs by chronic 

inhalation appears to involve secondary genotoxicity resulting from chronic inflammation and 

cell proliferation [IARC 2010; NIOSH 2011; Olin 

2000]. Thus, at low lung doses in rats (i.e., below 

lung overload), where inflammation and cell proliferation are not present, lung cancer would not be 

anticipated [Greim et al. 2001]. Mice also showed 

overloading of lung clearance but had lower inflammatory response than rats in a subchronic inhalation study of a PSLT (carbon black); hamsters 

did not show overloading or lung inflammation in 

that study [Elder et al. 2005].

The interpretation and use of rat dose-response 

data of inhaled particles in human hazard and risk 

assessment and OEL development has been discussed and debated for many years [Cherrie et al. 

2013; IARC 2010; Kuempel et al. 2014; Morfeld et 

al. 2015; Oberdörster 1995; Olin 2000; Pauluhn 

2014; Warheit et al. 2016; Yu 1996]. Yet, the rat 

chronic bioassay data have been shown to give fewer false negatives for inhaled particles classified by 

IARC as human carcinogens than have the mouse 

and hamster data [Mauderly 1997]. Moreover, human particle lung doses in workers in dusty jobs 

such as coal mining have been shown to be equivalent to the mass-overloading doses in rats exposed 

chronically to particles [IARC 2010; Kuempel et al. 

2014; NIOSH 2011].

In general, the rat is considered a useful model for 

human non-neoplastic lung responses to PSLT, and 

in the absence of mechanistic data to the contrary, 

the rat model is relevant to identifying potential 

carcinogenic hazards in humans [Olin 2000]. Rat 

chronic inhalation data of PSLT were used by IARC 

[2010] in its evaluation of the carcinogenicity of inhaled PSLT (titanium dioxide and carbon black) and 

by NIOSH [2011] in its hazard classification and 

REL for nanoscale and microscale titanium dioxide.

Scientific questions on rat lung overload that still 

need to be resolved were discussed by Borm et al. 

[2015], who cite two publications that contribute 

to the debate [Morfeld et al. 2015; Pauluhn 2014]. 

To date there is no clear resolution of this issue in 

the scientific community. Therefore, interpretations 

of the rat dose-response data for risk assessment 

have differed widely for inhaled PSLT, including 

those for nanoscale titanium dioxide, using the 
64 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

same basic data [NIOSH 2011; Relier et al. 2017; 

Warheit et al. 2016]. Although the scientific debate may continue, dosimetric adjustments to account for differences in PSLT aerosol particle size 

and respiratory tract disposition and/or clearance 

between rodents and workers have been used to account for toxicokinetic differences, and uncertainty 

factors can be applied to account for toxicodynamic differences [EPA 1994; ICRP 1994; Jarabek et al. 

2005; Kuempel et al. 2015; Oller and Oberdörster 

2016]. Animal inhalation studies used in risk assessment should include sufficient doses to characterize the dose-response relationship, including 

low doses to overloading doses [Kuempel et al. 

2014; Oberdörster 1997; Olin 2000; Pauluhn 2011].

Despite the differences in particle clearance and retention kinetics, the overloaded rat model may be 

relevant to predicting risk to workers exposed to 

inhaled particles. Overloading doses of microscale 

PSLT in rats have been observed as low as 0.5 mg/g 

lungs, with complete cessation of clearance at doses >10 mg/g lungs [Muhle et al. 1990; Oberdörster 

1995]. By comparison, workers in dusty jobs historically have had average retained particle mass doses 

>10 mg/g lungs [Douglas et al. 1986; Freedman and 

Robinson 1988; Stöber et al. 1965]. Thus, only at 

overloading doses does the particle lung burden in 

rats reach the higher levels that have been reported 

in coal miners. These findings suggest that the rat is 

an appropriate model for human health risk assessment of respirable particles. 

Studies in mice and hamsters are not as predictive 

of the human particle-associated lung responses, 

and results were negative for some particles that 

have been classified as known human carcinogens 

[Mauderly 1997]. In a quantitative comparison of 

lung cancer risk estimates in rats and humans associated with chronic exposure to various types of 

respirable PSLT (coal mine dust, carbon black, titanium dioxide, or crystalline silica), the rat- and human-based estimates were statistically consistent, 

given the level of imprecision in the animal and 

human data [Kuempel et al. 2009; NIOSH 2011]. 

These studies suggest that the rat may be the most 

reasonable and sensitive rodent model to estimate 

the risk of chronic exposure to respirable particles, 

despite the species differences in the clearance and 

retention kinetics, which can be adjusted for by using dosimetry modeling. 

5.6.2.9 Tools/Models (deposition and/or 

clearance) 

The most widely used dosimetry models for inhaled particles and fibers are found in the MPPD 

suite of models [ARA 2015; Price et al. 2002]. These 

models have largely replaced the U.S. EPA Regional 

Deposited Dose Ratio (RDDR) model, which allowed estimation of the equivalent deposited dose 

in the respiratory tract across species but did not 

include clearance [EPA 1994]. However, the MPPD 

does not include models for all test species. For example, the RDDR model is still needed for studies 

using hamsters, because this species is not included 

in the MPPD.

Several deposition and clearance models are included in MPPD, as described in the model overview and details in the software (MPPD v. 3.04). 

The MPPD has been developed over a decade or 

more with funding by various U.S. governmental 

sources (including EPA, Navy, and NIOSH) and 

nongovernmental sources. It is publicly available to 

download at https://www.ara.com/products/multiple-path-particle-dosimetry-model-mppd-v-211.

NIOSH-funded revisions to earlier versions include 

batch capability in running the deposition and 

clearance models in humans and rats (in MPPD 

v. 2.1 [ARA 2009]); addition of oronasal deposition in animals and humans, including olfactory 

deposition of nanoscale particles [Garcia and Kimbell 2009; Garcia et al. 2015]; and extension of the 

spherical particle model to include nonspherical 

and fibrous particles based on aerosol characterization and measurement of deposition efficiency 

in human respiratory tract replicas [Su and Cheng 

2015; Su and Cheng 2014]. 

The MPPD is for poorly soluble particles (spherical 

or nonspherical) but does not account for particle 

dissolution. It is also limited to the respiratory tract 

and does not include translocation to other organs. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 65

Other dosimetry/PBPK models are needed to estimate internal dose of soluble particles in the lungs 

or other organs. 

In general, data available for PBPK modeling are 

limited. When validated models are available, they 

are preferred to application of uncertainty factors 

to estimate human-equivalent dose because they 

account for material- and species-specific factors 

influencing the dose-to-target tissues. Many individual PBPK (dosimetry) models have been developed for inhaled particles and fibers; their use 

would need to be evaluated on a case-by-case basis. 

Some useful tools and references associated with 

dosimetry modeling are listed in Table 5-2.

When searching the literature for information 

on lung dosimetry models of aerosols, note that 

multiple databases might need to be used, such as 

PubMed, Web of Science or Scopus, Toxline, and/

or Embase. Although PubMed is a major research 

database and perhaps the most widely used, it does 

not provide citations for some of the journals in 

which aerosol research is published (e.g., the Journal of Aerosol Science). Past practices have demonstrated that broader search strategies may be needed to identify relevant articles in this area.

5.6.3 Gas and Vapor Exposures

In general, the major factors influencing the internal dose from gas or vapor inhalation are anatomy 

(ventilation rate), physiology (diffusion, dissolution, blood flow, metabolism, and elimination rates), 

and physicochemical properties (e.g., gas or vapor 

solubility, reactivity) of the chemical [Bogdanffy and 

Jarabek 1995; Hanna et al. 2001; Jarabek 1995; Kuempel et al. 2015]. The components of the inhalation 

dosimetry adjustment for gases are the following.

1. Conversion of units from ppm to mg/m3

: The 

concentration in the inhalation toxicity studies 

on gases are usually reported in units of ppm 

or mg/m3

. For exposure levels reported as ppm, 

this should be converted to the standard units of 

mg/m3

 by using the following formula:

 

where MW is the molecular weight in grams and 

22.45 is the volume occupied by 1 g-mol of any compound in the gaseous state at 25°C and 760 mm Hg.

2. Duration adjustment: Many inhalation toxicity 

studies in laboratory animals are conducted with 

discontinuous exposure, often with exposure 

frequencies of 6 to 8 hours per day and 5 days per 

week. Occupational risk estimates are derived 

with the intention to protect workers against the 

exposure of 8 hours per day for 5 days per week, 

totaling 40 hours per workweek. Therefore, 

duration-adjusted exposure level is

Adjusted concentration (mg ⁄ m3

) =

E (mg/m3

) × D ⁄ 8 ×W ⁄ 5

where E is the experimental exposure concentration, D is the work day adjustment of the 

number of hours exposed in 8-hour daily increments, and W is the workweek adjustment of 

the number of days of exposure in 5-day workweek increments. 

3. Human Inhalation Rate: The human inhalation 

rate for light exertion while doing work of 9.6 m3/8 

hours should be included in the risk estimate. 

4. Human Equivalent Concentration (HEC): The 

HEC is the concentration of a substance in humans 

that is believed to produce an effect equal to a 

dose in experimental animals, adjusted for exposure duration and physiological parameters 

such as breathing rate. 

5.6.3.1 NIOSH Practice

The current NIOSH practice for calculating HEC 

is as follows:

1. Experimental animal dose in ppm is converted 

to daily mg/kg inhaled dose.

2. In the absence of chemical-specific information on 

metabolism or dosimetry, this dose is extrapolated 

to humans, assuming dose equivalence in units of 

mg/kg∙day, scaled according to body weight to the 

0.75 power [Kleiber 1932; Sidhu 1992]. 

3. The human mg/kg∙day dose is then converted to 

ppm for an 8-hour workday.
66 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Table 5-2. Examples of available tools and resources for dosimetry modeling 

(adapted from Kuempel et al. [2015]).

Name of Tool or 

Resource Description Source and Availability

Multiple-path 

particle dosimetry 

model (MPPD)

Deposition, clearance, and 

retention estimation of inhaled 

particles in the respiratory tract 

of the human, rat, and mouse 

ARA [2015], http://www.ara.com/products/mppd.htm

Based on several models including Anjilvel and Asgharian 

[1995], Asgharian et al. [2001; 2014], and ICRP models 

[ICRP 1994; Paquet et al. 2015] 

Respiratory tract 

region deposited 

dose equations

Deposited-dose estimation of 

inhaled particles or vapors 

Interspecies dosimetric 

adjustments

Derivation of reference 

concentrations

U.S. EPA [1994; 2012a], https://www.epa.gov/risk/

methods-derivation-inhalation-reference-concentrationsand-application-inhalation-dosimetry

http://cfpub.epa.gov/ncea/cfm/recordisplay.

cfm?deid=244650

Human respiratory 

tract model 

Deposition, clearance, and 

retention estimation of inhaled 

particles (including nonradioactive) in the human 

respiratory tract 

ICRP Publication 66 [ICRP 1994], http://www.icrp.org/

http://www.sciencedirect.com/science/

journal/01466453/24/1-3

Physiologically 

Based 

Pharmacokinetic 

(PBPK) modeling 

guidance

Guidance on principles of 

characterizing and applying 

PBPK models in risk 

assessment

U.S. EPA [2006a], https://cfpub.epa.gov/ncea/risk/

recordisplay.cfm?deid=157668

IPCS [2010], http://www.who.int/ipcs/en/

http://www.inchem.org/documents/harmproj/harmproj/

harmproj9.pdf

Loizou et al. [2008] and McLanahan et al. [2012] 

Human reference 

values 

Anatomical and physiological 

parameters (reference values) 

in humans

Inter-individual variability by 

age and gender

Parameters for PBPK models

ICRP Publication 89 [Valentin 2002], http://www.icrp.org/

http://www.sciencedirect.com/science/

journal/01466453/32/3-4

Interspecies 

reference values

Physiological parameters for 

dose normalization or PBPK 

modeling 

Application to Biological 

Exposure Indices

Brown et al. [1997], Davies and Morris [1993], Mercer 

et al. [1994], Stone et al. [1992], Boxenbaum [1982], and 

Fiserova-Bergerova [1990]

Particle size 

definitions 

Criteria for airborne sampling 

of particle size fractions by 

probability of deposition in 

human respiratory tract regions

ACGIH [2015], ACGIH [1985], and Lioy et al. [1984] 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 67

The following example is taken from a hypothetical 

study of the effects from 1-bromopropane inhalation using a 2-year bioassay of B6C3F1 mice dosed 

6 hours per day. The PoD for lung tumors in female 

mice (the outcome of interest) in this study was 

0.64 ppm. The experimental animal dose in ppm is 

converted to a daily mg/kg inhaled dose:

Mouse BMDL = 0.64 ppm × 5.031 ×

0.060 m3/day × 6 hour/24 hour/

0.0353 kg

= 1.37 mg/kg .day

ppm

mg/m3

where 0.64 ppm is the dose; 5.031 mg/m3 per ppm 

is the molecular weight (122.99 g/mol) divided by 

24.45; 0.060 m3/day is the reference inhalation rate 

for female B6C3F1 mouse; and 0.0353 kg is the reference body weight for female B6C3F1 mouse in a 

chronic study. 

This dose is extrapolated to humans, assuming 

dose equivalence in units of mg/kg∙day scaled according to body weight to the 0.75 power:

Human BMDL=Mouse BMDL (1.37 

mg ⁄ kg∙day) × (0.0353 kg ⁄ 70 kg)0.25 = 

0.205 mg ⁄ kg∙day

where 70 kg is the reference human body weight. 

Here the exponent value of 0.25 reflects body 

weight (BW) scaling to the 3⁄4 power (i.e., BW3/4) in 

units of mg/kg-day (rather than mg/day), such that 

BW1/1/BW3/4 = BW1/4. 

The human mg/kg∙day dose is then converted to 

ppm for an 8-hour workday:

Human BMDL = 0.205 mg/kg.day ×

×

9.6 m3/8 – hour workday

70kg

= 0.297 ppm, or about 0.3 ppm

1ppm

5.031mg/m3

where 9.6 m3 per 8-hour workday is the reference 

inhalation rate for humans.

5.6.3.2 U.S. EPA Practice

The HEC is calculated from a PoD (e.g., NOAEL, 

LOAEL, or BMDL) by using the following formula:

PoD[HEC](mg ⁄m3)=PoD[ADJ](mg ⁄m3)×RGDR

where RGDR is the regional gas deposited ratio, 

which is the ratio of regional gas dose in laboratory animal species to that of humans for the target 

region.

The EPA categorized the gas, on the basis of the 

physicochemical properties and the regions of the 

effect in the respiratory tract, into categories 1, 

2, and 3. Category selection for a given chemical 

should be based on the properties of the chemical 

and its target effects in the body, as described in 

Table 5-3. 

5.6.3.3 Category 1 Gases

Category 1 gases are highly water soluble or reactive and thus produce an effect mostly in the respiratory tract itself. Because of the high-level deposition along with high reactivity, local tissue damage 

is expected from these gas exposures. Only a small 

fraction of these gases could penetrate deeper than 

the ET region under normal circumstances. However, during heavy exercise, fires, explosions, etc., 

these gases could penetrate deeper, leading to tissue damage in the distal respiratory tract. The following equations are used to calculate RGDR for 

different regions of Category 1 gas. If the calculated 

value is greater than 1, then RGDRET is set to 1.

1. RGDRET for Category 1 gas:

where VE is the minute volume (cm3/min), SAET is 

the surface area of the extrathoracic region (cm2), 

and terms A, H represent laboratory animal and 

human, respectively. Later EPA guidance suggested 

that there is internal dose equivalency in the ET region for rats and humans. This guidance suggests 

that in lieu of specific modeling data, the default 

RGDRET is 1, rather than 0.2–0.3 as predicted by 

the equation above [EPA 2012a].

RGDRET = (VE / SAET)A

(VE / SAET)H
68 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

2. RGDRTB for Category 1 gas:

 

where VE is the minute volume (cm3/min), SATB is 

the surface area of the TB region (cm2

), and terms 

A, H represent laboratory animal and human, 

respectively.

3. RGDRPU for Category 1 gas:

where Qalv is the alveolar ventilation rate (mL/

min) and is equal to 0.6 × VE.

5.6.3.4 Category 2 Gases

Category 2 gases are moderately water-soluble and 

have the potential to penetrate bronchi and thereby 

the blood. Therefore, both local and systemic effects could be observed following exposure to these 

gases. HECs for respiratory tract effects are calculated by the equations for a Category 1 gas, whereas 

HECs for extra-respiratory effects are calculated by 

the Category 3 equations. In cases where respiratory 

tract effects are caused by systemic distribution of 

the chemical, such as chloroform and naphthalene, 

the HEC should be calculated as a Category 3 gas.

 

5.6.3.5 Category 3 Gases

Category 3 gases have very low water solubility 

and limited reactivity with respiratory epithelium. 

These gases readily penetrate to the pulmonary 

region and are absorbed into the systemic circulation. Most of the effects are observed distal to the 

respiratory system except in cases where metabolism in the upper-respiratory tract leads to local 

effects. The following equation is used to calculate 

the RGDR for Category 3 gases:

 

where the value  is the ratio of the blood:gas 

(air) partition coefficient of the chemical for the laboratory animal species to the human value. A value 

of 1.0 is used for the ratio of (Hb/g)A > (Hb/g)H. A 

value of 1.0 is used as the default when one or both 

of the partition coefficients are not available. Blood: 

air partition coefficients for some chemicals are 

available from Gargas et al. [1989]. 

5.6.3.6 PBPK and Computational Fluid 

Dynamics Approaches

PBPK modeling (see Section 5.6.2.6) also applies 

to gas and vapor exposures to derive target tissue 

Table 5-3. Gas categories and characteristics.

Category

Water 

Solubility Reactivity

Accumulation 

in blood

Site of 

toxicity Examples

1 High Rapidly irreversibly 

reactive

Not significant Portal of entry Hydrogen 

fluoride, chlorine, 

formaldehyde, volatile 

organic acids and 

esters

2 Moderate Rapidly reversibly reactive, 

or moderately to slowly 

irreversibly metabolized in 

respiratory tract tissue

Potential Portal of 

entry, maybe 

systemic

Ozone, sulfur dioxide, 

xylene, propanol, 

isoamyl alcohol

3 Low Unreactive in surface 

liquid and tissue

Yes Systemic 

toxicity

Styrene

Adapted from EPA [2012a].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 69

dose estimates in various species. As stated previously, the construction and development of a PBPK 

model for an individual chemical involves a large 

amount of data and understanding of the absorption, metabolism, distribution, and elimination 

of the chemical in the test species and in humans. 

One of the examples of PBPK modeling used in occupational risk assessment is use of the methylene 

chloride PBPK model to derive target tissue dose 

estimates for lung tumors in mice [OSHA 1997].

Computational fluid dynamics–PBPK (CFD/PBPK) 

models are designed to model the fluxes of vapor 

between tissue phases (e.g., between epithelial and 

submucosal tissues) and allow for a differential 

blood flow and coupling the respiratory tract to the 

whole body. This type of model has been used to 

evaluate the dosimetry of many compounds, including diacetyl and styrene [Gloede et al. 2011; 

Sarangapani et al. 2002]. NIOSH used CFD/PBPK 

modeling in its risk assessment of occupational diacetyl exposure [NIOSH 2016].

5.7 Dose-Response Modeling 

with Epidemiologic Data

NIOSH prefers the direct estimation of occupational risks in working populations because: (1) 

data reflecting actual exposures and responses 

within the population of interest are inherently superior for risk assessment, and (2) the uncertainty 

in extrapolating data from animal toxicologic studies can be much larger than that in well-designed 

epidemiologic studies [Hertz-Picciotto et al. 1995; 

Smith 1988; Stayner et al. 1999]. Of the NIOSH risk 

assessments listed in Table 1-1, nine (70%) quantitatively examined the dose-response relationship 

by statistical models using epidemiologic data. In 

contrast, epidemiologic data have been used in less 

than 10% of Integrated Risk Information System 

(IRIS) assessments conducted by the EPA [Persad 

and Cooper 2008]. 

Many of the concepts discussed previously concerning animal data are also applicable to human 

data, especially for experimental designs or when 

modeling binary outcome data from observational studies without time-dependent variables (e.g., 

using logistic regression). Although methods of 

analyses may be identical, the majority of human 

data for risk assessment stems from observational 

studies, which have less control of extraneous factors and are more prone to bias than studies of experimental data (see Section 4.3.4). The design of 

epidemiologic studies contributing to risk assessment can vary, as can study aims, which also may 

not fully align with risk assessment goals. 

NIOSH risk assessors first decide, in a systematic 

way, if human data are suitable for quantitative doseresponse analyses, and if so, whether the data will 

serve as (1) the primary basis for risk extrapolation 

or (2) supporting information for a risk assessment 

primarily based on animal toxicologic data. The 

evaluation may be made concurrently with the WoE 

assessment in hazard identification, although data 

supporting hazard identification may lack the rigor 

necessary for dose-response analyses. In any event, 

all decisions on data suitability should be fully described in the risk assessment documentation. As a 

starting point, risk assessors have applied the framework first described by Hertz-Picciotto [1995], who 

suggested judging the suitability of epidemiologic 

data for quantifying dose-response by using five 

criteria. These criteria, slightly modified for NIOSH 

risk assessment purposes, are as follows:

1. The data consistently indicate a stable positive 

statistical association between the agent and adverse effect.

2. The data are abstracted from studies that are of 

high overall quality.

3. There is no substantial potential for confounding or other source of major bias.

4. There is a quantitative assessment of exposure 

that is deemed appropriate for dose-response 

analyses.

5. There is evidence of a monotonic dose-response.

Hertz-Picciotto [1995] suggested that compliance 

with criteria 1 through 4 provides a minimum basis 
70 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

for risk extrapolation using human data; compliance with 2 of the first 3 criteria is considered suitable for quantifying risks as a plausibility check 

with animal-based assessments. Thus, it is clear 

that more weight is to be placed on criteria 1, 2, and 

3. Criterion 1 is directly related to Hill’s guidelines 

on strength of association and consistency (see 

Section 4.1). This criterion differs from that originally specified, which included only moderate to 

strong positive associations [Hertz-Picciotto et al. 

1995]. This modification was made in recognition 

that excellent studies reporting weakly positive but 

consistent associations can inform the dose-response relationship. For example, Park et al. [2004] 

conducted a quantitative risk assessment of lung 

cancer from exposure to hexavalent chromium that 

used data from Gibbs et al. [2000], who reported 

modestly elevated lung cancer risk (e.g., SMR <2) 

in chromium production workers compared to the 

general population (SMR = 1.80; 95% CI: 1.49–

2.14). This risk assessment helped form the basis 

for the NIOSH REL on hexavalent chromium exposure [NIOSH 2013a]. Criteria 2 and 3 are related; 

both prefer study designs that reduce the potential 

for an inaccurate estimated effect. Criterion 4 was 

modified to recognize that quantitative exposure 

data at the individual level, as originally recommended [Hertz-Picciotto et al. 1995], is sparse in 

epidemiologic studies. The lack of individual-level 

measurements should not disqualify study data 

from quantitative risk assessment; however, their 

presence is preferred to aggregate exposure measures that are more vulnerable to bias from shared 

error or exposure misclassification (see Appendix 

B). Criterion 5 coincides with Hill’s guideline on 

a biologic gradient, which is not necessary in either case but certainly supports data use. Note that 

there are many explanations for a lack of observed 

monotonicity in dose-response data, such as measurement error, biologic saturation, and depletion 

of a susceptible population [Stayner et al. 2003]. 

Last, there may be exceptional circumstances in 

which other criteria may better apply or in which 

modification to existing criteria is prudent. Risk assessors should describe these exceptions in the risk 

assessment document. For example, a potential for 

substantive confounding may exist (criterion 3); 

however, data may allow for an examination and/

or adjustment of its effect on dose-response estimates. In this example, the risk assessment document should fully describe the potential for bias, 

the alternative analyses for examining the effects 

on dose-response estimates, and any consequent 

actions. 

NIOSH risk assessors strive to make the best use 

of epidemiologic data that are available in doseresponse modeling strategies, given that these data 

provide the important advantage of directly addressing human risk. When epidemiologic data 

are available and appear suitable for quantifying 

exposure-related effects, the risk assessor generally adopts a statistical modeling approach that includes an evaluation of potential sources of biases. 

NIOSH risk assessors endeavor to select statistical 

methods that best account for identified sources of 

uncertainty and therefore improve the reliability 

and validity of dose-response estimates. 

As discussed previously, exposure estimation in 

human studies is often fraught with limitations. 

Risk assessors consider ways to account for exposure uncertainty in developing a risk modeling approach. Given the relative uniqueness of most epidemiologic datasets, it is not feasible to describe all 

possible modeling strategies in this report. However, this section discusses some overall modeling 

approaches using human data methods that are 

unique to aggregate data from published reports 

and time-to-event data from longitudinal studies. 

5.7.1 Limited Data

Although dose-response analyses using individual 

exposure and outcome data are preferred, the lack 

of these data does not preclude examining statistical 

associations and regression relationships between 

aggregate measures of exposure and measures of 

population risk using summary estimates from human data. In fact, risk assessments have used limited data comprising only an aggregate exposure 

measure (e.g., average cumulative exposure) and a 

measure of relative risk (e.g., SMR, SIR, and OR). 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 71

For example, a simple dose-response model can be 

specified with data from a study reporting only a 

cohort SMR and average exposure by assuming a 

linear relationship exists between the SMR (or any 

measure of relative risk) and exposure (x): SMR = 1 

+ xβ, where the dose-response slope, β, represents 

the change in relative risk (e.g., the SMR) per unit 

dose [Smith 1988; Stayner et al. 1995]. Figure 5-7 

shows a plot of this relationship for a hypothetical 

cohort. 

An SMR from an occupational study may be negatively biased from a healthy worker hire effect (see 

Appendix B for more information). This effect can 

be countered by using an adjusted SMR, derived on 

the basis of the study type and outcome [Park et al. 

1991] or by fitting another parameter to the model 

to adjust for the background hazard rate [Stayner et 

al. 1995]. Similarly, information on other potential 

sources of bias can be included as model covariates. 

As another example, if SMRs are reported at different levels of exposure, then weighted least-squares 

regression or maximum likelihood estimation 

methods can be used. Examples of these techniques 

have been described in several reports [Breslow and 

Day 1987; Crump and Allen 1985; Hanley and Liddell 1985; Rothman et al. 2008; Smith 1988; Smith 

et al. 1994; Steenland and Savitz 1997]. 

There are some important limitations in the methods described above. First, we assume the true 

dose-response is linear, given that it is biologically 

plausible, generally appears conservative in the 

low-dose range compared to alternative models, 

and tends to fit epidemiologic data. Alternatively, 

a nonlinear dose-response function could be fitted, which may result in marked differences in estimates in the range of interest. In practice, however, 

adequate data preferring a nonlinear model are 

unlikely in most situations. Second, pooling SMRs 

in dose-response analysis is less than ideal, given 

that multiple SMRs (stratum-specific) may not be 

comparable because of indirect standardization. 

Pooling SMRs could bias estimates from differences in age, race, sex, or some other confounder 

across exposure groups. Fortunately, strong statistical confounding from stratum heterogeneity is not 

typically observed in most cases [Breslow and Day 

1987]. Nonetheless, risk assessors must address the 

potential for bias from heterogeneous comparison 

groups. Third, using weighted least squares to regress multiple responses at different exposure levels 

(e.g., SMRs or log-SMRs) does not account for correlations between response measures induced by 

sharing a common reference group. Methods have 

been developed to account for these correlations 

in trend estimation in both single study and metaanalytic (meta-regression) designs [Greenland and 

Longnecker 1992; Hamling et al. 2008; Orsini et 

al. 2012]. Of course, these methods are crude and 

generally inadequate to serve as the sole basis for 

NIOSH recommendations on OELs. Instead, they 

may be informative in risk characterization when 

used in conjunction with other supporting data.

A NIOSH example of a limited data approach is 

not available; however, examples are available in 

the literature [Chovil et al. 1981; Crump and Allen 1985; Hanley and Liddell 1985; Smith 1988; 

Steenland and Savitz 1997]. For example, Steenland 

and Savitz [1997] used a simple linear model to examine the dose-response between airborne nickel 

levels and lung cancer mortality. The dataset was 

abstracted from a previous epidemiologic study of 

Ontario nickel refinery workers (n = 495) followed 

from 1963 to 1978 [Chovil et al. 1981]. The relative 

risk per unit exposure was estimated by the slope 

parameter from a weighted least-squares linear regression of the SMRs at specified cumulative dose 

levels. The expected numbers of lung cancer deaths 

were used as weights and the model forced the 

intercept at unity. A simple estimate of lifetime excess cancer was estimated by RAR = R0(xβ), where 

RAR is the added lifetime risk from exposure x, R0 is 

the background lifetime risk of lung cancer death, 

and β is the upper 95% confidence limit on the 

slope parameter. 

5.7.2 Longitudinal Data

In longitudinal studies, data on observation time, 

demographics (e.g., age, race, and gender), and timevarying predictors (e.g., exposures) are available. 
72 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Figure 5-7. Linear dose-response slope estimates using average cumulative exposure and reported standardized 

mortality ratio (SMR) from an epidemiologic study (adapted from Smith et al. [1988]).

Approaches to modeling must be consistent with 

the data, although more than one approach may 

be available. For example, data from a cohort study 

of cause-specific mortality can be expressed as the 

amount of observation time and an observed count 

of adverse responses cross-classified on the basis of 

other predictors in order to estimate the incidence 

rate. 

In general, previous risk assessments have applied 

a tiered approach, whereby categorical analyses 

and splines are first used to evaluate the shape of 

the dose-response curve, which aids in defining a 

set of parametric models that are most appropriate for risk assessment [Steenland and Deddens 

2004]. The risk assessor may then select a preferred 

model from the set of models on the basis of prior 

knowledge of the expected response (biologic plausibility) and model fit. In any event, the choice of 

the best model should not rest solely on statistical 

grounds [Breslow 1990]. This is because competing 

statistical models can often yield roughly equivalent fits to the data in the observable-effect dose 

range, yet extrapolation below the observable range 

(i.e., in the range of interest) can result in estimates 

that are orders of magnitude apart [Brown and 

Koziol 1983; Stayner et al. 1995]. Methods within 

the framework of this approach can vary, and an 

exhaustive discussion of all modeling possibilities 

is beyond the scope of this report. More information is available in many important epidemiologic 

texts [Breslow and Day 1987; Rothman et al. 2008; 

Woodward 2013]. 

Dose-response modeling of longitudinal data has 

been approached with use of a wide array of methods but is generally conducted by regression of survival data (i.e., failure-time data) or person-years 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 73

data. Survival regression models can be fully parametric models of the distribution of failure times 

(e.g., Weibull models) or semi-parametric (i.e., Cox 

proportional hazards model). Poisson regression 

modeling is an example of a modeling approach for 

data on response counts and person-years of observation. Most epidemiologic studies have examined dose-response relationships from longitudinal study data by using general relative risk models 

with maximum likelihood estimates obtained from 

Cox proportional hazards or Poisson regression 

techniques for cohort data and conditional logistic 

regression for nested case-control designs.

5.7.2.1 Poisson Regression 

Follow-up data can be recorded as counts of responses, i.e., the number of events (e.g., deaths) and 

the number of person-years in strata of other variables (e.g., age, sex, exposure, etc.). Furthermore, 

dose-response curves can be fitted to the count data 

Yi and person-years based on Poisson regression 

modeling, i.e., Yi ~Poisson[ f (di, X1i, X2i, ..., Xci ; θ) · τi] 

where Yi represents the count observed during 

an accumulation of person-time τi ; if each record 

of the data is constructed from one person, then 

Yi ~Binomial [ f (di , X1i , X2i , ..., Xci ; θ) · τi | η1 = 1 may be 

substituted. If a reference population is available 

that provides information on the rate associated 

with age, sex, and other demographic variables, 

then it can be incorporated to improve estimate 

precision. However, the assumption of Poisson or 

binomial variations is a strong one, and it may be 

necessary to accommodate response variation exceeding that predicted by the model (i.e., over-dispersion), such as using quasi-likelihood methods 

or models that incorporate random effects. 

Examples of Poisson regression modeling in 

NIOSH risk assessment include dose-response 

models of the relationships between lung cancer 

and hexavalent chromium [NIOSH 2013a; Park et 

al. 2004] and asbestos [Stayner et al. 1997]. 

5.7.2.2 Cox Proportional Hazards Regression 

In survival analyses, the hazard function (or hazard) is the rate of failure at an instant in time, t, 

given that the individual survives up to t. In other 

words, it is the instantaneous risk that the event 

(e.g., death, cancer diagnoses) will occur at t. In 

most longitudinal studies, the time scale of interest 

is age. The hazard ratio (HR) is the hazard of one 

individual (e.g., the exposed) divided by another 

individual (e.g., the unexposed), typically holding all other predictors constant; therefore, it is a 

measure of the relative risk. Since its introduction 

in 1972, the Cox proportional hazards (CPH) regression model has become the most widely used 

approach to quantifying conditional hazards [Cox 

1972]. A general form of the CPH model for the 

hazard, h, cumulative dose, D, and attained age, t, is

h(t|D(t),Z(t)) = h0(t) f(D(t); β) exp[γTz],

where h0 is the baseline hazard, Z represents a vector of model covariates, model parameters β and 

γ are to be estimated, and f(D(t); β) is the relative 

rate as a function of cumulative dose at attained 

age. This model is semi-parametric because the 

baseline hazard is an unspecified function, but a 

parametric form is assumed for the effect of predictors on the hazard. Several options for specifying a 

dose rate function are available; the most common 

is an exponential form, i.e., (D(t); β) = exp (βD(t)), 

which is sometimes referred to as a log-linear doseresponse model. In this form, the CPH model is a 

simple additive model of the log of the hazard. Another common form is a linear response function, 

f(D(t); β) = 1+βD(t), where β is the excess relative 

rate per unit dose in the exposed individual relative 

to the unexposed. Validity of this model relies on 

a rather strong assumption that the hazards in the 

group of interest are proportional to the hazards in 

the referent group, and this proportionality is constant over time when D(t) is constant. A significant 

interaction between D(t) and t would be evidence 

against such proportional hazards. Additional statistical methods (e.g., stratification, fully parametric, 

or piecewise proportional models) may be necessary in the event of strong modification of the dose 

effect on the hazard over time [Allison 2010]. 

Examples of CPH regression methods in NIOSH 

risk assessment include dose-response modeling of 
74 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

the relationships between lung cancer and exposures 

to radon [Hornung and Meinhardt 1987] and cadmium [Stayner et al. 1992a; Stayner et al. 1992b]. 

5.7.2.3 Conditional Logistic Regression

Case-control designs are typically analyzed with 

logistic regression, as previously described. The 

fitting of matched or stratified logistic regression 

models is sometimes referred to as conditional logistic regression [Breslow and Day 1980; Rothman 

et al. 2008]. When time-dependent predictors are 

present, nested case-control studies often rely on 

conditional logistic regression. For nested casecontrol studies with one case per matched set (i.e., 

1:n matching), the form of the likelihood function 

for conditional logistic regression reduces to that 

of the CPH model for the continuous time scale. 

In both cases, the data are organized into risk sets 

(sometimes referred to as a matched set in conditional logistic regression), whereby a risk set is 

the collection of individuals at risk for the event 

at each time point in which a failure is observed. 

For example, a nested case-control study may 

specify a conditional logistic regression model that 

uses controls drawn from risk sets of individuals 

matched to cases on attained age. In this instance, 

the controls are selected by using incidence-density 

sampling methods [Beaumont et al. 1989; Richardson 2004]. Computational limitations may restrict 

full risk-set analyses (i.e., CPH model); therefore, a 

nested case-control study using conditional logistic 

regression is an appealing alternative to full cohort 

modeling.

5.7.2.4 Additional Considerations

In epidemiological studies, di is an exposure metric constructed from possibly complex employment 

histories, and identifying the optimum construct 

for di may itself be an important component of the 

modeling procedure (see Section 4.3.3.1). Moreover, predictors in the function f (di , X1i , X2i , ..., Xci ; θ) 

usually include age, gender, and other demographic 

variables and may confound or modify the effect of 

dose. Thus, the risk assessor must consider effects on 

estimates from the selected model or set of models 

that are due to the exposure metric construct and 

other predictors (see Appendix B for discussion on 

possible study biases). 

Attenuation of the dose-risk relationship at higher 

doses is a common observation in occupational 

epidemiologic studies [Stayner et al. 2003; Steenland et al. 2015]. This effect typically presents as 

a monotonically increasing slope at low exposure 

levels that diminishes or becomes negative at high 

exposure levels. Among possible explanations are 

a depletion of the susceptible population; healthy 

worker survivorship; a natural limit on the relative risk for diseases with a high background rate; 

measurement error; influence of unknown risk factors that may vary by the level of exposure adaptive 

responses; and biologic saturation. Regardless of 

cause, the risk assessor must consider the possible 

effects of high-dose attenuation when estimating 

responses at low doses, given that a “best fit” model 

may be a poorer choice for risk assessment. For example, a linear excess relative risk model that is best 

fit to the full range of exposures may underestimate 

the low-dose response as a result of risk attenuation 

(artifactual or otherwise) at high doses. Conversely, using a logarithmic transformation of exposure 

(i.e., a power model) may improve the model fit; 

however, this model is prone to overestimation of 

the response at low doses [Ginevan and Watkins 

2010; Steenland and Deddens 2004; Steenland et al. 

2011]. The potential for high-dose attenuation can 

be explored with categorical models, transformations of the exposure metric such as square-root 

or logarithmic, and the use of splines. However, 

the response in the low-dose region of the doseresponse curve can widely vary between these approaches [Steenland and Deddens 2004; Steenland 

et al. 2011]. When selecting preferred models for 

risk assessment, the risk assessor must evaluate the 

low-dose behavior of the models with respect to 

the potential effects of attenuation. Modification or 

replacement of the best-fit model may be required 

in order to avoid unrealistic estimates of effects in 

the range of dose that is most meaningful to the 

protection of workers. For example, simple piecewise linear models that allow for different slopes 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 75

between high- and low-dose regions (e.g., a twopiece linear spline) may be appealing, given they 

account for high-dose attenuation but allow for interpretation of risk at low dose that is suitable for 

risk assessment purposes [Steenland and Deddens 

2004; Steenland et al. 2011]. 

Dose-rate effects are of interest because they can 

have a substantial impact on low-dose extrapolation common in risk assessment. A dose-rate effect 

occurs when, for a given dose (e.g., the product of 

the chemical concentration and exposure duration), the dose-response relationship depends on 

the exposure intensity (e.g., the magnitude of the 

airborne chemical concentration). A positive doserate effect (i.e., higher exposure intensities having a 

greater effect) suggests that transient or peak exposures may have an important role in disease induction [Checkoway and Rice 1992; Esmen 1984; 

Rappaport 1991]. For example, peak exposure is 

obviously most important for evaluating acute toxicity. Peak exposure can also be the primary index 

when the agent is rapidly eliminated from the body 

or when nonlinear rates of biologic damage occur 

during periods of intense exposure [Esmen 1984; 

Rappaport 1991]. Nonlinearity may result from exposure-related responses that are reversible and/or 

have a threshold for the onset of biologic damage. 

Dose-rate effects that act to attenuate the response 

at higher exposure intensities are sometimes referred to as protraction enhancement or inverse 

dose-rate effects. These effects were evident in 

studies of the dose-response association between 

lung cancer and radon exposures in underground 

uranium miners [Lubin et al. 1995b]. The actual 

mechanisms involved in the radon inverse doserate effects are unknown; however, some suggested 

plausible explanations are nonlinear cellular responses, such as a bystander effect (i.e., a dose effect 

observed in non-irradiated cells) at very low dose 

rates [Brenner et al. 2001], or physical differences 

in the particle size distribution of radon progeny at 

different airborne concentrations [Leonard 2007]. 

Addressing dose-rate effects is challenging in doseresponse modeling. Interpretation will be largely 

dependent on the dose index used; however, given 

the complex and largely unknown biology associated with these nonlinear effects, mechanistic data 

are likely insufficient to inform a modeling strategy. Nevertheless, complete understanding of the 

underlying cause-and-effect relationship may not 

be necessary if a dose-response relationship between the chosen exposure metric and the adverse 

effect can be quantified.

Finally, there has been considerable advancement 

in statistical methods used in dose-response modeling of epidemiologic data since the onset of personal computing. New techniques are emerging to 

model nonlinear dose-response functions and to 

address data and modeling uncertainty. Modeling 

strategies might draw on newly developed statistical methods to account for modeling uncertainty 

and data incompleteness or imperfection, such as 

the following: 

■ Unobserved variables that affect observed 

exposure-response associations (e.g., using latent-variables methods and finite mixture-distribution models) [Leisch 2004; Rosseel 2012] 

■ Missing data values (e.g., using multiple imputation and conditional expectation methods, 

data augmentation, and the expectation-maximization [EM] algorithm) [Schafer and Olsen 

1998; Wei and Tanner 1990] 

■ Measurement errors and estimation errors in 

exposure and other explanatory variables (e.g., 

using regression calibration or Monte Carlo 

simulation techniques; see Appendix B) [Carroll et al. 2006] 

■ Model specification errors and model uncertainties (e.g., using model average methods; 

see Appendix B) [Buckland et al. 1997; Raftery 

et al. 1997] 

■ Inter-individual heterogeneity and variability 

(e.g., using mixed models accounting for random effects) 

■ Correlated or interdependent explanatory 

variables.
76 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

5.8 Alternative Analysis

The choice of modeling approach can markedly 

influence risk estimates. Moreover, limitations in 

available data often make the correct model specifications impossible to determine with confidence. 

When this occurs, risk assessors should candidly 

acknowledge the uncertainty in risk characterization. Limitations in available data often require 

scientific judgment in order to fill gaps in model 

specifications. Risk assessors identify and characterize these judgements by conducting additional 

analyses to test plausible alternative assumptions, 

examine the robustness of main analyses, and improve transparency in the risk assessment process. 

These alternative analyses comprise sensitivity and 

modeling uncertainty analyses. NIOSH defines 

sensitivity analysis as a study of the uncertainty in 

estimates from the mathematical model that can be 

apportioned to uncertainties in its inputs. In other 

words, it is a study of the robustness of the modeling outputs to uncertainty in model inputs. Model uncertainty addresses the possibility that the 

model itself—that is, the formula or algorithms for 

calculating outputs from inputs—is incorrect (see 

Appendix B). In alternative analyses, plausible alternative risk assessment strategies, defaults, and 

assumptions are quantitatively evaluated for their 

impact on risk estimates. As stated in Science and 

Decisions [NRC 2009], 

“... analysis could be performed when risk estimates for alternative hypotheses that are sufficiently supported by evidence are reported. 

This approach would require development of a 

framework with criteria for judging when such 

an analysis should be performed. The goal is not 

to present the multitude of possible risk estimates 

exhaustively but to present a small number of 

plausible cases to provide the risk manager a 

context for understanding additional uncertainty contributed by considering assumptions other 

than the default.”

This means a targeted, hypothesis-driven strategy 

for conducting alternative analyses is preferred. 

In large part, alternative analyses are examinations 

of risk estimates over a range of plausible values for 

uncertain data that are used in the risk assessment. 

Largely divergent estimates (or large uncertainties) suggest a high degree of model dependence, 

whereas reasonable agreement in findings suggests 

estimate robustness. Sensitivity analysis of a model 

that is known to correctly describe the causal relationship between inputs (including exposure 

histories) and outputs (including worker risks) is 

also useful for identifying how changes in factors 

change worker risks, which could then be targeted 

as priorities in risk management. Finally, alternative analyses can be a useful tool for model development and refinement [Frey and Patil 2002]. 

Alternative analysis should be part of the initial 

risk assessment plan. Analysis planning generally 

includes a description of any iterative methods intended for model development and refinement. 

Alternative analysis can be structured into the 

main analysis so that a variety of risk estimates is 

produced and the decision path to the final risk 

estimate is well supported and transparent. Risk 

assessors are cautioned against post hoc analyses 

as a substitution for planned alternative analyses. 

Nevertheless, there are instances when these analyses are appropriate or even expected. For example, 

subsequent analyses may occur in response to review comments. 

It is not practical to list all alternative analyses possible in risk assessment; however, Table 5-4 lists 

some examples of analyses that appear most often 

in the literature. Some areas of typical analyses are 

discussed in subsequent sections.

5.8.1 Choice of Adverse Effect

There may be more than one adverse effect available for dose-response analysis. Decisions about 

which adverse effect to analyze rely on consideration of the site of the effect and its relevance to 

the human toxicity of concern, severity of effect, reversibility of effect, MoA, sensitivity of the test species (or human subpopulation), and consistency of 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 77

effects across sex/species (or population groups). 

When multiple adverse effects are possible, and 

there is no clear choice for a single health effect 

of interest, it is a general course to examine multiple endpoints and choose the most limiting as 

the critical effect. Sensitivity analyses should include plausible alternative adverse effects and/or 

multiple indicators of a common effect. For example, different respiratory endpoints could include 

different measures of lung function (e.g., forced 

expiratory volume in 1 second [FEV1], forced vital capacity [FVC], and the ratio of FEV1 to FVC), 

self-report of symptoms, and/or diagnosed respiratory effects. For cancer studies, a variety of tumor 

sites could be analyzed. In animal toxicology studies, adverse effects could include analysis of both 

cancer and non-cancer effects, a selection of tumor 

sites, and more. The rationale for selecting the adverse effects in the main analysis and the sensitivity 

analysis should be explained.

Table 5-4. Example scenarios for alternative analyses.

Source of 

Uncertainty Research Question Possible Sensitivity Analysis 

Response variable Are there alternative definitions of 

the adverse effect? If so, how do these 

definitions affect dose-response estimates?

Alternative models using different 

specifications of the response variable

Is more than one adverse effect (not on the 

same causal pathway) associated with the 

hazardous exposure? If so, how do risks 

differ?

Alternative models using array of plausible 

responses

Explanatory variables How does measurement error in the 

primary exposure affect risk estimates? 

Alternative models using array of plausible 

estimates of exposure based on uncertainty

Are there alternative exposure metrics? 

If so, how do risk estimates differ across 

metrics?

Alternative multiple models using array of 

exposure definitions

If exposure is categorical, how does the 

choice of category cutpoints affect risk 

estimates?

Alternative models with varying exposure 

cutpoints

Is there a potential for unmeasured 

confounding (e.g., smoking data unavailable 

in analysis of cancer) by one or more 

sources? Can these effects be estimated?

Alternative models using array of plausible 

estimates of the confounder

Model specification How does model choice of dose-response 

function affect risk estimates?

Alternative models using array of plausible 

dose-response functions

How does the choice of confounding 

control (e.g., stratification versus covariate 

control) affect risk estimates?

Alternative models using array of methods 

for confounding control 

Animal to human risk 

extrapolation

How do assumptions on the animal-human 

relationship for metabolism, distribution, 

and toxicity affect risk estimates? 

Alternative models using array of plausible 

assumptions
78 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

5.8.2 Model Uncertainty Analyses

When practical, it is generally preferable to use 

non-parametric causal graph modeling techniques 

or model averaging methods to address model 

uncertainty (see Appendix B). That said, a standard practice in NIOSH dose-response modeling 

involving model selection is to specify models of 

interest beforehand and then test the specification 

by examining alternative specifications. When different plausible assumptions lead to very different 

estimates, then multiple ranges of plausible estimates, with the key assumptions leading to each, 

can be more informative and useful than a single 

range. Alternative models should be plausible and 

parsimonious. It is preferred that the model uncertainty analysis approach used, including the suite 

of alternative models to be examined, be specified 

a priori; however, model output information has 

been used in post hoc specification of alternative 

models in some analyses. This is likely to occur 

when results from main models point to the need 

for further development or if new information is 

found during analysis or in review afterward. 

The validity of modeling defaults should be examined in a sensitivity analysis when additional 

chemical-specific information is available that 

challenges those values. In addition, when there are 

alternative plausible assumptions on explanatory 

variables used in an analysis, it is reasonable to explore the impact of these assumptions in sensitivity 

analysis. Typical examples of key assumptions in 

epidemiologic studies include exposure lag times, 

homogenous dose-response among grouped outcomes (e.g., all cancers) and irreversible effects of 

chronic exposure, especially for non-cancer adverse effects. When examining alternative assumptions or default vales, it is important to use credible 

values that reflect the available data.

In some cases, the potential effects of measurement 

error or unmeasured confounding can be examined by sensitivity analysis [Chu et al. 2006; Greenland 1996; Groenwold et al. 2010]. For example, 

consider a study reporting a positive dose-response 

association between lung cancer and exposure to 

chemical X. Smoking data are unavailable. One 

could assume a range of plausible smoking behaviors (and their effects) that vary by degree of correlation with chemical X to examine the potential for 

residual confounding by smoking in main analyses. 

If a significant effect is not observed under plausible scenarios, then it is unlikely that smoking patterns explain the dose-response observed. 

The complexity of these sensitivity analyses can vary 

widely, from a simple examination of a single binary 

variable to complex computer simulations for examining joint effects of multiple factors. Some examples of sensitivity analyses over this range are readily 

available in several highly cited articles [Frey and 

Patil 2002; Greenland 1996; Greenland et al. 2005; 

Lash and Fink 2003; Lin et al. 1998]. Regardless of 

analysis design, it should be evident that reasonableness hinges on the range and values examined; 

therefore, risk assessors must carefully consider the 

choice of plausible values. 

In all modeling efforts, including sensitivity analyses of alternative models, NIOSH risk assessors must 

clearly describe the approach used in sufficient detail 

such that results can be replicated. Special attention 

should be given to providing a sound basis for any 

post hoc analyses conducted. Risk assessors should 

be aware that model specifications made using post 

hoc information, say, from a stepwise regression approach, can introduce bias from a lack of accounting 

for the informed choices made [Harrell 2015]. 

For example, consider a dose-response model that 

includes an exposure lag period accounting for disease latency. Risk estimates from models using a lag 

that is fixed a priori lack consideration of the uncertainty attributable to the lag choice. This uncertainty could be examined in a sensitivity analysis of 

models fit using alternative lags, although the main 

analysis still assumes a fixed lag. Alternatively, the 

lag could be estimated from the data (e.g., the lag 

that maximizes the likelihood) using methods fully 

accounting for the uncertainty in the lag [Richardson et al. 2011]. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 79

5.8.3 Extrapolation Methods

Model extrapolation occurs when inferences are 

made beyond the calibration or validation of the 

model [Frey and Patil 2002]. This can occur when 

model inputs used to predict risk are beyond the 

dataset used to develop the model. For example, 

the application of animal toxicologic data to assess 

human risk is a common extrapolation. When extrapolating risk from animals to humans, depending on the metabolism, distribution, and toxicity 

of the chemical, assessors may have a choice of extrapolation methods. In this case, it is reasonable 

to explore the impact of plausible extrapolation 

methods in the sensitivity analysis. An example of 

this can be found in the diacetyl risk assessment, in 

which the BW3/4 extrapolation method was compared to the EPA RGDR method for reactive gases/

vapors [NIOSH 2016].
This page intentionally left blank.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 81

6 Risk Characterization

Risk characterization is the third and final step in 

the NIOSH risk assessment process. It is the qualitative and, wherever possible, quantitative determination, including attendant uncertainties, of the 

probability of occurrence of known and potential 

adverse effects of an agent in a given organism, 

system, or population, under defined exposure 

conditions [IPCS 2004]. In environmental risk assessment, risk characterization describes the likelihood and severity of exposure-related adverse effects using information on the degree of potential 

exposure to the hazard within a population and its 

dose-response relationship with the adverse effect. 

Risk characterization in NIOSH risk assessments is 

restricted to health risks and focuses on the translation of information on the risk of workplace exposures into a basis for recommendations on limiting exposure. The process of extrapolating risks 

observed in animals in an experimental study to 

workers exposed over the course of their employment is an example of NIOSH risk characterization. In particular, NIOSH used the dose-response 

relationship observed in chronic inhalation studies 

in rats to predict lung cancer risks in humans from 

exposure to titanium dioxide to form the basis of 

recommended airborne exposure limits corresponding to increased risk in humans of about 10-3

over a working lifetime [NIOSH 2011]. That said, 

an underlying principle in NIOSH risk characterization is the preference for direct estimation of the 

risk in the affected working population, data permitting.

Risk characterization is the culmination of information gathered for the risk assessment in order 

to meet its intended purpose of informing risk 

management decisions. Risk characterization synthesizes and communicates the risk assessment 

science to a broad audience, primarily in NIOSH 

Criteria documents or Current Intelligence Bulletins 

containing RELs, RML-CAs, and alternative authoritative recommendations, such as the NIOSH 

occupational exposure banding process (see Appendix C). Thus, to the maximum extent practicable, NIOSH risk assessors follow the guiding 

principles of transparency, clarity, consistency, and 

reasonableness in risk characterization as first described by the EPA [Fowle and Dearfield 2000] (see 

Table 6-1). Above all, NIOSH risk characterization 

serves to (1) characterize and communicate the risk 

basis for NIOSH recommendations, (2) describe the 

overall confidence in this basis, and (3) provide other 

information that may assist in decision-making on 

mitigating risk.

6.1 Risk Definitions Common to 

NIOSH Risk Assessment

As described in Section 1.0, occupational risk (in 

the context of this document) is simply the potential (probability) and severity of adverse health effects in workers from their exposure to workplace 

hazards. This definition is consistent with that offered by the WHO for risk assessment: risk is the 

probability of an adverse effect in an organism, system, or (sub)population caused under specified circumstances by exposure to an agent [IPCS 2004]. 

In most NIOSH assessments risk is portrayed as 

the incidence of the adverse effect (e.g., disease onset) occurring in subject(s) over a specified period, 

given that the subject(s) were disease free at the 

beginning of that period. Under this definition, it is 

a measure corresponding to an average individualspecific risk (i.e., cumulative incidence). 

Quantitative risk assessment relies on estimates of 

excess risk per unit dose obtained from the doseresponse assessment. In most settings, NIOSH 

broadly defines excess risk as the increased incidence or prevalence of the adverse effect above a 
82 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

control level or background that is attributable to 

the exposure. However, one can express excess risk 

in multiple ways, such as the following.

■ Added Risk: The difference in risk (or in the probability of a response) between subjects exposed 

and those not exposed to a hazard. For example, 

it is the increment by which the probability of adverse effect exceeds background probability, calculated as P(d)–P(0), where P(d) is the probability of 

response at dose d and P(0) is the probability of 

response at zero dose (i.e., background risk). By 

this definition, added risk is also attributable risk.

■ Extra Risk: The measure of the proportional 

increase in risk of an adverse effect adjusted 

for the background incidence of the same effect. In other words, extra risk is the added 

risk relative to the proportion of the population not responding to the background 

risk, calculated as [P(d)–P(0)]/[1–P(0)]. For 

example, dose-response analyses of quantal 

toxicologic data often use a BMR of 10% 

extra risk, which generally coincides with 

the sensitivity of animal bioassays. Extra 

risk approaches added risk with decreasing 

Table 6-1. Guiding principles of NIOSH risk characterization.*

Principle Definition Criteria for Risk Characterization

Transparency Explicitness is key in the risk 

assessment process.

Use a risk analysis plan

Describe assessment approach, assumptions, 

extrapolations, and use of models 

Describe plausible alternative assumptions

Identify data gaps 

Distinguish science from policy

Describe uncertainty 

Describe relative strength of assessment

To the extent practical, use published information 

and make data available to other researchers

Clarity The assessment itself is free from 

obscure language and is easy to 

understand.

Be brief and concise†

Use plain English (avoid jargon)†

Avoid technical terms 

Use simple tables, graphics, and equations

Consistency The risk assessment conclusions 

harmonize with those in other 

risk assessments and with other 

NIOSH actions.

Follow NIOSH policies on technical writing and 

peer review

Place assessment in context with similar risk 

assessments 

Reasonableness The risk assessment uses sound 

science and sensible judgment.

Use review by peers 

Use best available scientific information 

Use good judgment 

*Adopted from the EPA Risk Characterization Handbook [Fowle and Dearfield 2000].

†

Because complex analyses may require detailed explanations for understandability, brevity may be at odds with clarity. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 83

contributions from background. This value 

is most commonly used in risk assessment 

[Haber et al. 2018]. 

■ Relative Risk: Typically reserved for human 

studies, the relative risk is the ratio of the risk 

in the exposed population to that observed in 

those unexposed (or exposed to a lesser degree). 

Relative risk is synonymous with risk ratio. One 

typically expresses or approximates relative risk 

as rate ratios, hazard ratios, odds ratios, SIRs, 

and SMRs. Some studies report excess relative 

risk (ERR), which is relative risk (or rate) - 1.

6.2 Risk Characterization 

Framework

The direct measurement of exposure-related risk in 

the region of interest is not practical in most cases, 

given that acceptable risks dwell below the observational level in most toxicologic and epidemiologic research. Instead, risk characterization relies 

on the extension of dose-response information by 

means of one of two general approaches to using 

dose-response data: 

■ Extrapolation Approach (Risk-based): Obtain quantitative estimates of low-dose risk by 

model-based extrapolation of the risk at doses 

below the observable data. For example, a linear non-threshold (LNT) model would support extrapolation by extending a line from 

the origin of the dose-response curve (i.e., the 

point of no exposure and no excess risk) to the 

human equivalent point of departure (PoD) in 

the observable range. This approach generally 

assumes an absence of a response threshold; 

therefore, one estimates the residual risk under 

one or more exposure scenarios using probabilistic means and target risk levels (e.g., the dose 

estimated to cause a lifetime excess risk of 1 in 

10,000). Most NIOSH risk assessments have 

used this approach (Table 1-1).

■ Point of Departure/Uncertainty Factor (PoD/

UF) Approach (Health-based): Divide the estimated PoD by factors (see Section 6.3.4) that 

account for identified sources of uncertainty to 

arrive at an estimate of safe dose. Here the term 

safe implies an exposure level in which the associated risk is absent or negligible. The PoD/

UF approach is appropriate for systemic toxicity, 

in which hemostasis and adaptive mechanisms 

must be overcome before an adverse effect can 

manifest (i.e., a threshold response). In this approach, risk is not explicitly quantified (i.e., the 

dose is implicitly risk-free); however, probabilistic means may be used to quantify risk from 

exposure above the safe level. 

The concept that toxic effects have exposureresponse thresholds is fundamental to toxicology 

[Aldridge 1986; Klaassen et al. 2013; Rhomberg et 

al. 2011; Rodricks et al. 2007]. As such, chemical 

risk assessments related to occupational diseases, 

excluding cancer, have mostly used a PoD/UF approach. In contrast, early risk assessments of cancer from ionizing radiation exposure recognized 

that induced mutagenesis exhibited effects that 

were proportional to dose and absent of a doseresponse threshold [NRC 1956; Sievert and Failla 

1959]. Continued research into low-dose radiation 

effects led to the now widely accepted notion of the 

LNT dose-response for radiocarcinogenesis [NRC 

2006; UNSCEAR 2015]. Assuming the LNT doseresponse was also applicable to chemical carcinogenesis, the EPA adopted LNT extrapolation in its 

risk assessments of carcinogens beginning in the 

late 1970s [Albert et al. 1977]. 

Refinements in risk assessment methods since the 

1970s have placed more emphasis on MoA evaluations, given that some carcinogens exhibit nonlinearity at low doses. In fact, many of the factors 

contributing to nonlinearity in the dose-response 

curve at low doses for noncarcinogenic agents 

(e.g., clearance pathways, cellular defenses, and repair processes) may also support nonlinearity at low 

doses for some carcinogens. Conversely, some noncancer endpoints may be better suited to risk extrapolation, with an allowance for a dose-response 

that appears LNT at low doses. In fact, reference 

points (e.g., occupational exposure limits, known as 

OELs) derived in PoD/UF assessments often reside 

at levels well below observation. Thus, quantitative 
84 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

substantiation of the absence of harm is not possible. Large interindividual variability in the low-dose 

threshold of a noncancer endpoint (i.e., widely varying susceptibility) can result in a dose-response that 

approaches linearity at low dose. 

Exceptions to the existing cancer/noncancer dichotomy have prompted calls for the harmonization of 

risk characterization methods [Barton et al. 1998; 

Crump 2011; Crump et al. 1997; NRC 2009; Rhomberg et al. 2011; White et al. 2009]. In response, some 

researchers have suggested a unified approach to risk 

characterization that is either extrapolation [NRC 

2009; White et al. 2009] or PoD/UF-based [Crump 

2011; Crump et al. 1997; Gaylor et al. 1999] or some 

combination of the two [Baird et al. 1996; Chen et 

al. 2007; Chiu and Slob 2015]. Still others have suggested a framework continuing to allow both approaches [Barton et al. 1998; Rhomberg et al. 2011]. 

For example, Rhomberg et al. [2011] suggested that 

the cancer/noncancer paradigm is valid in most 

cases, yet acknowledged that exceptions may occur; 

therefore, the choice between approaches should be 

based on the degree of compatibility of the methods 

on a case-by-case basis. 

The approach used for risk characterization can 

have major impact on its findings; therefore, its selection is a critical decision point in NIOSH quantitative risk assessment. Unfortunately, risk assessors 

often face a difficult choice given sparse MoA data, 

uncertainty in the dose-response at very low doses, 

and other limitations (such as measurement error 

and interindividual variability in the dose-risk 

relationship). For example, uncertainty in the presence of a toxicity threshold is unavoidable; therefore, risk-extrapolation may predict residual risk 

when the true (but unknown) risk is zero, and the 

POD/UF approach may underestimate risk in the 

absence of a response threshold. Therefore, methods can appear interchangeable and a preference for 

one over the other can appear less than objective. 

To avoid inconsistency among risk assessments 

and to ease transparency, NIOSH has developed a 

risk characterization framework that incorporates 

decision logic for systematically selecting a strategy 

for using extrapolation and PoD/UF approaches 

in conjunction with current science and NIOSH 

policy (Figure 6-1). NIOSH risk assessors are encouraged to follow this logic for planning and conducting risk characterization. NIOSH realizes that 

exceptions to the framework are possible, given nuances in every risk assessment; therefore, risk assessors are discouraged from forcing a fit. Above all, a 

WoE approach for evaluating and applying MoA 

must be the foundation of any method selected for 

risk characterization. 

For example, linear extrapolation is the default 

approach for characterizing the general class of 

chemical carcinogens. However, it is plausible that 

a non-genotoxic or non-DNA-reactive carcinogen 

may have sufficient MoA information to support 

nonlinearity or even a practical response threshold 

at low doses. In this instance, it may be more appropriate to use nonlinear extrapolation (or even 

a PoD/UF approach) rather than LNT. In addition, 

data availability is an important factor for deciding 

on a risk characterization approach. For example, a 

PoD can be determined from a NOAEL or LOAEL 

even if data are insufficient to quantify the doseresponse relationship. Furthermore, this framework is applicable only to NIOSH risk assessments; 

factors used in its development may be unrelated 

to, or may weigh differently on, risk characterization conducted elsewhere.

6.2.1 Carcinogens

NIOSH has separately published its policy on the 

classification and risk characterization of chemical 

carcinogens [NIOSH 2017]. The policy establishes 

the process for determining a Risk Management 

Limit for Carcinogens (RML-CA), which is an 

exposure limit for chemical carcinogens, preferably based on a target risk; this represents a starting place for controlling exposures.§

 In forming the 

policy, NIOSH surmised that for most chemical carcinogens there is no known safe level of exposure; 

§

Prior to the policy [NIOSH 2017], NIOSH expressed limits on carcinogens as RELs. 
NIOSH CIB 69 

• NIOSH Practices in Occupational Risk Assessment 85

 

Figure 6-1. Framework for risk characterization of a potential chemical hazard. The steps enclosed by the dashed line are explicitly addressed in the NIOSH Chemical Carcinogen Policy [NIOSH 2017]. 
86 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

therefore, an extrapolation approach is generally required for characterizing carcinogenic risk 

at low doses. However, there is emerging scientific 

evidence that some carcinogens may have enough 

MoA information to allow conclusion that the doseresponse is nonlinear at low doses. In these situations, simple linear extrapolation may substantially 

overestimate cancer risk. Thus, the policy allows for 

nonlinear extrapolation for chemical carcinogens 

with enough MoA evidence supporting nonlinear 

dose-response relationships at low doses. 

As shown in Figure 6-1, the NIOSH Chemical Carcinogen Policy does not explicitly address using a 

PoD/UF approach for carcinogens with threshold 

responses [NIOSH 2017]. Instead, when data are 

adequate for modeling the dose-response, NIOSH 

prefers addressing potential threshold responses in 

carcinogens by using a sublinear, but non-threshold, 

mathematical model. Nevertheless, the policy does 

not expressly disallow other methods, including a 

PoD/UF approach, when data support an alternative approach. 

NIOSH recognizes three general types of carcinogens, based on the weight of MoA evidence for carcinogenesis (adapted from Streffer et al. [2004]):

■ Genotoxic carcinogens consistent with LNT.

These include all mutagens and most directacting (DNA-reactive) genotoxic carcinogens, 

separated into two subgroups: (1) those in 

which the WoE supports LNT (e.g., ionizing 

radiation and vinyl chloride) and (2) those in 

which mechanisms are uncertain or generally 

unsupportive of a threshold at low doses (e.g., 

acrylamide, acrylonitrile). For the latter, LNT 

is used as a default health-protective measure. 

For example, acrylamide is clearly genotoxic 

at the chromosome level and is metabolized 

through the cytochrome P450 CYP2E1 pathway to a potentially reactive metabolite; therefore, it has generally been treated as a directacting mutagen [Streffer et al. 2004]. There is 

a growing body of evidence of nonlinearity 

in the slope of the response for acrylamide; 

however, underlying genotoxic mechanisms 

are still poorly understood [Maier et al. 2012; 

Shipp et al. 2006]. Until the WoE is supportive 

of an alternative approach, risk characterization for acrylamide would likely utilize LNT 

extrapolation as a primary risk characterization approach. 

■ Genotoxic carcinogens inconsistent with 

LNT. These genotoxic carcinogens have adequate evidence of underlying mechanisms 

suggesting nonlinearity in the response at low 

doses. These carcinogens are primarily nonDNA-reactive substances in which the interaction is with proteins or protein systems at 

the chromosome level (e.g., aneugenicity or 

clastogenicity). These substances have a weak 

potency for direct mutagenicity relative to secondary mechanisms. This group also includes 

those substances in which carcinogenesis is 

associated with repetitive local tissue damage 

and cell proliferation (e.g., chloroform and vinyl acetate). For example, evidence suggests 

that chloroform is a substance in which carcinogenicity occurs through cytolethality and regenerative cell proliferation. As such, the EPA 

considers chloroform to be a probable human 

carcinogen that is not likely to cause cancer 

in humans without exposure conditions that 

cause cell death and regrowth (i.e., a practical 

threshold exists) [EPA 2001]. 

■ Non-genotoxic carcinogens that act solely 

through secondary mechanisms (e.g., endocrine modification, tumor promotion, immunosuppression, and inflammation). These are 

non-genotoxic carcinogens that have widely 

varying MoA and tissue specificity but generally act through perturbation of cellular structures that can result in genomic instability. 

These processes tend to exhibit a threshold and 

be complex (i.e., requiring alteration of multiple pathways for cancer induction); therefore, non-genotoxic carcinogens are generally 

thought to be best described by sublinear or 

threshold responses at low doses [Hernández 

et al. 2009]. For example, TiO2 is not directly 

genotoxic; however, a plausible mechanism for 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 87

carcinogenesis is a nonchemical interaction 

of inhaled particles with the cells in the lung, 

causing persistent inflammation and mediation by secondary genotoxic processes. This 

complex mechanism may explain the sublinear 

carcinogenic response observed at low doses, 

as described in the NIOSH risk assessment 

[NIOSH 2011].

Genotoxic carcinogens consistent with LNT extrapolation have been the most commonly observed in risk assessment. The other types of carcinogens form a much smaller subset that either are 

non-genotoxic or have genotoxicity that is limited 

compared to other mechanisms (e.g., cell proliferation); therefore, nonlinear extrapolation may be 

preferred in risk characterization. For example, Bevan and Harrison [2017] identified a small number 

of genotoxic substances that have recommended 

health-based OELs, founded on MoA evidence of 

practical thresholds (Table 6-2). Similarly, Hernández et al. [2009] estimated that non-genotoxic carcinogens comprise about 12% of substances listed 

in IARC Groups 1, 2A, and 2B. 

Figure 6-2 shows a logic diagram for choosing an extrapolation approach. This diagram is a slight modification of concepts adopted by the Scientific Committee on Occupational Exposure Limits (SCOEL) 

[Bolt and Huici-Montagud 2008]. As in the risk 

characterization framework, this diagram is a generalization that may not accurately depict the specific 

situation encountered in an actual risk assessment. 

Mechanisms of carcinogenesis are highly complex 

and vary widely among chemicals; therefore, exceptions to this diagram are inevitable. Above all, 

the risk assessor must thoroughly evaluate the evidence of nonlinearity to determine the appropriate 

course. When the information is equivocal, it may 

be informative to characterize risks by using multiple approaches.

Methods for carcinogenic risk characterization have 

varied within the risk assessment community. In 

assessments supporting regulation in the United 

States, most have estimated the carcinogenic risk 

at low doses by using LNT, with the exception 

of the EPA’s assessment of chloroform [EPA 2001]. 

In contrast, others have used a PoD/UF approach 

to derive safe levels for non-genotoxic and some 

genotoxic carcinogens [Bevan and Harrison 2017; 

Kirman et al. 2016; Pecquet et al. 2018; Seeley et al. 

2001; Thompson et al. 2016]. For example, Pecquet 

et al. [2018] determined a no-significant-risk level 

for tetrabromobisphenol A (TBBPA), using a PoD/

UF approach, based in part on MoA data supportive of a threshold in carcinogenic response. NIOSH 

carcinogenic risk assessments have exclusively used 

extrapolation by mathematical models to quantify 

risks at low doses. Of agents assessed, only TiO2

demonstrated a nonlinear response, which NIOSH 

accounted for in the dose-response modeling. The 

lack of evidence of a threshold at low doses for any 

carcinogen does not prove the absence of an exposure level at which effects are negligible or zero. 

Similarly, strong evidence of a threshold may still 

be insufficient to estimate a numerical value for exposure that is considered risk-free, given statistical 

limitations, interindividual variability, and analysis 

uncertainty [Crump 2011]. Thus, PoD/UF methods and threshold-based mathematical models are 

absent in previous NIOSH risk assessments of occupational carcinogens, but they may be viable alternatives to linear and nonlinear extrapolation in 

future assessments. 

NIOSH generally assumes carcinogenic effects from 

exposures are cumulative and irreversible; therefore, NIOSH estimates lifetime carcinogenic risk. 

In most cases, nontrivial background cancer risk is 

expected in a population (i.e., due to factors other 

than the occupational exposure); therefore, a competing risk model is preferred. NIOSH typically 

estimates risks for an array of exposure scenarios 

and uses a target risk level to recommend a limit 

on exposure to carcinogens. The target risk level 

for cancer, as stated in the NIOSH Chemical Carcinogen Policy [NIOSH 2017], is one excess cancer 

case in 10,000 workers exposed in a 45-year working lifetime (i.e., 10-4 risk). In the absence of opposing evidence, NIOSH assumes that the attributable 

risk persists up to the age at death. The age at death 

used in NIOSH risk assessments has varied over 
88 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

 

Figure 6-2. Risk characterization of chemical carcinogens using WoE (adapted from Bolt and Huici-Montagaud [2008]).
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 89

time; however, recent assessments have projected 

risks to age 85 years, based on the availability of 

stable population rates (see Section 6.3.2).

Recent examples of NIOSH risk assessments include occupational carcinogens such as hexavalent 

chromium [NIOSH 2013a] and titanium dioxide 

[NIOSH 2011]. The MoA evidence for these materials differs, supporting low-dose linear response 

modeling for hexavalent chromium and nonlinear 

dose-response modeling for titanium dioxide. Risk 

characterization for these carcinogens follows the 

framework discussed, except NIOSH used a target 

risk level of 10-3 as the risk basis for RELs of both 

materials, according to the policy in place at the time. 

6.2.2 Non-Carcinogens

The NIOSH risk characterization framework generally considers a nonmalignant disease to have a 

toxicity threshold unless there is evidence to the 

contrary. Therefore, the PoD/UF approach is preferred 

for many noncancer endpoints. However, there can 

be exceptions to this rule. For illustration, consider 

the case of a chemical in which endogenous levels 

are near threshold levels (e.g., endogenous estrogens and androgens); thus, very low exogenous exposure may appear to have a non-threshold effect 

for some endpoints. Similarly, several endocrinedisrupting chemicals have been determined to be 

hormonally active at extremely low doses, making 

an assumption of a threshold for these chemicals 

untenable [Futran Fuhrman et al. 2015]. Finally, 

the response may vary so widely within the exposed population that it is best modeled with the 

assumption that there is no practical threshold. 

Ideally, adequate MoA information would be available to support the decision on risk characterization without equivocation; however, in practice this 

is rarely the case. Instead, a decision against PoD/

UF usually follows careful consideration of the nature and severity of the adverse effect, its observed 

association with the agent of interest, and the 

Table 6-2. Examples of genotoxic carcinogens with evidence against LNT response

 (adapted from Bevan and Harrison [2017]).

Substance Primary Cancers

Mechanism and OEL 

Supporting Document*

Cadmium (and cadmium 

compounds)

Lung, kidney, and 

prostate

Indirect genotoxic MoA characterized 

by different and non–mutually exclusive 

mechanisms, including oxidative DNA damage, 

induction of oxidative stress, inhibition of DNA 

repair, and deregulation of cell proliferation 

[SCOEL 2010]

Formaldehyde Nasopharynx Genotoxic amplification (at low exposures) by 

chronic proliferative processes caused by the 

cytotoxic effects [SCOEL 2008]

Nickel compounds (water 

soluble)

Lung, nasal cavity, and 

paranasal sinuses

Indirect genotoxic MoA characterized by 

interference with DNA repair systems and 

DNA methylation patterns, which lead to 

clastogenicity and an increased genomic 

instability [SCOEL 2011]

*All are OEL recommendations made by SCOEL, which advises the European Commission. 

Abbreviations: DNA, deoxyribonucleic acid; MoA, mode of action; OEL, occupational exposure limit; SCOEL, Scientific Committee on 

Occupational Exposure Limits.
90 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

heterogeneity of the population at risk. For example, 

occupational pneumoconioses (e.g., silicosis, asbestosis, and coal worker’s pneumoconiosis) are 

severe apical health effects in terms of disability, 

survivorship, and risk persistence. As such, lung 

diseases are among the most common noncancer 

endpoints investigated in NIOSH quantitative risk 

assessments, and most have invoked an extrapolation approach to risk characterization when quantitative dose-response data were available [NIOSH 

2016; Park et al. 2002; Park and Gilbert 2018]. An 

illustration is NIOSH’s recently completed assessment of the risks from diacetyl exposure in the 

workplace [NIOSH 2016]. Diacetyl, and some related chemicals such as 2,3-pentanedione, is used 

in the manufacture of food flavorings. Obliterative 

bronchiolitis is a rare, fibroproliferative, incurable, 

potentially fatal disease of the small airways of the 

lung linked to diacetyl exposure in some epidemiologic studies of flavoring workers. However, data 

from these studies were insufficient for direct quantification of the excess risk of obliterative bronchiolitis (i.e., the apical adverse effect) from diacetyl 

exposure. Instead, NIOSH assessed data on changes in lung function in exposed workers believed to 

precede obliterative bronchiolitis, given that respiratory obstruction is a common presentation of the 

disease. Pulmonary dysfunction observed among 

exposed diacetyl workers appeared irreversible. 

The natural history of obliterative bronchiolitis is 

highly variable, and there is little information on 

its pathology related to initiation by toxic exposure; 

therefore, a practical threshold for diacetyl toxicity is not known (although this is perhaps present 

on an individual basis). As with cancer risk assessments, NIOSH estimated airborne concentrations 

corresponding to a variety of target risk levels, while 

assuming a 45-year working lifetime. In this case, 

the risk-based REL was derived by using a target 

excess risk of one case in 1000 [NIOSH 2016]. 

6.3 Using Risk Assessment as a 

Basis for RELs or RML-CAs

NIOSH risk assessments provide the quantitative 

scientific basis for NIOSH recommendations, 

including RELs for noncancer agents and RMLCAs for carcinogens. Although the ultimate decision on a REL or RML-CA is a risk management 

decision and outside the scope of this report, it is 

important for risk assessors to understand the issues that contribute to those decisions in order to 

provide well-supported advice for the risk manager. 

Although NIOSH may develop RELs to protect 

against occupational exposures of any duration—

and, in fact, the bases of many RELs are adverse 

effects due to acute exposures—RELs (and RMLCAs) based on quantitative risk assessment usually focus on the prevention of chronic illnesses 

from longer duration exposures to lower levels 

of hazardous agents. In other words, NIOSH has 

typically conducted quantitative risk assessments 

for serious, chronic adverse effects such as cancer, 

pneumoconioses, neurological disorders, reproductive outcomes, and other exposure-related cumulative health effects. In part, this is in response 

to the NIOSH mandate in the Occupational Safety 

and Health Act of 1970 [29 USC 15] (as amended 

through January 1, 2004) to 

 “. . . develop criteria dealing with toxic 

materials and harmful physical agents 

and substances which will describe exposure 

levels that are safe for various periods of 

employment, including but not limited to the 

exposure levels at which no employee will 

suffer impaired health or functional capacities 

or diminished life expectancy as a result of his 

work experience” [29 USC 669 (a) (3)]. 

It also is in response to the codified directive to 

OSHA to ensure: 

“. . . on the basis of the best available 

evidence, that no employee will suffer 

material impairment of health or functional 

capacity even if such employee has regular 

exposure to the hazard . . . for the period of 

his working life” [29 USC 655 (b) (5)].

Because data describing health effects to workers 

exposed over a working lifetime are rare, risks are 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 91

estimated according to the guidelines described in 

this document. This includes integration of hazard 

identification and dose-response analysis. In applying these procedures, some assumptions and 

defaults are generally necessary to synthesize the 

information into risk estimates. The following sections describe the targets, defaults, and assumptions typically used in the NIOSH risk assessment 

process to provide an integrated picture of risks to 

workers.

6.3.1 Target Risk Levels

As previously discussed, model-based extrapolation in quantitative risk assessment assumes that 

any level of exposure to the agent, no matter how 

small, has an associated health risk (i.e., no response 

threshold). Complete removal of the agent, albeit 

ideally preferred, is not practical in many industrial 

settings; therefore, a continuum of exposure-related 

risk must be managed. This continuum represents 

a gradient of occupational health risks, ranging 

from high levels that are clearly unacceptable to 

extremely low levels in which efforts further reducing exposure result in a negligible reduction in risk 

[Hunter and Fewtrell 2001]¶

. The upper and lower 

boundaries of this gradient define the unacceptable

¶

The concept of acceptable risk implies a risk level that everyone will 

find acceptable. This notion is difficult to reconcile and rarely achievable given individual perceptions of risk. Nevertheless, many situations require a baseline for residual risk that is generally accepted 

[Hunter and Fewtrell 2001]. 

 

Figure 6-3. Framework for the tolerability of risk (adapted from HSE [2001]).
92 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

and broadly acceptable regions, respectively (Figure 6-3). Between these regions lies the tolerable

region, which is characterized by a general willingness to tolerate risks in the region, given assurances 

that the risk is managed to an extent that is reasonable and practical [HSE 2001; Tchiehe and Gauthier 

2017]. Here the terms reasonable and practical refer 

to using risk mitigation strategies that are proportionate to the magnitude of the risk involved [JonesLee and Aven 2011]. This is a commonly used riskreduction principle sometimes referred to as the As 

Low as Reasonably Practicable (ALARP) principle. 

Historically, NIOSH used this principle in its recommendations for exposure to carcinogens. Instead of a numerical REL, the carcinogen was given 

a “Ca” designation, which indicated that employers 

should implement substitution, engineering, work 

practice, and personal protective equipment strategies to reduce exposures as low as feasible [NIOSH 

2007; NIOSH 2017].

When there is residual risk, quantitative risk assessment will often estimate an array of risk levels for risk management purposes. For example, 

NIOSH uses quantitative risk assessment to estimate chemical exposures corresponding to risks 

ranging from one excess cancer case in 100 workers, or 10-2, to 10-6, assuming continuous workday 

exposure over a 45-year working lifetime [NIOSH 

2017]. NIOSH typically estimates the airborne 

concentration corresponding to a hazard at a single level within the tolerable region, designated 

as a target risk level. Multiple methods and principles are available for establishing risk acceptance 

criteria, and the adopted methods and principles 

will undoubtedly influence the choice of criteria 

[Vanem 2012]. Moreover, risk acceptance is often 

founded more on sociopolitical significance rather 

than science. Thus, risk acceptance (or tolerance) 

criteria are more likely to be unique to the situation 

at hand rather than be pre-defined [Rodrigues et al. 

2014; Vanem 2012]. Nevertheless, some examples 

of target risk levels are available for major hazards 

[HSE 2001; NIOSH 2017]. For example, the British Health and Safety Executive (HSE) established 

that the tolerable region for work-related fatality 

lies between an individual risk of 10-3 and 10-6 per 

annum [HSE 2001]. The HSE recommends using 

ALARP to manage risks within the tolerable range 

but toward the lower bound. 

As another example, NIOSH has established a 

target risk level for non-threshold carcinogens of 

one excess case per 10,000 workers continuously 

exposed over a 45-year working lifetime [NIOSH 

2017].** Prior to this policy, assessments have used 

a target risk of 10-3 lifetime catastrophic disease risk 

of cancer and non-malignant outcomes from occupational exposures [NIOSH 2011; NIOSH 2013a; 

NIOSH 2016]. As in the HSE, NIOSH target risk 

values have established reasonable starting places 

for risk mitigation strategies for chemical carcinogen exposure where residual risk is assumed. 

Figure 6-4 depicts a simple framework for determining target risk levels based on the relationship 

between the severity of the adverse effect and its 

probability of occurrence. For example, given a 

relationship between a catastrophic adverse effect 

and some hazardous exposure (e.g., leukemia from 

benzene exposure), the chart reveals a target level 

for a remote excess working-lifetime risk that lies 

between 10-3 and 10-5. 

The setting of target risk levels is a fundamental 

component of risk management; therefore, actions 

are primarily the responsibility of the decisionmakers and not the risk assessor. As such, a detailed discussion on the various principles in play 

for determining these levels is beyond the scope 

of this report, although discussion is available in 

several published reports [Aven 2016; HSE 2001; 

Rodrigues et al. 2014; Tchiehe and Gauthier 2017; 

Vanem 2012]. Finally, it should be clear that health 

risk is among many aspects considered to derive a 

target risk level, given that risk tolerance can depend on the combination of individual, societal, 

economic, and environmental impacts. Although 

these other factors may be considered by employers in managing risks, NIOSH quantitative risk 

**When practical, NIOSH bases its risk estimates on the 95% lower 

confidence limit of the central estimate [NIOSH 2017].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 93

assessment focuses solely on characterizing health 

risks according to its mandate; therefore, criteria for 

establishing NIOSH target risk levels to date have 

not considered costs and benefits [NIOSH 2017]. 

6.3.2 Working Lifetime and 

Persistent Risk

NIOSH currently defines a working lifetime of exposure to a chemical as an 8-hour shift, 5 days a 

week, 50 weeks a year, for 45 years of exposure (i.e., 

from age 20 to age 65). This represents the maximum amount of exposure anticipated for a worker. 

However, because the adverse effects of interest are 

typically chronic effects, the distribution of exposure over a week (or a year) does not usually affect 

the risk estimate. Therefore, whether a worker is 

exposed 4 days a week for 10 hours a day or 5 days a 

week for 8 hours a day does not usually make a difference in the final working lifetime risk estimate 

or the resulting 8-hour time-weighted average 

(TWA) REL or RML-CA. For chronic, cumulative 

hazards, NIOSH typically assumes that if the exposure duration was less than working lifetime, then 

risks would be lower than estimated. For risks that 

do not accumulate across a lifetime (for example, 

short-duration hazards or adverse effects with an 

exposure threshold), the 45-year working lifetime 

is not a relevant measure.

The exposure-related biologic insult may be irreversible for some toxicants, and the initiated toxicity pathway may progress throughout life after exposure has ended. For example, significant excess 

solid-cancer risk persists in the Japanese atomicbomb survivors 60 years after their acute exposure to ionizing radiation [Ozasa et al. 2012]. To 

account for risk persistence, NIOSH projects the 

added or extra risk used in developing the OEL 

to end of life. Recent NIOSH risk assessments assume a terminal age of 85 years. This value takes 

into account the limitations on data describing 

background rates of chronic illnesses at older 

ages. Examples of NIOSH risk assessments projecting persistent lifetime excess risk are available 

for diacetyl exposure and pulmonary impairment 

[NIOSH 2016]; nonmalignant respiratory disease 

and silica exposure [Park et al. 2002]; as well as 

lung cancer and exposure to asbestos [Stayner et 

al. 1997], hexavalent chromium [NIOSH 2013a; 

Park et al. 2004], silica [Rice et al. 2001], and cadmium [Stayner et al. 1992a; Stayner et al. 1992b].

6.3.3 Competing Risks in Projecting 

Lifetime Risk

Many exposure-related chronic illnesses present 

very late in life and have multiple risk factors other 

than occupational exposure to the agent of interest. 

Therefore, risk assessments include an accounting 

of competing risks of mortality and background 

disease rates when projecting lifetime risks. Among 

many available approaches, competing risk models 

have most commonly been accomplished by using 

actuarial methods (life-table analysis) that account 

for age-specific death rates and background disease 

incidence, under the common assumption that the 

relative risk, conditional on exposure, is independent of age [Cornfield 1957; Goldberg et al. 1956; 

NRC 1988; Zdeb 1977]. A life table provides a systematic record of the rate at which members of a 

hypothetical cohort (say, 10,000 workers who are 

‘risk free’ at beginning of working age) withdraw 

during follow-up by either death or the illness, 

based on reference mortality and incidence rates 

that vary by age. 

The life table predicts risks within age intervals that 

are conditional on survival to each age interval for 

intervals specified over the working lifetime period. 

The summation of the conditional probabilities of 

diagnoses (or death) in each interval using baseline 

disease rates provides an estimate of the lifetime 

risk in the unexposed (R0). Likewise, summing the 

conditional probabilities calculated from rates adjusted for exposure provides a corresponding risk 

measure, Rx , in the exposed. These measures can 

then be used to determine the excess lifetime risk 

[e.g., lifetime additive risk = (Rx – R0) or lifetime 

extra risk = (Rx – R0)/(1 – R0)]. NIOSH uses these 

excess lifetime risks to determine the health basis 

for the REL or RML-CA.
94 NIOSH CIB 69

• NIOSH Practices in Occupational Risk Assessment

 

Figure 6-4. Example of target risk based on adverse-effect severity and probability of occurrence.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 95

6.3.4 Application of 

Uncertainty Factors (UFs)

To address uncertainty in non-cancer adverse effects, NIOSH risk assessors have typically adjusted estimates by using UFs. The adjusted estimate 

represents a “safe” level of exposure, which is essentially the human equivalent PoD (e.g., NOAEL, 

LOAEL, or BMDL) for the critical effect, divided by 

a series of UFs. For example, NIOSH based its REL 

for occupational exposure to glycol ether on a PoD 

(i.e., NOAEL and LOAEL) and application of UFs 

[NIOSH 1991]. In this assessment, NIOSH determined a NOAEL from animal studies of reproductive and developmental toxicity (the most sensitive 

adverse effect), which was then adjusted to account 

for the animal inhalation rate, body weight, and 

fraction of the day exposed and converted to an 

equivalent exposure for humans. NIOSH applied 

two UFs: a factor of 10 for interspecies variability 

and another factor of 10 for intraspecies variability 

(i.e., a divisor of 100). Examples of NIOSH applications of UFs are in risk assessments for carbon 

nanotubes and nanofibers [NIOSH 2013b] and 

toluene diisocyanate [Daniels 2018].

In general, UFs are conservative approximations 

meant to protect workers against adverse effects 

from exposure to an agent by increasing the margin 

of safety [Dankovic et al. 2015]. UFs are preferred 

when data are insufficient to derive substancespecific or analogue-specific adjustment factors 

known as chemical-specific adjustment factors 

(CSAFs). For example, chemical-specific data on 

interspecies differences or human variability in 

toxicokinetics or toxicodynamics are useful for 

deriving CSAFs [Meek et al. 2002; WHO 2005]. 

The scientific bases for UFs have been previously 

described [Dourson et al. 1996; Dourson and Stara 

1983; Naumann and Weideman 1995]. 

6.3.4.1 Animal-to-Human Uncertainty 

Factor (UFA)

UFA accounts for the uncertainty in extrapolating 

laboratory animal data to average healthy workers. 

When using laboratory animal data to assess the 

risks to workers, the risk assessor applies UFA to 

address the differences in sensitivity between animals and humans, generally assuming that humans 

are more sensitive to substances than animals. It 

may be that humans are equally or less sensitive 

than animals for specific exposures, but unless 

this is demonstrated with experimental data, a UFA

should be applied.

The UFA comprises separate factors that account 

for toxicokinetic and toxicodynamic differences 

between species. Toxicokinetic differences arise 

because of differences in body size and metabolic 

rate. One way to address the toxicokinetic difference is by using an allometric scaling approach. 

In the absence of cross-species data on chemicalspecific metabolism, allometric scaling assumes 

that physiological parameters and basal metabolic 

rate are drivers of toxicological effects. As discussed 

previously in Section 5.6.3.1, risk assessors calculate an allometric scaling factor, or species-specific 

dosimetric adjustment factor (DAF), by

DAF = (BWa ⁄ BWh)0.25

for body weights (BW) of the animal (a) and human (h). Thus, different allometric scaling factors 

correspond to different species. Allometric scaling 

is generally applicable in most cases, except when 

the substances cause toxicity only at the portal of 

entry, such as can occur for the skin, respiratory 

tract, or gastrointestinal tract (i.e., not dependent 

on absorption or metabolic rate), and for the acute 

lethal effects [EPA 2006b]. Allometric adjustments 

replace the toxicokinetic portion of the UFA.

Other replacements for the toxicokinetic portion 

of the UFA are (1) a DAF that is applied when information is available, describing a more proximal 

(and presumably more relevant) dose; and (2) compound-related metabolic information that is available on humans and animals in the form of physiologically based pharmacokinetic (PBPK) modeling, 

provided that the model is validated and applicable 

to the specific agent. 
96 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Different agencies apply different defaults for UFA. 

For instance, WHO applies a sub-factor of 4 for 

toxicokinetics and 2.5 for toxicodynamics [WHO 

1994; WHO 2005], whereas the EPA typically uses 

equal sub-factors of √10, or approximately 3 [EPA 

2002]. The UFA of 1–10 should be applied based on 

available data on toxicokinetics and toxicodynamics. NIOSH risk assessors use the WHO values of 4 

for toxicokinetics and 2.5 for toxicodynamics.

6.3.4.2 Interindividual (Human) Variability 

Uncertainty Factor (UFH)

UFH accounts for response heterogeneity among 

the members of worker populations at risk. Like 

UFA, UFH is a result of toxicokinetic and toxicodynamic differences between the average and the 

most sensitive worker population. NIOSH considers the overall UFH to be a factor of 10, with the subfactors for toxicodynamics and toxicokinetics each 

accounting for 10 of the variability (often rounded 

to 3). UFH is modifiable with chemical-specific 

toxicokinetic information. For example, one typically adjusts the UFH to a value of 3 for chemicals 

that cause respiratory irritation upon inhalation, 

because irritation is unrelated to metabolism (i.e., a 

toxicokinetic sub-factor reduced to unity).

Some organizations consider a working population 

to be less heterogeneous than the general population and use a UFH of less than 10. For example, 

the European Chemicals Agency recommends a 

UFH of 5 to address interindividual variability in 

workers and a UFH of 10 for the general population 

when establishing derived no-effect levels [ECHA 

2008]. However, working populations might also 

include sensitive individuals such as asthmatics, 

pregnant women, older workers, and others who 

may be more susceptible. Therefore, NIOSH typically uses a factor of 10 for the overall UFH unless 

chemical-specific information is available to the 

contrary. Risk assessors must fully explain the rationale for using a factor other than 10. 

6.3.4.3 LOAEL-to-NOAEL Uncertainty 

Factor (UFL )

UFL accounts for the uncertainty in extrapolating 

from LOAELs to NOAELs. When the starting point 

for the exposure level of concern calculation is a 

LOAEL, an additional UFL between 3 (for minimal 

toxicological severity, such as fatty liver) and 10 

(for severe effects, such as hepatic necrosis) should 

be applied to estimate a dose where no adverse effect would occur [Dourson et al. 1996; Dourson 

and Stara 1983; Naumann and Weideman 1995].

When the starting point is a NOAEL or a BMDL, 

no additional UFL is required and the UFL value 

should be unity (equal to one). However, risk assessors might apply a UF >1 in certain cases, such 

as (1) with a poor-quality study in which very few 

animals and doses are used or (2) when very severe 

effects occur at the slightly higher next dose, which 

is the LOAEL. 

6.3.4.4 Shorter-Term-to-Longer-Term 

Uncertainty Factor (UFS )

Ideally, data from long-term (chronic) animal toxicology studies are available to estimate lifetime 

excess risks of chronic disease in humans. In practice, however, data may be limited to those from 

shorter-than-lifetime bioassays (e.g., two years 

for mice and rats). In these cases, a shorter-term 

to longer-term uncertainty factor (UFS) may be 

necessary to adjust for differences in duration of 

exposure. The UFS (also known as a subchronic to 

chronic factor) assumes that an effect observed at 

subchronic exposure levels will be seen at lower 

levels of chronic exposure [Dourson et al. 1996]. 

An exception would be evidence that risk (i.e., the 

incidence or severity) is unrelated to exposure duration or fully characterized by the shorter-term 

study. For example, some effects like sensory irritation of the skin/respiratory tract and effects 

caused by a reactive metabolite may not increase 

with duration. In these circumstances, additional correction using UFS >1 may be unwarranted 

[Dankovic et al. 2015].

Typically, a factor from 3 to 10 is applied for subchronic to chronic extrapolation, with allowance 

for factor modifications using chemical-specific 

experimental data. Extrapolation to chronic exposure from subacute (28-day) or acute (<24-hour) 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 97

studies is not generally recommended, although 

prediction of chronic effects from short-term studies is an active area of research. 

6.3.4.5 Database Inadequacy (Incomplete 

Data) Uncertainty Factor (UFD )

UFD accounts for the inability of the available toxicity database to address all likely adverse effects in 

humans. Evaluation of the total toxicological database should address whether the derived exposure 

level of concern is protective enough against all potential adverse outcomes for the substance. 

Risk assessors apply a UFD (1 to 10) in the absence 

of sufficient data. For example, if preliminary data 

indicate some evidence of neurotoxic/immunotoxic effects, then the absence of a detailed study 

evaluating neurotoxicity and immunotoxicity may 

be grounds for a UFD >1. The use of this factor is 

infrequent; NIOSH has not applied UFD in any of 

its risk assessments to date. 

6.3.4.6 The Composite Uncertainty 

Factor (UFC )

Generally, the product of individual UFs yields 

the overall composite factor (UFC ). Typically, when 

a UF of 10 is used, convention says that 10 is approximately 3.16, which is rounded to 3. However, 

when multiplying UFs of 3 and 3, the result used 

should be 10 and not 9. A risk estimate incorporating UFS of all five types should trigger concern 

over database sufficiency and appropriateness of 

risk characterization. Multiplying several factors 

could result in an overly conservative UFC value 

because the set of factors may not be independent 

[Calabrese and Gilbert 1993]. In general, the UFC

value is usually less than 300. A UFC value greater 

than 300 should be questioned. In rare instances, 

the UFC may be greater than 300 but should not 

exceed 3,000 when providing a basis for a REL.

6.3.4.7 Alternatives to Uncertainty Factors

In most situations, the use of standardized UFS

provides an appropriately conservative margin of 

safety against underestimation of the health risk 

from a hazardous agent. Occasionally, however, 

the product of several default factors can lead to 

incongruous overestimation of risk. Alternatives 

to applying UFS are available. Most recently, probabilistic methods for adjusting for uncertainty 

have been proposed [Chiu et al. 2018; Simon et 

al. 2016; Swartout et al. 1998]. Stemming from its 

review of the process used by the EPA to derive 

toxicity values for IRIS, the National Research 

Council (NRC) of the National Academies recommended a Bayesian hierarchical modeling approach as a means to modernize the use of UFS

in risk assessment [NRC 2014]. In general, the 

NRC proposed accounting for overall uncertainty by representing UFC as a Bayesian lognormal 

prior distribution to combine with a lognormal 

distribution representing uncertainty in the PoD. 

Simon et al. [2016] refined the NRC method to 

allow for simultaneous adjustment of individual 

factors rather than adjusting the UFC . Simon et al. 

[2016] provided several examples that cover most 

situations risk assessors may encounter. Overall, 

probabilistic methods tend to reduce the size of 

uncertainty adjustment compared to the standard 

UF approach, with the magnitude of the difference depending on the number of UFS and complexity of the assessment. 

6.3.5 Characterizing Multiple 

Outcomes

Many hazardous agents are associated with more 

than one adverse effect. In general, NIOSH risk 

assessment focuses on hazards with the greatest 

consequences affecting the largest group of workers and that have sufficient data to quantify occupational risk. Candidate endpoints arise from 

hazard identification and dose-response analyses 

that provide valid estimates of the utmost unmitigated risk associated with the hazard (e.g., per unit 

exposure to a hazardous chemical agent). Even so, 

a clear distinction in risk among some important 

stressor-outcome relationships may be lacking; therefore, risk characterization can involve comparisons 

of multiple endpoints that inform risk management 
98 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

decisions on which stressor-outcome relationship 

best supports the intended purpose of the risk assessment and end-user needs. NIOSH typically accomplishes this by drawing on the scientific judgement of a multidisciplinary team and through input 

from stakeholders. In basing a REL, NIOSH generally seeks health endpoints that are: 1) most relevant 

to risk management goals (i.e., the risk assessment 

purpose), 2) most specific to the stressor(s) of interest (e.g., the chemical exposure); and 3) most sensitive to apical adverse effects of interest. 

As a NIOSH example, recall that the apical health 

effect for diacetyl exposure is obliterative bronchiolitis (OB). Human data on the association between 

diacetyl exposure and OB were limited; therefore, 

NIOSH examined the associations between diacetyl 

exposure and several different markers of human 

lung function, assuming each may indicate future 

onset of OB. NIOSH presented all analyses in its 

published risk assessment [NIOSH 2016]. For the 

REL, NIOSH relied primarily on the human data 

showing significant exposure-associated reductions 

in the FEV1/FVC ratio and percent predicted FEV1

used in multiple outcome definitions. Rather than 

selecting data from a single outcome, NIOSH considered the range of estimated risk across outcome 

definitions and determined the REL accordingly. 

When multiple outcomes arise from apparently independent mechanistic origins, it may be appropriate to consider aggregating risk to base the REL. 

To illustrate, Park [2018] estimated excess lifetime 

risk from occupational exposure to metalworking 

fluids by combining data from published studies reporting excess cancer mortality from one of 

12 tumor sites. Park [2018] reported attributable 

cancer deaths (assuming 10 years lag) of 0.48 per 

thousand person-years at age 60 after 40 years of 

work from 1.0 mg/m3

-year exposure to metalworking fluids, with excess cancer in five sites (larynx, 

esophagus, brain, female breast, and uterine cervix) contributing over 90% of the aggregate risk. 

Mortality from cancer of the larynx was the largest 

contributor, resulting in nearly 40% of the attributable risk. 

6.4 Special Considerations 

When Developing a ShortTerm Exposure Limit (STEL)

In some cases, the available health effects data 

may elicit a concern for short-term exposure limits (STELs). For example, if peak exposures increase 

CNS symptoms, asthma attacks, or other acute-onset 

health effects, the data may be informative for developing STELs. Risk assessors should consider 

evidence of peak exposures causing specific health 

effects when evaluating the need for a STEL. For 

example, if the 8-hour TWA REL is 1 ppm and 

peak exposures at 25 ppm cause nasal and eye irritation, it is prudent to consider those data in developing a STEL. A NIOSH STEL is typically defined 

as a 15-minute TWA exposure that should not be 

exceeded at any time during a workday. Without a 

STEL, in this case, workers exposed for 15 minutes 

to 32 ppm and zero exposure for the remainder of 

the day are compliant with the REL (i.e., 32 ppm × 

0.25 hour/8 hours = 1 ppm) but still receive exposures related to adverse effects. If there is good doseresponse data at levels of concern for acute adverse 

effects, it is possible to conduct a quantitative risk 

assessment to support a numerical STEL. More often, quantitative data on the effects of peak exposures are not available. In those cases, a concern for 

acute exposures may be supported by data or a plausible concern may exist based on MoA, analogous 

chemicals, or other considerations. 

A simple way of testing this is to multiply the TWA 

REL or RML-CA by 32 (8 hours/workday ÷ 0.25 

hours). One should estimate a STEL if the resulting exposure would elicit concern for short-term 

effects. For example, consider a REL based on 

quantitative risk assessment of 10 ppm. If adverse 

effects are evident for the same chemical at shortterm exposure to 320 ppm, then estimate a STEL. 

If there are quantitative data showing health effects 

from short-term exposures (for example, respiratory irritation after exposure to 200 ppm for 10 minutes), that should be used to inform or establish the 

STEL. Alternatively, if there is a concern for shortterm exposures but no data for quantitative assess-
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 99

ment, then a STEL may be determined based on industrial hygiene practice (for example, STEL = 5 × 

8-hour TWA REL). This provides a maximum peak 

exposure that serves to both reduce peak exposures 

and reduce overall TWA exposure. 

6.5 Addressing Uncertainty 

and Variability

NIOSH risk estimates and RELs are most useful 

when combined with a reasonable understanding 

of the magnitude of the attendant uncertainty and 

variability. In risk assessment, the term variability 

describes the spread of true values of an estimated 

quantity (e.g., risk per unit dose) within the specified target population, given heterogeneity in the 

response in that population. Once a population is 

identified, variability is an irreducible property of 

the true distribution of the quantity in that population [Nayak and Kundu 2001]. In contrast, “uncertainty” refers to a lack of precise knowledge on 

the true dose-risk relationship between the agent 

and the adverse effect caused by the randomness 

(e.g., sampling error) and incompleteness of data. 

For example, imperfect knowledge of the relationships among components of a system being modeled (i.e., model uncertainty) is usually a dominant 

source of estimate uncertainty. Uncertainty and 

variability are unavoidable consequences in human health risk assessment. Moreover, uncertainty is pervasive in each assessment step; therefore, 

summed overall, the propagated cumulative uncertainty can become quite large. 

Appendix B provides details on common sources 

of uncertainty in NIOSH risk assessments. Briefly, 

in addition to model misspecification errors described above, some of the major sources are: 

■ Measurement error in exposure estimation 

(how much of the agent workers were exposed 

to in the study) 

■ The impact of exposure timing and limits on 

observation (e.g., projecting working lifetime 

risk from less than lifetime exposure) 

■ Errors in health effects ascertainment (how accurate the health effects data are and whether 

and how the health effects are related to the exposure of interest) 

■ Extrapolation from animal models or specific 

populations of workers to the general worker 

population (i.e., external validity)

■ Influence of mixed exposures and other risk 

factors in the study population. 

Risk assessors should evaluate these sources, to the 

extent practicable, and describe the approach taken 

to reduce their effects. The description should include an assessment of potential biases in risk estimates resulting from these sources. At a minimum, 

risk characterization should include an assessment 

of the analyst’s confidence in the estimate (e.g., 

confidence intervals or qualitative ratings), estimates of the magnitude and direction of potential 

biases, and areas of future research that will reduce 

uncertainty and improve risk estimates.

In some situations, NIOSH may conduct detailed 

quantitative analyses to present a reasonable range 

of plausible risks at a given dose. Typically, these 

analyses employ complex statistical methods, such 

as second-order distributions and two-dimensional 

Monte Carlo simulation techniques [Burmaster and 

Wilson 1996; Kelly and Campbell 2000; Nayak and 

Kundu 2001], to account for variability and uncertainty in data. However, these analyses often require 

considerable judgment on the distribution of values 

around true but unknown parameters used to derive risk estimates. There are several statistical techniques for dealing with data imperfections (e.g., 

latent variable analyses for unobserved variables, 

missing data imputation methods, and ensemble 

methods for countering model specification errors); 

however, there is no perfect remedy for data incompleteness other than obtaining more and better data. 

Nevertheless, the continued evolution of personal 

computing constantly improves the field of quantitative analyses; therefore, keeping current with uncertainty analysis methods is paramount to safeguarding best practices in NIOSH risk assessment.
100 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Appropriately communicating uncertainty and 

variability to risk managers and other stakeholders is challenging [Schulte 2003]. In fact, conveying these concepts can actually act to reduce public 

confidence in risk assessment findings [Johnson 

and Slovic 1995]. For example, Johnson and Slovic 

[1995] examined whether discussing uncertainty 

in risk assessments reduces the perceptions of risk 

and increases respect for the risk-assessing institution. They concluded from their findings that 

■ People are generally unfamiliar with uncertainty in science and in risk assessment.

■ People are more likely to recognize uncertainty 

when simply presented, although graphics had 

mixed results in communicating uncertainty.

■ People’s views appear less influenced by uncertainty manipulations than by attitudes toward 

risk, government, and authority.

■ Discussion of uncertainty in risk estimates appears to signal honesty (i.e., improved transparency).

■ Discussion of uncertainty in risk estimates may 

also signal incompetence for some.

■ The magnitude of estimated risk levels may affect views of expert knowledge (i.e., confidence 

in estimates diminishes with decreasing risk 

levels).

■ Although risk communication is beyond the 

scope of this document, risk characterization requires its consideration. Therefore, NIOSH risk 

assessors consider the findings above in developing and implementing a strategy for treating 

and portraying uncertainty and variability. 

To reiterate, uncertainty is a property of the risk 

assessment, whereas variability is a property of 

the population at risk. For a risk assessment to be 

informative on decision-making, it is imperative 

that risk characterization adequately describes the 

uncertainty and variability in its risk estimates. 

Whether describing them quantitatively or qualitatively, risk assessors should maintain a distinction 

between variability and uncertainty to improve risk 

communication [NRC 1994]. As practical, risk assessors should describe variability as the extent of 

actual risks to individuals within the affected population above and below the reference point. Risk assessors should also identify vulnerable populations 

and specific conditions of vulnerability. This information allows for tailoring risk management strategies to specific conditions or subgroups within a 

population to account for differences in susceptibility. For example, ionizing radiation exposures 

are limited in pregnant workers compared to other 

workers, to protect the more vulnerable fetus. For 

uncertainty, risk assessors should provide enough 

information to aid risk managers in judging the robustness or believability of the reference point in 

decision-making. 

Risk characterization should include a discussion on 

analysis limitations and known or suspected biases. 

Any discussion on bias, known or otherwise, should 

clearly describe its source, magnitude, and direction. For example, a common practice is to err on 

the side of worker protection to create a margin of 

safety, given incomplete information. For example, 

selecting a PoD based on the lower 95% confidence 

limit on the BMD rather than the central estimate is 

a “protective” practice common to benchmark doseresponse modeling. Risk assessors should describe 

steps to increase the safety margin in enough detail 

to inform risk managers. Conversely, it may be appropriate to describe a tendency to underestimate 

risk when comparing workers to other populations 

without accounting for selection differences (i.e., a 

bias from healthy worker effects [HWEs]). In summary, risk assessors should make clear to risk managers any tendency to underestimate or overestimate 

true risks as a part of risk characterization.

6.6 More on NIOSH RELs 

and RML-CAs

Adequate control of causative agents of occupational illness and disability is fundamental to the 

health and safety of the American workforce. To 

that end, NIOSH synthesizes relevant information on occupational hazards to formulate hazard
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 101

mitigation strategies, including publication of RELs 

and RML-CAs as discussed in this report. Preferably, these OELs stem from a quantitative assessment of the occupational risk associated with the 

hazard, although analytic and technical feasibility 

is also considered. For example, the NIOSH REL for 

occupational diacetyl inhalation exposure is 5 ppb 

and was based primarily on the findings from a 

quantitative risk assessment using epidemiologic 

data [NIOSH 2016]. From these data, NIOSH predicted that the risk of significant lung impairment 

was in the range of a target risk of 10-3 excess lifetime risk for workers exposed to 5 ppb over a 45-year 

working lifetime. 

The occupational risk to workers from exposure 

to a hazard is best characterized by a probability 

distribution rather than a point estimate, given unavoidable variability in exposure and response. 

In addition, NIOSH typically integrates both risk 

science and health policy, such as the feasibility of 

analytic methods, into deriving RELs and RMLCAs, which introduces further uncertainty (Figure 

6-5). OEL development often involves considerable 

uncertainty and generous professional judgment. 

Therefore, NIOSH recommends against treating 

RELs and RML-CAs as “bright lines” between safe 

and unsafe exposure for all workers. Instead, these 

OELs are better described as levels on the dose-risk 

continuum prompting evaluation and control (i.e., 

risk management). 

NIOSH recommends that in adopting a risk management strategy, decision-makers consider its 

recommendations on exposure levels, including 

their basis and the magnitude of occupational risk 

and attendant uncertainties, as well as competing 

risks from substitution or hazard controls. Fundamental to this strategy, employers should consider 

continued improvement in controls until exposure 

levels below the REL or RML-CA are confidently 

attained. However, actual exposure situations can 

vary widely; therefore, information used by risk 

managers in assessing the reasonableness and/or 

practicality of implementing hazard control strategies can differ by situation. Thus, underlying any 

risk management strategy is an assurance that risk 

mitigation efforts are not disproportionate to the 

magnitude of the risk involved. In the example case 

of diacetyl, it is important to understand that the 

REL protects against an excess risk that is relatively low compared to that faced in everyday life. At 

these low risk levels, situations can arise in which 

further reduction can be extremely difficult if not 

impractical to achieve. 

The REL and RML-CA are recommendations based 

on the best available science; neither places enforceable or binding limits on exposure. Application is 

solely at the discretion of the end-user. However, 

NIOSH derives RELs and RML-CAs to be protective for most workers and in most occupational settings; therefore, in the absence of situational risk 

management, the recommended level can serve 

as an appropriate control level. As such, many industries have voluntarily adopted NIOSH RELs 

and RML-CAs as a part of their risk management 

practices. Moreover, regulatory agencies have considered available NIOSH recommendations and 

supporting information when setting enforceable 

limits on exposure to some agents (e.g., coal dust 

and silica). NIOSH RELs and RML-CAs are determined exclusively for worker protection; therefore, 

these recommendations are not directly applicable 

to the protection of consumers or members of the 

public. However, the science behind these recommendations is likely to be useful for deriving similar public health standards.

In forming its recommendations on exposure, 

NIOSH identifies uses and manufacturing operations for the given hazard to recognize effective 

control strategies and describe engineering achievability. NIOSH may also indicate when the nature 

of job activities presents a challenge to meeting the 

REL or RML-CA. NIOSH has considered feasibility 

when setting RELs; however, engineering achievability is no longer considered in setting RML-CAs 

[NIOSH 2017]. Although routine attainment of 

exposures below the NIOSH-recommended limits 

may not occur in all work settings initially, it does 

represent a reasonable objective that employers can 

work to achieve through modification of work or 
102 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

the introduction or improvement of engineering 

controls. In this way, the REL and RML-CA encourage technological improvements to limit exposures. 

For some operations, additional protective measures 

such as administrative controls and personal protective equipment may be necessary to achieve risk 

mitigation goals. 

Finally, NIOSH RELs and RML-CAs facilitate hazard communication, as NIOSH urges employers 

to disseminate related information to workers and 

customers and encourages manufacturers to convey 

this information to downstream users. NIOSH also 

requests that professional and trade associations 

and labor organizations inform their members 

about workplace hazards. This communication 

should include a description of NIOSH recommendations and the risk associated with exposures 

at controlled levels. In communicating these risks, 

it may be helpful to include context, such as risks 

from other sources encountered in the human experience.
NIOSH CIB 69 

• NIOSH Practices in Occupational Risk Assessment 103 Figure 6-5. NIOSH process for deriving a Recommended Exposure Limit (REL) or Risk Management Limit for Carcinogens (RML-CA). The go ld boxes indicate occu- pational risk assessment. The blue boxes indicate other inputs to risk management decisions. The grey box indicates risk management decisions. Each step in the process induces uncertainty and variability in the selection of the exposure limit. Adapted from Waters et al. [2015].
This page intentionally left blank.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 105

7 Conclusions

Integrating all the pieces of a risk assessment requires careful attention to the purpose of the risk 

assessment. It is important to thoroughly investigate the robustness of key assumptions used and 

provide transparency for both the main analysis 

and analyses of alternative modeling strategies and 

defaults. If innovative or unusual modeling or analytical strategies are used, it is critical that these be 

presented in a clear manner, drawing the reader’s 

attention to departures from past practice. Ideally, 

novel or unusual methods would be published in 

the peer-reviewed scientific literature before they 

are used to develop NIOSH policies expressed in a 

NIOSH numbered publication, although this may 

not always be possible. One set of questions from 

the risk assessment plan at the beginning of this 

document deserves attention: 

■ How will risks be expressed and, if in quantitative 

analysis, what are the target risk levels used? 

■ What is the support for those decisions, and are 

there reasonable alternatives? 

■ How would using any reasonable alternatives 

influence the conclusions of the risk assessment?

Among completed NIOSH quantitative risk assessments, most have examined the risk from occupational carcinogens. The risk assessment assumptions regarding cumulative exposure and chronic 

expression of cancer are well supported and have 

numerous NIOSH precedents. Non-cancer risk assessments, on the other hand, have diverse health 

impacts and exposure profiles. These require 

thoughtful discussion of the assumptions in the 

dose-response analysis. Although harmonization 

of cancer and non-cancer risk assessments is a desirable goal, it is critical to keep in mind the differences in MoA and natural history of disease when 

using the risk assessment information to derive a 

REL for non-carcinogens.

NIOSH risk assessors must take care that the resulting risk assessments are a rational and clear 

portrayal of the available information. Risk assessors must thoroughly and accurately characterize 

uncertainties in risks caused by any level of exposure to the agent. If the risks at low doses cannot 

be determined with high accuracy, precision, and 

confidence from current information, then the uncertainty characterization step of the risk assessment process should clearly reveal this fact. Results 

of uncertainty and sensitivity analyses should be 

presented that highlight the value of specific additional pieces of scientific information in resolving 

or reducing current scientific uncertainties about 

low-dose risks. When data are uncertain, risk assessors tend to make protective assumptions; however, they should be mindful that an overestimation of risk could lead to unnecessary actions that 

can be detrimental to occupational health. Thus, 

the potential for bias in either direction must be 

clearly identified to inform risk managers. Above 

all, the risk assessment must balance the protection 

of workers with the strength of the data to ensure 

that all the NIOSH recommendations are supported by sound science. 

Risk assessment science is continuously evolving, 

given a wide array of uncharacterized hazards and a 

large community of risk assessment practitioners in 

academia, industry, and government. NIOSH risk 

assessments, although purposed for worker protection, can have relevance outside of the workplace. 

Similarly, activities intended for characterizing 

risks in other populations can also inform worker 

risks. Given these conditions, overlapping activities 

are anticipated among multiple agencies or risk assessment programs. For example, a recent review 

by the United States Government Accountability 

Office (GAO) examined overlap among federal and 

state chemical toxicity assessment programs [GAO 
106 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

2014]. The GAO findings suggest there was ample 

room for improvement in risk assessment through 

shared resources. Thus, routine exchange between 

NIOSH and the risk assessment community, both 

home and abroad, is paramount to ensuring best 

practices are followed, including improved efficiency and effectiveness by reducing duplication of 

effort. For these reasons, NIOSH maintains active 

collaborations within the risk assessment community and coordinates its risk assessment activities 

with stakeholders and the public. 

Methods currently under development provide 

additional, powerful tools to assess risks to workers on the basis of limited data. Validation of these 

approaches is a critical need. Occupational risk assessment should move forward and embrace new 

methodologies, but with caution and deliberate 

evaluation of new techniques and approaches. Appendix C shows examples of methods currently under development. As these methods are validated 

and demonstrate their utility for occupational risk 

assessment, it is anticipated that NIOSH risk assessors will adopt them.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 107

8 References 

ACGIH [1985]. Particle size-selective sampling 

in the workplace: report of the ACGIH Technical 

Committee on Air Sampling Procedures. Pub No. 

0830. Cincinnati, OH: American Conference of 

Governmental Industrial Hygienists.

ACGIH [2015]. Threshold limit values for chemical substances and physical agents and biological 

exposure indices. Cincinnati, OH: American Conference of Governmental Industrial Hygienists.

Ahlborg GA Jr. [1990]. Validity of exposure data 

obtained by questionnaire. Two examples from occupational reproductive studies. Scand J Work Environ Health 16(4):284–288.

Akaike H [1974]. A new look at the statistical model 

identification. IEEE Trans Automatic Control 19: 

716–723.

Albert RE, Train RE, Anderson E [1977]. Rationale 

developed by the Environmental Protection Agency 

for the assessment of carcinogenic risks. J Natl Cancer 

Inst 58(5):1537–1541.

Aldridge WN [1986]. The biological basis and 

measurement of thresholds. Annu Rev Pharmacol 

Toxicol 26:39–58.

Allen BC, Crump KS, Shipp AM [1988]. Correlation between carcinogenic potency of chemicals 

in animals and humans. Risk Anal 8(4):531–544.

Allison PD [2010]. Survival analysis using SAS: a 

practical guide. Cary, NC: SAS Institute.

Alyea RA, Moore NP, LeBaron MJ, Gollapudi BB, 

Rasoulpour RJ [2012]. Is the current product safety assessment paradigm protective for epigenetic 

mechanisms? J Pharmacol Toxicol Methods 66(3): 

207–214.

Andersen ME, Yang RS, Clewell III HJ, Reddy MB 

[2005]. Introduction: a historical perspective of the 

development and applications of PBPK models. In: 

Physiologically based pharmacokinetic modeling: 

science and applications. Reddy MB, Yang RSH, 

Clewell HJ III, Andersen ME, eds. Hoboken, NJ: 

John Wiley & Sons, Inc.

Anderson D, Sorsa M, Waters MD [1994]. The 

parallelogram approach in studies of genotoxic 

effects. Mutat Res 313(2–3):101–115.

Anjilvel S, Asgharian B [1995]. A multiple-path model 

of particle deposition in the rat lung. Fundam Appl 

Toxicol 28(1):41–50.

Ankley GT, Bennett RS, Erickson RJ, Hoff DJ, Hornung MW, Johnson RD, Mount DR, Nichols JW, 

Russom CL, Schmieder PK, Serrrano JA, Tietge JE, 

Villeneuve DL [2010]. Adverse outcome pathways: 

A conceptual framework to support ecotoxicology 

research and risk assessment. Environ Toxicol Chem 

29(3):730–741.

ARA [2009]. Multiple-path particle dosimetry (MPPD 

2.1) model. Raleigh, NC: Applied Research Associates, Inc.

ARA [2015]. Multiple-path particle dosimetry 

(MPPD) model, v. 3.04. Raleigh, NC: Applied 

Research Associates, Inc.

Arlot S, Celisse A [2010]. A survey of cross-validation 

procedures for model selection. Statist Surv 4:40–79.

Armstrong BG [1998]. Effect of measurement error 

on epidemiological studies of environmental and 

occupational exposures. Occup Environ Med 

55(10):651–656.

Arrighi HM, Hertz-Picciotto I [1993]. Definitions, 

sources, magnitude, effect modifiers, and strategies 

of reduction of the healthy worker effect. J Occup 

Med 35(9):890–892.

Arrighi HM, Hertz-Picciotto I [1994]. The evolving 

concept of the healthy worker survivor effect. 

Epidemiology 5(2):89–196.
108 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Arts JH, Hadi M, Irfan MA, Keene AM, Kreiling 

R, Lyon D, Maier M, Michel K, Petry T, Sauer UG, 

Warheit D, Wiench K, Wohlleben W, Landsiedel R [2015]. A decision-making framework for 

the grouping and testing of nanomaterials (DF4nanoGrouping). Regul Toxicol Pharmacol 71(2 

Suppl):S1–S27.

Asgharian B, Hofmann W, Bergmann R [2001]. 

Particle deposition in a multiple-path model of the 

human lung. Aerosol Sci Technol 34(4):332–339.

Asgharian B, Owen TP, Kuempel ED, Jarabek AM 

[2018]. Dosimetry of inhaled elongate mineral particles in the respiratory tract: The impact of shape 

factor. Toxicol Appl Pharmacol 361:27–35.

Asgharian B, Price OT, Yurteri CU, Dickens C, 

McAughey J [2014]. Component-specific, cigarette 

particle deposition modeling in the human respiratory tract. Inhal Toxicol 26(1):36–47.

Ash JS, Berg M, Coiera E [2004]. Some unintended 

consequences of information technology in health 

care: the nature of patient care information systemrelated errors. J Am Med Inform Assoc 11(2):104–

112.

Atkinson AJ Jr., Colburn WA, DeGruttola VG, 

DeMets DL, Downing GJ, Hoth DF, Oates JA, Peck 

CC, Schooley RT, Spilker BA, Woodcock J, Zeger 

SL [2001]. Biomarkers and surrogate endpoints: 

preferred definitions and conceptual framework. 

Clin Pharmacol Ther 69(3):89–95.

Atkinson TM, Ryan SJ, Bennett AV, Stover AM, 

Saracino RM, Rogak LJ, Jewell ST, Matsoukas 

K, Li Y, Basch E [2016]. The association between 

clinician-based common terminology criteria for 

adverse events (CTCAE) and patient-reported outcomes (PRO): a systematic review. Support Care 

Cancer 24(8):3669–3676.

Aven T [2016]. Risk assessment and risk management: review of recent advances on their foundation. Eur J Oper Res 253(1):1–13.

Axelson O, Steenland K [1988]. Indirect methods 

of assessing the effects of tobacco use in occupational studies. Am J Ind Med 13(1):105–118.

Baan RA [2007]. Carcinogenic hazards from inhaled carbon black, titanium dioxide, and talc not 

containing asbestos or asbestiform fibers: recent 

evaluations by an IARC Monographs Working 

Group. Inhal Toxicol 19 (Suppl 1):213–228.

Baetcke K, Hard G, Rodgers I, McGaughy R, Tahan L [1991]. Alpha 2u-globulin: association with 

chemically induced renal toxicity and neoplasia 

in the male rat. Washington, D.C.: Environmental 

Protection Agency, Risk Assessment Forum.

Bailer AJ, Stayner LT, Smith RJ, Kuempel ED, Prince 

MM [1997]. Estimating benchmark concentrations 

and other noncancer endpoints in epidemiology 

studies. Risk Anal 17(6):771–780.

Baird SJ, Cohen JT, Graham JD, Shlyakhter AI, Evans JS [1996]. Noncancer risk assessment: a probabilistic alternative to current practice. Hum Ecol 

Risk Assess 2(1):79–102.

Bareinboim E, Pearl J [2013]. A general algorithm 

for deciding transportability of experimental results. J Causal Inference 1(1):107–134.

Barnes DG, Dourson M [1988]. Reference dose 

(RfD): description and use in health risk assessments. Regul Toxicol Pharmacol 8(4):471–486.

Barrow C [1989]. Generation and characterization 

of gases and vapors. In: Henderson RF, McClellam 

R, Henderson R, eds. Concepts in Inhalation Toxicology. New York: Hemisphere Publishing Corp.

Barton HA, Andersen ME, Clewell HJ [1998]. 

Harmonization: developing consistent guidelines 

for applying mode of action and dosimetry information to cancer and noncancer risk assessment. 

Hum Ecol Risk Assess 4(1):75–115.

Baumgarten M, Siemiatycki J, Gibbs GW [1983]. 

Validity of work histories obtained by interview for 

epidemiologic purposes. Am J Epidemiol 118(4): 

583–591.

Beaumont JJ, Steenland K, Minton A, Meyer S 

[1989]. A computer program for incidence density 

sampling of controls in case-control studies nested 

within occupational cohort studies. Am J Epidemiol 129(1):212–219.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 109

Becker RA, Ankley GT, Edwards SW, Kennedy 

SW, Linkov I, Meek B, Sachana M, Segner H, Van 

Der Burg B, Villeneuve DL, Watanabe H, BartonMaclaren TS [2015]. Increasing scientific confidence in adverse outcome pathways: application of 

tailored Bradford-Hill considerations for evaluating weight of evidence. Regul Toxicol Pharmacol 

72(3):514–537.

Bellmann B, Muhle H, Creutzenberg O, Dasenbrock C, Kilpper R, MacKenzie JC, Morrow P, Mermelstein R [1991]. Lung clearance and retention of 

toner, utilizing a tracer technique, during chronic 

inhalation exposure in rats. Fundam Appl Toxicol 

17(2):300–313.

Benke G, Sim M, Forbes A, Salzberg M [1997]. 

Retrospective assessment of occupational exposure 

to chemicals in community-based studies: validity 

and repeatability of industrial hygiene panel ratings. Int J Epidemiol 26(3):635–642.

Benke G, Sim M, Fritschi L, Aldred G, Forbes A, 

Kauppinen T [2001]. Comparison of occupational 

exposure using three different methods: hygiene 

panel, job, exposure matrix (JEM), and self reports. 

Appl Occup Environ Hyg 16(1):84–91.

Bertke S, Hein M, Schubauer-Berigan M, Deddens 

J [2013]. A simulation study of relative efficiency 

and bias in the nested case-control study design. 

Epidemiol Method 2(1):85–93.

Bevan RJ, Harrison PTC [2017]. Threshold and 

non-threshold chemical carcinogens: a survey of 

the present regulatory landscape. Regul Toxicol 

Pharmacol 88:291–302.

Bingham E, Cohrssen B, Powell CH [2001]. Patty’s 

toxicology. Introduction to the field of toxicology. 

Vol. 1. 5th ed. New York: John Wiley and Sons.

Bjor O, Damber L, Jonsson H, Nilsson T [2015]. 

A comparison between standard methods and 

structural nested modelling when bias from a 

healthy worker survivor effect is suspected: an 

iron-ore mining cohort study. Occup Environ Med 

72(7):536–542.

Blair A, Stewart PA [1992]. Do quantitative exposure 

assessments improve risk estimates in occupational 

studies of cancer? Am J Ind Med 21(1):53–63.

Blettner M, Sauerbrei W, Schlehofer B, Scheuchenpflug T, Friedenreich C [1999]. Traditional reviews, 

meta-analyses and pooled analyses in epidemiology. Int J Epidemiol 28(1):1–9.

Bogaards JJ, Bertrand M, Jackson P, Oudshoorn 

MJ, Weaver RJ, van Bladeren PJ, Walther B [2000]. 

Determining the best animal model for human cytochrome P450 activities: a comparison of mouse, 

rat, rabbit, dog, micropig, monkey and man. Xenobiotica 30(12):1131–1152.

Bogdanffy MS, Daston G, Faustman EM, Kimmel 

CA, Kimmel GL, Seed J, Vu V [2001]. Harmonization of cancer and noncancer risk assessment: proceedings of a consensus-building workshop. Toxicol Sci 61(1):18–31.

Bogdanffy MS, Jarabek AM [1995]. Understanding mechanisms of inhaled toxicants: implications 

for replacing default factors with chemical-specific 

data. Toxicol Lett 82–83:919–932.

Boice JD Jr., Cohen SS, Mumma MT, Ellis ED, 

Eckerman KF, Leggett RW, Boecker BB, Brill AB, 

Henderson BE [2011]. Updated mortality analysis 

of radiation workers at Rocketdyne (Atomics International), 1948–2008. Radiat Res 176(2):244–258.

Bolt HM, Huici-Montagud A [2008]. Strategy of 

the scientific committee on occupational exposure 

limits (SCOEL) in the derivation of occupational 

exposure limits for carcinogens and mutagens. 

Arch Toxicol 82(1):61–64.

Bolton RE, Vincent JH, Jones AD, Addison J, Beckett ST [1983]. An overload hypothesis for pulmonary clearance of UICC amosite fibres inhaled by 

rats. Br J Ind Med 40(3):264–272.

Bond GG, Bodner KM, Sobel W, Shellenberger 

RJ, Flores GH [1988]. Validation of work histories obtained from interviews. Am J Epidemiol 

128(2):343–351.

Bonner JC, Silva RM, Taylor AJ, Brown JM, 

Hilderbrand SC, Castranova V, Porter D, Elder 
110 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

A, Oberdörster G, Harkema JR, Bramble LA, Kavanagh TJ, Botta D, Nel A, Pinkerton KE [2013]. 

Interlaboratory evaluation of rodent pulmonary 

responses to engineered nanomaterials: the NIEHS 

Nano GO Consortium. Environ Health Perspect 

121(6):676–682.

Boobis AR, Cohen SM, Dellarco V, McGregor D, 

Meek ME, Vickers C, Willcocks D, Farland W 

[2006]. IPCS framework for analyzing the relevance of a cancer mode of action for humans. Crit 

Rev Toxicol 36(10):781–792.

Boobis AR, Doe JE, Heinrich-Hirsch B, Meek ME, 

Munn S, Ruchirawat M, Schlatter J, Seed J, Vickers 

C [2008]. IPCS framework for analyzing the relevance of a noncancer mode of action for humans. 

Crit Rev Toxicol 38(2):87–96.

Borm P, Cassee FR, Oberdörster G [2015]. Lung 

particle overload: old school—new insights? Part 

Fibre Toxicol 12(1):10.

Bourbonnais R, Meyer F, Theriault G [1988]. Validity of self reported work history. Br J Ind Med 

45(1):29–32.

Boxenbaum H [1982]. Interspecies scaling, allometry, physiological time, and the ground plan 

of pharmacokinetics. J Pharmacokinet Biopharm 

10(2):201–227.

Bravata DM, Olkin I [2001]. Simple pooling versus combining in meta-analysis. Eval Health Prof 

24(2):218–230.

Brenner DJ, Little JB, Sachs RK [2001]. The bystander effect in radiation oncogenesis: II. A quantitative model. Radiat Res 155(3):402–408.

Breslow N [1990]. Biostatistics and Bayes. Stat Sci 

5(3):269–284.

Breslow NE, Day NE [1980]. Statistical methods in 

cancer research. Volume I—The analysis of casecontrol studies. IARC Scientific Publication No. 32. 

Lyon, France: International Agency for Research 

on Cancer.

Breslow NE, Day NE [1987]. Statistical methods in 

cancer research. Volume II—The design and analysis 

of cohort studies. IARC Scientific Publication No. 

82. Lyon, France: International Agency for Research on Cancer.

Breslow NE, Lubin JH, Marek P, Langholz B [1983]. 

Multiplicative models and cohort analysis. J Am 

Stat Assoc 78(381):1–12.

Brown CC, Koziol JA [1983]. Statistical aspects of 

the estimation of human risk from suspected environmental carcinogens. SIAM Review 25(2):151–181.

Brown JS, Gordon T, Price O, Asgharian B [2013]. 

Thoracic and respirable particle definitions for human health risk assessment. Part Fibre Toxicol 10:12.

Brown RP, Delp MD, Lindstedt SL, Rhomberg LR, 

Beliles RP [1997]. Physiological parameter values 

for physiologically based pharmacokinetic models. 

Toxicol Ind Health 13(4):407–484.

Buckland ST, Burnham KP, Augustin NH [1997]. 

Model selection: an integral part of inference. Biometrics 53(2):603–618.

Buckley JP, Keil AP, McGrath LJ, Edwards JK 

[2015]. Evolving methods for inference in the 

presence of healthy worker survivor bias. Epidemiology 26(2):204–212.

Budtz-Jørgensen E, Keiding N, Grandjean P [2001]. 

Benchmark dose calculation from epidemiological 

data. Biometrics 57(3):698–706.

Burello E, Worth AP [2011]. QSAR modeling of nanomaterials. Nanomed Nanobiotechnol 3:298–306.

Burmaster DE, Wilson AM [1996]. An introduction to second-order random variables in human 

health risk assessments. Hum Ecol Risk Assess 

2(4):892–919.

Burnham KP, Anderson DR [2002]. Model selection and multimodel inference; A practical information-theoretic approach. 2nd ed. New York: 

Springer-Verlag.

Burnham KP, Anderson DR [2004]. Multimodel 

inference: understanding AIC and BIC in model 

selection. Sociol Methods Res 33(2):261–304.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 111

Burnum JF [1989]. The misinformation era: the

fall of the medical record. Ann Intern Med 110(6): 

482–484.

Burstyn I, Hamra GB, Morel Symons J [2015]. 

Bayesian approach to ‘‘healthy worker hire effect’’ 

in standardized mortality ratio analysis. J Occup 

Environ Med 57(12):1311–1314.

Calabrese EJ, Gilbert CE [1993]. Lack of total independence of uncertainty factors (UFs): implications for the size of the total uncertainty factor. 

Regul Toxicol Pharmacol 17(1):44–51.

Calvert GM, Luckhaupt SE, Sussell A, Dahlhamer 

JM, Ward BW [2013]. The prevalence of selected 

potentially hazardous workplace exposures in the 

US: findings from the 2010 National Health Interview Survey. Am J Ind Med 56(6):635–646.

Cameron HM, McGoogan E [1981]. A prospective 

study of 1152 hospital autopsies: I. Inaccuracies in 

death certification. J Pathol 133(4):273–283.

Campbell DT, Stanley JC [1963]. Experimental and 

quasi-experimental designs for research. In: Handbook of research on teaching. Gage NL (Ed.). Chicago, IL: Rand McNally.

Carpenter DO, Arcaro K, Spink DC [2002]. Understanding the human health effects of chemical mixtures. Environ Health Perspect 110(Suppl 1):25.

Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu 

CM [2006]. Measurement error in nonlinear models: a modern perspective. 2nd ed. Boca Raton, FL: 

Chapman & Hall/CRC.

Chamblee RF, Evans MC [1982]. New dimensions 

in cause of death statistics. Am J Public Health 

72(11):1265–1270.

Checkoway H [1991]. Data pooling in occupational studies. J Occup Med 33(12):1257–1260.

Checkoway H, Pearce N, Kriebel D [2004]. Research methods in occupational epidemiology. 

Edition 2. New York: Oxford University Press.

Checkoway H, Rice CH [1992]. Time-weighted averages, peaks, and other indices of exposure in occupational epidemiology. Am J Ind Med 21(1):25–33.

Chen B, John W [2001]. Instrument calibration. In: 

Baron PA, Willeke K, eds. Aerosol measurement: 

principles, techniques, and applications. 2nd ed. 

New York: John Wiley & Sons, 627–666.

Chen CC, Chen JJ [2014]. Benchmark dose calculation for ordered categorical responses. Risk Anal 

34(8):1435–1447.

Chen JJ, Moon H, Kodell RL [2007]. A probabilistic 

framework for non-cancer risk assessment. Regul 

Toxicol Pharmacol 48(1):45–50.

Chen MJ [2001]. Development of beryllium exposure matrices for workers in a former beryllium 

manufacturing plant. Thesis. Cincinnati, OH: University of Cincinnati.

Cherrie JW, Brosseau LM, Hay A, Donaldson K 

[2013]. Low-toxicity dusts: current exposure guidelines are not sufficiently protective. Ann Occup Hyg 

57(6):685–691.

Chevrier J, Picciotto S, Eisen EA [2012]. A comparison of standard methods with g-estimation 

of accelerated failure-time models to address the 

healthy-worker survivor effect: application in a cohort of autoworkers exposed to metalworking fluids. Epidemiology 23(2):212–219.

Chiu WA, Axelrad DA, Dalaijamts C, Dockins C, 

Shao K, Shapiro AJ, Paoli G [2018]. Beyond the 

RfD: broad application of a probabilistic approach 

to improve chemical dose–response assessments 

for noncancer effects. Environ Health Perspect 

126(6):067009.

Chiu WA, Slob W [2015]. A unified probabilistic framework for dose–response assessment of 

human health effects. Environ Health Perspect 

123(12):1241–1254.

Chovil A, Sutherland RB, Halliday M [1981]. Respiratory cancer in a cohort of nickel sinter plant 

workers. Br J Ind Med 38(4):327–333.

Chu H, Wang Z, Cole SR, Greenland S [2006]. 

Sensitivity analysis of misclassification: a graphical 

and a Bayesian approach. Ann Epidemiol 16(11):

834–841.
112 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Cockburn M, Swetter SM, Peng D, Keegan THM, 

Deapen D, Clarke CA [2008]. Melanoma underreporting: why does it happen, how big is the problem, and how do we fix it? J Am Acad Dermatol 

59(6):1081–1085.

Cohen SM, Klaunig J, Meek ME, Hill RN, Pastoor 

T, Lehman-McKeeman L, Bucher J, Longfellow DG, 

Seed J, Dellarco V, Fenner-Crisp P, Patton D [2004]. 

Evaluating the human relevance of chemically induced animal tumors. Toxicol Sci 78(2):181–186.

Cole SR, Richardson DB, Chu H, Naimi AI [2013]. 

Analysis of occupational asbestos exposure and 

lung cancer mortality using the g formula. Am J 

Epidemiol 177(9):989–996.

Comber MHI, Walker JD, Watts C, Hermens J 

[2003]. Quantitative structure-activity relationships for predicting potential ecological hazard of 

organic chemicals for use in regulatory risk assessments. Environ Toxicol Chem 22(8):1822–1828.

Cooper HM [1982]. Scientific guidelines for conducting integrative research reviews. Rev Educ Res 

52(2):291–302.

Cornfield J [1957]. Estimation of the probability 

of developing a disease in the presence of competing risks. Am J Public Health Nations Health 

47(5):601–607.

Cote I, Andersen ME, Ankley GT, Barone S, Birnbaum LS, Boekelheide K, Bois FY, Burgoon LD, 

Chiu WA, Crawford-Brown D, Crofton KM, DeVito M, Devlin RB, Edwards SW, Guyton KZ, Hattis D, Judson RS, Knight D, Krewski D, Lambert J, 

Maull EA, Mendrick D, Paoli GM, Patel CJ, Perkins 

EJ, Poje G, Portier CJ, Rusyn I, Schulte PA, Simeonov A, Smith MT, Thayer KA, Thomas RS, Thomas R, Tice RR, Vandenberg JJ, Villeneuve DL, Wesselkamper S, Whelan M, Whittaker C, White R, Xia 

M, Yauk C, Zeise L, Zhao J, DeWoskin RS [2016]. 

The next generation of risk assessment multiyear 

study—highlights of findings, applications to risk 

assessment, and future directions. Environ Health 

Perspect 124(11):1671–1682.

Couch JR, Petersen M, Rice C, Schubauer-Berigan 

MK [2011]. Development of retrospective quantitative and qualitative job-exposure matrices for 

exposures at a beryllium processing facility. Occup 

Environ Med 68(5):361–365.

Coughlin SS, Chiazze L Jr. [1990]. Job-exposure 

matrices in epidemiologic research and medical 

surveillance. Occup Med 5(3):633–646.

Cowper DC, Longino Jr CF, Kubal JD, Manheim 

LM, Dienstfrey SJ, Palmer JM [2000]. The retirement migration of U.S. Veterans, 1960, 1970, 1980, 

and 1990. J Appl Gerontol 19(2):123–137.

Cox DR [1972]. Regression models and life-tables. 

J R Stat Soc Series B Stat Methodol 34(2):187–220.

Cox L Jr., Ricci P [2005]. Causation in risk assessment and management: models, inference, biases, 

and a microbial risk–benefit case study. Environ Int 

31(3):377–397.

Cox LA [2018]. Modernizing the Bradford Hill criteria for assessing causal relationships in observational data. Crit Rev Toxicol 48(8):1–31.

Crissman JW, Goodman DG, Hildebrandt PK, Maronpot RR, Prater DA, Riley JH, Seaman WJ, Thake 

DC [2004]. Best practices guideline: toxicologic 

histopathology. Toxicol Pathol 32(1):126–131.

Crump K, Allen B, Shipp A [1989]. Choice of dose 

measure for extrapolating carcinogenic risk from 

animals to humans: an empirical investigation of 

23 chemicals. Health Phys 57(Suppl 1):387–393.

Crump KS [1984]. A new method for determining allowable daily intakes. Fundam Appl Toxicol 

4(5):854–871.

Crump KS [1995]. Calculation of benchmark doses 

from continuous data. Risk Anal 15(1):79–89.

Crump KS [1998]. On summarizing group exposures in risk assessment: is an arithmetic mean or 

a geometric mean more appropriate? Risk Anal 

18(3):293–297.

Crump KS [2011]. Use of threshold and mode of 

action in risk assessment. Crit Rev Toxicol 41(8): 

637–650.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 113

Crump KS, Allen BC [1985]. Methods for quantitative risk assessment using occupational studies. 

Am Stat 39(4):442–450.

Crump KS, Chen C, Louis TA [2010]. The future 

use of in vitro data in risk assessment to set human exposure standards: challenging problems 

and familiar solutions. Environ Health Perspect 

118(10):1350–1354.

Crump KS, Clewell HJ, Andersen ME [1997]. Cancer and non-cancer risk assessment should be harmonized. Hum Ecol Risk Assess 3(4):495–499.

Daniels RD [2018]. Occupational asthma risk from 

exposures to toluene diisocyanate: a review and 

risk assessment. Am J Ind Med 61(4):282–292.

Daniels RD, Bertke S, Waters KM, SchubauerBerigan MK [2013]. Risk of leukaemia mortality 

from exposure to ionising radiation in U.S. nuclear 

workers: a pooled case-control study. Occup Environ Med 70(1):41–48.

Daniels RD, Kubale TL, Yiin JH, Dahm MM, Hales 

TR, Baris D, Zahm SH, Beaumont JJ, Waters KM, 

Pinkerton LE [2014]. Mortality and cancer incidence in a pooled cohort of U.S. firefighters from 

San Francisco, Chicago and Philadelphia (1950–

2009). Occup Environ Med 71(6):388–397.

Dankovic D, Kuempel E, Wheeler M [2007]. An 

approach to risk assessment for TiO2. Inhal Toxicol 

19(Suppl 1):205–212.

Dankovic DA, Naumann BD, Maier A, Dourson 

ML, Levy LS [2015]. The scientific basis of uncertainty factors used in setting occupational exposure 

limits. J Occup Environ Hyg 12(Suppl 1):S55–S68.

Dankovic DA, Smith RJ, Stayner LT, Bailer AJ [1993]. 

Time-to-tumour risk assessment for 1,3-butadiene 

based on exposure of mice to low doses by inhalation. IARC Sci Publ December(127):335–344.

Davies B, Morris T [1993]. Physiological parameters in laboratory animals and humans. Pharm Res 

10(7):1093–1095.

Davis JA, Gift JS, Zhao QJ [2011]. Introduction to 

benchmark dose methods and U.S. EPA’s benchmark 

dose software (BMDS) version 2.1.1. Toxicol Appl 

Pharmacol 254(2):181–191.

Delgado-Rodríguez M, Llorca J [2004]. Bias. J Epidemiol Community Health 58(8):635–641.

Dement JM, Harris RL Jr., Symons MJ, Shy CM 

[1983]. Exposures and mortality among chrysotile 

asbestos workers. Part I: exposure estimates. Am J 

Ind Med 4(3):399–419.

Dent AG, Sutedja TG, Zimmerman PV [2013]. Exhaled breath analysis for lung cancer. J Thorac Dis 

5(Suppl 5):S540–S550.

Donaldson K, Borm PJ, Oberdörster G, Pinkerton 

KE, Stone V, Tran CL [2008]. Concordance between 

in vitro and in vivo dosimetry in the proinflammatory effects of low-toxicity, low-solubility particles: 

the key role of the proximal alveolar region. Inhal 

Toxicol 20(1):53–62.

Dosemeci M, Wacholder S, Lubin JH [1990]. Does 

nondifferential misclassification of exposure always bias a true effect toward the null value? Am J 

Epidemiol 132(4):746–748.

Douglas AN, Robertson A, Chapman JS, Ruckley 

VA [1986]. Dust exposure, dust recovered from the 

lung, and associated pathology in a group of British 

coalminers. Br J Ind Med 43(12):795–801.

Dourson ML, Felter SP, Robinson D [1996]. Evolution of science-based uncertainty factors in noncancer risk assessment. Regul Toxicol Pharmacol 

24(2 Pt 1):108–120.

Dourson ML, Stara JF [1983]. Regulatory history 

and experimental support of uncertainty (safety) 

factors. Regul Toxicol Pharmacol 3(3):224–238.

Drew NM, Kuempel ED, Pei Y, Yang F [2017]. A 

quantitative framework to group nanoscale and 

microscale particles by hazard potency to derive 

occupational exposure limits: Proof of concept 

evaluation. Regul Toxicol Pharmacol 89:253–267.

Driscoll KE, Costa DL, Hatch G, Henderson R, 

Oberdörster G, Salem H, Schlesinger RB [2000]. 

Intratracheal instillation as an exposure technique 

for the evaluation of respiratory tract toxicity: uses 

and limitations. Toxicol Sci 55(1):24–35.
114 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Easterbrook PJ, Gopalan R, Berlin JA, Matthews 

DR [1991]. Publication bias in clinical research. 

Lancet 337(8746):867–872.

ECHA [2008]. Guidance on information requirements and chemical safety assessment. Helsinki, 

Finland: European Chemicals Agency.

Edwards JK, Lesko CR, Keil AP [2017]. Invited 

commentary: causal inference across space and 

time—quixotic quest, worthy goal, or both? Am J 

Epidemiol 186(2):143–145.

Eisen EA, Holcroft CA, Greaves IA, Wegman DH, 

Woskie SR, Monson RR [1997]. A strategy to reduce healthy worker effect in a cross-sectional 

study of asthma and metalworking fluids. Am J Ind 

Med 31(6):671–677.

Eisen EA, Smith TJ, Wegman DH, Louis TA, Froines J [1984]. Estimation of long term dust exposures in the Vermont granite sheds. Am Ind Hyg 

Assoc J 45(2):89–94.

Elder A, Gelein R, Finkelstein JN, Driscoll KE, 

Harkema J, Oberdörster G [2005]. Effects of subchronically inhaled carbon black in three species. I. 

Retention Kinetics, lung inflammation, and histopathology. Toxicol Sci 8(2):614–629.

Engel LW, Strauchen JA, Chiazze L, Heid M [1980]. 

Accuracy of death certification in an autopsied 

population with specific attention to malignant 

neoplasms and vascular diseases. Am J Epidemiol 

111(1):99–112.

Fox AJ, Collier PF [1976]. Low mortality rates in 

industrial cohort studies due to selection for work 

and survival in the industry. Br J Prev Soc Med 

30(4):225–230.

Fox MA, Brewer LE, Martin L [2017]. An overview of literature topics related to current concepts, 

methods, tools, and applications for cumulative 

risk assessment (2007–2016). Int J Environ Res 

Public Health 14(4).

Freedman A, Robinson S [1988]. Noninvasive magnetopneumographic studies of lung dust retention 

and clearance in coal miners, University Park, PA: 

Penn State University Press.

French NHF, Goovaerts P, Kasischke ES [2004]. Uncertainty in estimating carbon emissions from boreal forest fires. J Geophys Res Atmos 109(14):1–12.

Frey HC, Patil SR [2002]. Identification and review of sensitivity analysis methods. Risk Anal 

22(3):553–578.

Friedenreich CM [1993]. Methods for pooled 

analyses of epidemiologic studies. Epidemiology 

4(4):295–302.

Fritschi L, Siemiatycki J, Richardson L [1996]. Selfassessed versus expert-assessed occupational exposures. Am J Epidemiol 144(5):521–527.

Fuller WA [1987]. Measurement error models. New 

York: Wiley & Sons. 

Futran Fuhrman V, Tal A, Arnon S [2015]. Why 

endocrine disrupting chemicals (EDCs) challenge 

traditional risk assessment and how to respond. J 

Hazard Mater 286:589–611.

Gail MH, Benichou J [2000]. Encyclopedia of epidemiologic methods. New York: John Wiley & Sons. 

Gangwal S, Brown JS, Wang A, Houck KA, Dix DJ, 

Kavlock RJ, Hubal EA [2011]. Informing selection 

of nanomaterial concentrations for ToxCast in vitro testing based on occupational exposure potential. Environ Health Perspect 119(11):1539–1546.

GAO [2014]. Chemical assessments: agencies coordinate activities, but additional action could enhance efforts. Report to the chairman, subcommittee on oversight, Committee on Science, Space, and 

Technology, House of Representatives. GAO-14-

763. Washington, DC: United States Government 

Accountability Office.

Garcia GJ, Kimbell JS [2009]. Deposition of inhaled 

nanoparticles in the rat nasal passages: dose to the 

olfactory region. Inhal Toxicol 21(14):1165–1175.

Garcia GJ, Schroeter JD, Kimbell JS [2015]. Olfactory deposition of inhaled nanoparticles in humans. Inhal Toxicol 27(8):394–403.

Gardner MJ, Winter PD, Pannett B, Powell CA 

[1986]. Follow up study of workers manufacturing 

chrysotile asbestos cement products. Br J Ind Med 

43(11):726–732.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 115

Gargas ML, Burgess RJ, Voisard DE, Cason GH, 

Andersen ME [1989]. Partition coefficients of 

low-molecular-weight volatile chemicals in various liquids and tissues. Toxicol Appl Pharmacol 

98(1):87–99.

Gaylor DW, Kodell RL, Chen JJ, Krewski D [1999]. 

A unified approach to risk characterization. Inhal 

Toxicol 11(6–7):575–578.

Gernand JM, Casman EA [2014]. A meta-analysis of 

carbon nanotube pulmonary toxicity studies—how 

physical dimensions and impurities affect the toxicity of carbon nanotubes. Risk Anal 34(3):583–597.

Gibb HJ, Lees PSJ, Pinsky PF, Rooney BC [2000]. 

Lung cancer among workers in chromium chemical production. Am J Ind Med 38(2):115–126.

Ginevan ME, Watkins DK [2010]. Logarithmic dose 

transformation in epidemiologic dose-response 

analysis: Use with caution. Regul Toxicol Pharmacol 

58(2):336–340.

Glass DC, Pircher S, Del Monaco A, Hoorn SV, 

Sim MR [2016]. Mortality and cancer incidence in 

a cohort of male paid Australian firefighters. Occup 

Environ Med 73(11):761–771.

Gloede E, Cichocki JA, Baldino JB, Morris JB [2011]. 

A validated hybrid computational fluid dynamicsphysiologically based pharmacokinetic model for 

respiratory tract vapor absorption in the human 

and rat and its application to inhalation dosimetry 

of diacetyl. Toxicol Sci 123(1):231–246.

Goldberg ID, Levin ML, Gerhardt PR, Handy VH, 

Cashman RE [1956]. The probability of developing 

cancer. J Natl Cancer Inst 17(2):155–173.

Goldstein L, Langholz B [1992]. Asymptotic theory 

for nested case-control sampling in the Cox regression model. Ann Statist 20(4)1903–1928.

Greenland S [1987]. Quantitative methods in the 

review of epidemiologic literature. Epidemiol Rev 

9(1):1–30.

Greenland S [1996]. Basic methods for sensitivity 

analysis of biases. Int J Epidemiol 25(6):1107–1116.

Greenland S, Copas J, Jones DR, Spiegelhalter D, 

Rice K, Armstrong B, Senn S, Carpenter J, Kenward 

M, De Stavola B, Nitsch D, Nitsch D, De Stavola 

B, Muirhead CR, Hodges J, Longford NT, Gelman 

A, Draper D, Gustafson P, McCandless L, Rubin 

DB [2005]. Multiple-bias modelling for analysis 

of observational data. J R Stat Soc Ser A Stat Soc 

168(2):267–306.

Greenland S, Gustafson P [2006]. Accounting 

for independent nondifferential misclassification 

does not increase certainty that an observed association is in the correct direction. Am J Epidemiol 

164(1):63–68.

Greenland S, Longnecker MP [1992]. Methods for 

trend estimation from summarized dose-response 

data, with applications to meta-analysis. Am J Epidemiol 135(11):1301–1309.

Greenland S, Robins JM, Pearl J [1999]. Confounding and collapsibility in causal inference. Stat Sci 

14(1):29–46.

Gregoratto D, Bailey MR, Marsh JW [2010]. Modelling particle retention in the alveolar-interstitial 

region of the human lungs. J Radiol Prot 30(3):491–

512.

Greim H, Borm P, Schins R, Donaldson K, Driscoll 

K, Hartwig A, Kuempel E, Oberdörster G [2001]. 

Toxicity of fibers and particles—report of the 

workshop held in Munich, Germany, 26–27 October 2000. Inhal Toxicol 13:737–754.

Griffin PJ [1986]. Predictive value of animal toxicity studies. In: Walker SR, Dayan AD, eds. Longterm animal studies: their predictive value for 

man. Lancaster, PA: Springer Netherlands.

Grimes DA, Schulz KF [2002]. Bias and causal 

associations in observational research. Lancet 

359(9302): 248–252.

Groenwold RH, Nelson DB, Nichol KL, Hoes AW, 

Hak E [2010]. Sensitivity analyses to estimate the 

potential impact of unmeasured confounding in 

causal research. Int J Epidemiol 39(1):107–117.
116 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Guha N, Roy A, Kopylev L, Fox J, Spassova M, 

White P [2013]. Nonparametric Bayesian methods for benchmark dose estimation. Risk Anal 

33(9):1608–1619.

Haber LT, Dourson ML, Allen BC, Hertzberg 

RC, Parker A, Vincent MJ, Maier A, Boobis AR 

[2018]. Benchmark dose (BMD) modeling: current 

practice, issues, and challenges. Crit Rev Toxicol 

48(5):387–415.

Hallock MF, Smith TJ, Woskie SR, Hammond SK 

[1994]. Estimation of historical exposures to machining fluids in the automotive industry. Am J Ind 

Med 26(5):621–634.

Hamilton RF, Wu N, Porter D, Buford M, Wolfarth 

M, Holian A [2009]. Particle length-dependent titanium dioxide nanomaterials toxicity and bioactivity. Part Fibre Toxicol 6:35.

Hamling J, Lee P, Weitkunat R, Ambuhl M [2008]. 

Facilitating meta-analyses by deriving relative effect and precision estimates for alternative comparisons from a set of estimates presented by exposure 

level or disease category. Stat Med 27(7):954–970.

Han X, Corson N, Wade-Mercer P, Gelein R, Jiang J, Sahu M, Biswas P, Finkelstein JN, Elder A, 

Oberdörster G [2012]. Assessing the relevance 

of in vitro studies in nanotoxicology by examining 

correlations between in vitro and in vivo data. Toxicology 297(1–3):1–9.

Hanley J, Liddell D [1985]. Fitting relationships between exposure and standardized mortality ratios. 

J Occup Environ Med 27(8):555–560.

Hanna LM, Lou SR, Su S, Jarabek AM [2001]. Mass 

transport analysis: inhalation rfc methods framework for interspecies dosimetric adjustment. Inhal 

Toxicol 13(5):437–463.

Harrell F [2015]. Regression modeling strategies: 

with applications to linear models, logistic and ordinal regression, and survival analysis. New York: 

Springer-Verlag. 

Haseman JK, Elwell MR [1996]. Evaluation of 

false positive and false negative outcomes in NTP 

long-term rodent carcinogenicity studies. Risk 

Anal 16(6):813–820.

Hayes AW [2008]. Principles and methods of toxicology. Boca Raton, FL: CRC Press. 

Head ML, Holman L, Lanfear R, Kahn AT, Jennions 

MD [2015]. The extent and consequences of phacking in science. PLoS Biology 13(3):e1002106.

Health effects testing guidelines. 40 CFR 798 [2014].

Hedrich HJ, Bullock G [2004]. The laboratory 

mouse. London, England: Elsevier Academic Press.

Helsel DR [2005]. Nondetects and data analysis: 

statistics for censored environmental data. Hoboken, NJ: Wiley-Interscience.

Hempel S, Xenakis L, Danz M [2016]. Systematic 

reviews for occupational safety and health questions: resources for evidence synthesis. RR 1463. 

Santa Monica, CA: RAND Corporation.

Hernán MA, Vanderweele TJ [2011]. Compound 

treatments and transportability of causal inference. 

Epidemiology 22(3):368–377.

Hernández LG, van Steeg H, Luijten M, van Benthem J [2009]. Mechanisms of non-genotoxic carcinogens and importance of a weight of evidence 

approach. Mutat Res 682(2–3):94–109.

Hertzberg RC [1989]. Fitting a model to categorical 

response data with application to species extrapolation of toxicity. Health Phys 57(Suppl 1):405–409.

Hertzberg RC, Teuschler LK [2002]. Evaluating 

quantitative formulas for dose-response assessment of chemical mixtures. Environ Health Perspect 110(Suppl 6):965.

Hertz-Picciotto I, Wartenberg D, Simon R [1995]. 

Epidemiology and quantitative risk assessment: a 

bridge from science to policy. Am J Public Health 

85(4):484–493.

Higgins JP, Green S [2008]. Cochrane handbook 

for systematic reviews of interventions. Cochrane 

Book Series. Chichester, UK: Wiley-Blackwell. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 117

Hill AB [1965]. The environment and disease: association or causation? Proc R Soc Med 58:295–300.

Hinderliter PM, Minard KR, Orr G, Chrisler WB, 

Thrall BD, Pounds JG, Teeguarden JG [2010]. ISDD: 

A computational model of particle sedimentation, 

diffusion and target cell dosimetry for in vitro toxicity studies. Part Fibre Toxicol 7(1):36.

Hinds WC [1999]. Aerosol technology: properties, 

behavior, and measurement of airborne particles. 

2nd ed. New York: John Wiley & Sons.

Hoffman FO, Ruttenber AJ, Apostoaei AI, Carroll RJ, Greenland S [2007]. The Hanford Thyroid 

Disease Study: an alternative view of the findings. 

Health Phys 92(2):99–111.

Hoffmann S, Laurier D, Rage E, Guihenneuc C, 

Ancelet S [2018]. Shared and unshared exposure 

measurement error in occupational cohort studies 

and their effects on statistical inference in proportional hazards models. PLoS One 13(2):e0190792

Höfler M [2005]. The Bradford Hill considerations 

on causality: a counterfactual perspective. Emerg 

Themes Epidemiol 2:11.

Homburger F [1987]. The necessity of animal 

studies in routine toxicology. Comments Toxicol 

1(5):245–255.

Hornung RW, Greife AL, Stayner LT, Steenland NK, 

Herrick RF, Elliott LJ, Ringenburg VL, Morawetz 

J [1994]. Statistical model for prediction of retrospective exposure to ethylene oxide in an occupational mortality study. Am J Ind Med 25(6):825–836.

Hornung RW, Meinhardt TJ [1987]. Quantitative 

risk assessment of lung cancer in U.S. Uranium 

miners. Health Phys 52(4):417–430.

Hothorn LA [2014]. Statistical evaluation of toxicological bioassays—a review. Toxicol Res 3(6):418–432.

Howard J, Piacentino J, MacMahon K, Schulte P 

[2017]. Using systematic review in occupational 

safety and health. Am J Ind Med 60(11):921–929.

Howell D, Molloy S, Wilkinson K, Green E, Orchard 

K, Wang K, Liberty J [2015]. Patient-reported outcomes in routine cancer clinical practice: a scoping 

review of use, impact on health outcomes, and implementation factors. Ann Oncol 26(9):1846–1858.

Howick J, Glasziou P, Aronson JK [2009]. The evolution of evidence hierarchies: what can Bradford 

Hill’s ‘guidelines for causation’ contribute? J R Soc 

Med 102(5):186–194.

Howlader N, Noone AM, Krapcho M, Miller D, 

Bishop K, Altekruse SF, Kosary CL, Yu M, Ruhl 

J, Tatalovich Z, Mariotto A, Lewis DR, Chen HS, 

Feuer EJ, Cronin KA [2016]. SEER cancer statistics 

review, 1975–2013. Bethesda, MD: National Cancer Institute. 

Hristozov D, Zabeo A, Alstrup Jensen K, Gottardo 

S, Isigonis P, Maccalman L, Critto A, Marcomini A 

[2016]. Demonstration of a modelling-based multicriteria decision analysis procedure for prioritization of occupational risks from manufactured 

nanomaterials. Nanotoxicology 10(9):1215–1228.

HSE [2001]. Reducing risks, protecting people. HSE’s 

decision-making process. Sudbury, UK: Health and 

Safety Executive. 

Hughes EW, Martin-Body RL, Sarelius IH, Sinclair 

JD [1982]. Effects of urethane-chloralose anaesthesia on respiration in the rat. Clin Exp Pharmacol 

Physiol 9(2):119–127.

Hunter PR, Fewtrell L [2001]. Acceptable Risk. In: 

Water Quality: Guidelines, Standards and Health. 

L. Fewtrell and J. Bertram, eds. London: IWA Publishing.

IARC [2010]. IARC monographs on the evaluation 

of carcinogenic risks to humans: Carbon black, titanium dioxide, and talc. Vol. 93. Lyon, France: International Agency for Research on Cancer. 

IARC [2012]. IARC monographs on the evaluation 

of carcinogenic risks to humans: chemical agents 

and related occupations. Vol. 100F. Lyon, France: 

International Agency for Research on Cancer.

IARC [2019]. Tumour site concordance and mechanisms of carcinogenesis. IARC Scientific Publication No. 165. Lyon, France: International Agency 

for Research on Cancer.
118 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

ICRP [1994]. Human respiratory tract model for 

radiological protection. A report of a task group 

of the International Commission on Radiological 

Protection. ICRP Publication 66. Ann ICRP 24(1–

3):1–482.

Ioannidis JP [2016]. Exposure-wide epidemiology: 

revisiting Bradford Hill. Stat Med 35(11):1749–1762.

Ioannidis JPA [2008]. Why most discovered true associations are inflated. Epidemiology 19(5):640–648.

IPCS [2004]. Harmonization project document no. 

1: IPCS risk assessment terminology. International 

Programme on Chemical Safety. Geneva, Switzerland: World Health Organization.

ISO [2018]. ISO 31000: Risk Management—Guidelines. Geneva, Switzerland: International Organization for Standardization. 

Israel RA, Rosenberg HM, Curtin LR [1986]. Analytical potential for multiple cause-of-death data. 

Am J Epidemiol 124(2):161–179.

Jackson GB [1980]. Methods for integrative reviews. 

Rev Educ Res 50(3):438–460.

Jarabek AM [1995]. The application of dosimetry 

models to identify key processes and parameters 

for default dose-response assessment approaches. 

Toxicol Lett 79(1–3):171–184.

Jarabek AM, Asgharian B, Miller FJ [2005]. Dosimetric adjustments for interspecies extrapolation 

of inhaled poorly soluble particles (PSP). Inhal 

Toxicol 17(7–8):317–334.

Joffe M [1992]. Validity of exposure data derived 

from a structured questionnaire. Am J Epidemiol 

135(5):564–570.

Johnson BB, Slovic P [1995]. Presenting uncertainty in health risk assessment: initial studies of 

its effects on risk perception and trust. Risk Anal 

15(4):485–494.

Jones-Lee M, Aven T [2011]. ALARP—What does 

it really mean? Reliab Eng Sys Saf 96(8):877–882.

Julien E, Boobis AR, Olin SS, ILSI Research Foundation Threshold Working Group [2009]. The key 

events dose-response framework: a cross-disciplinary 

mode-of-action based approach to examining 

dose-response and thresholds. Crit Rev Food Sci 

Nutr 49(8):682–689.

Jurek AM, Maldonado G, Greenland S, Church TR 

[2006]. Exposure-measurement error is frequently 

ignored when interpreting epidemiologic study results. Eur J Epidemiol 21(12):871–876.

Kanj RS, Kang JL, Castranova V [2005]. Measurement of the release of inflammatory mediators 

from rat alveolar macrophages and alveolar type 

II cells following lipopolysaccharide or silica exposure: a comparative study. J Toxicol Environ Health 

A 68(3):185–207.

Kaplan S, Garrick BJ [1981]. On the quantitative 

definition of risk. Risk Anal 1(1):11–27.

Kaptchuk TJ [2003]. Effect of interpretive bias on 

research evidence. BMJ 326(7404):1453–1455.

Kauppinen TP [1994]. Assessment of exposure in 

occupational epidemiology. Scand J Work Environ 

Health 20(Special Issue):19–29.

Kavlock RJ, Austin CP, Tice RR [2009]. Toxicity 

testing in the 21st century: implications for human 

health risk assessment. Risk Anal 29(4):485–487; 

discussion 492–487.

Kavlock RJ, Schmid JE, Setzer RW, Jr. [1996]. A 

simulation study of the influence of study design 

on the estimation of benchmark doses for developmental toxicity. Risk Anal 16(3):399–410.

Kelly EJ, Campbell K [2000]. Separating variability 

and uncertainty in environmental risk assessment—

making choices. Hum Ecol Risk Assess 6(1):1–13.

Khalil F, Laer S [2011]. Physiologically based 

pharmacokinetic modeling: methodology, applications, and limitations with a focus on its role in 

pediatric drug development. J Biomed Biotechnol 

2011:907461.

Kienhuis AS, Van de Poll MCG, Wortelboer H, Van 

Herwijnen M, Gottschalk R, Dejong CHC, Boorsma A, Paules RS, Kleinjans JCS, Stierum RH, Van 

Delft JHM [2009]. Parallelogram approach using 

rat-human In vitro and rat in vivo toxicogenomics 

predicts acetaminophen-induced hepatotoxicity in 

humans. Toxicol Sci 107(2):544–552.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 119

Kirkeleit J, Riise T, Bjørge T, Christiani DC [2013]. 

The healthy worker effect in cancer incidence studies. Am J Epidemiol 177(11):1218–1224.

Kirman CR, Hughes B, Becker RA, Hays SM 

[2016]. Derivation of a no-significant-risk-level 

(NSRL) for dermal exposures to diethanolamine. 

Regul Toxicol Pharmacol 76:137–151.

Klaassen C, Amdur M, Doull J [2013]. Casarett & 

Doull’s toxicology: the basic science of poisons. 

New York: McGraw-Hill Education. 

Kleiber M [1932]. Body size and metabolism. Hilgardia 6(11):315–353.

Klimisch HJ, Andreae M, Tillmann U [1997]. A 

systematic approach for evaluating the quality of 

experimental toxicological and ecotoxicological 

data. Regul Toxicol Pharmacol 25(1):1–5.

Knol MJ, VanderWeele TJ [2012]. Recommendations for presenting analyses of effect modification 

and interaction. Int J Epidemiol 41(2):514–520.

Kreiss K, Gomaa A, Kullman G, Fedan K, Simoes 

EJ, Enright PL [2002]. Clinical bronchiolitis obliterans in workers at a microwave-popcorn plant. N 

Engl J Med 347(5):330–338.

Kreyling WG, Semmler-Behnke M, Takenaka S, 

Moller W [2013]. Differences in the biokinetics of 

inhaled nano- versus micrometer-sized particles. 

Acc Chem Res 46(3):714–722.

Kriebel D, Checkoway H, Pearce N [2007]. Exposure and dose modelling in occupational epidemiology. Occup Environ Med 64(7):492–498.

Krinke GJ ed. [2000]. The laboratory rat. London: 

Elsevier Academic Press.

Kromhout H, Oostendorp Y, Heederik D, Boleij 

JS [1987]. Agreement between qualitative exposure estimates and quantitative exposure measurements. Am J Ind Med 12(5):551–562.

Kuempel E, Smith R, Dankovic D, Stayner L [2009]. 

Rat-and human-based risk estimates of lung cancer from occupational exposure to poorly-soluble 

particles: a quantitative evaluation. J Phys Conf Ser 

151(1):012011.

Kuempel E, Tran C [2002]. Comparison of human 

lung dosimetry models: implications for risk assessment. Ann Occup Hyg 46(Suppl 1):337–341.

Kuempel ED [2000]. Comparison of human and rodent lung dosimetry models for particle clearance 

and retention. Drug Chem Toxicol 23(1):203–222.

Kuempel ED [2011]. Carbon nanotube risk assessment: implications for exposure and medical monitoring. J Occup Environ Med 53(6 Suppl):S91–97.

Kuempel ED, Attfield MD, Stayner LT, Castranova 

V [2014]. Human and animal evidence supports 

lower occupational exposure limits for poorly-soluble 

respirable particles: letter to the editor re: ‘Lowtoxicity dusts: current exposure guidelines are not 

sufficiently protective’ by Cherrie, Brosseau, Hay 

and Donaldson. Ann Occup Hyg 58(9):1205–1208.

Kuempel ED, Castranova V, Geraci CL, Schulte 

PA [2012]. Development of risk-based nanomaterial groups for occupational exposure control. J 

Nanopart Res 14:1029.

Kuempel ED, O’Flaherty EJ, Stayner LT, Smith RJ, 

Green FH, Vallyathan V [2001]. A biomathematical model of particle clearance and retention in 

the lungs of coal miners. Regul Toxicol Pharmacol 

34(1):69–87.

Kuempel ED, Smith RJ, Attfield MD, Stayner LT 

[1997]. Risks of occupational respiratory diseases 

among U.S. Coal miners. Appl Occup Environ Hyg 

12(12):823–831.

Kuempel ED, Sweeney LM, Morris JB, Jarabek AM 

[2015]. Advances in Inhalation Dosimetry Models 

and Methods for Occupational Risk Assessment 

and Exposure Limit Derivation. J Occup Environ 

Hyg 12 (Suppl 1):S18–S40.

Kuppusamy SP, Kaiser JP, Wesselkamper SC [2015]. 

Epigenetic regulation in environmental chemical carcinogenesis and its applicability in human health 

risk assessment. Int J Toxicol 34(5):384–392.

Lantz PM, House JS, Lepkowski JM, Williams 

DR, Mero RP, Chen J [1998]. Socioeconomic factors, health behaviors, and mortality: results from 

a nationally representative prospective study of US 

adults. JAMA 279(21):1703–1708.
120 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Lasagna L [1987]. Predicting human drug safety 

from animal studies: Current issues. J Toxicol Sci 

12(4):439–450.

Lash TL, Fink AK [2003]. Semi-automated sensitivity analysis to assess systematic errors in observational data. Epidemiology 14(4):451–458.

Laskin DL, Pendino KJ [1995]. Macrophages and 

inflammatory mediators in tissue injury. Annu Rev 

Pharmacol Toxicol 35:655–677.

Lawson CC, Schnorr TM, Whelan EA, Deddens 

JA, Dankovic DA, Piacitelli LA, Sweeney MH, 

Connally LB [2004]. Paternal occupational exposure to 2,3,7,8-tetrachlorodibenzo-p-dioxin and 

birth outcomes of offspring: birth weight, preterm 

delivery, and birth defects. Environ Health Perspect 112(14):1403–1408.

Lee DJ, LeBlanc W, Fleming LE, Gómez-Marín O, 

Pitman T [2004]. Trends in U.S. smoking-rates in 

occupational groups: The National Health Interview Survey 1987–1994. Int J Occup Environ Med 

46(6):538–548.

LeFevre ME, Green FH, Joel DD, Laqueur W 

[1982]. Frequency of black pigment in livers and 

spleens of coal workers: correlation with pulmonary pathology and occupational information. Hum 

Pathol 13(12):1121–1126.

Leisch F [2004]. FlexMix: a general framework for 

finite mixture models and latent class regression in 

R. J Stat Softw 11(8):1–18.

Lentz TJ, Dotson GS, Williams PRD, Maier A, 

Gadagbui B, Pandalai SP, Lamba A, Hearl F, Mumtaz M [2015]. Aggregate exposure and cumulative 

risk assessment-integrating occupational and nonoccupational risk factors. J Occup Environ Hyg 

12(Suppl 1):S112–S126.

Leonard BE [2007]. Examination of underground 

miner data for radon progeny size reduction as 

cause of high radon “inverse” dose rate effect. 

Health Phys 93(2):133–150.

Lesko CR, Buchanan AL, Westreich D, Edwards JK, 

Hudgens MG, Cole SR [2017]. Generalizing study 

results: a potential outcomes perspective. Epidemiology 28(4):553–561.

Lewis AS, Sax SN, Wason SC, Campleman SL 

[2011]. Non-chemical stressors and cumulative 

risk assessment: an overview of current initiatives 

and potential air pollutant interactions. Int J Environ Res Public Health 8(6):2020–2073.

Lewis RW, Billington R, Debryune E, Gamer A, 

Lang B, Carpanini F [2002]. Recognition of adverse 

and nonadverse effects in toxicity studies. Toxicol 

Pathol 30(1):66–74.

Li R, Wang X, Ji Z, Sun B, Zhang H, Chang CH, 

Lin S, Meng H, Liao YP, Wang M, Li Z, Hwang 

AA, Song TB, Xu R, Yang Y, Zink JI, Nel AE, Xia 

T [2013]. Surface charge and cellular processing of 

covalently functionalized multiwall carbon nanotubes determine pulmonary toxicity. ACS Nano 

7(3):2352–2368.

Lin DY, Psaty BM, Kronmal RA [1998]. Assessing 

the sensitivity of regression results to unmeasured 

confounders in observational studies. Biometrics 

54(3):948–963.

Lin L, Piegorsch WW, Bhattacharya R [2015]. 

Nonparametric benchmark dose estimation with 

continuous dose-response data. Scand Stat Theory 

Appl 42(3):713–731.

Linet MS, Schubauer-Berigan MK, Weisenburger 

DD, Richardson DB, Landgren O, Blair A, Silver S, Field RW, Caldwell G, Hatch M, Dores GM 

[2007]. Chronic lymphocytic leukaemia: an overview of aetiology in light of recent developments 

in classification and pathogenesis. Br J Haematol 

139(5):672–686.

Lioy P, Lippmann M, Phalen R [1984]. Rationale 

for particle size-selective air sampling. Ann Am 

Conf Ind Hyg 11:27–34.

Liu CS, Liou SH, Loh CH, Yu YC, Uang SN, Shih 

TS, Chen HI [2005]. Occupational bladder cancer 

in a 4,4-methylenebis(2-chloroaniline) (MBOCA)-

exposed worker. Environ Health Perspect 113(6): 

771–774.

Lochner C, Kinnear CJ, Hemmings SM, Seller C, 

Niehaus DJ, Knowles JA, Daniels W, MoolmanSmook JC, Seedat S, Stein DJ [2005]. Hoarding in 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 121

obsessive-compulsive disorder: clinical and genetic 

correlates. J Clin Psychiatry 66(9):1155–1160.

Loizou G, Spendiff M, Barton HA, Bessems J, Bois 

FY, d’Yvoire MB, Buist H, Clewell HJ, Meek B, 

Gundert-Remy U [2008]. Development of good 

modelling practice for physiologically based pharmacokinetic models for use in risk assessment: the 

first steps. Regul Toxicol Pharmacol 50(3):400–411.

Loomis D, Salvan A, Kromhout H, Kriebel D 

[1999]. Selecting indices of occupational exposure 

for epidemiologic studies. Occupational Hygiene 

5(1):73–91.

Lu FC, Kacew S [2002]. Lu’s Basic Toxicology: fundamentals, target organs and risk assessment. 4th 

ed. New York: Taylor & Francis. 

Lubin JH, Boice JD Jr., Edling C, Hornung RW, 

Howe G, Kunz E, Kusiak RA, Morrison HI, Radford EP, Samet JM, Tirmarche M, Woodward 

A, Xiang Yao S [1995b]. Radon-exposed underground miners and inverse dose-rate (protraction 

enhancement) effects. Health Phys 69(4):494–500.

Lubin JH, Boice JD, Edling C, Hornung RW, Howe 

GR, Kunz E, Kusiak RA, Morrison HI, Radford EP, 

Samet JM, Tirmarche M, Woodward A, Yao SX, 

Pierce DA [1995a]. Lung cancer in radon-exposed 

miners and estimation of risk from indoor exposure. J Natl Cancer Inst 87(11):817–827.

Luck J, Peabody JW, Dresselhaus TR, Lee M, Glassman P [2000]. How well does chart abstraction 

measure quality? A prospective comparison of 

standardized patients with the medical record. Am 

J Med 108(8):642–649.

MacMahon B, Yen S, Trichopoulos D, Warren K, 

Nardi G [1981]. Coffee and cancer of the pancreas. 

N Engl J Med 304(11):630–633.

Maier A, Kohrman-Vincent M, Hertzberg R, Allen B, 

Haber LT, Dourson M [2012]. Critical review of doseresponse options for F344 rat mammary tumors for 

acrylamide—additional insights based on mode of 

action. Food Chem Toxicol 50(5):1763–1775.

Maier MS [2011]. Setting occupational exposure 

limits for unstudied pharmaceutical intermediates 

using an in vitro parallelogram approach. Toxicol 

Mech Methods 21(2):76–85.

Maldonado G [2008]. Adjusting a relative-risk estimate for study imperfections. J Epidemiol Community Health 62(7):655–663.

Mallick B, Hoffman FO, Carroll RJ [2002]. Semiparametric regression modeling with mixtures of Berkson and classical error, with application to fallout 

from the Nevada test site. Biometrics 58(1):13–20.

Marquart J, Brouwer DH, Gijsbers JH, Links IH, 

Warren N, van Hemmen JJ [2003]. Determinants 

of dermal exposure relevant for exposure modelling in regulatory risk assessment. Ann Occup Hyg 

47(8):599–607.

Martignoni M, Groothuis GM, de Kanter R [2006]. 

Species differences between mouse, rat, dog, monkey and human CYP-mediated drug metabolism, 

inhibition and induction. Expert Opin Drug Metab 

Toxicol 2(6):875–894.

Mauderly JL [1997]. Relevance of particle-induced 

rat lung tumors for assessing lung carcinogenic

hazard and human lung cancer risk. Environ Health 

Perspect 105(Suppl 5):1337–1346.

Maudsley G, Williams EMI [1996]. ‘Inaccuracy’ in 

death certification—where are we now? J Public 

Health Med 18(1):59–66.

McClellan RO [1997]. Use of mechanistic data in 

assessing human risks from exposure to particles. 

Environ Health Perspect 105(Suppl 5):1363.

McFadden E, Luben R, Wareham N, Bingham S, 

Khaw KT [2008]. Occupational social class, educational level, smoking and body mass index, and 

cause-specific mortality in men and women: a prospective study in the European Prospective Investigation of Cancer and Nutrition in Norfolk (EPICNorfolk) cohort. Eur J Epidemiol 23(8):511–522.

McLanahan ED, El-Masri HA, Sweeney LM, Kopylev LY, Clewell HJ, Wambaugh JF, Schlosser P 

[2012]. Physiologically based pharmacokinetic 

model use in risk assessment—why being published is not enough. Toxicol Sci 126(1):5–15.
122 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

McMichael AJ [1976]. Standardized mortality ratios and the “healthy worker effect”: scratching beneath the surface. J Occup Med 18(3):165–168.

McNamee R [2003]. Confounding and confounders. Occup Environ Med 60(3):227–234.

Meek ME, Boobis AR, Crofton KM, Heinemeyer 

G, Raaij MV, Vickers C [2011]. Risk assessment 

of combined exposure to multiple chemicals: A 

WHO/IPCS framework. Regul Toxicol Pharmacol 

60(2, suppl):S1–S14.

Meek ME, Renwick A, Ohanian E, Dourson M, 

Lake B, Naumann BD, Vu V [2002]. Guidelines for 

application of chemical-specific adjustment factors 

in dose/concentration-response assessment. Toxicology 181–182:115–120.

Meliker JR, Goovaerts P, Jacquez GM, Nriagu JO 

[2010]. Incorporating individual-level distributions of exposure error in epidemiologic analyses: 

An example using arsenic in drinking water and 

bladder cancer. Ann Epidemiol 20(10):750–758.

Mercer RR, Russell ML, Roggli VL, Crapo JD [1994]. 

Cell number and distribution in human and rat airways. Am J Respir Cell Mol Biol 10(6):613–624.

Mercer RR, Scabilloni JF, Hubbs AF, Wang L, Battelli 

LA, McKinney W, Castranova V, Porter DW [2013]. 

Extrapulmonary transport of MWCNT following 

inhalation exposure. Part Fibre Toxicol 10:38.

Mete F, Kilic E, Somay A, Yilmaz B [2012]. Effects of 

heat stress on endocrine functions & behaviour in 

the pre-pubertal rat. Indian J Med Res 135:233–239.

Miller FJ [2000]. Dosimetry of particles in laboratory animals and humans in relationship to issues 

surrounding lung overload and human health risk 

assessment: a critical review. Inhal Toxicol 12(1–

2):19–57.

Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, 

Stroup DF [1999]. Improving the quality of reports 

of meta-analyses of randomised controlled trials: The 

QUOROM statement. Lancet 354(9193):1896–1900.

Moher D, Schulz KF, Altman D [2001]. The CONSORT statement: Revised recommendations for 

improving the quality of reports of parallel-group 

randomized trials. JAMA 285(15):1987–1991.

Moon H, Kim HJ, Chen JJ, Kodell RL [2005]. Model 

averaging using the Kullback information criterion 

in estimating effective doses for microbial infection 

and illness. Risk Anal 25(5):1147–1159.

Morales KH, Ibrahim JG, Chen C-J, Ryan LM 

[2006]. Bayesian model averaging with applications to benchmark dose estimation for arsenic in 

drinking water. J Am Stat Assoc 101(473):9–17.

Moretto A, Bachman A, Boobis A, Solomon KR, 

Pastoor TP, Wilks MF, Embry MR [2017]. A framework for cumulative risk assessment in the 21st 

century. Crit Rev Toxicol 47(2):85-97.

Morfeld P, Bruch J, Ngiewich Y, Chaudhuri I, Muranko H, Myerson R [2015]. Translation toxicology 

in setting occupational exposure limits for dusts 

and hazard classification—a critical evaluation of a 

recent approach to translate dust overload findings 

from rats to humans. Part Fibre Toxicol 12:3.

Morrow PE [1988]. Possible mechanisms to explain dust overloading of the lungs. Fundam Appl 

Toxicol 10(3):369–384.

Morrow PE, Muhle H, Mermelstein R [1991]. Chronic inhalation study findings as a basis for proposing 

a new occupational dust exposure limit. Int J Toxicol 

10(2):279–290.

Moss O, Cheng Y [1989]. Generation and characterization of test atmospheres: particles. In: Henderson RF, McClellam R, Henderson R, eds. Concepts in inhalation toxicology. New York: Hemisphere Publishing Corp.

Muhle H, Bellmann B, Creutzenberg O, Heinrich 

U, Ketkar M, Mermelstein R [1990]. Dust overloading of lungs after exposure of rats to particles 

of low solubility: comparative studies. J Aerosol Sci 

21(3):374–377.

Mumtaz MM, Knauf LA, Reisman DJ, Peirano 

WB, DeRosa CT, Gombar VK, Enslein K, Carter 

JR, Blake BW, Huque KI, Ramanujam VMS [1995]. 

Assessment of effect levels of chemicals from 

quantitative structure-activity relationship (QSAR) 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 123

models. I. Chronic lowest-observed-adverse-effect 

level (LOAEL). Toxicol Lett 79(1–3):131–143.

Naimi AI, Cole SR, Hudgens MG, Brookhart MA, 

Richardson DB [2013]. Assessing the component 

associations of the healthy worker survivor bias: 

occupational asbestos exposure and lung cancer 

mortality. Ann Epidemiol 23(6):334–341.

Naimi AI, Cole SR, Hudgens MG, Richardson DB 

[2014]. Estimating the effect of cumulative occupational asbestos exposure on time to lung cancer mortality: using structural nested failure-time 

models to account for healthy-worker survivor 

bias. Epidemiology 25(2):246–254.

NASEM [2017]. Controlled human inhalationexposure studies at EPA. National Academies of 

Sciences, Engineering, and Medicine (NASEM). 

Washington, DC: National Academies Press.

Naumann BD, Weideman PA [1995]. Scientific basis for uncertainty factors used to establish occupational exposure limits for pharmaceutical active 

ingredients. Hum Ecol Risk Assess 1(5):590–613.

Nayak TK, Kundu S [2001]. Calculating and describing uncertainty in risk assessment: the Bayesian approach. Hum Ecol Risk Assess 7(2):307–328.

NCRP [2010]. Radiation dose reconstruction: principles and practices. NCRP report no. 163. Bethesda, MD: National Council on Radiation Protection 

and Measurements.

Needham LL, Calafat AM, Barr DB [2007]. Uses 

and issues of biomonitoring. Int J Hyg Environ 

Health 210(3–4):229–238.

Nel AE, Nasser E, Godwin H, Avery D, Bahadori 

T, Bergeson L, Beryt E, Bonner JC, Boverhof D, 

Carter J, Castranova V, Deshazo JR, Hussain SM, 

Kane AB, Klaessig F, Kuempel E, Lafranconi M, 

Landsiedel R, Malloy T, Miller MB, Morris J, Moss 

K, Oberdörster G, Pinkerton K, Pleus RC, Shatkin 

JA, Thomas R, Tolaymat T, Wang A, Wong J [2013]. 

A multi-stakeholder perspective on the use of alternative test strategies for nanomaterial safety assessment. ACS Nano 7(8):6422–6433.

Nelson G [1992]. Gas mixtures: preparation and 

control. Boca Raton, FL: CRC Press. 

Neophytou AM, Picciotto S, Costello S, Eisen EA 

[2016]. Occupational diesel exposure, duration of 

employment, and lung cancer: an application of the 

parametric g-formula. Epidemiology 27(1):21–28.

Nieuwenhuijsen MJ [2010]. Exposure assessment 

in occupational and environmental epidemiology. 

New York: Oxford University Press. 

Nikula KJ, Vallyathan V, Green FH, Hahn FF 

[2001]. Influence of exposure concentration or 

dose on the distribution of particulate material 

in rat and human lungs. Environ Health Perspect 

109(4):311–318.

Nilsson C, Raun K, Yan FF, Larsen MO, TangChristensen M [2012]. Laboratory animals as surrogate models of human obesity. Acta Pharmacol 

Sin 33(2):173–181.

NIOSH [1982]. Comments to OSHA on identification, classification and regulation of potential 

occupational carcinogens. Cincinnati, OH: U.S. 

Department of Health and Human Services, Centers for Disease Control and Prevention, National 

Institute for Occupational Safety and Health.

NIOSH [1986a]. Informal public meeting on OSHA’s proposed rule: occupational exposure to benzene. NIOSH testimony presented on March 20, 

1986. Washington, DC: Occupational Safety and 

Health Administration.

NIOSH [1986b]. Testimony to OSHA on the proposed rule: occupational exposure to formaldehyde. NIOSH testimony presented May 5, 1986. 

Washington DC: Occupational Safety and Health 

Administration.

NIOSH [1987]. A recommended standard for occupational exposure to radon progeny in underground mines. Washington, DC: U.S. Department 

of Health and Human Services, Public Health Service, Centers for Disease Control, National Institute for Occupational Safety and Health, DHHS 

(NIOSH) Publication No. 88-101.
124 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

NIOSH [1988]. Occupational safety and health guideline for benzene: potential human carcinogen. Cincinnati, OH: U.S. Department of Health and Human 

Services, Centers for Disease Control and Prevention, 

National Institute for Occupational Safety and Health, 

DHHS (NIOSH) Publication No. 89-104.

NIOSH [1991]. Criteria for a recommended standard: occupational exposure to ethylene glycol 

monomethyl ether, ethylene glycol monoethyl ether, 

and their acetates. Cincinnati, OH: U.S. Department of Health and Human Services, Centers for 

Disease Control and Prevention, National Institute for Occupational Safety and Health, DHHS 

(NIOSH) Publication No. 91-119.

NIOSH [1995a]. Criteria for a recommended standard: occupational exposure to respirable coal mine 

dust. Cincinnati, OH: U.S. Department of Health 

and Human Services, Centers for Disease Control 

and Prevention, National Institute for Occupational Safety and Health, DHHS (NIOSH) Publication 

No. 95-106. 

NIOSH [1995b]. Guidelines for air sampling and 

analytical method development and evaluation. 

Cincinnati, OH: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety 

and Health, DHHS (NIOSH) Publication No. 95-117. 

NIOSH [1998]. Criteria for a recommended standard: occupational noise exposure. Cincinnati, 

OH: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, 

National Institute for Occupational Safety and 

Health, DHHS (NIOSH) Publication No. 98-126. 

NIOSH [2007]. NIOSH pocket guide to chemical hazards. Cincinnati, OH: U.S. Department of 

Health and Human Services, Centers for Disease 

Control and Prevention, National Institute for Occupational Safety and Health, DHHS (NIOSH) 

Publication No. 2005-149. 

NIOSH [2011]. Current intelligence bulletin 63: occupational exposure to titanium dioxide. Cincinnati, 

OH: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, 

DHHS (NIOSH) Publication No. 2011-160. 

NIOSH [2013a]. Criteria for a recommended standard: occupational exposure to hexavalent chromium. Cincinnati, OH: U.S. Department of Health 

and Human Services, Centers for Disease Control 

and Prevention, National Institute for Occupational Safety and Health, DHHS (NIOSH) Publication 

No. 2013-128. 

NIOSH [2013b]. Current intelligence bulletin 65: 

occupational exposure to carbon nanotubes and 

nanofibers. Cincinnati, OH: U.S. Department of 

Health and Human Services, Centers for Disease 

Control and Prevention, National Institute for Occupational Safety and Health, DHHS (NIOSH) 

Publication No. 2013-14. 

NIOSH [2016]. Criteria for a reommended standard: occupational exposure to diacetyl and 

2,3-pentanedione. Cincinnati, OH: U.S. Department of Health and Human Services, Centers for 

Disease Control and Prevention, National Institute for Occupational Safety and Health, DHHS 

(NIOSH) Publication No. 2016-111. 

NIOSH [2017]. Current intelligence bulletin 68: 

NIOSH chemical carcinogen policy. Cincinnati, 

OH: U.S. Department of Health and Human Services, 

Centers for Disease Control and Prevention, National 

Institute for Occupational Safety and Health, DHHS 

(NIOSH) Publication Number 2017-100. 

NIOSH [2019]. Technical Report: the NIOSH occupational exposure banding process for chemical 

risk management. Cincinnati, OH: U.S. Department of Health and Human Services, Centers for 

Disease Control and Prevention, National Institute for Occupational Safety and Health, DHHS 

(NIOSH) Publication No. 2019-132.

Noble RB, Bailer AJ, Park R [2009]. Model-averaged 

benchmark concentration estimates for continuous 

response data arising from epidemiological studies. 

Risk Anal 29(4):558–564.

NRC [1956]. The biological effects of atomic radiation; a report to the public. Washington, DC: National 

Academy of Sciences, National Research Council. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 125

NRC [1981]. Mammalian models for research on aging. National Academy of Sciences, National Research 

Council. Washington, DC: The National Academy 

Press. 

NRC [1983]. Risk assessment in the federal government: managing the process. National Academy of 

Sciences, National Research Council. Washington, 

DC: National Academy Press. 

NRC [1988]. Health risks of radon and other internally deposited alpha-emitters: BEIR IV. National 

Academy of Sciences, National Research Council. 

Washington, DC: National Academy Press.

NRC [1990]. Health Effects of exposure to low levels of ionizing radiation: BEIR V. National Academy of Sciences, National Research Council. Washington, DC: National Academies Press.

NRC [1991]. Environmental epidemiology, volume 

1: public health and hazardous wastes. National 

Academy of Sciences, National Research Council. 

Washington, DC: National Academies Press.

NRC [1994]. Science and judgment in risk aassessment. National Academy of Sciences, National Research Council. Washington, DC: National Academy Press.

NRC [2006]. Health risks from exposure to low levels of ionizing radiation: BEIR VII Phase 2. National 

Academy of Sciences, National Research Council. 

Washington, DC: National Academies Press. 

NRC [2007]. Toxicity testing in the 21st century: 

a vision and a strategy. National Academy of Sciences, National Research Council. Washington, 

DC: National Academies Press.

NRC [2009]. Science and decisions: advancing risk 

assessment. National Academy of Sciences, National 

Research Council. Washington, DC: National Academy Press. 

NRC [2012]. Exposure science in the 21st century: 

a vision and a strategy. National Academy of Sciences, National Research Council. Washington, DC: 

The National Academies Press.

NRC [2014]. Review of EPA’s integrated risk information system (IRIS) process. National Academy 

of Sciences, National Research Council. Washington, DC: The National Academies Press. 

NTP [2015a]. Handbook for conducting a literaturebased health assessment using OHAT approach 

for systematic review and evidence integration. 

Washington, DC: Office of Health Assessment and

Translation (OHAT), Division of the National Toxicology Program, National Institute of Environmental Health Sciences, U.S. Department of Health 

and Human Services.

NTP [2015b]. Handbook for preparing report on 

carcinogens monographs. Washington, DC: Office 

of the Report on Carcinogens, Division of the National Toxicology Program, National Institute of 

Environmental Health Sciences, U.S. Department 

of Health and Human Services.

NTP [2015c]. OHAT risk of bias rating tool for human and animal studies. Washington, DC: Office 

of Health Assessment and Translation (OHAT), 

Division of the National Toxicology Program, National Institute of Environmental Health Sciences, 

U.S. Department of Health and Human Services.

Oates CJ, Kasza J, Mukherjee S [2016]. Discussion of 

“Causal inference using invariant prediction: identification and confidence intervals” by Peters Bühlmann 

and Meinshausen. Preprint. Ithaca, NY: Cornell 

University, https://arxiv.org/abs/1605.08328.

Oberdörster G [1989]. Dosimetric principles for 

extrapolating results of rat inhalation studies to 

humans, using an inhaled Ni compound as an example. Health Phys 57(Suppl 1):213–220.

Oberdörster G [1995]. Lung particle overload: implications for occupational exposures to particles. 

Regul Toxicol Pharmacol 21(1):123–135.

Oberdörster G [1997]. Pulmonary carcinogenicity of inhaled particles and the maximum tolerated 

dose. Environ Health Perspect 105 Suppl 5:1347–1355.

Oberdörster G [2012]. Nanotoxicology: in vitro-in 

vivo dosimetry. Environ Health Perspect 120(1):A13; 

author reply A13.
126 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Oberdörster G, Ferin J, Soderholm S, Gelein R, Cox 

C, Baggs R, Morrow P [1994]. Increased pulmonary toxicity of inhaled ultrafine particles: Due to 

lung overload alone? Ann Occup Hyg 38(inhaled 

particles VII):295–302.

Oberdörster G, Maynard A, Donaldson K, Castranova V, Fitzpatrick J, Ausman K, Carter J, Karn 

B, Kreyling W, Lai D, Olin S, Monteiro-Riviere N, 

Warheit D, Yang H, ILSI Research Foundation/Risk 

Science Institute Nanomaterial Toxicity Screening 

Working Group [2005a]. Principles for characterizing the potential human health effects from exposure to nanomaterials: elements of a screening 

strategy. Part Fibre Toxicol 2:8.

Oberdörster G, Oberdörster E, Oberdörster J 

[2005b]. Nanotoxicology: an emerging discipline 

evolving from studies of ultrafine particles. Environ Health Perspect 113(7):823–839.

OECD [1981a].Test no. 410: repeated dose dermal 

toxicity: 21/28-day study. OECD guidelines for the 

testing of chemicals, section 4. Paris: Organisation 

for Economic Co-operation and Development.

OECD [1981b]. Test no. 411: subchronic dermal 

toxicity: 90-day study. OECD guidelines for the testing of chemicals, section 4. Paris: Organisation for 

Economic Co-operation and Development.

OECD [2005]. Good laboratory practice: OECD 

principles and guidance for compliance monitoring. Paris: Organisation for Economic Co-operation and Development. 

OECD [2009]. Test no. 403: acute inhalation toxicity. OECD guidelines for the testing of chemicals, 

section 4. Paris: Organisation for Economic Co-operation and Development.

OECD [2010]. Test no. 429: skin sensitisation local 

lymph node assay. OECD guidelines for the testing of chemicals, section 4. Paris: Organisation for 

Economic Co-operation and Development.

OECD [2015]. Test no. 404: acute dermal irritation/corrosion. OECD guidelines for the testing of 

chemicals, section 4. Paris: Organisation for Economic Co-operation and Development.

OECD [2017]. Test no. 402: acute dermal toxicity. 

OECD guidelines for the testing of chemicals, section 4. Paris: Organisation for Economic Co-operation and Development.

OECD [2018a]. Considerations for assessing the 

risks of combined exposure to multiple chemicals, 

series on testing and assessment no. 296, Environment, Health and Safety Division, Environment 

Directorate. Paris: Organisation for Economic Cooperation and Development.

OECD [2018b]. Test no. 412: Subacute inhalation 

toxicity: 28-day study. OECD guidelines for the 

testing of chemicals, section 4. Paris: Organisation 

for Economic Co-operation and Development.

OECD [2018c]. Test no. 413: subchronic inhalation 

Toxicity: 90-day study. OECD guidelines for the 

testing of chemicals, section 4. Paris: Organisation 

for Economic Co-operation and Development.

OECD [2018d]. Test no. 452: chronic toxicity studies. OECD guidelines for the testing of chemicals, 

section 4. Paris: Organisation for Economic Cooperation and Development.

Oh E, Liu R, Nel A, Gemill KB, Bilal M, Cohen Y, 

Medintz IL [2016]. Meta-analysis of cellular toxicity for cadmium-containing quantum dots. Nat 

Nanotechnol 11(5):479–486.

Olin, SS [2000]. Risk Science Institute workshop 

participants, the relevance of the rat lung response 

to particle overload for human risk assessment: a 

workshop consensus report. Inhal Toxicol 12:1–17.

Oller AR, Oberdörster G [2016]. Incorporation of 

dosimetry in the derivation of reference concentrations for ambient or workplace air: a conceptual approach. J Aerosol Sci 99(Suppl C):40–45.

Orsini N, Li R, Wolk A, Khudyakov P, Spiegelman 

D [2012]. Meta-analysis for linear and nonlinear 

dose-response relations: examples, an evaluation 

of approximations, and software. Am J Epidemiol 

175(1):66–73.

OSHA [1997]. 29 CFR Parts 1910, 1915 and 1926: 

Occupational exposure to methylene chloride. Final rule. Fed Regist 62:1494–1619.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 127

Overmyer KA, Thonusin C, Qi NR, Burant CF, 

Evans CR [2015]. Impact of anesthesia and euthanasia on metabolomics of mammalian tissues: 

studies in a C57BL/6J mouse model. PLoS One 

10(2):e0117232.

Ozasa K, Shimizu Y, Suyama A, Kasagi F, Soda M, 

Grant EJ, Sakata R, Sugiyama H, Kodama K [2012]. 

Studies of the mortality of atomic bomb survivors, 

report 14, 1950–2003: an overview of cancer and 

noncancer diseases. Radiat Res 177(3):229–243.

Panchal SK, Brown L [2011]. Rodent models for 

metabolic syndrome research. J Biomed Biotechnol 

2011:351982.

Paquet F, Etherington G, Bailey MR, Leggett RW, Lipsztein J, Bolch W, Eckerman KF, Harrison JD, ICRP 

[2015]. ICRP Publication 130: occupational intakes 

of radionuclides: part 1. Ann ICRP 44(2):5–188.

Park R, Rice F, Stayner L, Smith R, Gilbert S, Checkoway H [2002]. Exposure to crystalline silica, silicosis, and lung disease other than cancer in diatomaceous earth industry workers: a quantitative risk 

assessment. Occup Environ Med 59(1):36–43.

Park RM [2018]. Risk assessment for metalworking fluids and cancer outcomes. Am J Ind Med 

61(3):198–203.

Park RM, Bena JF, Stayner LT, Smith RJ, Gibb HJ, 

Lees PSJ [2004]. Hexavalent chromium and lung 

cancer in the chromate industry: a quantitative risk 

assessment. Risk Anal 24(5):1099–1108.

Park RM, Berg SL [2018]. Manganese and neurobehavioral impairment. A preliminary risk assessment. Neurotoxicology 64:159–165.

Park RM, Bowler RM, Roels HA [2009]. Exposureresponse relationship and risk assessment for cognitive deficits in early welding-induced manganism. J Occup Environ Med 51(10):1125–1136.

Park RM, Gilbert SJ [2018]. Pulmonary impairment and risk assessment in a diacetyl-exposed 

population: microwave popcorn workers. J Occup 

Environ Med 60(6):496–506.

Park RM, Maizlish NA, Punnett L, Moure-Eraso R, 

Silverstein MA [1991]. A comparison of PMRs and 

SMRs as estimators of occupational mortality. Epidemiology 2(1):49–59.

Park RM, Stayner LT, Petersen MR, Finley-Couch 

M, Hornung R, Rice C [2012]. Cadmium and lung 

cancer mortality accounting for simultaneous arsenic exposure. Occup Environ Med 69(5):303–309.

Pauluhn J [2010]. Multi-walled carbon nanotubes 

(Baytubes): approach for derivation of occupational 

exposure limit. Regul Toxicol Pharmacol 57(1):78–89.

Pauluhn J [2011]. Poorly soluble particulates: 

searching for a unifying denominator of nanoparticles and fine particles for DNEL estimation. Toxicology 279(1):176–188.

Pauluhn J [2014]. Derivation of occupational exposure levels (OELs) of low-toxicity isometric biopersistent particles: how can the kinetic lung overload 

paradigm be used for improved inhalation toxicity 

study design and OEL-derivation? Part Fibre Toxicol 11:72.

Paustenbach DJ [2000]. The practice of exposure 

assessment: a state-of-the-art review. J Toxicol Environ Health B Crit Rev 3(3):179–291.

Pearl J [2010]. An introduction to causal inference. 

Int J Biostat 6(2):Article 7.

Pearl J, Bareinboim E [2014]. External validity: From 

do-calculus to transportability across populations. 

Statist Sci 29(4):579–595.

Pecquet AM, Martinez JM, Vincent M, Erraguntla 

N, Dourson M [2018]. Derivation of a no-significantrisk-level for tetrabromobisphenol A based on a 

threshold non-mutagenic cancer mode of action. J 

Appl Toxicol 38(6):862–878.

Persad AS, Cooper GS [2008]. Use of epidemiologic data in Integrated Risk Information System (IRIS) assessments. Toxicol Appl Pharmacol 

233(1):137–145.

Phalen RF [1997]. Methods in inhalation toxicology. Vol. 3. Boca Raton, FL: CRC Press.

Phillips CV, Goodman KJ [2004]. The missed lessons of Sir Austin Bradford Hill. Epidemiol Perspect Innov 1(1):3.
128 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Picciotto S, Ljungman PL, Eisen EA [2016]. Straight 

metalworking fluids and all-cause and cardiovascular mortality analyzed by using g-estimation of 

an accelerated failure time model with quantitative 

exposure: Methods and interpretations. Am J Epidemiol 183(7):680–688.

Piegorsch WW, Xiong H, Bhattacharya RN, Lin L 

[2012]. Nonparametric estimation of benchmark 

doses in environmental risk assessment. Environmetrics 23(8):717–728.

Piegorsch WW, Xiong H, Bhattacharya RN, Lin L 

[2013]. Benchmark dose analysis via nonparametric regression modeling. Risk Anal 34(1)135–151.

Pierce DA, Vaeth M, Cologne JB [2008]. Allowance for random dose estimation errors in atomic 

bomb survivor studies: a revision. Radiat Res 

170(1):118–126.

Pinsky PF, Yu K, Black A, Huang WY, Prorok PC 

[2016]. Active follow-up versus passive linkage 

with cancer registries for case ascertainment in a 

cohort. Cancer Epidemiol 45:26–31.

Poet TS, McDougal JN [2002]. Skin absorption 

and human risk assessment. Chem Biol Interact 

140(1):19–34.

Poirier MC [2016]. Linking DNA adduct formation 

and human cancer risk in chemical carcinogenesis. 

Environ Mol Mutagen 57(7):499–507.

Posada D, Buckley TR [2004]. Model selection 

and model averaging in phylogenetics: advantages of akaike information criterion and bayesian approaches over likelihood ratio tests. Syst Biol 

53(5):793–808.

Price O, Asgharian B, Miller F, Cassee F, de WinterSorkina R [2002]. Multiple path particle dosimetry 

model (MPPD v 1.0): a model for human and rat 

airway particle dosimetry. Bilthoven, The Netherlands: National Institute for Public Health and the 

Environment (RIVM). 

Prince MM, Gilbert SJ, Smith RJ, Stayner LT [2003]. 

Evaluation of the risk of noise-induced hearing 

loss among unscreened male industrial workers. J 

Acoust Soc Am 113(2):871–880.

Raabe OG, Al-Bayati MA, Teague SV, Rasolt A 

[1988]. Regional deposition of inhaled monodisperse coarse and fine aerosol particles in small 

laboratory animals. Ann Occup Hyg 32(Inhaled 

Particles VI):53–63.

Raftery AE [1995]. Bayesian model selection in social research. In: Sociological methodology. Vol 25. 

Washington, DC: American Sociological Association.

Raftery AE, Madigan D, Hoeting JA [1997]. Bayesian model averaging for linear regression models. J 

Am Stat Assoc 92(437):179–191.

Raj AT, Patil S, Sarode S, Salameh Z [2017]. Phacking: a wake-up call for the scientific community. Sci Eng Ethics 24(6):1813–1814.

Ramachandran G, Banerjee S, Vincent JH [2003]. 

Expert judgment and occupational hygiene: application to aerosol speciation in the nickel primary production industry. Ann Occup Hyg 47(6):461–475.

Rappaport SM [1991]. Assessment of long-term 

exposures to toxic substances in air. Ann Occup 

Hyg 35(1):61–122.

Redelings MD, Wise M, Sorvillo F [2007]. Using 

multiple cause-of-death data to investigate associations and causality between conditions listed on the 

death certificate. Am J Epidemiol 166(1):104–108.

Relier C, Dubreuil M, Lozano Garcìa O, Cordelli 

E, Mejia J, Eleuteri P, Robidel F, Loret T, Pacchierotti F, Lucas S, Lacroix G, Trouiller B [2017]. 

Study of TiO2 P25 nanoparticles genotoxicity on 

lung, blood, and liver cells in lung overload and 

non-overload conditions after repeated respiratory 

exposure in rats. Toxicol Sci 156(2):527–537.

Rhomberg LR, Goodman JE, Bailey LA, Prueitt 

RL, Beck NB, Bevan C, Honeycutt M, Kaminski 

NE, Paoli G, Pottenger LH, Scherer RW, Wise KC, 

Becker RA [2013]. A survey of frameworks for best 

practices in weight-of-evidence analyses. Crit Rev 

Toxicol 43(9):753–784.

Rhomberg LR, Goodman JE, Haber LT, Dourson 

M, Andersen ME, Klaunig JE, Meek B, Price PS, 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 129

McClellan RO, Cohen SM [2011]. Linear low-dose 

extrapolation for noncancer heath effects is the exception, not the rule. Crit Rev Toxicol 41(1):1–21.

Rice C, Harris Jr RL, Lumsden JC, Symons MJ 

[1984]. Reconstruction of silica exposure in the 

North Carolina Dusty Trades. Am Ind Hyg Assoc J 

45(10):689–696.

Rice FL, Park R, Stayner L, Smith R, Gilbert S, 

Checkoway H [2001]. Crystalline silica exposure 

and lung cancer mortality in diatomaceous earth 

industry workers: a quantitative risk assessment. 

Occup Environ Med 58(1):38–45.

Richard AM, Judson RS, Houck KA, Grulke CM, 

Volarath P, Thillainadarajah I, Yang C, Rathman J, 

Martin MT, Wambaugh JF, Knudsen TB, Kancherla 

J, Mansouri K, Patlewicz G, Williams AJ, Little SB, 

Crofton KM, Thomas RS [2016]. ToxCast chemical 

landscape: paving the road to 21st century toxicology. Chem Res Toxicol 29(8):1225–1251.

Richardson DB [2004]. An incidence density sampling program for nested case-control analyses. 

Occup Environ Med 61(12):e59.

Richardson DB [2010]. Occupational exposures and 

lung cancer: Adjustment for unmeasured confounding by smoking. Epidemiology 21(2):181–186.

Richardson DB, Cole SR, Chu H, Langholz B 

[2011]. Lagging exposure information in cumulative exposure-response analyses. Am J Epidemiol 

174(12):1416–1422.

Rinsky RA, Hornung RW, Silver SR, Tseng CY 

[2002]. Benzene exposure and hematopoietic mortality: A long-term epidemiologic risk assessment. 

Am J Ind Med 42(6):474–480.

Rinsky RA, Smith AB, Hornung R, Filloon TG, 

Young RJ, Okun AH, Landrigan PJ [1987]. Benzene and leukemia. An epidemiologic risk assessment. N Engl J Med 316(17):1044–1050.

Rodricks JV, Gaylor DW, Turnbull D [2007]. Quantitative extrapolations in toxicology. In: Hayes AW, 

ed. Principles and methods of toxicology. 5th ed. 

Boca Raton, FL: CRC Press.

Rodrigues MA, Arezes P, Leão CP [2014]. Risk 

criteria in occupational environments: critical 

overview and discussion. Procedia Soc Behav Sci 

109:257–262.

Ron E, Hoffman FO [1999]. Uncertainties in radiation dosimetry and their impact on dose-response 

analyses. Bethesda, MD: U.S. Department of Health 

and Human Services, Public Health Service, National Institutes of Health. 

Rosseel Y [2012]. Lavaan: An R package for structural equation modeling. J Stat Softw 48(2):1–36.

Rothman KJ, Gallacher JEJ, Hatch EE [2013]. Why 

representativeness should be avoided. Int J Epidemiol 42(4):1012–1014.

Rothman KJ, Greenland S, Lash TL [2008]. Modern epidemiology. Philadelphia, PA: Wolters Kluwer Health/Lippincott Williams & Wilkins. 

Roy K, Kar S, Das RN [2015]. Understanding the 

basics of QSAR for applications in pharmaceutical 

sciences and risk assessment. London: Academic 

Press.

Ruddon RW [2007]. Cancer biology. 4th ed. New 

York: Oxford University Press.

Rushton EK, Jiang J, Leonard SS, Eberly S, Castranova V, Biswas P, Elder A, Han X, Gelein R, Finkelstein J, Oberdörster G [2010]. Concept of assessing nanoparticle hazards considering nanoparticle 

dosemetric and chemical/biological response metrics. J Toxicol Environ Health A 73(5):445–461.

Russo F, Williamson J [2007]. Interpreting causality in the health sciences. Int Stud Philos Sci 

21(2):157–170.

Sackett DL [1979]. Bias in analytic research. J Chronic 

Dis 32(1–2):51–63.

Sager TM, Wolfarth MW, Andrew M, Hubbs A, 

Friend S, Chen TH, Porter DW, Wu N, Yang F, 

Hamilton RF, Holian A [2014]. Effect of multiwalled carbon nanotube surface modification on 

bioactivity in the C57BL/6 mouse model. Nanotoxicology 8(3):317–327.
130 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Sahu SC, Casciano DA [2009]. Nanotoxicity: from 

in vivo and in vitro models to health risks. Chichester, West Sussex, UK: John Wiley & Sons. 

Salem H, Katz SA [2014]. Inhalation toxicology. 

Boca Raton, FL: CRC Press. 

Sand S, Portier CJ, Krewski D [2011]. A signal-tonoise crossover dose as the point of departure for 

health risk assessment. Environ Health Perspect 

119(12):1766–1774.

Sanderson WT, Petersen MR, Ward EM [2001]. 

Estimating historical exposures of workers in a 

beryllium manufacturing plant. Am J Ind Med 

39(2):145–157.

Sarangapani R, Teeguarden JG, Cruzan G, Clewell 

HJ, Andersen ME [2002]. Physiologically based 

pharmacokinetic modeling of styrene and styrene 

oxide respiratory-tract dosimetry in rodents and 

humans. Inhal Toxicol 14(8):789–834.

Sathiakumar N, Brill I, Leader M, Delzell E [2015]. 

1,3-Butadiene, styrene and lymphohematopoietic cancer among male synthetic rubber industry 

workers—preliminary exposure-response analyses. 

Chem Biol Interact 241:40–49.

Schafer DW, Lubin JH, Ron E, Stovall M, Carroll RJ 

[2001]. Thyroid cancer following scalp irradiation: 

a reanalysis accounting for uncertainty in dosimetry. Biometrics 57(3):689–697.

Schafer JL, Olsen MK [1998]. Multiple imputation for 

multivariate missing-data problems: a data analyst’s 

perspective. Multivariate Behav Res 33(4):545–571.

Schlesinger RB [1985]. Comparative deposition of inhaled aerosols in experimental animals and humans: 

a review. J Toxicol Environ Health 15(2):197–214.

Schoeny RS, Margosches E [1989]. Evaluating 

comparative potencies: developing approaches to 

risk assessment of chemical mixtures. Toxicol Ind 

Health 5(5):825–837.

Schottenfeld D, Fraumeni Jr JF [2006]. Cancer epidemiology and prevention. 3rd ed. New York: Oxford University Press.

Schubauer-Berigan MK, Anderson JL, Hein MJ, 

Little MP, Sigurdson AJ, Pinkerton LE [2015]. 

Breast cancer incidence in a cohort of U.S. flight 

attendants. Am J Ind Med 58(3):252–266.

Schubauer-Berigan MK, Couch JR, Deddens JA 

[2017]. Is beryllium-induced lung cancer caused 

only by soluble forms and high exposure levels? 

Occup Environ Med 74(8):601–603.

Schubauer-Berigan MK, Couch JR, Petersen MR, 

Carreón T, Jin Y, Deddens JA [2011]. Cohort mortality study of workers at seven beryllium processing plants: update and associations with cumulative and maximum exposure. Occup Environ Med 

68(5):345–353.

Schuhmacher-Wolz U, Kalberlah F, Oppl R, van 

Hemmen JJ [2003]. A toolkit for dermal risk assessment: toxicological approach for hazard characterization. Ann Occup Hyg 47(8):641–652.

Schulte P [2003]. Challenges for risk assessors. 

Hum Ecol Risk Assess 9(1):439–445.

Schulte PA [1993]. Use of biological markers in occupational health research and practice. J Toxicol 

Environ Health 40(2–3):359–366.

Schulte PA, Murashov V, Zumwalde R, Kuempel 

ED, Geraci CL [2010]. Occupational exposure limits for nanomaterials: state of the art. J Nanoparticle 

Res 12(6):1971–1987.

Schulte PA, Whittaker C, Curran CP [2015]. Considerations for using genetic and epigenetic information in occupational health risk assessment and 

standard setting. J Occup Environ Hyg 12(Suppl 1): 

S69–S81.

Schulte PA, Kuempel ED, Drew NM [2018]. Characterizing risk assessments for the development of 

occupational exposure limits for engineered nanomaterials. Regul Toxicol Pharmacol 95:207–219. 

doi: 10.1016/j.yrtph.2018.03.018.

Schulz H, Brand P, Heyer NJ [2000]. Particle deposition in the respiratory tract. In: Gehr P, Heyder 

J, eds. Particle-lung interactions. New York: Marcel 

Dekker, Inc.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 131

Schwarz G [1978]. Estimating the dimension of a 

model. Ann Stat 6(2):461–464.

SCOEL [2008]. Recommendation from the scientific committee on occupational exposure limits 

for formaldhyde. Brussels, Belgium: European Comission.

SCOEL [2010]. Recommendation from the scientific committee on occupational exposure limits for 

cadmium and its inorganic compounds. Brussels, 

Belgium: European Commission.

SCOEL [2011]. Recommendation from the scientific committee on occupational exposure limits for 

nickel and inorganic nickel compounds. Brussels, 

Belgium: European Comission.

Seed J, Carney EW, Corley RA, Crofton KM, DeSesso JM, Foster PM, Kavlock R, Kimmel G, Klaunig 

J, Meek ME, Preston RJ, Slikker W, Jr., Tabacova S, 

Williams GM, Wiltse J, Zoeller RT, Fenner-Crisp P, 

Patton DE [2005]. Overview: using mode of action 

and life stage information to evaluate the human 

relevance of animal toxicity data. Crit Rev Toxicol 

35(8–9):664–672.

Seeley MR, Tonner-Navarro LE, Beck BD, Deskin 

R, Feron VJ, Johanson G, Bolt HM [2001]. Procedures for health risk assessment in Europe. Regul 

Toxicol Pharmacol 34(2):153–169.

Sehdev AES, Hutchins GM [2001]. Problems with 

proper completion and accuracy of the cause-ofdeath statement. Arch Intern Med 161(2):277–284.

Seixas NS, Checkoway H [1995]. Exposure assessment in industry specific retrospective occupational epidemiology studies. Occup Environ Med 

52(10):625–633.

Seixas NS, Heyer NJ, Welp EA, Checkoway H 

[1997]. Quantification of historical dust exposures 

in the diatomaceous earth industry. Ann Occup 

Hyg 41(5):591–604.

Seixas NS, Robins TG, Moulton LH [1988]. The 

use of geometric and arithmetic mean exposures 

in occupational epidemiology. Am J Ind Med 

14(4):465–477.

Sexton K [2012]. Cumulative risk assessment: an 

overview of methodological approaches for evaluating combined health effects from exposure to 

multiple environmental stressors. Int J Environ Res 

Public Health 9(2):370–390.

Sexton K, Hattis D [2007]. Assessing cumulative health risks from exposure to environmental 

mixtures—three fundamental questions. Environ 

Health Perspect 115(5):825.

Shipp A, Lawrence G, Gentry R, McDonald T, Bartow H, Bounds J, Macdonald N, Clewell H, Allen 

B, Van Landingham C [2006]. Acrylamide: review 

of toxicity data and dose-response analyses for 

cancer and noncancer effects. Crit Rev Toxicol 

36(6):481–608.

Sidhu KS [1992]. Basis for body weight exponent 

(0.75) as a scaling factor in energy metabolism and 

risk assessment. J Appl Toxicol 12(5):309–310.

Sievert R, Failla G [1959]. Recommendations of the 

international commission on radiological protection. Health Phys 2:1–20.

Silver SR, Hiratzka SL, Schubauer-Berigan MK, 

Daniels RD [2007]. Chronic lymphocytic leukemia 

radiogenicity: A systematic review. Cancer Causes 

Control 18(10):1077–1093.

Silver SR, Rinsky RA, Cooper SP, Hornung RW, Lai 

D [2002]. Effect of follow-up time on risk estimates: 

a longitudinal examination of the relative risks of 

leukemia and multiple myeloma in a rubber hydrochloride cohort. Am J Ind Med 42(6):481–489.

Silverman DT, Hoover RN, Swanson GM, Hartge 

P [1983]. The Prevalence of coffee drinking among 

hospitalized and population-based control groups. 

JAMA 249(14):1877–1880.

Simmons JP, Nelson LD, Simonsohn U [2011]. 

False-positive psychology: undisclosed flexibility 

in data collection and analysis allows presenting 

anything as significant. Psychol Sci 22(11):1359–

1366.

Simon SL, Hoffman FO, Hofer E [2015]. The twodimensional Monte Carlo: a new methodologic 

paradigm for dose reconstruction for epidemiological studies. Radiat Res 183(1):27–41.
132 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Simon TW, Zhu Y, Dourson ML, Beck NB [2016]. 

Bayesian methods for uncertainty factor application for derivation of reference values. Regul Toxicol Pharmacol 80:9–24.

Smith AH [1988]. Epidemiologic input to environmental risk assessment. Arch Environ Health 

43(2):124–129.

Smith CM, Christiani DC, Kelsey KT [1994]. Chemical risk assessment and occupational health: current applications, limitations, and future prospects. 

Westport, CT: Greenwood Publishing Group. 

Smith MT, Guyton KZ, Gibbons CF, Fritz JM, Portier CJ, Rusyn I, DeMarini DM, Caldwell JC, Kavlock RJ, Lambert PF, Hecht SS, Bucher JR, Stewart 

BW, Baan RA, Cogliano VJ, Straif K [2016]. Key 

characteristics of carcinogens as a basis for organizing data on mechanisms of carcinogenesis. Environ Health Perspect 124(6):713–721.

Snipes MB [1989]. Long-term retention and clearance of particles inhaled by mammalian species. 

Crit Rev Toxicol 20(3):175–211.

Sobels FH [1977]. Some problems associated with 

the testing for environmental mutagens and a perspective for studies in “comparative mutagenesis.” 

Mutat Res 46(4):245–260.

Spiegelman D, Valanis B [1998]. Correcting for bias 

in relative risk estimates due to exposure measurement error: a case study of occupational exposure 

to antineoplastics in pharmacists. Am J Public 

Health 88(3):406–412.

Stahlhofen W, Rudolf G, James A [1989]. Intercomparison of experimental regional aerosol deposition data. J Aerosol Med 2(3):285–308.

Stayner L, Dankovic D, Smith R, Steenland K 

[1998]. Predicted lung cancer risk among miners 

exposed to diesel exhaust particles. Am J Ind Med 

34(3):207–219.

Stayner L, Smith R, Bailer AJ, Luebeck EG, Moolgavkar SH [1995]. Modeling epidemiologic studies 

of occupational cohorts for the quantitative assessment of carcinogenic hazards. Am J Ind Med 

27(2):155–170.

Stayner L, Smith R, Bailer J, Gilbert S, Steenland K, 

Dement J, Brown D, Lemen R [1997]. Exposureresponse analysis of risk of respiratory disease associated with occupational exposure to chrysotile 

asbestos. Occup Environ Med 54(9):646–652.

Stayner L, Smith R, Thun M, Schnorr T, Lemen R 

[1992a]. A dose-response analysis and quantitative 

assessment of lung cancer risk and occupational 

cadmium exposure. Ann Epidemiol 2(3):177–194.

Stayner L, Smith R, Thun M, Schnorr T, Lemen R 

[1992b]. A quantitative assessment of lung cancer 

risk and occupational cadmium exposure. IARC 

Sci Publ (118):447–455.

Stayner L, Smith RJ, Gilbert S, Bailer AJ [1999]. 

Epidemiologic approaches to risk assessment. Inhal Toxicol 11(6–7):593–601.

Stayner L, Steenland K, Dosemeci M, Hertz-Picciotto I 

[2003]. Attenuation of exposure-response curves in 

occupational cohort studies at high exposure levels. Scand J Work Environ Health 29(4):317–324.

Stayner L, Vrijheid M, Cardis E, Stram DO, Deltour 

I, Gilbert SJ, Howe G [2007]. A Monte Carlo maximum likelihood method for estimating uncertainty 

arising from shared errors in exposures in epidemiological studies of nuclear workers. Radiat Res 

168(6):757–763.

Stayner LT, Dankovic DA, Smith RJ, Gilbert SJ, 

Bailer AJ [2000]. Human cancer risk and exposure 

to 1,3-butadiene—a tale of mice and men. Scand J 

Work Environ Health 26(4):322–330.

Steckler A, McLeroy KR [2008]. The importance of 

external validity. Am J Public Health 98(1):9–10.

Steenland K, Deddens J, Stayner L [1998]. Diesel 

exhaust and lung cancer in the trucking industry: 

Exposure-response analyses and risk assessment. 

Am J Ind Med 34(3):220–228.

Steenland K, Deddens JA [2004]. A practical 

guide to dose-response analyses and risk assessment in occupational epidemiology. Epidemiology 

15(1):63–70.

Steenland K, Karnes C, Darrow L, Barry V [2015]. 

Attenuation of exposure-response rate ratios at 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 133

higher exposures a simulation study focusing on 

frailty and measurement error. Epidemiology 

26(3):395–401.

Steenland K, Savitz DA [1997]. Topics in environmental epidemiology. New York: Oxford University Press. 

Steenland K, Seals R, Klein M, Jinot J, Kahn HD 

[2011]. Risk estimation with epidemiologic data 

when response attenuates at high-exposure levels. 

Environ Health Perspect 119(6):831–837.

Steenland K, Whelan E, Deddens J, Stayner L, 

Ward E [2003]. Ethylene oxide and breast cancer 

incidence in a cohort study of 7576 women (United 

States). Cancer Causes Control 14(6):531–539.

Stellman SD, Boffetta P, Garfinkel L [1988]. Smoking habits of 800,000 American men and women 

in relation to their occupations. Am J Ind Med 

13(1):43–58.

Stewart PA, Blair A, Cubit DA, Bales RE, Kaplan 

SA, Ward J, Gaffey W, O’Berg MT, Walrath J [1986]. 

Estimating historical exposures to formaldehyde 

in a retrospective mortality study. Appl Ind Hyg 

1(1):34–41.

Stewart PA, Lees PS, Francis M [1996]. Quantification 

of historical exposures in occupational cohort studies. Scand J Work Environ Health 22(6):405–414.

Stewart WF, Tonascia JA, Matanoski GM [1987]. 

The validity of questionnaire-reported work history 

in live respondents. J Occup Med 29(10):795–800.

Stöber W, Einbrodt H, Klosterkötter W [1965]. 

Quantitative studies of dust retention in animal 

and human lungs after chronic inhalation. Inhaled 

Particles and Vapours II: proceedings of an international symposium organized by the British Occupational Hygiene Society, Cambridge, Sept. 28 

to Oct 1, 1965. Cambridge, UK: Pergamon Press, 

Symposium Publications Division.

Stöber W, Morrow PE, Morawietz G [1990]. Alveolar retention and clearance of insoluble particles in 

rats simulated by a new physiology-oriented compartmental kinetics model. Fundam Appl Toxicol 

15(2):329–349.

Stone KC, Mercer RR, Gehr P, Stockstill B, Crapo 

JD [1992]. Allometric relationships of cell numbers 

and size in the mammalian lung. Am J Respir Cell 

Mol Biol 6(2):235–243.

Stram DO, Preston DL, Sokolnikov M, Napier 

B, Kopecky KJ, Boice J, Beck H, Till J, Bouville A 

[2015]. Shared dosimetry error in epidemiological 

dose-response analyses. PLoS One 10(3):e0119418.

Stratmann G, Sall JW, May LD, Loepke AW, Lee MT 

[2010]. Beyond anesthetic properties: the effects of 

isoflurane on brain cell death, neurogenesis, and 

long-term neurocognitive function. Anesth Analg 

110(2):431–437.

Streffer C, Bolt H, Follesdal D, Hall P, Hengstler JG, 

Jacob P, Oughton D, Prieß K, Rehbinder E, Swaton 

E [2004]. Low dose exposures in the environment: 

dose-effect relations and risk evaluation. New York: 

Springer-Verlag. 

Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, Moher D, Becker BJ, Sipe TA, 

Thacker SB [2000]. Meta-analysis of observational 

studies in epidemiology: a proposal for reporting. 

JAMA 283(15):2008–2012.

Sturm R, Hofmann W [2009]. A theoretical approach to the deposition and clearance of fibers 

with variable size in the human respiratory tract. J 

Hazard Mater 170(1):210–218.

Su WC, Cheng YS [2014]. Carbon nanotubes size 

classification, characterization and nasal airway 

deposition. Inhal Toxicol 26(14):843–852.

Su W-C, Cheng YS [2015]. Estimation of carbon 

nanotubes deposition in a human respiratory tract 

replica. J Aerosol Sci 79:72–85.

Sutter TR [1995]. Molecular and cellular approaches to extrapolation for risk assessment. Environ 

Health Perspect 103(4):386–389.

Suvrathan A, Tomar A, Chattarji S [2010]. Effects 

of chronic and acute stress on rat behaviour in the 

forced-swim test. Stress 13(6):533–540.

Swartout JC, Price PS, Dourson ML, Carlson-Lynch 

HL, Keenan RE [1998]. A probabilistic framework 

for the reference dose (probabilistic RfD). Risk 

Analysis 18(3):271–282.
134 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Symonds MRE, Moussalli A [2011]. A brief guide 

to model selection, multimodel inference and 

model averaging in behavioural ecology using 

Akaike’s information criterion. Behav Ecol Sociobiol 65(1):13–21.

t’ Mannetje A, Fevotte J, Fletcher T, Brennan P, 

Legoza J, Szeremi M, Paldy A, Brzeznicki S, Gromiec J, Ruxanda-Artenie C, Stanescu-Dumitru R, 

Ivanov N, Shterengorz R, Hettychova L, Krizanova 

D, Cassidy A, Van Tongeren M, Boffetta P [2003]. 

Assessing exposure misclassification by expert assessment in multicenter occupational studies. Epidemiology 14(5):585–592.

Tchiehe DN, Gauthier F [2017]. Classification of 

risk acceptability and risk tolerability factors in occupational health and safety. Saf Sci 92:138–147.

Teeguarden JG, Hinderliter PM, Orr G, Thrall BD, 

Pounds JG [2007]. Particokinetics in vitro: dosimetry considerations for in vitro nanoparticle toxicity assessments. Toxicol Sci 95(2):300–312.

Teixeira PM [2018]. p-Hacking—a call for ethics. 

Pulmonology 24(3):207–208.

Teschke K, Kennedy SM, Olshan AF [1994]. Effect of 

different questionnaire formats on reporting of occupational exposures. Am J Ind Med 26(3):327–337.

Teschke K, Olshan AF, Daniels JL, De Roos AJ, 

Parks CG, Schulz M, Vaughan TL [2002]. Occupational exposure assessment in case-control studies: 

opportunities for improvement. Occup Environ 

Med 59(9):575–593; discussion 594.

Thacker SB [1988]. Meta-analysis: a quantitative approach to research integration. JAMA 

259(11):1685–1689.

Thomas D, Stram D, Dwyer J [1993a]. Exposure 

measurement error: influence on exposure-disease 

relationships and methods of correction. Annu Rev 

Public Health 14:69–93.

Thomas D, Stram D, Dwyer J [1993b]. Exposure 

measurement error: influence on exposure-disease. 

Relationships and methods of correction. Annu 

Rev Public Health 14:69–93.

Thomas RS, Clewell HJ III, Allen BC, Wesselkamper SC, Wang NC, Lambert JC, Hess-Wilson JK, 

Zhao QJ, Andersen ME [2011]. Application of 

transcriptional benchmark dose values in quantitative cancer and noncancer risk assessment. Toxicol 

Sci 120(1):194–205.

Thomas RS, Wesselkamper SC, Wang NC, Zhao QJ, 

Petersen DD, Lambert JC, Cote I, Yang L, Healy E, 

Black MB, Clewell HJ III, Allen BC, Andersen ME 

[2013]. Temporal concordance between apical and 

transcriptional points of departure for chemical 

risk assessment. Toxicol Sci 134(1):180–194.

Thompson CM, Suh M, Mittal L, Wikoff DS, Welsh 

B, Proctor DM [2016]. Development of linear and 

threshold no significant risk levels for inhalation 

exposure to titanium dioxide using systematic review and mode of action considerations. Regul 

Toxicol Pharmacol 80:60–70.

Thygesen LC, Andersen GS, Andersen H [2005]. A 

philosophical analysis of the Hill criteria. J Epidemiol Community Health 59(6):512–516.

Tielemans E, Heederik D, Burdorf A, Vermeulen 

R, Veulemans H, Kromhout H, Hartog K [1999]. 

Assessment of occupational exposures in a general 

population: comparison of different methods. Occup Environ Med 56(3):145–151.

Tollefsen KE, Scholz S, Cronin MT, Edwards SW, 

de Knecht J, Crofton K, Garcia-Reyero N, Hartung 

T, Worth A, Patlewicz G [2014]. Applying adverse 

outcome pathways (AOPs) to support integrated 

approaches to testing and assessment (IATA). 

Regul Toxicol Pharmacol 70(3):629–640.

Tooth L, Ware R, Bain C, Purdie DM, Dobson A 

[2005]. Quality of reporting of observational longitudinal research. Am J Epidemiol 161(3):280–288.

Tran C, Buchanan D [2000]. Development of a biomathematical lung model to describe the exposuredose relationship for inhaled dust among UK 

coal miners. IOM Report TM/00/02. Edinburgh, 

Scotland, UK: Institute of Occupational Medicine 

(IOM).
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 135

Tran CL, Buchanan D, Cullen RT, Searl A, Jones 

AD, Donaldson K [2000]. Inhalation of poorly 

soluble particles. II. Influence of particle surface 

area on inflammation and clearance. Inhal Toxicol 

12(12):1113–1126.

Triantafillou S, Tsamardinos I [2015]. Constraintbased causal discovery from multiple interventions 

over overlapping variable sets. J Mach Learn Res 

16:2147–2205.

United Nations [2013]. Globally harmonized system 

of classification and labelling of chemicals (GHS). 

ST/SG/AC.10/30/Rev.5. New York and Geneva, 

Switzerland: United Nations.

UNSCEAR [2015]. Sources, effects and risks of 

ionizing radiation: UNSCEAR 2012 report to the 

General Assembly, with scientific annexes. United 

Nations Scientific Committee on the Effects of 

Atomic Radiation. New York: United Nations. 

UNSCEAR [(2000]. Sources and effects of ionizing 

radiation. Annex I. Epidemiological evaluation of 

radiation-induced cancer. United Nations Scientific Committee on the Effects of Atomic Radiation. 

New York: United Nations.

Ury HK [1975]. Efficiency of case control studies 

with multiple controls per case: continuous or dichotomous data. Biometrics 31(3):643–649.

Valentin J [2002]. Basic anatomical and physiological 

data for use in radiological protection reference values. ICRP Publication 89: Approved by the commission in September 2001. Ann ICRP 32(3–4):1–277.

van Ravenzwaay B, Leibold E [2004]. The significance of in vitro rat skin absorption studies to human 

risk assessment. Toxicol In Vitro 18(2):219–225.

Van Wendel De Joode B, Van Hemmen JJ, Meijster 

T, Major V, London L, Kromhout H [2005a]. Reliability of a semi-quantitative method for dermal 

exposure assessment (DREAM). J Expo Anal Environ Epidemiol 15(1):111–120.

Van Wendel De Joode B, Vermeulen R, Van Hemmen JJ, Fransman W, Kromhout H [2005b]. Accuracy of a semiquantitative method for Dermal 

Exposure Assessment (DREAM). Occup Environ 

Med 62(9):623–632.

Vanderweele TJ [2009]. On the distinction between 

interaction and effect modification. Epidemiology 

20(6):863–871.

Vanem E [2012]. Ethics and fundamental principles 

of risk acceptance criteria. Saf Sci 50(4):958–967.

Vesterinen HM, Sena ES, Egan KJ, Hirst TC, Churolov L, Currie GL, Antonic A, Howells DW, Macleod MR [2014]. Meta-analysis of data from animal studies: a practical guide. J Neurosci Methods 

221:92–102.

Villeneuve DL, Garcia-Reyero N [2011]. Vision & 

strategy: predictive ecotoxicology in the 21st century. Environ Toxicol Chem 30(1):1–8.

von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche 

PC, Vandenbroucke JP [2007]. The Strengthening 

the Reporting of Observational Studies in epidemiology (STROBE) statement: guidelines for reporting 

observational studies. Lancet 370(9596):1453–1457.

Wacholder S [1995]. When measurement errors 

correlate with truth: Surprising effects of nondifferential misclassification. Epidemiology 6(2):157–161.

Wacholder S, Dosemeci M, Lubin JH [1991]. Blind 

assignment of exposure does not always prevent 

differential misclassification. Am J Epidemiol 134(4): 

433–437.

Wakeford R [2015]. Association and causation in 

epidemiology—half a century since the publication 

of Bradford Hill’s interpretational guidance. J R Soc 

Med 108(1):4–6.

Wallace RB, Kohatsu N [2008]. Wallace/MaxcyRosenau-Last public health & preventive medicine. 

New York: McGraw-Hill Medical. 

Wang L, Shaw PA, Mathelier HM, Kimmel SE, 

French B [2016]. Evaluating risk-prediction models using data from electronic health records. Ann 

Appl Stat 10(1):286–304.

Wang X, Xia T, Duch MC, Ji Z, Zhang H, Li R, Sun 

B, Lin S, Meng H, Liao YP, Wang M, Song TB, Yang 

Y, Hersam MC, Nel AE [2012]. Pluronic F108 
136 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

coating decreases the lung fibrosis potential of 

multiwall carbon nanotubes by reducing lysosomal 

injury. Nano Lett 12(6):3050–3061.

Wang X, Xia T, Ntim SA, Ji Z, Lin S, Meng H, Chung 

CH, George S, Zhang H, Wang M, Li N, Yang Y, 

Castranova V, Mitra S, Bonner JC, Nel AE [2011]. 

Dispersal state of multiwalled carbon nanotubes 

elicits profibrogenic cellular responses that correlate with fibrogenesis biomarkers and fibrosis in the 

murine lung. ACS Nano 5(12):9772–9787.

Warheit DB, Kreiling R, Levy LS [2016]. Relevance 

of the rat lung tumor response to particle overload 

for human risk assessment—update and interpretation of new data since ILSI 2000. Toxicology 

374(Supplement C):42–59.

Waters M, McKernan L, Maier A, Jayjock M, 

Schaeffer V, Brosseau L [2015]. Exposure estimation and interpretation of occupational risk: enhanced information for the occupational risk manager. J Occup Environ Hyg 12(Suppl 1):S99–S111.

Weed DL [2005]. Weight of evidence: a review of 

concept and methods. Risk Anal 25(6):1545–1557.

Weed DL [2018]. Analogy in causal inference: rethinking Austin Bradford Hill’s neglected consideration. Ann Epidemiol 28(5):343–346.

Wei GCG, Tanner MA [1990]. A Monte Carlo 

implementation of the em algorithm and the poor 

man’s data augmentation algorithms. J Am Stat Assoc 85(411):699–704.

Weingand K, Brown G, Hall R, Davies D, Gossett 

K, Neptun D, Waner T, Matsuzawa T, Salemink 

P, Froelke W, Provost JP, Dal Negro G, Batchelor 

J, Nomura M, Groetsch H, Boink A, Kimball J, 

Woodman D, York M, Fabianson-Johnson E, Lupart M, Melloni E [1996]. Harmonization of animal clinical pathology testing in toxicity and safety 

studies. The Joint Scientific Committee for International Harmonization of Clinical Pathology Testing. Fundam Appl Toxicol 29(2):198–201.

Weitzel JN, Blazer KR, MacDonald DJ, Culver JO, 

Offit K [2011]. Genetics, genomics, and cancer risk 

assessment: state of the art and future directions in 

the era of personalized medicine. CA Cancer J Clin 

61(5):327–359.

Wheeler M, Bailer AJ [2012]. Monotonic Bayesian semiparametric benchmark dose analysis. Risk 

Anal 32(7):1207–1218.

Wheeler MW, Bailer AJ [2007]. Properties of modelaveraged BMDLs: a study of model averaging in 

dichotomous response risk estimation. Risk Anal 

27(3):659–670.

Wheeler MW, Bailer AJ [2013]. An empirical comparison of low-dose extrapolation from points of 

departure (PoD) compared to extrapolations based 

upon methods that account for model uncertainty. 

Regul Toxicol Pharmacol 67(1):75–82.

Wheeler MW, Shao K, Bailer AJ [2015]. Quantile 

benchmark dose estimation for continuous endpoints. Environmetrics 26(5):363–372.

White E, Armstrong BK, Saracci R, Armstrong BK 

[2008]. Principles of exposure measurement in epidemiology: collecting, evaluating, and improving 

measures of disease risk factors. New York: Oxford 

University Press. 

White RH, Cote I, Zeise L, Fox M, Dominici F, 

Burke TA, White PD, Hattis DB, Samet JM [2009]. 

State-of-the-science workshop report: issues and 

approaches in low-dose-response extrapolation 

for environmental health risk assessment. Environ 

Health Perspect 117(2):283.

WHO [1977]. Manual of the international statistical classification of diseases, injuries, and causes of 

death: based on the recommendations of the Ninth 

Revision Conference, 1975 and adopted by the 

Twenty-Ninth World Health Assembly. Geneva, 

Switzerland: World Health Organization.

WHO [1994]. Assessing human health risk of 

chemicals: derivation of guidance values for healthbased exposure limits. Geneva, Switzerland: World 

Health Organization.

WHO [2000]. Evaluation and use of epidemiological 

evidence for environmental health risk assessment: 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 137

WHO guideline document. EUR/00/5020369. Copenhagen, Denmark: World Health Organization, 

Regional Office for Europe.

WHO [2005]. Chemical-specific adjustment factors for interspecies differences and human variability: guidance document for use of data in dose/

concentration-response assessment. Harmonization project document no. 2. Geneva, Switzerland: 

World Health Organization (WHO).

Wicherts JM, Veldkamp CLS, Augusteijn HEM, 

Bakker M, van Aert RCM, van Assen MALM 

[2016]. Degrees of Freedom in Planning, Running, 

Analyzing, and Reporting Psychological Studies: 

A Checklist to Avoid p-Hacking. Frontiers in Psychology 7(1832).

Wignall JA, Shapiro AJ, Wright FA, Woodruff TJ, 

Chiu WA, Guyton KZ, Rusyn I [2014]. Standardizing 

benchmark dose calculations to improve sciencebased decisions in human health assessments. Environ Health Perspect 122(5):499–505.

Wong BA [1999]. Inhalation exposure systems design, methods, and operation. In: Toxicology of the 

lung. Gardner DE, Crapo JD, McClellan RO, eds. 

Philadelphia, PA: Taylor and Francis, 1–53.

Wong BA [2007]. Inhalation exposure systems: 

design, methods and operation. Toxicol Pathol 

35(1):3–14.

Woodruff TJ, Sutton P [2014]. The navigation guide 

systematic review methodology: A rigorous and 

transparent method for translating environmental 

health science into better health outcomes. Environ 

Health Perspect 122(10):1007–1014.

Woodward M [2013]. Epidemiology: study design 

and data analysis. 3rd ed. Boca Raton, FL: CRC Press. 

Worster A, Haines T [2004]. Advanced statistics: 

understanding medical record review (MRR) studies. Acad Emerg Med 11(2):187–192.

Woskie SR, Smith TJ, Hammond K, Schenker MB, 

Garshick E, Speizer FE [1988]. Estimation of the 

diesel exhaust exposures of railroad workers: I. 

Current exposures. Am J Ind Med 13(3):381–394.

Xia T, Hamilton RF, Bonner JC, Crandall ED, 

Elder A, Fazlollahi F, Girtsman TA, Kim K, Mitra S, Ntim SA, Orr G, Tagmount M, Taylor AJ, 

Telesca D, Tolic A, Vulpe CD, Walker AJ, Wang 

X, Witzmann FA, Wu N, Xie Y, Zink JI, Nel 

A, Holian A [2013]. Interlaboratory evaluation 

of in vitro cytotoxicity and inflammatory responses to engineered nanomaterials: the NIEHS 

Nano GO Consortium. Environ Health Perspect 

121(6):683–690.

Yu C [1996]. Extrapolation modeling of particle 

deposition and retention from rats to humans. 

Particul Sci Technol 14(1):1–13.

Zaccai JH [2004]. How to assess epidemiological 

studies. Postgrad Med J 80(941):140.

Zdeb MS [1977]. The probability of developing 

cancer. Am J Epidemiol 106(1):6–16.

Zhang H, Ji Z, Xia T, Meng H, Low-Kam C, Liu R, 

Pokhrel S, Lin S, Wang X, Liao YP, Wang M, Li L, 

Rallo R, Damoiseaux R, Telesca D, Madler L, Cohen Y, Zink JI, Nel AE [2012]. Use of metal oxide 

nanoparticle band gap to develop a predictive paradigm for oxidative stress and acute pulmonary inflammation. ACS Nano 6(5):4349–4368.

Zhang Z, Preston DL, Sokolnikov M, Napier BA, 

Degteva M, Moroz B, Vostrotin V, Shiskina E, 

Birchall A, Stram DO [2017]. Correction of confidence intervals in excess relative risk models using 

Monte Carlo dosimetry systems with shared errors. 

PLoS One 12(4):e0174641.
This page intentionally left blank.
APPENDIX A 

Glossary
140 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 

Terms Def initions

absolute risk The probability that a disease-free individual will develop a given disease over a specified time, given age and other risk factors, and in the 

presence of competing risk.

adverse effect Changes in the morphology, physiology, growth, development, reproduction, or life span of an organism, system, or population that result 

in an impairment of functional capacity, an impairment of the capacity to compensate for additional stress, or an increase in susceptibility 

to other influences. 

adverse outcome pathway A conceptual construct describing a sequential chain of causally 

linked events at different levels of biological organization leading to 

an adverse effect.

aerodynamic equivalent diameter The diameter of a sphere with a standard density of one gram per cubic 

centimeter (1 g/cm3

), having the same terminal velocity when settling 

under gravity as the particle under consideration.

agent A chemical, biological, or physical entity that contacts a target.

allometric scaling Adjustment of data to allow for differences and making comparisons 

between species having dissimilar characteristics (e.g., in size, shape, 

and metabolism). Allometric scaling commonly refers to the relationship between metabolic rate and body size among mammalian species 

(e.g., the metabolic scaling between a laboratory rat and human).

aneuploidy A change in chromosome number from the species’ normal diploid or 

haploid number, other than an exact multiple of the haploid number 

(polyploidy).

associative causation Causal discovery based on relative risk or similar statistical measures 

of association. It is used in most papers that assert causal relationships 

and is often interpreted and applied by using Hill’s guidelines or similar qualitative weight-of-evidence criteria.

apical effect An observable outcome in a whole organism, such as a clinical sign 

or pathologic state, that is indicative of a disease state resulting from 

exposure to a toxicant.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 141

Terms Def initions

asymptotic relative efficiency The relative quality of two estimators as the sample size increases to infinity. 

This concept is best described with an example. Suppose an unknown parameter, θ, is to be estimated by using data y1, y2, ..., yn and that two estimators, 

θi= θi (y1,y2, ...,yn ); ί =1, 2, are to be compared. If the quality of each estimator 

is inversely proportional to its mean squared error MSE(θί)=E{(θί - θ)2

}, then 

the MSE efficiency of θ1 relative to that of θ2 is MSE(θ2) ⁄ MSE(θ1) which is a 

function of the sample size η and the asymptotic relative efficiency is given 

by {MSE(θ2) ⁄ MSE(θ1)}. The asymptotic relative efficiency from performing conditional logistic regression on sampled risk-sets, with m controls 

matched to each case, relative to the full risk-sets is m/(m+1).

attributable risk The risk that is regarded as being caused by exposure to the agent; also 

referred to as risk difference, excess risk, or added risk.

benchmark dose (BMD) A dose or concentration that produces a predetermined change in the 

response rate of an adverse effect, relative to the background response 

rate of the effect.

benchmark dose lower limit 

(BMDL)

The statistical lower confidence limit on the benchmark dose, which 

is typically used as the point of departure in dose-response analyses. 

The BMDL is usually set at the one-sided 95% lower confidence limit 

on the BMD.

benchmark response (BMR) A predetermined change in the response rate of an adverse effect relative to the background response rate of the effect.

biomarker Indicator of changes or events in biological systems. Biomarkers of exposure refer to cellular, biochemical, analytical, or molecular measures 

that are obtained from biological media such as tissues, cells, or fluids 

and are indicative of exposure to an agent.

chronic exposure A continuous or intermittent long-term contact between an agent and 

a target. (Other terms such as “long-term exposure” and “protracted 

exposure” are also used.)

clastogenicity The disruption or breakage of chromosomes, leading to sections of the 

chromosome being deleted, added, or rearranged.

confounding The mixing of the effects from the exposure of interest with the effects 

from other factor(s) on the risk of the adverse effect.

cytotoxicity The harmful effects to cell structure or function that ultimately lead to 

cell death.

dichotomous Dividing or branching into two parts. For example, dichotomous data 

can have only two values.
142 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 

Terms Def initions

dose Total amount of an agent administered to, taken up by, or absorbed by 

an organism, system, or (sub)population. Dose is also described as the 

amount of agent that enters a target after crossing an exposure surface. 

The term is often used interchangeably with exposure.

dose-response The relationship between the amount of an agent administered to, 

taken up by, or absorbed by an organism, system, or population and 

the change developed in that organism, system, or population in reaction to the agent. In NIOSH risk assessment, the dose-response is 

typically described as the conditional probability of the adverse effect 

in exposed workers at different levels of occupational exposure to the 

agent, given assumed levels for other direct causes of the adverse effect. 

Related term: exposure-response.

dose-response assessment The analysis of dose-response association between exposure to the 

agent and adverse effects. Ideally, the products of the dose-response 

assessment are unbiased estimates of the risk per unit dose that are 

used in risk characterization.

dosimetry The determination or measurement of the amount of an agent administered to, taken up by, or absorbed by an organism, system, or (sub)

population (see dose).

ecological fallacy An erroneous inference that may occur because an association observed between variables on an aggregate level does not necessarily 

represent or reflect the association at an individual level; a causal relationship that exists on a group level or among groups may not exist 

among the group’s individuals.

etiology The science of origins, causes, or causality of diseases or conditions.

excess relative risk A measure of association equivalent to the relative risk -1.

exposure Contact between an agent and a target. Contact takes place at an exposure surface over an exposure period.

exposure assessment The process of estimating or measuring the magnitude, frequency, and 

duration of exposure to an agent, along with the number and characteristics of the population exposed. Ideally, it describes the sources, 

pathways, routes, and uncertainties in the assessment. The use of this 

term is reserved for the description of risk assessments by the National 

Research Council [NRC 1983].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 143

Terms Def initions

exposure duration The length of time over which continuous or intermittent contacts 

occur between an agent and a target.

exposure event The occurrence of continuous contact between an agent and a target.

exposure frequency The number of exposure events in an exposure duration.

exposure index A measured or estimated quantity of exposure or dose.

exposure model A conceptual or mathematical representation of the exposure process.

exposure pathway The course an agent takes from the source to the target.

exposure route The way in which an agent enters a target after contact (e.g., by ingestion, inhalation, or dermal absorption).

exposure scenarios A combination of facts, assumptions, and inferences that define a discrete situation where potential exposures may occur. These may include 

source, exposed population, period of exposure, microenvironment(s), 

and activities. Scenarios are often created to aid exposure assessors in 

estimating exposure.

exposure-response The relationship between the intensity, frequency, or duration of exposure to a stressor or agent and the intensity, frequency, or duration of 

the subsequent biological response of the organism. Given varied use 

of the terms dose and exposure in many settings, exposure-response 

and dose-response are often used interchangeably. Related terms: 

concentration-response, dose-response.

external validity The degree to which study findings may apply, be generalized, or be 

transported to populations or groups that did not participate in the 

study.

extra risk The measure of the proportional increase in risk of an adverse effect, 

adjusted for the background incidence of the same effect.

genotoxicity A general description of all types of DNA or chromosome damage, 

such as breaks, adducts, mutations, aberrations, and aneuploidy.

hazard A condition or a set of circumstances that present a potential for harm 

(e.g., occupational illness or injury) to an organism, system, or population.

hazard function In survival analyses, the rate of failure at an instant in time, t, given 

that the individual survives until t.
144 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 

Terms Def initions

hazard identification Identification of the type and nature of adverse effects that an agent 

has an inherent capacity to cause in an organism, system, or population. Hazard identification is the initial stage of the risk assessment. 

The products of hazard identification are definitions of the agent and 

outcome used in dose-response analysis.

hazard ratio In survival analysis, the hazard function (rate) of one individual (e.g., the 

exposed), divided by that of another individual (e.g., the unexposed), 

typically holding all other predictors constant (i.e., a rate ratio).

hematotoxicity Adverse changes to the blood, caused by exposure to an agent.

human equivalent concentration 

(HEC)

The human concentration (for inhalation exposure) or dose (for other 

routes of exposure) of an agent that is believed to induce the same 

magnitude of toxic effect as the experimental animal species concentration or dose.

immediately dangerous to life or 

health (IDLH)

An exposure condition or environment that is likely to cause death 

or immediate or delayed permanent adverse health effects or prevent escape from such an environment. The IDLH values developed 

by NIOSH characterize these high-risk exposure concentrations and 

conditions.

information bias A bias in the effect estimate that occurs from systematic inaccuracies 

in the measurement of either the exposure or the adverse effect.

intake The process by which an agent crosses an outer exposure surface of a 

target without passing an absorption barrier (i.e., through ingestion or 

inhalation).

internal validity The degree to which study findings are free from bias.

interpretation bias A bias that arises from improper inference or speculation based on a 

naïve or deliberate lack of impartiality by the interpreter.

job-exposure matrix (JEM) A cross-classification of jobs/tasks and exposure level spanning a 

specified period. The JEM is used to estimate exposure indices that 

vary by job and time.

key event An empirically observable precursor step that is itself a necessary element of the mode of action or is a biologically based marker for such 

an element.

knot The boundary between categories of the regressor in a regression 

model using a spline function.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 145

Terms Def initions

limit of detection (LOD) For an analytical procedure, the lowest amount or concentration of 

the analyte that is reliably distinguishable from the absence of analyte 

(i.e., low false-negative rate). For example, for air sampling methods, 

NIOSH defines the LOD as the mass of the analyte that gives a mean 

signal that is three standard deviations above the mean blank signal.

limit of quantification (LOQ) For an analytical procedure, the amount or concentration of the analyte at which quantitative results can be reported with a high degree 

of confidence. The high degree of confidence is based on a set of acceptance criteria that are assay-specific. For example, for air sampling 

methods, NIOSH defines the LOQ as the larger of: (a) the mass corresponding to the mean blank signal + 10 standard deviations of the blank 

signal or (b) the mass above which recovery is ≥75%.

lowest observed adverse effect level 

(LOAEL)

The lowest dose or concentration at which there are biologically significant increases in frequency or severity of adverse effects between 

the exposed population and its appropriate control group.

manipulative causation Causal discovery based on the intuition that changing causes also 

changes their effects. Manipulative causation is implied by mechanistic causation, where given a network of mechanisms that affect the 

frequency or severity of adverse effects, manipulation of these mechanisms will change the frequency or severity of adverse effects.

measurand A quantity intended to be measured.

measurement accuracy The agreement between a measured quantity value and a true quantity 

value of a measurand.

measurement error Any discrepancy between the true quantity and the measurand. The 

true quantity is considered unique and is unknowable.

measurement precision The agreement between measured quantities obtained by replicate 

measurements on the same or similar objects under specified conditions.

mechanism of action The underlying biochemical interactions, usually at the molecular 

level, that lead to the mode of action at the cellular level and ultimately the expression of the adverse effect. The mechanism of action is 

a more detailed understanding and description of events than is mode 

of action.

medical follow-back A data collection technique in epidemiologic studies where data from 

one source (e.g., self-report or registry linkage) is verified or augmented 

by using medical records.
146 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 

Terms Def initions

mode of action (MOA) A sequence of key events and processes, starting with interaction of 

an agent with a cell, proceeding through operational and anatomical 

changes, and resulting in the adverse effect.

monotonic A relationship, sequence, or trend is said to be monotonically increasing if each value is greater than or equal to the previous one and monotonically decreasing if each value is less than or equal to the previous 

one. Monotonic responses may be linear or nonlinear, but the slope 

does not change sign.

mutagenicity The ability of an agent to induce or generate heritable changes (mutations) of the genotype in a cell because of alterations in or loss of 

genetic material.

no observed adverse effect level 

(NOAEL)

The highest dose level at which there are no biologically significant 

increases in the frequency or severity of adverse effects between the 

exposed population and its appropriate control; some effects may be 

produced at this dose level, but they are not considered adverse or 

precursors of adverse effects observed.

occupational exposure limit (OEL) The allowable concentration or intensity of a hazardous agent in the 

worker’s work environment over a period. Generally expressed as an 

8-hour time-weighted average or as a short-term exposure limit of 15 

or 30 minutes.

occupational risk The potential and severity of adverse effects in workers from their exposure to workplace hazards.

odds ratio A measure of association in comparative studies, particularly casecontrol studies. It is the ratio of the odds that an outcome will occur, 

given a particular exposure to the odds of the outcome occurring in 

the absence of that exposure.

parametric model A mathematical model of association that is wholly dependent on assumptions about the distribution of the data that are represented by a 

finite set of explicit parameters. For example, a simple linear regression with one variable has two parameters (the coefficient and the intercept), which completely explain the data.

pharmacokinetics The study of the absorption, distribution, metabolism, and elimination of exogenous chemicals in biological systems.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 147

Terms Def initions

physiologically based 

pharmacokinetic (PBPK) 

model

A multicompartmental mathematical model based on the known 

physiology of an organism used to quantify the absorption, distribution, metabolism, and elimination of exogenous chemicals following 

exposure. In this context, the compartments represent actual tissue 

and organ spaces and their volumes are the physical volumes of those 

organs and tissues.

point of departure (PoD) The estimate of dose-response at an exposure in the low range of (or 

just below) the observable data. Various approaches are available for 

its estimation; the simplest defines the PoD as the no observed adverse effect level (NOAEL) or the lowest observed adverse effect level 

(LOAEL) from an animal toxicologic or human epidemiologic study.

pneumoconioses The group of interstitial lung diseases, mostly of occupational origin, 

caused by the inhalation of mineral or metal dusts, such as asbestosis, silicosis, coal workers’ pneumoconiosis (black lung disease), and 

chronic beryllium disease.

publication bias A bias from an editorial preference for publishing particular findings, 

which distorts inferences made from available evidence.

random error Variation of results and inferences from the truth, occurring only 

because of chance. Random measurement error is a component of 

measurement error that in replicate measurements varies in an unpredictable manner.

rate ratio A measure of association that quantifies the relationship between an 

exposure and a health outcome from an epidemiologic study, calculated as the ratio of incidence rates or mortality rates of two groups.

recommended exposure limit (REL) An exposure limit recommended by NIOSH, expressed as an 8-hour 

time-weighted average airborne concentration of a chemical during a 

40-hour workweek over a 45-year working lifetime. RELs are usually 

based on a quantitative risk assessment, when available, but may also 

depend on the limit of quantification of the analytical exposure measurement method. RELs are published in NIOSH Criteria Documents 

and Current Intelligence Bulletins and are compiled in the NIOSH 

Pocket Guide to Chemical Hazards.

relative risk The ratio of the risk (disease probability) observed in the exposed (intervention) group to that observed in the unexposed (control) group; 

also used as a general term for measures of association on a relative 

scale, including risk ratio, rate ratio, hazard ratio, odds ratio, standardized incidence ratio, and standardized mortality ratio.
148 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 

Terms Def initions

reliability The extent to which multiple assessments are consistent.

risk The probability and severity of an adverse effect in an organism, system, or population caused under specified circumstances by exposure 

to an agent. In NIOSH risk assessment, risk is more narrowly defined 

as the incidence of the adverse effect (e.g., disease onset) occurring 

in subject(s) over a specified period of time, given that the subject(s) 

were disease-free at the beginning of that period. Under this definition, risk is a measure corresponding to an average individual-specific 

risk (i.e., cumulative incidence).

risk assessment Determination of the relationship between the predicted exposure 

and adverse effects, in four major steps: hazard identification, doseresponse assessment, exposure assessment, and risk characterization. 

The NIOSH risk assessment process is limited to occupational exposures and primarily includes hazard identification, dose-response 

assessment, and risk characterization, while omitting exposure assessment. In NIOSH assessments, exposure data are evaluated as a part of 

hazard identification.

risk-based decision A risk management decision using risk assessment as the basis for 

decision-making.

risk characterization The qualitative and, wherever possible, quantitative determination, 

including attendant uncertainties, of the probability of occurrence 

of known and potential adverse effects of an agent in workers under 

defined exposure conditions. In NIOSH risk assessment, risk characterization is the culmination of planning, problem formulation, and 

analysis phases to form a risk basis for NIOSH recommendations on 

limiting occupational exposure levels.

risk-informed decision A risk management decision using risk assessment as an input to 

decision-making.

risk management The managerial, decision-making, and control process intended to 

avert intolerable risk.

risk management limit for 

carcinogens (RML-CA)

An exposure limit for chemical carcinogens that represents a starting 

place for controlling exposures, preferably based on target risk. For 

example, NIOSH will set the RML-CA at a risk of one excess cancer 

case in 10,000 workers in a 45-year working lifetime, when analytically 

feasible. For more information, see the NIOSH Chemical Carcinogen 

Policy [NIOSH 2017].
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 149

Terms Def initions

risk ratio A measure of association that quantifies the association between an 

exposure and a health outcome from an epidemiologic study, calculated as the ratio of incidence proportions of two groups.

short-term exposure limit (STEL) Generally, a 15-minute TWA exposure that should not be exceeded 

at any time during a workday. Some NIOSH STELs are based on a 

different short-term exposure duration.

similar exposure groups Workers having the same exposure profile because of similarity and 

frequency of tasks performed.

spline function A mathematical function that is used for regression model interpolation or smoothing. The boundary between categories of the regressor 

are called the knots or joint points of the spline.

standardized incidence ratio (SIR) The ratio of the observed number of disease cases in the study population to the number of cases that would be expected, based on disease 

rates in the referent population that are applicable to the characteristics (such as age, race, gender, and calendar period) in the study 

population.

standardized mortality ratio (SMR) The ratio of the observed number of deaths in a study population to 

the number of deaths that would be expected, based on death rates in 

the referent population that are applicable to the characteristics (such 

as age, race, gender, and calendar period) in the study population.

stressor Any physical, chemical, biological, or psychosocial entity that can 

induce an adverse effect.

systematic error A component of measurement error that in replicate measurements 

remains constant or varies in a predictable manner.

target Any biological entity that receives an exposure or a dose (for example, 

an organ, an individual, or a population).

tolerable risk The region in the risk continuum that can, for the time being, be 

tolerated, assuming that the risk is minimized by appropriate control 

procedures.

toxicity The inherent property of an agent having the potential to cause an adverse effect when an organism, system, or population is exposed to that 

agent. Toxicity is usually defined with reference to the dose, the way 

the agent is administered and distributed in time (single or repeated 

doses), the type and severity of injury, the time needed to produce the 

injury, the nature of the organism(s) affected, and other relevant conditions. In some settings, toxicity refers to the adverse effect.
150 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Terms Def initions

uncertainty factor (UF) A “safety” factor ≥1 that is used as a divisor to adjust the safe-dose 

point of departure downward to account for variation and uncertainty 

in estimates from experimental or observational data.

underlying cause of death The disease or injury that initiated the train of events leading directly 

to death or the circumstances of the accident or violence that produced the fatal injury.

uptake The process by which an agent crosses an absorption barrier.

validity The quality of being logically or factually sound; the extent to which 

the measure describes that which is being measured. It is the degree 

to which inferences drawn are valid. Study validity comprises internal 

and external validity.

weight of evidence on 

toxicity (WoE)

The nature and extent to which the available data support the hypothesis that an agent causes a defined adverse effect (e.g., cancer in humans).
APPENDIX B 

Sources of Uncertainty in Risk Estimation
152 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

B.1 Validity

Validity is the quality of being logically or factually 

sound; the extent to which a measure describes that 

which is being measured; and the degree to which 

inferences drawn are valid. Study validity comprises internal and external validity [Campbell and 

Stanley 1963]. The former is the degree to which 

study findings are free from bias, whereas the latter is the degree to which study findings may apply, 

be generalized, or be transported to populations 

or groups that did not participate in the study. The 

two components of validity are not independent 

and can entail tradeoffs [Grimes and Schulz 2002]. 

In general, sound observational and experimental 

studies are aimed to provide internally valid causal 

effect estimates from a study sample, rather than 

estimates of an externally valid causal effect for a 

target population [Edwards et al. 2017]. Internal 

validity is the sine qua non of research supporting 

risk assessment; therefore, information on sources of potential bias (e.g., selection, information, 

and confounding) is the focus of this appendix. 

External validity has received much less attention, 

although it can also be an important consideration when applying research findings to assessing risks in a broad population of workers [Steckler and McLeroy 2008]. External validity has been 

described as generalizability and transportability 

[Lesko et al. 2017]. In this context, generalizability 

refers to the quality of inference from a potentially 

biased sample of the target population to the full 

target population. Transportability refers to inference in the target population from observations 

in a partial or completely non-overlapping sample 

population. In either case, measuring external validity requires data and assumptions beyond those 

needed for internal validity. Thus, relating a causal effect observed in a single study to the target 

population can be difficult and, in some cases, impractical. Generalizing and transport methodologies in observational research are sparse [Hernán 

and Vanderweele 2011; Lesko et al. 2017; Pearl 

and Bareinboim 2014]. Potential exceptions are 

statistical methods for combining diverse studies, such as meta-analyses and data pooling [Pearl 

and Bareinboim 2014]. A general algorithm for the 

transport of experimental results to populations is 

provided by Bareinboim and Pearl [2013].

The tendency to conflate statistical generalizability 

or representativeness with scientific generalizability should be avoided. There are many examples in 

which representativeness can detract from research 

goals and ultimately hinder scientific inference 

[Rothman et al. 2013]. Instead, external validity 

can be considered as the explanation of the reasons 

in which the scientific findings are valid beyond a 

specific study. In practice, evaluation of external 

validity in risk assessment requires more scientific 

judgment compared to internal validity.

B.2 Selection Bias

The term selection bias is used to describe many biases that are themselves a distortion in the estimate 

of effect that results from the manner in which the 

study subjects are selected from the source population [Gail and Benichou 2000]. These include 

biases resulting in differential follow-up, recall, 

self-selection, volunteering or non-response, and 

sampling frames. Selection bias is possible in all 

observational studies and particularly so in casecontrol studies, because the outcome is known at 

study inception. For example, MacMahon et al. 

[1981] conducted a hospital-based case-control 

study that reported a strong association between 

coffee drinking and pancreatic cancer. Controls 

were selected from “... all other patients who were 

under the care of the same physician in the same 

hospital at the time of an interview with a patient 

with pancreatic cancer” [MacMahon et al. 1981]. 

This selection process resulted in a large proportion of controls who presented mainly with gastrointestinal disorders; therefore, these patients 

may have been advised by physicians not to consume coffee [Feinstein et al. 1981; MacMahon et 

al. 1981; Silverman et al. 1983]. The abnormally 

low odds of coffee consumption among controls 

would cause a spurious positive association between coffee intake and pancreatic cancer. This 

bias may have been avoided by selecting controls 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 153

from patients hospitalized for conditions not requiring a change in diet [Silverman et al. 1983]. 

Primary control of selection bias is managed by 

study design. The avoidance of selection bias in 

case-control studies is accomplished by drawing 

cases and controls from the same study base; therefore, it is imperative that the study base be welldefined before sampling. Other methods include 

maximizing participation rates, using randomized 

sampling protocols, and applying sound inclusion/

exclusion criteria. 

There is often overlap between confounding and 

selection bias; therefore, secondary control of selection bias can sometimes be achieved by treating 

identifying factors as confounders in analyses and 

controlling for confounding accordingly. For example, if union workers are more likely than office 

workers to participate in a study and be exposed, 

then partial control of the bias may be realized by 

including job information as a confounder. Finally, 

sensitivity analyses using an array of inclusion criteria can help characterize the potential for significant selection bias and define a dataset for use in 

risk assessment. 

The risk assessor should be able to recognize potential sources of selection bias in an evaluation 

of the study design. The risk assessor should give 

more weight to studies that have best addressed 

this source of bias through design, control, and 

sensitivity analyses. The risk assessor should pay 

special attention to (non-nested) case-control studies, which are most vulnerable to selection biases. 

When reviewing the design of studies for the potential for selection bias, risk assessors should consider the following questions:

• What study design was used and where does 

this design fall in the hierarchy for WoE? Preferred studies will provide a detailed description of the study design, which includes limitations that are inherent to the design.

• Has the study population been sufficiently 

described to determine potential differences 

between study and control groups (i.e., do 

inclusion criteria differ between groups)? 

Preferred studies will include a detailed description of the characteristics of the study 

and control groups.

• What methods were used to select study participants? How could those excluded from 

study have affected study results had they 

been included? Preferred studies will include 

a description of the exclusion and inclusion 

criteria used for study participation and the 

methods used to reduce the potential for bias. 

• What steps were taken to maximize participation rates? Low participation is indicative 

of a potential for selection bias. 

• Is participation non-differential with respect to 

exposure? Case-control studies are particularly 

vulnerable to differential selection to study and 

control groups with respect to exposure, given 

that case status is known at enumeration. 

• Is there significant loss to follow-up? Loss to 

follow-up is typically less than 10% in welldesigned studies.

B.3 Information Bias

Information bias, sometimes referred to as data collection bias, measurement error, or misclassification 

bias, is a distortion in the effect estimate that occurs 

when the measurement of either the exposure or the 

adverse effect is systematically inaccurate [Gail and 

Benichou 2000]. In this context, information bias 

is a study execution bias that is restricted to data 

on study participants (i.e., the sample population). 

Information biases may stem from errors in the 

measurement instrument (instrument bias), data 

source (data source bias), the observer or investigator (observer bias), and/or the subject (subject bias). 

Given limitations in available data, observational 

studies are particularly prone to several sources of 

information bias. For example, exposure data can 

be biased when collected with prior knowledge of 

case status (as in a case-control study). If exposure 

is self-reported, then a recall bias (a form of subject 

bias) may result from differential self-reporting of 
154 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

exposure status among cases and the control group 

when cases are aware of a potential association between exposure and their disease. If exposure data 

are collected by interview, then the interviewer 

must be blinded to case status to reduce the potential for an observer bias. Likewise, if measurement 

data are collected, then care must be taken to ensure that identical procedures were used for both 

cases and controls. In general, when assessing the 

presence of information bias in a study under review, the risk assessor should ask these questions: 

• Was the study information obtained in the 

same way for all comparison groups?

• Was the information on exposure and other 

explanatory factors collected by persons 

blinded to case status? 

Errors in the data are usually separated by data 

type, such that the term measurement error is reserved for errors in continuous data and the term 

misclassification error refers to errors in discrete 

data. Measurement error of explanatory variables 

used in analyses is unavoidable, even in the bestdesigned studies. Risk assessors should have a firm 

understanding of the potential effects from these 

errors in studies selected for dose-response assessment; therefore, a detailed discussion is provided 

in the following section. This discussion is primarily in the context of errors in measurement of the 

exposure of interest; however, the concepts presented are shared by all data sources vulnerable to 

an information bias.

B.3.1 Measurement Error and 

Misclassification

In the context of exposure, measurement (observation) error refers to any discrepancy between 

the true exposure, X, and the imperfect measured 

value, W; thus, it is analogous to exposure misclassification*

. By this strict definition, measurement 

*

Although used to describe all measurement error, the term misclassification is sometimes limited to errors in qualitative indices and 

replaced with misspecification when referencing errors in numerical 

indices.

error comprises both systematic and random components. Random errors are stochastic fluctuations 

in observed values around the true (but unknown) 

value, without directional preference. Systematic 

error or bias refers to inaccuracies in measured values that are inherent to the measurement system. 

Systematic error (bias) is a difference between the 

mean of observed values and the true mean value, 

for a given true value, and typically arises from 

inaccuracy in measured or estimated values that 

are inherent to the measurement process. Bias can 

be unintentional or deliberate, such as the use of 

conservative assumptions to increase the margin 

of safety in a protection system. A common source 

of systematic error in exposure estimates are methods used to report “nondetects,” i.e., measurements 

below a detection threshold [Helsel 2005]. In these 

cases, the true value lies somewhere between the 

null and the detection threshold. In practice, nondetects are typically recorded as zero (likely underestimation of exposure), the limit of detection 

(LOD, which is a likely overestimation of exposure), or simply omitted (a bias in either direction, 

depending on the use of the data). Here, the LOD is 

the lowest amount or concentration of the analyte 

that is reliably distinguishable from the absence of 

analyte. For example, in developing air sampling 

methods, NIOSH defined the LOD as the mass of 

the analyte that gives a mean signal that is three 

standard deviations above the mean blank signal 

[NIOSH 1995b]. Methods to account for nondetects can range from simple substitution (e.g., 

substituting with LOD/2 or LOD/20.5) to complex 

parametric and nonparametric statistical modeling 

[Helsel 2005; NCRP 2010]. 

Measurement error can be shared or unshared 

[Hoffmann et al. 2018]. If the error in dose among 

individuals, groups, and time is independent and 

identically distributed (a general assumption in 

most error structure models), the error is unshared. In contrast, correlations in error components between individuals, groups, or time represent shared error. Consider the case in which a set 

of data from personal air samples used a calibration 

coefficient that overestimated dose by 20%. This error is shared among workers in the set of data.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 155

In general, measurement error reduces statistical 

power for trend tests because of added variance 

and may bias effect measures in dose-response 

analyses. The influence on risk estimates depends 

on the combination of error characteristics (Berkson or classical random errors, or shared errors) 

and model specification, and this influence can 

range from negligible effects to a strong bias in either direction [Armstrong 1998; Nieuwenhuijsen 

2010]. Furthermore, measurement error is often 

thought of only in terms of the primary predictor; 

however, risk assessors should be mindful that a 

dose-response relationship could also be strongly 

influenced by measurement error in covariates that 

confound or mediate effects of interest. 

Risk assessors should be reasonably assured that 

the data selected for dose-response analyses are 

free of a potential for significant bias. This assurance is partly gained through rigorous adherence 

to estimation methods designed to avoid bias, 

such as observance of the data hierarchy, blinding 

assessors to case status, using and comparing multiple indices, and validating estimates. Well-conducted epidemiologic studies typically pay careful 

attention to obvious sources of systematic error 

in exposure estimates, but analyses have been 

generally conducted without considering residual measurement error effects [Jurek et al. 2006]. 

This is because assessments of measurement errors often require elaborate tests of reliability 

and validity, which are infrequently performed, 

if even feasible. Furthermore, many investigators 

assume that random measurement error always 

induces bias toward a null association; therefore, 

they incorrectly conclude it cannot cause spurious positive findings. Consequently, information 

needed to account for measurement error in risk 

analyses may be lacking. When data are available, 

researchers have suggested methods for adjusting 

estimates to account for random error or assessing its potential effects in dose-response analyses 

[Carroll et al. 2006; French et al. 2004; Hoffman 

et al. 2007; Maldonado 2008; Mallick et al. 2002; 

Meliker et al. 2010; Schafer et al. 2001; Spiegelman 

and Valanis 1998; Stayner et al. 2007; Thomas et 

al. 1993a]. These methods can involve (1) integrating validation data into regression calibration 

or likelihood-based methods to produce adjusted 

estimates [Hoffman et al. 2007; Mallick et al. 2002; 

Schafer et al. 2001; Spiegelman and Valanis 1998] 

or (2) using Monte Carlo simulation to predict a 

range of plausible estimates [French et al. 2004; 

Meliker et al. 2010; Stayner et al. 2007]. In the 

former case, risk assessors should be cautious of 

adjustments made on the basis of inadequate information that could induce a potentially stronger 

bias relative to unadjusted values. 

In summary, there may be few options available 

to risk assessors regarding limiting the potential 

effects of measurement error in dose-response 

analyses. Nevertheless, it is important for risk 

assessors to have a fundamental understanding 

of measurement error and its associated effects 

so that they can better describe and account for 

the limitations in analyses that support quantitative risk assessment. Seminal works on measurement error and dose-response modeling should 

be reviewed [Armstrong 1998; Carroll et al. 2006; 

Fuller 1987; Ron and Hoffman 1999; Thomas et 

al. 1993b]. Some general concepts are discussed 

below. 

B.3.1.1 Differential versus 

Nondifferential Error

Exposure measurement error is either differential 

or nondifferential, contingent on its relation to the 

dependent variable (e.g., disease status). Error that 

is independent of case status (and other predictors) is said to be nondifferential. It is commonly 

thought that nondifferential error results in bias 

toward a null association, which is a proven condition of binary variables or continuous variables in 

which the magnitude and direction of the measurement error are independent of the true value (i.e., 

the classical error model; see Table B-1). However, 

there are examples of nondifferential error in polytomous and continuous exposure measures that induce bias away from the null [Dosemeci et al. 1990; 

Greenland and Gustafson 2006; Wacholder 1995].
156 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Table B-1. Direction of bias caused by nondifferential measurement 

error of the primary predictor variable.

Predictor Scale Bias Expected

Binary Biases the effect measure toward a null association

Polytomous Estimates of trend across adjacent categories are biased downward. Estimates 

from comparison of categories can be biased in either direction. 

Numerical Classical error biases regression coefficients toward zero. Berkson error (i.e., 

random error that is statistically independent from the observed variable) leads 

to little or no bias in coefficients in most regression models.

Differential error can result in serious bias in either 

direction. For example, workers diagnosed with 

leukemia may be more apt to report or may have 

more thorough histories of benzene exposure than 

workers who are cancer free. In this instance, leukemia cases will appear to have higher exposures, 

thus biasing the association between benzene exposure and leukemia away from the null. Differential exposure error is unlikely if exposure data are 

collected prior to the disease outcome or without 

prior knowledge of the hypothesized association. 

Therefore, the primary means to avoid differential error is to ensure that exposure estimates were 

made while blinded to case status and that case ascertainment is, to the extent practical, independent 

of exposure status. When using data from previous 

studies, risk assessors should examine the study 

design for any potential weaknesses that may lead 

to differential measurement error. Common sources of exposure information that are vulnerable to 

differential measurement error are self-reports or 

proxy reports, medical records, and compensation 

records. Cautious interpretation is also warranted 

for studies involving case ascertainment by differential diagnoses of “occupational diseases” (e.g., 

silicosis, asbestosis, malignant mesothelioma, and 

chronic beryllium disease). In certain situations, 

nondifferential exposure information can be restructured to induce differential misclassification 

[Dosemeci et al. 1990; Flegal et al. 1991; Wacholder et al. 1991]. For example, combining categories 

of a polytomous exposure variable or constructing 

exposure categories from continuous exposure 

data can result in differential measurement error.

B.3.1.2 Categorical Indices

Measurement error in qualitative data is typically 

described as the probability of exposure misclassification. For example, the error in a dichotomous 

exposure index can be expressed by its probability of correctly classifying an exposed worker (i.e., 

sensitivity) and the probability of correctly classifying an unexposed worker (i.e., specificity). A matrix of misclassification probabilities can be used 

to describe errors in indices with more than two 

levels. Misclassification probabilities are generally 

determined in validity studies comparing exposure 

estimates for a sample of workers in the study to 

estimates derived from another source that is believed to be as precise or better. Random (nondifferential) measurement error in a dichotomous exposure variable will always attenuate its effect, i.e., 

suggest that the agent under study is less toxic than 

it truly is. Trends across ordered categories of polytomous exposure variables will also be attenuated 

by nondifferential measurement error; however, 

comparisons between categories can be biased in 

either direction [Armstrong 1998]. 

B.3.1.3 Error Models for Numerical Indices: 

Classical versus Berkson Error

Two approaches to modeling random measurement error for numerical data are the classical and 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 157

Berkson models. Settings where observations are 

subject to random variation from factors such as 

instrument imprecision and recording errors may 

be amenable to a classical model of measurement 

error, e.g., W=X+U, where measurement error, 

U, is a random variable with mean zero variance, 

VARu , and is independent of X. The observed exposure is equal to the true (but unobserved) exposure plus some measurement error; therefore, 

average values obtained from replicate measures 

are unbiased estimates of the true exposure but 

will always have greater variability than the true 

exposure. In contrast, the Berkson error model, expressed as E(X|W=w) = w, arises when a single estimate, w, is applied to several individuals who have 

differing values of the quantity being estimated that 

average to w. In this model, the true exposure is 

more variable than the observed exposure. For example, assigning the average measured concentration from an ambient air monitor to the group of 

workers can be modeled by Berkson error. Modeling of the measurement errors may be approached 

with the use of additive or multiplicative structures 

under each of these approaches. 

The error form is significant regarding dose-response 

analyses. For example, consider the simple case of 

a univariate linear dose-response model: E(Y)= 

α+βXX, where the regression of response variable, 

Y, on the independent variable, X (with variance, 

VAR), has parameters α and β. If X is unavailable 

and exposure measure W with classical additive error (i.e., W=X+U) is substituted, then the resulting 

regression model E(Y) = α*+ βWW has the slope 

parameter βW = βX VARX ⁄ (VARX + VARU), where 

the quantity (VARX + VARU) is the variance of the 

measured variable. The ratio of true to measured 

value variances (referred to by Fuller [1987] as the 

reliability ratio λ) must be less than unity; therefore, classical error results in attenuation of the 

observed linear dose-response [Fuller 1987]. The 

degree of attenuation is relative to the quantity 

VARU ⁄ VARX , such that smaller measurement error 

or larger spread of true values reduces bias. For example, the effect of classical error in a cumulative 

dose estimate is likely less than in a single measurement of the same magnitude, given that the error of 

the single measurement is larger relative to that of 

multiple measurements comprising the cumulative 

dose. In contrast, attenuation of linear regression 

coefficients does not result from Berkson error. 

Recall that for Berkson error, E(X |W) = W, thus 

E(Y |W) = α+βXW, and therefore the estimator (βX) 

is not attenuated [Carroll et al. 2006].

B.3.1.4 Additive, Multiplicative, and Mixed 

Error Structures

Error structures (for numerical values) are generalized by two limiting cases: additive error, in which 

the variance is constant for different magnitudes of 

the measurand; and multiplicative error, in which 

the variance increases with increasing values of 

the measurand. The classical multiplicative error 

model can be expressed by W=XeU, such that there 

is additivity on the logarithmic scale [i.e., ln(W) = 

ln(X) + U]. Measurement error can be modeled by 

using additive, multiplicative, or a combination of 

each, resulting in a mixed error structure. Replicate 

measurements of several occupational agents have 

shown a multiplicative error structure. As in the 

additive measurement error model, the increased 

variance from multiplicative error attenuates the 

observed dose-response; however, the effect is larger at higher exposures, resulting in the appearance 

of downward curvature with increasing values of 

the error-prone measurements of exposure [Carroll et al. 2006]. An attenuated response at higher 

exposure levels has been observed in numerous 

occupational studies and in simulations [Carroll et 

al. 2006; Stayner et al. 2003; Steenland et al. 2015]. 

Nevertheless, there is considerably less literature 

on accounting for multiplicative or mixed error 

structures in predictor variables of dose-response 

regression models. The subsequent effects on these 

models vary by structure; therefore, some notion of 

the error structure is important for understanding 

subsequent model limitations. 

B.3.1.5 Errors in Confounders and Effect 

Modifiers

As a general rule, random measurement error in 

a confounder in which the error is not correlated 
158 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

with other measures or the exposure of interest 

tends to increase confounding from that covariate 

[Armstrong 1998]. This means that the effect measure of interest is likely to lie between the unadjusted value (crude measure) and a value obtained 

under complete control of confounding (i.e., residual confounding from incomplete control). The 

directionality of induced bias depends on the direction of the confounding effect. The amount of 

bias depends on the strength of the confounder and 

the reliability ratio. As in confounding, random 

measurement error in an effect modifier tends to 

attenuate its effect modification; therefore, the ability to observe risk difference among groups is diminished [Armstrong 1998].

B.3.1.6 Errors in Adverse Effect Definitions

Information bias is also plausible from misclassification or measurement error in the adverse effect. 

For example, consider a cancer incidence study of 

U.S. workers in which cases in the exposed population are ascertained from a single state registry. If 

incidence rates in the exposed group are compared 

to standardized national rates, then the resulting 

effect measure (e.g., SIR) is likely biased from underascertainment of cases due to some migration 

out of the state by the workforce. Thus, cancer incidence studies within the U.S. are improved with 

ascertainment involving multiple states. In this 

case, the misclassification is differential and the 

bias is likely toward a null association. As a similar example, consider a case control study in which 

the study population and adverse effect data are 

drawn from electronic health records (EHRs). As 

in the previous example, cases may be missed if diagnosed outside of the EHR catchment area (e.g., a 

single clinic or group of clinics), and the potential 

for error increases with decreasing catchment area 

size. In this scenario, the affluent workers in the 

study have a more flexible health insurance plan; 

therefore, they are more likely to be diagnosed outside of the catchment area (and be missed). These 

same workers may have less exposure because of 

their job assignment. Under these conditions, the 

misclassification is differential with respect to exposure. As previously discussed, the resulting bias 

can be in either direction [Wang et al. 2016]. 

Misclassification can also occur from differences 

in diagnostic criteria used for defining the adverse 

effect. These criteria can vary by data source and 

by time. For example, the ICD published by the 

World Health Organization has been the standard 

diagnostic tool used for epidemiology since the late 

1940s. Multiple revisions to the ICD have occurred 

throughout the years, given changes in diagnostic 

criteria. The definitions of certain diseases (e.g., 

hematopoietic cancers) have dramatically changed 

over the course of the ICD; therefore, studies published at different times may not have comparable 

disease definitions. As another example, consider an adverse effect defined on the basis of 

data abstracted from medical records. The reliability and validity of data in each individual medical 

record are vulnerable to different interpretations of 

different scenarios and often by different observers 

[Worster and Haines 2004]. 

Researchers acquire adverse effect data with several different approaches tailored to the response 

definition, data availability, and study feasibility. 

Thus, the adverse effect data may stem from direct 

measurements (e.g., lung function tests), existing 

health outcome databases (e.g., National Death Index, disease registries, compensation databases), 

medical records (paper or EHRs), and patient (or 

proxy) self-reports. These sources are not without 

error, and the potential for bias is dependent on 

the magnitude of these errors. For example, EHRs 

appear to be a promising source of medical information suitable for risk assessment. However, data 

residing in these systems are inputted by imperfect 

systems and persons. Sources of misinformation 

associated with medical records include physician 

misdiagnoses, flawed laboratory results, and erroneous patient self-reporting [Ash et al. 2004; Burnum 1989; Luck et al. 2000; Worster and Haines 

2004]. Thus, data collected prospectively with use 

of study criteria that were defined beforehand are 

likely to be superior to data abstracted from EHRs.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 159

In summary, the potential for bias from mismeasurement of the adverse effect is reduced when case 

definitions and ascertainment are the same among 

comparison groups. Nondifferential misclassification of the adverse effect with respect to risk factor 

exposure will likely result in an underestimation 

of the effect (i.e., bias toward a null association), 

whereas differential misclassification may result 

in a bias in either direction. When selecting studies for data synthesis, the risk assessor should confirm consistency in adverse effect definition among 

comparison groups in data used for risk assessment. 

Studies with well-defined adverse effects that are 

consistent throughout observation should be given 

more weight. For example, data from a compulsory 

reporting system (e.g., cancer registry) is preferred 

to information gathered by self-report. Studies with 

poorly defined adverse effects should be avoided. 

Risk assessors must also consider limitations that 

are inherent to the sources of adverse effect data. The 

risk assessor must consider the potential bias in estimates that may result from errors in the source data 

and weight the evidence accordingly.

B.4 Confounding

With respect to causal inference, confounding has 

been described as a mixing of the effects from extraneous factors (confounders) with the effect of interest [Checkoway et al. 2004]. This mixing occurs 

when the comparison groups (e.g., exposed and 

unexposed workers) have differing background 

risks of disease. There are many definitions of confounders; however, perhaps the most complete is 

that suggested by McNamee (2003), who posited 

that a factor should be considered a confounder if 

three conditions are met:

1. The factor is a cause of the disease, or a surrogate measure of a cause, in unexposed people. 

Factors satisfying this condition are called risk 

factors.

2. The factor is correlated, positively or negatively, 

with exposure in the study population. If the 

study population is classified into exposed and 

unexposed groups, this means that the factor 

has a different prevalence in the two groups. 

3. The factor is not affected by the exposure (i.e., 

does not reside on the causal pathway) [McNamee 2003].

Disease risk factors can comprise a wide array, 

including demographic factors (age, sex, race), 

lifestyle factors (smoking habits, diet, and alcohol 

use), or exposures to other agents in the workplace 

or elsewhere. In study planning, all known or suspected risk factors should be identified, especially 

those factors most apt to confound dose-response 

associations. This information is needed to achieve 

appropriate confounding control and characterize 

the potential influence on effect measures from residual confounding.

Methods to control for confounding are generally 

related to study design or analysis. Design methods 

are meant to ensure that the exposed group is comparable to or exchangeable with the referent group 

with respect to the potential confounders [Greenland et al. 1999]. Exchangeability is the concept 

that response distributions in exchangeable comparison groups are identical under the same exposure conditions. These methods include restriction, 

randomization (i.e., clinical trial), and matching on 

potential confounders. In practice, there is limited 

success in finding exchangeable comparison groups 

in observational studies; therefore, these studies 

tend to rely on analytical methods for controlling 

confounding, such as stratified analyses and multiple regression. For example, dose-response analysis 

in a longitudinal study may use Poisson regression 

to control for confounding effects of age (an important confounder for most chronic illnesses) on the 

exposure interest by either background stratifying 

on age or including age as a covariate in the model. 

Similarly, a nested case-control study of the same 

cohort may use conditional logistic regression with 

age (attained age of the case) as the time scale. Both 

approaches are used extensively in occupational 

epidemiology.

In general, reasonable control for important demographic risk factors (e.g., age, sex, and race) and calendar period is achieved in most published epidemiologic studies. However, measures of association 
160 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

are still vulnerable to confounding effects from incomplete control of measured risk factors or from 

no control of unmeasured risk factors. Smoking is 

known to cause several types of cancer and nonmalignant disease. If smoking prevalence is also 

related to exposure status, then smoking might be 

a confounder. The resultant bias could be in either 

direction (i.e., positive or negative confounding), 

depending on the smoking characteristics of the 

comparison populations. For example, consider 

that blue-collar workers tend to use tobacco products more than white-collar workers. If blue-collar 

workers are also more likely to be exposed than 

white-collar workers (a reasonable assumption in 

some workplaces), then smoking can be a correlate 

of exposure [Lee et al. 2004; Stellman et al. 1988]. 

Under these conditions, smoking could confound 

the effect of occupational exposure on a smokingrelated disease. The expected effect in this case is 

positive confounding of the exposure effect by 

smoking, which means the measure of association 

will be biased away from the null without control 

for smoking. Unfortunately, information on the 

smoking habits of workers in most longitudinal 

studies is rarely available; therefore, direct adjustment for confounding effects of smoking are seldom seen. Instead, researchers might use indirect 

methods for adjustment [Axelson and Steenland 

1988; Richardson 2010]. In the example above, job 

descriptions could be used as a proxy for smoking. 

Socioeconomic status is a well-known proxy for 

many lifestyle factors, including smoking, which 

may confound a dose-risk relationship [Lantz et al. 

1998; McFadden et al. 2008]. One could also examine alternative adverse effects that are strongly 

associated with the unknown confounder but not 

with the exposure of interest [Richardson 2010]. 

An observed (but unexpected) dose-risk relationship between the agent of interest and alternative 

adverse effects is indicative of residual confounding. At the very least, researchers should provide 

some information on the potential for significant 

bias because of incomplete control of known or 

suspected confounders.

Confounding can also occur in occupational studies that do not account for concomitant workplace 

exposures. For example, Sathiakumar et al. [2015] 

examined the relationship between styrene exposure and leukemia in a large pooled study of workers in North American synthetic rubber plants 

[Sathiakumar et al. 2015]. Styrene-exposed workers were also exposed to 1, 3-butadiene, which is a 

known human leukemogen [IARC 2012]. Quantitative estimates of cumulative exposure to 1,3-butadiene and styrene were calculated. Statistical 

analyses used Cox proportional hazards regression 

models with age as the time scale and adjustment 

for race, year of birth, and plant. Modest positive 

dose-response associations between leukemia and 

cumulative exposures to both agents were observed 

in separate models; however, the independent effects of styrene exposure could not be determined 

because of its strong correlation with 1,3-butadiene. Thus, the carcinogenic effects of these agents 

in combination appear hopelessly entangled in 

these workers, and the dose-response observed for 

styrene could be due wholly, or in part, to unmeasured confounding by 1,3-butadiene. 

Whether a study is a valid contributor to hazard 

identification depends on how well the published 

results address the potential for confounding. In 

turn, resultant datasets must also inform and support the analytical approaches used in the subsequent dose-response assessment. Thus, the risk assessor should evaluate the adequacy for control of 

measured and unmeasured confounders in studies 

under review. When unmeasured risk factors are 

identified, the risk assessor should evaluate the 

steps taken by researchers to reduce the potential 

for significant bias from residual confounding by 

these risk factors. The risk assessor should also 

consider the potential for unknown risk factors 

and assess their potential impact on internal validity. In all instances, the risk assessor should give 

more weight to studies with measures of association that are least likely to be affected by residual 

confounding. 

B.5 Healthy Worker Effects

Another important potential source of bias in occupational studies is healthy worker effects. These 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 161

effects primarily occur from two points of participant selection—into the study at the time of hire 

and out of the workforce at time of termination—

and as such are commonly referred to as the “hire 

effect” and “survivor effect,” respectively [Arrighi 

and Hertz-Picciotto 1993; Arrighi and Hertz-Picciotto 1994; Fox and Collier 1976]. A third aspect 

to healthy worker effects is the natural decline in 

health status with time since hire [Checkoway et 

al. 2004]. Finally, healthy worker effects do not apply equally to all outcomes or to all groups within a 

study population (e.g., different races, ages, or demographics). For example, healthy worker effects 

are generally greater among minority populations 

compared to Caucasian males because of different 

class structures in working and referent populations [McMichael 1976]. Given distinct differences 

in the sources of potential bias and methods available for control, risk assessors should consider 

these aspects separately as they are discussed in the 

following sections. 

B.5.1 Healthy Worker Hire Effect

The healthy worker hire effect (HWHE) results 

from increased employment eligibility among 

healthier persons, which can be exacerbated by hiring practices that screen against poor health (e.g., 

pre-employment exams). These conditions can result in a group of persons of interest (i.e., workers) 

who are in better overall health than the comparison group, irrespective of exposure status. HWHE 

is typically observed in external comparisons (e.g., 

SMR or SIR studies using the general population 

as referent); however, some employers have used 

medical screening information for job placement 

within the industry, which could bias results from 

internal comparisons. For external comparisons, 

the HWHE results in a deficit in risk compared 

to true effects in some outcomes, particularly in 

chronic diseases most associated with lifestyle factors (e.g., diabetes mellitus and cardiovascular diseases). An alternative comparison group, such as 

a similar working population that is unexposed to 

the agent of interest, may reduce this bias. When 

job assignment is influenced by medical screening, 

the direction of the potential bias depends on the 

relationship between job assignment and exposure. 

Risk assessors need to be wary of the potential for a 

strong HWHE in data from external comparisons. 

Unfortunately, there are few options afforded to risk 

assessors for accounting for HWHE. Risk estimates 

from external comparisons can be adjusted with 

use of information on potential biases [Burstyn et 

al. 2015; Kirkeleit et al. 2013; Park et al. 1991]; however, data availability to support adjustments may 

be limited. Moreover, adjustments to published 

SMRs are crude approximations at best and could 

result in bias themselves. When available, data 

from internal comparisons should be preferred for 

dose-response analyses of working populations. 

When internal comparison data are available, the 

risk assessor should evaluate the potential for bias 

from continued medical surveillance. 

B.5.2 Healthy Worker Survivor Effect

The healthy worker survivor effect (HWSE) occurs when healthy workers continue to work and 

unhealthy workers leave employment prematurely 

or are reassigned to less hazardous work because of 

their poor health. A potential exacerbating factor 

is a possible health benefit from employment compared to unemployment, such as the beneficial effect of physical exertion in reducing cardiovascular 

risk. In any case, exposure is a condition of employment, which itself may be conditional on exposure, 

health status, or both. The likely effect from these 

relationships is attenuation of the estimated doseresponse. 

Methods for mitigating HWSE vary. Control of 

these effects in longitudinal studies tends to involve 

one or more factors: age at hire, employment duration, employment status, time since hire, and age at 

risk [Checkoway et al. 2004]. Methods have typically involved confounding control by restriction, 

by matching (stratifying), or by covariate adjustment. For example, HWSE controls have included 

restricting the analysis to participants alive after a 

minimum length of time since hire; adjusting for 

employment status as a time-dependent variable; 
162 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

 

Figure B-1. Directed acyclic graph representing healthy worker survivor effects. Let j be index age; A, exposure 

cumulated over follow-up; X, continuous exposure; W, index employment status; U, a common cause of W and 

T; and T, the index survival time. The three components of survival bias, as expressed by Naimi et al. [2013], are 

shown as C1 , C2 , and C3. Adapted from Naimi et al. [2013].

and using time lags (exposure windows). However, 

it is now recognized that the nature of HWSE may 

preclude complete control of the effect by standard 

approaches [Buckley et al. 2015; Naimi et al. 2013]. 

This is perhaps best explained by the causal diagram in Figure B-1, in which Naimi et al. [2013] 

described three components of HWSE. Component 1 (C1

) is the association between prior exposure 

(X(j-1)) and employment at age j (Wj), component 2 

(C2

) is the association between employment status 

and subsequent exposure (X(j)), and component 

3 is the association between employment status 

and survival time (T). If all three components are 

present, then a standard confounding control that 

treats employment status as a time-varying confounder in Cox, Poisson, or logistic regression (i.e., 

standard methods) is inappropriate and may result 

in bias (Table B-2). When all three components are 

likely, then the use of more sophisticated nonstandard analytic methods, such as G-estimation, parametric G-formula, and inverse-probability-weighted 

marginal structural models, is preferred to appropriately control for healthy worker survivor bias 

[Buckley et al. 2015]. This is because the deleterious health effect from a prior exposure may affect 

employment status (i.e., violates the third aspect of 

a confounder).

Of course, a key consideration is whether it is reasonable for exposure to influence employment status. For example, strong survivor effects are much 

less likely to occur in late-onset adverse effects (e.g., 

malignant mesothelioma) compared to debilitating 

effects (or precursor effects) that present during

employment years (e.g., occupational asthma). 

Thus, risk assessors must evaluate the severity and 

likelihood of survivor effects on the basis of spatial 

and temporal relationships between employment, 

exposure, and outcome. 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 163

Table B-2. Associations contributing to Healthy Worker Survivor Effect (as shown in Figure B-1) 

and recommended analytic methods. (Adapted from Naimi et al. [2013].)

Component

C1 C2 C3

Confounding 

by employment 

status?

Employment 

status affected by 

prior exposure? Analysis method*

1 1 1 Yes Yes Non-standard

1 1 0 No Yes Standard, employment status unadjusted

1 0 1 No Yes Standard, employment status unadjusted

0 1 1 Yes No Standard, employment status adjusted

1 0 0 No Yes Standard, employment status unadjusted

0 1 0 No No Standard, employment status unadjusted

0 0 1 No No Standard, employment status unadjusted

0 0 0 No No Standard, employment status unadjusted

*Examples of standard analytic methods are logistic, Poisson, and Cox modeling. Non-standard methods are G-formula and G-estimation of a 

structural nested model.

When there is a potential for strong HWSE, studies have recently employed methods, such as Gestimation [Bjor et al. 2015; Chevrier et al. 2012; 

Naimi et al. 2014; Picciotto et al. 2016] and Gformula [Cole et al. 2013; Neophytou et al. 2016], 

in structural nested models or accelerated-failuretime models. Although promising, studies using 

these methods are currently sparse; therefore, the 

evidence available for hazard identification will 

likely be restricted to studies in which residual 

HWSE is likely when prior exposure affects employment status. Nevertheless, these new methods 

may be well suited for dose-response modeling in 

the dose-response assessment.

HWSE may also result from underestimation of 

prevalent cases in cross-sectional studies when the 

adverse effect causes persons to leave employment 

or move to less hazardous jobs. For example, Eisen 

et al. [1997] identified significant selection bias in 

estimates of asthma prevalence in a cross-sectional 

study of workers exposed to metalworking fluids. 

In that study, workers transferred to a job with less 

exposure because of the onset of asthma symptoms. 

This resulted in underestimating disease prevalence 

in those exposed and subsequently overestimating 

prevalence among unexposed persons at the time of 

the health survey (i.e., a negative dose-response). 

However, a reanalysis of the data using exposure 

and disease status at the time of asthma onset instead of time of survey revealed significant excess 

risk. When evaluating occupational cross-sectional 

studies for risk assessment, risk assessors should 

determine whether there is a potential for the adverse effect to influence work status. Studies in 

which influence is likely should be avoided, unless 

HWSE has been adequately addressed in the study 

design and execution.

B.5.3 Length of Follow-up

Although often given less attention, the length 

that a working population is followed in longitudinal studies is an important consideration when 

evaluating the potential for healthy worker effects. 

The strength of HWHE diminishes with increasing time since first employment and is essentially 

no longer present at post-retirement age. Retirees 

also lose the survival advantage of continued employment. Collectively, this suggests a decrease in 

healthy worker effects with increasing follow-up 
164 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

[McMichael 1976]. Risk assessors should be wary 

of studies of chronic (or latent) adverse effects that 

have relatively short follow-up periods. For example, a recent cohort study of mortality patterns 

among paid Australian firefighters reported a statistically significant deficit in risk of cancer death 

(SMR = 0.81; 95% CI, 0.72–0.90); however, the 

average length of follow-up was less than 16 years 

[Glass et al. 2016]. In contrast, a comparable study 

of U.S. career firefighters reported excess cancer 

mortality (SMR = 1.14; 95% CI, 1.10–1.18) in a 

cohort with average follow-up of 29 years [Daniels 

et al. 2014]. The relatively short follow-up period 

in the Australian study is unlikely to counter the 

selection effects due to pre-employment health criteria for firefighters. In addition to person-years at 

risk, the percent deceased is a useful indicator of 

the length and quality of follow-up in cohort studies, especially in examinations of adverse effects 

that generally occur late in life (e.g., cancer). In 

the previous example, fewer than 5% of the cohort 

of full-time Australian firefighters were deceased, 

compared to over 40% of the U.S. firefighter cohort. 

In the previous examples, we discussed the potential for selection bias toward a null association 

with decreased follow-up. Studies of chronic diseases characterized by a short latent period and 

short-lived risk after exposure may provide for excess risks that decrease with increasing length of 

follow-up. For example, leukemia risks that were 

attenuated with increased follow-up have been observed in follow-on studies of working populations 

exposed to benzene and ionizing radiation [Boice 

et al. 2011; Daniels et al. 2013; Rinsky et al. 2002; 

Silver et al. 2002]. Thus, the risk assessor must also 

consider effect modification by temporal factors 

that are associated with the length of follow-up. 

B.6 Other Potentially Important 

Sources of Bias 

B.6.1 Model-form 

Misspecification Error

It is generally understood that any mathematic 

model of the dose-response is not the “true model” 

that generated the observed data; at best, it only 

approximates truth [Posada and Buckley 2004]. 

Thus, model-form misspecification results when 

the mathematical structure of the assumed causal 

model substantively differs from the true biologic 

dose-response. The error can be sizeable when the 

assumptions of the model are not appropriate for 

the data. Errors in variable selection and/or definition (i.e., variable coding) can lead to spurious 

dose-response associations. This error is particularly problematic in hazard identification because 

investigators rarely discuss statistical methods used 

to counter model-form error. At a minimum, not 

accounting for this error results in overvaluing estimate precision.

Various methods are available to reduce modelform bias and aid in model selection, such as penalized criteria (e.g., AIC, BIC, and Mallow’s Cp), 

cross validation methods, and model averaging 

(aggregation) techniques [Arlot and Celisse 2010; 

Buckland et al. 1997; Burnham and Anderson 

2002; Burnham and Anderson 2004; Cox Jr and 

Ricci 2005; Posada and Buckley 2004; Symonds 

and Moussalli 2011]. In particular, the use of multimodel ensemble methods (e.g., Bayesian model 

averaging and simulated inference) that combine 

results from multiple plausible models typically 

provides for greater accuracy than that obtained 

from any single model [Buckland et al. 1997; Raftery et al. 1997]. For this reason, in dose-response 

analyses NIOSH generally prefers ensemble methods to statistical model selection criteria when 

data allow. An exception is a case with compelling 

mechanistic evidence of a specific dose-response 

function (see section 5.4).

B.6.2 Publication, Interpretation, 

and Analysis Biases

A publication bias refers to an editorial preference for publishing findings, which distorts inferences made from available evidence. For example, 

a positive results bias may occur when authors and 

editors are more likely to publish positive findings 

rather than null findings. Publication bias can also 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 165

occur when there is reluctance to publish disparate 

or controversial results, or when an emerging issue drives publication such that preliminary data 

are more likely to be published. Publication bias is 

plausible in all studies; however, observational and 

experimental animal studies are more susceptible 

than randomized clinical trials [Easterbrook et al. 

1991]. In all cases, published data can misinform 

on the consistency of evidence used for hazard 

identification. Risk assessors should be cognizant 

of the potential for publication bias and give appropriate attention to all findings, including those 

from negative studies. Attempts should be made to 

uncover relevant unpublished works.

Interpretation bias arises from improper inference 

or speculation based on a naïve or deliberate lack of 

impartiality by the interpreter. In this case, the interpreters are the study researchers, who interpret 

their findings at publication, or risk assessors, who 

translate findings for risk assessment. Research objectivity is always challenged by the ever-present 

interaction between data and judgement; therefore, 

interpretation is never completely independent of 

opinion, notion, or conviction [Kaptchuk 2003]. 

The potential for significant interpretation bias is 

likely smaller in the peer-reviewed literature than 

in trade journals and commercially funded technical reports. Information on potential conflicts of 

interest or disclosures can be useful in assessing 

the potential for interpretation bias. A willingness 

to examine alternative interpretations by investigators and risk assessors alike will lessen the potential 

for bias. Rigorous peer and public reviews also aid 

in avoiding interpretation bias.

Analyses bias results for errors in analyzing the 

data, such as inappropriate analytical strategies 

(e.g., overmatching, model misspecification, and 

post hoc analyses). In the course of any study, researchers make several decisions on data collection 

and analysis, including exploration of analytic alternatives. In some instances, these decisions cannot or are not made beforehand; therefore, they 

are likely informed by study information. This 

construct is referred to as “researcher degrees of 

freedom” [Simmons et al. 2011], which can lead 

to higher false-positive rates and inflated effect 

sizes (inflation bias) [Ioannidis 2008; Wicherts et 

al. 2016]. For example, exhaustive exploitation of 

study data to achieve significant findings is a source 

of inflation bias called p-hacking or data dredging 

[Head et al. 2015; Raj et al. 2017]. As in publication bias, p-hacking results in a conditioning of 

the literature by presenting only true or false positives. Uncovering the bias can be difficult, in part 

because many researchers do not recognize it as a 

real problem. The bias can be reduced by specifying 

statistical analyses a priori in an analysis plan. In 

general, post hoc analyses should be avoided. Study 

designs that are best described as exploratory are 

most vulnerable to significant bias from p-hacking 

[Teixeira 2018]. Ideally, studies should be restricted 

to testing specific hypotheses.

B.6.3 Effect Modification and 

Interaction

The terms effect modification and interaction have 

been used interchangeably in the literature; the former is seemingly preferred by epidemiologists and 

the latter by statisticians. It has been proposed that, 

in the strictest sense, these terms describe different phenomena [Vanderweele 2009]. Effect modification is described as a condition in which the 

exposure-related effect varies by levels of an extraneous factor [Checkoway et al. 2004]. Typically, the 

extraneous factor is a descriptor of subpopulations 

(e.g., gender, race); therefore, effect modification 

may elucidate susceptibility differences in the population. For example, suppose a study reported an 

association between exposure to agent X and lung 

cancer in women but not in men. In this case, gender is the effect modifier of agent X for causing lung 

cancer. There is one intervention (exposure), and 

the susceptible population is women. In contrast, 

an interaction specifically refers to the effect of two 

exposures together to be different (either more or 

less) than the combination of the two effects considered separately. Thus, an interaction describes 

the causal effects of the two exposures combined. 

For example, the joint effects of radon exposure 

and smoking status on lung cancer differ such that 

the excess relative risk per unit of radon exposure 

among nonsmokers is higher than that of smokers 
166 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

[Lubin et al. 1995a]. In this case, there are two possible interventions (smoking and radon exposure). 

Interaction effects can range from profoundly antagonistic to strongly synergistic. Unfortunately, 

most studies available for risk assessment have not 

examined effect modification by factors other than 

race or gender, and information is usually insufficient to draw conclusions on potential interactions 

[Knol and VanderWeele 2012]. 

B.6.4 Random Error 

Random error is the variation of results and inferences from the truth, occurring only because of 

chance. Effect measures are influenced by random 

variation in many components of an epidemiologic 

study. For example, a major contributor to random 

error in human studies is the process used to select study participants. This process is referred to 

as sampling, and the random error contribution 

is known as sampling variation or sampling error 

[Rothman et al. 2008]. Random variation around 

true values related to estimates used in statistical 

models is another source of random error. 

The common measure of random error in an estimation process is its variance, and the inverse of 

variance is a measure of statistical precision of the 

estimate. Precision can be improved by increasing 

the sample size, thus reducing the variance. This 

variance can also be reduced for a given sample size 

through design improvements; this is referred to as 

increasing study efficiency [Rothman et al. 2008]. 

Typically, the random error that is associated with 

the point estimate reported in a study is reflected 

by its associated confidence interval or p-value. 

In hazard identification, more weight is generally 

given to effect estimates with better precision (e.g., 

narrower confidence intervals). Nevertheless, estimate precision does not reflect a lack of bias from 

systematic errors. Moreover, random measurement 

error can also lead to biased estimates of the doseresponse [Carroll et al. 2006]. For example, as previously discussed, when explanatory variables (e.g., 

dose) are measured with error, the regression coefficient in dose-response models is typically biased 

toward the null. 

Unfortunately, deleterious effects of random error 

are rarely accounted for in epidemiologic studies, 

although recently some studies of health effects associated with ionizing radiation have made headway. In particular, regression calibration and Monte Carlo simulation have been used sparingly to 

account for uncertainty in dose-response analyses 

in studies relying on complex dosimetry systems 

subject to shared and unshared measurement error 

[Fearn et al. 2008; Pierce et al. 2008; Simon et al. 

2015; Stram et al. 2015; Zhang et al. 2017].
APPENDIX C 

EMERGING PRACTICES
168 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

C.1 Occupational Exposure 

Banding

Occupational Exposure Limits (OELs) play a critical role in protecting workers and emergency response personnel from exposure to dangerous 

concentrations of hazardous materials [Schulte et 

al. 2010]. In the absence of an OEL, determining 

the appropriate controls needed to protect workers from chemical exposures can be challenging. 

According to the EPA, the Toxic Substances Control Act (TSCA) Chemical Substance Inventory 

currently contains over 85,000 chemicals that are 

commercially available [EPA 2015], yet only about 

1,000 of these chemicals have been assigned an 

authoritative (government, consensus, or peer reviewed) OEL. Furthermore, the rate at which new 

chemicals are being introduced into commerce 

significantly outpaces OEL development, creating 

a need for guidance on thousands of chemicals that 

lack reliable exposure limits. Occupational exposure banding, also known as hazard banding or 

health hazard banding, is a systematic process that 

uses both qualitative and quantitative hazard information on selected health effect endpoints to identify potential inhalation-based exposure ranges 

or categories. The NIOSH occupational exposure 

banding process seeks to create a consistent, documented process to characterize chemical hazards 

so that timely, well-informed risk management 

decisions can be made for chemicals lacking OELs 

[NIOSH 2019].

The concept of using hazard-based categories to 

communicate potential health concerns, alert workers to the need for risk management, and inform exposure control requirements is not new. Numerous 

hazard classification and category-based systems 

have seen extensive use in the occupational setting. 

Such systems are deeply embedded in occupational 

hygiene practice, particularly in the pharmaceutical industry, and are also elements of well-developed, modern hazard communication programs 

(e.g., United Nations 2013 Globally Harmonized 

System of Classification and Labelling of Chemicals [2013]). The NIOSH occupational exposure 

banding process is distinguished from other hazard 

classification and category-based systems in several 

ways. These unique attributes of the NIOSH process include (1) a three-tiered system that allows 

users of varying expertise to utilize the process, (2) 

determination of potential health impacts based on 

nine toxicological endpoints separately, (3) hazardbased categories linked to quantitative exposure 

ranges, and (4) assessment of the process via extensive evaluation exercises to determine accuracy 

and repeatability.

To apply the NIOSH occupational exposure banding process, it is important to understand the three 

tiers of the process. Each tier has different requirements for data sufficiency, which allows a variety 

of stakeholders to use the process in many different situations. The most appropriate tier for banding depends on the availability and quality of the 

data, how the data will be used, and the training 

and expertise of the user. Whereas Tier 1 requires 

relatively little information and modest specialized training, each successive tier necessitates more 

chemical-specific data and more user expertise to 

assign an Occupational Exposure Band (OEB) successfully. A primary goal of Tier 1 is to give the user 

a quick summary of the most important health effects associated with exposure to the chemical of 

interest and quickly identify extremely toxic chemicals that should be considered for substitution or 

elimination. Tier 2 requires the user to examine several publicly available databases and extract relevant 

toxicological and weight-of-evidence data to be used 

in the NIOSH banding algorithm. Tier 3 employs 

expert judgment to critically evaluate experimental 

data and discern toxicological outcomes. 

Another important component of the NIOSH occupational exposure banding process is the inclusion of five exposure bands. Occupational exposure banding uses limited chemical toxicity data 

to group chemicals into one of five bands, ranging 

from A through E. These bands, or OEBs, define 

the range of exposures expected to be protective of 

worker health. Band E is the most protective band 

for the most dangerous chemicals, whereas band A 

is the least protective for the least dangerous. One 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 169

major benefit of occupational exposure banding is 

that the amount of time and data required to categorize a chemical into an OEB is significantly less 

than that required to develop an OEL.

The burden of worker and responder exposure to 

potentially hazardous chemicals that lack authoritative OELs is considerable, and the need for risk management for these chemicals is clear. Occupational 

exposure banding is one way to provide this type of 

guidance. An OEB provides more than a range of exposures that is expected to be protective of worker 

health. Rather, an OEB can be utilized to identify 

potential health effects and target organs, inform 

implementation of control interventions and preparedness plans, inform medical surveillance decisions, and provide critical information quickly.

For more information about the NIOSH occupational exposure banding process and the companion electronic banding tool (e-Tool)), refer to 

the NIOSH occupational exposure banding safety 

and health web topic page at: https://www.cdc.gov/

niosh/topics/oeb/default.html 

C.2 Emerging Alternatives To 

Assessing Apical Endpoints

An apical endpoint is an observable outcome in a 

whole organism, such as a clinical sign or pathologic state, that is indicative of a disease state that 

can result from exposure to a toxicant [NRC 2007]. 

For risk assessment, it is typically the final stage 

of disease progression. Adverse effects are generally related to traditional apical endpoints such as 

death, reproductive failure, or developmental dysfunction [Villeneuve and Garcia-Reyero 2011]. In 

some instances, data on the apical effect are not 

available; therefore, the risk assessment may rely 

on a non-apical surrogate that lies on the adverse 

effect pathway (Figure C-1) between the molecular 

initiating event and the adverse effect (e.g., functional genomics and biomarkers). These sub-organism 

effects are sometimes referred to as precursor effects. 

It has been suggested that the future of risk assessment is likely to shift away from toxicity testing of 

apical endpoints and move toward research evaluating biologically significant perturbations in toxicity pathways at earlier stages of the disease state. 

This research is anticipated to use a combination 

of computational biology and high-throughput in 

vitro tests of human cells and tissues [NRC 2007].

C.3 Biomarkers 

A biomarker is defined as a characteristic that is 

objectively measured and evaluated as an indicator 

of normal physiologic processes, pathologic processes, pharmacologic responses to a therapeutic 

intervention, or susceptibility [Atkinson et al. 2001; 

Schulte 1993]. In the context of an adverse effect, 

the biomarker refers to a biological analyte that 

predicts the individual’s disease state. Biomarkers 

include conventional measures, such as blood pressure, blood cholesterol, and enzyme levels; however, recent advancements have focused on cellular, 

genetic, and molecular markers that are sought as 

screening tools for early diagnosis of a severe disease (e.g., lung cancer and cardiovascular disease). 

The utility of defining sets of responses based on 

multiple genomic, transcriptomic, proteomic, and 

metabolomic markers and processes and/or second 

messenger and other biochemical pathways is an 

evolving area of work [Cote et al. 2016]. 

Risk assessors prefer to measure early indicators 

of serious health effects rather than wait for frank 

expression of disease. For example, lung cancer is 

a rare (<60 cases per 100,000 person-years) and 

serious (<20% survival after 5 years from diagnosis) adverse effect observed predominantly at ages 

65 years or older and at later stages of disease progression [Howlader et al. 2016]. Epidemiologic 

studies of occupational lung cancer require large 

populations who are observed over a long period 

to ensure adequate statistical power for effect sizes 

typically observed. A biomarker intended for early 

indication of lung cancer could act to relax some of 

these design requirements. Research suggests that 

exhaled breath contains organic compounds from 

metabolic processes that can vary between healthy 

subjects and subjects with lung cancer, making it a 
170 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

Chemical

Properties

Metabolites

Receptor/Ligand

Interaction

DNA Binding

Protein Oxidation

Gene Activation

Protein Production

Altered Signaling

Protein Depletion

Altered Physiology

Disrupted Homeostasis

Altered Tissue

Development

or Function

Lethality

Impaired Development

Impaired Reproduction

Cancer

Toxicant

MacroMolecular 

Interact ions

Cellular

Responses

Organ 

Responses

Organism

Responses

Toxicity Pathway

Chemical Characterization Non-apical Endpoints Apical Endpoints

Direct Molecular

Initiating Event

Figure C-1. Diagram of the adverse outcome pathway. A chemical (or its metabolites) interacts with a molecular 

target to initiate leading a sequence of higher-order effects to produce an adverse effect relevant to risk assessment. Adapted from Ankley et al. [2010].

potentially viable biomarker for early onset of disease [Dent et al. 2013]. If the relationships between 

dose, the exhaled breath condensate analytes of interest, and lung cancer can be adequately characterized, then exhaled breath condensate may also 

be a useful response metric in future dose-response 

analyses of lung carcinogens. 

Quantitative risk assessment of biomarkers is an 

area of active research and development and has 

been successfully used for risk assessment only in 

very limited situations [Cote et al. 2016; Poirier 

2016]. As such information evolves, the risk assessor must be prepared to consider whether such 

exposure-biomarker associations are useful relationships to model in occupational risk assessment.

C.4 Use of Genetics and 

Epigenetics in Risk 

Assessment

A growing body of literature demonstrates that genetic and epigenetic factors condition biological 

responses to occupational and environmental hazards or serve as targets of them. Generally, genetic 

and epigenetic data might be used as endpoints in 

hazard identification, indicators of exposure, effect modifiers in exposure estimation and doseresponse modeling, and descriptors of mode of 

action (MoA) in characterization of toxicity pathways. Vast amounts of genetic and epigenetic data 

may be generated by high-throughput technologies. Ideally, these data can be useful for assessing 

variability and reducing uncertainty in extrapolations and to help identify previously unidentified 

biological perturbations that may be of interest in 

risk assessment [Schulte et al. 2015].

One of the most critical areas to understand in the 

incorporation of genetic and epigenetic information 

in risk assessment is in the area of gene-environment interactions. The term gene-environment interaction can involve a range of interpretations of 

joint effects, including the risk of a single genotype 

across a range of environmental exposures, or the 

risk of exposure across a range of genotypes. Many 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 171

of the potential approaches to evaluating the impact 

of gene-environment interactions are reviewed in 

Schulte et al. [2015]. 

Future risk assessments may involve acquired 

changes in the somatic genome or changes in the 

epigenetics, which comprise the factors influencing 

expression of the genome. Techniques for addressing these require deep knowledge of mechanisms of 

action of toxic agents and well-defined experimental 

designs to address specific risk assessment questions. 

The elucidation of perturbations in genetic and epigenetic information on human health is likely to be 

a rich area for future risk assessment. A framework 

for organizing the research around these types of 

risk assessment questions can be found in Schulte et 

al. [2015]. One area where some progress has been 

made in developing genetics for quantitative risk assessment is in the use of high-throughput analyses, 

as described in the next section.

C.5 Molecular Toxicology and 

High-Throughput Analysis

Because of the lack of full toxicologic data on most 

chemicals, NIOSH is investigating the utility of 

high-throughput screening and in vitro short-term 

tests for occupational risk assessment. In the past 

decade, there has been an exponential increase in 

the publication of new toxicity data focusing on 

genomic analysis using high-throughput screening 

and in vitro short-term exposure. The paradigm for 

assessing chemical risks to human health is rapidly 

changing because of the availability of this toxicogenomic information and because of increased understanding of the gene-environment interactions. 

In 2007, the U.S. Environmental Protection Agency 

(EPA) requested that the National Research Council (NRC) conduct a complete review of toxicity 

testing methods and strategies. NRC presented its 

long-range vision and strategy to advance toxicity 

testing [NRC 2007]. By recognizing the importance of NRC’s vision, several federal agencies (the 

EPA, National Institutes of Environmental Health 

Sciences/National Toxicology Program, National 

Institutes of Health, and Food and Drug Administration) formed a collaborative program known as 

Toxicity Testing in the 21st Century (Tox 21). This 

program uses high-throughput screening methods and computational toxicology approaches to 

screen, rank, and prioritize chemicals for further 

testing and assessment. The Tox 21 program has 

screened more than 10,000 chemicals by using 

approximately 70 in vitro cell-based assays with 

15-point dose-response at the NIH Chemical Genomics Center, with innovative robotic technology 

[Kavlock et al. 2009]. In addition to Tox 21, the 

EPA’s Toxicity Forecaster, simply known as ToxCast, has generated data for over 1,800 subsets of 

chemicals from Tox21 inventory by expanding into 

more biological endpoints. ToxCast screens chemicals for dose-related changes in at least six doses in 

over 700 high-throughput assays (both cell-based 

and cell-free) and 300 signaling pathways that 

cover a wide range of cell responses [Richard et al. 

2016]. The EPA describes in “Next Generation Risk 

Assessment: Recent Advances in Molecular, Computational, and Systems Biology” how new molecular, computational, and systems biology data and 

approaches could better inform risk assessment 

[EPA 2014].

Overall, the screening data generated by these programs are used to predict the toxicity of chemicals 

and to prioritize the chemicals that need further 

comprehensive toxicity evaluation. In addition, 

the results from high-throughput analysis could be 

used in adverse outcome pathway (AOP) analysis, 

although the specifics are still being worked out 

[Tollefsen et al. 2014]. 

Thomas et al. [2011; 2013] demonstrated a high 

degree of correlation between the BMD values for 

transcriptional changes and the corresponding apical endpoint changes in male Sprague-Dawley and 

F344 rats and in female B6C3F1 mice exposed to 

various chemicals. The authors went on to suggest that the transcriptional points of departure (PoD) values could be used as potential surrogates for both cancer and non-cancer points of 

departure. Kuppusamy et al. [2015] and Alyea et 

al. [2012] demonstrated concordance between the 
172 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

changes in epigenetics and apical endpoints. In addition, Schulte et al. [2015] discussed the utilization of genetic and epigenetic data in occupational 

health risk assessment.

Applications of the molecular toxicology approach 

could include screening out problematic chemicals, 

identifying critical in vivo testing, prioritizing datapoor chemicals, and replacing traditional testing 

with more efficient alternatives. Although the current effort to use molecular toxicology data looks 

promising, additional data and methods of analysis are needed. For these efforts to continue, strong 

collaboration between agencies is needed as well. 

C.6 Quantitative StructureActivity Relationships 

The literature on Quantitative Structure-Activity 

Relationship (QSAR) models is vast (for a review, 

see Roy et al. [2015]), and the models have been 

widely applied in pharmaceutical research and 

risk assessment. In order to predict a response in 

untested chemicals, QSAR models link chemical, 

physical, and structural properties to a biological outcome by means of a mathematical model. 

Currently, no human health risk assessment has 

been based solely on a QSAR analysis; however, 

as toxicity testing moves away from animal testing [NRC 2007], such approaches may become 

more common. Ideally, QSAR approaches can link 

fundamental chemical properties to adverse outcome pathways and, eventually, whole-organism 

response (e.g., cancer, death) [Ankley et al. 2010]. 

Although QSAR modeling in risk assessment is 

an emerging discipline, general guidelines for its 

use are outlined below. However, because the science is still in its infancy, an individual approach 

should be taken in tailoring the guidelines to a 

given situation.

The first issue in using QSAR modeling in a risk 

assessment framework is the adverse outcome predicted and its relevance to human health. To date, 

most QSAR models focus on prediction of single 

outcomes such as the median lethal dose (i.e., LD50) 

from chemical structural properties. Such outcomes 

are often a gross measure of toxicity and say little about low levels of exposure. Others, which 

compute the lowest observed adverse effect level 

(LOAEL), or an equivalent endpoint [Mumtaz et al. 

1995], may be directly applicable to the risk assessment but require strong assumptions that should 

be carefully reviewed. QSAR modeling is under 

development to predict the entire dose-response 

curve, which would provide additional information on toxicity. In sum, care should be taken when 

choosing the endpoint for a risk assessment. If the 

endpoint is a gross measure of toxicity, then it may 

be useful to classify a chemical on the basis of its 

relative toxicity but unreasonable to provide an exposure level in the nature of an occupational exposure limit (OEL). Predicting the entire dose-response curve may have additional applications for 

quantitative risk assessment. Any QSAR-based risk 

assessment should start with exploring the limitations of the model and the predicted endpoint a 

priori, and subsequent assessment should carefully 

consider these limitations. 

Once the endpoint/model is chosen, it is important 

to assess the validity of the QSAR model. This is 

usually done in a statistical analysis of the prediction in terms of a “leave one out” (or “leave many 

out”) hold-out analysis. Here, the model is fit to a 

reduced dataset and the held-out data are predicted. 

Such analyses provide a useful tool to measure the 

accuracy of the model within the context of the entire dataset tested. Note that chemicals beyond the 

scope of the dataset will be less likely to behave as 

predicted. The model should have a high degree of 

accuracy in prediction for the chemicals of interest, 

where accuracy is defined relative to the analysis at 

hand. Further, the model should be validated and 

a sensitivity analysis (including various plausible 

assumptions and defaults for the model structure) 

should be performed. Finally, because the estimates are based upon limited or no data, the 

preliminary nature of the assessment should be 

stressed. If new data are made available that suggest 

the chemical is more or less toxic, the risk assessment should be updated with the new data within a 

reasonable timeframe.
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 173

C.6 Nanomaterials Risk 

Assessment

C.6.1 Overview

Given the large and growing number of engineered 

nanomaterials (ENMs) with limited data, as for 

other emerging and existing substances produced 

or used in the workplace, alternative test strategies 

(i.e., toxicological approaches other than primary 

animal testing) such as high-throughput screening 

and in vitro exposures may help to fill the gaps by 

providing data that could be used in validated hazard and risk assessment models [Drew et al. 2017; 

Kuempel et al. 2012]. 

C.6.2 Dose Normalization In Vitro

and In Vivo

As risk assessments begin to rely largely on in vitro

data and in silico modeling, accurate description of 

dose both in vitro and in vivo will be key to evaluating these dose-response relationships and validating alternative test strategies for use in risk assessment [Gangwal et al. 2011; Oberdörster 2012]. 

Many in vitro studies have used doses that are much 

higher than occupationally equivalent lung doses 

[Gangwal et al. 2011]. Such studies could be useful 

for hazard identification and screening evaluations 

but may overpredict the in vivo response. In vitro

studies are also limited in the cell types represented 

and interactions among cells. 

A challenge in quantifying the dose-response relationships in vitro is estimating the effective dose, 

i.e., the dose that reaches the target cells. The particle surface area doses to cells can differ significantly 

at a given mass concentration (μg/ml), because of 

the differences in the specific surface area (m2/g) 

of particles of different sizes and differences in the 

sedimentation and diffusion properties of particles 

in liquid-based systems [Hinderliter et al. 2010]. 

The In Vitro Sedimentation, Diffusion, and Dosimetry (ISDD) model was developed to account for 

differences in settling velocity in the liquid media 

based on particle size, density, and specific surface 

area [Hinderliter et al. 2010]. Adjusting the in vitro

dose to estimate the total surface area of nanoparticles that reach cells in the petri dish was shown 

to better correlate with acute in vivo endpoints 

[Hinderliter et al. 2010; Teeguarden et al. 2007]. 

C.6.3 Correlation of In Vitro and

In Vivo Responses 

Several studies of ENMs have shown good correlation between the in vitro and acute in vivo

inflammation-related responses to poorly soluble 

particles [Donaldson et al. 2008; Rushton et al. 

2010; Zhang et al. 2012]. The dose metric in these 

studies differed, including comparison of either the 

total particle surface area to the total cell surface 

area in vitro or in vivo (cm2/cm2) [Donaldson et al. 

2008], the response per unit particle surface area 

[Rushton et al. 2010], or the area under the doseresponse curve [Zhang et al. 2012]. The steepest 

portion of the dose-response slope showed the best 

correlation of in vitro with in vivo responses [Han 

et al. 2012; Rushton et al. 2010]. Responses included cell-free generation of reactive oxygen species 

(ROS), rat lung epithelial cell release of lactate dehydrogenase (LDH) or induction of protein oxidation endpoints, and rat pulmonary inflammation 

measured as polymorphonuclear (PMN) leukocyte 

response in bronchoalveolar lavage fluid (BALF) 

after intratracheal instillation (IT) exposure to different ENMs. 

Pulmonary fibrosis in vivo (in rodents) and fibrosis-related markers in vitro (in rodent or human 

lung cells) have been shown to be correlated with 

exposure to some ENMs. Specifically, activation of 

the NLRP3 inflammasome and pro-fibrogenic endpoints in vitro or fibrosis in vivo have been associated with exposure to carbon nanotubes [Hamilton 

et al. 2009; Li et al. 2013; Sager et al. 2014; Wang et 

al. 2012; Wang et al. 2011]. With further validation, 

an in vitro inflammasome activation assay may be 

useful for assessing the potential for chronic adverse effects of carbon nanotubes and other ENMs. 
174 NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment

C.7 Alternative Methods for 

Nanomaterials 

C.7.1 Comparative Potency 

Estimation

One promising use of alternative test strategies data 

is comparative potency analyses between nanomaterials and benchmark materials for use in the development of OEBs [Kuempel et al. 2012]. Benchmark materials can serve as points of reference 

for comparison to ENMs. Benchmark materials 

are well-characterized substances within biological MoA categories for which the health hazards 

are well known and quantitative risk estimates 

have been (or could be) developed [Kuempel et al. 

2012; Nel et al. 2013]. Possible benchmark materials to evaluate inhalation hazards may include fine 

crystalline silica, asbestos, and ultrafine titanium 

dioxide and/or carbon black [Oberdörster et al. 

2005a]. These comparative toxicity analyses would 

be conducted in vitro for a set of ENMs, along with 

benchmark particles (including positive and negative controls or references), to which the new materials could be compared. The in vitro to in vivo

dose-response relationships would be validated 

for the benchmark materials in specific assays. A 

parallelogram approach [Schoeny and Margosches 

1989; Sobels 1977; Sutter 1995] has been used for 

comparative toxicity and risk estimation and has 

also been proposed for use in setting provisional 

OELs of pharmaceutical intermediates, including 

using in vitro data [Maier 2011]. The use of in 

vitro dose-response data to estimate a PoD directly 

has been proposed; it would involve using methods similar to those used for in vivo data, including adjustment of the PoD by uncertainty factors 

(initially, until more evidence is available) [Crump 

et al. 2010]. Such comparative approaches could be 

used in deriving initial OELs or OEBs for individual ENMs or groups of ENMs [Kuempel et al. 2012].

Given the limited data for developing OELs for 

ENMs, methods have been developed to prioritize 

or group ENMs based on the available subchronic or 

chronic dose-response data for benchmark materials 

and the utilization of shorter-term in vivo data for 

many ENMs [Arts et al. 2015; Drew et al. 2017; Hristozov et al. 2016]. Several QSAR models have been 

developed, which describe the important factors 

influencing the toxicity and allow for hazard grouping and ranking [Burello and Worth 2011; Gernand 

and Casman 2014; Hristozov et al. 2016; Oh et al. 

2016]; however, these models have not been used in 

human health risk assessment. Still lacking in these 

frameworks is an integrated methodology to utilize 

quantitative dose-response data to group EMNs by 

hazard potency, using biological responses and dose 

metrics that allow for the estimation of humanequivalent concentration and development of categorical OELs [Schulte et al. 2018]. More comprehensive data across a range of ENMs and experimental 

designs are needed to develop predictive models using alternative data to the rodent bioassays typically 

used in risk assessment.

C.7.2 Hazard Classification/

Clustering 

NIOSH and others are exploring methods to utilize 

physicochemical properties—such as particle size, 

shape, solubility, crystal structure, and chemical 

composition—as predictors of a material’s hazard 

potency, such as tested in high-throughput cellular studies and validated in limited rodent studies. 

Potency is the inverse of dose (i.e., higher potency 

substances are those with a lower dose associated 

with an adverse effect). In these ongoing analyses, 

NIOSH is investigating the dose-response relationships and substance-specific physicochemical data, 

and it is using statistical learning methods such 

as Random Forests to identify groups of similarly 

behaving materials with respect to hazard potency 

[Drew et al. 2017]. The adverse outcomes of interest 

include pulmonary inflammation, fibrosis, cancer, 

and systemic effects associated with inhaled nanoscale particles. In current analyses of acute pulmonary inflammation, a set of 16 microscale and 

nanoscale particles in a training dataset have been 

grouped into four potency clusters, including three 

groups for nanoscale particles, which are 4–175 

times more potent than a fourth group containing 
NIOSH CIB 69 • NIOSH Practices in Occupational Risk Assessment 175

a microscale reference particle. These analyses illustrate proof of concept for grouping particles by 

pulmonary hazard potency [Drew et al. 2017]. 

Next steps are to evaluate an in vitro dataset of some 

of the same materials as in the in vivo dataset to investigate the possible utility of in vitro studies of cellular responses to particle exposure that are involved 

in the in vivo mechanism of activation of pulmonary 

inflammation associated with particle exposure, including cytokine, gene transcription, and cell toxicity 

endpoints. Ultimately, it is envisioned that extended 

and validated analyses will be used as a framework 

to develop initial OEL categories or OEBs as hazard inputs into nanomaterial control banding tools 

[Drew et al. 2017; Kuempel et al. 2012]. 

C.7.3 Validation

A key challenge to utilizing alternative test strategies data is the development and application of 

validation criteria. Validation would include evaluation of variability across laboratories and selected 

assays of reference particles. Such evaluations for 

ENMs have shown considerable interlaboratory 

variability in dose-response relationships for the 

same ENMs across laboratories [Bonner et al. 2013], 

especially in the in vitro assays [Xia et al. 2013]. 

To facilitate the validation and implementation 

of alternative test strategies data, standard sets of 

particle descriptors, dose metrics, and response 

parameters are needed to compare biological MoA 

and dose–response relationships across different 

studies [Kuempel et al. 2012]. In vitro data could be 

used in a tiered toxicology testing such that selected 

materials (e.g., highest and lowest toxicity within a 

category) in the in vitro assays would go on for in 

vivo testing.

C.8 Non-Chemical and 

Cumulative Risk Assessment

With the exclusion of ionizing radiation, quantitative risk assessment of non-chemical stressors has 

received little attention in the risk assessment community. Moreover, risk assessment methods have 

largely focused on a single stressor, although risks 

often involve complex exposures to multiple stressors from multiple routes and pathways. Recently, 

there has been interest in assessing risks from 

non-chemical stressors separately and in combination with chemical exposures. For example, the 

National Research Council has recommended that 

risk assessors consider exposures to both chemical 

and non-chemical stressors as sources of cumulative risk [NRC 2009]. Such stressors can include 

physical, operational, and psychosocial domains. 

Examples include work stress, heat stress, noise exposures, and vibrational exposures. 

Research into risk assessment methods for nonchemical stressors and cumulative risks is ongoing 

[Lentz et al. 2015; Lewis et al. 2011]. For example, 

NIOSH has developed methods similar to chemical 

risk assessment to assess some non-chemical hazards, such as ionizing radiation and noise [NIOSH 

1987; NIOSH 1998]. Expanding these methods to 

include other non-chemical hazards, the joint effects of multiple stressors, and the contribution of 

non-occupational stressors to occupational risk are 

areas of interest in NIOSH risk assessment.

Concerning cumulative risk, the literature has 

focused on the potential for interactions among 

combined chemicals producing synergistic effects 

[Carpenter et al. 2002; Fox et al. 2017; Hertzberg 

and Teuschler 2002; Sexton 2012; Sexton and Hattis 2007]. The EPA has developed a series of reports 

describing its framework for assessing chemical 

mixtures and conducting cumulative risk assessment [EPA 1986; EPA 2000; EPA 2003; EPA 2007]. 

These reports provide detailed guidance on cumulative risk that can be adapted to occupational 

settings. Other frameworks are available [Meek 

et al. 2011; Moretto et al. 2017]. More recently, 

an informative report on risk assessments involving combined exposures to multiple chemicals 

has been published by the OECD [OECD 2018a]. 

The report provides an overview of the technical 

aspects, limitations, and uncertainties associated 

with various approaches and methodologies available to assess health risks from combined exposures to multiple chemicals.
Promoting productive workplaces through 

safety and health research

DHHS (NIOSH) Publication No. 2020-106

DOI: https://doi.org/10.26616/NIOSHPUB2020106