United States Government Accountability Office

GAO Report to the Chairman, Committee on 

Environment and Public Works, U.S. 

Senate 

HUMAN HEALTH 

RISK ASSESSMENT 

EPA Has Taken Steps 

to Strengthen Its 

Process, but 

Improvements Needed 

in Planning, Data 

Development, and 

Training 

May 2006 

 

GAO-06-595 
What GAO Found

United States Government Accountability Office

Why GAO Did This Study

Highlights

Accountability Integrity Reliability

May 2006

HUMAN HEALTH RISK ASSESSMENT

EPA Has Taken Steps to Strengthen Its 

Process, but Improvements Needed in 

Planning, Data Development, and 

Training 

Highlights of GAO-06-595, a report to the 

Chairman, Committee on Environment 

and Public Works, U.S. Senate 

Over 100,000 chemicals, pollutants, 

and toxic substances are used in

the United States and regulated by

the Environmental Protection 

Agency (EPA). EPA uses risk 

assessment to determine the health

risk from exposure to these 

substances, collectively referred to

as contaminants. In the last 12

years, independent reviewers have

examined this process and made

recommendations for how it could

be improved. GAO was asked to

(1) identify the significant 

recommendations that have been

made to improve human health risk 

assessment; (2) describe what EPA 

has done to modify its human 

health risk assessment process; (3) 

determine the effects these past 

modifications have had on the 

preparation of risk assessments; 

and (4) identify any additional 

actions experts believe EPA could 

take to improve its process, and the 

barriers it would face in doing so. 

What GAO Recommends

GAO recommends that EPA 

enhance early planning of each risk

assessment, identify and 

communicate data needs to the 

public and private research

community, and support 

development and implementation

of in-depth training for risk

assessors and managers. EPA 

neither agreed nor disagreed with

our findings and recommendations.

However, the agency provided 

specific technical comments, which

we incorporated as appropriate. 

Since 1994, independent reviewers recommended that EPA better plan its 

risk assessments. In doing so, they said EPA should better utilize scientific 

data it has and identify other data it needs on the potential adverse effects 

from exposure to contaminants, and prioritize and support research to meet 

those needs. Furthermore, reviewers recommended that EPA better evaluate 

the analytic tools it uses and employ more powerful tools when appropriate. 

Reviewers also recommended that EPA better analyze and characterize the 

sources of uncertainty in its risk assessments. Finally, they recommended 

that EPA enhance its analysis of variability in exposure to contaminants and 

in susceptibility to harm from exposure, and improve how it considers the 

effects of exposure to multiple contaminants and through many sources. 

EPA has strengthened its risk assessment process since 1994 and 

improvement efforts are ongoing. For example, EPA has increased planning 

for assessments and has initiated actions to develop missing or incomplete 

scientific data. EPA has also begun to embrace new methodologies, such as 

ones to predict how the body will react to a contaminant. Furthermore, EPA 

now uses a tiered approach to conducting uncertainty analysis, employing 

more sophisticated analysis as warranted. Finally, EPA has made progress in 

characterizing variability due to differences in both exposure and 

susceptibility of exposed individuals and has begun to take steps to address 

exposure to multiple contaminants and through multiple sources. 

According to EPA’s risk assessors, the modifications EPA has made have 

generally helped improve risk assessments. Many EPA risk assessors believe 

that agencywide guidance has helped them prepare risk assessments and 

have resulted in greater consistency across program offices. Furthermore, 

while most assessors report collaboration with internal and external entities 

is effective and has improved the quality of risk assessments, some said 

conflicting priorities and poor communication hindered collaboration among 

some EPA offices. Finally, while risk assessors said training has helped 

them gain skills and knowledge, over 70 percent said that more in-depth or 

relevant training would improve their risk assessment abilities. 

Experts identified additional actions EPA could take to further improve its 

risk assessment process, recognizing that it may face barriers in doing so. 

Experts said EPA could improve its planning process by better focusing on 

scientific data needs and involving stakeholders early to obtain their 

concurrence with EPA’s approach. Experts also said EPA could more 

thoroughly evaluate methods and models, transparently document its 

analytic choices, and enhance internal review. Finally, experts said EPA 

could provide additional training for risk assessors, managers, and 

stakeholders on the risk assessment process. Experts, however, said that 

the scientific complexity of risk assessment, the difficulty of obtaining and 

applying data, and a cultural resistance to deviating from established 

methods could act as obstacles to successfully making such changes. 

www.gao.gov/cgi-bin/getrpt?GAO-06-595.

To view the full product, including the scope 

and methodology, click on the link above. To 

view the results of GAO’s survey of EPA’s

risk assessors, click www.gao.gov/cgibin/getrpt?GAO-06-637SP. For more 

information, contact John Stephenson at 

(202) 512-6225 or stephensonj@gao.gov. 
Contents 

Letter 1

Results in Brief 5 

Background 8

Independent Reviewers Have Recommended Improvements to 

EPA’s Risk Assessment Process Since 1994 12 

EPA Has Strengthened Many Facets of Its Risk Assessment 

Process Since 1994, and Efforts Are Ongoing 24 

EPA Risk Assessors Responding to Our Survey Reported That 

Process Modifications Have Helped Them Prepare Better Risk 

Assessments but That Collaboration and Training Limitations 

Hamper Further Progress 51 

Enhanced Risk Assessment Planning, Improved Analysis and 

Review, and Added Training Could Further Improve EPA’s 

Process, but Barriers Could Limit Progress 62 

Conclusions 67

Recommendations for Executive Action 69 

Agency Comments 69 

Appendix I Objectives, Scope, and Methods 71 

Appendix II GAO Contact and Staff Acknowledgments 75 

Figures 

Figure 1: Typical Sequence of Risk Assessment and Risk 

Management Processes 9 

Figure 2: Helpfulness of Office-Specific Guidance 53 

Figure 3: Risk Assessors That Generally or Strongly Agree That 

Organizations Help Fill Data Needs 55 

Figure 4: Areas in Which Training Has Moderately or Greatly 

Improved Risk Assessor Knowledge and Skills 58 

Figure 5: Types of Training Taken by Risk Assessors 59 

Figure 6: Modifications in Training That Would Moderately or 

Greatly Improve the Preparation of Risk Assessments 61 

Page i GAO-06-595 Human Health Risk Assessment
Abbreviations 

ASPH Association of Schools of Public Health 

ATSDR Agency for Toxic Substances and Disease Registry 

BOSC Board of Scientific Counselors 

Commission Presidential/Congressional Commission on Risk 

Assessment and Risk Management 

CREM Council for Regulatory Environmental Modeling 

EPA Environmental Protection Agency 

FQPA Food Quality Protection Act 

GIS Geographic Information System 

HEI Health Effects Institute 

HPV High Production Volume 

ILSI-RSI International Life Sciences Institute—Risk Science 

Institute 

IRIS Integrated Risk Information System 

NAS National Academy of Sciences 

NCCT National Center for Computational Toxicology 

NCEA National Center for Environmental Assessment 

NCER National Center for Environmental Research 

NERL National Exposure Research Laboratory 

NHEERL National Health and Environmental Effects Research 

 Laboratory 

NHEXAS National Human Exposure Assessment Survey 

NIEHS National Institute for Environmental Health Sciences 

NRMRL National Risk Management Research Laboratory 

OAQPS Office of Air Quality Planning and Standards 

OMB Office of Management and Budget 

OP Office of Pesticides 

OPPT Office of Pollution Prevention and Toxics 

ORD Office of Research and Development 

OW Office of Water 

STAR Science to Achieve Results 

TRIM Total Risk Integrated Methodology 

TSCA Toxic Substances Control Act 

Page ii GAO-06-595 Human Health Risk Assessment
This is a work of the U.S. government and is not subject to copyright protection in the 

United States. It may be reproduced and distributed in its entirety without further 

permission from GAO. However, because this work may contain copyrighted images or 

other material, permission from the copyright holder may be necessary if you wish to 

reproduce this material separately. 

Page iii GAO-06-595 Human Health Risk Assessment
United States Government Accountability Office

Washington, DC 20548 

May 31, 2006

The Honorable James M. Inhofe 

Chairman 

Committee on Environment and Public Works 

United States Senate 

Dear Mr. Chairman: 

Chemicals play an important role in people’s everyday lives. Consumers 

use products containing chemicals, such as cleansers and pesticides, and 

companies use a variety of toxic substances as solvents or additives to 

their industrial processes. Although these substances have beneficial uses 

and are important in producing goods and services, some may adversely 

affect human health. Over 100,000 chemicals, pollutants, and toxic 

substances are used in the United States and regulated by the 

Environmental Protection Agency (EPA). These substances, whether 

produced in the United States or imported, include 187 hazardous air 

pollutants, nearly 20,000 pesticide products on the market, and a growing 

number of substances used in industrial processes—over 82,000 as of 

December 2005. EPA uses a process known as risk assessment to estimate 

the health risk from exposure to these substances, collectively referred to

in this report as contaminants.1

 While highly technical by nature, risk 

assessments, along with other relevant information, serve as a basis for 

regulatory decisions that protect human health. EPA’s risk assessments 

are shaped by available scientific information and by provisions in major 

environmental statutes, such as the Clean Air Act, and they affect a range 

of stakeholders, including regulated industries, federal agencies, 

environmental advocacy groups, academic and other researchers, and the 

public. 2

Chemicals play an important role in people’s everyday lives. Consumers 

use products containing chemicals, such as cleansers and pesticides, and 

companies use a variety of toxic substances as solvents or additives to 

their industrial processes. Although these substances have beneficial uses 

and are important in producing goods and services, some may adversely 

affect human health. Over 100,000 chemicals, pollutants, and toxic 

substances are used in the United States and regulated by the 

Environmental Protection Agency (EPA). These substances, whether 

produced in the United States or imported, include 187 hazardous air 

pollutants, nearly 20,000 pesticide products on the market, and a growing 

number of substances used in industrial processes—over 82,000 as of 

December 2005. EPA uses a process known as risk assessment to estimate 

the health risk from exposure to these substances, collectively referred to

in this report as contaminants.1

 While highly technical by nature, risk 

assessments, along with other relevant information, serve as a basis for 

regulatory decisions that protect human health. EPA’s risk assessments 

are shaped by available scientific information and by provisions in major 

environmental statutes, such as the Clean Air Act, and they affect a range 

of stakeholders, including regulated industries, federal agencies, 

environmental advocacy groups, academic and other researchers, and the 

public. 2

 1

Although other regulatory agencies, such as the Food and Drug Administration, also use 

risk assessments as part of regulatory decision making, this report discusses only the risk

assessment process used by EPA. 

2

EPA’s mission is to protect human health and to safeguard the natural environment—air, 

water, and land—upon which life depends. This report focuses only on EPA’s efforts to

protect human health. 

Page 1 GAO-06-595 Human Health Risk Assessment
In 1994, as part of a congressionally mandated review of the methods used 

by EPA to estimate the risk of developing cancer from exposure to 

hazardous air pollutants, the National Academy of Sciences (NAS) issued a 

report entitled Science and Judgment in Risk Assessment. This report 

focused primarily on hazardous air pollutants and provided an overview 

assessment of EPA’s risk assessment methods.3

 In providing this overview, 

the NAS committee identified several themes, largely focusing on the 

quality and availability of essential information, that serve as a useful 

framework for discussing the risk assessment process: 

• Implementation. EPA faces certain overarching, institutional issues that 

affect its implementation of the risk assessment process. 

• Data needs. Data necessary to complete scientifically plausible risk 

assessments are often unavailable on such topics as the actual levels of 

exposure to contaminants and how those exposures affect human health.

• Default options. In the absence of convincing scientific knowledge or 

data, EPA relies on assumptions, often conservative in nature, about such 

questions as how exposure to low doses of a contaminant affects human 

health. 

• Method and model evaluation. The predictive accuracy of methods and 

models, such as those used to predict how a contaminant will be 

processed once it enters the body, is not always known. 

• Uncertainty. The lack of precise knowledge about the type, likelihood, 

and extent of adverse effects from exposure to a contaminant results in 

uncertainty in risk assessment that can be reduced only by advances in 

scientific understanding or the collection of better data. 

• Variability. Variability, in exposure or in the biological differences among 

humans that determine how exposure to contaminants affects health, can 

be better characterized with more data but cannot be reduced or 

eliminated. 

 3

Pollutants are generally categorized as hazardous air pollutants under the Clean Air Act if 

they cause or may cause cancer or other serious health effects, such as reproductive 

effects or birth defects, or adverse environmental and ecological effects. Currently, the

Clean Air Act regulates 187 chemicals and chemical categories as hazardous air pollutants. 

Page 2 GAO-06-595 Human Health Risk Assessment
• Aggregate and cumulative effects. Aggregate effects are the results from 

exposure to a single contaminant by any combination of means—including 

inhalation, contact with the skin, and ingestion; cumulative effects are the 

results from exposure to multiple contaminants by any combination of 

means. 

Science and Judgment in Risk Assessment was one of several reports by

NAS that discussed risk assessment in general and made 

recommendations for ways to improve it.4

 In addition, the 

Presidential/Congressional Commission on Risk Assessment and Risk 

Management issued a report in 1997 that reviewed risk assessment, as well 

as risk management, in regulatory programs.5 Furthermore, the Office of 

Management and Budget (OMB) issued guidance related to peer review 

and ensuring the quality, objectivity, utility, and integrity of information 

released to the public.6 More recently, in January 2006, OMB issued a 

proposed bulletin that advocates minimum standards for the scientific 

quality of risk assessments to enhance their technical quality and 

objectivity. 

In the context of these prior reviews, you asked us to (1) identify the 

significant recommendations to improve human health risk assessment 

that have been made since 1994; (2) describe what EPA has done to 

modify its human health risk assessment process over the same period; (3) 

determine the effects these past modifications have had on the preparation 

of risk assessments; and (4) identify any additional actions experts believe 

EPA could take to improve its risk assessment process in the future, and 

the barriers EPA would face in doing so. 

To identify significant recommendations to improve human health risk 

assessment since 1994, we reviewed EPA documents, including those 

produced by EPA’s Risk Assessment Forum, Science Policy Council, 

 4

National Academy of Sciences, Risk Assessment in the Federal Government: Managing 

the Process (Washington, D.C., 1983).

5

The Presidential/Congressional Commission was created by Pub. L. No. 101-549 (1990). 

6

Peer review is a documented critical review of a specific scientific or technical work 

product, conducted by qualified individuals who are independent of those who performed

the work, but who are collectively equivalent in technical expertise—for example, EPA’s 

Science Advisory Board or the National Academy of Sciences. According to EPA’s 2006

Peer Review Policy, peer review can be internal, in which the reviewers are independent 

experts from inside EPA, or external, in which the reviewers are independent experts from

outside EPA. 

Page 3 GAO-06-595 Human Health Risk Assessment
Council for Regulatory Environmental Modeling, and Science Advisory 

Board, as well as each of EPA’s program offices. We also reviewed our 

own reports and documents produced by NAS and the Presidential/ 

Congressional Commission on Risk Assessment and Risk Management. We 

also spoke with experts in the risk assessment field, who identified many 

of these documents in the course of our discussions and provided insight 

into some of the recommendations made therein. To describe what EPA 

has done to modify its human health risk assessment process, we 

interviewed program office managers from EPA’s Office of Air and 

Radiation, Office of Pesticide Programs, Office of Pollution Prevention and 

Toxics, Office of Water, and Office of Research and Development (ORD). 

While we recognize that EPA’s regional offices and the Office of Solid 

Waste and Emergency Response conduct risk assessments, for the most 

part their work focuses on site-specific assessments and thus was not 

included in this effort. Within ORD, we interviewed managers in two of 

EPA’s laboratories (the National Health and Environmental Effects 

Laboratory and the National Exposure Research Laboratory) and three of 

EPA’s research centers (the National Center for Environmental 

Assessment, National Center for Environmental Research, and National 

Center for Computational Toxicology). Furthermore, we attended various

EPA and stakeholder group training sessions and meetings. 

To determine the effects these modifications have had on the preparation 

of risk assessments, we surveyed risk assessors from the program offices 

mentioned above. We sought their views on the usefulness of many of 

these modifications, as well as on aspects of EPA’s risk assessment 

process, including guidance documents, training, organizational structure, 

and collaboration. The survey and results can be viewed at 

http://www.gao.gov/cgi-bin/getrpt?GAO-06-637SP. In addition, we 

interviewed experts in the risk assessment field and obtained their 

perspectives on the extent to which these modifications were helpful to 

EPA’s risk assessment process. To identify additional actions experts 

believe EPA could take to improve its process and to identify barriers to 

such actions, we contacted experts representing a range of stakeholders in

the process. Specifically, we contacted risk assessment scientists; 

toxicologists; scientific advisers to EPA; state officials; and 

representatives from regulated industries, government agencies, and 

environmental advocacy groups who have an expertise in risk assessment. 

We used an iterative process (often referred to as the “snowball sampling” 

technique) to identify these knowledgeable experts and selected for 

interviews those who would provide us with a broad and balanced range 

of perspectives on EPA risk assessment practices. We used a standard set 

of questions to interview each of these experts to ensure we consistently 

Page 4 GAO-06-595 Human Health Risk Assessment
discussed each aspect of EPA risk assessment policies and practices. (See

app. I for a more detailed description of the methodology we employed.)

In addition, we consulted with NAS’s Board on Environmental Studies and 

Toxicology at its June 2005 meeting to obtain input on the areas in which 

EPA has made the most progress in improving its risk assessment 

practices and areas EPA will need to focus on in the future. We used the 

experts present at this meeting as a starting point for our snowball 

sampling technique used to identify subsequent experts. We conducted 

our work from February 2005 through March 2006 in accordance with 

generally accepted government auditing standards. 

Independent reviewers, including NAS, have made a number of significant 

recommendations to improve EPA’s human health risk assessment 

process since 1994 that can be grouped and discussed in terms of the 

seven themes originally identified by NAS in 1994. First, reviewers have 

said that EPA should improve its overall implementation of the process 

with such steps as more thorough planning of its risk assessments and 

greater use of independent reviews by scientists (called peer reviews). 

Second, to improve the quantity and quality of the data upon which risk 

assessments are based, reviewers have recommended that EPA better 

utilize the scientific data it has and identify additional data needs on the 

potential adverse effects from exposure to contaminants, and prioritize 

and support research to meet the data needs identified. Third, because 

EPA lacks data to fully assess the health risk from exposures, reviewers 

have recommended that when the agency relies on assumptions—known 

as default options—it should, among other things, more clearly indicate 

when it relies on default options and how it chooses them. Fourth, to 

improve the accuracy of EPA’s risk assessments, reviewers have stated

that EPA should better evaluate the methods and models used in its 

analysis and incorporate newer, more powerful tools when appropriate. 

Fifth, to address the inevitable uncertainties associated with gaps in 

scientific knowledge and general unknowns about model and data 

accuracy, EPA should, according to reviewers, more explicitly analyze and 

characterize the sources of uncertainty in its risk assessments and, when 

possible, discuss the uncertainties both descriptively (qualitatively) and

numerically (quantitatively). Sixth, reviewers have also recommended that

EPA enhance its analysis of variability in levels of exposures to 

contaminants, as well as differences in individual reactions to exposure. 

Finally, because people are typically exposed to a mixture of contaminants

through a variety of means, such as contact with skin and breathing air, 

reviewers have recommended that EPA improve how it considers the 

Results in Brief 

Page 5 GAO-06-595 Human Health Risk Assessment
effects of combinations of contaminants and all possible means of 

exposure. 

EPA has strengthened many aspects of its risk assessment process since 

1994, and improvement efforts are ongoing. First, EPA has improved 

implementation of its risk assessment process by, for example, beginning

to improve risk assessment planning and creating scientific leadership 

positions, such as the Office of the Science Advisor. Second, EPA has 

initiated actions to develop missing or incomplete scientific data on the 

potential adverse effects from exposure to contaminants. For example, 

EPA officials told us they have begun to use the planning and review 

processes to determine what data are needed and communicate these 

needs to both EPA and outside researchers. Third, EPA has cited the need 

for risk assessments to be more transparent about their use of default 

options. For example, EPA has recently issued guidance describing default 

options that are appropriate for certain purposes and directing risk 

assessments to disclose the default options used in a particular risk 

assessment. Fourth, EPA is enhancing its use of models and embracing 

new methodologies. For example, EPA has established a group—the 

Council for Environmental Regulatory Modeling—to review models and 

provide guidance in model selection. Furthermore, some program offices 

now use more advanced models to predict how a contaminant will be 

processed once it enters the body. Fifth, EPA generally characterizes 

uncertainty descriptively, but has begun to incorporate quantitative 

techniques into its uncertainty analyses. For example, EPA typically uses a 

tiered approach to conducting uncertainty analysis, starting as simply as 

possible, describing uncertainty qualitatively, and sequentially employing 

more sophisticated analysis, such as probabilistic analysis, as warranted. 

Sixth, EPA has made progress in describing variability due to differences 

in both the exposure and the susceptibility of exposed individuals to 

contaminants. For example, when determining a level of exposure that is 

unlikely to be harmful, EPA includes adverse effects for people who might 

be at increased risk because of their age or the state of their health. 

Finally, EPA has begun to take steps to consider the combined effects of 

exposure to multiple contaminants through multiple means of exposure. 

For example, EPA has developed a framework to assess the combined, or 

cumulative, risk and has directed its program offices to include cumulative

risks when planning major risk assessments. 

EPA risk assessors believe the modifications EPA has made over the past 

10 years—particularly issuing additional guidance—have generally helped 

them improve risk assessments, but collaboration and training could be

improved. Most EPA risk assessors believe that agencywide guidelines and 

Page 6 GAO-06-595 Human Health Risk Assessment
policy and reference documents have helped them prepare risk 

assessments by, for example, providing useful frameworks for evaluating

potential harm from chemicals, and have resulted in greater consistency 

among risk assessments prepared by different offices within EPA. 

Furthermore, EPA’s collaboration with external researchers, including 

other federal research entities, academia, and industry, has improved 

EPA’s ability to conduct risk assessments by providing expertise and 

research not always available within the agency, according to risk 

assessors. In addition, risk assessors reported that internal collaboration 

among EPA offices is moderately to very effective, but could be improved. 

For example, a few risk assessors commented that the program offices’ 

knowledge of the regulatory context in which research will be used helps

the researchers structure their work. However, risk assessors also 

reported that collaboration is hindered within EPA by, for example, 

conflicting priorities among the various offices and the poor 

communication between some of them. Finally, while risk assessors said 

improved training has also helped them gain relevant skills and 

knowledge, over 70 percent of the risk assessors responding to our survey 

stated that more in-depth or relevant training would improve their ability 

to prepare risk assessments. In addition, they believe training for risk 

assessors and managers in specific technical and scientific areas, such as 

emerging scientific issues and the use of newer models, is lacking. 

While the experts we spoke with said the modifications EPA has made 

over the past 10 years have been beneficial overall, they identified 

additional actions EPA could take to improve its risk assessment process, 

recognizing that EPA may face barriers to doing so. Specifically, experts 

said EPA could improve the planning it undertakes prior to starting a risk 

assessment by better focusing on what data are needed for the assessment 

and by involving stakeholders early in this planning process. For example, 

several experts said that increased involvement with a broad range of 

stakeholders early in the planning process would help identify alternative 

methods and models and obtain stakeholder concurrence with the 

agency’s approach. In addition, experts said EPA could more thoroughly

evaluate methods and models, transparently document its analytic 

choices, and enhance internal review. For example, several experts said 

that EPA should more transparently communicate which default 

assumptions were used in risk assessments, why the defaults were chosen, 

and what judgments EPA was making when it employed certain methods. 

Finally, experts said EPA could provide additional training for risk 

assessors, managers, and stakeholders on all elements of the risk 

assessment process, such as how to use and apply models and how to 

interpret data from emerging scientific fields. While these efforts would 

Page 7 GAO-06-595 Human Health Risk Assessment
further improve the risk assessment process, experts pointed out inherent

barriers that EPA may face in carrying them out, such as the scientific 

complexity of risk assessment, the difficulty of obtaining and applying

data, and a cultural resistance to deviating from established methods. For 

example, several experts said that EPA’s risk assessments have grown 

more technically challenging and require risk assessors and managers to 

have different skills from what they had in the past. 

To further improve the risk assessment process, GAO recommends that 

EPA enhance early planning of each risk assessment, identify and 

communicate data needs to the research community, and support 

development and implementation of in-depth training for risk assessors 

and managers. 

We provided EPA with a draft of this report for review and comment. EPA 

neither agreed nor disagreed with our findings and recommendations. 

However, the agency provided specific comments to improve the report’s 

technical accuracy, which we incorporated as appropriate. 

Risk assessment, as used in public health and environmental settings, is 

inherently a complex and highly technical process that provides a 

systematic scientific description of potential adverse effects from 

exposure to contaminants. Risk assessments are conducted to estimate 

whether and how much harm can be expected from exposure to a given 

contaminant or mixture of contaminants and to help determine whether 

the harm is significant enough to require regulatory or other corrective 

action. The adverse effects from exposure can be acute, such as 

respiratory illnesses or poisonings, or chronic, such as cancer or 

reproductive or genetic abnormalities. The result of a risk assessment is a 

statement of the likelihood that an exposed population will be harmed and 

to what extent. EPA’s human health risk assessment process generally

consists of the four stages recommended in 1983 by the National Academy 

of Sciences (NAS): (1) hazard identification (determining whether a 

substance could cause adverse affects and what those adverse affects 

would be), (2) dose-response assessment (determining the relationship 

between the magnitude of exposure to a contaminant and the probability 

and severity of adverse effects), (3) exposure assessment (identifying the 

extent to which exposure is predicted to occur), and (4) risk 

characterization (combining the information from the preceding analyses

into a conclusion about the nature and magnitude of risk). 

Background 

Page 8 GAO-06-595 Human Health Risk Assessment
Risk assessment plays a critical role in EPA’s regulatory decisions, but the 

characterization of risk that results from a risk assessment is only one 

piece of information used to inform decisions on regulatory standards for 

protecting human health. This analysis is integrated with other 

information, such as economic information on the costs and benefits of 

mitigating the risk, technological information on the feasibility of 

managing the risk, and the concerns of various stakeholders. The 

combined analysis informs agency officials who ultimately make 

regulatory decisions. This overall process is generally called risk 

management. (See fig. 1.) According to NAS, the risk assessment process 

should be conceptually distinguished from how the results of that process 

are used in the risk management process. However, in practice, 

maintaining this separation is difficult because the two processes are 

fundamentally linked by the complementary needs of the risk assessors 

and the risk managers. Specifically, risk assessors need to be aware of the

context in which the assessment will be used and to communicate their 

key findings, as well as their confidence in them, in a way that risk 

managers can understand and apply. Risk managers need to ensure that 

the risk assessment has been properly performed and can be integrated 

with other information to make and justify a sound regulatory decision. 

Figure 1: Typical Sequence of Risk Assessment and Risk Management Processes 

Hazard

identification

Risk

management

decisions

Control

options Legal

considerations

Other economic

and social factors

Exposure

assessment

Dose-response

assessment

Risk assessment Risk management

Risk

characterization

Source: EPA.

Page 9 GAO-06-595 Human Health Risk Assessment
EPA’s human health risk assessment process is limited to a great extent by

the amount of scientific data available on contaminants and by the level of

scientific understanding about how contaminants act in the body to 

produce adverse effects. Data are often unavailable, and science cannot 

always provide definitive answers. The complex and sophisticated analytic

tools used in risk assessment cannot overcome basic scientific unknowns. 

As a result, EPA’s risk assessments almost always include assumptions 

about potential adverse effects. Some assumptions are conservative—that 

is, they are intended to help ensure that the agency does not 

underestimate health risks. Some critics of EPA’s risk assessment 

practices believe EPA uses assumptions that are unjustifiably

conservative, given new scientific data and methods, and thereby 

produces estimates that overstate actual risks. Other stakeholders, 

however, criticize some agency assumptions for not being conservative 

enough in the face of scientific uncertainties. These criticisms often result 

in legal challenges to EPA’s decisions. 

EPA’s human health risk assessment process is also greatly dependent on 

its degree of knowledge about the population’s level of exposure to 

contaminants. This knowledge includes the extent to which people are 

exposed to potentially harmful contaminants in their daily lives, the 

chemicals to which they are most often exposed, the levels of such 

exposures, how exposures change over time, and the sources of exposure. 

Risk assessors, researchers, and policymakers must often rely on 

estimates of human exposure that are often derived from data showing the 

extent the chemicals are found in the air, water, food, or other 

environmental media and assumptions about how and at what rate the 

body absorbs the chemicals it contacts. In addition to estimates from 

models, extrapolations from experiments involving animals, and 

measurements of chemicals in the environment, EPA also relies on more 

direct methods to measure exposure and more accurately assess exactly 

how much of a contaminant has been absorbed in the body. For example, 

EPA uses population activity models—models based on actual human 

behavior, such as the time spent outdoors or, for children, the amount of 

time spent on the floor—to better estimate an individual’s true exposure. 

Risk assessment activities at EPA are carried out by both the agency’s 

Office of Research and Development (ORD)—its principal scientific and

research arm—and its program and regional offices, including the Office 

of Air and Radiation, Office of Pesticide Programs, Office of Pollution 

Prevention and Toxics, Office of Solid Waste and Emergency Response, 

and Office of Water. ORD carries out all steps of highly complex, 

precedent-setting risk assessments for specific contaminants, such as 

Page 10 GAO-06-595 Human Health Risk Assessment
dioxin. In addition, ORD often has responsibility for the first two steps of 

the risk assessment process—hazard identification and dose-response 

assessment—in support of the program offices. In such cases, the last two 

steps—exposure assessment and risk characterization—are the 

responsibility of the various program offices. Three notable exceptions 

exist to this division of responsibility. The Office of Pesticide Programs 

and the Office of Pollution Prevention and Toxics often conduct all steps 

of risk assessments independently, in part, because the relevant statutes 

for these program offices place strict time frames on decision making and 

the confidential nature of data provided to the agency under these 

programs. Furthermore, the Office of Water also does all of the stages for

some of the assessments for purposes of the Safe Drinking Water Act. 

EPA’s approach to risk assessment varies across program offices, often as 

a result of different regulatory and legal requirements. For example, a 

branch of the Office of Pollution Prevention and Toxics assesses data 

submitted by industry applicants on approximately 2000 new chemicals 

annually under the Toxic Substances Control Act (TSCA) of 1976. TSCA 

generally requires EPA to evaluate the chemicals within 90 days, but does 

not require all applicants to conduct laboratory tests on the potential 

hazards and risks of the chemicals. In contrast, the Clean Air Act requires 

EPA to establish National Ambient Air Quality Standards, to review the 

scientific basis for those standards at least every five years, and to revise 

the standards as appropriate. As part of this process, ORD summarizes the 

most current scientific information on the pollutant in question. ORD’s 

findings and conclusions are then combined with other exposure and risk 

analyses to determine what, if any, revisions should be made to the 

standards. 

Risk assessment has been the center of numerous reports, analyses, and

regulations over the years. One of the earliest was the National Academy 

of Sciences’ 1983 report Risk Assessment in the Federal Government: 

Managing the Process, often referred to as the “Red Book” because of the 

color of its cover. Subsequently, NAS released several related reports, 

including Science and Judgment in Risk Assessment (1994),

Understanding Risk: Informing Decisions in a Democratic Society

(1996), and Strengthening Science at the U.S. Environmental Protection 

Agency (2000). In addition, Congress, via the Clean Air Act Amendments of

1990, created the Presidential/Congressional Commission on Risk 

Assessment and Risk Management (Commission) and required it to 

investigate the policy implications and appropriate uses of risk assessment 

and risk management in various regulatory programs designed to protect 

people from cancer and other chronic health effects that may result from 

Page 11 GAO-06-595 Human Health Risk Assessment
exposure to hazardous substances. The Commission published its twovolume final report in 1997. 

In addition, the Office of Management and Budget (OMB) has issued 

guidelines, reports, and bulletins that have affected the practice of risk 

assessment. For example, in October 2001, OMB issued its Guidelines for 

Ensuring and Maximizing the Quality, Objectivity, Utility, and 

Integrity of Information Disseminated by Federal Agencies, which 

required agencies to issue their own guidelines to ensure the quality of 

information being disseminated. Furthermore, in September 2003, OMB 

issued a report to Congress on the costs and benefits of federal regulation. 

As part of this effort, OMB sought public comment on the practice of risk 

assessment, many of which were directed at, and passed along to, EPA. 

OMB also issued the Final Information Quality Bulletin for Peer Review

in December 2004 that provided guidance to federal agencies on what 

information is subject to peer review and defined a planning process for 

peer review. Most recently, on January 9, 2006, OMB released a draft 

bulletin on procedures agencies should use to conduct risk assessments 

and information that should be included. Public comment on this draft will 

be accepted until June 15, 2006. 

Independent reviewers, including the National Academy of Sciences (NAS) 

and the Presidential/Congressional Commission on Risk Assessment and 

Risk Management have made a number of significant recommendations to 

improve EPA’s human health risk assessment process since 1994. These 

recommendations cover a range of actions, including improving planning

of what will be required to complete a particular risk assessment and what 

key stakeholders need to be involved, supporting development of new 

scientific data on the potential adverse effects from exposure to the 

contaminant under review, and enhancing analysis of the potential risk 

and its uncertainties. All of them can be summarized and discussed in 

terms of the seven themes—implementation, data needs, default options, 

method and model evaluation, uncertainty, variability, and exposure to 

multiple contaminants and routes of exposure—reported in NAS’s report 

Science and Judgment in Risk Assessment.

7

Independent 

Reviewers Have 

Recommended 

Improvements to 

EPA’s Risk 

Assessment Process 

Since 1994 

 7

National Academy of Sciences, Science and Judgment in Risk Assessment (Washington,

D.C., 1994). 

Page 12 GAO-06-595 Human Health Risk Assessment
Independent reviewers made a number of recommendations to EPA to 

improve the overall implementation of its risk assessment process. The 

reviewers recommended that EPA improve its planning of risk 

assessments before beginning the process and adopt an iterative risk 

assessment approach. Reviewers also stressed that EPA should increase 

its use of risk assessment reviews by independent scientists, known as 

peer reviews; increase stakeholder involvement in the risk assessment 

process; and ensure it has the workforce needed to conduct quality risk 

assessments. Finally, the reviewers recommended that EPA improve the 

way it characterizes the risk it finds and strengthen the link between risk 

assessment and risk management. 

In 1996, NAS reviewers stressed the importance of planning from the 

beginning of a risk assessment to incorporate the perspectives and 

knowledge of interested and affected parties.8

 Because of the large number 

of stakeholders interested in the results of a risk assessment, it is 

important to ensure that the risk assessment addresses all of the 

stakeholders’ needs. NAS pointed out that one way to do this is to be 

aware of the stakeholders’ concerns from the outset and incorporate them 

into the analysis and characterization of risk. By involving stakeholders 

early, risk assessors can ensure that they ask the right questions, make 

appropriate assumptions, and determine the best way to summarize 

information, thereby potentially making the resulting message more 

credible to these parties.

In addition to improved planning, NAS suggested in 1994, and the 

Commission concurred, that EPA develop the ability to conduct risk 

assessments iteratively—that is, begin with a screening analysis to 

ascertain the potential risk and refine that analysis as appropriate. Such an

approach would start with relatively inexpensive screening techniques and 

move to more resource-intensive data gathering and modeling as the 

particular situation warranted. To guard against the possibility of 

underestimating risk, these screening techniques should be constructed to 

err on the side of caution where there is uncertainty. 

Each of the NAS reports we reviewed, as well as the report by the 

Commission, recommended ways EPA could improve its peer review 

process and better involve stakeholders in risk assessment. These 

Improve Planning and 

Review Process and 

Ensure an Appropriately 

Skilled Workforce to 

Conduct Risk Assessment 

Improve Planning and Adopt an 

Iterative Risk Assessment 

Approach 

Increase Peer Review and 

Stakeholder Involvement in the 

Risk Assessment Process 

 8

National Academy of Sciences, Understanding Risk: Informing Decision in a

Democratic Society (Washington, D.C., 1996).

Page 13 GAO-06-595 Human Health Risk Assessment
independent reviewers said peer review is critical to evaluate the accuracy

and appropriateness of technical information, scientific observations, and 

interpretations used in regulatory decision making. The Commission 

observed that peer review can also help in the planning stages of a risk 

assessment to put a problem in context and draw on the knowledge of 

experienced researchers, public health officials, and scientists. 

In particular, NAS recommended in 2000 that EPA change its peer review 

policy to ensure greater independence of these reviews from the control, 

or appearance of control, of the program managers.9

 EPA’s Science 

Advisory Board had expressed concern about potential conflicts of 

interest because agency policy allowed the same individual to serve as 

both project manager for a particular work product and peer review 

leader.10 The NAS committee concurred with this concern. 

In addition, NAS’s 1994 report recommended that EPA collaborate more 

with outside parties to improve the overall risk assessment process. For 

example, EPA could use external advisory groups, such as its Science 

Advisory Board, to help ensure that risk assessment decisions use the best

science and incorporate full public discussion and participation from the 

scientific community. Such collaboration could also include a process for 

public review and comment. The Commission also said that agencies 

should use advisory groups, composed of stakeholders, to periodically 

evaluate the use of technical information. In addition, the Commission 

recommended that agencies establish clear, written guidelines for peer 

review and match the use of peer review to the importance of the decision

to be made.

In 1996, NAS commented on the importance of stakeholder involvement in

the risk assessment process. NAS suggested that the risk assessment 

process involve the spectrum of interested and affected parties, including 

decision makers and specialists in risk analysis, at each step of the 

process. According to NAS, such involvement will help ensure that as 

much important, relevant knowledge as possible enters the process and 

 9

National Academy of Sciences, Strengthening Science at the U.S. Environmental

Protection Agency (Washington, D.C., 2000).

10EPA’s Science Advisory Board provides independent advice and peer review on scientific 

and technical aspects of environmental problems and issues. Experts, including scientists, 

engineers, and economists, provide independent, balanced, and scientifically sound advice

to EPA. 

Page 14 GAO-06-595 Human Health Risk Assessment
that the process remains open and inclusive. NAS also mentioned another 

benefit of involving these stakeholders in the process: the participation of 

a spectrum of people helps ensure that the process will be framed 

correctly and the resulting risk characterization will be accurate, balanced, 

and informative. 

Each of the NAS reports we reviewed contained recommendations 

stressing the importance of an organization’s ability to ensure that it has 

the expertise and leadership needed to conduct risk analyses. In addition, 

according to current workforce models, agencies need to ensure that they

have effective training and the capability to identify what skills and 

competencies the employees and the organization need. In 1994, NAS 

recommended that EPA acquire additional expertise, as needed, to better 

interpret data and reach sound conclusions about the risks to human 

health from a contaminant. For example, many of the conclusions 

regarding the potential risks from a particular contaminant rely upon risk

assessment models based on animal data or, when available, human 

evidence from epidemiological studies. The quality of the risk assessment 

will depend upon how well these data are interpreted to predict health 

effects in human populations. NAS recommended that EPA acquire staff 

with specialized skills in fields such as toxicology to successfully complete 

this type of analysis. 

NAS also recommended in 2000 that EPA continue to place high priority 

on ORD’s graduate fellowship and postdoctoral program. The report 

stated that to achieve scientific and technical excellence, EPA must first 

attract, retain, and properly support a dedicated professional staff. NAS 

also noted that while EPA has many outstanding scientists and engineers, 

ORD’s workforce is aging and many staff may retire in the relatively near 

future. Periodic hiring freezes, combined with high competition from the 

private sector, had made it difficult for ORD to recruit the new talent 

needed to sustain and enhance the research workforce. NAS concluded 

that ORD’s graduate fellowships and postdoctoral programs brought a 

stream of fresh scientific and technical talent into EPA’s research program 

and helped train future researchers in environmental science, engineering,

and other disciplines. NAS urged EPA to continue to place a high priority 

on these programs. 

In 2000, NAS made a series of recommendations related to scientific 

leadership and talent at EPA. Specifically, NAS recommended that EPA 

establish a new position—deputy administrator for science and 

technology—to address concerns that science at EPA was not perceived to

be strong and that many EPA scientists believed their scientific knowledge 

Ensure EPA Has the Skilled 

Workforce Needed to Conduct 

Quality Risk Assessments 

Page 15 GAO-06-595 Human Health Risk Assessment
and resources were not effectively used. Traditionally, EPA’s most senior 

science official had been the assistant administrator for research and 

development, but that official lacked agencywide responsibility or 

authority to oversee the scientific and technical basis for regulatory and 

policy decision making. In addition, EPA’s program offices were not 

required to follow scientific advice from ORD. Consequently, the NAS 

panel concluded that EPA needed an appropriately qualified science 

official at a sufficiently high level with both the authority and 

responsibility for agencywide scientific performance. 

Finally, reviewers recommended ways EPA could improve its 

characterization of risk and better link its risk assessment and risk 

management processes to harmonize the scientific aspects of risk 

assessment and the broader policy objectives of risk management. NAS’s 

1996 report, which primarily focused on risk characterization and the role 

it plays in risk assessment, stressed that successful risk characterization 

can result in better and more widely accepted risk assessment and risk 

management decisions. The Commission echoed NAS’s position and 

recommended that risk characterizations should include information 

useful to everyone in the risk management process, such as information on

who is at risk, how they might be affected, what the severity of an adverse 

effect might be, and how confident the risk assessors are about their 

predictions. 

Reviewers also suggested that EPA better link its risk assessment and risk 

management processes. EPA had tried to separate these processes entirely 

to avoid the possible perception that EPA made scientific judgments on 

the risk posed by a contaminant on the basis of its willingness to regulate 

the substance rather than on the science. However, NAS pointed out that 

while some degree of judgment is necessary, the science-policy judgments 

EPA makes in the course of risk assessment should more clearly be 

informed by the agency’s risk management priorities and goals. According 

to NAS, better linkage between risk assessment and risk management will 

help ensure that the risk assessments will be more accurate and relevant

to risk managers. The Commission concurred, stating that risk assessment 

should be guided by an understanding of the issues important to risk 

managers’ decisions about how to protect public health and the 

environment. 

Improve Characterization of 

Risk and More Closely Link 

Risk Assessment with the 

Needs of Risk Managers 

Page 16 GAO-06-595 Human Health Risk Assessment
In 1994, NAS found, in general, that EPA did not have the full range of data

needed to conduct adequate risk assessments to estimate the health risks 

associated with most contaminants—for example, data related to the 

effect a contaminant has on the body and the extent of exposure to the 

contaminant. These data are essential to ensure the accuracy and 

precision of the risk assessment. In addition, NAS recommended that EPA 

increase its collaboration with researchers and improve the transparency

of its research priorities. 

To help address EPA’s dearth of data, NAS recommended, among other 

things, that EPA review its existing databases, such as the Integrated Risk

Information System (IRIS), and identify data gaps.

11 As part of its database 

review, NAS recommended that EPA compile an inventory of the 

chemical, toxicological, and epidemiological literature in the databases to

determine what gaps exist. NAS further suggested that EPA prioritize its 

future research on the basis of the significance of the missing data to risk 

assessments, convey its data needs to other public and private 

researchers, such as the Agency for Toxic Substances and Disease 

Registry,12 and develop incentives to expedite the generation of needed

data by these entities. 

Furthermore, in 2000, NAS recommended, among other things, that EPA 

expand the multiyear research planning by its Office of Research and 

Development (ORD). For example, ORD had developed plans for research 

on topics such as particulate matter, endocrine disruptors,13 and drinking 

water. The NAS committee pointed out that these planning efforts will 

contribute to research program continuity and the achievement of 

strategic goals. In addition, NAS recommended that the plans be both 

continued for problem-driven research—research targeted at a particular 

environmental problem—and expanded to include core research—

Review Existing Data, 

Identify Data Gaps, 

Prioritize Research Needs, 

Foster Development of 

New Data, and Improve 

Strategic Research 

Planning and 

Collaboration 

 11IRIS contains information on the human health effects that may result from exposure to

various chemicals in the environment and helps provide consistent information on

chemical substances for use in risk assessments.

12The Agency for Toxic Substances and Disease Registry is the primary public health 

agency involved with hazardous waste issues and works to prevent or reduce the harmful

effects of exposure to hazardous substances on human health by, for example, supporting 

research that assists with risk assessment. 

13Endocrine disrupting chemicals are thought to mimic natural human hormones that 

influence important regulatory and development mechanisms such as blood pressure,

metabolism, and reproduction. Important endocrine glands include the thyroid, pancreas, 

and male and female gonads (testes and ovaries). 

Page 17 GAO-06-595 Human Health Risk Assessment
broader, more generic research that will help improve current and future

understanding of a scientific issue. Research should be balanced between

these two types, since core research will help ORD better understand and 

anticipate environmental risks and will enable ORD to provide better 

problem-specific research and technical assistance to the agency. 

NAS has also made recommendations to EPA in the area of research 

accountability. Specifically, NAS recommended that EPA improve the 

documentation and transparency of the decision-making process ORD 

uses to set research and technical assistance priorities and allocate funds. 

For example, NAS pointed to an EPA Science Advisory Board finding that 

ORD’s lack of transparency in its process for setting research priorities 

made it difficult to evaluate the adequacy of proposed budgets. NAS also 

recommended that EPA expand on its efforts to create an inventory of 

science projects and programs across EPA by documenting and publishing 

a comprehensive and detailed inventory. The inventory should include 

information such as goals and objectives of each project, milestones, 

schedules, and staff allocations and should be used to ensure that science 

activities are properly coordinated through the agencywide science 

planning and budgeting process and are appropriately peer reviewed. 

Once the research strategy has been established, research collaborations 

can help execute it. NAS suggested that EPA should recognize the limits of 

its own research capabilities and develop a strategy to obtain outside 

support in research areas where ORD is not pre-eminent. Specifically, NAS 

recommended EPA enhance its research collaborations by developing and 

implementing a strategy to stimulate, acquire, and apply the results of 

research conducted or sponsored by other federal agencies, state 

agencies, universities, and industry in this country and abroad. NAS said 

such collaboration is especially important given the budget constraints 

EPA and the federal government face and the breadth of knowledge 

required to conduct the full range of risk assessments. In addition, NAS 

recommended that EPA develop additional mechanisms to promote and 

facilitate research interactions among grantees in EPA’s Science to 

Achieve Results (STAR) grant program and ORD research staff.14 For 

example, NAS suggested that grant applicants could identify in their 

proposals how their research might complement or supplement ongoing or 

planned research in the ORD laboratories. NAS also recommended that

 14The STAR program funds peer reviewed competitive grants on research topics selected 

by ORD and postgraduate training for scientists in environmental fields. 

Page 18 GAO-06-595 Human Health Risk Assessment
EPA increase its efforts to disseminate ORD’s research products, explain 

their significance, and assist others inside and outside the agency in 

applying the research. NAS noted that while EPA’s policy and regulatory 

work receives a great deal of public attention, the agency’s science 

research typically receives a similar degree of attention only when the 

scientific basis for a decision is questioned. They also pointed out that 

even internal EPA offices may be unaware of important benefits from 

ORD’s research program. 

When EPA needs to rely on assumptions—known as default options—

because it lacks data to fully assess the potential health risks from 

exposure to a contaminant, reviewers said EPA should, among other 

things, more clearly indicate that it relied on these defaults and how they 

were chosen. EPA’s defaults, which are based on general scientific 

knowledge and policy judgment, are, for the most part, conservative.15 One 

example of a commonly used default option is that laboratory animals are 

a surrogate for humans in assessing health risks: An adverse effect in the 

animals is taken as evidence of a chemical’s potential to harm humans. 

Independent reviewers agreed with EPA’s use of default options as a 

reasonable way to cope with the lack of available data. However, NAS 

recommended that EPA be clearer about the scientific and policy basis for 

each default option and when it uses a default. Furthermore, reviewers 

said that EPA should fully explain under what conditions they would 

depart from these default options. According to NAS, giving greater 

formality to the criteria for a departure would result in greater guidance

for the public and lessen the possibility of undocumented departures that 

would undercut the scientific credibility of a risk assessment. The 

Commission concurred, stressing that the defaults used in risk 

assessments and the uncertainty associated with their results should be 

clearly identified and justified. 

More Clearly Indicate Use 

of Default Options and 

How They Were Chosen 

 15EPA’s defaults represent a choice that, although scientifically plausible given the existing 

uncertainty, is more likely to result in overestimating than underestimating human health

risk. 

Page 19 GAO-06-595 Human Health Risk Assessment
The overall accuracy of a risk assessment largely depends on the validity 

of the various analytic methods and models EPA uses to assess the 

toxicity of, and exposure to, a particular contaminant. However, according 

to NAS, EPA often does not clearly understand the extent to which the 

methods or models it chooses accurately predict the toxicity or exposure 

to a contaminant. For example, because of limited scientific information 

on how contaminants actually work in the body to produce adverse effects 

in humans, EPA frequently uses a method that relies on studies involving 

laboratory animals to understand the toxicity of a substance. The 

concentrations of the contaminant introduced into the animals as part of 

these studies are higher and administered for shorter periods of time than 

humans would normally experience. Consequently, to determine the 

expected response in people, EPA extrapolates the response from 

laboratory animals to humans. While such extrapolations are useful to 

predict potential harmful effects of a contaminant, different analytic 

methods may better predict the effect. 

In 1994, NAS recommended, among other things, that EPA evaluate the 

accuracy of its methods and models for assessing toxicity and, when 

appropriate, incorporate more advanced tools. For example, models that 

are based on the underlying mechanisms at the cellular or molecular level 

can more accurately estimate the dose of the contaminant that would have 

an adverse biological effect on a specific part of the body. Regarding the 

risk of cancer from a particular contaminant, NAS recommended that EPA 

continue to both use methods involving animal studies to evaluate the 

possibility of adverse effects in humans and explore, when appropriate, 

mechanistic models. Furthermore, to better convey the cancer risk 

associated with a particular substance, NAS recommended that EPA 

develop a classification scheme that provides narrative statements 

regarding the hazards posed by carcinogens and a descriptive evaluation 

of the strength and nature of the evidence used to estimate the substance’s 

potential for causing cancer. 

Regarding exposure, NAS noted that EPA had traditionally characterized 

exposure according to two criteria: exposure of the total population and 

exposure of a specified highly exposed subpopulation. While these two 

criteria can help assess whether any particular exposure might occur 

above a regulatory threshold, only considering the highly exposed 

subpopulation is likely to overestimate the exposures of most of the 

population. Consequently, NAS recommended, and the Commission 

concurred, that EPA consider the entire range of a population’s exposure, 

rather than just the exposures of a highly exposed subpopulation. The 

Commission also recommended that EPA identify and evaluate highly

Better Evaluate Available 

Methods and Models and 

Incorporate More 

Advanced Tools When 

Appropriate 

Page 20 GAO-06-595 Human Health Risk Assessment
exposed populations separately. Moreover, NAS recommended that EPA 

use population-activity models—models based on actual human activity 

patterns—to better estimate an individual’s true exposure and expand 

efforts to use personal monitoring data to better understand actual 

exposures and variances across the population. 

To address the inevitable uncertainties associated with gaps in scientific 

knowledge and general unknowns about model and data accuracy, 

reviewers recommended that EPA more explicitly analyze and 

characterize the sources of uncertainty in its risk assessments. Numerous 

gaps in scientific knowledge exist regarding the health effects of various 

contaminants, such as the exact amount of exposure to a particular 

contaminant that can cause an adverse effect or the biological effect of a 

contaminant on the body. In addition, knowledge is often lacking about 

which model or method might be most appropriate to estimate risks to 

human health from a particular contaminant. NAS recommended that EPA 

develop guidelines for how to analyze and report the different types of 

uncertainty, both for the overall assessment and for the different stages of

the risk assessment, such as hazard identification and exposure 

assessment. For example, during hazard identification, uncertainty can be 

related to the quality, type, and results of scientific studies; however, 

during exposure assessment, uncertainty can be related to model choice 

or available data. 

NAS also recommended in 1994 that EPA conduct uncertainty analysis of 

the risk estimate and present the identified uncertainties as explicitly, 

accurately, and fully as feasible. Analysis of the effects of uncertainties 

can help inform EPA decision makers and the public about the extent of 

uncertainty associated with the risk assessment. The analysis can also 

show where additional research might resolve major uncertainties and 

where it might not. NAS recommended that the uncertainty analysis be 

presented both descriptively (qualitatively) and, where possible, 

numerically (quantitatively). For example, some sources of uncertainty, 

such as those related to estimating exposures, can be reduced through the 

use of more advanced statistical methods. Other types of uncertainty, such 

as those associated with extrapolating data from animal testing to predict

the effect on humans, are more difficult to quantify. The Commission 

concurred and recommended that risk characterizations include narrative

descriptions of the primary reasons for uncertainty, as this information is 

likely to be more understandable and useful than quantitative estimates or 

model results. 

More Explicitly Analyze 

and Characterize the 

Sources of Uncertainty in 

Risk Assessments 

Page 21 GAO-06-595 Human Health Risk Assessment
Variability, which refers to the natural diversity in a population, can be 

better understood or described, but not reduced. To address the two main 

categories of variability—one related to differences in levels of individual

exposures to contaminants and the other related to differences in 

individual reactions to exposure—reviewers recommended that EPA 

enhance its analysis of both types and carefully state in each risk 

assessment what assumptions it made about what is and is not accounted 

for. Specifically, variability related to different exposures depends on the 

various concentrations of a contaminant as it disperses in the 

environment, different breathing rates, and different food consumption 

and personal activity patterns. For example, infants and children generally 

consume more fruits, vegetables, and fruit juices per body weight than

adults, and some people, such as agricultural workers, are more exposed 

to pesticides through breathing and skin contact. The Commission 

recommended that risk assessments identify groups of people who are 

likely to have higher exposures to contaminants and consult these groups 

in the early stages of an assessment to obtain information about all known 

sources of exposure. 

Reviewers also recommended that EPA revise the way it estimates how 

long a person is exposed to a contaminant. NAS and the Commission 

recommended that EPA move away from estimates of exposure based on 

a hypothetical “maximally exposed individual,” who was assumed to be 

the person at greatest risk in a worst-case scenario, because these 

estimates do not account for a number of other factors that may affect 

exposure patterns and rates, such as the time the person spends indoors 

or going to work. Furthermore, estimates based on the hypothetical 

maximally exposed individual likely overestimate the exposures of most of 

the population and underestimate the exposures of subpopulations, such 

as agricultural workers, who may be more highly exposed than the general 

population. While EPA’s 1992 exposure assessment guidelines suggest the 

use of ranges and high-end exposure estimates chosen from the high end 

of those ranges, according to NAS, EPA had not sufficiently documented 

the reliability of such estimates when data are limited. 

The second type of variability—differences in human susceptibility—is 

related to inherent differences among people, such as age, physiologic 

characteristics, lifestyle, genetics, sex, and ethnicity. Reviewers found that 

EPA’s approach for reducing risks associated with chemical exposures 

generally did not include information on differences in individual 

susceptibility or encourage gathering evidence to identify these 

differences. The reviewers recommended, among other things, that EPA 

consider this “interindividual” variability and adopt a default option for 

Enhance Analysis of 

Variability in Exposure 

Levels and Health Risks to 

Exposed Individuals 

Page 22 GAO-06-595 Human Health Risk Assessment
differences in susceptibility among humans. In addition, NAS 

recommended that EPA assess risks to infants and children whenever it

appears that their risks might be greater than those of adults. For example, 

the developing brains of infants and young children have an increased 

susceptibility to contaminants that harm the nervous system, such as lead. 

NAS specifically recommended that EPA sponsor research to examine the 

causes and extent of interindividual variability in susceptibility to cancer 

and the possible connection between susceptibility and age, race, 

ethnicity, and sex. The Commission generally concurred, adding that risk 

assessments should also identify especially susceptible subpopulations, 

such as people with asthma who may have an increased responsiveness to 

allergens and respiratory irritants. The Commission also stated that, where 

possible, available information about the range of the population’s 

susceptibility should be considered and used in place of default 

assumptions. 

Reviewers recommended that EPA improve how it considers the effects of 

combinations of contaminants (cumulative exposure) and all possible 

paths of exposure to a single contaminant (aggregate exposure). People 

are typically exposed to a mixture of contaminants through a variety of 

pathways, such as contact with skin or eating food, each of which might 

be associated with an increased probability of one or more health effects. 

However, most risk assessments address a single contaminant and often 

focus on a single pathway of exposure, such as inhalation. As a result, NAS 

recommended that EPA should consider all possible exposure pathways. 

For example, when assessing risk from mercury, EPA should consider the 

risk to residents from inhaling mercury emitted from a nearby industrial 

smoke stack, as well as the possibly greater health risk of consuming 

mercury that has accumulated in the tissue of fish that are caught and 

eaten locally after mercury from the smoke stack was deposited into 

water.16

When assessing the risks of exposure to chemical mixtures, most risk 

assessments estimate the risks from individual contaminants, then 

calculate the combined risk by simple addition. However, this method 

ignores potentially synergistic interactions that may make the effects more 

Better Consider the 

Human Health Effects of 

Exposure to Multiple 

Contaminants and Routes 

of Exposure 

 16Contaminants can progressively accumulate in the tissues of an organism, such as a 

human or a fish, as a result of uptake by the body from all routes of exposure. This process,

called bioaccumulation, occurs because the rate of intake exceeds the organism’s ability to 

eliminate the substance from the body. 

Page 23 GAO-06-595 Human Health Risk Assessment
damaging to human health than anticipated or antagonistic interactions 

that may make the effects less damaging than anticipated. Consequently, 

this method could either under- or overestimate the total risk. NAS said 

that simple addition of the risks from multiple contaminants may be 

appropriate for screening-level risk estimates. However if a more refined 

quantitative estimate is needed, EPA should consider using statistical 

procedures to combine the risks from exposures to multiple contaminants, 

which would help produce a more comprehensive estimate of risk. The 

Commission concurred but stressed that combining risks may not always 

be feasible; the risk analyses for exposure to each contaminant may not be 

compatible because the risk assessments may differ in accuracy. Further, 

the Commission recommended that for risk assessments involving 

exposures to low concentrations of multiple chemicals, the risks from

each exposure should be added in the absence of information on exactly 

how the chemicals affect the body. However, if the multiple chemicals 

affect the body in different ways—for example, if one chemical affected 

development while another affected the nervous system—the impact of 

each chemical on the body should be considered independently and not 

added together. 

EPA has modified its human health risk assessment process since 1994 in 

several ways. First, the agency enhanced implementation of its risk 

assessment process by, for example, issuing guidance and realigning staff 

resources. In addition, EPA has taken steps to identify the scientific data it 

has on the potential adverse effects from exposure to various

contaminants and has established collaborative relationships with external 

researchers to foster the development of needed additional data. 

Furthermore, EPA has begun to improve its use of default options, 

enhance its modeling capabilities, and explore new methodologies. EPA 

has also begun to characterize uncertainty quantitatively and analyze and

communicate variability more thoroughly. Finally, EPA is more often 

considering the combined effects of exposure to multiple contaminants 

through multiple pathways. Most of these efforts are ongoing and can be 

discussed in terms of the themes presented in the previous section. 

EPA has taken a number of steps to improve implementation of its risk 

assessment process. Specifically, EPA has developed guidance and policy 

documents at the agency and program office levels; built scientific 

capacity; modified components of its approach to risk assessment; and 

refined its peer review and quality-assurance practices. 

EPA Has 

Strengthened Many 

Facets of Its Risk 

Assessment Process 

Since 1994, and 

Efforts Are Ongoing 

EPA Has Enhanced 

Implementation of Key 

Aspects of Its Risk 

Assessment Process 

Page 24 GAO-06-595 Human Health Risk Assessment
The guidance and policy documents EPA has issued over the past decade

were intended to help staff develop and use risk assessments and to 

provide basic information to the public about EPA’s risk assessment 

methods. While some of these documents have remained unchanged over 

time, many have been revised, or will be revised, as science, knowledge, 

and analytic methods have improved. Among these documents are the 

guidelines issued by EPA’s Risk Assessment Forum, a committee of senior

EPA scientists established to promote agencywide consensus on risk 

assessment issues, which cover such topics as neurotoxicity,17 exposure 

assessment, and carcinogenic risk assessment. Each of the original five 

guideline documents created in 1986 has been updated at least once, and 

some are slated to be revised again. For example, the exposure guidelines 

were revised in 1992 and revisions are currently being planned. In 

addition, the forum has issued two entirely new sets of guidelines since 

1994—Guidelines for Reproductive Toxicity Risk Assessment and 

Guidelines for Neurotoxicity Risk Assessment—and has developed a 

number of policies on, among other things, risk characterization, peer 

review, and evaluating risk to children. EPA has also issued interim policy 

memorandums and position papers on scientific issues such as genomics 

and endocrine disruption. 

Similarly, much of EPA’s agencywide guidance issued since 1994 has 

undergone revision and has its origins in earlier policy documents. For 

example, EPA’s Risk Characterization Handbook, issued in 2000, has its 

roots in the 1995 Policy for Risk Characterization. The handbook stresses 

that risk characterization should be transparent, clear, consistent, and 

reasonable. EPA’s policy on the use of peer review was originally issued in

1994 and was followed up by issuance of peer review handbooks in 1998 

and again in 2000.

18 The peer review policy was recently updated in 

January 2006. EPA has also issued technical guidance, such as its review 

of the processes to estimate a daily or continuous exposure to humans 

that is likely to be without appreciable adverse effects during a lifetime. 

In addition to the EPA-wide guidance, many of the program offices have 

also issued guidance documents that support their particular risk 

assessment efforts. In general, the office-specific guidance documents 

provide risk assessors with analytic tools and exposure scenarios 

EPA Has Issued Many 

Agencywide and Program 

Office-Specific Guidance and 

Policy Documents to Improve 

Risk Assessment Practices 

 17Neurotoxicity is an adverse change in the structure or function of the central or 

peripheral nervous system following exposure to a chemical, physical, or biological agent. 

18U.S. Environmental Protection Agency, Peer Review Handbook (Washington, D.C., 2000). 

Page 25 GAO-06-595 Human Health Risk Assessment
pertinent to the statutory responsibilities of the office. For example, the 

Office of Pollution Prevention and Toxics created guidance in the form of 

an analytic tool to screen chemicals in the absence of data, which 

frequently occurs because of the lack of a requirement for industry to 

develop extensive data on new chemicals or new uses of existing 

chemicals. In addition, the Office of Pesticide Programs has issued 

pesticide-specific guidance documents. For example, to help ensure 

consistency in pesticide chemical risk assessments, the Office of Pesticide 

Programs issued guidance for developing residential exposure 

assessments and developed a template for making and documenting 

registration eligibility decisions. 

EPA enhanced its scientific leadership through the creation of the position

of science advisor in 2002 and the Office of the Science Advisor in 2003, 

the increased reliance on research advisory groups composed of senior 

EPA scientists and external experts, and the continuation of its research 

fellowship programs. The overarching responsibility of the science advisor 

is to coordinate and oversee the scientific activities of the program offices 

at EPA to ensure the best use of science. The Office of the Science Advisor 

provides further leadership by establishing specific mechanisms to ensure 

that scientific results, combined with technical evaluation and peer 

review, play a prominent role in regulatory decisions and that EPA staff 

interpret and enforce regulations consistent with the science supporting 

them. 

In addition to enhancing its scientific leadership, EPA has also increased 

its reliance on research advisory groups since 1994. The Science Policy 

Council and the Risk Assessment Forum play key roles in advancing the 

practice of risk assessment at EPA. The council reviews the adequacy of

existing policies, establishes science policy as needed, and coordinates 

EPA efforts related to methods, modeling, risk assessment, and 

environmental technology. The Science Policy Council staff facilitate ad 

hoc work groups, encourage communication and consensus building 

within the agency, and participate in technical work-group activities and 

deliberations. 

The Risk Assessment Forum is a standing committee of senior EPA 

scientists established to promote agencywide consensus on difficult and 

controversial risk assessment issues and to ensure that this consensus is 

incorporated into guidance. According to an agency official, the forum is 

designed as a venue where staff can meet and discuss common risk 

assessment issues across program offices. One of the forum’s main 

contributions to risk assessment at EPA has been the issuance of a series 

EPA Has Built Scientific 

Capacity through Increased 

Focus on Scientific Leadership, 

Greater Reliance on Research 

Advisory Groups, and 

Development of Future 

Scientific Talent 

Page 26 GAO-06-595 Human Health Risk Assessment
of risk assessment guidelines. The forum is currently working on new 

guidelines, such as one related to adverse effects on the immune system. 

When more specificity is needed on an existing guideline, the forum issues 

companion pieces, known as “purple books” because of the color of their 

cover, that provide additional or updated information. 

The Board of Scientific Counselors (BOSC) provides objective and 

independent advice, information, and recommendations about ORD’s 

research program to ORD’s assistant administrator. BOSC is composed of 

scientists and engineers from academia, industry, and environmental 

organizations who are recognized as experts in their fields. In 1998, BOSC 

completed a peer review of ORD’s laboratories and centers.19 BOSC 

completed a second review of the laboratories and centers in 2002 and 

2003 that identified key accomplishments of the laboratories and centers, 

as well as areas for future improvement. In addition, after EPA’s Office of 

the Science Advisor issued its 2004 staff paper,20 it asked BOSC to host a 

workshop for EPA staff and other interested stakeholders, such as 

industry, environmental groups, and researchers, to provide feedback to 

refine EPA’s current practices and to suggest alternative approaches for 

specific aspects of risk assessment. 

EPA has also worked to foster scientific excellence and enhance the skills 

of its existing workforce through its graduate and postdoctoral fellowship 

programs. One such program, the EPA/ORD Postdoctoral Fellowship 

Program, began in 1998, and, as of May 2003, 205 individuals had 

participated in the program. One benefit of the program is that it helps 

provide a ready pool of talented candidates for EPA vacancies. In fact, 

according to a study of the program conducted by the National Council for 

Science and the Environment, nearly half of the former postdoctoral 

participants had taken permanent positions at EPA. The Science to 

Achieve Results (STAR) fellowship program is designed to encourage 

masters and doctoral students to pursue careers in an environmental field.

The STAR fellowship program has provided new environmental research 

in the biological and health sciences, two fields related to the development 

 19ORD’s laboratories and centers are the National Center for Environmental Assessment 

(NCEA), National Health and Environmental Effects Research Laboratory (NHEERL),

National Exposure Research Laboratory (NERL), National Center for Environmental 

Research (NCER), National Risk Management Research Laboratory (NRMRL), and 

National Homeland Security Research Center. 

20U.S. Environmental Protection Agency, Risk Assessment Principles and Practice 

(Washington, D.C., 2004). 

Page 27 GAO-06-595 Human Health Risk Assessment
of human health risk assessment. EPA has also partnered with the 

Association of Schools of Public Health (ASPH) to offer 1-year placements

of graduates in EPA laboratories, centers, and program offices to work on 

public health issues. In the announcement of opportunities for the 2006 

ASPH Fellows Program, a dozen fellowships are being offered in areas 

related to human health risk assessment. For example, the National Center 

for Environmental Assessment is offering a fellowship to develop health

assessments for various chemicals, and the National Center for 

Environmental Research is offering a position for someone interested in 

working on developing models or analyzing uncertainty in risk 

assessments.

To improve planning, which is a part of all risk assessments to some 

degree, EPA has issued various guidance documents and held workshops 

for staff. One of EPA’s earliest related guidance documents,21 released in

1997, was designed to help risk assessors and risk managers plan and 

document the scope of risk assessments and to consider input from 

appropriate stakeholders and experts, especially in those assessments 

involving the effects of combinations of contaminants.22 EPA followed up 

this guidance with workshops to help staff apply it in risk assessments. In

January 2002, EPA issued a handbook to reflect some of the lessons 

learned from implementation of the 1997 guidance to make risk 

assessments more useful to decision makers and other stakeholders. 

These lessons, conveyed through case studies, include that planning can 

be particularly valuable when the assessment is complex, controversial, or 

precedent setting, and that explaining uncertainty to stakeholders can help 

develop trust, credibility, and support for the decision-making process. 

The 2000 Risk Characterization Handbook also strongly advocates the 

use of planning and presents a number of topics for both risk managers 

and risk assessors to consider, such as identifying the stakeholders in the

process; scope of the effort; relevant management goals, issues, and 

policies; available data; and data needs. For example, during planning, risk 

assessors, risk managers, and stakeholders need to identify the key data 

gaps and discuss how best to fill them, such as whether to use existing 

data or conduct additional short- or long-term tests to evaluate exposure 

EPA Has Begun to Incorporate 

Planning Activities and 

Stakeholder Input into Its Risk 

Assessment Approach 

 21U.S. Environmental Protection Agency, Guidance on Cumulative Risk Assessment—Part 

1: Planning and Scoping (Washington, D.C., 1997). 

22U.S. Environmental Protection Agency, Lessons Learned on Planning and Scoping for 

Environmental Risk Assessments (Washington, D.C., 2002).

Page 28 GAO-06-595 Human Health Risk Assessment
and effects. The Office of Air and Radiation recognized the need for 

planning and developed planning guidance as part of its Air Toxics Risk 

Assessment Reference Library, issued in 2004. EPA acknowledged in its 

2004 staff paper that it needs to continue to stress the importance of 

concerted and conscious planning with risk assessors and risk managers 

before a risk assessment is started. According to EPA, risk assessors need 

to outline early in the development of a risk assessment what will and will 

not be addressed and how they will develop the risk assessment. 

Stakeholders and the public play a key role in the planning, as well as at 

later stages in the development of a risk assessment. Stakeholders, at 

various levels and in various forms, can help ensure better understanding 

of the risk assessment results and may promote support for the selected 

risk reduction strategies. Program offices involve stakeholders in various 

ways. For example, the branch of the Office of Air Quality Planning and 

Standards (OAQPS) responsible for setting certain air quality standards for 

six principal pollutants solicits input from stakeholders in the planning 

phase of its periodic updates to the standards it sets.23 In addition, the 

public may officially comment on draft air quality standards once they are 

publicly released. The Office of Water pursues stakeholder and public 

involvement that includes working with the environmental community, 

industry, trade associations, risk assessor organizations, states, and 

bordering countries. In addition, the office’s periodic reviews of water 

quality standards and other nonregulatory actions, such as health 

advisories, are all open processes that allow for public input on various 

stages of the analysis. 

For risk assessments involving the reregistration of pesticides, the Office 

of Pesticide Programs (OPP) established a process that provides several 

opportunities for public participation.24 Depending on the potential health

risks posed by a pesticide product, the public has anywhere from one to 

 23These six principal pollutants, known as “criteria pollutants,” are carbon monoxide, lead,

nitrogen dioxide, ozone, particulate matter, and sulfur dioxide. 

24EPA is reviewing older pesticides (those initially registered prior to November 1984) 

under the Federal Insecticide, Fungicide, and Rodenticide Act to ensure they meet current

scientific and regulatory standards. EPA is also reassessing tolerances (pesticide residue 

limits in food) to ensure they meet safety standards established under the Food Quality 

Protection Act (FQPA) of 1996. Under FQPA, EPA must reassess all tolerances established 

before August 3, 1996, within 10 years. The Office of Pesticide Programs reviews the 

registrations of all pesticide products every 15 years to determine whether they still meet 

safety standards as part of its registration review process. 

Page 29 GAO-06-595 Human Health Risk Assessment
four separate opportunities to comment. For example, if risk assessors 

estimate that the product poses little risk to human health, the public will 

have one opportunity to comment before OPP decides whether to approve 

the pesticide product. For higher-risk products, the public will have as 

many as four opportunities to comment. The first opportunity to comment 

occurs after OPP has completed a preliminary risk assessment. This 

preliminary assessment contains all of the elements of a risk assessment 

and has undergone internal review, but is not yet finalized. Notice of the 

opportunity to comment is distributed to people who have elected to sign

up for such notifications, as well as through a “notice of availability” 

published in the Federal Register. The public can also comment on risk 

assessments prepared by the Office of Pesticide Programs through the 

office’s Science Advisory Panel—which holds periodic public meetings on 

pesticide-related risk assessment issues, such as methods to assess skin 

sensitivity to exposure to pesticides or models used to estimate dietary 

exposures. 

EPA has also adopted an iterative approach to many of its risk 

assessments. An iterative approach begins with a screening assessment 

and progressively grows in depth and scope in relation to the estimated 

risks to human health. When a screening assessment identifies a potential

for a nontrivial risk, EPA decides if pursuing that risk is appropriate based

on its current priorities and available resources. If EPA decides to pursue 

the risk, a more detailed, refined risk assessment is performed. The degree 

of refinement is based on the type of decision, the available resources, and 

the needs of the risk manager. After refinement of the estimate, EPA 

reviews it to see if it will be sufficient to answer the questions posed. 

Refinements proceed iteratively until the assessment provides an adequate

answer for the decision maker within the resources available. Both the 

revised cancer guidelines and EPA’s 1995 Policy for Risk 

Characterization support an iterative approach to risk assessment. Some 

program offices have also adopted an iterative—or tiered—approach to 

risk assessment. For example, the air toxics program follows a tiered 

approach, beginning with an analysis that includes few data and many 

conservative assumptions. If this analysis indicates that the risk may be 

relatively high, assessors pursue more intensive analysis to determine if 

the risk is realistic or an artifact of the lower tier’s conservative 

assumptions. Despite this move toward greater use of an iterative 

approach, EPA acknowledges it could be clearer about when it is taking 

such an approach. For example, EPA could be more transparent about 

when and why it makes a risk management decision based on a screeninglevel assessment rather than a more detailed assessment. 

Page 30 GAO-06-595 Human Health Risk Assessment
In the years since the issuance of Science and Judgment in Risk 

Assessment, EPA has made strides to improve its peer review practices. 

EPA uses peer review to help ensure the quality of its risk assessments 

and keep them as objective and consistent as possible. EPA’s Peer Review

Policy states that scientifically and technically based work products 

related to agency decisions should be peer reviewed. In 2000, EPA issued 

its revised Peer Review Handbook, an update of the original 1998 edition. 

In the intervening 3 years, EPA’s Science Advisory Board, EPA’s Office of 

Inspector General, the National Research Council, and GAO scrutinized 

the peer review process. In response, in part, to recommendations made 

by many of these groups, EPA issued the current edition of the handbook.

Among other things, it instructs EPA to balance peer review panels in 

terms of expertise and biases to help ensure a reasonable and scientific 

review. 

In addition to issuing the Peer Review Handbook, EPA has undertaken a 

number of actions to help ensure the quality of its data and information. In 

May 2000, EPA established an agencywide quality-assurance system and 

issued the EPA Quality Manual for Environmental Programs. Key 

components of this system include assigning a quality-assurance manager 

to conduct independent oversight of data quality, developing a 

management plan, and conducting an annual assessment of the quality 

system. In addition, the system calls for an assessment of the data EPA 

used to support agency decisions to verify that they were of sufficient 

quantity and quality for their intended use. In 2002, EPA developed its 

information-quality guidelines in response to those issued by OMB, which 

stated that agencies must ensure the quality, objectivity, utility, and 

integrity of information released to the public.25 EPA’s guidelines outline 

its policy and procedures to ensure and maximize the quality of the 

information it disseminates and describe the mechanisms by which EPA 

reviews information prior to dissemination. EPA issued a complement to 

these guidelines in 2003 to raise awareness among the public about EPA’s 

ongoing interest in high-quality data and to serve as an additional resource 

for staff as they evaluate the quality and relevance of information.26

EPA Has Refined Its Peer 

Review and Quality-Assurance 

Practices 

 25Office of Management and Budget, Guidelines for Ensuring and Maximizing the 

Quality, Objectivity, Utility, and Integrity of Information Disseminated by Federal 

Agencies (Washington, D.C., 2002). 

26U.S. Environmental Protection Agency, A Summary of General Assessment Factors for 

Evaluating the Quality of Scientific and Technical Information (Washington, D.C., 2003).

Page 31 GAO-06-595 Human Health Risk Assessment
Since 1994, EPA has initiated a number of actions to develop missing or 

incomplete scientific data on the potential adverse effects from exposure 

to contaminants, including refocusing and expanding the Office of 

Research and Development human health risk assessment program, 

improving strategic research planning, increasing collaboration with 

outside researchers, and enhancing databases. 

In 1995, EPA restructured ORD to more effectively generate and gather

information needed for the risk assessment process, combining 12 existing

laboratories into 3 national laboratories and 2 national centers—the 

National Health and Environmental Effects Research Laboratory 

(NHEERL), National Exposure Research Laboratory (NERL), National 

Risk Management Research Laboratory (NRMRL), National Center for 

Environmental Research (NCER), and National Center for Environmental 

Assessment (NCEA).27 Specifically, one laboratory, NHEERL, conducts 

research on the effects of various exposure routes and rates, dose levels 

and duration, and cumulative doses on health outcomes. Its main research 

areas, all of which help improve risk assessments, target (1) the use of 

mechanistic information—how a substance causes an effect at a biological 

level—to reduce uncertainties in risk assessment; (2) the cumulative risks 

posed by exposure to contaminant mixtures; (3) the health risks to 

particularly susceptible populations; and (4) the evaluation of public 

health outcomes to determine the effectiveness of actions designed to 

reduce health risks. This research is developed in coordination with the 

program offices to target issues in risk assessment for air pollutants, safe 

foods and pesticides, water, and contaminated lands. 

A second laboratory, NERL, researches and develops improved methods 

and models to assess and predict human exposures to harmful 

contaminants in air, water, soil, and food. For example, it developed 

methods to better characterize pollution sources, and models to quantify 

the effects on exposure from various individual behaviors. NERL works 

closely with the program offices to set research priorities and help ensure 

that its results are useful to the program offices. For example, NERL 

coordinated with OAQPS to format data from several studies on airborne 

EPA Has Enhanced Its 

Strategic Planning and 

Refocused Staff Resources 

to Initiate Review of 

Existing Data and 

Development of Needed 

Data 

EPA Restructured and 

Augmented Its Research Office 

 27In September 2002, EPA formed the National Homeland Security Research Center to,

among other things, provide appropriate, affordable, effective, and validated technologies 

and methods for assessing risks posed by chemical, biological, and radiological terror

attacks. 

Page 32 GAO-06-595 Human Health Risk Assessment
particulate matter in a comparable manner that allowed NERL and OAQPS 

staff to develop more powerful analytic results than would have been 

possible from the individual studies alone. NERL also developed tools to 

enhance the exposure assessment portion of the risk assessment, such as 

a database of human activities by age, sex, and location, to better 

characterize exposure risk based on personal activities. The third 

laboratory, NRMRL, focuses its research more on ways to minimize 

exposure to contaminants that cause health risks than on ways to improve 

the preparation of risk assessments. For example, NRMRL researches 

sources of chemicals that disrupt the endocrine system and strategies to 

minimize exposure to such chemicals. 

The two centers, NCER and NCEA, serve as focal points for external 

researchers and risk assessors, respectively. NCER funds innovative 

environmental research by academic scientists to reduce uncertainty in 

risk assessment. Specifically, NCER has sponsored research to develop

data for use in models, thereby helping to reduce the model’s 

uncertainties. NCER also coordinates with EPA’s laboratories and 

program offices to develop its research topics. The Science to Achieve

Results (STAR) program, one of NCER’s grant programs, funds 

competitive research proposals and graduate fellowships in environmental 

science and engineering fields to complement ORD’s research in its 

strategic and research plans, such as the health effects of contaminants on 

airborne particulate matter and in drinking water, and, more generally, on 

children’s health. For example, the STAR program has funded research to

generate data on human exposures that will improve risk estimates. NCER 

has also established academic research centers in such areas as children’s 

health. NCEA, EPA’s national resource center for human health risk 

assessment, supports EPA’s work in three main ways. First, NCEA 

conducts risk assessments of national significance—for example, 

assessments of dioxin and diesel emissions—and prepares the air quality 

criteria documents that reflect the state of the science for six principal air 

pollutants. Second, it develops scientifically sound, defensible risk 

assessment methods to improve the use of science in risk assessment, 

such as software to estimate a benchmark dose—the dose that produces 

change in the risk of an adverse effect. Third, it provides guidance and 

support to risk assessors and risk managers through such means as its 

management of the Integrated Risk Information Systems (IRIS), a database 

of the potential human health effects of exposure to various chemicals in 

the environment. 

In addition to the six labs and centers, EPA in 2005 established the 

National Center for Computational Toxicology (NCCT) to coordinate and

Page 33 GAO-06-595 Human Health Risk Assessment
implement EPA’s research on computational toxicology, a cutting-edge 

field that uses mathematical models to predict adverse effects and to 

better understand the mechanisms through which a given contaminant 

causes harm. Given advances in such newly emerging disciplines as the 

study of genes and their functions, computational toxicology offers the 

potential for scientists to develop a more detailed understanding of the 

risks posed by a much larger number of chemicals than is currently 

possible. NCCT research is designed to develop tools to conduct 

quantitative risk assessments more rapidly and to improve the 

identification of chemicals that may pose substantial health risks. NCCT 

staff have begun to collaborate with other ORD laboratories and centers to 

effectively target their research efforts. 

Since 1994, EPA has undertaken a number of strategic activities to more 

closely link the data needs of program offices to research agendas of EPA 

and ORD.28 In connection with the goals presented in EPA’s and ORD’s 

strategic plans, ORD defined 16 high-priority research areas such as 

human health, endocrine disruption, airborne particulate matter and other

air pollutants, and safe pesticides. Research strategies for each area are

either in place or under development. For example, EPA’s Human Health

Research Strategy identifies four broad, overarching research areas to 

guide ORD’s human health research over the next 5 to 10 years.29

To carry out its 16 research strategies, ORD began in 2000 to develop a 

multiyear implementation plan for each. The first plans emerged in 2001,

with most finalized by 2003 following widespread participation and input

from many stakeholders as well as review by senior ORD managers. ORD 

invites input on the plans from ORD and program office staff, federal 

research partners, and outside peer groups, such as EPA’s Science 

Advisory Board, the Board of Scientific Counselors, and the National 

Research Council. These plans establish the short- and long-term goals and 

timelines required for ORD’s laboratories or centers to implement each of

the strategies. The National Program Director, a newly established 

EPA Has Enhanced Strategic 

Research Planning 

 28To comply with the Government Performance and Results Act (GPRA), EPA every 3 years

generates an agencywide 5-year strategic plan that highlights high-level environmental 

issues. The most recent plan for fiscal years 2003-2008 identified key research needs 

related to EPA’s mission. 

29The four strategic research directions are harmonizing cancer and noncancer risk

assessments, assessing aggregate and cumulative risk, determining risk to susceptible 

human subpopulations, and conducting research to enable evaluation of public health 

outcomes from risk management decisions. 

Page 34 GAO-06-595 Human Health Risk Assessment
position for each major ORD research area including human health, helps 

ensure that ORD’s time and staff resources are used strategically and that

the overall planning effort links to the needs of the program offices. 

Program offices use various planning and review approaches to determine 

data needs. For example, the Office of Air Quality Planning and Standards 

develops a research-needs paper at the conclusion of each periodic review

of the air quality standards it establishes to inform the research agenda for 

the next review, which occurs about every 5 years. Each paper helps 

ensure that research in key areas will be available for the next review. In 

contrast, the data needs of the Office of Pesticide Programs are defined by

law.30 Applicants who wish to register a pesticide product must submit the 

data defined in the statute and regulations, and OPP staff determine 

whether the data are of sufficient quality and quantity to assess the risk 

from the pesticide product.31 This list of required data is currently 

undergoing revision and additional data requirements may be added. 

Since 1994, EPA has strengthened and formalized collaboration with a 

range of other federal researchers to better leverage its limited research 

dollars and foster the development of data to improve human health risk 

assessments. Specifically, EPA has developed relationships with agencies 

such as the National Institute for Environmental Health Sciences (NIEHS) 

and the Agency for Toxic Substances and Disease Registry (ATSDR). For 

example, in 1998, EPA established a cooperative agreement with NIEHS to 

develop a body of research on the relationship between exposures and 

children’s health. This collaboration jointly funded Children’s 

Environmental Health Research Centers at seven U.S. universities and one

medical center to research children’s asthma and other respiratory 

diseases, as well as ways to reduce farm children’s exposure to pesticides.

In addition, EPA works closely with ATSDR to help fill research gaps and 

develop chemical-specific toxicological assessments used in risk 

assessments. In 2004, EPA and ATSDR entered into a formal agreement to 

ensure close collaboration to avoid duplicating efforts to fill data gaps. 

Under the agreement, the two agencies formed a work group to coordinate

their efforts to develop toxicological assessments for ATSDR’s work at 

EPA Program Offices Have 

Established Collaborative 

Relationships with External 

Researchers 

 30The Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), 7 U.S.C. §§ 136-136y. 

31Applicants must submit data on the acute and chronic toxicity of the pesticide product 

under different conditions, such as how and at what rate it can be applied. See 40 C.F.R. 

Part 158 (2005). 

Page 35 GAO-06-595 Human Health Risk Assessment
specific highly contaminated locations and for EPA’s Integrated Risk 

Information System (IRIS) database.

EPA, NIEHS, and ATSDR also jointly develop and annually review a list of 

approximately 275 hazardous substances commonly found at the nation’s 

highly contaminated sites and for which ATSDR will prepare toxicological 

assessments. At each annual review, agency staff may add chemicals to 

the list and identify priority research to fill gaps in knowledge. Of these 

275 chemicals, approximately 150 have been identified by EPA as highpriority needs. Each toxicological assessment contains almost everything

that is known about the chemical, including its potential to harm human 

health or the environment. A key difference between these toxicological 

assessments and the ones in EPA’s IRIS database is that ATSDR includes 

chronic cancer and noncancer effects, as well as acute effects, while IRIS 

generally includes only chronic cancer and noncancer effects. 

In addition, EPA has begun to establish collaborative relationships with 

scientific and industry-related researchers. For example, EPA has 

cooperative agreements with the International Life Sciences Institute’s 

Risk Science Institute (ILSI-RSI), an organization that researches critical 

scientific issues in risk assessment, such as the development of risk 

assessment methodologies.32 These cooperative agreements were 

specifically designed to engage the scientific community and bring 

together scientists from different affiliations (including academia, other 

parts of government, and the private sector including industry) to address 

risk assessment issues. Under one agreement, ILSI-RSI is to research risk 

assessment approaches for cumulative and aggregate exposures. In 

addition, EPA has used research provided by CIIT Centers for Health 

Research, a chemical research laboratory funded by EPA, industry, and 

other federal agencies, to provide information for its formaldehyde IRIS 

assessment. Furthermore, EPA and industry jointly fund the Health Effects 

Institute (HEI)—an organization that researches the health effects of 

various air pollutants, including airborne particulate matter and ozone. 

HEI has provided data for risk assessments and convened panels of 

experts to review and issue reports related to risk assessment, recently on

diesel exhaust. In its 2004 staff paper, EPA noted that it needs to continue 

to encourage development of the specific data necessary to more 

 32ILSI-RSI is primarily funded through cooperative agreements with regulatory agencies

such as EPA and Health Canada, but also receives financial support from the European

Commission, the National Institute of Environmental Health Sciences, and industry trade

groups. 

Page 36 GAO-06-595 Human Health Risk Assessment
accurately assess potential risks, particularly with researchers responsible 

for generating appropriate data, such as those seeking approval to 

manufacture or use a chemical. 

The Office of Pollution Prevention and Toxics has two programs to work 

with industry to develop data on contaminants that can be used to better 

understand risks. The first is the High Production Volume (HPV) Challenge 

Program. This program was officially launched in late 1998 to ensure that a 

baseline set of data would be made available to the public on

approximately 2,800 chemicals that are manufactured or imported in 

amounts greater than 1 million pounds per year. Diverse stakeholders, 

including the American Chemistry Council, Environmental Defense, and

the American Petroleum Institute participate in the program. The HPV 

Challenge Program provides an opportunity for all stakeholders, including 

the public, to comment on the tests and data summaries from the chemical 

sponsors—companies and consortia that volunteered to make publicly 

available screening-level data that allow EPA, industry, and other 

stakeholders to more effectively gauge the potential hazards of HPV 

chemicals. All comments are publicly available on the World Wide Web. As 

of January 2006, EPA had commitments from industry sponsors to provide 

data for 2,247 of the chemicals. The second program, the Voluntary 

Children’s Chemical Evaluation Program, is designed to provide data that 

will allow the public to better understand the potential health risks to 

children associated with certain chemical exposures. EPA asked 

companies that manufacture or import 23 chemicals that have been found

in human tissues in various biological monitoring programs to voluntarily 

sponsor the evaluation of specific chemicals in a pilot program. Thirty-five 

companies and 10 consortia volunteered to sponsor 20 chemicals. This 

program was developed only after considering comments and concerns 

from stakeholders. Of the 23 chemicals chosen for this pilot, data 

gathering has been completed for 9 and is under way for another 11. The 

remaining 3 chemicals in the pilot program have no sponsors. 

EPA maintains both an agencywide and several program-specific 

databases of information to help in the development of risk assessments. 

The primary database used by risk assessors in the program offices is the 

Integrated Risk Information System (IRIS), an electronic database of 

descriptive and quantitative information on human health effects that may 

result from exposure to various chemicals in the environment. Program 

office staff combine the IRIS data with exposure information they develop 

to characterize the health risks of a given chemical in a given situation. 

EPA Maintains Databases of 

Information Related to Risk 

Assessment 

Page 37 GAO-06-595 Human Health Risk Assessment
Since 1994, EPA has changed the IRIS assessment process in several ways. 

For example, each IRIS file now contains a discussion of the key studies, 

as well as a description of the decisions and default assumptions used in 

the assessment. EPA has also expanded the review that IRIS assessments 

undergo. For example, internal peer reviewers, including EPA senior 

health scientists representing program offices and regions, review the IRIS 

summary and accompanying detailed technical information. After this 

review, ORD releases the document for external peer review. EPA makes 

draft assessments available to the public at this time and, following peer 

review, the IRIS assessment discusses the key issues reviewers raised and 

EPA’s response. In addition, EPA has added a tracking system that allows 

IRIS users to readily determine where an individual assessment is in its 

development. 

In September 2003, EPA completed a congressionally requested review to 

assess the need to update information in IRIS, based on concerns that EPA 

and state regulators rely on potentially outdated scientific information. 

Input from EPA program and regional offices, the public, and other 

stakeholders indicated that EPA should, among other things, increase the 

number of new or updated assessments completed each year to 50. To

date, EPA has fallen considerably short of this goal. According to a 

program official, EPA completed 8 IRIS assessments in 2005, plans to 

complete 16 in 2006, and has approximately 75 assessments under way. 

EPA officials said a number of factors, such as the complexity of the 

assessment process, resource limitations, and extensive peer review, had 

limited EPA’s ability to complete more assessments last year. EPA has 

increased the number of staff working on IRIS assessments from 6 to 23 

and may ultimately increase the number to 29. The review also indicated

that EPA needs to assign staff to develop health assessments for IRIS, and 

provide funding for extramural research and contracts to develop IRIS 

files and subject them to external peer review. 

EPA also changed how it sets priorities for which chemicals need new or

updated IRIS assessments.33 Annually, EPA asks its program offices, 

regions, and the public to identify contaminants for which it should 

 33EPA uses four criteria to prioritize chemicals for IRIS reviews: (1) EPA statutory, 

regulatory, or program needs; (2) availability of new scientific information that might 

significantly change the current information; (3) interest from state and local government 

or the public; and (4) the amount of additional information that would be needed to

complete the review. Ultimately, the decision to assess any given substance hinges on 

available agency resources. 

Page 38 GAO-06-595 Human Health Risk Assessment
develop or revise IRIS assessments. EPA publishes the list in the Federal 

Register and requests the public and scientific community to submit any 

relevant data on substances undergoing review. EPA is currently 

reviewing ways to increase coordination with other governmental 

agencies that develop chemical assessments, outreach to stakeholders 

earlier in the development of IRIS assessments, and consultation with 

independent external reviewers. In 2004, the IRIS program also initiated a 

review of available scientific literature for the 460 chemicals in the 

database that are not under active reassessment to determine whether a 

reassessment based on new literature could significantly change existing

toxicity information. For 63 percent of the chemicals reviewed, no major 

new health effects studies were found. Such literature reviews will be 

conducted annually and the findings noted in the IRIS database. 

In addition, some program offices maintain databases to enhance their risk 

assessments. For example, the Office of Air Quality Planning and 

Standards (OAQPS) maintains a database of dose-response values 

developed by various sources, including IRIS, ATSDR, and the California 

Environmental Protection Agency, as an aide for its risk assessors. OAQPS 

staff update this database as better data become available. As part of its 

National Air Toxics Assessment—an ongoing comprehensive evaluation of 

hazardous air pollutants in the United States—EPA assessed 32 air 

pollutants plus particulate matter in diesel exhaust in 1996. The national

assessment is designed to identify air pollutants with the greatest potential 

to harm human health, and the results will help set priorities for collecting 

additional data. As part of its assessment, EPA compiled a national 

emissions inventory of hazardous air pollutants from outdoor sources, 

estimated population exposures to the pollutants, and characterized the 

potential cancer and noncancer health risks from breathing the pollutants. 

ORD also maintains personal monitoring data on the chemicals in the air, 

foods and beverages, water, and dust in an individual’s personal indoor 

and outdoor environments. For example, in its National Human Exposure 

Assessment Survey (NHEXAS) program, which was completed in 1998,

ORD collected human exposure data from hundreds of subjects from 

several areas of the country. NHEXAS provided data on background levels 

of total exposure to environmental contaminants that can be used as a 

baseline in exposure and risk assessments to estimate whether specific 

populations are exposed to increased levels of environmental 

contaminants. 

Page 39 GAO-06-595 Human Health Risk Assessment
EPA has explicitly stated the need for risk assessors to identify when they 

relied on a default option, why they chose it, and when they departed from

using a standard default option, but the agency acknowledges more could 

be done. To carry out its mission to protect human health, EPA’s risk 

assessment procedures, including its default options, are protective of 

human health. In three recent guidance documents—the 2004 staff paper,

the 2005 cancer guidelines, and the risk characterization handbook—EPA 

advocated more transparency in the choice of default options. These 

documents summarize a significant change in EPA’s approach. 

Specifically, EPA first critically examines all relevant and available data to 

assess health risks, then uses the default options only in the absence of 

adequate contaminant-specific data. EPA also states in its staff paper that

it bases its default assumptions on peer reviewed published studies, 

empirical observations, extrapolation from related observations, and 

scientific theory. Moreover, the cancer guidelines include an appendix that 

defines the basis for each of the default options that may be used in a 

cancer risk assessment. The Risk Characterization Handbook notes that 

risk assessments should describe the full range of default options that 

were used, including ones to address uncertainty. Moreover, the handbook 

states that when defaults are used, the risk assessment should reference 

the relevant EPA guidance that explains them. 

EPA program offices also advocate greater transparency when using 

default options. Specifically, the majority of IRIS assessments completed 

since 1997 describe the defaults used in the analysis and any departures 

from those defaults. The Office of Air Quality Planning and Standard’s Air 

Toxics Risk Assessment Reference Library contains a number of 

references to defaults that should be used in the course of preparing a 

human health risk assessment.34 For example, to estimate an individual’s 

exposure to an air pollutant, the guidance presents the default option to 

use for exposure in a screening-level assessment, namely that the 

individual remains at a single location and continuously breathes polluted

air. 

Despite the increased focus on more transparency in the use of defaults, 

EPA acknowledges it could more consistently describe how the default 

was developed and explain why it is a reasonable assumption. In its staff 

EPA Has Improved Its 

Choice and 

Communication of Default 

Options 

 34The Air Toxics Risk Assessment Reference Library is a multivolume technical resource

manual that provides information on the fundamental principles of risk-based assessment

for hazardous air pollutants and how to apply those principles in different settings. 

Page 40 GAO-06-595 Human Health Risk Assessment
paper, EPA acknowledges it needs to ensure that the defaults are 

supported by the best available data and should look for opportunities to 

increase certainty and confidence in the defaults and extrapolations used.

EPA also acknowledges it may need to re-examine older risk assessments 

that relied on defaults that can now be replaced with relevant data. To a 

large degree, the use of defaults is intertwined with EPA’s ability to get the 

data it needs. As was discussed previously, EPA has targeted research, 

both within EPA and through its grant programs, to understand variability 

and uncertainty in the data derived from studies of laboratory animals, and

this research may further reduce EPA’s need to rely on default options. 

Since 1994, EPA has taken several steps to enhance its modeling 

capabilities and embrace new methodologies for risk assessment through 

improved guidance and workshops. To help improve its models, EPA’s 

Agency Task Force on Environmental Regulatory Modeling published a 

report that concluded a need existed for, among other things, training and

technical support, agency guidance on external peer review of

environmental regulatory modeling, and creation of a Committee on 

Regulatory Environmental Modeling.

35 Also in 1994, EPA’s Risk 

Assessment Forum developed a draft protocol to evaluate models for 

exposure assessments.36 In 1997, ORD and program offices conducted an

agencywide conference, called the Models 2000 Workshop, to facilitate 

adherence to existing guidance on modeling, to define and implement 

improvements in how the agency developed and used models, and to 

recommend an implementation plan for improving modeling within the 

agency. 

EPA followed up these activities in 2000 by creating the Committee on 

Regulatory Environmental Modeling (CREM) to promote consistency and 

consensus within the agency on modeling issues (including modeling 

guidance, development, and application) and to enhance internal and 

external communications on modeling activities. CREM supports and 

enhances the existing modeling activities in the program offices and 

provides EPA with tools to support environmental decision making. CREM 

EPA Has Taken Steps to 

Enhance Its Modeling 

Capabilities and Embrace 

New Methodologies 

 35U.S. Environmental Protection Agency, Report of the Agency Task Force on

Environmental Regulatory Modeling—Guidance, Support Needs, Draft Criteria and

Charter (Washington, D.C., 1994). 

36U.S. Environmental Protection Agency, Model Validation for Predictive Exposure 

Assessments (Washington, D.C., 1994). 

Page 41 GAO-06-595 Human Health Risk Assessment
also provides the public and EPA staff with a central point of inquiry about 

EPA’s use of models. In 2000, CREM launched agencywide activities 

designed to enhance the development, use, and selection of regulatory 

environmental models at EPA. One such activity—a workshop to facilitate 

discussion of good modeling practices—resulted in the development of 

modeling guidance. 

In 2003, CREM developed guidance and created a database—called the 

Models Knowledge Base—of the models most frequently used in EPA.37

The guidance recommends best practices to help determine when a model, 

despite its uncertainties, can be appropriately used to inform a decision.

Specifically, it recommends that model developers and users subject their 

model to credible, objective peer review, assess the quality of the data they

use as inputs, and perform sensitivity and uncertainty analysis to 

determine which of the model inputs has the greatest impact on the 

modeled results. 

EPA has also incorporated efforts to improve models in its research 

strategies and implementation plans. For example, in its plan for research 

on hazardous air pollutants, EPA established a long-term goal to reduce 

uncertainties in risk assessments through methods, data, and models of 

acute and chronic exposures and exposures through multiple pathways at

both the national and regional levels. In addition, one of ORD’s 

laboratories established an exposure modeling research branch and 

develops population exposure models, such as the Stochastic Human 

Exposure and Dose Simulation model for inhalation and exposures of 

general and sensitive subpopulations through multiple pathways. EPA has 

also begun to use geographic information systems (GIS) to present risk 

information spatially. For example, a GIS system is being developed that

maps all of the drinking water intakes in the United States and their 

associated watersheds, so that the agency can better assess risks to 

drinking water supplies stemming from activities in the related watershed. 

For risk assessments of hazardous air pollutants, GIS can display and

analyze data during planning, scoping, and problem formulation, during 

the exposure assessment, and during the characterization of risks. GIS can 

also help communicate information to risk managers and other 

stakeholders. 

 37U.S. Environmental Protection Agency, Draft Guidance on the Development, Evaluation,

and Application of Regulatory Environmental Models (Washington, D.C., 2003). 

Page 42 GAO-06-595 Human Health Risk Assessment
In addition to models, EPA is beginning to embrace such new risk 

assessment methodologies as probabilistic risk assessment and mode of 

action analysis. Probabilistic risk assessment characterizes the variability 

or uncertainty in risk estimates as the range or distribution of the number 

of times each possible outcome will occur. In probabilistic risk 

assessment, one or more variables in the risk equation, such as the 

exposure rate, is defined as a distribution rather than as a single number. 

A primary advantage of probabilistic risk assessment is that it provides a 

quantitative description of the degree of variability or uncertainty. EPA’s 

1997 policy states that probabilistic techniques, such as Monte Carlo 

analysis, can be viable statistical tools to analyze variability in risk 

assessments, when they are based on adequate supporting data and 

credible assumptions.38 The guidance presents a general framework and 

broad set of principles to ensure the use of good scientific practices when 

conducting probabilistic analyses of variability and uncertainty. EPA 

currently uses a number of models that include probabilistic analyses and 

is developing a new modeling framework, known as the Multimedia 

Integrated Modeling System, that will further enhance the agency’s ability 

to probabilistically model uncertainty. 

EPA’s recently revised cancer guidelines advocate the use of a mode of 

action analysis—based on the sequence of biological events that must 

occur to produce a harmful effect—to improve the accuracy of risk 

assessments. As a general rule, EPA assumes that toxic responses 

observed in laboratory animals indicate that the same responses are likely 

to occur in people even though differences in such areas as metabolic 

rates can result in different sensitivities between laboratory animals and 

humans. Mode of action analysis will more clearly indicate whether a 

difference exists between animals and humans in their response to 

contaminants. In addition, the guidelines present a new cancer 

characterization system consisting of five summary descriptors, to be used 

in conjunction with narrative, to describe the extent to which available 

data support the conclusion that a contaminant causes cancer in humans 

and to justify the summary descriptor selected. For noncancer risk 

assessments, EPA has used, and continues to refine, the benchmark dose 

methodology, which identifies the dose or concentration of a contaminant 

that slightly increases the likelihood of an adverse effect. 

 38U.S. Environmental Protection Agency, Policy for Use of Probabilistic Analysis in Risk

Assessment (Washington, D.C., 2003). 

Page 43 GAO-06-595 Human Health Risk Assessment
Uncertainty is inherent in all phases of risk assessment, from hazard 

identification through risk characterization. Over the years, EPA has relied 

more on qualitative, or descriptive, characterizations of uncertainty and

less on quantitative, or numeric, characterizations. EPA’s practice now is 

to use a tiered approach to analyze uncertainty. That is, EPA starts with a 

simple description of uncertainty and sequentially employs more 

sophisticated quantitative analysis, such as sensitivity analysis, provided 

the additional analysis reduces the uncertainty. To characterize risk 

quantitatively, EPA has typically used approaches that produce a single

number to characterize the risk in terms of the level of a contaminant that 

does not cause harm, as opposed to presenting a range of possible values.

Although EPA is beginning to use probabilistic approaches in exposure 

assessments, and has done so for six principal air pollutants, it does not 

typically do so to analyze uncertainty in its dose-response analyses, though 

its Science Advisory Board encouraged development of such approaches

in 2000. EPA acknowledged in its 2004 staff paper that probabilistic risk 

assessment could be used more frequently and could provide useful 

information beyond screening-level assessments. However, the staff paper

said probabilistic analysis may not be appropriate in all situations and the 

accuracy of the analysis will depend largely on the availability and quality 

of the data used in the analysis. 

Another quantitative approach to uncertainty is the use of uncertainty 

factors to account for such unknowns as variation in sensitivity among 

members of the human population or the appropriateness of extrapolating 

animal data to humans. EPA routinely uses uncertainty factors when it 

estimates the daily exposure to the human population that is likely to be

without appreciable risk of adverse effects during a lifetime. This daily 

exposure estimate is called a reference dose for contaminants that are 

consumed and a reference concentration for inhaled contaminants. EPA 

states in its 2004 staff paper that it applies uncertainty factors in health 

assessments based on available data and the scientific judgment of EPA 

risk assessors and peer reviewers. According to EPA, most IRIS 

toxicological assessments, which contain a chemical’s reference dose and 

reference concentration, provide justifications for the uncertainty factors 

applied to a particular chemical. Moreover, the factors undergo rigorous 

internal, and independent, external scientific peer review before being 

used in IRIS assessments.

EPA has issued a number of documents that delineate the need to clearly

and consistently characterize uncertainty in risk assessments. In 1995, 

EPA issued a risk characterization policy that stated that the risk 

EPA Is Introducing More 

Quantitative 

Characterization of 

Uncertainty into Its Risk 

Assessments 

Page 44 GAO-06-595 Human Health Risk Assessment
assessment should fully, openly, and clearly characterize risks, and should 

disclose the scientific analyses, uncertainties, assumptions, and policies 

underlying the decisions. This policy was followed in December 2000 by 

the Risk Characterization Handbook, which includes guidance on how to

address, among other things, uncertainty in risk assessment and describes 

the need and methods to present the sources and magnitude of uncertainty 

to the risk manager. More recently, the 2005 revised cancer guidelines 

discuss each of the major uncertainties, such as model uncertainty or 

uncertainty related to human variation, and stress that assessments should 

discuss the significant uncertainties encountered in the analysis. For 

example, the guidance calls for the assessments of hazard, dose-response, 

and exposure to have accompanying technical characterizations covering 

the strengths and limitations of data and a discussion of uncertainty. 

EPA has made progress in describing variability due to differences in both 

the exposure individuals receive and the susceptibility of exposed 

individuals to adverse effects. A key document EPA risk assessors use to 

account for variation in exposure is its Exposure Factors Handbook, 

originally issued in 1989 and revised in 1997. The handbook summarizes 

data on human behaviors and characteristics that influence exposure to 

environmental contaminants and recommends values to account for those 

factors in assessing risk. Specifically, the handbook contains a series of 

over 150 data tables that provide information on how much time 

individuals spend at various activities and in various environments. 

Assessors can use these data to develop exposure duration estimates for 

exposure scenarios. For example, the tables contain statistics—broken 

down by age, gender, race, education, and some medical conditions, such 

as asthma or emphysema—for time spent in various outdoor locations. 

The handbook also provides general guidance to risk assessors on the 

types of variability relevant to a risk assessment and ways variability can 

be analyzed and addressed. 

All program offices address exposure variability in their risk assessments, 

although they do so in different ways. For example, risk assessors in the 

Office of Air Quality Planning and Standards who set certain air quality 

standards for six principal pollutants said they consider individual activity 

patterns for sensitive populations like children or asthmatics in exposure 

modeling by including a distribution of breathing rates to reflect variability 

inherent in the population. Furthermore, by modeling to protect the most 

sensitive or at-risks groups, they are assured of protecting the rest of the 

population. Variability in exposure to the six principal pollutants is 

generally described qualitatively in scientific summaries for each 

EPA Better Analyzes and 

Communicates Variability 

in both Exposure and 

Susceptibility 

Page 45 GAO-06-595 Human Health Risk Assessment
pollutant. The Office of Water includes an analysis of risks to various 

subpopulations and a narrative discussion of the strengths and 

weaknesses of the studies it used to estimate exposure, but generally does 

not include a quantitative analysis. The Office of Pesticide Programs 

considers 24 different population subgroups in its exposure estimates, 

including differences in age, gender, ethnicity, and geographic dispersion. 

When data allow, the Office of Pesticide Programs develops a distribution 

of exposures and risks for its more refined risk assessments. 

To further its understanding of variability in exposure, EPA has 

undertaken a number of research projects. For example, one of ORD’s 

laboratories conducted the National Human Activity Pattern Survey to 

provide detailed human exposure information for specific populations and

allow EPA to better understand actual human exposure to pollutants in 

real-world situations. The survey results are stored in the Consolidated 

Human Activity Database to help risk assessors estimate the time that 

exposed people spend in various environments and their inhalation, 

ingestion, and dermal absorption rates while in those environments. This 

laboratory also conducts research to define, quantify, and reduce the 

uncertainty associated with the exposure and risk assessments, to develop 

improved methods to more accurately measure exposure and dose, and to

develop technical information and quantitative tools to predict the nature

and magnitude of human exposures to environmental contaminants. A 

recent EPA study was designed to identify chemicals commonly used in 

homes or day care centers, and whether children in these environments 

encountered the chemicals in the course of their daily activities.39 The 

research sought to identify the major routes (i.e., breathing and ingestion) 

and sources (i.e., dust, food, air, soil, and water) through which children 

come into contact with chemicals. 

Variability also exists with regard to susceptibility to adverse affects 

because of inherent differences among humans. EPA most recently 

addressed variability in susceptibility in the 2005 revision of its cancer 

guidelines, which describe the importance of separate risk assessments for 

all potentially sensitive life stages, including adults and children. The 

supplementary cancer guidelines for children address issues pertaining to

cancer risks associated with early-life exposures. Legislation can also 

require EPA to consider potentially susceptible populations and life 

 39U.S. Environmental Protection Agency, Children’s Total Exposure to Persistent

Pesticides and Other Persistent Organic Pollutants (Washington, D.C., 1999). 

Page 46 GAO-06-595 Human Health Risk Assessment
stages. For example, the Safe Drinking Water Act Amendments mandate 

that EPA consider risks to groups within the general population that are at 

greater risk of adverse health effects, including children, the elderly, and 

people with serious illnesses. In addition, the Food Quality Protection Act 

contains special provisions for the consideration of risks to children from 

pesticides. In 1995, EPA’s Science Policy Council called for EPA to 

consider the risks to infants and children consistently and explicitly as 

part of its risk assessments. In 1997, the White House issued an executive 

order that required EPA and other federal agencies to identify and assess 

environmental health and safety risks that may disproportionately affect 

children and to ensure that policies, programs, activities, and standards 

address such disproportionate risks.40

In its 2004 staff paper, EPA acknowledges that characterizing variability 

for susceptible populations and life stages is an area where it could 

improve, although the absence of data limits its efforts. In 2002, EPA 

pointed out that data are limited to identify susceptible populations and 

life stages for many contaminants.41 In these situations, EPA typically 

relies on default options, such as the use of uncertainty factors to account

for variations in susceptibility. Many of the exposure assumptions and 

default values used to assess plausible current and future exposure 

scenarios can be found in EPA’s 1997 Exposure Factors Handbook, and 

recent updates to the handbook are available online. 

Another way EPA addresses variability is through research. One of ORD’s 

four strategic research directions in its Human Health Research Strategy

is designed to improve the understanding of why some people and groups 

are more susceptible and highly exposed than others. According to this 

strategy, ORD’s research on subpopulations will focus on three factors—

life stage, genetic factors, and pre-existing diseases—that have been 

identified by a program office and the scientific community as having a 

high priority for risk assessment. In 2000, ORD released its Strategy for 

Research on Environmental Risks to Children to strengthen the scientific

foundation of risk assessment and management decisions that affect

children and guide EPA’s research needs and priorities over the following 

5 to 10 years. Approximately 75 percent of the funding for this strategy will 

 40Exec. Order No. 13045, 62 Fed. Reg. 19885 (Apr. 21, 1997). 

41U.S. Environmental Protection Agency, Review of the Reference Dose and Reference 

Concentration Processes (Washington, D.C., 2002). 

Page 47 GAO-06-595 Human Health Risk Assessment
be dedicated to research grants under the STAR program, such as those 

designed to evaluate children’s exposure to pesticides. 

To help risk assessors analyze the health effects of exposure to multiple 

contaminants (cumulative exposure) and through multiple routes 

(aggregate exposure), EPA has issued guidance, developed methods and 

models, and supported research. In 1997, EPA’s Science Policy Council 

issued guidance on cumulative risk assessment. This guidance directs each 

office to consider cumulative risk in planning major risk assessments and, 

where relevant data are available, to broaden the scope of the assessment 

to integrate multiple sources, effects, pathways, stressors, and populations

for cumulative risk analyses. The guidance also highlights the need to 

ensure that the public and other stakeholders have an opportunity to help

define the way EPA assesses an environmental or public health problem 

and calls for ongoing communication and coordination among EPA’s risk 

assessors, risk managers, economists, engineers, and other technical 

experts. 

In 2000, EPA updated its 1986 guidance on chemical mixtures, to generate 

a consistent agencywide approach to assess health risks from exposures 

to multiple chemicals.42 The guidance is organized according to the type of 

data available to risk assessors, ranging from data-rich to data-poor 

situations, to help risk assessors select an appropriate methodology. For

example, if data are of poor quality or quantitative data are very limited on 

chemical mixtures, the risk assessor may choose to perform a qualitative 

analysis of the potential human health impacts from exposure to the 

mixture. The guidance also contains procedures to develop toxicity 

equivalency factors, based on the toxicity of components of the mixture, 

to assess the risk from mixtures in the absence of data on the specific 

mixture. 

In 2003, EPA’s Risk Assessment Forum developed a simple, flexible 

framework to help risk assessors consistently conduct and evaluate 

cumulative risk assessments.43 The framework is conceptually similar to 

the one used in human health assessments of a single contaminant in that 

EPA Has Begun to 

Consider the Combined 

Effects of Exposure to 

Multiple Contaminants 

through Multiple Pathways 

 42U.S. Environmental Protection Agency, Supplementary Guidance for Conducting Health

Risk Assessment of Chemical Mixtures (Washington, D.C., 2000). 

43U.S. Environmental Protection Agency, Framework for Cumulative Risk Assessment

(Washington, D.C., 2003). 

Page 48 GAO-06-595 Human Health Risk Assessment
it follows a three-stage approach of (1) planning, scoping, and problem 

formulation; (2) analysis; and (3) risk characterization. In addition, the 

document also highlights needed areas of research and methods 

development that may be important to the evaluation of cumulative risks, 

such as understanding how the sequence and timing of exposure may 

influence the ultimate risk for effects. 

EPA risk assessors use a number of models and methodologies to analyze 

multiple routes of exposure. For example, the branch of the Office of Air 

Quality Planning and Standards that regulates hazardous air pollutants 

employs the Multiple Pathways of Exposure model to assess and predict 

the movement and behavior of chemicals in the environment. For 

example, the Multiple Pathways of Exposure model includes procedures 

to estimate human exposures and health risks that result from the transfer 

of pollutants from the air to soil and surface water bodies and the 

subsequent uptake of the pollutant by plants, animals, and humans. The 

model specifically addresses exposures from breathing; consuming food, 

water, and soil; and contact with skin. More recently, EPA developed the 

Total Risk Integrated Methodology (TRIM) and created the TRIM Fate, 

Transport, and Ecological Exposure model that describes the movement 

of air pollutants emitted from any type of stationary source as well as their 

transformation over time in water, air and soil. 

Some program offices have also taken steps to explicitly consider the risks 

associated with more than one route of exposure or more than one 

chemical. Specifically, the Office of Pesticide Programs issued guidance in

2001 and 2002 in response to statutory requirements to assess the risk of 

aggregate exposure—exposure to a single chemical by multiple pathways 

and routes.44 The first set of guidance focuses on how to assess aggregate 

risk in those cases where more extensive data and more sophisticated 

exposure assessment methods and tools are available; this guidance also 

emphasizes, when data are available, the use of distributional data—

aggregate exposures of many individuals in the population of interest—for 

all pathways of exposure.45 This approach allows the risk assessor to more 

fully evaluate exposure and resulting risk across the entire population, 

rather than the exposure of a single highly exposed individual. The second 

 44See § 405 of the Food Quality Protection Act of 1996, amending 21 U.S.C. § 

346a(b)(2)(D)(vi). 

45U.S. Environmental Protection Agency, General Principles for Performing Aggregate 

Exposure and Risk Assessments (Washington, D.C., 2001).

Page 49 GAO-06-595 Human Health Risk Assessment
set of guidance describes a framework to assess potential human health 

risks from all pathways of exposure to multiple pesticides that share a 

common mechanism of toxicity—that is, the pesticides produce a similar 

toxic effect on the same organ or tissue.46

In addition, the branch of the Office of Air Quality Planning and Standards 

that regulates hazardous air pollutants developed the Integrated Urban Air 

Toxics Strategy, which is used to consider cumulative risks presented by 

exposures to hazardous air pollutants emitted from various sources. Staff 

can assess risk at both a national and an urban or a neighborhood scale. 

Furthermore, the hazardous air pollutant office developed guidance on 

multipathway risk assessments that are particularly important for 

hazardous air pollutants, such as mercury and dioxins, because human 

exposure occurs both from breathing air containing the toxins and from 

consuming plants, water, and soil where the pollutants were deposited. 

Moreover, these pollutants persist in the environment for long periods of

time and may also accumulate in the tissues of commonly consumed 

plants and animals to levels that are harmful to humans.47 The guidance, 

maintained in the Air Toxics Risk Assessment Reference Library, 

describes how to plan, scope, and formulate the problem, conduct the 

analysis, and characterize the risk for such cases. 

The extent to which program offices assess the effects of cumulative and 

aggregate exposures is related to the regulatory responsibilities of each

office and by the availability of data. For example, the hazardous air 

pollutant office routinely analyzes a mix of chemicals from various 

emitting sources, such as petroleum refineries, to regulate hazardous air 

pollutants.48 Similarly, as mentioned above, the Office of Pesticide 

Programs is required to consider exposure to pesticides from various 

pathways, such as food, drinking water, and residential uses, and various 

routes, such as eating, breathing, and contact with skin. In contrast, the 

Toxic Substances Control Act does not require the Office of Pollution 

 46U.S. Environmental Protection Agency, Guidance on Cumulative Risk Assessment of 

Pesticide Chemicals That Have a Common Mechanism of Toxicity (Washington, D.C.,

2002). 

47For example, metals released into the air may be deposited on the ground, where they

remain in surface soils for long periods of time. The chemicals in the soil may be taken up 

into plants through the roots and accumulate in the tissues of foraging animals. 

48By statute, hazardous air pollutants are regulated not as individual pollutants but by 

emission sources that consist of a group of similar industrial processes or industries that

release multiple pollutants.

Page 50 GAO-06-595 Human Health Risk Assessment
Prevention and Toxics to assess the risks of a new chemical that may 

occur through its interaction with other chemicals. The office also 

assesses the risks of existing chemicals but cannot conduct cumulative 

risk assessment for classes of chemical that share a common mode of 

action because no data exist. Program managers hope such data will 

become available in the future. 

In its 2004 staff paper, EPA commented that while it has increased its 

emphasis on evaluating cumulative risks, it needs to expand on

approaches to do so, and it needs to produce a rigorous scientific base to 

support such evaluations. To that end, one of ORD’s four strategic 

research directions in its Human Health Research Strategy is to improve 

assessments of aggregate and cumulative risks. Specific research 

objectives are to develop exposure models and methods, provide a 

scientific basis to predict interactive effects of contaminants in mixtures, 

and determine the most appropriate approaches to combine effects and 

risks from mixtures. 

EPA risk assessors responding to our survey reported that some 

modifications to its risk assessment processes, such as new or updated 

EPA guidance issued over the last 10 years, have been helpful. They also 

said that although collaboration among internal and external researchers 

has improved, problems remain with communication and coordination. 

Finally, risk assessors said that the training they’ve taken in the last 5 

years has been beneficial, but they need additional training on analytic

tools, such as modeling, and on other scientific disciplines related to risk 

assessment. 

EPA Risk Assessors 

Responding to Our 

Survey Reported That 

Process Modifications 

Have Helped Them 

Prepare Better Risk 

Assessments but That 

Collaboration and 

Training Limitations 

Hamper Further 

Progress 

Page 51 GAO-06-595 Human Health Risk Assessment
At least two-thirds of risk assessors responding to our survey who 

reported using guidelines or reference documents indicated that these 

documents were moderately to very helpful in preparing risk 

assessments.

49 In addition, between one-third and two-thirds of 

respondents who reported using policy documents said these documents 

were moderately to very helpful in preparing risk assessments. More 

specifically, many risk assessors said agencywide guidelines and reference

documents provide a framework to assess risks to human health that help 

make risk assessments more consistent. For example, some risk assessors 

noted the usefulness of agency reviewed or approved procedures to 

support their assessments. In addition, some risk assessors said the 

guidelines and reference documents helped clarify issues, and several 

assessors said they were a good source for data needed to conduct 

assessments. Risk assessors responding to our survey cited the Guidelines

for Carcinogen Risk Assessment as the document most frequently used 

when preparing human health risk assessments. More specifically, several

risk assessors noted that the carcinogen guidelines provide a useful 

framework for preparing risk assessments. Many risk assessors 

commented that agencywide guidelines and reference documents are 

helpful or provide useful examples. For example, a few risk assessors 

stated that the Exposure Factors Handbook helps provide consistency 

among EPA offices that conduct exposure assessments because it defines 

standard values for exposure, and the rationale behind those values. 

Another assessor said that the Review of the Reference Dose and 

Reference Concentration Processes provides comprehensive guidance on

setting reference values and contains a case study that serves as a model 

for concise and well-written hazard identification. Although risk assessors 

responding to our survey reported that guidance documents are generally

helpful, many expressed concerns about them. For example, some risk 

assessors consider the documents too general or too difficult to decipher.

EPA Has Issued Numerous 

Guidance Documents That 

Have Been Generally 

Helpful to Risk Assessors 

 49Guidelines refer to the Guidelines for Carcinogen Risk Assessment, Guidelines for 

Neurotoxicity Risk Assessment, Guidelines for Reproductive Toxicity Risk Assessment, 

and Supplemental Guidelines for Chemical Mixtures. Reference documents refer to the

Assessment Factors Handbook, Exposure Factors Handbook, Framework for Cumulative 

Risk Assessment, Guiding Principles for Monte Carlo Analysis, Peer Review Handbook, 

Review of Reference Dose and Reference Concentration Processes, Risk Assessment 

Principles and Practices, and Risk Characterization Handbook. Policy documents refer to 

the Policy on Evaluating Health Risks to Children, Policy for Use of Probabilistic 

Analysis in Risk Assessment, Interim Genomics Policy, and Interim Position on 

Environmental Endocrine Disruption. 

Page 52 GAO-06-595 Human Health Risk Assessment
In addition, 82 percent of the risk assessors whose offices have officespecific guidance said that the guidance is very or moderately helpful with

regard to preparing risk assessments. (See fig. 2.) According to many risk 

assessors, office-specific guidance provides information in a format 

relevant to each office’s specific needs. For example, the Office of 

Pesticide Programs periodically issues “hot sheets” that describe how to 

apply general guidance to pesticide product risk assessments. In addition, 

the Office of Air and Radiation created the Air Toxics Risk Assessment 

Reference Library that provides information on how to analyze the risks 

from hazardous air pollutants. Over 65 percent of risk assessors reported 

that EPA and program offices were moderately to very effective at 

disseminating guidance. 

Figure 2: Helpfulness of Office-Specific Guidance 

Note: Percentages do not total to 100 because of some risk assessors did not respond to this 

question. 

0

10

20

30

40

50

60

70

80

90

100

Helpfulness of guidance

 My office does

 not produce

 or use guidance

Not at all

 helpful

Slightly helpful

Moderately helpful

Very helpful

Percentage of risk assessors

Source: GAO analysis of survey data.

Page 53 GAO-06-595 Human Health Risk Assessment
To prepare risk assessments, EPA relies on external peer review and 

scientific data from a variety of sources on the potential health risks from

exposure to contaminants. Collectively, this collaboration has improved 

EPA’s ability to conduct those assessments. For example, 35 percent of 

risk assessors responding to our survey reported that external peer 

review, which is often conducted by independent researchers, has 

definitely helped improve the overall quality of EPA’s risk assessments, 

with an additional 48 percent noting that it has probably helped improve 

the overall quality. Furthermore, at least 32 percent noted that peer review

has definitely helped ensure that the science used in risk assessments is 

appropriately characterized, helped advance the use of new scientific 

methods or models, and helped EPA produce risk assessments that are 

more defensible. Another 38 percent said that peer review has probably 

helped with these improvements. In addition, responding EPA risk 

assessors rely primarily on other federal research entities, academia, and 

industry, and to a slightly lesser extent, ORD’s research laboratories, to 

meet their offices’ research needs. (See fig. 3.) 

Collaboration Has 

Improved EPA’s Ability to 

Conduct Risk Assessment, 

but Problems Remain 

Page 54 GAO-06-595 Human Health Risk Assessment
Figure 3: Risk Assessors That Generally or Strongly Agree That Organizations Help 

Fill Data Needs 

More specifically, 63 percent of risk assessors generally or strongly agreed

that they relied on other federal research entities, such as the National 

Toxicology Program; 50 the National Institute for Environmental Health 

Sciences (NIEHS) and the National Cancer Institute—both within the 

National Institutes of Health; and the Agency for Toxic Substances and

Disease Registry (ATSDR) to help fill their offices’ needs for scientific 

data. EPA has established formal collaborative agreements with both 

NIEHS and ATSDR to research children’s health and to develop 

toxicological data useful to both agencies, respectively. In addition, EPA 

0

10

20

30

40

50

60

70

80

90

100

 

The STAR and center

grant programs

Other governmental

entities

ORD’s research labs

Industry

Academia

Other federal 

research entities

Percentage of risk assessors

Organizations

Source: GAO analysis of survey data.

 50The National Toxicology Program is an interagency program established by the 

Department of Health and Human Services that provides information about potentially 

toxic chemicals to health, regulatory, and research agencies, scientific and medical 

communities, and the public. 

Page 55 GAO-06-595 Human Health Risk Assessment
has also collaborated with the U.S. Geological Service to identify 

contaminants in ground and surface waters. 

In addition to federal research entities, over 57 percent of respondents 

generally or strongly agreed that their offices rely on research from 

academia and industry to meet their research needs. For example, EPA 

has formal agreements with the International Life Sciences Institute’s Risk 

Science Institute and the Health Effects Institute to develop research on 

approaches to analyze cumulative and aggregate exposures and the health 

effects of various air pollutants, respectively. EPA also relies heavily on 

industry-generated research on specific chemical substances. For 

example, under the laws that govern registration of pesticide products and 

new chemicals, applicants must supply specific data for relevant EPA 

offices to review when deciding whether to approve the pesticide products

or chemicals in question. 

Furthermore, 46 percent of risk assessors said they relied on ORD’s 

laboratories to generate research that helps fill scientific data needs. For 

example, some risk assessors said scientists from ORD’s laboratories 

provide useful technical guidance on scientific issues or the risk 

assessment process. Another risk assessor commented that ORD’s 

expertise is very useful to help interpret unusual findings or to advise on 

emerging issues. One risk assessor stated that ORD helped develop a

specific model to use in probabilistic risk assessments. Other types of 

collaboration with ORD’s laboratories include help to develop models for 

assessing dose and response relationships, to interpret toxicity data, to

conduct epidemiological studies, and to develop scientific summaries for 

risk assessments of priority air pollutants. 

Overall, of the risk assessors who said they often or always collaborate 

with other EPA offices, at least 46 percent said that the collaboration was 

very effective. A few risk assessors commented that collaboration has 

become more effective in recent years, in part because staff in the various

offices have more contact with one other, established scientist-to-scientist 

relationships, or learned whom to contact to address a particular question. 

For example, one risk assessor pointed out that cross-agency workgroups 

help to facilitate agencywide collaboration. Furthermore, a few risk 

assessors commented that the program offices’ knowledge of the 

regulatory context in which research will be used helps ORD’s researchers 

structure their work. For example, NERL collaborated with the Office of

Air to fund studies that coordinated the format of data produced by 

different researchers to enhance the consistency of research approaches, 

which created more powerful results and made the data more useful to 

Page 56 GAO-06-595 Human Health Risk Assessment
program offices. Finally, because some chemicals may be assessed by 

more than one office, collaboration across program offices helps ensure 

the consistency of risk assessments across EPA. For example, the Office 

of Pesticide Programs and the Office of Water may both prepare risk 

assessments for certain contaminants that may be found on food and in 

drinking water. 

Despite the improvements to collaboration at EPA, some risk assessors 

pointed out two barriers that limit collaboration. Specifically, assessors 

noted that conflicting priorities or goals among EPA offices and poor 

communication between some offices hinder the effectiveness of 

collaboration. For example, although some chemicals are studied by more 

than one office within EPA, the approaches and timelines differ among 

offices because the laws and responsibilities for each program office can 

differ significantly. As a result, what may be a priority chemical in one 

program office may not be a priority in another, thereby hindering timely 

collaboration. Furthermore, a couple of risk assessors found collaboration 

challenging because they could not find the right person in another office

to communicate with on a specific issue. 

Several risk assessors suggested ways to improve and increase 

communication among program offices, ORD, and non-EPA organizations. 

For example, some risk assessors suggested more interagency work 

groups or meetings as a way to address research needs and foster 

information exchange on the development of methods. A few risk 

assessors suggested that a central library of risk assessment information 

would facilitate collaboration and avoid duplicating work already done by 

others. Specifically, one risk assessor said EPA could provide centralized 

databases of work conducted by different agencies and organizations, 

such as chemical-specific toxicity data, specific exposure or other values, 

and points of contact at each office.

At least 80 percent of risk assessors responding to our survey said that the 

training they received, whether on the job, self-directed, office specific, or

agencywide, was moderately to very useful. Moreover, over half of these 

risk assessors said that training improved their ability to prepare risk 

assessments. Nevertheless, risk assessors surveyed and agency officials 

interviewed reported that both risk assessors and managers would benefit 

from more in-depth training on subjects such as analytic tools and 

emerging scientific issues. 

EPA’s Training Has Been 

Helpful, but Risk 

Assessors and Managers 

Lack Sufficient Training on 

Analytic Tools and 

Emerging Issues 

Page 57 GAO-06-595 Human Health Risk Assessment
Over half of the risk assessors reported that training had moderately or 

greatly improved their abilities in at least seven different risk assessment 

skill and knowledge areas. More specifically, over 75 percent of risk 

assessors reported that training has helped them learn about a particular 

risk assessment method or model, enhance the quality of risk assessments 

or risk characterizations they prepare, and maintain or refresh their 

existing knowledge (see fig. 4). To a slightly lesser extent, training also 

helped risk assessors apply EPA guidance and policies, more effectively

prepare risk assessments, and understand EPA’s four-stage paradigm for 

risk assessment. 

Figure 4: Areas in Which Training Has Moderately or Greatly Improved Risk 

Assessor Knowledge and Skills 

Over three-quarters of risk assessors reported that they participated in onthe-job (82 percent) or self-directed (77 percent) training to enhance their 

skills. (See fig. 5.) According to risk assessors in the office that reviews 

Risk Assessors Have Enhanced 

Their Skills through Training in 

Various Areas 

0

10

20

30

40

50

60

70

80

90

100

Communicate effectively

 with stakeholders and

 the public

Communicate effectively

 with risk managers

Evaluate alternate

 methods

and models

Understand EPA’s paradigm

for risk assessment

More effectively prepare

 risk assessments

Apply EPA guidance

 and policies

Sustain/refresh existing

 knowledge

Enhance the quality of

 risk assessments or risk

 characterizations

Learn about a particular

risk assessment

method or model

Percentage of risk assessors

Source: GAO analysis of survey data.

Areas

Page 58 GAO-06-595 Human Health Risk Assessment
new chemical applications, training is primarily on the job, largely because

the office makes risk assessment decisions under tight time frames, which 

limits the time available for formal training, according to program officials. 

Similarly, workload constraints affect opportunities for formal training at

the Office of Water, which instead relies primarily on self-directed or onthe-job training. Some on-the-job training takes the form of mentoring, 

such as in the Office of Research and Development, where senior staff are 

expected to mentor newer staff. 

Figure 5: Types of Training Taken by Risk Assessors

In addition, over 70 percent of risk assessors who responded to our survey 

reported that they participated in office-specific training to enhance their 

risk assessment skills. For example, the Office of Pesticide Programs 

(OPP) has a comprehensive risk assessment training program focusing 

mostly on scientific issues, but also on other issues to improve the overall

quality of risk assessments. This training features speakers, including 

some from outside OPP, such as from other EPA offices, academia, and 

industry. These biweekly sessions are broadcast live over the office’s 

internal computer system and taped for future use as well as broader 

distribution, so that staff can access them as needed. Moreover, OPP’s 

0

10

20

30

40

50

60

70

80

90

100

EPA-wide 

training

Non-EPA

training

Officespecific

training

Selfdirected

training

On-the-job

training

Percentage of risk assessors

Types of training

Source: GAO analysis of survey data.

Page 59 GAO-06-595 Human Health Risk Assessment
training program includes having experienced risk assessors nearing 

retirement share their knowledge with other staff. Of the risk assessors 

responding to our survey that work in OPP, over 90 percent took 

advantage of its office-specific training. A similarly high percentage of staff 

in the hazardous air pollutant branch of the Office of Air and Radiation 

reported that they took that office’s training on preparing risk 

assessments.

In addition, 50 percent of risk assessors said they attended external 

training, such as professional society meetings sponsored by the Society

for Risk Analysis, Health Effects Institute, and the Society of Toxicology

and university courses related to scientific methods and disciplines. 

Finally, 46 percent of risk assessors said they participated in agencywide 

training. According to an EPA official, the agency offers a broad array of 

agencywide risk assessment training. For example, EPA’s Risk Assessment 

Forum develops training to accompany the agency’s risk assessment 

guidelines. More specifically, forum staff conducted briefings and 

orientation sessions on the 2005 cancer risk assessment guidelines. In 

addition, the forum sponsors colloquia approximately twice a year for staff 

to exchange information on risk assessment issues. Recent colloquia dealt

with dermal exposure assessment and whether additional guidance is 

needed on the use of Monte Carlo analysis. Colloquia usually result in 

reports that summarize the findings and may inform future guidance 

documents. 

While risk assessors reported taking advantage of and benefiting from 

various forms of training, they also said additional training would improve 

their ability to prepare risk assessments, a belief echoed by agency 

officials we spoke with. Over 70 percent of risk assessors stated that more 

in-depth or relevant training would improve their ability to prepare risk 

assessments. More specifically, some risk assessors expressed the need 

for training on analytic tools, such as modeling the dose-response 

relationship and statistical analysis and software. Several risk assessors 

also expressed a strong interest in training on scientific topics, including 

toxicology and such emerging issues as genomics, as well as nonscientific 

topics, such as public communications and public relations. Some risk 

assessors suggested EPA provide formal, comprehensive training for new 

risk assessors to educate them on how to use the guidance documents and 

more advanced courses for experienced risk assessors. Furthermore, 

many risk assessors reported that more time (69 percent) and more funds 

(65 percent) for training would moderately or greatly improve their ability 

to prepare risk assessments. (See fig. 6.) 

Additional Training Is Needed 

Page 60 GAO-06-595 Human Health Risk Assessment
Figure 6: Modifications in Training That Would Moderately or Greatly Improve the 

Preparation of Risk Assessments 

In addition to our survey respondents, some program officials we 

interviewed pointed out the need for additional training for both risk 

assessors and risk managers. For example, risk assessors need training in

emerging issues, such as genomics, so that they can use these types of 

data in risk assessments, according to one agency official. In contrast, risk 

managers need training on the elements of risk assessment, so that they

will be better able to interpret the information provided and apply it to risk 

management decisions. According to one program manager, risk managers 

often want the “bottom line” (numerical) results of a risk assessment 

without understanding the nature of the uncertainties in the assessment, 

or the potential value of obtaining more information to help clarify those 

uncertainties. 

Finally, the changing nature of the workforce may have implications for 

training at EPA. According to an agency official, many experienced risk 

assessors who possess years of institutional knowledge are retiring or 

nearing retirement age. Consequently, the agency needs to educate newer 

0

10

20

30

40

50

60

70

80

90

100

More 

general

Broader

audience

(including

risk

managers)

More

frequently

More

funds

More

time

More

relevant

More

in-depth

Percentage of risk assessors

Modifications to training

Source: GAO analysis of survey data.

Page 61 GAO-06-595 Human Health Risk Assessment
risk assessment staff as quickly and thoroughly as possible to help ensure 

that the agency’s ability to accurately and effectively produce risk 

assessments does not decline. While recently hired risk assessors come to 

EPA with specialized knowledge in fields related to risk assessment, they 

may not understand the broader context of risk assessment. For example, 

a new employee with degrees in biology and toxicology may not know 

how to integrate that knowledge with other scientific information to 

prepare a risk assessment. Furthermore, one EPA official noted that the 

agency has no formal training course, or set of courses, to help develop 

staff’s ability to prepare risk assessments. 

The experts we spoke with, including representatives of federal and state 

agencies, regulated industry, environmental advocacy groups, and outside 

researchers and consultants, said the modifications EPA has made over 

the past 10 years have been beneficial overall. However, they identified 

additional actions EPA could take to improve its risk assessment process, 

recognizing that EPA would face barriers to doing so. Specifically, EPA 

could improve its planning process of what will be required to complete a 

risk assessment by better identifying the scientific data it has and data it 

needs on the potential adverse effects from exposure to a contaminant and

by seeking stakeholder input early in this planning process. In addition, 

EPA could more thoroughly evaluate methods and models, transparently 

document its analytic choices, and enhance internal review. Finally, 

experts believe EPA could provide additional training for risk assessors, 

managers, and stakeholders. While these efforts would further improve the 

risk assessment process, EPA could face barriers in carrying them out, 

such as the scientific complexity of risk assessment, the difficulty of 

obtaining and applying data, and a cultural resistance to deviating from 

established methods. 

In order to ensure that EPA has the data needed for risk assessment, it 

needs to better identify data that are available, prioritize its data needs, 

and collaborate with the external research community during the planning

phase. For example, several experts said EPA should generate a 

searchable database of studies conducted by different agencies and 

organizations related to the chemicals being evaluated, so that researchers 

and risk assessors could more easily identify what studies are available 

and what additional research is needed. Experts also suggested several 

ways for EPA to prioritize its data needs. For example, sensitivity analysis 

can be used within individual risk assessments to determine which data 

gaps are the most critical to the risk assessment result. Some experts also 

Enhanced Risk 

Assessment Planning, 

Improved Analysis 

and Review, and 

Added Training Could 

Further Improve 

EPA’s Process, but 

Barriers Could Limit 

Progress 

Enhance Planning by 

Increasing Focus on Data 

Needs and Involving a 

Broad Range of 

Stakeholders 

Page 62 GAO-06-595 Human Health Risk Assessment
said EPA could better prioritize its data needs by increasing its use of data 

on the amounts of contaminants in people’s bodies to help concentrate its

research on the chemicals to which humans are actually exposed. Finally, 

several experts suggested that EPA increase its collaboration with 

external researchers, in part because the agency lacks the resources to 

independently generate all of the data that are needed. If EPA more 

effectively collaborated with other federal research organizations, such as

the National Toxicology Program and the National Institute of

Environmental Health Sciences, federal research dollars could be better 

harnessed to help EPA protect the public from exposure to contaminants. 

For example, the National Toxicology Program has the technology to 

assist the Office of Pesticide Programs with its screening of inert 

ingredients, which are all of the “other” chemicals in a pesticide product. 

In addition, experts said EPA should use all relevant data, including data 

from industry research laboratories, provided EPA takes steps to ensure 

the data are generated in an unbiased and scientifically defensible way. 

For example, experts suggested that EPA could subject studies to 

independent peer review and evaluate the sufficiency of data produced by

these organizations to increase confidence about using these data in EPA 

risk assessments. 

Experts also said that EPA could improve the quality of risk assessments if 

the agency enhanced its planning by more consistently involving 

stakeholders, especially early in the process. Several experts said that 

increased involvement with a broad range of stakeholders early in the 

planning process would help identify alternate methods and models to use 

and obtain stakeholder concurrence with the agency’s approach. Although

all stakeholders might not agree on the methods chosen, some experts 

believe that by seeking stakeholder input on these issues early in the 

process, EPA may minimize arguments later. In addition to stakeholder 

involvement with early planning, several experts recommended that EPA 

increase coordination with stakeholders throughout the process. For 

example, one expert said EPA could more transparently acknowledge and

address comments from the public and other stakeholders, regardless of 

whether the agency planned to implement their suggestions. 

EPA could more thoroughly evaluate its analytic choices and incorporate

or develop a wider variety of analytic tools. Some experts said that, with 

regard to EPA’s use of default options, risk assessors should more 

thoroughly document why available data are insufficient to allow EPA to 

use another analytic approach and commented that the revised cancer 

guidelines may provide a useful framework for making this decision. 

More Thoroughly Evaluate 

and Transparently 

Document Analytic 

Choices, and Increase 

Internal Review 

Page 63 GAO-06-595 Human Health Risk Assessment
Furthermore, several experts said EPA could use a tool like sensitivity 

analysis to assess and clearly communicate the extent to which the choice 

of a method or default assumption affects the risk assessment outcome. 

For example, if sensitivity analysis demonstrated that the impacts were 

significant, EPA could use the analysis to identify the critical areas where 

additional studies might reduce the need to rely on a default assumption 

for that assessment. One expert pointed out that the Office of Pesticide 

Programs uses sensitivity analysis in its exposure assessments to estimate 

which uses of a pesticide present the greatest risk to workers and how to 

mitigate those risks. In addition, several experts recommended that EPA 

make better use of existing analytic tools and develop new ones, where 

needed. For example, some experts said EPA should more frequently 

employ probabilistic analyses in risk assessments and incorporate the 

latest scientific tools, such as genomics and computational toxicology, to 

better assess uncertainty and variability. In addition, several experts noted 

that EPA needs to develop tools and methodologies to better analyze 

certain aspects of risk assessment, such as the combined effects of 

exposure to multiple chemicals through multiple pathways. 

Experts also said EPA risk assessments should clearly describe the 

sufficiency of the data and the scientific basis for its choice of a default 

assumption, method, or model. Some experts pointed out that risk 

assessments should identify and clearly discuss any data that are not 

available for the analysis, including the form the data need to be in and the 

most appropriate study design or methodology to obtain the needed data. 

In addition, several experts said EPA needs to more explicitly

communicate which default assumptions were used in a risk assessment 

and why the defaults were chosen. For example, one expert said that even

though a risk assessment may be perfect, if the public does not understand 

the rationale behind the agency’s choices, the risk assessment might be 

seen as flawed. Furthermore, in individual risk assessments, the agency 

could more transparently identify which critical studies would help the 

agency avoid relying on default assumptions. Some experts also suggested 

that EPA use as case studies completed assessments for which the agency

had sufficient data to use models and other analytic tools rather than 

default assumptions to more accurately assess risks. Finally, some experts 

said that EPA should more transparently consider alternate methods and 

models in each risk assessment. For example, EPA should be more 

transparent about the judgments it makes when it employs certain 

methods, such as the benchmark dose method, which identifies the dose 

that produces a small increase in the risk of an adverse effect. 

Page 64 GAO-06-595 Human Health Risk Assessment
Finally, experts suggested that EPA increase internal reviews of risk 

assessments by staff members with extensive risk assessment experience.

Internal reviews could improve the risk assessment process in two ways: 

first, to assure the quality of risk assessments and second, to ensure that 

the design of its risk assessments match the needs of risk managers. For

example, one expert suggested EPA reinstate a senior peer review group, 

composed of experienced risk assessors from throughout the agency. 

Others suggested that EPA could also internally peer review risk 

assessments prepared by less experienced staff to ensure that default 

assumptions are applied appropriately and transparently explained. In 

addition to increased review of individual risk assessments, some experts 

also felt the risk assessment process could benefit from additional 

examination of agencywide cross-cutting issues applicable to all program 

offices. For example, one expert said that some analytic tools, such as 

Monte Carlo analysis, were not developed specifically for use in risk 

assessments and suggested that EPA work with ORD’s National Center for

Computational Toxicology to define how these tools could be used in risk 

assessments across EPA. Moreover, according to several experts, 

agencywide discussions and activities promote consistency in risk 

assessment practices. For example, some experts thought EPA could 

benefit from a systematic agencywide discussion of the sources of 

uncertainty in risk assessment. 

Experts emphasized the importance of training for risk assessors, risk 

managers, and the stakeholder community on all elements of the risk 

assessment process. Several experts said risk assessors are not adequately 

trained in basic risk assessment principles, such as available default 

assumptions and when they should be used or replaced. Some experts also

suggested risk assessors receive training in using and applying models and 

in how to interpret data from emerging scientific fields to improve their 

ability to use these data, as appropriate, in risk assessments. Several 

experts also believe that training for risk managers would help improve 

risk assessments because risk managers need to better understand the role

risk assessment plays in risk management. According to some experts, risk 

managers who are more familiar with the process are better equipped to 

support risk assessors and ensure that the risk assessment considers all 

appropriate factors. Finally, a few experts also suggested that EPA hold 

training for stakeholders in the risk assessment process. For example, one 

expert suggested that EPA develop Web-based training for both the 

regulated community and regulators themselves to help ensure

consistency in how they understand the process. As part of this training, 

More Training Could 

Improve Risk Assessor,

Risk Manager, and 

Stakeholder 

Understanding of the 

Process 

Page 65 GAO-06-595 Human Health Risk Assessment
EPA could explain how risk assessment fits into the overall risk 

management process. 

While experts identified a number of actions EPA could take to improve 

the risk assessment process, they said EPA may face barriers such as the 

highly complex, technical, and time-intensive nature of preparing risk 

assessments, challenges in acquiring and applying data from all available

sources, and a general reluctance to deviate from its established methods 

and assumptions. Several experts pointed out that EPA’s risk assessments 

have grown more technically challenging and require risk assessors and 

managers to possess different skills than in the past. For example, some 

experts told us risk managers have different levels of expertise and 

background in risk assessment, and may not fully understand how risk 

assessment helps inform regulatory decisions. Moreover, some experts 

said that because risk assessment is just one piece of information used to 

make a regulatory decision, it is difficult to explain to stakeholders and the 

public the impact of risk assessments on risk management decisions. 

Experts also pointed out that scientific knowledge on subjects, such as 

uncertainty and variability, is limited and analytic tools are still being 

developed. For example, several experts said that while it would be useful

for EPA to more fully consider the risks of exposure to a single chemical

from all exposure pathways, at present it is an emerging science with few 

well developed analytic tools to use in risk assessments. In addition, using

tools, such as probabilistic analysis, to assess variability requires large 

amounts of data that are seldom available. Finally, several experts said 

that improving the process by such steps as incorporating new analytic 

techniques and conducting thorough internal review requires more time 

and coordination. For example, one expert pointed out that EPA does not 

always have the staff and time to analyze all sets of data or to examine 

alternative methods or models that might provide a more robust risk 

estimate. 

In addition to barriers attributable to the complexity of preparing risk 

assessments, experts also said EPA may face barriers in acquiring and 

applying data from all available sources. Many experts commented that 

data are expensive to obtain, and EPA has limited financial resources to 

devote to such activities. For example, some experts pointed out that 

some of the more direct studies of human exposure, such as 

epidemiological or biomonitoring studies, are quite expensive to conduct. 

Furthermore, EPA may be reluctant to use available data from all sources. 

As several experts pointed out, data from industry-sponsored researchers 

might be perceived as biased, potentially subjecting EPA to criticism. 

EPA Faces Barriers to 

Improving Its Risk 

Assessment Process, Such 

as the Complexity of Risk 

Assessment, Difficulty of 

Acquiring and Applying 

Data, and a Culture 

Resistant to Change 

Page 66 GAO-06-595 Human Health Risk Assessment
Despite potential perceptions of bias, some experts thought EPA should 

have the ability to use all available data, regardless of its source, as long as 

the data in question have been appropriately peer reviewed. In addition, 

some experts said statutory requirements may limit EPA’s ability to use 

certain data. For example, the Toxic Substances Control Act limits EPA’s 

authority to require extensive data from industry before deciding whether 

to approve a new chemical. Some experts also pointed out that research

does not always produce clear-cut results. For example, one expert 

commented that epidemiological studies of the general population may not 

account for confounding factors, such as exposure to other chemicals, 

which complicate efforts to draw conclusions about the effects of a single 

chemical. In addition, some experts said that variability, an important but 

scientifically complicated issue, often creates inconsistencies across 

studies because many factors such as geography, lifestyle, and food intake

affect an individual’s response. 

Finally, experts said that EPA has a general reluctance to deviate from 

using methods and assumptions it has used in the past. As a result, experts 

said EPA prefers to use techniques that have been generally accepted in 

the scientific community than to use methods that rely on recent scientific 

advances. For example, some experts told us EPA is often reluctant to 

deviate from its established default assumptions. Furthermore, some 

experts also commented that risk assessors may not have an incentive to 

deviate from methods and assumptions they have used in the past because

it may make the risk assessment more easily challenged by those who 

disagree with it. In addition, some experts said the level of comfort in 

using new methods varies throughout the agency. For example, one expert

believes that probabilistic models have been applied inconsistently 

because some risk assessors have been unwilling to deviate from the 

standard models. 

While technical and difficult to understand by nature, risk assessment is a 

key element in EPA’s efforts to protect human health from the potentially

harmful effects of chemicals, pollutants, and toxic substances that people 

encounter in their everyday lives. Since 1994, EPA has taken a number of 

steps, including greater involvement by the public and other stakeholders, 

to strengthen and improve its process for preparing assessments of the 

risks posed by contaminants in the environment. Independent reviewers 

as well as the experts and EPA risk assessors we contacted said overall 

EPA’s efforts have improved the agency’s risk assessments. However, the 

agency itself and the individuals we contacted acknowledge that EPA 

needs to do more. While some barriers to further improvement depend on

Conclusions 

Page 67 GAO-06-595 Human Health Risk Assessment
scientific advances that are largely beyond EPA’s control, other actions to

improve its risk assessment process are within its control. Specifically, 

when EPA engages the stakeholder and research communities after the 

risk assessment has largely been completed, it misses opportunities to 

benefit from their expertise. By working with stakeholders early and 

periodically throughout the process to identify key issues, studies, 

methods, and default assumptions that need to be considered in the 

analysis, EPA would help ensure consistent, transparent, and high-quality 

risk assessments. On the other hand, failure to take full advantage of 

stakeholders’ knowledge and points of view is likely to contribute to the 

perception among stakeholders that their concerns are not adequately 

represented in the risk assessments and that EPA’s decisions lack 

transparency. While EPA has issued a number of guidance and policy 

documents advocating the benefits of early planning, it acknowledges it 

could do more to ensure that such planning and consultation take place 

and involve relevant stakeholders. Furthermore, EPA does not always 

systematically communicate its data needs to the research community. 

While EPA has begun to better identify and prioritize its specific data 

needs, it has not been able to consistently develop data it needs in a timely 

manner. A more proactive approach to communicating its research needs 

to outside public and private researchers would help EPA more efficiently 

use the limited resources it has to obtain the data it needs. Furthermore, 

this approach would increase the likelihood that EPA would have data it 

needs to complete risk assessments now and into the future and that 

appropriately designed research projects would be conducted.

Transparently communicating its research needs would also enhance 

EPA’s ability to produce high-quality, scientifically defensible risk 

assessments and reduce the uncertainty associated with the effects of 

many contaminants on human health. Although experts we interviewed 

said EPA may hesitate to seek and use data from a wide range of sources 

because it could be seen as biased, EPA could take steps to ensure the 

quality of data generated by others. By doing so, EPA would expand its 

cache of available data and, potentially, reduce its reliance on default 

assumptions. Finally, current workforce models of high-performing 

organizations stress the need to formally and comprehensively assess the 

skill and competency requirements for staff and to identify related training 

and developmental needs to ensure that the workforce retains a high level

of needed skills. In recent years, EPA’s emphasis on training for its risk 

assessors and managers has declined in the areas risk assessors and 

experts say are needed to improve the quality of risk assessments and take

advantage of recent scientific and analytic advances. Without an 

agencywide training program for its risk assessment and risk management 

workforce, the quality, consistency, and transparency of risk assessments 

Page 68 GAO-06-595 Human Health Risk Assessment
and risk management decisions will likely continue to be challenged by 

stakeholders and the public. 

To improve the overall quality, consistency, and transparency of its risk 

assessments, we recommend that the Administrator of EPA direct the 

appropriate agency entities to take the following three actions: 

• Develop a strategy to ensure that offices engage in early planning to 

identify and seek the expertise needed, both within the EPA workforce 

and from external subject matter experts. The strategy should delineate 

such things as how EPA could use the available expertise to determine the

needed data, the relevant default assumptions, the extent of internal and 

external review that needs to be included in the assessment, and the 

approach used to consistently involve a broad range of stakeholders—

including the public, regulated industry, federal agencies, and advocacy 

groups—as appropriate to the risk assessment. 

• More proactively identify the data most relevant to the current risk 

assessment needs, including the specific studies required and how those 

studies should be designed, and communicate those needs to the research 

community. Increased collaboration among program offices in identifying

needed data would help ensure that the resulting data will meet the needs 

of multiple offices. In addition, EPA should better communicate these data

needs and better coordinate research planning with the external public 

and private research community to help focus EPA’s limited resources. 

• Ensure that risk assessors and risk managers have the skills needed to

produce quality risk assessments by developing and implementing indepth training. This training should address the needs of risk assessors 

and managers with varying levels of expertise by including basic courses, 

such as an overview of risk assessment, as well as more advanced courses 

on topics such as modeling, toxicology, and other advanced scientific 

techniques. 

We provided EPA with a draft of this report for review and comment. EPA 

neither agreed nor disagreed with our findings and recommendations. 

However, the agency provided specific comments to improve the report’s 

technical accuracy, which we incorporated as appropriate. 

Recommendations for 

Executive Action 

Agency Comments 

Page 69 GAO-06-595 Human Health Risk Assessment
As agreed with your office, unless you publicly announce the contents of

this report earlier, we plan no further distribution until 30 days from the 

report date. At that time, we will send copies to the Administrator, EPA, as 

well as to appropriate congressional committees, and other interested 

Members of Congress. We also will make copies available to others upon 

request. In addition, the report will be available at no charge on the GAO 

Web site at http://www.gao/gov. 

If you or your staff have questions about this report, please contact me at

(202) 512-6225 or stephensonj@gao.gov. Contact points for our Offices of 

Congressional Relations and Public Affairs may be found on the last page 

of this report. GAO staff who made key contributions to this report are 

listed in appendix II. 

Sincerely yours, 

John B. Stephenson 

Director, Natural Resources 

 and Environment 

Page 70 GAO-06-595 Human Health Risk Assessment
Appendix I: Objectives, Scope, and Methods Appendix I: Objectives, Scope, and Methods 

Our objectives for this review were to (1) identify the significant 

recommendations to improve human health risk assessment that have

been made since 1994; (2) describe what the Environmental Protection 

Agency (EPA) has done to modify its human health risk assessment 

process over the same period; (3) determine the effects these past 

modifications have had on the preparation of risk assessments; and (4) 

identify any additional actions experts believe EPA could take to improve 

its risk assessment process in the future, and the barriers EPA would face 

in doing so.

To identify significant recommendations to improve human health risk 

assessment since 1994, we reviewed EPA documents, including those 

produced by EPA’s Risk Assessment Forum (RAF), Science Policy Council

(SPC), Council for Regulatory Environmental Modeling (CREM), and 

Science Advisory Board (SAB) as well as each of EPA’s program offices. 

We also reviewed our own reports and documents produced by the 

National Academy of Sciences and the Presidential/Congressional 

Commission on Risk Assessment and Risk Management. To gain an 

external perspective, we spoke with experts in the risk assessment field, 

who identified many of these documents in the course of our discussions 

and provided insight into some of the documents’ recommendations. 

To describe what EPA has done to modify its human health risk 

assessment process, we interviewed program office managers from the 

Office of Air and Radiation (OAR), Office of Pesticide Programs (OPP), 

Office of Pollution Prevention and Toxics (OPPT), Office of Water (OW), 

and Office of Research and Development (ORD). We did not include the

site-specific risk assessment activities of the Office of Solid Waste and 

Emergency Response in our review. Within ORD, we interviewed 

managers in two of EPA’s laboratories (the National Health and 

Environmental Effects Laboratory and the National Exposure Research 

Laboratory) and three of EPA’s research centers (the National Center for 

Environmental Assessment, National Center for Environmental Research, 

and National Center for Computational Toxicology). We also interviewed 

officials from RAF, SPC, CREM, SAB, and the Office of the Science 

Advisor. Furthermore, we attended various EPA and stakeholder group 

training sessions and meetings. Since we limited our review to the human

health aspects of risk assessment since 1994, our analysis does not 

highlight EPA’s modifications prior to 1994, including publication of 

guidance documents that are highly relevant to risk assessment practices,

and does not address issues specifically related to ecological risk 

assessment. 

Page 71 GAO-06-595 Human Health Risk Assessment
Appendix I: Objectives, Scope, and Methods 

To assess the effects these modifications have had on the preparation of 

risk assessments, we conducted a Web-based nonprobability survey of all 

human health risk assessors from ORD and four EPA program offices that 

conduct human health risk assessment (OAR, OPP, OPPT, and OW). We 

used the survey to obtain an internal perspective on the usefulness of 

many of the modifications EPA made since 1994, as well as on aspects of 

EPA’s risk assessment process, including guidance documents, training, 

organizational structure, and collaboration. In developing the Web-based 

questionnaire, we met with EPA officials from the five offices surveyed to 

gain a thorough understanding of the risk assessment issues specific to 

each office and identify the sampling frame. In order to identify human 

health risk assessors—a label that is not an EPA job series—we obtained

from EPA officials in each program office being reviewed the names of 

agency staff who worked on any part of the human health risk assessment 

process since January 2001. Our sampling frame consisted of 270 staff that

met this criterion. This report does not contain all the results from the 

survey. The survey and results can be viewed at http://www.gao.gov/cgibin/getrpt?GAO-06-637SP. 

The practical difficulties of conducting any survey may introduce 

nonsampling error. For example, differences in how a particular question

is interpreted, the sources of information available to respondents, or the 

types of people who do not respond can introduce unwanted variability 

into the survey results. In order to reduce nonsampling error, we pretested 

the questionnaire with five risk assessors, one from each of the offices 

surveyed. During these pretests, we asked agency officials to complete the 

survey as we observed the process. We then interviewed the respondents 

to ensure that (1) the questions were clear and unambiguous, (2) the terms 

used were precise, (3) the questionnaire did not place an undue burden on 

the agency officials completing it, and (4) the questionnaire was 

independent and unbiased. On the basis of the feedback from the pretests, 

we modified the questions, as appropriate. Information about accessing 

the questionnaire was provided via e-mail for all survey participants. The 

survey was activated, and staff informed of its availability on October 17, 

2005; it was available until January 13, 2006. To ensure security and data 

integrity, we provided all participants with a personal password that 

allowed them to access and complete a questionnaire. No one else could 

access that questionnaire or edit its data. We included steps in both the 

data collection and data analysis stages for the purpose of minimizing such 

nonsampling errors. To reduce survey nonresponse, we sent e-mail 

reminders and conducted follow-up telephone calls with nonrespondents. 

Overall, 82 percent of the 270 risk assessors in our sampling frame 

Page 72 GAO-06-595 Human Health Risk Assessment
Appendix I: Objectives, Scope, and Methods 

responded to our survey, and all offices had a response rate of at least 80 

percent. 

We used general modifiers (i.e., many, several, some, a few, and a couple) 

to characterize written responses to some open ended survey questions. 

We used the following method to assign these modifiers to our statements 

about risk assessor’s survey responses: “many” represents 22 to 44 

respondents (roughly 10 to 20 percent), “several” represents 12 to 21 

respondents (5 to 10 percent), “some” represents 4 to 11 respondents, “a 

few” represents 3 respondents, and “a couple” represents 2 respondents. 

These divisions do not represent technically established categories; rather, 

we chose these divisions because they aligned with natural breaks in 

response “themes” highlighted in the report. 

To assess further actions EPA could take to improve its risk assessment 

process and to identify barriers it may face in doing so, we interviewed 

experts representing a range of stakeholders in the process. Specifically, 

we contacted risk assessment scientists; toxicologists; scientific advisers 

to EPA; state officials; and representatives from regulated industries, 

government agencies, and environmental advocacy groups who have an 

expertise in risk assessment. We used an iterative process (often referred

to as the “snowball sampling” technique) to identify these knowledgeable 

experts and selected for interviews those who would provide us with a 

broad and balanced range of perspectives on EPA risk assessment 

practices. 

We first contacted the National Academy of Sciences’ Board of 

Environmental Studies and Toxicology, which is the academy’s principal 

study unit for environmental pollution problems affecting human health 

and the assessment and management of related risks to human health and

the environment. We presented our engagement to the board and sought 

its input on the areas in which EPA has made the most progress improving 

its risk assessment practices and areas EPA will need to focus on in the 

future. We also asked members if they would be willing to participate in a 

future interview and solicited the names of other experts who would be 

appropriate for us to contact about this engagement. We selected for 

interviews experts who would provide us with a broad and balanced range 

of perspectives on EPA risk assessment practices. We continued 

interviewing and soliciting names until we determined we had appropriate

coverage from all the relevant stakeholder groups. Our sampling identified

22 experts, listed alphabetically, as follows: Elizabeth L. Anderson, Ph.D.; 

Gail Charnley, Ph.D.; Harvey J. Clewell, M.A.; Shannon Cunniff; Kerry 

Dearfield, Ph.D.; Michael L. Dourson, Ph.D.; Elaine M. Faustman, Ph.D.; 

Page 73 GAO-06-595 Human Health Risk Assessment
Appendix I: Objectives, Scope, and Methods 

Paul Gilman, Ph.D.; Gary Ginsberg, Ph.D.; Sherri Goodman, Esq.; Judith A. 

Graham, Ph.D.; Dan Greenbaum; Leslie J. Hushka, Ph.D.; Annie M. 

Jarabek, B.S.; James H. Johnson, Ph.D.; Elizabeth Julien, Ph.D.; Dorothy 

Patton, Ph.D.; Jonathan M. Samet, Ph.D.; Jennifer Sass, Ph.D.; Chris 

Whipple, Ph.D.; Richard Wiles, M.A.; and Lauren Zeise, Ph.D.

We used a standard set of questions to interview each of these experts to 

ensure we consistently discussed each aspect of EPA risk assessment 

policies and practices. To develop the questions, we reviewed 

documentation on EPA’s risk assessment process and reports prepared by

the National Academy of Sciences. We pretested our questions with two of

the experts and refined the questions accordingly. We used content 

analysis to identify the main themes among their responses. In addition, 

we asked the experts for their opinions about the many risk assessment 

modifications EPA has made since 1994, and used content analysis to 

synthesize their comments. 

We conducted our work from February 2005 through March 2006 in 

accordance with generally accepted government auditing standards. 

Page 74 GAO-06-595 Human Health Risk Assessment
Appendix II: 

A

GAO Contact and Staff 

cknowledgments 

Page 75 GAO-06-595 

Appendix II: GAO Contact and Staff 

Acknowledgments 

John B. Stephenson, (202) 512-6225 or stephensonj@gao.gov

In addition to the contact named above, Cheryl Williams (Assistant 

Director), Jennifer Cook, Michelle Cooper, Elizabeth Erdmann, and 

Rebecca Shea made key contributions to this report. Also contributing to 

this report were Nancy Crothers, Richard Frankel, and Roderick Moore. 

GAO Contact 

Staff 

Acknowledgments 

(360546) Human Health Risk Assessment
GAO’s Mission The Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 

constitutional responsibilities and to help improve the performance and 

accountability of the federal government for the American people. GAO 

examines the use of public funds; evaluates federal programs and policies; 

and provides analyses, recommendations, and other assistance to help

Congress make informed oversight, policy, and funding decisions. GAO’s 

commitment to good government is reflected in its core values of 

accountability, integrity, and reliability. 

The fastest and easiest way to obtain copies of GAO documents at no cost

is through GAO’s Web site (www.gao.gov). Each weekday, GAO posts 

newly released reports, testimony, and correspondence on its Web site. To

have GAO e-mail you a list of newly posted products every afternoon, go 

to www.gao.gov and select “Subscribe to Updates.” 

The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 

Documents. GAO also accepts VISA and Mastercard. Orders for 100 or 

more copies mailed to a single address are discounted 25 percent. Orders 

should be sent to: 

U.S. Government Accountability Office 

441 G Street NW, Room LM 

Washington, D.C. 20548 

To order by Phone: Voice: (202) 512-6000 

TDD: (202) 512-2537 

Fax: (202) 512-6061 

Contact: 

Web site: www.gao.gov/fraudnet/fraudnet.htm

E-mail: fraudnet@gao.gov

Automated answering system: (800) 424-5454 or (202) 512-7470 

Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

U.S. Government Accountability Office, 441 G Street NW, Room 7125 

Washington, D.C. 20548 

Paul Anderson, Managing Director, AndersonP1@gao.gov (202) 512-4800

U.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 

Obtaining Copies of 

GAO Reports and 

Testimony

Order by Mail or Phone 

To Report Fraud,

Waste, and Abuse in 

Federal Programs 

Congressional 

Relations 

Public Affairs 

PRINTED ON RECYCLED PAPER