BBooaarrdd ooff SScciieennttiiffiicc CCoouunnsseelloorrss 

OOffffiicceeooffRReesseeaarrcchhaannddDDeevveellooppmmeenntt

UUnniitteeddSSttaatteessEEnnvviirroonnmmeennttaallPPrrootteeccttiioonnAAggeennccyy

Human Health Research 

Program Review

Final Report of the Subcommittee on 

Human Health 

May 18, 2005 

Revised July 27, 2005 
This report was written by the Human Health Subcommittee of the Board 

of Scientific Counselors, a public advisory committee chartered under the 

Federal Advisory Committee Act (FACA) that provides external advice, 

information, and recommendations to the Office of Research and Development (ORD). This report has not been reviewed for approval by EPA, 

and therefore, the report’s contents and recommendations do not necessarily represent the views and policies of the EPA, or other agencies of the 

federal government. Further, the content of this report does not represent 

information approved or disseminated by EPA, and, consequently, it is not 

subject to EPA’s Data Quality Guidelines. Mention of trade names or 

commercial products does not constitute a recommendation for use. Reports of the Board of Scientific Counselors are posted on the Internet at 

http://www.epa.gov/osp/bosc. 
PROGRAM REVIEW REPORT

OF THE

BOARD OF SCIENTIFIC COUNSELORS

HUMAN HEALTH RESEARCH PROGRAM

Office of Research and Development 

U.S. Environmental Protection Agency 

MAY 18, 2005 

REVISED JULY 27, 2005

SUBCOMMITTEE ON HUMAN HEALTH

James E. Klaunig (Chair) – Indiana University School of Medicine 

James R. Clark (Vice-Chair) – Exxon Mobil Research and Engineering Company 

Timothy J. Buckley – Johns Hopkins Bloomberg School of Public Health 

Harvey Clewell – CIIT Centers for Health Research 

Michael Jayjock – The LifeLine Group 

Joseph Landolph – University of Southern California 

Donald Mattison – National Institutes of Health 

Elaine Symanski – University of Texas 

TABLE OF CONTENTS

I. EXECUTIVE SUMMARY............................................................................................................................. 1 

Overall Review Goals and Charge...................................................................................................... 1 

Subcommittee Members...................................................................................................................... 1 

Structure of the Program Review........................................................................................................ 2 

The Human Health Research Program: Background .......................................................................... 3 

Conclusions and Recommendations for the Program ......................................................................... 4 

Overall Program Strengths .......................................................................................................... 4 

Overall Program Opportunities..................................................................................................... 5 

Conclusions and Recommendations for Each Long-Term Goal ................................................... 5 

 

II. OVERVIEW COMMENTS ON HUMAN HEALTH PROGRAM ................................................................... 8 

Overall Strengths ................................................................................................................................ 8 

Overall Opportunities.......................................................................................................................... 8 

Program Relevance ............................................................................................................................. 9 

Program Quality................................................................................................................................ 10 

Program Performance........................................................................................................................ 11 

Program Leadership .......................................................................................................................... 11 

III. LONG-TERM GOAL 1: USE OF MECHANISTIC RESEARCH IN RISK ASSESSMENT .......................... 13 

Relevance .......................................................................................................................................... 13 

Quality .............................................................................................................................................. 17 

Performance ...................................................................................................................................... 18 

Scientific Leadership......................................................................................................................... 24 

IV. LONG-TERM GOAL 2: AGGREGATE/CUMULATIVE RISK ASSESSMENT ........................................ 25 

Relevance .......................................................................................................................................... 26 

Quality .............................................................................................................................................. 29 

Performance ...................................................................................................................................... 29 

Scientific Leadership......................................................................................................................... 30 

V. LONG-TERM GOAL 3: EVALUATION OF RISK TO SUSCEPTIBLE SUBPOPULATIONS ....................... 31 

Relevance .......................................................................................................................................... 32 

Quality and Performance................................................................................................................... 34 

Scientific Leadership......................................................................................................................... 35 

VI. LONG-TERM GOAL 4: EVALUATION OF PUBLIC HEALTH OUTCOMES .......................................... 37 

Relevance .......................................................................................................................................... 37 

Quality .............................................................................................................................................. 38 

Performance ...................................................................................................................................... 38 

Scientific Leadership......................................................................................................................... 39 

VII. TESTIMONIALS ................................................................................................................................... 40

VIII. APPENDIX A: CHARGE QUESTIONS................................................................................................. 42

IX. APPENDIX B: MEETING AGENDA ..................................................................................................... 48 

X. APPENDIX C: LIST OF ACRONYMS...................................................................................................... 51 
Human Health Research Program Review Report 1 

I. EXECUTIVE SUMMARY

Overall Review Goals and Charge 

Independent expert reviews are used extensively by industry, federal agencies, Congressional 

committees, and academia and have been recommended by the National Academy of Sciences as 

an approach for evaluating federal research programs. Accordingly, the U.S. Environmental 

Protection Agency’s (EPA) Office of Research and Development (ORD) enlisted its Board of 

Scientific Counselors (BOSC) to conduct independent expert reviews of its environmental 

research programs. ORD plans to use the findings from these evaluations to improve the 

management and performance of its programs; establish “best practices” in federal research 

program design, management, and evaluation; and strengthen research accountability as it 

complies with Office of Management and Budget (OMB) requirements. 

At a public meeting in September 2004, the Executive Committee of the BOSC agreed to form a 

new subcommittee to conduct a program review of ORD’s Human Health Research Program. It 

is expected that the review will provide guidance that will help ORD to: (1) plan, implement, 

and strengthen the Human Health Research Program; (2) compare the Human Health Research 

Program with programs designed to achieve similar outcomes in other parts of EPA and in other 

federal agencies; (3) make research investment decisions over the next 5 years; (4) prepare 

EPA’s performance and accountability reports to Congress under the Government Performance 

and Results Act of 1993; and (5) respond to evaluations of federal research, such as those 

conducted by OMB, which highlight the value of recommendations from independent expert 

panels. 

The new eight-member Human Health Subcommittee created by the BOSC is composed of 

representatives from academia, industry, and government. The Subcommittee was charged with 

evaluating ORD’s Human Health Research Program with respect to four criteria: relevance, 

quality, performance, and scientific leadership (see Appendix A for the charge questions). A 

series of questions associated with each of these four criteria was posed to the Subcommittee, 

and the Subcommittee chose to organize its response according to the long-term goals (LTGs) 

outlined in the Human Health Multi-Year Plan (MYP). Charge questions were answered in the 

context of each LTG. The salient points within each LTG have been captured and aggregated 

across the Human Health Research Program as a whole in this Executive Summary. 

Subcommittee Members 

The members of the Human Health Subcommittee have considerable expertise in the area of 

human health research, including formal education, training, and research experience in biology, 

chemistry, biochemistry, environmental carcinogenesis, pharmacology, molecular biology and 

molecular mechanisms of carcinogenicity and toxicity, toxicology, physiologically based 

pharmacokinetic (PBPK) modeling, exposure modeling, risk assessment, epidemiology, 

biomarkers and biological monitoring, and public health, with additional expertise in the areas of 

children’s health, community-based human exposure studies, and clinical experience. The 
2 Human Health Research Program Review Report

Subcommittee was chaired by Dr. James Klaunig from Indiana University School of Medicine. 

Dr. James Clark of Exxon Mobil Research and Engineering Company served as Vice Chair of 

the Subcommittee. Other members included Drs. Timothy Buckley, Johns Hopkins University; 

Harvey Clewell, CIIT Centers for Health Research; Michael Jayjock, The LifeLine Group; 

Joseph Landolph, University of Southern California; Donald Mattison, National Institutes of 

Health (NIH); and Elaine Symanski, University of Texas Health Science Center at Houston.

Structure of the Program Review 

The format of the program review’s face-to-face meeting was modeled after the successful 

reviews conducted at the division level by the National Health and Environmental Effects 

Research Laboratory (NHEERL). The meeting opened with an explanation of Federal Advisory 

Committee Act (FACA) rules by the Designated Federal Officer (DFO), after which a synopsis 

of the Human Health Research Program was presented. A series of presentations organized 

along the Program’s four LTGs followed. For each LTG, an overview of the key issues and the 

research approaches used to address these issues was presented by the ORD Lead for the LTG. 

This information was augmented by a poster session in which intramural and extramural 

researchers presented studies and results in greater scientific detail, also noting the impact and 

outcomes of their research and the extent to which funding was leveraged with other programs. 

The poster sessions gave the members of the Subcommittee the opportunity for one-on-one 

discussions with principal investigators. Following each poster session, the Subcommittee 

reconvened to summarize and discuss its findings; this discussion session was moderated by the 

Subcommittee member assigned as Lead for the LTG. The 2 1⁄2-day meeting concluded with 

testimonials presented by various EPA program and regional office representatives. These 

individuals discussed their use of the science produced by the Human Health Research Program 

and offered their perspectives on its relevance and impact. At the close of the meeting, a draft 

oral report was presented by the BOSC Human Health Subcommittee. Work sessions for the 

Subcommittee were held at various times throughout the face-to-face meeting, and time was 

allotted for public comment. 

Prior to the face-to-face meeting of the Subcommittee in North Carolina, two public conference 

calls were held where the charge to the Subcommittee was defined, individual writing 

assignments were made, and the expectations of the Subcommittee’s report was discussed. The 

members received several electronic and hardcopy documents describing the overall strategy of 

the Human Health Research Program, as well as a recent review of the strategy by a Science 

Advisory Board (SAB) Panel. In addition, printed legal-size paper copies of the posters were 

provided to the panel members upon arriving at the meeting. The Subcommittee members 

concurred that the structure of the review was well organized, and the presentation of the 

materials in both platform and poster forms was a very good approach, allowing for interchange 

between the scientists involved in the four programs and the Subcommittee members conducting 

the review. 

The Subcommittee recognized and appreciates the amount of work that ORD put into developing 

preparatory materials. Evidence that ORD has responded to previous reviews by the SAB’s 

Human Health Research Strategy Review Panel, the SAB’s Drinking Water Committee, the 

BOSC, and the National Research Council (NRC) is present both in written documentation as 
Human Health Research Program Review Report 3 

well as verbal presentations. The Subcommittee expressed disappointment, however, that the 

information provided to the members was not presented in a format that specifically addressed 

the review criteria (i.e., relevance, quality, performance, and scientific leadership). Future 

BOSC reviews can be facilitated and enhanced by: (1) providing to the Subcommittee the 

reports/critiques from previous reviews; and (2) tailoring the EPA presentations to the review 

criteria and critiques from previous reviews. 

The Human Health Research Program: Background 

In 1998, EPA sponsored a meeting of scientists and managers from ORD and the regional and 

program offices. The objective of the meeting was to identify cross-cutting issues of high 

priority that would form the basis of a core human health research program at EPA. Subsequent 

to that meeting, ORD developed the Human Health Research Strategy (2003), which provided a 

conceptual framework for human health research and identified and prioritized the research 

needed to improve the scientific foundation for health risk assessments. Four areas of research 

were emphasized in the strategy: harmonizing cancer and noncancer risk assessments (which 

has been subsequently refocused as the use of mechanistic information in risk assessment), 

assessing aggregate and cumulative risk, evaluating the risk to susceptible human 

subpopulations, and evaluating public health outcomes. These priority research areas set the 

strategic direction for the Human Health Research Program in EPA and provided the scientific 

basis for the development of the LTGs later identified in the Human Health MYP. The Human 

Health Research Strategy Review Panel of the SAB reviewed EPA’s Human Health Strategy 

document in 2003 and made numerous recommendations. EPA adopted these recommendations, 

making the program very strong. 

The Human Health MYP (2003) is a document that describes in greater detail ORD’s plans for a 

core research program in human health. It provides information to aid and support decisions 

during budget formulation, focuses on key research questions and scientific results, and 

demonstrates how the research program will contribute to Agency outcomes and strategic goals. 

 In the MYP, ORD identified four LTGs, each linked to one of the four research areas articulated 

in the Human Health Research Strategy that represent the highest priority research needs. The 

four LTGs are: 

Ê LTG 1: Use of Mechanistic Information in Risk Assessment 

The key research questions for this goal are: 

• What modes/mechanisms of action (MOA) are important for understanding the impact of 

environmental stressors on human health? 

• What are the attributes (e.g., shape of the dose-response, species specificity) of the MOA 

that impact risk assessment? 

• How do we measure, model, and/or predict the key attributes of the MOA that could 

impact risk assessment? 

• How do we incorporate mechanistic tools into risk assessment? 

Ê LTG 2: Aggregate/Cumulative Risk Assessment 

The key research questions for this goal are: 

• What are people’s real world aggregate exposures? 
4 Human Health Research Program Review Report

• What contributes to aggregate exposures? 

• How do we predict cumulative risk from aggregate exposures? 

• How do we mitigate aggregate/cumulative risk? 

Ê LTG 3: Evaluation of Risk to Susceptible Subpopulations 

The key research questions for this goal are: 

• Which subpopulations have differential risk to environmental stressors? 

• What is the basis for differential risk? 

• What is the risk to each subpopulation? 

• How can differential risk be mitigated? 

Ê LTG 4: Evaluation of Public Health Outcomes 

The key research questions for this goal are: 

• What public health outcomes need to be examined to evaluate Agency regulatory 

decisions? 

• What approaches/tools are needed to evaluate (and attribute) changes in public health 

outcomes to Agency actions? 

• Did Agency actions have an impact on public health outcomes? 

By addressing these questions, it is the goal of the Human Health Research Program to provide 

fundamental understanding of the physical and biological processes that underlie environmental 

systems and human populations at risk. It is expected that the products of this program will 

provide an integrated information base for scientifically defensible risk assessment and risk 

management decisions. 

Conclusions and Recommendations for the Program 

Overall Program Strengths

Ê The research of the Human Health Research Program was found to be of high quality and 

appropriately focused. 

• It was multidisciplinary, displayed good stakeholder participation, informed risk 

assessments, and achieved the goal of reducing uncertainty. 

Ê The hallmarks of the program are that the research is multidisciplinary, yet coherent and 

coordinated. 

• The scientists display an overall high level of enthusiasm and commitment. 

• Development of new data is encouraged, and extant data are fully explored and utilized 

to inform the risk assessment decisions of stakeholders. 

Ê Interaction is occurring among investigators involved in different LTGs, and this interaction 

should be acknowledged and encouraged through both formal (e.g., research retreats) and 

informal means. 
Ê The research benefits from managerial excellence across all aspects of the program. 

• Managers set a tone and expectation that encourages, spawns, and rewards problemdriven cross-discipline teams at the operational level. 

• Excellent managerial support is reflected in the overall high level of excitement, 

commitment, and passion displayed by the scientists in describing their work. 

Ê A remarkably positive and valuable aspect of this program is a decisive propensity within the 

program to encourage the mining of available data and science to inform the risk assessment 

decisions of stakeholders. 

Overall Program Opportunities

Ê To some extent, the direction, choice, and focus of research topics are undoubtedly areas 

where national politics interact and shape the development of specific scientific programs. 

• Although this is a reality of conducting research in support of the Agency, it is important 

that ORD priorities continue to integrate the status of scientific knowledge. 

• The overall criteria and framework for decisions regarding why specific elements are 

vital and have been included in the research program could be enhanced further in the 

written material presented to the Subcommittee. 

• A more transparent explanation of prioritization and justifications would be most 

valuable. 

Ê The Human Health Research Program should monitor, engage in, and advise research efforts 

of other international organizations such as the European Union (EU). 

Ê The creation of the new National Center for Computational Toxicology may produce 

challenges with regard to teamwork. 

• The most quantitatively oriented individuals in the NHEERL and the most biologically 

oriented individuals of the National Exposure Research Laboratory (NERL) that are 

moving to the Center should not lose contact with scientists and issues in their former 

organizations. 

Ê A greater level of interaction between the externally funded University Centers and in-house 

research could result in more significant research progress (e.g., the case of the potential role 

of glutathione S-transferase [GST] polymorphisms in autism). 

Conclusions and Recommendations for Each Long-Term Goal 

LONG-TERM GOAL 1 

Use of Mechanistic Information in Risk Assessment 

Ê ORD scientists have been and continue to be leaders in developing research to support EPA 

risk assessments, which has allowed EPA to be a leader for many years in conducting 

credible, nationally/internationally accepted risk assessments of chemicals of environmental 

Human Health Research Program Review Report 5 
concern. Veteran scientific administrators are providing exceptional leadership to the 

Human Health Research Program. 

Ê Mechanistic research by ORD scientists has positively and significantly affected risk 

assessments and will benefit future assessments of other chemicals. Stakeholders (the 

Offices of Air, Water, Pesticides, and Toxics and regional offices) are involved in 

planning/prioritization of ORD’s Human Health Research Program. 

Ê The Human Health Research Program has a logical, comprehensive MYP that describes an 

appropriate flow of work within/across LTG 1 and has made significant progress toward each 

major area of research within this LTG. 

Ê The Agency has successfully utilized its extramural grants program to advance its research 

agenda. These programs, however, need to be better advertised and perhaps even better 

financed and expanded to attract the widest possible competitive applicant pool. 

Ê New, broad strategies should be developed by the Human Health Research Program to 

manage the risks from the thousands of new chemicals that are being synthesized and put 

into the environment. 

LONG-TERM GOAL 2 

Aggregate/Cumulative Risk Assessment 

Ê The research efforts on cumulative risk assessment are highly relevant to the problems faced 

by the Agency in assessing the risks faced by the public, who are typically exposed to 

multiple toxic chemicals. The research described in the area of cumulative risk assessment is 

particularly creative, effective, and well conducted. 

Ê The work shows a propensity to develop new data and to mine available data to inform risk 

assessment decisions. Assessments should be more comprehensive, however, and include a 

wider array of chemicals important to human exposure. 

Ê The overall criteria and framework for decisions regarding why specific elements are vital 

and have been included in the research program could be further enhanced in the written 

material presented to the Subcommittee. A more transparent explanation of these aspects 

would be most valuable. 

Ê The Human Health Research Program plays a leadership role in advancing the realm of 

scientific development in the areas that are currently addressed. The scientific staff is 

excellent, as are the facilities, and the funding appears to be adequate. Within this realm, the 

managers and researchers are overseeing and participating in a program that is leading the 

state-of-the-science. 

6 Human Health Research Program Review Report
LONG-TERM GOAL 3 

Evaluation of Risk to Susceptible Subpopulations 

Human Health Research Program Review Report 7 

Ê The susceptible subpopulations program is well grounded within the ORD strategic plan. 

There is evidence of good coordination with other programs in the Human Health Research 

Program, as well as with outside research organizations (nationally and internationally), and 

there is leveraging of effort across federal agencies. In addition, Agency scientists involved 

in the research on children’s susceptibility are internationally recognized experts in 

children’s environmental health and play a strong role in fostering a continuing emphasis on 

this research area. 

Ê The current level of involvement of program offices, regional offices, and other stakeholders 

provides strength to the program; it should be sustained and possibly upgraded. There is a 

need to expand EPA expertise to include community-based participatory research. 

Ê The single dimensional model presented in this program did not fully represent what is a 

dynamic multidimensional research program. Although the Agency’s focus on children as a 

susceptible population subgroup appears well justified, the justification can be strengthened 

by the Agency’s own scientific assessment of the public health benefit to be achieved 

through a research focus on children as a particular subpopulation. 

LONG-TERM GOAL 4 

Evaluation of Public Health Outcomes 

Ê This area of focus is relatively new to ORD and the Human Health Research Program and is 

consistent with the overall mission of EPA in the protection of human health. The program 

on public health outcomes is being built upon the same conceptual framework that supports 

the three other LTGs, namely the exposure-dose-effect continuum. Overall, the program on 

public health outcomes is highly relevant to the mission of the Agency and has the potential 

to serve as the nucleus for integrating and evaluating ORD research. 

Ê The Subcommittee believes that the goals of this program could be further focused to guide 

future activities and that a process needs to be articulated for making decisions about which 

actions to evaluate, which endpoints to study, and which environmental indicators to apply. 

Long-term success of this program will be dependent in part on the ability to develop strong 

interactions with other programs in EPA and utilize research from other LTGs. Thus, it is 

recommended that a mechanism be put into place with formal and informal components to 

promote dialogue among investigators involved in the different LTGs and to provide a 

process for assessing research outputs. 

 

Ê The program will require additional monies and personnel to broaden expertise in areas of 

public health, especially in biostatistics and environmental epidemiology. 
8 Human Health Research Program Review Report

II. OVERVIEW COMMENTS ON HUMAN HEALTH

RESEARCH PROGRAM

Overall Strengths 

The research of the Human Health Research Program was found to be of high quality and 

appropriately focused. It was multidisciplinary, displayed good stakeholder participation, 

informed risk assessments, and achieved the goal of reducing uncertainty. The hallmarks of the 

program are that the research is multidisciplinary and yet coherent and coordinated, the scientists 

display an overall high level of enthusiasm and commitment, and development of new data is 

encouraged at the same time that extant data are fully explored and utilized to inform the risk 

assessment decisions of stakeholders. 

Interaction is occurring among investigators involved in different LTGs, and this interaction 

should be acknowledged and encouraged through both formal (e.g., research retreats) and 

informal means. The Subcommittee was impressed by the high level of cooperation and 

teamwork among investigators and across organizations that was displayed in the Human Health 

Research Program. Effective interaction and collaboration across disciplines (e.g., computer 

modeling, statistics, pharmacokinetics, health endpoint evaluation) and across organizations 

(NHEERL, NERL), critical to an innovative, effective, and productive human health program, 

were achieved. ORD management is to be commended for fostering such an effective team 

research environment. 

Managerial excellence appears to be a hallmark of the overall program. Setting a tone and 

expectation that encourages, spawns, and rewards problem-driven cross-discipline teams at the 

operational level is a clear and positive result of this managerial effort. Interactions with 

scientists and programs external to EPA are less formally organized, but where they occur, they 

appear to be appropriately happening at the lead scientist/principal investigator level. Additional 

evidence of excellent managerial support is reflected in the overall high level of excitement, 

commitment, and passion displayed by the scientists in describing their work. It is suggested 

that this strong positive aspect of this program be explicitly recognized, continually rewarded, 

and carefully guarded. 

Overall Opportunities 

To some extent, the direction, choice, and focus of research topics are undoubtedly areas where 

national politics interact and shape the development of specific scientific programs. Today, 

there are regulatory mandates playing out in the rest of the world that are driving the overall 

scientific development of human health exposure assessment for the multitude of common and 

relatively unstudied substances to which humans are exposed. Significant resources are about to 

be committed, especially in the EU, to develop the exposure and risk assessment tools needed to 

reasonably accomplish these mandates. 
Human Health Research Program Review Report 9 

Even if EPA ORD is not designing this type of research, it should monitor, engage in, and advise 

these research efforts of others. Scientists within the Human Health Research Program should 

contribute their considerable skill and knowledge to the EU Research Planning. 

The creation of the new National Center for Computational Toxicology may produce challenges 

with regard to teamwork. It appears that the most quantitatively oriented individuals in 

NHEERL and the most biologically oriented individuals in NERL will become part of the new 

Center. Although this reorganization apparently does not involve a change in location, there is a 

potential for the founding staff of the Center to become focused on the mission of the new 

organization and to some extent lose contact with the scientists in their former organizations. 

This transition should be monitored to ensure that collaboration continues to be encouraged and 

not organizationally impeded. 

Program Relevance 

Strategic drivers for the Agency’s Human Health Research Program are mandates from 

Congressional actions, mandates from the Administration, needs of the regional and program 

offices, needs of the scientific community, recommendations of external advisory groups, and 

goals established by EPA in response to the Governmental Performance and Results Act (GPRA) 

under Sound Science. Based on these strategic drivers, the Agency has identified two strategic 

research directions: (1) research to improve the scientific foundation of human health risk 

assessment, including harmonizing cancer and noncancer risk assessments (which has been 

subsequently refocused as the use of mechanistic information in risk assessment), assessing 

aggregate and cumulative risk, and determining risk to susceptible human subpopulations; and 

(2) research to enable evaluation of public health outcomes from risk management decisions. 

These two strategic research directions are consistent with EPA’s strategic goals, its Human 

Health Research Strategy, and the NRC’s recommendations for Core Research Program items. 

The presentations and review materials provided by ORD scientists showed that ORD has been 

diligent in implementing the recommendations in the report of the NRC’s Committee on 

Research Opportunities and Priorities for EPA (referred to as the ROPE report) into its Core 

Research activities. The formalization of a balance between Core Research and Applied 

Research is excellent and serves the Agency well. 

The core Human Health Research Program is effective and strategic in its coordination of its 

intramural and extramural research programs. Research programs are defined to achieve 

separate but complementary objectives. These programs are effectively coordinated, resulting in 

good synergy with minimal redundancy. Effective participation of program and regional offices 

in research planning, execution, and evaluation is a major strength of the Human Health 

Research Program. It is through this participation that EPA’s research is grounded in the 

practical needs of regulatory offices and communities. The interaction with program and regional 

offices is already occurring, although the strength of the interaction appears to vary across 

program offices. These strong and valuable interactions take place because of the initiative of 

ORD scientists, as well the facilitation of ORD management. Of course, constant vigilance is 

required to assure that practice-oriented research is balanced with basic hypothesis-driven 

research. 
10 Human Health Research Program Review Report

The Subcommittee noted that a greater level of interaction between investigators in the 

externally funded University Centers and in-house researchers could result in more significant 

research progress, as seen, for instance, in the case of the potential role of GST polymorphisms 

in autism. EPA has established an Extramural Research Grants Program and an Extramural 

Environmental Health Centers Program that both utilize the efforts of scientists at research 

institutes and universities to aid the Agency in developing research projects in areas of 

toxicology and carcinogenesis that need to be explored beyond the Agency’s immediate 

capabilities. 

The EPA also has established a program by which outside scientists can be hired on a temporary 

basis (Interagency Personnel Agreement [IPA], 4-year terms) to work inside the Agency and 

bring it new expertise. Cooperative Research and Development Agreements (CRADAs) also are 

available to extend the expertise of the Agency by incorporating the expertise of external 

scientists as extramural contractors. The posters and publications of ORD scientists showed 

clear evidence that collaboration is occurring between Agency scientists and scientists from 

other governmental agencies (e.g., the National Institute of Environmental Health Sciences 

[NIEHS]). A listing of intergovernmental agency collaborations between the Human Health 

Research Program of ORD and its sister governmental agencies, however, was missing from the 

review documents, so the full extent of this partnering could not be judged accurately and given 

appropriate credit. 

The public benefits from doing good science could be further enhanced in the written materials 

presented to the review Subcommittee. Certainly, understanding and substantially reducing 

cancer and noncancer risks are vitally important and clearly recognized in this plan. The 

Subcommittee advises that it also is important and valuable that the technical work products be 

able to render the scientific determination of de minimus or acceptably low levels of human 

health risk from chemical exposure. The value of confident knowledge regarding the relative 

safety offered by improved exposure and risk assessment tools of previously feared exposures 

and putative risks represents an arguably significant improvement to the public health. 

Program Quality 

The Human Health Research Program is of high quality and appropriately focused. There appear 

to be good multidisciplinary approaches to the research, effectively mixing multidisciplinary 

research from the level of the cell to human populations and back again. In addition, there is an 

overall high level of excitement and commitment displayed by the scientists in describing their 

work. The research described was creative, effective, and well conducted. The Agency 

scientists have shown considerable skill in applying quantitative techniques ranging from 

statistical analysis to PBPK modeling, as required for the various aspects of the research. In 

conducting research on cumulative risk, EPA scientists have been able to simultaneously provide 

rapid response to the needs of the Agency’s regulatory program while still maintaining a strong, 

long-term research effort. In addition, peer review is recognized as a critical component of 

EPA’s Human Health Research Program. 

To better evaluate the quality and performance of the Human Health Research Program, 

reviewers would have benefited from a bibliometric analysis of publications; such an analysis 
Human Health Research Program Review Report 11 

would be a useful parameter for showing the impact of EPA research on the field. This analysis 

should differentiate the intramural from the extramural programs to facilitate assessments of 

Agency capabilities and the efforts of funded scientists, as well as assess collaborative efforts. It 

was brought to the attention of the reviewers that a bibliometric analysis was underway at the 

time of the review, but it was not available in time for the reviewers to take the results into 

consideration. 

Program Performance 

A remarkably positive and valuable aspect of this program is a decisive propensity within the 

program to encourage the mining of available data and science to inform the risk assessment 

decisions of stakeholders. The Subcommittee encourages the use of consensual scientific 

deliberation by teams of experts to provide the best advice possible relative to the critical 

questions facing risk assessors and risk managers. 

EPA’s overarching conceptual framework description for the core Human Health Research 

Program that represents the LTGs and their interaction could be expanded and more fully 

developed. The overall criteria and framework for decisions regarding why specific elements are 

vital and have been included in the research program were not always clear to the Subcommittee 

members. Although the research program appears appropriately directed and focused, its 

scientific basis, justification, and conceptualization could be further developed. The presentation 

of the justification of research priorities appeared to some members of the Subcommittee to be 

largely defined by external advisory bodies, such as the NRC. Although advice from such 

advisory groups provides an important element of justification, further clarification of the role of 

the scientists of the Human Health Research Program in defining and setting these priorities is 

suggested. The presented materials lacked sufficient detail relating the specific program 

elements to be able to reasonably conclude that the focus is consistent with the stated goals. 

Details such as exactly how the work is going to be planned and processed are critical, and 

although not presented in the premeeting materials, were in most cases clearly articulated during 

the meeting. 

Program Leadership 

The majority of the Subcommittee members thought that the scientific leadership of the Human 

Health Research Program was excellent. The team of veteran scientific administrators provides 

professional leadership to the Human Health Research Program and ORD scientists in EPA. The 

NRC’s ROPE report stressed strengthening the core research of ORD within EPA and using this 

core research program to aid that Agency and its programs. ORD has worked hard to implement 

recommendations of the NRC report and demonstrated success in supporting key advances in 

human exposure and health effects research. The scientific leadership of the program is strong. 

A minority of the Subcommittee members expressed concern, however, that the direction and 

leadership of the program was strongly influenced by external advisory groups. 

Program scientists contribute and frequently have taken on leadership roles in environmental 

science, toxicology, carcinogenesis, risk assessment, exposure assessment, and public health. 

Most of the researchers participating in the Human Health Research Program have made 
12 Human Health Research Program Review Report

significant contributions to the peer-reviewed scientific literature. The Subcommittee reviewed 

an extensive bibliography of publications derived from the Human Health Research Program. 

ORD scientists also serve in key positions in advisory committees and technical panels for 

governmental and industrial research programs, in key positions for scientific societies, and in 

undergraduate and graduate teaching programs at universities. Also, within EPA, ORD 

scientists have been leaders in developing research to support EPA risk assessments for many 

years, and they continue to lead in this area. This research strength of ORD has allowed EPA to 

be a leader in conducting credible, nationally and internationally accepted risk assessments on 

chemicals of environmental concern for many years. The Subcommittee noted an area of 

opportunity for further participation (i.e., examining involvement and potential collaboration 

with similar programs in the EU and Health Canada). These interactions should be coordinated 

at a much higher level within the Agency than is currently occurring. 

Another area of opportunity relates to leadership transition. There is evidence of a gap in the 

Human Health Research Program between the number of senior, established scientists currently 

holding leadership positions and younger, less experienced researchers. Some thought should be 

given by EPA to culturing and developing individuals for leadership positions (perhaps through 

mentorship) and making the emerging roles and responsibilities clear to all stakeholders. The 

Subcommittee recommends that EPA plan for leadership succession in both the technical and 

management arenas. 
Human Health Research Program Review Report 13 

III. LONG-TERM GOAL 1:

USE OF MECHANISTIC INFORMATION IN RISK ASSESSMENT

Relevance 

Strategic drivers for EPA’s Human Health Research Program are mandates from Congressional 

actions, mandates from the Administration, needs of the regional and program offices, needs of 

the scientific community, recommendations of external advisory groups, and goals established 

by EPA in response to GPRA under Sound Science. Based on these strategic drivers, the 

Agency has identified two strategic research directions: (1) research to improve the scientific 

foundation of human health risk assessment, including harmonizing cancer and noncancer risk 

assessments (which has been subsequently refocused as the use of mechanistic information in 

risk assessment), assessing aggregate and cumulative risk, and determining risk to susceptible 

human subpopulations; and (2) research to enable evaluation of public health outcomes from risk 

management decisions. These two strategic research directions are consistent with EPA’s 

strategic goals, its Human Health Research Strategy, and the NRC’s recommendations for Core 

Research Program items. 

The presentations and review materials provided by ORD scientists showed that ORD has 

worked hard to implement the recommendations of the NRC’s ROPE report into its Core 

Research activities. ORD has worked to balance its research efforts between Core Research and 

Applied Research to the extent of 50 percent in each of these activities, as recommended by the 

ROPE report. Presently, the balance stands at approximately 60 percent Core Research and 40 

percent Applied Research, which is appropriate. This formalization of a balance between Core 

Research and Applied Research is excellent and serves the Agency well. 

Consistency With Agency Strategic Goals, the Human Health Research Strategy, and 

Recommendations of the NRC

Development of mechanistic data to guide risk assessments is crucial to ensure risk assessments 

produced by the Agency are accurate. Within EPA, ORD scientists have been leaders in 

developing research to support EPA risk assessments for many years, and they continue to lead 

in this area. This research strength of ORD has allowed EPA to be a leader in conducting 

credible, nationally and internationally accepted risk assessments on chemicals of environmental 

concern for many years. The work of ORD scientists on molecular mechanisms of the 

carcinogenicity of arsenic and dioxin has contributed significantly to cancer risk assessment in 

these areas. 

The EPA research program on dioxins has proved to be of high value in supporting EPA’s risk 

characterization for these chemicals. Important products of this research effort that have been 

incorporated into the dioxin risk assessment include: (1) the demonstration that steady-state 

body burden should be used as the measure of exposure for cross-species equivalence, instead of 

the traditional default of dose rates (mg/kg/day); (2) scientific support for the toxicity 
14 Human Health Research Program Review Report

equivalency factor (TEF) approach for assessing exposure to mixtures of dioxins; and 

(3) establishment of the human relevance of the mode of action for dioxin effects. This work has 

been used by other agencies internationally. Moreover, the importance of using body burden 

instead of dose rate in risk assessments for persistent chemicals is a major step forward that will 

impact many future risk assessments. 

Mechanistic research on atrazine by ORD scientists has had a similarly valuable impact on the 

risk assessment for this chemical and will benefit future assessments of other chemicals that 

disrupt the luteinizing hormone (LH) surge. This work demonstrates the importance of using 

mode of action studies to elucidate the interactions between toxic chemicals and biological 

systems instead of assuming simple equivalence of effects across species. The research efforts 

on atrazine demonstrated that effects in animals that were the initial concern (mammary cancer) 

were not likely to be relevant to humans. The mode of action for atrazine suggests that humans 

would be at risk for different toxic effects. The results have been highly influential in the 

Agency’s consideration of the risk from this chemical. The efforts to identify shortcomings in 

the standard developmental test protocols that failed to detect the developmental effects of 

atrazine provide important insight into the limitations of current animal testing protocols. These 

research efforts of ORD are very useful to the Office of Pesticides at EPA. Both the dioxin and 

atrazine research efforts clearly demonstrate the valuable interactions taking place in the Agency 

between the research program and the regulatory program. 

A further example of research relevant to EPA’s strategic mission was the demonstration that, 

for drinking water disinfection byproducts (DBPs) that share a common mechanism of action, 

effects of these chemicals can be added to generate a common additive risk. Starting with 

adjustments to internal animal dose, these investigators adjusted that dose to an internal human 

dose equivalent, allowing these investigators to develop dose-response curves for human 

response. This approach has significant utility in the calculation of the risk of humans to a 

mixture of DBPs. 

ORD researchers contributed an excellent piece of research on component-based methods for 

investigating chemical mixtures in toxicology. These investigators showed that at lower 

concentrations (i.e., close to relevant environmental exposure to pesticides) there is an additivity 

or antagonism in the hepatotoxicity of four trihalomethanes when these chemicals are 

administered to animals as mixtures, based on mixing and concentrations administered to 

animals. For organochlorine pesticides, there were greater than additive (synergistic) activities 

of these chemicals for inhibition of blood and brain cholinesterase, inhibition of motor activity, 

and gait score. They also studied chemicals that disrupt homeostasis of the thyroid gland 

(thyroid disrupting [TD] chemicals) and found that addition of mixtures of TD chemicals caused 

a dose-dependent decrease in thyroid hormone (T4) levels. The researchers found that at low 

doses, they obtained additivity for mixtures of TD chemicals. Intriguingly, at high doses, they 

found synergistic effects on thyroid hormone disruption. This interesting work is relevant to 

environmental exposure to these chemicals in mixing ratios to which humans are exposed, and 

the work should continue. 
Human Health Research Program Review Report 15 

Public Benefits

The public benefits of LTG 1 in the Human Health Research Program are clearly articulated. 

The utility of collecting mechanistic data as part of ORD’s basic science programs in toxicology 

and carcinogenesis is obvious and clearly dedicated to generating the basic science necessary to 

allow the development of mechanistically based risk assessments. This is clearly shown by the 

support and enthusiasm of the Office of Pesticides for the mechanistic research ongoing at ORD 

in the area of conazoles and atrazine. The public has clearly benefited by the strict regulation of 

these pesticides based on mechanistic data generated by ORD researchers. Further effort needs 

to be invested in articulating the benefits of the program to the public, so they appreciate the 

many past, present, and future successes of the EPA in protecting human health and the 

environment. 

Stakeholder Involvement

Stakeholders (e.g., program and regional offices) are involved in the planning and prioritization 

of research. As best represented by the flow diagrams presented during the meeting by the 

Executive Director of the Human Health Research Program and by the Region 5 Regional 

Science Liaison (RSL) to ORD, the program offices (Air, Water, Pesticides, and Toxics) and the 

regions communicate their needs to ORD, which communicates them to the Agency. The 

program offices and regions also communicate their needs directly to the Agency itself. Hence, 

the two major groups of stakeholders— the program offices and the EPA regional offices—are 

involved in the planning and prioritization of the research by communicating their needs for 

information to the Agency and to ORD. An excellent specific example of this is the very strong 

program on mechanisms of arsenic carcinogenesis research, which originated as a result of the 

needs of the Office of Water and the Office of Pesticides for scientifically sound, credible 

mechanism-based cancer risk assessments for arsenic in drinking water and for arseniccontaining pesticides in air. An area that needs to be better addressed by ORD is the Office of 

Water’s need for information on the carcinogenicity of compounds containing hexavalent 

chromium when administered by the oral (drinking water) route. NIEHS’ National Toxicology 

Program (NTP), however, already is conducting animal carcinogenesis studies by the ingestion 

route. ORD scientists are encouraged to track and leverage with these studies. 

Industry stakeholders also have been working with scientists in the Human Health Research 

Program and in ORD to be involved in the planning and prioritization of research conducted by 

ORD. A good example of this is the collaboration between the industrial consortium called the 

Triazole Group and the Conazole Research Group of the ORD. The Triazole Group provides 

animal feed dosed with various conazole fungicides to ORD investigators for their use and has 

an active dialogue with these investigators on the research results that they acquire. 

Several presentations were provided on interactions of ORD scientists with EPA offices and 

programs, including the Extramural Research Program, the Office of Pesticides, regional offices, 

and the Program on Children’s Environmental Health. From these presentations, it is clear that 

ORD scientists collaborate closely with the program and regional offices. Many examples were 

provided of the nature of this partnering. An excellent example includes the continuing support 

ORD scientists have provided to the Office of Pesticides to support cumulative exposure of 
16 Human Health Research Program Review Report

pesticides and pesticide cancer risk assessment. ORD scientist support in the Regional Applied 

Research Effort (RARE) Program and the crucial role of ORD scientists in collaborating with the 

Program on Children’s Environmental Health are two more examples. 

These strong and valuable interactions take place largely because of the willingness of ORD 

scientists and the strong support of the management of ORD. The Agency is to be commended 

for maintaining a highly collaborative scientific environment. Of course, constant vigilance is 

required to assure that practice-oriented research is balanced with basic hypothesis-driven 

research. 

There was less evidence of interaction between the intramural scientists and the extramural 

Children’s Environmental Health Research Centers. A greater level of interaction between 

investigators in the externally funded University Centers and in-house researchers could result in 

more significant research progress (e.g., in the case of the potential role of GST polymorphisms 

in autism). 

Coordination With Outside Research Organizations

The Agency already has Congressional and Administrative mandates placed upon it, which it 

must accomplish. There has been good collaboration between investigators in ORD and in the 

NIEHS in areas relevant to LTG 1. A good example of this is the collaborative work showing 

that methylated arsenic metabolites generate oxygen radicals and damage DNA. This could be 

an important mechanism by which arsenic compounds contribute to the molecular mechanism of 

arsenic carcinogenesis. More such collaborations should be encouraged, where appropriate. 

EPA’s extramural grants program is well coordinated to extend capabilities of ORD scientists 

into needed research areas that are lacking within ORD. This promotes synergistic 

collaborations and avoids duplication of effort. The Human Health Research Program focuses 

on environmental health, rather than pure basic science, hence minimizing overlap with the NIH 

and other federal agencies. 

Utilization of Other Agencies To Advance EPA’s Research Agenda

EPA has established an Extramural Research Grants Program and an Extramural Environmental 

Health Centers Program, both of which utilize the efforts of scientists at research institutes and 

universities to aid the Agency in developing areas of toxicology and carcinogenesis that need to 

be explored beyond the Agency’s immediate capabilities. A specific example of this is studies 

by the extramural Children’s Environmental Health Centers in the mechanisms of autism in 

children and causes of autism in children. This is a very effective collaboration. 

EPA also has established a program by which outside scientists can be hired on a temporary 

basis (IPA, 4-year terms) to work inside the Agency and bring it new expertise. CRADAs also 

are available to extend the capabilities of the Agency by incorporating the expertise of external 

scientists as extramural contractors. The posters and publications of ORD scientists showed 

clear evidence that collaboration is occurring between Agency scientists and scientists from 

other governmental agencies (i.e., NIEHS). A listing of intergovernmental agency collaborations 

between ORD’s Human Health Research Program and its sister governmental agencies, 
Human Health Research Program Review Report 17 

however, was not provided during the review, so the full extent of this partnering could not be 

judged accurately and given appropriate credit. 

 

The Subcommittee was impressed by the high level of cooperation and teamwork among 

investigators and across organizations that was displayed in the Human Health Research 

Program. This collaborative atmosphere is particularly remarkable given the number of natural 

and organizational impediments that must typically be overcome to develop a team approach. 

Natural impediments include the natural tendency for scientists to pursue their personal research 

interests and extend their chosen areas of expertise. There also are likely to be difficulties in 

communication and collaboration between scientists in different disciplines (e.g., engineering, 

computer programming, statistics, and toxicology). In the case of ORD, it would be expected 

that these natural impediments might be exacerbated by the fact that the investigators come from 

several different organizations (e.g., NHEERL, NERL, etc.), which could result in conflicts over 

organizational responsibility (e.g., “turf battles”) and manpower/budget issues (e.g., time 

utilization, travel funds, etc.). It was clear from the presentations and discussions with Agency 

scientists that these impediments have to a large extent been overcome. ORD management is to 

be commended for the efforts that it is taking to maintain such an effective team research 

environment. 

The creation of the new National Center for Computational Toxicology may produce additional 

challenges in this regard. It appears that the most quantitatively oriented individuals in 

NHEERL and the most biologically oriented individuals in NERL will become part of the new 

Center. Although this reorganization apparently does not involve a change in location, there is a 

potential for the founding staff of the Center to become focused on the mission of the new 

organization and to some extent lose contact with the scientists in their former organizations. A 

high level of management attention will be required during the transition period to assure that 

current and future collaborations are not impeded by this change of organizational structure. 

Quality 

The Human Health Research Program of ORD uses the efforts of external advisory committees 

extensively to review its research products at regular intervals. Examples of this are the reviews 

of the drinking water research of ORD by the Drinking Water Committee of the SAB, reviews of 

the human health effects research by the Human Health Research Strategy Review Panel of the 

SAB, and reviews of the peer-reviewed manuscripts of scientists in the Human Health Research 

Program, as well as all programs in ORD by the Science and Technology Achievement Awards 

Committee. These reviews utilize external scientists to conduct peer review of the scientific 

accomplishments of scientists from ORD’s Human Health Research Program. 

The Agency ensures high quality research initially through the Extramural Research Funding 

Programs, which are highly competitive, merit-based funding. These programs do, however, 

need to be better advertised and perhaps even better financed and expanded to attract the widest 

possible competitive applicant pool. 

Funds are competitively awarded in the extramural grants program (Science To Achieve Results 

[STAR] Program). Each intramural investigator is reviewed internally every year. Established 
18 Human Health Research Program Review Report

programs are peer reviewed every 4-5 years at the level of the laboratory and at the level of 

ORD. Criteria for the review process are established by the Laboratory Director with approval of 

the Assistant Administrator of the Agency. For new initiatives, such as Computational 

Toxicology and the Children’s Health Initiative, there is an internal Request for Proposal (RFP) 

process. 

Overall, the work of the EPA scientists is of very high quality, particularly that on arsenic, 

dioxins, conazoles, and atrazines. The research on atrazine, in particular, demonstrates the high 

scientific quality of the research being conducted at the Agency. EPA scientists have conducted 

a high quality, hypothesis-derived research strategy that uses a systems biology approach. Here, 

investigations of the mode of action of atrazine are informed by an understanding of the 

underlying biological system. This work has provided important insights into the nature of the 

interaction of atrazine with crucial processes in the hypothalamus-pituitary-thyroid axis. 

Performance 

Research Themes

The research theme, use of mechanistic data to guide cancer and noncancer risk assessments, is 

clearly defined in the overview section. It is necessary and appropriate in LTG 1 for risk 

assessors and risk managers to use ORD’s methods and models to increase accuracy and 

decrease uncertainty in risk assessments. This will reduce the adverse health risks of humans 

exposed to environmental stressors. It is appropriate to ask the four questions in the overview 

regarding: (1) mode/mechanisms of action that are important to understand the impact of 

environmental stressors on human health; (2) attributes of the MOA that impact risk assessment; 

(3) how to measure, model, and/or predict key attributes of the MOA that impact risk 

assessment; and (4) how to incorporate mechanistic data and computational tools into risk 

assessment. The Human Health Research Program and all of ORD’s research programs clearly 

define use of mechanistic data in risk assessment. 

 

The research theme for LTG 1 is clearly addressed in several areas: 

1. EPA has taken a leadership role in developing a mode of action for chlortriazine pesticides, 

which they showed acted on the hypothalamic-pituitary-gonadal axis and blocked the LH 

surge, which blocked fertilization. Development of this MOA for chlortriazines allowed EPA 

scientists to predict the risk of adverse health outcomes to humans as a result of chlortriazine 

exposure by extrapolating from data in mice. 

2. NHEERL scientists, in collaboration with NIEHS scientists, made a pioneering contribution 

to the toxicology and carcinogenicity of arsenic showing that metabolism of arsenic to its 

methylated, reduced species [MMA(V), DMA(V), MMA(III), DMA(III)]—which generate 

oxygen radicals, damage DNA, and are genotoxic—is critical for its toxicity and 

carcinogenicity to be exerted. These important insights into molecular mechanisms of arsenic 

carcinogenesis advance cancer risk assessment for arsenic delivered by inhalation or 

ingestion routes of exposure. 
Human Health Research Program Review Report 19 

3. ORD scientists began a new initiative to study the role of oxidative stress in asthma, 

atherosclerosis, cancer, infertility, and neurodegenerative disorders. ORD developed 

methods to quantitate oxygen radicals in exposed tissues and showed dietary antioxidants 

lower levels of oxidative stress in rodents and humans. They are studying if and how chronic 

diseases, such as asthma, involve lowering antioxidant defenses and if biomarkers of 

oxidative stress, such as four single nucleotide polymorphisms (SNPs) in oxidative damagerelated genes, can predict adverse outcomes. They showed that in women in China exposed 

to polycyclic aromatic hydrocarbons, these SNPs correlate with an increased lung cancer 

risk. 

4. ORD scientists demonstrated that specific members of the conazole family of fungicides 

induce liver cancer, thyroid cancer, and/or reproductive effects, and investigated why these 

conazole members are carcinogenic. They found that metabolism of specific conazoles 

explains part of their tissue specificity but not patterns of thyroid tumor development. The 

scientists are working to determine molecular mechanisms of conazole-induced thyroid 

carcinogenesis. 

5. EPA also has been a leader in developing new cancer risk assessment guidelines 

incorporating all biologically relevant information, including data on MOA by which a 

chemical causes cancer in laboratory animals and/or humans. They finalized cancer risk 

assessment documents for chloroform, ethyl tertiary butyl ether, vinyl acetate, and 

formaldehyde. EPA scientists also initiated computational approaches to dose-response 

modeling through development of specific software that utilizes MOA data for cancer and 

noncancer outcomes, such as the benchmark dose software (BMDS). The BMDS is now 

utilized by 2,000 scientists from industry, academia, and governments in 80 countries. 

6. ORD scientists developed approaches to collect data from newer genomics technologies and 

convert them into mechanistic data to support the risk assessment process. This includes 

creation of the National Center for Computational Toxicology, as recommended by EPA’s 

own internal groups, the SAB’s Drinking Water Committee, and the Human Health Research 

Strategy Review Panel. 

In the research on dioxins and atrazines, EPA scientists supported the short-term needs of the 

regulatory offices while maintaining focus and programs on longer term, hypothesis-driven 

research geared toward elucidating mechanisms of adverse health effects. For dioxins, the risk 

assessment model was based on binding of dioxins to the Ah receptor and use of PBPK 

modeling to show that this MOA was concordant across a range of species, including rodents 

and humans. This research led to the interesting and logical conclusion that steady-state body 

burdens of dioxins should be used as the dose metric for cross-species extrapolation. This 

approach has been widely accepted. It is interesting and surprising that even after this exercise 

was completed on a carcinogen that binds to the Ah receptor to exert its effects, the models 

resulted in a linear extrapolation for cancer endpoints. It is even more interesting that linear 

extrapolation was found to be optimal for noncancer endpoints, based on biochemical and 

toxicological endpoints. EPA should be commended for its scientific leadership in resolving 

uncertainties for dioxin risk assessment. 
20 Human Health Research Program Review Report

ORD’s Program has clearly articulated its focus and the rationale behind its approach to study 

the use of mechanistic data in risk assessment (Research Theme #1). ORD has many demands 

placed upon it. ORD must satisfy the Agency’s strategic priorities, and must also support EPA’s 

program offices, regions, other governmental and nongovernmental organizations, and the public 

through scientific and technical advice and assistance. ORD also provides scientific leadership in 

identifying, studying, and resolving critical environmental health and ecological effects issues 

and in shaping the environmental health and ecological effects research agenda. ORD scientists 

do this primarily by developing data on the molecular and cellular mechanisms of 

carcinogenicity, neurotoxicity, and reproductive toxicity of chemicals of environmental interest. 

ORD scientists then provide these data to risk assessors and risk managers of EPA program and 

regional offices that need such data. 

The Human Health Research Program has clearly stated that its key research questions for 

mechanistic research are the following: (1) What MOAs are important for understanding the 

impact of environmental stressors on humans? (2) What are the attributes (e.g., shape of the 

dose-response curve, specificity of the MOA) that impact risk assessment? (3) How do we 

measure, model, or predict the key attributes of the MOA that could impact risk assessment? 

(4) How do we incorporate mechanistic data into risk assessment? These four questions clearly 

articulate the focus and rationale behind the approach of ORD’s Human Health Research 

Program to study this theme and to use the results for accurate risk assessments. 

There is evidence of integration across themes. A good generic example of this is the 

contribution of ORD scientists to multilaboratory, multiagency risk assessments based on data 

developed outside EPA. A specific example of this is the integrated characterization of human 

health and eco-toxicological risk for perchlorate based on the proposed MOA. This involved 

each center and laboratory within ORD, the program and regional offices, NIEHS, and the 

National Institute for Occupational Safety and Health (NIOSH). This effort is necessary because 

perchlorate causes toxicity, carcinogenesis, and developmental abnormalities as a result of the 

hypothyroidism that it can induce. 

The theme of development and use of mechanistic data in risk assessments is a logical and 

important theme that needs to be pursued by ORD to place the risk assessments on sound 

scientific footings. It is crucial to determine whether the risk assessments follow a linear or 

threshold model to make these risk assessments scientifically robust and accurate to enhance the 

confidence of scientists, stakeholders, regulatory agencies (e.g., the Office of Water and the 

Office of Pesticides), and the public in the credibility of these risk assessments. 

 

Use of a Multi-Year Plan 

The Human Health Research Program has a logical design. The SAB’s Human Health Research 

Strategy Review Panel thoroughly reviewed the program 2 years ago. As a result of this review, 

EPA’s Human Health Research Program was revised appropriately and is now very strong. The 

design of the program is logical and comprehensive, tempered with a necessary and appropriate 

focus on the important areas relevant to human health, the needs of EPA regions and program 

offices, and the Agency’s strategic needs as mandated by the Administration and Congress. The 

goals and the priorities of the Human Health Research Program are clear. There also are well- 
Human Health Research Program Review Report 21 

delineated schedules for the program to be implemented. There are some apparent discrepancies, 

however, between the specific projects and performance measures listed in the 2003 LTG 1 and 

the current suite of projects relevant to today’s Human Health Research Program and their 

deliverables and performance measures. This should be addressed by updating these items in the 

next iteration of the MYP. 

 

The MYP describes an appropriate flow of work within and across Research Theme #1, 

Development and Use of Mechanistic Data in Risk Assessment. This research theme of 

development of mechanistic data for the actions of environmentally relevant toxins and 

carcinogens has been well developed. There is collaboration between basic scientists and risk 

assessors and between these two collaborating groups and regulatory offices and policy makers. 

The resultant data generated then are passed in collaborative fashion to clients and stakeholders 

within and outside the Agency. 

The Human Health Research Program uses the MYP to address a logical sequence of questions. 

It also uses this plan as the basis for prioritizing its work. The MYP, however, is in need of 

updating to reflect current research activities. 

Progress to Meet the Long-Term Goals 

ORD has made significant progress toward each of its LTGs for the Human Health Research 

Program. Particularly impressive has been the progress made toward elucidating molecular 

mechanisms of arsenic carcinogenesis and arsenic toxicity, conazole carcinogenesis, and atrazine 

toxicity. Further significant progress has been made toward elucidating molecular mechanisms 

of dioxin toxicity and carcinogenesis. Risk assessments based on these mechanistic advances for 

arsenic, dioxin, and conazole carcinogenesis should shortly result in finalized cancer risk 

assessment documents for arsenic and dioxin. 

Certainly, within the areas of arsenic, dioxins, atrazines, conazoles, and LHs, the program is 

clearly addressing key research questions for each area. Little is known about mechanisms of 

conazole carcinogenesis. The research is showing that there may be novel, unique mechanisms 

of conazole-induced thyroid carcinogenesis. Rapid progress is being made in the area of LHs 

and carcinogenesis. It is impressive that this group of investigators utilizes a broad, 

interdisciplinary array of scientific approaches, including chemistry, biochemistry, cell biology, 

animal carcinogenicity studies, epidemiology and human studies, and population-based studies. 

It is noteworthy that in this program, toxicologists, molecular biologists, epidemiologists, and 

risk assessors collaborate effectively. 

Rationales to address all questions for the current suite of projects in the Human Health Research 

Program have been clearly articulated. Many of these rationales, based on elucidation of the 

mechanisms of action of toxic and carcinogenic chemicals, are straightforward and have been 

articulated properly. The rationale for many of the research questions derives directly from the 

needs of the program offices (Offices of Air, Water, and Pesticides) and the regional offices. As 

such, the rationales to address the questions have been clearly articulated and address meeting 

the objectives of these EPA offices for regulatory purposes. The end goal that is clearly 

articulated 
22 Human Health Research Program Review Report

throughout is to gather relevant data on mechanisms of toxicity and carcinogenesis for 

environmentally important chemicals to support risk assessments conducted on them. 

Research questions are being addressed in a reasonably timely manner. This is particularly true 

in the context that development of new scientific information occurs along a timeline that often 

embodies unexpected delays and changes in strategy based on unexpected research findings. A 

good example of this is the newly acquired data on the molecular mechanisms of the 

carcinogenicity of arsenic. This project took many years to come to fruition, because it has been 

a very difficult problem. EPA, however, has made rapid progress on this problem within the last 

5 years. 

Another good example of EPA progress in this area is the steady advancement that the Agency 

scientists have continued to make on understanding the molecular mechanisms of the toxicities 

and carcinogenicities of dioxin and dioxins. 

A third example of ORD scientists addressing questions in a timely manner is their work on 

elucidating the molecular mechanisms of the carcinogenicity of conazoles. EPA’s Office of 

Pesticides asked ORD for assistance to establish risk assessment procedures for conazole 

fungicides, a high priority for the Office of Pesticides. ORD responded rapidly to elucidate the 

molecular mechanisms of the toxicity and carcinogenicity of conazoles and has provided these 

data to the Office of Pesticides. In addition, an industrial working group, the Triazole Task 

Force, has collaborated with ORD in this effort. ORD researchers responded in a timely fashion 

to the necessity for regulation of conazoles by providing basic science data on mechanisms of 

action of conazoles for use in risk assessments for these chemicals. 

 

Use of Outputs by Stakeholders

The program has generally met stakeholder needs in a timely and useful way. Stakeholder needs 

often arise in a precipitous way, when no database exists on the chemicals in question and an 

environmental crisis arises. An example is the carcinogenicity of arsenic in drinking water. 

Progress was hindered because of lack of understanding of the molecular mechanisms of arsenic 

carcinogenesis. The work of ORD scientists gave significant insight into this problem by 

showing that arsenic needs to be methylated and reduced to MMA(III) and DMA(III), which 

then could generate oxygen radicals and cause DNA damage. These breakthroughs by ORD 

scientists are being utilized in the new risk assessment calculations for the carcinogenicity of 

arsenic in drinking water. These data eventually will be used by the Office of Water and also will 

aid the Office of Pesticides in regulating arsenical-containing pesticides. 

There was extensive confirmation that ORD scientists are helpful to the various EPA regions in 

terms of hosting regional scientists in ORD laboratories, collaborating on regional environmental 

problems, providing scientific consultation to help ameliorate regional environmental problems, 

and providing scientific consultation when they are asked by the public for information on 

specific problems in environmental toxicology that arise in the regions. According to testimony 

by the Region 5 RSL, by representatives from the Office of Water and the Office of Pesticide 

Programs, and by the Science Director of the Office of Children’s Health Protection, the research 

of ORD scientists in the Human Health Research Program has and continues to meet 
Human Health Research Program Review Report 23 

stakeholders’ needs in useful ways and in a timely manner. Studies on perturbations in breast 

development by exposure to dioxins and pesticides and the resultant increased cancer risk in 

children are examples of the utility and timeliness of the research efforts of ORD scientists. 

ORD has contributed to and collaborated with the Office of Children’s Health Protection in 

developing RFPs for targeted studies. 

The extramural Environmental Health Center Grants Program was discussed by the Director of 

one of these Centers (Rutgers, New Jersey). His testimonial made it clear that these Centers are 

making progress in identifying the causes of and understanding the molecular mechanisms of 

asthma, neurodevelopmental defects, autism, cancer, and other chemically induced adverse 

health effects in children. This research output is very useful to EPA’s Office of Children’s 

Health Protection, to the community of environmental health scientists, and to the public in 

general. 

The Human Health Research Program has been effective in developing outputs that support the 

risk assessment/risk management process. An important example is data generated by ORD 

showing that methylated arsenic metabolites [MMA(V), MMA(III), DMA(V), DMA(III), etc.] 

generate oxygen radicals and cause DNA damage in mammalian cells. These data will support 

the risk assessment calculations for arsenic cancer risk assessment and arsenic risk management 

processes to control arsenic in air and water. These data eventually will resolve if there is a 

threshold in the cancer risk assessment for arsenic in drinking water as a result of metabolism of 

arsenic, which follows a reductive methylation set of processes. Data generated by the TEF 

approach has supported risk assessment for dioxins. Additional research efforts by ORD 

scientists to understand MOA of and toxicity and carcinogenicity of mixtures of conazoles, 

carbamates, atrazine, and pyrethroids have aided and will continue to aid risk assessments for 

these chemicals. 

The panel heard extensive testimony from EPA offices and programs regarding the utility of 

research products developed by ORD scientists in the Human Health Research Program. 

Research on arsenic carcinogenesis by ORD researchers is used by the Office of Water and the 

Office of Pesticides and will play an important role in cancer risk assessments for arsenic in 

drinking water and in pesticide formulations. The finding by ORD scientists of the genotoxicity 

of methylated arsenic metabolites should answer the question of whether arsenic cancer risk 

assessment curves are linear or have a threshold. These important data will be used by the Office 

of Water and the Office of Pesticides. The Office of Pesticides is very enthusiastic about the 

research products from ORD on the toxicity and carcinogenicity of pesticides (e.g., conazoles, 

atrazines, carbamates, and pyrethroids). The Office of Water also is highly enthusiastic about 

the research products being developed by ORD scientists to aid it in regulating toxic and 

carcinogenic water contaminants, including arsenic, conazoles, and DBPs. 

The Region 5 RSL testified regarding the strong enthusiasm of all 10 regions for the value of the 

mechanistic data developed by ORD scientists and its use in mechanism-based risk assessments 

developed by ORD scientists. He described regional workshops (Emerging Pollutants 

Workshop, Chicago, 2003; Cumulative Risk Workshop, Dallas, 2002), seminars, regional 

methods initiatives, and regional research partnership programs in which ORD scientists 

collaborate with EPA regions, and regional personnel spend time in ORD laboratories working 
24 Human Health Research Program Review Report

with ORD scientists. These programs make strong connections between EPA regions and ORD 

scientists. ORD scientists clearly help the regions with scientific consultation and collaboration. 

Scientific Leadership 

The team of veteran scientific administrators provides professional leadership to the Human 

Health Research Program and ORD scientists in EPA. The NRC’s ROPE report stressed 

strengthening the core research of ORD within EPA and using this Core Research Program to aid 

EPA and its programs. ORD has worked hard to implement recommendations of the NRC report 

and demonstrated success in supporting key advances in human exposure and health effects 

research. ORD’s scientific leadership is strong. Research scientists in the Human Health 

Research Program are internationally recognized experts on dioxin toxicology and are applying 

their unique expertise to understand toxicities of other persistent chemicals, such as 

polychlorinated biphenyls and polychlorinated dibenzofurans. 

Researchers conducted studies to support LTG 1 of the Human Health Research Program have 

made significant contributions to the peer-reviewed scientific literature on MOA of the toxicity 

and carcinogenicity of environmental toxicants. The Subcommittee reviewed an extensive 

bibliography of publications derived from this program. ORD scientists also serve in key 

positions in advisory committees and technical panels for governmental and industrial research 

programs, in key positions for scientific societies, and in undergraduate and graduate teaching 

programs at universities. Several ORD scientists have won numerous scientific awards within 

and outside EPA and are well known nationally and internationally. Younger scientists at ORD 

should continue to develop and establish strong scientific reputations in human health research. 

ORD needs to do some transition planning to ensure a smooth transition to new leaders when 

senior leaders retire. 
Human Health Research Program Review Report 25 

IV. LONG-TERM GOAL 2:

AGGREGATE/CUMULATIVE RISK ASSESSMENT

Managerial excellence appears to be a hallmark of the overall Human Health Research Program. 

 Setting a tone and expectation that encourages, spawns, and awards problem-driven crossdiscipline teams at the operational level is a clear and positive result of this managerial effort. 

Indeed, the Subcommittee observed established networks in place to keep ORD researchers 

connected within the Agency, with particular reliance on interactions among lead 

scientists/principal investigators. Interactions with scientists and programs external to EPA are 

less formally organized but they appear to be appropriately happening at the lead 

scientist/principal investigator level. 

Additional evidence of outstanding managerial support is reflected in the overall high level of 

excitement, commitment, and passion displayed by the scientists in describing their work. It is 

suggested that this strong positive aspect of this program be explicitly recognized, continually 

rewarded, and carefully guarded. 

Another remarkably positive and valuable aspect of this program is a decisive propensity within 

the program to encourage the mining of available data and science to inform the risk assessment 

decisions of stakeholders. The Subcommittee encourages the use of consensual scientific 

deliberation by teams of experts to provide the best advice possible relative to the critical 

questions facing risk assessors and risk managers. 

The overall criteria and framework for decisions regarding why specific elements are vital and 

have been included in the research program were not clear to the panel. A more transparent 

explanation of these aspects would be most valuable. 

 

The public benefits from doing good science are not clearly or completely presented within the 

proposed work. Certainly, understanding and substantially reducing cancer and noncancer risks 

are vitally important and clearly recognized in this plan. The Subcommittee advises that it also 

is important and valuable that the technical work products be able to render the scientific 

determination of de minimus or acceptably low levels of human health risk from chemical 

exposure. The value of confident-knowledge regarding the relative safety offered by improved 

exposure and risk assessment tools of previously feared exposures and putative risks represents 

an arguably significant improvement to the public health. In addition to calming fear and its 

potential adverse consequences, these scientifically supported determinations allow for the 

continuing focus of finite resources on other potentially significant health risks. Some of the 

“delisting” examples in the plan point up these successful scientific determinations, but this 

factor is not articulated as a decided and very important benefit of the program. 

It appears as if EPA’s program and regional offices are involved in the planning and 

prioritization of the program’s research; however, the Subcommittee suggests a broadening of 
26 Human Health Research Program Review Report

this list to include other members listed as stakeholders. This would include qualified scientists 

with professional standing in the other stakeholder groups mentioned in the background material. 

To some extent, the direction, choice, and focus of research topics are undoubtedly areas where 

national politics interact and shape the development of specific scientific programs. The current 

focus of EPA ORD research on pesticides and only a relatively few substances has remained 

unchanged for some time and was the subject of some previous comments during Agency 

reviews in past years. The difference today is that there are regulatory mandates playing out in 

the rest of the world that are driving the overall scientific development of human health exposure 

assessment for the multitude of common and relatively unstudied substances to which humans 

are exposed. Significant resources are about to be committed, especially in the EU, to develop 

the exposure and risk assessment tools needed to reasonably accomplish these mandates. 

Because these mandates are quite different, the tools being developed will have a dissimilar 

perspective and approach than that displayed in the ORD Research Plan. Even if ORD is not 

designing this type of research it should monitor, engage in, and advise these research efforts of 

others. This commitment should happen with greater intensity and at a significantly higher level 

within the EPA research organization than has occurred to date. This will allow the Agency to 

advise and participate in the development of this particular piece of science. At the very least, 

scientists within ORD should contribute their considerable skill and knowledge to the EU 

research planning effort. Given this interaction and interchange, the outstanding scientific 

management extant within ORD might also benefit these international programs simply by virtue 

of its powerful example. This contact and participation would assure that the Agency’s scientific 

staff remains in touch with and knowledgeable about what is transpiring in this critical realm. It 

also would put them in an excellent position to enlist and act as full collaborating and 

operational partners in developing these tools if the decision is made to do so. 

Relevance 

The list of themes, topics, and poster titles, along with the other background materials, certainly 

contains all of the elements of the Agency’s strategic goals and the Human Health Research 

Strategy. Inspection indicates reasonable consistency and relevance. That is, the outlined work 

as presented in the background materials looks fine; however, these materials did not present 

enough substance or details relating the specific program elements to be able to reasonably 

conclude that the focus is consistent with the stated goals. Details, such as exactly how the work 

is going to be planned and processed, are critical and, although not presented in the premeeting 

materials, were in most cases clearly articulated during the meeting. 

The research efforts on cumulative risk assessment are highly relevant to the problems addressed 

by the Agency in assessing the risks faced by the public; people are typically exposed to multiple 

toxic chemicals. The high level of public concern, together with the large extent of uncertainty 

regarding the effects of mixtures, provides ample support for the Agency’s research emphasis in 

this area. 

The overall ORD research program for Aggregate/Cumulative Risk appears to remain focused 

on pesticides and a few other specific toxicants, such as dioxin, chlorinated solvents, metals, and 
Human Health Research Program Review Report 27 

glycol ethers. Focusing a program on pesticides and specific toxicants makes sense for the early 

efforts in the nascent field of exposure and risk assessment. These areas certainly are the most 

data rich and are well supported by a strong U.S.-based regulatory mandate, as well as the 

recommendations of various advising groups. There was no evidence, however, provided in the 

reference material or presented during the meeting of a general research effort to understand and 

evaluate the exposure and risk to the literally thousands of existing chemicals to which people 

are exposed today. 

It is reasonably well established that the risks of most types of personal chemical exposures are 

not being assessed at this point. These are the exposures that are happening predominately from 

residential exposure sources. Many, perhaps the vast majority, of these exposures and risks may 

be de minimus or insignificant; however, any scientific research plan designed to render answers 

about the aggregate and cumulative risk to humans from chemical exposure should reasonably 

address this significant portion of the total amount of chemical exposure experienced by humans. 

Any technical program that aspires to lead in the realm of human health risk assessment from 

chemicals should not ignore this reality. Similarly, any rational plan should have a specific and a 

systematic research strategy to address the multitude of substances to which individuals are 

exposed everyday. 

Given the well-defined source-exposure-dose-effect continuum that currently exists in the plan, 

the actual research should logically start with defining a critical taxonomy and characterization 

of the universe of sources extant or entering into typical human microenvironments. Given this 

universe, a reasonable number of hypothesis-driven models should be formulated and tested 

within it. All of this should be followed up with the development of fate and transport models to 

characterize the contact and delivery of these substances to people via various routes. 

Methods, Measures, and Models to Advance the Science of Aggregate and Cumulative Risk 

Assessment

This is an important effort focused on methods, measures, and models to advance the science of 

aggregate and cumulative risk assessment. It involves new analytical methods, measurements, 

and mathematical and computational models. The development of the Human Exposure 

Database System (HEDS) is very important and will capture data for human exposure to 

chemicals in a Web-based format, making it available to all investigators and stimulating further 

epidemiological studies of exposure to toxic chemicals and adverse health outcomes. The 

Consolidated Human Activity Database (CHAD) also is very important in compiling 

demographic information for each subject in questionnaire format and is very useful in 

exposure/intake dose modeling and/or statistical analysis. The National Human Exposure 

Assessment Survey, designed to evaluate comprehensive human exposure to multiple chemicals 

on a community and regional scale, also is a very important database, containing data on 

exposure factors from the pilot studies. This tool provides available data to exposure assessors 

so that they may summarize data on exposure factors for conducting human health exposure 

assessments. This database has provided consistency among exposure assessments, which is 

excellent. 

Component-based methods for investigating chemical interactions in toxicology are promising. 

Studies discussed with the Subcommittee showed new and interesting results. At lower 
28 Human Health Research Program Review Report

concentrations, close to those relevant to environmental exposure to pesticides, the studies find 

indicate that there is an additivity or antagonism in the hepatotoxicity of four trihalomethanes 

and for organochlorine pesticides that inhibit brain cholinesterase activity, when they are 

administered to animals as mixtures, depending on the mixing ratios and upon the concentrations 

administered to animals. For organochlorine pesticides, there were greater-than-additive 

(synergistic) activities of these chemicals for inhibition of blood and brain cholinesterase, 

inhibition of motor activity, and gait score. ORD scientists also studied TD chemicals and found 

that addition of mixtures of TD chemicals caused a dose-dependent decrease in thyroid hormone 

(T4) levels. They found that in the lower portion of the curve, at low doses, they obtained 

additivity for mixtures of TD chemicals. Interestingly, at higher doses, they found synergistic 

effects on thyroid hormone disruption. This is interesting and important work relevant to 

environmental exposures to these chemicals in mixing ratios to which humans are exposed. 

Case Studies and Risk Assessment Application of Aggregate and Cumulative Risk: An 

Overview 

An interesting presentation of cumulative risk assessment of DBPs showed that for DBPs acting 

through a common mechanism, the effects of these chemicals can be added to generate a 

cumulative dose, and a common, additive risk can be calculated. Starting with adjustment to 

internal animal dose, these investigators adjusted that dose to internal human equivalent dose 

(HED). PBPK modeling or allometric scaling generated an internal HED. This allowed these 

investigators to develop a dose-response curve for human responses. This approach has 

significant utility to estimate the risk to humans of DBPs as a mixture, which is very important. 

Source to-exposure pathways for dioxin-like compounds under the dioxin exposure initiative are 

a critical area of study. It is very important to delineate and quantify the pathways of dioxin 

exposure from all sources of dioxin release through the environment and into the food to assist 

development of a regulatory strategy for this important class of compounds. It is likely that 

these compounds cause cancer even at environmental levels, and this needs to be thoroughly 

studied. There is equivalent work ongoing that focuses on the use of the TEF methodology for 

cumulative risk for dioxins, with justified conclusions that TEF methodology provides a 

reasonable approximation of the toxic effects of a mixture of dioxin-like chemicals. These 

predictions are best for effects strictly mediated by the Ah receptor (enzyme induction). Thyroid 

hormone decrements are under predicted by the TEF methodology. 

Research on the relative potency factor assessment of organophosphate pesticides showed that 

the organophosphate group operated through a common mechanism of inhibition of 

acetylcholinesterase; therefore, the dose of congeners should be additive to predict toxicity of 

mixtures of these compounds. This research helps to answer important questions in this area and 

to make risk assessment for these compounds for neurotoxicity more reliable. 

The development of an ORD probabilistic exposure model—Stochastic Human Exposure and 

Dose Simulation (SHEDS) Model—allowed ORD to conduct a health risk assessment for 

children who contact chromated copper arsenic (CCA) on treated playsets and wooden decks. 

This useful work on applying SHEDS generated an exposure assessment model that was used by 

regulators in collaboration with ORD to generate accurate risk assessments for CCA-treated 
Human Health Research Program Review Report 29 

wood. Other important work from this group of scientists linked exposure measurements with 

human activity data to assess dose in human tissues by applying the exposure models. They used 

a PBPK model to determine interactions among and individual contributions of multiple 

chemicals from aggregate exposure on an absorption, distribution, metabolism, and excretion 

basis. This model accurately predicted clearance of trichloroethylene from the body and 

formation of 1,1,1-trichloroethane in the body. They used very important uncertainty analysis 

employing Monte Carlo Techniques and made good predictions. Another set of very important 

experiments showed that all carbamates produced dose- and time-dependent inhibition of 

cholinesterase and decreased motor activity. ORD scientists are working with PBPK models to 

address these questions. They found that for pyrethroids there was a common endpoint for use in 

cumulative risk calculations. They showed that adding in vitro clearance data improves the 

PBPK rat model. They eventually will generate models for use in determining the cumulative 

risk of carbamates and pyrethroids and allow EPA to conduct state-of-the-art cumulative 

assessment for carbamate and pyrethroids and reduced uncertainties in risk assessments for these 

two important classes of pesticides. 

Quality 

The research described in the area of cumulative risk assessment is particularly creative, 

effective, and well conducted. The Agency scientists have shown considerable innovation and 

skill in developing and applying quantitative techniques ranging from statistical analysis to 

PBPK modeling, as required for the various aspects of the research. The leadership of the ORD 

in defining research directions in the field of exposure assessment and risk assessment is 

evidenced by several large-scale efforts that include the National Human Exposure Assessment 

Survey and work focused on evaluating risks associated with exposure to multiple contaminants, 

as well as the development of HEDS, SHEDS, and CHAD. 

To the extent that Agency scientists typically seek to have their work published in peer-reviewed 

journals, the program appears to be functioning very well. In general, the program scientists 

have an excellent publication record and reputation for the quality of their publication 

submissions. 

Performance 

In conducting research on cumulative risk, ORD scientists have been able to simultaneously 

provide rapid response to the needs of the Agency’s regulatory program while still maintaining a 

strong, long-term research effort. The ability of the Agency to promote effective collaboration 

across disciplines (e.g., computer modeling, statistics, pharmacokinetics, health endpoint 

evaluation) and across organizations (e.g., NHEERL, NERL) is remarkable. The high 

effectiveness of the research program owes a great deal to the atmosphere of cooperation and 

teamwork that has been created and maintained by EPA management. It is crucial that efforts to 

maintain this highly effective environment be continued and perhaps increased when the new 

National Center for Computational Toxicology is created. 
30 Human Health Research Program Review Report

In general, there appears to be a concerted effort to provide excellent coordination and 

integration across the program’s research themes; however, there seems to be little integration of 

exposure assessment across themes that deal primarily with health effects. 

As mentioned above, the general outline and definition of the program and these themes appears 

to be reasonably well defined. The rational level of detail regarding how the program is 

specifically set up to identify and address critical issues is not obvious, however. 

The research theme aggregate/cumulative risk assessment is an overall term designed to include 

another critical theme of better exposure assessment models. It appears that this critical aspect 

may have gotten somewhat lost in the combination, and any reasonable focus or attention 

relative to exposure assessment model development under aggregate/cumulative risk is not 

plainly stated. 

Scientific Leadership 

The program definitely plays a leadership role in advancing the realm of scientific development 

in the areas that currently are addressed. The scientific staff and facilities are excellent, and the 

current funding appears to be adequate. Within this realm, the managers and researchers are 

overseeing and participating in a program that is leading the state-of-the-science. 

As noted above, there are specific scientific activities occurring in the EU that should receive 

intense Agency interest, interaction, and potential coordination. It is highly probable that 

specific elements of the ORD research effort will not overlap with these EU programs, but both 

entities could certainly benefit from such interaction. 

The EU has significant resources and a strong regulatory mandate for the assessment of existing 

chemicals. Health Canada also has a strong regulatory mandate to conduct exposure and risk 

assessments on literally thousands of existing chemicals in commerce. EPA interaction and 

potential collaboration with these programs should be coordinated at a much higher level within 

the Agency than is currently occurring. 

The human health research initiatives conducted by these nations could subsequently be viewed 

as being very relevant. A lack of significant EPA participation in these research efforts could 

possibly threaten the Agency’s current leadership position. 
Human Health Research Program Review Report 31 

V. LONG-TERM GOAL 3:

EVALUATION OF RISK TO SUSCEPTIBLE SUBPOPULATIONS

EPA’s overarching conceptual framework for the core human health research program that 

represents the long-term training goals and their interaction needs to be more clearly and fully 

developed. The conceptual models provided in EPA’s Human Health Research Strategy (i.e., 

Figures 1-3, 1-4, and 1-5) do not clearly represent its risk assessment context, the long-term 

research goals, or the importance of their interaction. The conceptual model should reflect the 

fundamental features of the core human health research program including: (1) it resides within 

a risk assessment framework; (2) the Long Term Research Goals; and (3) the strength of the 

program comes through interaction across LTGs. 

Related to the need for a more fully developed conceptual framework for EPA’s core human 

health research strategy is the need to provide a more clear health rationale. The actual research 

strategy appears well defined and appropriately directed, but its rationale (i.e., how and why this 

research has been selected) is less clear. To a large extent, ORD assumes its public health 

rationale based on the advice and consultation of external advisory groups including the National 

Academy of Sciences. Although the perspective of such science-based bodies is an important 

element to the rationale, ORD needs to clearly articulate its own rationale. At the level of the 

research program and the individual research project, the core Human Health Research Program 

can be strengthened by presenting a clear public health rationale. 

The Subcommittee noted that this program has emphasized/embraced the strength/benefits of 

multidisciplinary interaction within and between the researchers in the different LTGs. This is 

likely to be fertile ground for environmental health discovery. This interaction appears to be 

occurring and is a strength to the current program. EPA should acknowledge the importance of 

this interaction, take credit for it, and encourage its continued development. 

Peer review is recognized as a critical component of EPA’s Human Health Research Program. 

The program review process can be facilitated and enhanced by: (1) providing the 

reports/critiques from previous reviews; and (2) tailoring the EPA presentations to the review 

criteria and critiques from previous reviews. 

The core Human Health Research Program is effective and strategic in its coordination of its 

intramural and extramural research programs. These research programs are defined to achieve 

separate but complementary objectives. Both programs are effectively coordinated, resulting in 

good synergy with minimal redundancy. 

Effective participation of program and regional offices in research planning, execution, and 

evaluation is a major strength to the ORD Human Health Research Program. It is through this 

participation that EPA’s research is grounded in the practical needs of regulatory offices and 

communities. Furthermore, considerable talent and complementary expertise resides within 

program and regional offices that can only enhance and leverage ORD’s Human Health Research 

Program capability. Similar to the interaction between LTGs, this interaction already is 
32 Human Health Research Program Review Report

occurring but the strength of the interaction seems to vary across program offices. The 

Subcommittee noted that interaction with the Pesticides Program is especially effective. 

Relevance 

Consistency With the Agency’s Strategic Goals, the Human Health Research Strategy, and 

Recommendations of the National Research Council

Organization of EPA’s core Human Health Research Program around susceptible subpopulations 

is both relevant and strategic to environmental health and the establishment of a risk assessment 

scientific foundation. There is strong historical evidence that much of the environmental health 

impact is related to susceptible populations where exposures are excessive, individuals are 

biologically susceptible, or both (e.g., lead and chlorpyrifos). Because environmental exposures 

in general are quite low, population adverse health effects are likely only manifested on the tails 

of the exposure or biologically susceptible distribution. The need for research related to 

susceptible subpopulations has been identified as an important area where human health risk 

assessments could be improved by: (1) NRC’s 1994 Science and Judgment in Risk Assessment; 

(2) NRC’s 1993 Pesticide in the Diets of Infants and Children; (3) the Food Quality Protection 

Act (FQPA) of 1996; and (4) the Safe Drinking Water Act of 1996. Whereas ORD was effective 

in describing the relevance of this LTG to ORD’s Strategic Plans, as well as the Human Health 

Research Plan, it failed to discuss this particular LTG relative to the NRC core research 

priorities. 

EPA has identified children as the susceptible subpopulation of interest. Emphasis on children 

(which includes all early life stages beginning at gestation) as a susceptible population is 

appropriate in view of the high level of societal and Congressional interest, as well as the fact 

that toxicity testing has traditionally been performed primarily on adult animals and may not be 

optimal for detecting effects in early life. Furthermore, this research focus is broadly supported 

by EPA program offices, the NRC, and legislative concerns. The products of the research will 

not only improve the Agency’s ability to perform risk assessments that appropriately consider 

children’s health, but they will also provide data that will be useful to many other agencies and 

health research organizations. 

Although the Agency’s focus on children as a susceptible population subgroup appears well 

justified, the justification presented was largely based on a consensus of recommendations across 

external advisory bodies (e.g., Office of Pollution Prevention and Toxics, NRC). This 

justification can be strengthened by the Agency’s own scientific assessment of the public health 

benefit to be achieved through a research focus on children as a particular subpopulation. Such 

justification is likely to become more important in considering future potential subpopulation 

research foci that may be less obvious than children. 

ORD has identified susceptible and highly exposed groups defined by: life stage, genetic 

factors, and/or health status. Intramural and extramural research within this LTG spans a wide 

range of research categories of significant environmental health relevance including: (1) 

pesticides and children; (2) modeling source-to-effects in children; (3) exposure assessment 

support to the National Children’s Study; (4) risk assessment/risk management; (5) aging; (6) 

asthma; and 
Human Health Research Program Review Report 33 

(7) uniquely vulnerable populations. Within each of these categories, research activities appear 

strategic and systematic, addressing short-term methodological needs while developing longer 

term community-based measurement and modeling approaches. This research is appropriately 

focused and directed to satisfy critical research needs particularly related to children as a 

susceptible subpopulation. Furthermore, there appears to be effective coordination and 

interaction within and between both intramural and extramural research programs. 

The National Children’s Study is a very interesting study, with study locations selected with the 

help of expert statisticians to provide an appropriate sample. The potential benefits of this 

research are numerous and can lead to mitigation of exposures that contribute to adverse health 

effects in children. It is expected that in approximately 7 years, information from these projects 

will begin to be published in the peer-reviewed literature as part of this long-term (20 years or 

more) study. A series of exposure projects to support the National Children’s Study have been 

proposed. These efforts marry the unique capabilities of EPA in analytical chemistry with its 

capabilities in epidemiology. Newer techniques, such as satellite sensing to detect toxic air 

pollutants and toxic pollutants in rivers on a geographic basis, and novel sensing methodologies 

also will be incorporated into this study. It is a broad-based study dedicated to answering the 

question, “Can we identify exposures that correlate with adverse health effects on children?” 

Initially, EPA will look broadly to determine correlations. Although this study will take a long 

time to complete, it is an appropriate use of ORD resources. 

The Agency’s asthma research involves establishment of a new Cell Biology Group, which 

should be a dynamic group with very strong leadership. The finding of this group that exposure 

of transgenic IL-5 mice to diesel combustion products caused airflow obstruction in the presence 

of a methacholine challenge is very interesting. There is a very good set of collaborations here 

between immunologists, cell biologists, and engineers. It would be very useful to have regular 

group meetings within ORD and for the group to participate in interagency meetings at which 

regulatory representatives from EPA are present, to strengthen the focus of this group. 

Research related to susceptibility from aging is focused on a limited number of pollutants. Chief 

among these are trichloroethylene (TCE), benzene, pesticides, and particulate matter (PM10). It 

appears that this group meets frequently and is very active in attending meetings of the Society 

of Gerontology and other aging societies and presenting its research at these meetings. It is 

important that this group meets with the Children’s Health Group at least once each year in a 

Super-Group of Susceptible Populations to share data and approaches and determine whether 

common approaches can be adopted by both groups in studying these two types of susceptible 

populations. 

The studies described in the area of source-to-effects modeling of early life exposure have 

primarily focused on the kinetic half of the modeling spectrum, from source to target-tissue dose. 

Future efforts should begin to extend quantitative evaluation into the area of dynamics, from 

target-tissue dose to response. The creation of the new National Center for Computational 

Toxicology will provide the opportunity to apply the latest genomic and systems biology 

techniques to investigate child-adult differences in tissue response. 
34 Human Health Research Program Review Report

Stakeholder Involvement

There are clear indications that EPA human health research stakeholders, including program and 

regional offices, are substantively involved in the research planning and prioritization for the 

program, although it also appears that the extent of involvement varies by region and program 

office. Stakeholder participation in research planning and prioritization provides a clear and 

compelling advantage to ORD’s Human Health Research Program in assuring research relevance 

and providing research opportunity. Involvement of regional and program office stakeholders 

varies. The Office of Pollution Prevention and Toxics is very effectively involved in acquiring 

research findings and in planning and defining the research agenda, but involvement of other 

program offices, such as the Office of Air and Radiation and the Office of Drinking Water, and 

regional offices does not appear to be as consistently strong. This variability, however, may 

stem from the relevance of the program office to the core Human Health Research Program. 

Community-based research permeates LTG 3 and is one of its strengths. There is growing 

recognition of the value of community-based participatory research as a means for conducting 

such research. Although STAR grantees have embraced principles of community-based 

participatory research, there appears to be little or no ORD intramural capacity or expertise. If 

community research continues to be a part of this LTG (as it should), ORD should acquire this 

intramural capability. 

Coordination With Outside Research Organizations

The LTG 3 research activities appear to be well coordinated with outside research organizations 

(nationally and internationally), providing two significant benefits: minimizing duplication and 

leveraging research opportunities. A good example of this coordination is the National 

Children’s Study where EPA is substantively involved and contributing to the projects. 

Effective and substantive collaboration also are occurring with the U.S. Food and Drug 

Administration (FDA), the U.S. Department of Housing and Urban Development (HUD), and the 

Centers for Disease Control and Prevention (CDC). In general, research coordination appears to 

occur at two levels and varies across the program. The most effective level of coordination 

occurs informally at the level of the bench scientist. Laboratory managers play an important role 

in enabling and encouraging this interaction. The second level of coordination occurs at the 

level of the laboratory manager where research programs are coordinated more generally and at a 

broader scale. Information about research coordination was largely available through 

conversations with scientists and managers. This feature of ORD’s Human Health Research 

Program is a strength that should be more prominently described and presented. 

Quality and Performance 

In general, the quality and performance of research under LTG 3 is very high. The research is 

characterized by hallmarks of quality and success, including its multidisciplinary focus and the 

extensive publication of the research in high-quality peer-reviewed technical journals. Over the 

5-year period 1999-2005, there were more than 500 peer-reviewed articles published, providing 

an indication of the productivity of this research program. The studies described in the area of 

source-to-effects modeling of early life exposure have been very effectively planned and 
Human Health Research Program Review Report 35 

conducted. The scientific quality of the research in this area provides a standard of excellence 

for investigations of other susceptible populations. 

Research within LTG 3 is effective in systematically addressing a wide range of research issues 

associated with improving risk assessment for susceptible populations. Research is being 

conducted to assess children’s exposure, susceptibility, and differential risks from pesticides. 

This work relies on a variety of approaches, including laboratory, exposure, and epidemiologic 

studies to inform children’s pesticide susceptibility and risk. These measurement-based studies 

form the basis for the development and/or validation of sophisticated exposure to dose models. 

These modeling efforts are state-of-the-art and are effective in providing a quantitative 

conceptual framework for estimating exposure and risk and elucidating data and research gaps. 

Plans are underway to link the source-to-exposure-to-dose models with PBPK modeling efforts 

to derive comprehensive models for source-to-effect modeling. EPA has played a productive 

and strategic role in the multi-agency National Children’s Study initiative. Under LTG 3, EPA 

has contributed substantively in conducting pilot studies to inform planning related to exposure 

measurement methods. Although children are a current focus of research within LTG 3, ORD 

also appropriately recognizes the elderly as an important susceptible group. ORD is conducting 

appropriate initial work to effectively build this program, applying many of the relevant concepts 

and approaches from its children’s work. Susceptibility as a result of asthma is well justified 

because of this disease’s high prevalence rate and evidence that indoor and outdoor air pollutants 

play an important but uncertain role in its occurrence. Given emerging evidence of the 

significance of particulate matter, the complexities of exposure to air pollution mixtures, and 

exposure assessment issues associated with indoor allergens, ORD is poised to provide a 

significant research contribution. 

Scientific Leadership 

ORD scientists involved in the research on children’s susceptibility are internationally 

recognized experts in children’s environmental health and play a strong role in fostering a 

continuing emphasis on this research area. EPA’s leadership in children’s exposure assessment 

is especially valuable because this research focus appears to be unique to EPA. Very significant 

contributions are being provided related to methods, measurements, and models for susceptible 

population exposure. 

 

Scientific leadership is demonstrated at multiple levels within this LTG. First and foremost, 

much of the research being conducted is cutting edge. This is particularly true within research 

realms of relevance to the Agency regulatory purview (e.g., methods, measurements, and models 

for assessing children’s exposure to pesticides). ORD’s scientists and their research agendas are 

clearly out front and defining the science in a number of areas under this LTG. Through 

effective use of its intramural and extramural research programs, ORD is defining the state-ofthe-art in the development of measurements and models for children’s exposure assessment, 

including biological monitoring, quantifying activity patterns, and residential measurement 

methods. This developmental work feeds into observational studies providing rare objective 

measures of children’s pesticide exposure through multiple pathways. At the same time, ORD is 

providing leadership with research to better understand the biological basis for children’s 

differential sensitivity to pesticides. Significant public health discoveries are emerging from 

ORD’s 
36 Human Health Research Program Review Report

extramural research program from epidemiologic studies identifying significant associations 

between mothers’ pesticide exposure and birth outcomes. ORD scientists are taking on a 

valuable and significant leadership role with the National Children’s Study in defining methods 

for exposure assessment and the development and implementation of pilot projects to address 

key data gaps necessary to optimally design and implement the study. 
Human Health Research Program Review Report 37 

VI. LONG-TERM GOAL 4:

EVALUATION OF PUBLIC HEALTH OUTCOMES

Relevance 

ORD carries out the research activities of the Agency and provides the scientific basis upon 

which its programs, policies, and regulations are developed. In articulating the LTGs for 

research over a 10-year period, ORD’s Human Health Research Strategy (EPA, 2003a) and 

Human Health MYP (EPA, 2003b) have identified an area of focus on public health outcomes. 

Research on the evaluation of public health outcomes builds upon an earlier initiative to develop 

indicators for measuring improvements in human and ecological health and thereby demonstrates 

quantifiable benefits of the Agency’s policies and regulatory actions (EPA, 2003c). Evaluating 

public health outcomes associated with environmental policies and regulations is consistent with 

the overall mission of the Agency to demonstrate that EPA is protecting human health, as well as 

with the accountability directive issued by former EPA Administrator Christine Whitman. The 

program also is consistent with the outcomes-based approach that is being applied in many areas 

of public health. Given the shift in the Agency to develop environmental health indicators to 

support decision making within the Agency, a new area of research is being developed within 

ORD to address this issue. 

The program on public health outcomes is being built upon the same conceptual framework that 

supports the three other major areas of research, namely the exposure-dose-effects continuum. 

Unlike the programs that support LTGs 1, 2, and 3, however, this program will not only benefit 

from integration with other programs, but such integration is viewed as critical for its success. 

To facilitate coordination, it is recommended that a mechanism be put into place that has both 

formal and informal components, not only to promote dialogue but also to elaborate a process for 

evaluating research outputs as suitable inputs for the activities carried out by the program. 

Within this rubric, the Human Health Research Program has the potential—in the future—of 

providing the nucleus for evaluating the research of the ORD, in terms of its relevance to 

environmental health. As an emerging area of research with a clear public health focus, it will 

be incumbent upon the leadership to highlight the relevance and importance of this program as it 

relates to the research carried out in the areas of: (1) harmonization of cancer and noncancer risk 

assessments; (2) aggregate and cumulative exposure and risk; and (3) susceptible 

subpopulations. 

 

In addition to the need for substantive interaction between the Public Health Outcomes Program 

with the three other core areas of the Human Health Research Program, collaborations with other 

public health entities that monitor trends in environmental quality and human health (e.g., the 

CDC or the Agency for Toxic Substances and Disease Registry) are critical. With respect to 

interagency interactions, a memorandum of understanding (MOU) has already been established 

between EPA (via ORD and the Office of Environmental Information) and the CDC (signed 

September 30, 2003) to create a partnership to develop an environmental health tracking network 

at the national level. As part of the MOU, a pilot project has been launched (the Public Health 

Air Surveillance Evaluation [PHASE] project) to evaluate existing air quality and public health 
38 Human Health Research Program Review Report

surveillance data and determine if useful linkages between the two data sources can be 

established. In addition to EPA (NERL and the Office of Air Quality, Planning, and Standards) 

and the CDC, collaborators in the PHASE project include state health departments from New 

York, Maine, and Wisconsin. Other collaborative activities should be identified to allow for the 

sharing of expertise and for leveraging effort across agencies. 

Quality 

The primary goal of the research program on evaluation of public health outcomes is to develop 

and apply reliable measures to assess the effectiveness of Agency policies and regulations in 

reducing human health risks. Clearly, this LTG is overarching and relates to every action and 

program that supports the EPA mission to improve public health. Given the magnitude of the 

scope of this evaluation, it is recommended that the program specify focused goals that will 

guide its activities over the near term, as well as articulate a process for making decisions 

regarding which action to evaluate, which health endpoint to study, and, most importantly, which 

environmental health indicator to apply. Without question, the greatest challenge will lie in 

developing, selecting, and applying environmental health indicators that might provide the 

linkages between risk management decisions and specific health endpoints. 

There is evidence that a deliberative process has guided the initial activities of the program. For 

example, the pilot project initiated under the MOU established between EPA and the CDC was 

selected on the basis of its applicability and relevance. The involvement of state health 

departments enhances the project, as they (together with their partners at the regional, county, 

and city level) will play an important role in evaluating existing data, identifying data gaps, and 

facilitating changes in data collection methods to better meet the needs of efforts to assess public 

health outcomes. With respect to the intramural “Demonstration” project that was initiated 

recently, the Subcommittee was informed that criteria will be developed for evaluating and 

selecting projects on the basis of quality, relevance, and feasibility over the short term. It is 

recommended that these criteria be made explicit and communicated to the program and regional 

offices so that projects are developed and selected with the greatest potential for success. The 

importance of these early efforts is underscored, as they will provide the lessons that may serve 

to catapult the program into the future. 

Performance 

The evaluation of public health outcomes as a result of policies of the Agency is a relatively new 

research program within ORD and, therefore, cannot be assessed using the same evaluation 

criteria as those used for assessing the other components of the Human Health Research 

Program. Several surveys of existing databases already have been carried out, including an 

inventory of exposure databases conducted by the National Risk Management Research 

Laboratory (NRMRL) as part of the annual performance measures supporting the LTG of the 

public health outcomes research agenda. The intramural program has been launched with a call 

for preproposals from the program and regional offices to develop the tools to evaluate public 

 health outcomes associated with policies or actions. In addition, a Request for Applications 

(RFA) is being drafted with NCER to focus on studies to develop and establish linkages among 
Human Health Research Program Review Report 39 

monitoring, exposure, and health effects databases that might be used to develop indicators of 

the effectiveness of risk management decisions made by the Agency. 

The program is new and, thus, has shared a relatively small proportion of the available budget of 

ORD. In the MYP (EPA, 2003b), less than 7 percent of the FY2004 budget earmarked for 

human health research was allocated for public health outcomes research. Likewise, a relatively 

small percentage (< 4%) of the total full-time equivalents committed to the Human Health 

Research Program was directly involved in public health outcomes research. The program has 

experienced increases in the budget for FY2005, with monies allocated to support the intramural 

“Demonstration projects”, as well as an RFA (approximately $1 million) to focus on studies that 

evaluate the utility of existing monitoring, exposure, and public health data for developing 

indicators of the effectiveness of risk management decisions. 

It is anticipated that as the program matures, additional monies will need to be made available to 

provide adequate support for the activities that are conducted. In addition, the program will 

likely require additional expertise that currently is not represented within ORD. Specifically, it 

is anticipated that the program would benefit from biostatistical support and additional expertise 

in environmental epidemiology. Additional areas of needed expertise may emerge as the 

program develops. 

Scientific Leadership 

Current leadership for the program has the appropriate background and professional leadership 

to take the Human Health Research Program to the next steps. It is recommended that as the 

program expands, recruiting scientists with expertise in the area of evaluating public health 

outcomes would strengthen the leadership of the program. As the program gains definition, it is 

also recommended that ORD solicit external review of its activities on a periodic basis to aid the 

leadership in evaluating the program’s activities as they relate to short-term goals and in 

articulating the scope of activities that are likely to allow the program to achieve its LTGs. 

References 

U.S. Environmental Protection Agency. Human Health Research Strategy. Washington, DC: 

Office of Research and Development, 2003a, EPA/600/R-02/050. 

U.S. Environmental Protection Agency. Human Health Multi-Year Plan. Washington, DC: 

Office of Research and Development, 2003b. 

U.S. Environmental Protection Agency. Draft Report on the Environment. Washington, DC: 

Office of Research and Development, 2003c, EPA 260-R-02-006. 
40 Human Health Research Program Review Report

VII. TESTIMONIALS

Testimony on the interactions of the Human Health Research Program with EPA’s regional 

offices, other programs, and extramurally supported investigators was provided during the faceto-face site visit of the Subcommittee. According to testimony by the Region 5 RSL, by 

representatives from the Office of Water and the Office of Pesticide Programs, and by the 

Science Director of the Office of Children’s Health Protection, the research of ORD scientists in 

the Human Health Research Program has met and continues to meet stakeholders’ needs in 

useful ways and in a timely manner. Effective participation of program and regional offices in 

research planning, execution, and evaluation is clearly a major factor in maintaining the 

relevance of the ORD research program. It is through this participation that EPA’s research is 

grounded in the practical needs of regulatory offices and communities. Furthermore, 

considerable talent and complementary expertise resides within program and regional offices 

that can only enhance and leverage ORD’s capability. Like the interaction between LTGs, this 

interaction is routinely occurring, although the strength of the interaction seems to vary across 

program offices. The Subcommittee noted that interaction is particularly effective with the 

Pesticides Program. 

 

The panel heard extensive testimony from EPA offices and programs regarding the utility of 

research products developed by ORD scientists in the Human Health Research Program. 

Research on arsenic carcinogenesis by ORD researchers is used by the Office of Water and the 

Office of Pesticides, and it will play an important role in cancer risk assessments for arsenic in 

drinking water and in pesticide formulations. The Office of Pesticides is very enthusiastic about 

the research products from ORD on the toxicity and carcinogenicity of pesticides (conazoles, 

atrazines, carbamates, and pyrethroids). The Office of Water also is highly enthusiastic about 

the research products being developed by ORD scientists to aid it in regulating toxic and 

carcinogenic water contaminants, including arsenic, conazoles, and DBPs. ORD also has 

contributed to the goals of the Office of Children’s Health Protection and has collaborated with 

them in developing RFPs for targeted studies. 

There was extensive confirmation that ORD scientists are helpful to the various EPA regions in 

terms of hosting region scientists in ORD laboratories, collaborating with the regions on regional 

environmental problems, providing scientific consultation to the regions to help ameliorate their 

environmental problems, and providing scientific consultation to the regions when they are 

asked by the public for information on specific problems in environmental toxicology that arise 

in the regions. The Region 5 RSL testified regarding the strong enthusiasm of all 10 regions for 

the value of the mechanistic data developed by ORD scientists and its use in mechanism-based 

risk assessments developed by ORD scientists. He described regional workshops (Emerging 

Pollutants Workshop, Chicago, 2003; Cumulative Risk Workshop, Dallas, 2002), seminars, 

regional methods initiatives, and regional research partnership programs in which ORD scientists 

collaborate with EPA regions, and regional personnel spend time in laboratories of ORD 

scientists. These programs make strong connections between EPA regions and ORD scientists. 

The Subcommittee concluded that ORD scientists clearly help the regions with scientific 

consultation and collaboration. 
Human Health Research Program Review Report 41 

There was less evidence of an interaction of the in-house program with the extramural Children’s 

Centers. A greater level of interaction might result in useful in-house research initiatives, for 

example in the area of the potential role of GST polymorphisms in autism. EPA has established 

an Extramural Research Grants Program and an Extramural Environmental Health Centers 

Program, both of which utilize the efforts of scientists at research institutes and universities to 

aid the Agency in developing areas of toxicology and carcinogenesis that need to be explored 

beyond EPA’s immediate capabilities. The extramural Environmental Health Center Grants 

Program was discussed by the Director of one of these Centers (Rutgers, New Jersey). His 

testimony made it clear that these Centers are making progress in identifying the causes of and 

understanding the molecular mechanisms of asthma, neurodevelopmental defects, autism, 

cancer, and other chemically induced adverse health effects in children. This research output is 

very useful to the Office of Children’s Health Protection, to the community of environmental 

health scientists, and to the public in general. Specific examples of this are studies by the 

extramural Environmental Health Centers in the mechanisms of autism in children and causes of 

autism in children. A greater level of interaction between the externally funded University 

Centers and in-house research could result in more significant research progress, for instance, in 

the case of the potential role of GST polymorphisms in autism. 

Summary

Several presentations were provided on the interactions of ORD with other EPA offices and 

programs, including the Extramural Research Program, the Office of Pesticide Programs, the 

regional offices, and the Program on Children’s Environmental Health. It is clear from these 

presentations that the program is very closely coupled with the program and regional offices. 

Many examples were provided of the benefits of this partnering. Particularly laudable examples 

include the continuing support ORD has been providing to the Office of Pesticides to support 

cumulative exposure and risk assessment, the interactions with regional scientists through the 

RARE Program, and the crucial role of ORD in the Program on Children’s Environmental 

Health. 

These strong and valuable interactions could not take place without the willingness of the ORD 

scientists and the support of ORD management. The Agency is to be commended for 

maintaining such a highly collaborative environment. It must be recognized, of course, that 

constant vigilance is required to assure that the amount of time spent on these collaborative 

efforts does not interfere with the productivity of the hypothesis-driven research program. 
42 Human Health Research Program Review Report

VIII. APPENDIX A:

CHARGE QUESTIONS

DRAFT Charge for the Human Health Research Program 

1.0 Objective. The objective of this review is to evaluate the relevance, quality, 

performance, and scientific leadership of the Office of Research and Development’s (ORD’s) 

Human Health Research Program. The independent external peer-review panel=s evaluation and 

recommendations will provide guidance to ORD to help: 

Ê Plan, implement, and strengthen the program; 

Ê Compare the Human Health Research Program with other programs designed to achieve 

similar outcomes in other parts of the Environmental Protection Agency (the Agency) and in 

other federal agencies; 

Ê Make research investment decisions over the next 5 years; 

Ê Prepare the Agency’s performance and accountability reports to Congress under the 

Government Performance and Results Act; and 

Ê Respond to evaluations of federal research, such as those conducted by the Office of 

Management and Budget (which highlights the value of recommendations from independent 

expert panels in guidance to federal agencies1,2). 

2.0 Background Information. Independent expert review is used extensively in industry, 

federal agencies, Congressional committees, and academia. The National Academy of Sciences 

has recommended this approach for evaluating federal research programs.3

 

Because of the nature of research, it is not easy to measure the creation of new knowledge as it 

develops or the pace at which research progresses or scientific breakthroughs occur. 

Demonstrating research contributions to outcomes is very challenging4

 when federal agencies 

conduct research to support regulatory decisions and then rely on third parties5

 (such as state 

environmental agencies) to enforce the regulations and demonstrate environmental 

improvements. Typically, many years may be required for practical research applications to be 

developed, and decades may be required for some research public benefit outcomes to be 

achieved. 

Most of the Agency’s environmental research programs investigate complex environmental 

problems and processes, combining use-inspired basic research6,7 with applied research and 

integrating several scientific disciplines across a conceptual framework8

 that links research to 

environmental decisions or outcomes. In multidisciplinary research programs such as these, 

progress toward outcomes cannot be measured by outputs created in a single year. Rather, 
Human Health Research Program Review Report 43 

research progress occurs over several years, as research teams explore hypotheses with 

individual studies, interpret research findings, and then develop hypotheses for future 

investigations. 

In designing and managing its research programs, ORD emphasizes the importance of 

identifying priority research questions or topics to guide the research directions. Similarly, ORD 

recommends that its programs develop a small number of performance goals that serve as 

indicators of progress to answer the priority questions and to accomplish outcomes. Short-term 

outcomes are accomplished when research is applied by specific clients to strengthen 

environmental decisions or regulations. These decisions and resulting actions (e.g., reducing or 

preventing exposure of humans to environmental stressors posing a high risk) ultimately 

contribute to the improved health of the American public. 

In a comprehensive evaluation of science and research at the Agency, the National Research 

Council recommended9

 that EPA substantially increase its efforts to explain the significance of 

its research products and to assist clients inside and outside the Agency in applying them. In 

response to this recommendation, ORD has engaged science advisors from client organizations 

to serve as members of its research program teams. These teams help identify research 

contributions with significant decision-making value and help plan for their transfer and 

application. 

For the Agency’s environmental research programs, periodic retrospective analysis at intervals 

of 4 or 5 years is needed to characterize research progress, to identify when clients are applying 

research to strengthen environmental decisions, and to evaluate client feedback about the 

research. Conducting program evaluation at this interval enables assessment of research 

progress, the scientific quality and decision-making value of the research, and if research 

progress has resulted in short-term outcomes for specific clients. 

A description of the Office of Science and Technology Policy/Office of Management and Budget

Research and Development Investment Criteria is included in Appendix I. These investment 

criteria of relevance, quality, performance, and leadership of the scientific program on human 

health risk assessment are pertinent to the draft charge questions. 

3.0 Draft Charge Questions for ORD=s Human Health Research Program

The following charge questions will help evaluate the relevance, quality, performance, and 

scientific leadership of ORD’s human health research: 

Relevance

1. Is the focus of ORD’s Human Health Research Program relevant to and consistent with the 

Agency’s strategic goals, the Agency’s Human Health Research Strategy, and 

recommendations for core research priorities developed by the National Research Council? 

2. Are potential public benefits of the program clearly articulated? 

3. Are stakeholders (e.g., program and regional offices) involved in the planning and 

prioritization of the research? 
44 Human Health Research Program Review Report

4. Is the program well coordinated with outside research organizations, nationally and 

internationally, to avoid duplication of effort and promote synergistic collaboration? 

5. To what extent has EPA established and utilized other agencies (inside and outside the 

government) in advancing EPA’s research agenda? What are the impediments, if any, to 

collaboration with other organizations? 

Quality

1. Does the program use peer review to ensure the quality of its products? 

2. Does the program ensure high-quality research through competitive, merit-based funding? 

3. If funds are not competitively awarded, what process does the program use to allocate funds? 

Does this process ensure that quality is maintained? 

Performance

Research Themes

1. Are the four research themes (i.e., use of mechanistic data in risk assessment, 

aggregate/cumulative risk, susceptible subpopulations, and evaluation of public health 

outcomes) clearly defined? 

2. Has ORD’s program clearly articulated its focus and the rationale behind its approach to 

study the four research themes? 

3. Is there evidence of integration across themes? 

4. Do these four themes represent a logical framework for organizing the research and for 

identifying long-term goals that meet the needs of the Agency, science, and program 

customers? 

Use of a Multi-Year Plan 

1. Does the program have a logical, comprehensive design and Multi-Year Plan (MYP) with 

clear goals, schedules, and priorities? 

2. Does the MYP describe an appropriate flow of work within and across the research themes? 

3. Does the program use this MYP to address a logical sequence of questions and does it use 

the plan as a basis for prioritizing its work? 

Progress to Meet the Long-Term Goals 

1. Has the program made significant progress toward each of the long-term goals? 

2. Does the research address the key research questions? 

3. Is the rationale to address the questions clearly articulated? 

4. Are the questions being addressed in a timely manner? 

Use of Outputs by Stakeholders 

1. Has the program met stakeholder needs in a timely and useful way? 

2. Has the program been effective in developing outputs that support the risk assessment/risk 

management process? 

3. Are outputs from the program used by stakeholders? 
Human Health Research Program Review Report 45 

Scientific Leadership 

1. Has the program played a leadership role in advancing the state-of-the-science of human 

health research and solving important research problems? 

2. Have ORD’s human health researchers demonstrated leadership in their respective human 

health disciplines? 

 

4.0 Potential Peer-Review Panel Approach for Program Review

Ê Hold up to two conference calls in the month preceding a face-to-face meeting. 

• Allows ORD to present background materials to the peer-review panel. 

• Allows the peer-review panel to review and comment on the charge. 

• Allows the peer-review panel to ask clarifying questions about the program under review. 

Ê The Contractor shall distribute background materials and documents requested by the peer-review 

panel in advance of the progress review. 

Ê The peer-review panel Chair makes review and writing assignments to panel members in advance 

of a face-to-face meeting. 

Ê Hold a 2-3 day face-to-face meeting for the program review at a location where a critical mass of 

ORD scientists is located. 

• The first 2 days of the meeting will involve ORD presentations and poster sessions. 

• On the morning of the 3rd day of the meeting, the peer-review panel prepares a draft report that 

addresses all of the charge questions. 

• It is a goal to have a draft report available for circulation among the panel members and 

comment at the end of the face-to-face meeting. 

Ê If needed, hold one to two conference calls to finalize the report within 1 month after the face-toface meeting. 

• It is a goal to have a final report approved by the peer-review panel available to ORD within 1 

month following the face-to-face meeting. 

References

1

 Budget Data Request 04-31. Executive Office of the President, Office of Management and 

Budget. March 22, 2004. “Completing the Program Assessment Rating Tool (PART) for 

the FY06 Review Process,” pp. 50-56. 

2

 Memorandum for the Heads of Executive Departments and Agencies. Executive Office of the 

President, Office of Management and Budget. June 5, 2003. “FY 2005 Interagency 

Research and Development Priorities,” pp. 5-10.
46 Human Health Research Program Review Report

3 Evaluating Federal Research Under the Government Performance and Results Act, National 

Research Council, 1999.

4 The House Science Subcommittee. Letter to Dr. Bruce Alberts, President of the National 

Academy of Sciences, from F. James Sensenbrenner, Jr. and George E. Brown, October 

23, 1997. 

5 The Government Performance and Results Act: 1997 Governmentwide Implementation Will 

Be Uneven. U.S. General Accounting Office, GAO/GGD, 1997.

6 Building a Foundation for Sound Environmental Decisions. (National Research Council, 1997). 

7 “Renewing the Compact between Science and Government,” Stokes DE, in 1995 Forum 

Proceedings, Vannevar Bush II—Science for the 21st Century, pp. 15-32, Sigma Xi, 1995. 

8

 Risk Assessment in the Federal Government: Managing the Process, National Research 

Council, 1983.

9 Strengthening Science at the U.S. Environmental Protection Agency, National Research 

Council, 2000, p. 141.
 

Appendix I: 

OSTP/OMB Research and Development 

Investment Criteria 

(Provided in hardcopy only) 
IX. APPENDIX B:

MEETING AGENDA

48 Human Health Research Program Review Report

U.S. EPA BOARD OF SCIENTIFIC COUNSELORS 

Human Health Subcommittee Meeting 

AGENDA 

February 28 - March 2, 2005 

 Environmental Protection Agency 

Room C-111A/B/C 

109 T.W. Alexander Drive 

B 

• 

O 

• 

S 

• 

C Research Triangle Park, NC 27711 

Monday, February 28, 2005

8:00-8:30 a.m. Registration 

8:30-8:40 a.m. Welcome and Opening Remarks Dr. James Klaunig 

Chair, Human Health (HH) 

 Subcommittee 

Dr. James Clark, 

Vice-Chair, HH Subcommittee 

8:40-8:45 a.m. DFO Welcome and Charge Virginia Houk (EPA) 

 - Administrative Procedures and DFO, HH Subcommittee 

 FACA Rules 

- Objective of This Subcommittee 

 and Charge 

8:45-8:50 a.m. ORD’s Welcome Dr. William Farland (EPA) 

Acting DAA–Science, ORD 

8:50-9:10 a.m. Introduction to the Review of ORD’s Dr. Lawrence Reiter (EPA) 

HH Research Program Director, NHEERL 

9:10-9:45 a.m. Overview of the HH Research Program Dr. Larry Cupitt (EPA) 

 ORD/NERL 

 
Human Health Research Program Review Report 49 

HH Research Program LTG 1: Use of Mechanistic Data in Risk Assessment 

9:45-10:15 a.m. LTG 1: Overview - Use of Mechanistic Dr. Julian Preston (EPA) 

 Data in Risk Assessment ORD/NHEERL 

10:15-10:30 a.m. Break 

 

10:30 a.m.-12:00 noon LTG 1: Poster Session (Atrium) HH Subcommittee 

12:00-12:30 p.m. LTG 1: Discussion HH Subcommittee 

12:30-1:30 p.m. Working Lunch HH Subcommittee 

HH Research Program LTG 2: Aggregate/Cumulative Risk

1:30-2:00 p.m. LTG 2: Overview - Aggregate/ Dr. Jerry Blancato (EPA) 

Cumulative Risk ORD/NERL 

2:00-3:30 p.m. LTG 2: Poster Session (Atrium) HH Subcommittee 

3:30-4:00 p.m. LTG 2: Discussion HH Subcommittee 

4:00-4:15 p.m. Break 

4:15-5:30 p.m. Discussion and Work Session HH Subcommittee 

- Work on Draft Report 

5:30 p.m. Adjourn 

Tuesday, March 1, 2005 

8:30-8:40 a.m. Review of Yesterday’s Activities Dr. James Klaunig 

Overview of Today’s Agenda Chair, HH Subcommittee 

HH Research Program LTG 3: Susceptible Subpopulations

 

8:40-9:10 a.m. LTG 3: Overview - Susceptible Dr. John Vandenberg (EPA) 

 Subpopulations ORD/NCEA 

9:10-11:15 a.m. LTG 3: Poster Session (Atrium) HH Subcommittee 

11:15-11:30 a.m. Break 

11:30 a.m.-12:15 p.m. LTG 3: Discussion HH Subcommittee 

12:15-1:30 p.m. Working Lunch HH Subcommittee 

 
50 Human Health Research Program Review Report

HH Research Program LTG 4: Evaluating Public Health Outcomes

1:30-1:50 p.m. LTG 4: Overview - Evaluating Public Dr. Hal Zenick (EPA) 

Health Outcomes ORD/NHEERL 

1:50-2:10 p.m. LTG 4: Discussion HH Subcommittee 

2:10-2:25 p.m. Public Comments 

2:25-2:40 p.m. Break 

2:40-3:10 p.m. Relevance of HH Research Program Dr. George Lambert 

EPA Extramural Perspective Robert Wood Johnson 

Medical School 

3:10-5:30 p.m. Discussion and Work Session HH Subcommittee 

 - Work on Draft Report 

 

5:30 p.m. Adjourn 

Wednesday, March 2, 2005 

8:30-8:40 a.m. Review of Prior Day’s Activities Dr. James Klaunig 

Overview of Today’s Agenda Chair, HH Subcommittee 

8:40-9:00 a.m. Relevance of HH Research Program Dr. Randy Perfetti (EPA) 

EPA Program Office Perspective Office of Science 

Coordination and Policy 

9:00-9:20 a.m. Relevance of HH Research Program Dr. David Macarus (EPA) 

EPA Regional Perspective Region 5 

9:20-9:40 a.m. Relevance of HH Research Program Dr. Michael Firestone (EPA) 

EPA Office of Children’s Health OCHP 

 Protection (OCHP) Perspective 

9:40-9:55 a.m. Break 

 

9:55-11:00 a.m. Discussion and Work Session HH Subcommittee 

- Develop Oral Report 

 

11:00 a.m.-12:00 noon Oral Report on Charge Questions HH Subcommittee 

12:00 noon Adjourn 
Human Health Research Program Review Report 51 

X. APPENDIX C:

LIST OF ACRONYMS

BMDS Benchmark Dose Software 

BOSC Board of Scientific Counselors 

CDC Centers for Disease Control and Prevention 

CHAD Consolidated Human Activity Database 

CRADA Cooperative Research and Development Agreement 

DFO Designated Federal Officer 

EPA U.S. Environmental Protection Agency 

EU European Union 

FACA Federal Advisory Committee Act 

FDA U.S. Food and Drug Administration 

GPRA Government Performance and Results Act 

GST Glutathione S-Transferase 

HED Human Equivalent Dose 

HEDS Human Exposure Database System’ 

HUD U.S. Department of Housing and Urban Development 

LH Luteinizing Hormone 

LTG Long-Term Goal 

MOA Modes/Mechanisms of Action 

MOU Memorandum of Understanding 

MYP Multi-Year Plan 

NERL National Exposure Research Laboratory 

NHEERL National Health and Environmental Effects Research Laboratory 

NIEHS National Institute of Environmental Health Sciences 

NIH National Institutes of Health 

NIOSH National Institute for Occupational Safety and Health 

NRC National Research Council 

NRMRL National Risk Management Research Laboratory 

NTP National Toxicology Program 

OMB Office of Management and Budget 

ORD Office of Research and Development 

PHASE Public Health Air Surveillance Evaluation 

PBPK Physiologically Based Pharmacokinetic 

PM Particulate Matter 

RARE Regional Applied Research Effort 

RFA Request for Applications 

RFP Request for Proposals 

ROPE Research Opportunities and Priorities for EPA 

RSL Regional Science Liaison 

SAB Science Advisory Board 

SHEDS Stochastic Human Exposure and Dose Simulation 

SNPs Single Nucleotide Polymorphisms 

STAR Science To Achieve Results 

TCE trichloroethylene 

TEF Toxicity Equivalency Factor