U.S. ENVIRONMENTAL PROTECTION AGENCY

OFFICE OF INSPECTOR GENERAL

Improvements to EPA 

Policies and Guidance 

Could Enhance Protection 

of Human Study Subjects 

Report No. 14-P-0154 March 31, 2014 

Scan this mobile 

code to learn more 

about the EPA OIG. 
 

 

 

 

 

Report Contributors: Rick Beusse 

 Chris Dunlap 

 Kevin Good 

 Alicia Mariscal 

 Renee McGhee-Lenart 

 Julie Narimatsu

 Tempestt Woodard 

Abbreviations 

CAA Clean Air Act 

CAPS Concentrated Air Particles 

CFR Code of Federal Regulations 

CITI Collaborative IRB Training Initiative 

DHRO Division Human Research Officer 

EPA U.S. Environmental Protection Agency 

GAO Government Accountability Office 

HRPO Human Research Protocol Office 

HSRB Human Studies Review Board 

HSRRO Human Studies Research Review Official 

IRB Institutional Review Board 

NAAQS National Ambient Air Quality Standards 

NHEERL National Health and Environmental Effects Research Laboratory 

OHRE Office of Human Research Ethics 

OIG Office of Inspector General 

ORD Office of Research and Development 

PM Particulate matter 

PM2.5 Fine particulate matter 

SOP Standard operating procedure 

μg/m3 Micrograms per cubic meter of air 

UNC University of North Carolina at Chapel Hill 

Cover photo:	 A human subjects air pollution test chamber used during the OMEGACON 

study in the U.S. EPA Human Studies Facility in Chapel Hill, North Carolina. 

(EPA OIG photo) 

Hotline Suggestions for Audits or Evaluations 

To report fraud, waste or abuse, contact To make suggestions for audits or evaluations, 

us through one of the following methods: contact us through one of the following methods: 

email: OIG_Hotline@epa.gov email: OIG_WEBCOMMENTS@epa.gov 

phone: 1-888-546-8740 phone: 1-202-566-2391 

fax: 1-202-566-2599 fax: 1-202-566-2599 

online: http://www.epa.gov/oig/hotline.htm online: http://www.epa.gov/oig/contact.html#Full_Info 

write:	 EPA Inspector General Hotline write: EPA Inspector General Hotline 

1200 Pennsylvania Avenue, NW 1200 Pennsylvania Avenue, NW 

Mailcode 2431T Mailcode 2431T 

Washington, DC 20460 Washington, DC 20460 
 

 

 

U.S. Environmental Protection Agency 14-P-0154 

March 31, 2014 Office of Inspector General 

At a Glance 

Why We Did This Review 

In response to a congressional 

request, we conducted this 

review to determine whether 

the U.S. Environmental 

Protection Agency (EPA) 

followed applicable laws, 

regulations, policies, 

procedures and guidance when 

it exposed human subjects to 

diesel exhaust emissions or 

concentrated airborne particles. 

In particular, we reviewed five 

studies that the EPA conducted 

during 2010 and 2011 to 

determine whether the agency 

(1) obtained sufficient approval 

to conduct these studies; 

(2) obtained adequate informed 

consent from the human study 

subjects; and (3) adequately 

addressed adverse events that 

occurred during the studies. 

The EPA’s human studies are 

governed by 40 Code of 

Federal Regulations (CFR) 

Part 26, also known as the 

Common Rule, which 

establishes minimum 

standards. The EPA conducts 

human research studies to 

better understand the health 

effects of pollution on humans. 

This report addresses the 

following EPA theme: 

 Addressing climate change 

and improving air quality. 

For further information, 

contact our public affairs office 

at (202) 566-2391. 

The full report is at: 

www.epa.gov/oig/reports/2014/ 

20140331-14-P-0154.pdf 

Improvements to EPA Policies and Guidance Could 

Enhance Protection of Human Study Subjects

 What We Found 

The EPA followed applicable regulations when The EPA can enhance its 

it exposed 81 human study subjects to human studies by improving 

concentrated airborne particles or diesel how it obtains approval for 

studies; how it communicates exhaust emissions in five EPA studies 

risk to people who participate conducted during 2010 and 2011. However, we in EPA studies; and how it identified improvements that could be made to addresses adverse events in the EPA’s policies and guidance to enhance its guidance. protection of study subjects. 

The EPA obtained approval to conduct the five human research studies, including 

approval from a biomedical Institutional Review Board (IRB) and the EPA Human 

Studies Research Review Official (HSRRO). However, the EPA’s policies and 

guidance do not address when HSRRO approval is needed for significant study 

modifications. Developing guidance for when HSRRO must approve significant 

modifications would ensure their independent review. 

The EPA obtained informed consent from the 81 human study subjects before 

exposing them to pollutants. While the consent forms met the requirements of 

40 CFR Part 26, we found that exposure risks were not always consistently 

represented. Further, the EPA did not include information on long-term cancer 

risks in its diesel exhaust studies’ consent forms. An EPA manager considered 

these long-term risks minimal for short-term study exposures. We believe 

presenting consistent information about risks further ensures that study subjects 

can make the most informed choice about participating in a study. 

The EPA addressed six adverse events during its studies, reported them to the 

IRB, and provided clinical follow-up after the events. While the clinical follow-up 

appeared to be reasonable, the EPA’s policies, guidance and consent forms do not 

establish the EPA’s clinical follow-up responsibilities. According to EPA managers, 

the agency uses the latest University of North Carolina at Chapel Hill (UNC) IRB’s 

adverse event definitions and reporting timeframes to respond to adverse events. 

However, the agency’s guidance provides different definitions and reporting 

timeframes and does not state that the EPA has adopted the UNC-IRB definitions 

and timeframes. Using EPA’s guidance, the EPA reported two of the six adverse 

events later than required and did not report two other events to IRB. 

 Recommendations and Planned Corrective Actions 

We recommend that the EPA establish procedures for obtaining HSRRO approval 

of significant study modifications, ensure consent forms consistently address 

pollutant risks, update its guidance to include the EPA’s clinical follow-up 

responsibilities, and address a number of other recommendations. The EPA 

concurred with all recommendations and provided planned corrective actions and 

completion dates that meet the intent of the recommendations. All 

recommendations have been resolved. 
 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY 

WASHINGTON, D.C. 20460 

THE INSPECTOR GENERAL 

March 31, 2014 

MEMORANDUM 

SUBJECT:	 Improvements to EPA Policies and Guidance Could Enhance Protection of 

Human Study Subjects 

 Report No. 14-P-0154 

FROM:	 Arthur A. Elkins Jr. 

TO:	 Lek Kadeli, Acting Assistant Administrator 

 Office of Research and Development 

This is our report on the subject evaluation conducted by the Office of Inspector General (OIG) of the 

U.S. Environmental Protection Agency (EPA). This report contains findings that describe the problems 

the OIG has identified and corrective actions the OIG recommends. This report represents the opinion of 

the OIG and does not necessarily represent the final EPA position. Final determinations on matters in 

this report will be made by EPA managers in accordance with established audit resolution procedures. 

The EPA office having primary responsibility over the issues evaluated in this report is the Office of 

Research and Development’s National Health and Environmental Effects Research Laboratory. 

Action Required 

The agency agreed with all recommendations and provided planned corrective actions and completion 

dates that meet the intent of these recommendations. Therefore, the agency is not required to provide a 

written response for these recommendations. Recommendation 2 is closed and no further action is 

required. For the remaining recommendations, which are in an open status, please update the EPA’s 

Management Audit Tracking System as you complete the planned corrective actions. Please notify my 

staff if there is a significant change in the agreed-to corrective actions. Should you choose to provide a 

response to this final report, we will post your response on the OIG’s public website, along with our 

memorandum commenting on your response. You should provide your response as an Adobe PDF file 

that complies with the accessibility requirements of Section 508 of the Rehabilitation Act of 1973, as 

amended. 

We will post this report to our website at http://www.epa.gov/oig. 

If you or your staff have any questions regarding this report, please contact Carolyn Copper, 

Assistant Inspector General for Program Evaluation, at (202) 566-0829 or copper.carolyn@epa.gov; 

or Rick Beusse, Director, Air Evaluations, at (919) 541-5747 or beusse.rick@epa.gov. 
 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Improvements to EPA Policies and Guidance Could 14-P-0154 

Enhance Protection of Human Study Subjects 

Table of Contents 

Chapters 

1 Introduction ........................................................................................................ 1

 Purpose ....................................................................................................... 1 

Background ................................................................................................. 1 

 Scope and Methodology .............................................................................. 9 

2 Studies Obtained Required Approvals, but Sequence of Approvals Not 

Followed and Procedures Could Be Improved................................................ 12
 

Required Approvals Obtained to Conduct Five Human Research Studies, 

 but Sequence of Approvals Did Not Follow NHEERL Guidance .......... 12 

EPA Could Strengthen Protection of Human Subjects by 

 Developing Guidance Concerning Study Modifications ........................ 15
 

Formal Ethics Training Requirements Met, but Procedure 

 Needed to Track Annual Continuing Education Requirements ............ 16 

One Human Research Study Exposed a Study Subject Above 

 Concentration Targets but Followed Approved Protocol ...................... 16 

Conclusions................................................................................................. 17 

Recommendations ...................................................................................... 17 

Agency Comments and OIG Evaluation ...................................................... 18 

3 EPA Obtained Informed Consent From Human Subjects, but 

Consent Forms Addressed Pollutant Exposure Risks Differently ................ 19 

EPA Followed Requirements of 40 CFR 26.116 ......................................... 19 

Consent Forms Addressed Risks of Studies’ Medical Procedures ............. 21 

Studies’ Consent Forms Addressed Risks of 

 Pollutant Exposures Inconsistently ....................................................... 21 

Consent Forms for Diesel Exhaust Studies Did Not Include 

 Potential Cancer Risks From Long-Term Exposure ............................. 23 

Conclusions................................................................................................. 24 

 Recommendations ...................................................................................... 25 

Agency Comments and OIG Evaluation ...................................................... 25 

4 Improvements Needed in EPA Guidance to Address 

Adverse Event Definitions, Reporting Timeframes and 

Clinical Follow-Up Responsibilities.................................................................. 26 

Clinical Follow-Up Provided but Guidance and Study Protocols 

Lack Clinical Follow-Up Responsibilities .............................................. 26 

NHEERL Guidance Does Not Reflect the Adverse Event Definitions 

 and the Reporting Timeframes the Agency Reports Using ................. 29 

Other Matters .............................................................................................. 32 

Conclusions................................................................................................. 32 

 Recommendation ........................................................................................ 33 

Agency Comments and OIG Evaluation ...................................................... 33 

-continued- 
 

 

Improvements to EPA Policies and Guidance Could 14-P-0154 

Enhance Protection of Human Study Subjects 

Status of Recommendations and Potential Monetary Benefits.............................. 34 

Appendices 

A EPA Air Quality Index for Particle Pollution .................................................... 36
 

B Agency Comments on Draft Report ................................................................. 37
 

C OIG Evaluation of Agency Comments.............................................................. 42
 

D Definitions of Adverse Events and Unanticipated Problems......................... 45 

E Distribution ......................................................................................................... 46 
 

 

 

Chapter 1

Introduction 

Purpose 

In response to a congressional request, the Office of Inspector General (OIG) 

conducted a review of the U.S. Environmental Protection Agency’s (EPA’s) 

research involving human subjects to determine whether the EPA followed 

applicable laws, regulations, policies, procedures and guidance when it exposed 

human subjects to concentrated airborne particles or diesel exhaust emissions. Our 

specific objectives were to determine whether the EPA, in conducting five studies1 

that exposed human subjects to diesel exhaust emissions or Concentrated Air 

Particles (CAPS) during 2010 and 2011: 

1) Obtained sufficient approval to expose subjects to specific levels of diesel 

exhaust emissions or concentrated airborne particles; 

2) Obtained adequate informed consent from human study subjects before 

exposing them to diesel exhaust emissions or concentrated airborne 

particles; and 

3) Adequately addressed any adverse events that occurred, including: 

(a) Notifying the University of North Carolina at Chapel Hill’s 

(UNC’s) Institutional Review Board (IRB), the Human Studies 

Review Board (HSRB) and the EPA Human Subjects Research 

Review Official (HSRRO); 

(b) Revising consent forms as needed; and 

(c) Providing clinical follow-up in accordance with the approved 

protocol. 

Background 

What Is Human Subjects Research? 

According to the EPA, any project that collects data from or about humans may 

constitute human subjects research. As such, it would be subject to EPA-issued 

regulations regarding the protection of human subjects under Title 40, Part 26 of 

the Code of Federal Regulations (CFR). Under these regulations, the EPA defines 

“human subject” as a “living individual about whom an investigator conducting 

1 The studies were entitled XCON, KINGCON, OMEGACON, DEPOZ and LAMARCK. 

14-P-0154 1 
 

 

 

research obtains 1) data through intervention or interaction with the individual, or 

2) identifiable private information.” Interventions include physical procedures or 

manipulations of the subject or the subject’s environment by which data are 

gathered. Interaction includes communication or interpersonal contact between 

investigator and subject. Research is defined as a “systematic investigation... 

designed to develop or contribute to generalizable knowledge.” 

The EPA has been conducting controlled exposure studies for about 40 years. In 

controlled exposure studies, human subjects are intentionally exposed to 

pollutants under controlled conditions. These studies allow investigators to isolate 

and explain health events related to such exposures. According to National Health 

and Environmental Effects Research Laboratory (NHEERL) guidance, the studies 

can also help estimate “safe threshold exposures for humans.” The EPA is among 

15 federal agencies2

 that have adopted rules governing the protection of human 

subjects used in this and other types of human subjects research. 

Why Does the EPA Conduct Research Involving Human Subjects? 

The Clean Air Act (CAA) authorizes the EPA Administrator to establish a 

national research and development program for the prevention and control of air 

pollution. According to the CAA, the Administrator shall conduct a research 

program on the short-term and long-term effects of air pollutants on human 

health. When conducting the research program, the Act states that the 

Administrator shall conduct studies, including epidemiological, clinical and 

laboratory and field studies as necessary to identify and evaluate exposure to, and 

effects of, air pollutants on human health. The Administrator has established this 

program within the agency’s Office of Research and Development (ORD). 

The CAA amendments of 1990 require the EPA to set and periodically review 

National Ambient Air Quality Standards (NAAQS) for pollutants considered 

harmful to public health and the environment. The EPA’s controlled exposure 

studies help set these standards by identifying exposure-response relationships 

and providing more information on how the body interacts with particular 

pollutants. The EPA’s Office of Research and Development prepares multiyear 

plans in conjunction with the Office of Air and Radiation to determine the various 

human research studies that ORD will need to conduct to answer its key research 

questions. From fiscal years 2009 to 2013, the agency spent an average of about 

$3.46 million per year, or slightly over $17.3 million for the 5 years, and 

maintained about 19 full-time equivalents (FTEs) to support the EPA’s human 

subjects research conducted by ORD’s Clinical Studies Branch. 

2

According to the Department of Health and Human Services, 15 federal agencies adopted the Common Rule in 

1991 by issuing regulations. 

14-P-0154 2 
 

 

 

EPA’s Human Studies Facility 

The EPA’s Human Studies Facility, located on the UNC campus, is primarily 

intended for research to support EPA standards and regulations and is equipped to 

study the health effects of airborne pollutants on humans. The facility has the 

capability to deliver gaseous pollutants at precise concentrations across a broad 

range of atmospheric conditions. Human subjects research exposure systems used 

by EPA include two small (36 square feet) and two large (300 square feet) study 

chambers and one neurophysiological test room. 

A small chamber (36 square feet) used for diesel studies (left) and a large chamber 

(300 square feet) used for ozone studies (right). (EPA OIG photos) 

The facility also has several exam rooms 

where EPA staff conduct physical exams on 

study subjects prior to their participation in a 

study. According to ORD managers, the 

agency uses the physical exams to exclude 

individuals who might be at risk for 

experiencing an adverse event during the 

study. 

Key Criteria Governing EPA Research 

Involving Human Subjects 

The EPA conducts research involving 

intentional exposures of human subjects to 

pollutants. The agency performs this 

research under a number of statutory, 

regulatory and agency orders, policies and 

guidance. 

Exam room where NHEERL conducts 

initial and follow-up physical exams on 

study participants. (EPA OIG photo) 

14-P-0154 3 
 

 

Title 40 CFR 26 

Title 40 CFR 26 (the Common Rule) provides the regulatory framework under 

which the EPA conducts research involving human subjects. The regulation 

requires that the agency provide “written assurance” that it will comply with the 

Common Rule. The assurance must include, among other things, a “statement of 

principles” governing its responsibilities for protecting human subjects. It must 

also include the designation of an IRB to review research proposals. Additionally, 

it must include procedures for ensuring that unanticipated problems involving 

risks to subjects are reported to the IRB and other agency officials. The EPA 

possessed an active Federalwide Assurance from the Department of Health and 

Human Services to conduct human subject research during 2010 and 2011. 

The regulation also sets forth the requirements for an IRB. These include the 

membership, functions and operations, and criteria for approval of research. In 

addition, the regulation sets requirements for informed consent. According to the 

regulation, the investigator must give the prospective study subject sufficient 

opportunity to consider whether or not to participate. In addition, the regulation 

requires that the consent form: 

	 Present the information in language understandable to the subject. 

	 Provide a description of any reasonably foreseeable risks or discomforts to 

the subject. 

	 Disclose alternative courses of treatment, benefits to the subject and 

identification of any experimental procedures. 

	 Contain a statement that participation is voluntary. 

The regulation also requires that the prospective study subject be provided a copy 

of the consent form. 

EPA Order 1000.17 Change A1 

The agency issued EPA Order 1000.17 in 1977. In 1999, the policy was replaced 

with EPA Order 1000.17 Change A1, and it was amended in 2011 to ensure that 

the EPA complied with the Common Rule. It requires that the EPA’s HSRRO 

approve all research involving human subjects conducted or supported by the 

EPA unless otherwise exempt. The Order establishes a presumption that “studies 

involving risk of substantial injury to a human subject from the conduct of the 

study and that studies testing for irreversible health effects in humans will not be 

approved..., unless strongly persuasive additional justification acceptable to the 

Review Official (HSRRO) is submitted.” 

14-P-0154 4 
 

 

National Health and Environmental Effects Research Laboratory Policy 

and Guidance 

ORD’s NHEERL is the only EPA laboratory that conducts controlled human 

exposure studies. The Laboratory issued a policy in 2004 and updated guidance in 

2010 governing NHEERL human subjects research activities. The policy, along 

with the EPA Order, requires that all human subjects research be approved or 

determined to be exempt by the HSRRO. The HSRRO is independent from and 

not organizationally located within NHEERL. The NHEERL Human Research 

Guidance provides more detailed information on the NHEERL approval process. 

This includes which officials and staff must review the protocol prior to IRB 

submission. It also includes the special reviews required for controlled exposure 

studies. According to NHEERL guidance, any full scale controlled-exposure 

study of subjects to known pollutants must be reviewed by two external reviewers 

and receive a medical review if there is more than minimal risk. Once the IRB 

approves the study protocol, a study justification document is prepared which 

describes the risks to study participants versus the benefits to society, as well as 

why the study could not be conducted using animals. The entire package is then 

reviewed and approved by the Director of the Human Research Protocol Office 

(HRPO), the Environmental Public Health Division Director, the NHEERL 

Associate Director for Health, and the HSRRO. 

The NHEERL guidance also provides detailed information on the informed 

consent process. For example, the document states that “informed consent is a 

process, not a form.” Information must be presented in a way to allow subjects to 

decide voluntarily whether to participate. It must use “lay language,” especially in 

describing the study purpose, duration, risks and benefits. The document suggests 

language be at the 8th to 10th grade reading level. 

NHEERL guidance defines an adverse event as undesirable and unintended, 

though not necessarily unanticipated, injury or physical or emotional consequence 

to a human subject. These events can be serious and/or unanticipated. They also 

have different reporting requirements according to their severity. (Table 9 in 

Chapter 4 provides detailed definitions for each of these events). The more 

serious the event, the more quickly the principal investigator must report the event 

to the IRB. The report must be written and include information for the IRB to 

“judge whether or not the event raises new questions about risks to participants or 

the research design.” 

Health Impacts and Exposure Levels for Two Key Air Pollutants 

ORD has conducted research involving intentional exposures of human subjects 

to fine particulate matter and diesel exhaust for over 10 years. Based on years of 

research, the EPA has established acceptable levels of exposure for fine particles, 

but has not established a level for diesel exhaust other than as a source of fine 

particulate matter. 

14-P-0154 5 
 

 

Fine Particulate Matter 

Particulate matter (PM) is a complex mixture of harmful solid and liquid particles. 

Fine particles that are less than or equal to 2.5 microns in diameter are known as 

fine particulate matter (PM2.5). These particles are about 1/30th the thickness of a 

human hair. The greatest number of particles is usually concentrated in the 

“ultrafine” range. This range represents particles that are less than or equal to 

0.1 microns in diameter or about 1/1000th to 1/10,000th the thickness of a human 

hair. Because fine and ultrafine particles can penetrate deeply into the respiratory 

tract, these smaller particles may be more likely to cause adverse health effects 

than larger particles. CAPS are fine particles collected from the immediate 

environment that are concentrated using specialized equipment. By concentrating 

air particles, ORD can mimic the levels of pollution that humans are exposed to in 

other areas, such as severe nonattainment areas in the United States and more 

polluted cities in the world. 

NHEERL has two ambient-air-particle concentrators for studying the effects of 

concentrated particulate matter. Available systems include a coarse-particle 

concentrator for particle sizes from 2.5 to 10 micrometers and a fine/ultrafineparticle concentrator for particles 2.5 micrometers and smaller. 

The turquoise, gray and red pipes are used to control the airstream conditions, 

temperature and humidity in a test chamber (left) and a bank of analyzers used for 

pollutant monitoring (right). (EPA OIG photos) 

Diesel Exhaust 

Diesel exhaust is produced when an engine burns diesel fuel. Diesel exhaust is a 

complex mixture of more than 40 toxic air contaminants. These include 19 known 

or suspected carcinogens, such as benzene, formaldehyde and 1,3-butadiene. 

Particles emitted from diesel engines are usually concentrated in the “ultrafine” 

range. Their small size makes them highly respirable and able to reach the deep 

lung. The improvement in the development of diesel technologies over the last 

few decades has led to diesel engines with reduced emissions. 

14-P-0154 6 
 

 

 

 

Table 1 describes the health impacts from short-term and long-term exposures to 

PM2.5 and diesel exhaust. While some of these impacts are serious, the persons 

most at risk are children, elderly people and people with heart and lung disease. 

According to several EPA managers and documents we reviewed, EPA excluded 

persons most at risk from the subject population. 

Table 1: Health impacts from exposure to fine particulate matter and diesel exhaust 

Pollutant 

PM2.5

Exposure 

Short-term 

exposurea 

Respiratory impacts 

Aggravated lung disease, 

causing asthma attacks and 

acute bronchitis, and may 

also increase susceptibility 

to respiratory infections. 

Cardiovascular impacts 

In people with heart 

disease, exposure linked 

to heart attacks and 

arrhythmias. 

Other impacts 

Mortality 

Long-term Respiratory-related disease Cardiovascular-related Mortality and suggestive 

exposure and respiratory effects. mortality and 

cardiovascular effects. 

of lung and other cancers 

and reproductive and 

developmental effects. 

Diesel 

exhaust/PM 

Short-term 

exposurea Respiratory effects including 

irritation to throat and lungs, 

a cough, nausea and 

exacerbated asthma. 

Cardiovascular effects 

such as worsening heart 

diseaseb 

Irritation to the eyes and 

nose and neurological 

effects such as 

lightheadedness. 

Long-term 

exposure 

Respiratory effects including 

lung inflammation. 

Lung cancer and mortality. 

Source: OIG analysis of the EPA’s Integrated Science Assessment for PM, EPA websites on particulate matter and 

diesel particulate matter, National Ambient Air Quality Standards for PM (December 2012) and the U.S. EPA. Health 

Assessment Document for Diesel Engine Exhaust (Final 2002). U.S. EPA Office of Research and Development, 

National Center for Environmental Assessment, Washington, D.C., EPA/600/8-90/057F, 2002.
 

a

According to the agency, short-term epidemiological studies generally involved exposures ranging from 1 to 5 days. 

The length of the exposure sessions for the five studies the OIG reviewed can be found in table 3. b

Health impacts for diesel exhaust were combined because sources did not distinguish between short- and long-term 

effects. 

Exposure Levels for PM2.5 

The Clean Air Act required that the EPA establish standards, or acceptable levels 

of exposure, with an “adequate margin of safety” for each of the criteria 

pollutants. The EPA has set NAAQS for particulate matter. PM2.5 is one of the six 

principal or "criteria" pollutants. Table 2 lists the levels of exposure for PM2.5 for 

the 24-hour and annual levels. Primary standards provide public health protection, 

including protecting the health of “sensitive” populations such as asthmatics, 

children, and the elderly. Secondary standards provide public welfare protection, 

including protection against decreased visibility and damage to animals, crops, 

vegetation and buildings. Appendix A lists the PM2.5 concentrations associated 

with the EPA’s Air Quality Index for Particle Pollution. 

14-P-0154 7 
 

 

 

 

 

 

 

Table 2: NAAQS Standards for PM 

Pollutant 

[final rule cite] 

Primary/ 

secondary 

Averaging 

time Levelb Form 

Particle 

pollution 

Dec 14, 2012a PM2.5 

Primary Annual 12 μg/m3 Annual mean, averaged over 3 years 

Secondary Annual 15 μg/m3 Annual mean, averaged over 3 years 

Primary and secondary 24-hour 35 μg/m3 98th percentile, averaged over 3 years 

Source: The EPA’s summary of NAAQS standards, December 14, 2012. 

a

On December 14, 20123, the EPA revised its existing standards for PM. b

Units of measure are micrograms per cubic meter of air (μg/m3). 

Exposure Levels for Diesel Exhaust 

Diesel engines are one source of PM. The EPA has not established levels of 

acceptable exposure for diesel exhaust other than as particulate matter. The EPA 

has stated that long-term inhalation exposure to diesel exhaust is likely to pose a 

lung cancer hazard to humans. 

EPA’s Human Research Studies for PM and Diesel Exhaust 

Over the last 10 years, the EPA has conducted 13 human exposure studies using 

CAPS and four studies using diesel exhaust. According to an EPA principal 

investigator, the exposure levels selected for a study reflect a balance between being 

high enough to produce biological responses but not so high as to produce clinical 

responses in a study subject. For example, a biological response would be a shortterm, reversible response such as inflammation that goes away in a few days. In 

contrast, a clinically significant response could put a study subject at medical risk. 

Table 3 describes the five CAPS and diesel exhaust studies conducted in 2010 and 

2011 that we reviewed. 

3

EPA published the final rule for 40 CFR Parts 50, 51, 52 et al., NAAQS for PM, in the Federal Register on 

January 15, 2013. The final rule became effective on March 18, 2013. 

14-P-0154 8 
 

 

Table 3: EPA particulate matter and diesel exhaust human research studies conducted in 2010-2011 

Study 

Pollutant(s) 

that 

humans 

were 

exposed to 

Pollutant 

concentration 

approved by 

IRB (PM or 

diesel exhaust) 

Length of pollutant 

exposure approved by 

IRB 

Study protocol 

description of 

study subjects 

approved by the 

IRB 

Number of 

study 

subjects 

exposed in 

2010 and 2011 

XCON CAPS (PM) Up to 600,000 

particles per cc 

2 two-hour exposures: 

1 PM and 1 clean air 

exposurea 

Adults 25 to 

70 years of age 

with metabolic 

syndromeb 

23 

OMEGACON CAPS (PM) Up to 600 μg/m3 2 two-hour exposures: 

1 PM and 1 clean air 

exposure 

Healthy adults 

50 to 75 year olds 

17 

KINGCON CAPS (PM) Up to 600 μg/m3 2 two-hour exposures:

 1 PM and 1 clean air 

exposure 

45 to 65 years of 

age with mild 

asthma who were 

non-smokers 

5 

LAMARCK Diesel 

exhaust and 

ozone 

300 μg/m3 3 two-hour exposures: 

1 ozone, 1 diesel exhaust, 

and 1 clean air exposure 

Healthy adults and 

adults with mild to 

moderate asthma 

25 

DEPOZ Diesel 

exhaust and 

ozone 

300 μg/m3 4 two-hour exposures: 

1 diesel exhaust and 

ozone, 1 ozone only, 

1 diesel exhaust only, and 

1 clean air exposure 

followed by a two hour 

exposure to ozone the 

next day 

Healthy adults 

18 to 55 years old 

18 

TOTAL 88c 

Source: OIG analysis of information provided by the EPA ORD NHEERL. 

a

All of the studies exposed human subjects to clean air as a scientific control. b

The National Cholesterol Education Program characterizes metabolic syndrome patients as having three of the 

following: increased abdominal girth, elevated blood pressure, dyslipidemia (abnormal amount of fats and cholesterol 

in the blood), elevated fasting triglycerides and elevated fasting glucose. c

Seven individuals participated in two studies or 81 unique individuals participated in the 5 studies during 2010 and 2011.
 

The EPA offered to pay the human subjects participating in the five studies a 

maximum of about $950 to about $3,700. Study subjects are paid more to 

participate in lengthier, more complex studies. The UNC-IRB approved the 

payment amounts for these five studies. 

Scope and Methodology 

In response to a request by Congress, we assessed whether the EPA followed 

applicable laws, regulations, policies, procedures and guidance when it exposed 

41 human subjects to concentrated airborne particles or diesel exhaust emissions 

during three specific studies (XCON, KINGCON and OMEGACON) that the 

EPA conducted in 2010 and 2011. Additionally, because of Congressional interest 

in the exposure of subjects to diesel exhaust emissions, we also reviewed two 

other EPA studies (DEPOZ and LAMARCK) where subjects were exposed to 

diesel exhaust during the 2010–2011 timeframe. For these studies, we evaluated 

(1) the sufficiency of the approval process; (2) the adequacy of the process for 

14-P-0154 9 
 

 

obtaining the study subject’s informed consent; and (3) the adequacy of the EPA’s 

response to any adverse events. 

To determine if the EPA obtained sufficient approval to expose subjects to 

specific levels of CAPS or diesel exhaust emissions, we obtained and reviewed 

federal and EPA regulations, policies and guidance concerning human subjects 

research. We reviewed the study protocols for all five studies, including the study 

applications, consent forms, and EPA HSRRO and IRB review and approval 

documents. We compared the EPA’s and the IRB’s documentation of decisions, 

actions and events with the requirements of the Common Rule (40 CFR Part 26). 

We also compared them to EPA Order 1000.17 Change A1 and ORD and 

NHEERL policies. We interviewed current and former managers and staff from 

ORD and the Office of the Science Advisor. We interviewed the principal 

investigators for the five studies we reviewed. We also interviewed managers and 

staff from the Office of Human Research Ethics (OHRE) at UNC who oversee the 

IRB for EPA’s human subjects research conducted at the Human Studies Facility 

in Chapel Hill. 

To determine if the EPA obtained adequate informed consent from human study 

subjects before exposing them to concentrated airborne particles or diesel exhaust 

emissions, we obtained and reviewed the 88 signed consent forms from 81study 

subjects that were exposed to pollutants during 2010 and 2011 in the five studies. 

Seven individuals participated in two of the five studies we reviewed. Thus, 

81 unique individuals participated in the five studies from 2010 to 2011. In 

addition, we interviewed nine study subjects who participated in at least one of 

the five studies. This included three human subjects who experienced adverse 

events, two study subjects who did not complete all exposure sessions, and four 

study subjects who did not experience an event. We also discussed informed 

consent procedures with the principal investigators for each of the five studies. 

The interviews provided us with information about how the EPA obtained 

informed consent from the study subjects, addressed adverse events and provided 

clinical follow-up. 

To determine whether the EPA adequately addressed any adverse events that 

occurred, for each adverse event we obtained and reviewed the documentation of 

EPA’s notification to the IRB. We also reviewed other notifications such as to the 

EPA HSRRO. Additionally, we determined whether the EPA revised the consent 

forms after an event. We also assessed whether the EPA provided clinical 

follow-up in accordance with the approved protocol. We reviewed the EPA and 

the IRB documentation of adverse events and medical notes about the subjects. 

We interviewed the principal investigators for the five studies we reviewed, as 

well as ORD managers and staff about the adverse events that occurred. We also 

interviewed managers and staff from OHRE at UNC. The EPA office having 

primary responsibility over the issues evaluated in this report is ORD’s National 

Health and Environmental Effects Research Laboratory. 

14-P-0154 10 
 

 

 

We conducted our review from November 2012 to January 2014 in accordance 

with generally accepted government auditing standards. Those standards require 

that we plan and perform our review to obtain sufficient, appropriate evidence to 

provide a reasonable basis for our findings and conclusions based on our 

objectives. We believe that the evidence obtained provides a reasonable basis for 

our findings and conclusions based on our objectives. 

Prior Reports 

Prior reports by the U.S. Government Accountability Office (GAO) applicable to 

the topics addressed by this evaluation included: 

	 GAO Report No. 09-448T: Undercover Tests Show the Institutional 

Review Board System Is Vulnerable to Unethical Manipulation, 

March 26, 2009. 

	 GAO Report No. 01-775T: HUMAN SUBJECTS RESEARCH: HHS 

Takes Steps to Strengthen Protections, but Concerns Remain, May 23, 

2001. 

	 GAO Report No. HEHS-96-72: SCIENTIFIC RESEARCH: Continued 

Vigilance Critical to Protecting Human Subjects, March 8, 1996. 

14-P-0154 11 
 

 

Chapter 2

Studies Obtained Required Approvals, 

but Sequence of Approvals Not Followed and 

Procedures Could Be Improved 

The EPA obtained required approvals to expose study subjects to specific levels 

of concentrated airborne particles and diesel exhaust emissions. For each study, 

the EPA obtained approval to conduct the study from one of UNC’s biomedical 

IRBs, internal NHEERL division and senior management, and the EPA’s 

HSRRO. The EPA’s HSRRO resides within the Office of the Science Advisor 

and is independent of ORD’s NHEERL. However, in four of five studies, the 

branch chief approved the study on the NHEERL sign-off sheet after the initial 

IRB approval. In addition, several of the reviews did not occur in the order called 

for in NHEERL guidance, and information was missing on the NHEERL sign-off 

sheet for several studies. While EPA regulations and policy require that the IRB 

approve all study modifications, the EPA’s regulations, policy and guidance do 

not address the review and approval process for study modifications, including 

when a study modification would be significant enough to obtain the HSRRO’s 

approval. For example, we found a study modification approved by the IRB that 

doubled the concentration level of pollutant exposure but was not reviewed by the 

HSRRO. In our view, the EPA should revise its NHEERL guidance to include the 

review and approval process for study modifications, including a definition of 

significant study modifications that should be reviewed by the HSRRO. By 

revising its guidance, the EPA would help ensure that significant modifications 

receive a second independent review. In addition, while the investigators fulfilled 

their initial formal ethics training, we were unable to confirm whether they had 

met their continuing education requirements because NHEERL does not have a 

procedure for documenting the completion of the annual ethics refresher training. 

Required Approvals Obtained to Conduct Five Human Research 

Studies, but Sequence of Approvals Did Not Follow NHEERL Guidance 

The EPA obtained required approvals for the five human research studies we 

reviewed. However, the EPA did not follow NHEERL guidance for its NHEERL 

reviews and approvals. According to NHEERL guidance, the branch chief must 

review and approve the protocol, informed consent form, and all other items to be 

sent to the IRB before the principal investigator sends these items to the IRB. The 

guidance also states that all required reviews and responses are obtained in 

writing and approvals are recorded on the NHEERL sign-off sheet, and that initial 

reviews, such as the medical and statistical review, occur before the branch chief 

and IRB review. NHEERL requires that peer reviewer comments and principal 

investigator responses be sent to the IRB with the protocol. This helps ensure that 

the IRB sees the full research proposal, including any peer review comments. 

14-P-0154 12 
 

 

After the IRB approval, the Division Quality Assurance Officer reviews the 

protocol, then the HRPO Director, and then the Division Director. In our review 

of the sign-off sheets, we found: 

 In four of the five studies, the branch chief approved the study on the 

NHEERL sign-off sheet after the initial IRB approval. 

 In three of the five studies, the physician signed the NHEERL sign-off 

sheet after the IRB approval. 

 In two studies, the HRPO signed the NHEERL sign-off sheet after the 

division director, and in one study the HRPO signed before the Division 

Quality Assurance Officer. 

 In two studies, NHEERL received peer reviews after the initial IRB 

approval of the study. 

 In one study, the statistician’s review occurred after the initial IRB 

approval of the study. 

In one study, the sign-off sheet did not have dates for the statistician, physician, 

peer reviews, and the division director approval. When requested, the principal 

investigator was unable to provide us documentation with the missing dates. 

According to an NHEERL manager, there is no obligation to obtain sign-off in a 

particular order other than all signatures must be obtained before sending the 

package on for the approval of the Associate Director of Health. The EPA needs 

to ensure that NHEERL properly documents its review and approval steps for 

human research studies as stated in NHEERL guidance. 

The EPA’s regulations (40 CFR 26.109) require that an IRB review and approve 

initial study protocols as well as any study modifications before a study can begin 

or be modified. An IRB is an independent institution designated to review, approve 

and monitor approved research involving human subjects. The IRB reviews the 

study protocol, the consent form or forms and advertisements to recruit study 

subjects. The IRB also asks questions as needed to clarify issues before giving its 

approval. The protocols for each of the studies included information about the 

levels of pollutant exposure and the associated risks. In one case, the IRB made its 

approval contingent upon additional statements about the study risks being added to 

the consent forms. Once the EPA had addressed the IRB’s questions or 

requirements to the IRB’s satisfaction, the IRB approved each study. 

EPA Order 1000.17 Change A1 requires the EPA HSRRO to approve human 

subjects research studies supported by the EPA. The EPA’s HSRRO serves as an 

expert technical resource in matters of human research ethics and subject safety. 

The HSRRO also provides guidance and leadership in these areas. According to 

NHEERL guidance, each human research study should also receive NHEERL 

branch, division and management level reviews, in addition to IRB review and 

approval, before being sent to the EPA’s HSRRO for review and approval. 

Figure 1 is a flowchart of the review and approval process. 

14-P-0154 13 
 

 

 Figure 1: Flowchart of review and approval of NHEERL study protocols

Draft 

Protocol 

Initial branch-level 

reviews 

(extramural, medical, 

Branch Chief, etc.) 

Revised 

protocol 

IRB 

review 

Changes? 

Changes? 

Changes? 

Division-level reviews 

(Quality Assurance, Dosing, Branch 

Chief, Division Director) 

NHEERL-level reviews 

(HRPO Director, Associate 

Director for Health) 

HSRRO review 

Begin research 

Changes? 

Yes 

Yes 

Yes 

Yes 

No 

No 

No 

No 

 Source: OIG flowchart based on NHEERL Guidance, 2010. 

ORD’s NHEERL guidance incorrectly lists the HSRRO review under the 

NHEERL level reviews. The guidance also identifies the Division Human 

Research Officer (DHRO) as a division-level reviewer when no one has filled the 

position according to an NHEERL manager. NHEERL policy requires each 

NHEERL division conducting human subjects research to appoint a DHRO to 

ensure compliance with federal regulations and NHEERL human research policy. 

The HRPO has been fulfilling this role according to an NHEERL manager. 

Clarifying ORD’s guidance and vesting the review responsibilities of the DHRO 

14-P-0154 14
 

 

in an existing position, such as the HRPO Director, will better ensure that initial 

study protocols and significant modifications receive thorough reviews. 

The EPA received approvals from one of the UNC’s biomedical IRBs and the 

EPA’s HSRRO for each of the five studies as required by 40 CFR 26.109, 

EPA Order 1000.17 Change A1 and the NHEERL guidance. According to ORD 

managers, this two-level review process provides an additional level of protection 

to the study subjects that does not exist at most other research institutions. The 

EPA also documented the NHEERL branch and divisional reviews for the five 

studies using NHEERL sign-off sheets. We confirmed that the IRB reapproved 

the five protocols we reviewed annually and approved modifications to the 

studies. We also verified that the HSRRO approved the five studies before they 

began. 

EPA Could Strengthen Protection of Human Subjects by Developing 

Guidance Concerning Study Modifications 

While 40 CFR Part 26.109(a) and EPA Order 1000.17 Change A1 require the IRB 

to approve all modifications, the EPA’s policies and guidance do not address the 

EPA internal review and approval process for modifications. In addition, the 

EPA’s policy and guidance do not address when the approval of the HSRRO 

would be necessary for significant study modifications. Study modifications are 

common. The IRB approved 18 modifications for the XCON study, 19 

modifications for KINGCON and 24 modifications for OMEGACON over the life 

of the three studies. Each modification application usually contained multiple 

changes to the study. 

While some study modifications are considered minor, such as when new 

investigators are added to the study, other study modifications may be significant. 

The KINGCON study’s June 2008 application stated that human subjects would 

be exposed to concentrated air particle levels between 50 and 300 μg/m3

. The 

application stated that an exposure session would be shut down if the exposure 

concentration exceeded 400 μg/m3

. However, in November 2008, the principal 

investigators modified the study to increase the upper limit of concentration to 

600 μg/m3

. The EPA’s application for IRB approval of the modification stated 

that the concentration increase did “not increase risk” to study subjects in the 

study. This study modification, which some may consider significant, was not 

sent to the HSRRO for review. NHEERL management told us that the Director of 

the HRPO within NHEERL determines whether to send a modification to the 

HSRRO. By revising its guidance, EPA would help ensure that significant 

modifications receive an independent review and properly address any changes in 

risks to human subjects. 

14-P-0154 15 
 

 

 

 

Formal Ethics Training Requirements Met, but Procedure Needed to 

Track Annual Continuing Education Requirements 

According to NHEERL guidance, the conduct of human subjects research carries 

special responsibilities with regard to ethical, medical and scientific issues. 

Society has imposed special requirements on investigators because of concern of 

potential maltreatment of human research subjects. Ethics training helps ensure 

that risks to human subjects are not overlooked. NHEERL guidance requires that 

all EPA investigators involved in human subjects research attend formal human 

research ethics training. The Collaborative IRB Training Initiative (CITI) is a 

Web-based program that is used to satisfy the ethics training requirement. CITI 

training contains modules on topics including ethical principles, IRB regulations, 

informed consent and vulnerable populations. UNC maintains an electronic 

database of individuals who have completed the CITI basic educational 

requirements. For the five studies we evaluated, the EPA investigators involved 

with human subject research had completed the formal human research ethics 

training requirement. 

NHEERL policy and guidance also require that all personnel engaged in human 

subjects research complete continuing education (refresher training) on an annual 

basis. The annual continuing education requirement can be fulfilled by: 

1) completion of the on-line CITI Refresher course, or 2) attendance at one lecture 

or seminar with a primary focus on human research issues. We could not 

determine if the annual continuing education requirements were met by the EPA 

investigators involved in human subjects research because there was no 

documentation of their completion. We asked the Environmental Public Health 

Division Director whether a procedure exists to record and track annual 

continuing education training. The division director told us that NHEERL lacked 

a procedure for tracking this requirement, but that a mechanism would be 

implemented immediately. In September 2013, the division director provided us 

with a copy of the database page that will be used to track the training and 

proposed revisions to NHEERL guidance. 

One Human Research Study Exposed a Study Subject Above 

Concentration Targets but Followed Approved Protocol 

The five human research studies we reviewed allowed subjects to be exposed 

above the study concentration targets. For example, the OMEGACON protocol 

stated that an exposure was to be shut down if particulate concentrations exceeded 

600μg/m3 for over six minutes. Evidence shows that EPA exposed one 

OMEGACON study subject to pollutant concentrations that reached 751μg/m3 

, 

which exceeded the IRB-approved concentration target of 600 μg/m3

. EPA 

computer-generated real time data from the exposure chamber showed that the 

exposure session was shut down six minutes after the first concentration of 

600 μg/m3 was recorded. Additionally, the Director of OHRE at UNC stated that 

studies’ approved pollutant concentrations are targets and not absolute safety 

14-P-0154 16 
 

 

 

 

limits, and in this instance, where the level did exceed the targeted range, the 

protocol was followed and the study session was halted. 

The other four human research studies we reviewed did not expose study subjects 

above the concentration limits. 

Conclusions 

The EPA obtained the required approvals for the five studies we reviewed. 

However, in four of five studies, the branch chief approved the study on the 

NHEERL sign-off sheet after the initial IRB approval. In addition, several of the 

reviews did not occur in the order called for in NHEERL guidance, and 

information was missing on the NHEERL sign-off sheet for several studies. 

While the EPA regulations and policies require that the IRB approve all 

modifications, the EPA could improve its protection of human subjects if it 

established guidance for determining when study modifications are significant 

enough to require HSRRO approval. Additionally, while the investigators fulfilled 

the initial formal ethics training, we were unable to confirm that they had met the 

continuing education requirements because there was no procedure for 

documenting the completion of the annual refresher training. The EPA needs to 

ensure that the human research study team members obtain annual ethics training 

to properly protect study subjects. 

Recommendations 

We recommend that the Assistant Administrator for Research and Development: 

1. Revise the NHEERL Human Research Guidance to include: 

a.	 An EPA internal review and approval process for significant study 

modifications which include a definition and illustrative examples of 

significant study modifications. The review and approval process 

should indicate when significant study modifications should be sent 

to the HSRRO for review and approval. 

b.	 A revised flowchart of the protocol review process listing the 

HSRRO as an independent reviewer and not part of the NHEERLlevel review process and eliminating the DHRO review. 

2.	 Implement a procedure for documenting that human subjects research 

study investigators have met the requirement for continuing annual ethics 

education. 

3.	 Revise the NHEERL Human Research Policy to eliminate the DHRO 

position and transfer the duties of the DHRO to the HRPO Director. 

14-P-0154 17 
 

 

 

4.	 Develop management controls to ensure NHEERL management reviews 

and approvals are properly documented and follow NHEERL guidance. 

Agency Comments and OIG Evaluation 

The agency concurred with all recommendations in Chapter 2 and provided 

acceptable planned corrective actions and completion dates for recommendations 

1(a), 1(b), 3, and 4. Recommendations 1(a), 1(b), 3, and 4 are resolved and open 

with corrective actions ongoing. NHEERL provided evidence that it has already 

completed the corrective actions for recommendation 2. This recommendation is 

closed and no further action is required. Appendix B contains the agency’s 

response to our draft report. Appendix C contains our detailed evaluation of that 

response. 

14-P-0154 18 
 

 

  

 

Chapter 3

EPA Obtained Informed Consent From 

Human Subjects, but Consent Forms Addressed 

Pollutant Exposure Risks Differently
 

The EPA obtained informed consent from human study subjects before exposing 

them to concentrated airborne pollutants or diesel exhaust in 2010 and 2011. Our 

interviews with nine human study subjects confirmed that: 

 They read the consent form.
 

 They had the opportunity to review the consent form and ask questions.
 

 An EPA representative explained the form to them prior to their 

participation in the study.
 

 They signed the consent form. 

In the five studies’ consent forms, the EPA addressed the risks of medical 

procedures and described certain risks of being exposed to CAPS or diesel 

exhaust. However, we found that the five studies’ consent forms inconsistently 

addressed the risks of being exposed to CAPS and diesel exhaust. Although the 

regulation (40 CFR 26, the Common Rule) requires that consent forms describe 

any “reasonably foreseeable risks or discomforts to the subject,” the regulation 

does not define this phrase. In our view, the lack of such a definition contributed 

to this inconsistency. Furthermore, the EPA did not include the potential longterm cancer risks in the diesel exhaust studies’ consent forms. According to an 

EPA NHEERL manager, this was because they only planned to perform shortterm exposures and the risk of getting cancer from a single 2-hour exposure was 

minimal. In our view, the agency should inform study subjects of any potential 

cancer risks of a pollutant to which they are being exposed so that study subjects 

can make the most informed decision possible about whether to participate in a 

study. 

EPA Followed Requirements of 40 CFR 26.116 

According to 40 CFR 26.116, EPA investigators need to obtain informed consent 

from study subjects before research begins. NHEERL guidance states that 

procedures used to obtain informed consent should educate the potential study 

subjects using language they understand. We obtained copies of the 88 consent 

forms for the five studies and found that the 81 study subjects had signed the 

consent forms and, in some cases, had initialed each page of the form. We 

analyzed the consent forms for the five studies and determined that the EPA met 

the regulation’s requirements as shown in table 4. 

14-P-0154 19 
 

 

 

 

 Table 4: Basic elements of informed consent in the five studies’ consent forms 

Basic elements of informed consent XCON 

Element fulf

KINGCON 

i

OMEGACON 

lled in studies

LAMARCK 

 listed below 

DEPOZ 

1) A statement that the study involves 

research, an explanation of the purposes of 

the research and the expected duration of ✓ ✓ ✓ ✓ ✓ 

the subject's participation, a description of 

the procedures to be followed and 

identification of any procedures which are 

experimental. 

2) A description of any reasonably 

foreseeable risks or discomforts to the 

✓ ✓ ✓ ✓ ✓ 

subject. 

3) A description of any benefits to the 

subject or to others which may reasonably 

✓ ✓ ✓ ✓ ✓ 

be expected from the research. 

4) A disclosure of appropriate alternative Not Not Not Not Not 

procedures or courses of treatment, if any, applicable applicable applicable applicable applicable 

that might be advantageous to the subject. 

5) A statement describing the extent, if any, 

to which confidentiality of records identifying ✓ ✓ ✓ ✓ ✓ 

the subject will be maintained. 

6) For research involving more than minimal 

risk, an explanation as to whether any 

compensation and an explanation as to ✓ ✓ ✓ ✓ ✓ 

whether any medical treatments are 

available if injury occurs and, if so, what 

they consist of, or where further information 

may be obtained. 

7) An explanation of whom to contact for 

answers to pertinent questions about the 

research and research subjects' rights, and ✓ ✓ ✓ ✓ ✓ 

whom to contact in the event of a researchrelated injury to the subject. 

8) A statement that participation is voluntary, 

refusal to participate will involve no penalty 

or loss of benefits to which the subject is ✓ ✓ ✓ ✓ ✓

otherwise entitled and the subject may 

discontinue participation at any time without 

penalty or loss of benefits to which the 

subject is otherwise entitled. 

 Source: OIG analysis comparing the five studies’ consent forms with the requirements in 40 CFR 26.116(b). 

Furthermore, interviews with nine study subjects confirmed that the principal 

investigator or other EPA representative met with them and reviewed the content 

of the consent forms with them. Eight of nine study subjects stated that they were 

allowed to take the consent form home prior to their participation in the study; 

one study subject could not remember. According to the study subjects we 

interviewed, an EPA representative went over the consent form with them to a 

great extent or to a very great extent. 

Seven of nine study subjects stated that the consent form was easy to understand. 

One study subject stated the consent form was very wordy and that, in the 

subject’s opinion, less scientific language should be used; another thought an 

executive summary was needed. Although both believed that improvements could 

14-P-0154 20
 

 

 

 

 

be made to the consent form to make it easier to understand, the two study 

subjects stated that EPA went over the consent form with them to a great extent. 

Consent Forms Addressed Risks of Studies’ Medical Procedures 

The consent forms for the five studies addressed the risks and discomforts of the 

medical procedures involved in the studies. The consent forms discussed the 

minimal risks of heart rhythm and blood pressure monitoring and blood sampling, 

as well as the risks of more complex procedures such as brachial artery ultrasound. 

For example, an optional procedure in the KINGCON study was a bronchoalveolar 

lavage, which is used to obtain fluids and cells from the respiratory tract to analyze 

the pulmonary response to particle exposure. In this procedure, a bronchoscope is 

passed through the nostril to the back of the throat and wedged in an airway in the 

right lung. Sterile saline is injected into the lung and then suctioned into the 

bronchoscope. The EPA used five pages of the KINGCON 18-page consent form to 

describe the bronchoalveolar lavage procedure, including the purpose of the 

procedure, how it would be performed, reasons a potential subject should not 

participate in the procedure and the potential risks. A similar description was 

provided in the LAMARCK consent form. 

Studies’ Consent Forms Addressed Risks of Pollutant Exposures 

Inconsistently 

We found inconsistencies in the content of the consent forms with respect to the 

risks of pollutant exposure to CAPS (PM) and diesel exhaust. As shown in 

table 5, the consent forms for all five studies compared the subject’s exposure to 

the exposure they would receive while visiting a large city on a smoggy day. 

However, only one of five studies’ consent forms provided the subject with 

information on the upper range of the pollutant he or she would be exposed to and 

only two of five alerted study subjects to the risk of death for older individuals 

with cardiovascular disease.

 Table 5: Statements made in consent forms of the five studies reviewed 

Studies 

Comparison of study 

exposure to exposure 

visiting large city on a 

smoggy day 

Information about the 

upper range of pollutant 

exposure 

Information about risk of 

death for those with 

cardiovascular disease 

XCON Yes Yes Yes 

OMEGACON Yes No No 

KINGCON Yes No No 

LAMARCK Yes No No 

DEPOZ Yes No Yes 

 Source: OIG analysis of consent forms for XCON, OMEGACON, KINGCON, DEPOZ and LAMARCK studies. 

In a 2003 fact sheet4

, the EPA’s message to the public about PM2.5 was that longterm exposure is associated with reduced lung function and even premature death, 

4 Particle Pollution and Your Health (2003), U.S. EPA Office of Air and Radiation, EPA-452/F-03-001. 

14-P-0154 21 
 

 

 

 

and short-term exposure is linked to heart attacks and arrhythmias for people with 

heart disease. A 2006 EPA assessment document5

 further reports associations 

between short-term PM exposures and mortality and morbidity. 

The XCON and DEPOZ study consent forms warned the study subjects that 

exposure to high levels of selected air pollutants (i.e., PM, the pollutant being 

tested in the XCON study and diesel exhaust, the pollutant being tested in the 

DEPOZ study) could lead to death in older people with cardiovascular problems. 

This warning was not in the OMEGACON, KINGCON, or LAMARCK consent 

forms, even though these studies also exposed study subjects to PM 

(OMEGACON, KINGCON) and diesel exhaust (LAMARCK). According to an 

NHEERL manager, the exposure risk for healthy individuals is minimal. Because 

the three studies’ consent forms (OMEGACON, KINGCON and LAMARCK) 

lacked the warning that PM exposure can cause death in older people with 

cardiovascular disease, they are significantly different in their disclosure of 

exposure risk than the XCON and DEPOZ consent forms. This lack of warning 

about PM in OMEGACON, KINGCON and LAMARCK is also different from 

the EPA’s public message about PM. 

Only the XCON study consent form identified the upper range of pollutant 

exposure for each study subject. The other four studies’ consent forms did not 

mention the level of pollutant exposure. Instead, the forms for the other four 

studies (DEPOZ, KINGCON, LAMARCK and OMEGACON) compared the 

subject’s level of exposure during the study to the exposure they would receive 

visiting major cities on smoggy days. According to an NHEERL manager, this 

comparison to a major city was a practical way for the subjects to understand the 

relative risk of exposure. The manager explained that a person breathing 420 

μg/m3

 for 2 hours would inhale the same concentration as they would breathing 

35 μg/m3 (the EPA’s 24-hour standard for PM2.5) for 24 hours in a city such as 

Los Angeles. The manager also stated that PM risk is focused on susceptible 

populations and that the risk is small for those with no overt disease. One human 

subject that we interviewed believed, in hindsight, that the pollutant exposure 

range should be included in the consent form. 

The EPA has not defined the term “reasonably foreseeable risks” in its 

regulations. According to an August 2013 article in the Journal of Clinical Best 

Practices: 

Since the regulations do not define “reasonably foreseeable risks, 

investigators, IRBs and oversight agencies might each interpret this 

phrase differently. These different interpretations can lead to confusion 

and controversy.... Inconsistent interpretations, by definition, lead to 

5

 Provisional Assessment of Recent Studies on Health Effects of Particulate Matter Exposure (2006), U.S. EPA 

Office of Research and Development, Washington, D.C., National Center for Environmental Assessment, 

EPA/600/R-06/063. 

14-P-0154 22 
 

 

 

 

 

inconsistent protection of human research subjects, which can lead to 

inadequate protections in some instances.6 

An NHEERL manager told us that consent forms differed because each was 

written by a different principal investigator. The manager also told us that 

NHEERL is developing new guidance for principal investigators that would help 

them harmonize the risk language in the consent forms. In our view, the lack of a 

definition of “reasonably foreseeable risks” in the EPA’s guidance for conducting 

human subjects research contributed to this inconsistency. 

Consent Forms for Diesel Exhaust Studies Did Not Include 

Potential Cancer Risks From Long-Term Exposure 

The LAMARCK and DEPOZ study consent forms did not include the potential 

cancer effects of long-term exposure to diesel exhaust. The EPA classifies diesel 

exhaust as “likely to be carcinogenic to humans by inhalation” and stated in its 

2002 Health Assessment Document for Diesel Engine Exhaust7

 that long-term 

inhalation exposure is likely to pose a lung cancer hazard to humans, as well as 

damage the lung in other ways depending on the length of the exposure. 

According to EPA’s 2002 Health Assessment document, the human evidence 

from occupational studies is considered strongly supportive of a finding that 

diesel exhaust exposure is causally associated with lung cancer, though the 

evidence is less than that needed to definitively conclude that diesel exhaust is 

carcinogenic to humans. 

According to an NHEERL manager, long-term cancer risks from 2-hour 

exposures would be minimal. The manager also stated that cancer risk is 

calculated considering a lifetime exposure of 40 years. 

The LAMARCK and DEPOZ protocols that went to the IRB contained language 

about the substance being carcinogenic and stated that the risk was minimal. 

Although the IRB was made aware of the principal investigator’s basis for 

considering the risk from short-term exposure to be minimal, human subjects 

were not informed of this risk in the consent forms. Table 6 shows what the 

principal investigator told the IRB about the risks from each study. 

6

 Resnick, D. “What is a ‘Reasonably Foreseeable Risk’? The Support Study Controversy.” Journal of Clinical
 

Research Best Practices 9.8 (2013): 1-12. 7

 U.S. EPA Health Assessment Document for Diesel Engine Exhaust (Final 2002). U.S. EPA Office of Research and 

Development, National Center for Environmental Assessment, Washington, D.C., EPA/600/8-90/057F.
 

14-P-0154 23 
 

 

 Table 6: Principal investigator’s explanation of risk to the IRB 

Study name / 

pollutants Statement of risk provided to the IRB in the EPA’s study application 

LAMARCK “Diesel exhaust particles contain some probable carcinogenic polycyclic aromatic hydrocarbons, 

which in high enough concentrations and/or with repeated exposures may induce tumors. Diesel 

(Diesel exhaust also contains aldehydes, some of which are possibly carcinogenic in high enough dose 

exhaust and and with long enough exposure. However, the exposure concentrations to be used in this protocol 

ozone) are minimal. Overall, it appears that at the low DEP [diesel exhaust particle] concentration to be 

given one time for the exposure in this study, the risk of cancer, if it exists at all, is extremely low 

and certainly no more than what one would experience if one were to visit for a few days a 

particulate-polluted city in the US [United States] such as Los Angeles or New York City.” (2009) 

DEPOZ 

(Diesel 

exhaust and 

ozone) 

“DE [diesel exhaust] particles contain some probable carcinogenic polycyclic aromatic 

hydrocarbons and other components, which in high enough concentrations and/or with repeated 

exposures may induce tumors. Diesel exhaust also contains aldehydes, some of which are 

possibly carcinogenic in high enough dose and with long enough exposure. However, the 

exposure time and concentrations to be used in this protocol are minimal relative to the durations 

required to induce lung cancer. Overall, there are no known long-term health risks in healthy 

individuals acutely exposed to DE at the PM concentrations given in this study.” (2009)

 Source: DEPOZ and LAMARCK study applications to the IRB. 

According to a risk analysis conducted for another diesel exhaust study in 1999, 

the long-term cancer risk for an individual exposed to 20 or 100 micrograms per 

cubic meter of diesel exhaust for 2 hours was estimated to be in the range of 1 in 

1 billion. The study subjects in LAMARCK and DEPOZ were exposed to levels 

that ranged from about 250 to 320 micrograms per cubic meter. The DEPOZ 

principal investigator estimated the risk to be about 3 in 1 billion because the 

exposure was three times higher. 

An August 2013 article in the Journal of Clinical Best Practices states that “most 

people would want to know whether a medical procedure involves a risk of death, 

even if the chance of dying is very small.” One study subject that we interviewed 

stated that it would have been useful to have had information about known longterm effects of exposure to diesel exhaust. In our view, the EPA should inform 

study subjects of the potential long-term cancer risk of any pollutant to which it 

exposes human subjects so that study subjects can make the most informed 

decision possible about whether to participate in a study. 

Conclusions 

The EPA obtained informed consent from the 81 study subjects that participated 

in the five studies in 2010 and 2011 as required by 40 CFR 26.116. However, the 

EPA inconsistently addressed pollutant risk in its consent forms. Only two of the 

five studies’ consent forms included the risk of death from exposure to high levels 

of selected air pollutants such as PM and diesel exhaust, and only one study’s 

consent form included the upper limits of exposure levels. Because EPA’s 

regulations do not define “reasonably foreseeable risks,” EPA investigators, the 

IRB and the HSRRO must define the term using their professional judgment, 

which leads to inconsistencies in addressing risks in the study consent forms. 

Such inconsistencies could lead to inconsistent protection of human subjects. The 

EPA needs to develop guidance to help ensure more consistent interpretation of 

14-P-0154 24
 

 

 

reasonably foreseeable risks. Furthermore, the EPA should provide the study 

subjects with a summary of the EPA assessments about the short- and long-term 

effects of the pollutants to which human study subjects will be exposed. 

The EPA’s diesel exhaust studies did not include language about the long-term 

cancer risks of diesel exhaust. The NHEERL manager explained that the cancer 

risk from diesel exhaust was not relevant to the 2-hour exposures included in the 

LAMARCK study. However, evidence suggests that at least some human study 

subjects would like to know if a study involves risk of death, even if the risk is 

very small. In the future, the EPA should include the long-term risk of cancer to 

potential subjects in its consent forms so study subjects can make the most 

informed decision about whether to participate in a study. 

Recommendations 

We recommend that the Assistant Administrator for Research and Development: 

5. Revise NHEERL Human Research Guidance to include a definition for 

“reasonably foreseeable risks” including illustrative examples of the types 

of information that should be included in the consent forms. 

6. Revise NHEERL Human Research Guidance to include procedures for 

ensuring that human subjects research consent forms consistently present 

the risks of the pollutants to which human subjects are exposed, including 

a summary of EPA assessments of short-term and long-term health effects 

and the upper pollutant concentration level for the pollutant to which the 

human subjects will be exposed. 

7. Include in its consent forms any known or likely carcinogenic effects of 

pollutants that the EPA uses in human exposure studies, based on EPA, 

other federal health agency, or other organization’s (as appropriate) 

assessment of such risks. If EPA uses the work of non-federal entities, the 

agency should document the basis for using non-federal information as 

opposed to the assessments of federal health agencies. 

Agency Comments and OIG Evaluation 

The agency concurred with all recommendations in Chapter 3 and provided 

acceptable planned corrective actions and completion dates for the 

recommendations. Recommendations 5, 6, and 7 are resolved and open with 

corrective actions ongoing. Appendix B contains the agency’s response to our 

draft report. Appendix C contains our detailed evaluation of that response. 

14-P-0154 25 
 

 

 

Chapter 4

Improvements Needed in EPA Guidance to 

Address Adverse Event Definitions, Reporting 

Timeframes and Clinical Follow-Up Responsibilities 

The EPA addressed six adverse events in the five CAPS and diesel exhaust 

studies conducted in 2010 and 2011 by reporting them to the IRB and providing 

clinical follow-up after the events. The EPA’s clinical follow-up for the six 

adverse events ranged from 1 day to 3 months after the event and included phone 

calls and emails by the EPA nurses. While the EPA’s clinical follow-up appeared 

to be reasonable, the EPA’s policies and guidance do not establish the EPA’s 

clinical follow-up responsibilities. In our view, the EPA should revise its 

guidance to establish the agency’s clinical follow-up responsibilities after an 

adverse event. 

According to ORD managers, NHEERL is required to use the latest UNC-IRB 

standard operating procedures (SOPs) adverse event definitions and reporting 

timeframes to respond to adverse events. For the five studies we reviewed, this 

would have been the 2009 UNC-IRB SOPs. However, NHEERL’s policies and 

guidance do not state that NHEERL has adopted UNC-IRB SOP’s adverse event 

definitions and reporting timeframes. NHEERL guidance is outdated, with 

definitions for adverse events and reporting timeframes that are the same as the 

2003 UNC-IRB SOP definitions and reporting timeframes. When evaluated 

against the NHEERL guidance reporting timeframes, the EPA reported two of the 

six adverse events to the IRB later than required. In addition, when evaluated 

against the EPA’s NHEERL guidance definition for an adverse event, the EPA 

did not report two study events to the IRB in which both subjects experienced 

cardiac arrhythmias. In both instances, the subjects were not allowed to continue 

participating in the study and were advised to consult their physicians. The EPA 

should update its NHEERL guidance to make it clear what adverse event 

definitions and reporting requirements they should use when managing their 

human subject research studies to ensure the consistent protection of study 

subjects. 

Clinical Follow-Up Provided but Guidance and Study Protocols Lack 

Clinical Follow-Up Responsibilities 

The EPA identified six adverse events during three studies (DEPOZ, 

OMEGACON, and XCON) conducted in 2010 and 2011 and reported them to the 

IRB. The EPA did not identify any adverse events in two studies, KINGCON and 

LAMARCK. Table 7 summarizes each of the six adverse events. 

14-P-0154 26 
 

 

 

 

 

Table 7: Summary of the six adverse events in 2010 and 2011 

Study and date 

(month/year) of 

adverse event Description of adverse event 

Actions taken after adverse event was reported including 

revising the study’s consent forms 

OMEGACON, 

May 2010 

Subject developed a migraine 

during exposure. 

The EPA revised the consent forms to exclude future human 

subjects with a history of migraine headaches from 

participating in the study. 

OMEGACON, 

May 2010 

Subject had cardiac 

arrhythmiaa

 after exposure to 

clean air. 

Not applicable because the IRB did not consider the adverse 

event to be reportable. 

OMEGACON, 

June 2010 

Subject had cardiac arrhythmia 

after exposure to clean air. 

Not applicable because the IRB did not consider the adverse 

event to be reportable. 

DEPOZ, 

September 2010 

Subject experienced 

decreased lung function after 

ozone exposures. 

None. DEPOZ consent form already addressed the potential 

risk of airway obstruction. 

XCON, 

October 2010 

Subject experienced 

tachycardiab

 and atrial 

fibrillationc while exposed to 

ambient air pollution particles. 

The subject was later 

hospitalized overnight for 

observation. 

NHEERL management met and determined that no screening 

could have feasibly been done to have predicted this issue. 

The XCON consent form already warned study participants 

not to participate if they had cardiovascular disease including 

coronary artery disease, heart failure, or rhythm disturbances. 

DEPOZ, 

April 2011 

Subject developed a persistent 

cough. 

NHEERL established a corrective action plan where subjects 

who presented a cough within the first 15 minutes of 

exposure would be removed from the chamber. 

DEPOZ consent form already stated that exposure to ozone 

could cause a cough. 

Source: NHEERL’s information on human subjects who experienced adverse events. 

a

An arrhythmia is a problem with the rate or rhythm of the heartbeat. During an arrhythmia, the heart can beat too 

fast, too slow, or with an irregular rhythm.

b

Tachycardia is a faster than normal heart rate. A healthy adult heart normally beats 60 to 100 times a minute when a 

person is at rest. If you have tachycardia, the heart rate in the upper chambers or lower chambers of the heart, or 

both, are increased significantly. 

c

Atrial fibrillation is an irregular and often rapid heart rate that commonly causes poor blood flow to the body. During 

atrial fibrillation, the heart's two upper chambers (the atria) beat chaotically and irregularly—out of coordination with 

the two lower chambers (the ventricles) of the heart. 

After each of the six EPA-identified adverse events took place, agency medical 

staff provided clinical follow-up, which we define as the necessary monitoring of 

the condition of a human research subject who experienced an adverse event to 

ensure their well being. The EPA’s follow-up included subsequent 

communications with the subject regarding their health. The length of clinical 

follow-up varied from 1 day to 3 months. The EPA doctors and/or nurses 

provided the follow-up care. After several events, clinical follow-up included 

advising the study subject to make an appointment with their private physician or 

another specialist for medical care. Table 8 describes the clinical follow-up the 

EPA medical staff provided for each of the six EPA-identified adverse events. 

14-P-0154 27 
 

 

 

 

Table 8: Clinical follow-up provided by the EPA for the six adverse events in 2010 and 2011 

Study and timeframes 

of adverse event Clinical follow-up provided by the EPA 

OMEGACON, 

May 2010 

Two days of follow-up including (1) giving the subject medicine for pain relief and a visit 

by NHEERL’s on duty physician on the first day and (2) a follow-up conversation with the 

principal investigator on the second day. 

OMEGACON, 

May 2010 

Two days of follow-up including review of subject’s holter monitor recordinga

 by one 

doctor and two nurses. On the second day, the EPA medical staff advised the study 

subject to see a private physician because the principal investigator believed that the 

study subject had an underlying medical condition. The EPA provided the study subject 

with a copy of medical test results. 

OMEGACON, 

June 2010 

One day of follow-up plus an email from an NHEERL nurse three weeks afterwards. 

During the follow-up day, an NHEERL doctor and the principal investigator reviewed the 

study subject’s holter monitor recording, and the NHEERL doctor reviewed the study 

subject’s electrocardiogram. EPA medical staff advised the study subject to see a health 

care provider and a private cardiologist. 

DEPOZ, 

September 2010 

Follow-up occurred the day following the event. Follow-up included assessments by an 

EPA doctor and nurse. The study subject continued to participate in the study. 

XCON, 

October 2010 

Follow-up occurred over a span of seven days including (1) a nurse consultation with two 

NHEERL physicians; (2) review of holter monitor recordings; (3) observation of study 

subject; (4) study subject transfer to the emergency room with overnight monitoring at 

the UNC hospital; and (5) phone calls to the study subject by an NHEERL nurse. 

DEPOZ, 

April 2011 

Follow-up provided for 3 months after event including (1) being seen by an NHEERL 

physician; (2) receiving medication for 1 week; (3) emails and phone calls by an 

NHEERL nurse; and (4) scheduling an appointment for the study subject at the UNC 

Ambulatory Care Center Pulmonary Clinic. 

Source: NHEERL’s information on human subjects who experienced adverse events. 

a

A holter monitor is a type of portable electrocardiogram which keeps a continuous record of the heart rhythm, 

typically over a 24-hour period. 

The EPA’s consent forms for the five studies addressed what happens if an injury 

occurs to a study subject and informed the subjects that a physician would be 

available during the exposure session, if needed. 

The console the EPA uses to observe and monitor human study 

subjects during exposures (foreground) and a human subjects 

air pollution test chamber (background). (EPA OIG photo) 

14-P-0154 28 
 

 

However, the EPA’s policies, guidance, the five study protocols and consent forms 

do not include the EPA’s clinical follow-up responsibilities. While the follow-up 

NHEERL provided after each adverse event appeared reasonable, we could not 

compare it to any existing EPA criteria. ORD managers said that their clinical 

follow-up responsibilities do not include serving as a study subject’s health care 

provider and they are not aware of circumstances under which they would be 

allowed to serve as a health care provider. We interviewed three study subjects who 

experienced adverse events. Two study subjects commented that the EPA 

investigators handled the adverse events in a professional manner. One subject 

suggested that the EPA follow up with study subjects who experience an adverse 

event with an inquiry letter about 2 to 3 months after the event to determine how 

they are doing. In our view, clinical follow-up is an ethical responsibility and the 

EPA should revise its guidance to establish the agency’s clinical follow-up 

responsibilities for adverse events to ensure principal investigators and study team 

members understand their responsibilities. Once established, including a summary 

of the EPA’s clinical follow-up responsibilities in the consent forms would better 

ensure that human subjects understand the EPA’s clinical follow-up responsibilities 

prior to the occurrence of an adverse event. 

NHEERL Guidance Does Not Reflect the Adverse Event Definitions 

and the Reporting Timeframes the Agency Reports Using 

According to ORD managers, NHEERL uses the latest UNC-IRB SOP adverse 

event definitions and reporting timeframes to respond to adverse events. The 

Director of OHRE for UNC confirmed that it is the UNC-IRB’s expectation that 

the EPA follows the UNC-IRB SOP. However, this expectation is not in writing. 

For the five studies we reviewed, this would have been the 2009 UNC-IRB SOP. 

However, the NHEERL guidance does not state that the EPA adopted the UNCIRB SOP definitions and reporting timeframes. Further, NHEERL’s guidance 

definitions and timeframes are out of date because it still uses the 2003 UNC-IRB 

definitions and reporting timeframes. As a result, we evaluated how the EPA 

addressed adverse events using NHEERL guidance definitions and timeframes. 

Additionally, we also evaluated the adverse events using the 2009 UNC-IRB SOP 

definitions of adverse events and reporting requirements. Appendix D summarizes 

the NHEERL guidance and 2009 UNC-IRB definitions for various types of 

adverse events and unanticipated problems. In our view, the EPA needs to update 

its NHEERL guidance to clarify the adverse event definitions and reporting 

timeframes it is using during their human research subject studies, in order to 

ensure the consistent protection of study subjects. 

Two Adverse Events Not Reported In a Timely Manner When 

Evaluated Against Outdated NHEERL Guidance 

The EPA did not report two of the six adverse events it identified in a timely 

manner when evaluated against NHEERL guidance, because NHEERL guidance 

was out of date. According to agency managers NHEERL has been following the 

14-P-0154 29 
 

 

 

UNC-IRB SOP reporting requirements; however, these definitions and 

timeframes have not been reflected in the 2010 NHEERL guidance. Table 9 

summarizes the reporting timeframes contained in the NHEERL guidance and the 

2009 UNC-IRB SOP. 

Table 9: Reporting timeframes for adverse events and unanticipated problems 

Event/Problem 

NHEERL guidance reporting 

timeframes for adverse events 

and unanticipated problems 

2009 UNC-IRB SOP reporting timeframes for adverse 

events and unanticipated problems 

Serious adverse 

events 

Within 24 hours or by next working 

day if unanticipated; 5 working days 

if anticipated. 

Unanticipated problems that are serious adverse events 

should be reported to the IRB within 1 week of the 

investigator becoming aware of the event. 

Adverse event 

or experience 

Within 10 working days if 

unanticipated. 

Unanticipated problems that are adverse events should 

be reported to the IRB within 2 weeks of the investigator 

becoming aware of the problem. Adverse events that are 

not unanticipated problems are not required to be 

reported to the IRB. 

Unanticipated 

problems 

Within 10 working days. Unanticipated problems that are not serious adverse 

events should be reported to the IRB within 2 weeks of 

the investigator becoming aware of the problem. 

Sources: The EPA NHEERL Guidance, 2010 and UNC IRB SOP Number 19.0, 2009. 

In the XCON October 2010 adverse event, the study subject was hospitalized 

which, according to NHEERL’s guidance, would be defined as a serious adverse 

event. The principal investigator reported the unanticipated adverse event to the 

IRB 3 working days after the event. Serious and unanticipated adverse events 

should be reported within 24 hours according to NHEERL guidance. An 

NHEERL manager explained to us that the event was considered unanticipated 

but not a serious adverse event. The EPA took the study subject to the hospital for 

overnight observation, but she was not admitted. The manager interpreted the 

NHEERL guidance as stating that a serious adverse event would require a study 

subject to be hospitalized for a prolonged period of time. 

The EPA also did not report the DEPOZ April 2011 adverse event to the IRB in a 

timely manner according to 2010 NHEERL guidance. The study subject reported 

experiencing a persistent cough after an ozone exposure session. This adverse 

event was not serious but was unanticipated. The EPA reported the event to the 

IRB after 11 working days when the event should have been reported within 10 

working days according to NHEERL guidance. Adverse events must be reported 

in a timely manner so that the IRB can expeditiously determine if it needs to take 

any action concerning a study, such as requiring the EPA to revise the study 

protocols or the consent forms. 

Only one of the two events (DEPOZ, April 2011) would have been reported as 

untimely when evaluated against the 2009 UNC-IRB SOP reporting timeframes. 

According to the UNC-IRB SOP, unanticipated problems that are adverse events 

should be reported to the IRB within 2 weeks of the investigator becoming aware 

of the event; as noted above, the agency reported this adverse event after 

11 working days, or 15 calendar days. 

14-P-0154 30 
 

 

 

The other event (XCON, October 2010) was reported after 3 working days, or 

5 calendar days. This event would not have been classified as a serious adverse 

event using the UNC-IRB definitions because the study subject did not require 

inpatient hospitalization. The NHEERL guidance defines a serious adverse event 

as one in which the study subject “requires hospitalization.” 

Two Adverse Events Not Reported to the IRB When Evaluated 

Against Outdated NHEERL Guidance 

Despite meeting the NHEERL guidance definition for an adverse event, the EPA 

did not report two study events to the IRB in which both subjects experienced 

cardiac arrhythmias. These events met the 2010 NHEERL guidance definition for 

an adverse event, which is an event that is “undesirable and unintended, though 

not necessarily unanticipated, [with] injury or physical or emotional consequence 

to a human subject.” However, when evaluated against the 2009 UNC-IRB SOP 

definitions, neither of these adverse events would have been reported because 

they did not meet the criteria. In both instances, the subjects were not allowed to 

continue participating in the study and were advised to consult their 

physicians. Table 10 describes the two unreported events and the EPA’s 

explanations for not reporting them. 

Table 10: 2010 and 2011 events not reported as adverse events 

Study and date 

(month/year) 

of event Description of event The EPA explanation for not reporting as adverse event 

DEPOZ, Study subject had ventricular According to an NHEERL manager, the heart irregularity was 

November 2010 ectopic heart beats during 

holter monitoring on the 

follow-up day of the study. 

The EPA medical staff told 

the study subject to follow-up 

with a physician. 

benign. However, because of this irregularity, the EPA could 

monitor the heartbeat, but the results could not be interpreted, 

which was one of the data points being measured in the study. 

Therefore, the individual was disqualified from further 

participation in the study. 

XCON, Within seconds of the PM After the study subject was removed from the exposure 

February 2011 exposure, the study subject 

had an increase in heart rate 

that lasted several seconds, 

as well as variable blood 

pressure readings over the 

next half an hour. The EPA 

medical staff told the study 

subject to follow-up with a 

physician. 

chamber, the study subject told two EPA physicians about 

experiences with a “racing heart” although the study subject 

had responded during a screening visit that heart rate seems 

normal and that there were no episodes of very rapid heart 

rate which starts and stops suddenly. NHEERL staff 

determined that the event did not relate to the research since 

the study subject was only exposed to CAPS for a short time. 

The EPA did not inform the IRB at the time of the event, but 

the principal investigator sent the IRB a memo in October 

2012 (about 1.5 years later) explaining why the incident was 

not reported to the IRB. In an email dated October 15, 2012, 

the Director of the Office of Research Ethics for UNC 

concurred that this incident did not rise to the level of a 

reportable event. 

Source: NHEERL medical notes on study subject participants. 

During the OMEGACON study, the EPA reported two adverse events to the IRB 

(May 2010 and June 2010 in table 7) that were similar to the DEPOZ event 

(November 2010 in table 10). For the XCON event (February 2011 in table 10), 

14-P-0154 31 
 

 

the study’s principal investigator stated that the incident was not reported because 

the event was likely not related to the research. 

When evaluated against the 2009 UNC-IRB SOP definitions, neither of these 

adverse events would have been reported because they did not meet the criteria, 

which is that the event must be both unanticipated in nature and related to the 

research. The two events were likely not related to the research. An EPA study 

team member discovered the heart rhythm problems of one study subject (DEPOZ, 

November 2010) the day after the pollutant exposure session on a testing follow-up 

day. The other study subject (XCON, February 2011) was only exposed to CAPS 

for a short time and informed the EPA study team after the short exposure about 

having racing heart symptoms prior to the study. 

Other Matters 

The congressional request also asked that we determine whether the HSRB and 

the HSRRO were notified of adverse events and if consent forms were revised as 

needed after an adverse event. 

HSRB and HSRRO Notification Not Required After an Adverse Event 

Neither the Common Rule nor the EPA’s policy and guidance require the HSRB 

or HSRRO to be notified when adverse events occur. The EPA did not notify the 

HSRB about any of the five studies’ adverse events. Although not required, the 

EPA notified the HSRRO about one adverse event (XCON, October 2010) where 

the subject was sent to the hospital. 

One Study’s Consent Forms Revised After an Adverse Event 

The EPA revised one study’s consent form due to an adverse event. During the 

OMEGACON study one human study subject developed a migraine during an 

exposure session (OMEGACON, May 2010). The EPA decided to revise the 

OMEGACON study’s consent form to exclude potential human subjects with a 

history of migraines from participating in the study. The EPA did not revise the 

consent forms for the other adverse events. See explanations in table 7. We did 

not find that the consent forms for any of the other studies needed to be revised 

based on the identified adverse events. 

Conclusions 

The EPA needs to update its NHEERL guidance to clarify which adverse event 

definitions and reporting timeframes it is using. It is important to do so to provide 

consistent protection to the study subjects participating in NHEERL’s human 

subjects research studies. Principal investigators must understand what adverse 

definitions and reporting requirements they should use so they can identify and 

report adverse events in a timely manner. 

14-P-0154 32 
 

 

 

 

Although the EPA’s policies and guidance do not include the EPA’s clinical 

follow-up responsibilities, agency medical staff provided clinical follow-up to 

study subjects after each of the six EPA-identified adverse events. Establishing 

guidance for the agency’s clinical follow-up responsibilities after an adverse event 

would enhance the protection of human subjects who experience adverse events. 

The EPA should revise its guidance to address this key area. The EPA should also 

include a summary of its clinical follow-up responsibilities in its consent forms so 

that human subjects understand the EPA’s clinical follow-up responsibilities prior 

to the occurrence of an adverse event. 

Recommendation 

We recommend that the Assistant Administrator for Research and Development: 

8. Revise the EPA’s NHEERL Human Research Guidance to: 

a. Clearly state that NHEERL has adopted UNC-IRB SOP definitions and 

reporting timeframes for adverse events and unanticipated problems 

and will continue to follow the most updated version of the SOPs. 

b. Establish the EPA’s clinical follow-up responsibilities after adverse 

and serious adverse events, including general timeframes for clinical 

follow-up. 

c. Require principal investigators to include a summary of the agency’s 

clinical follow-up responsibilities in each study’s protocol and 

consent forms. 

Agency Comments and OIG Evaluation 

The agency concurred with all recommendations in Chapter 4 and 

provided acceptable planned corrective actions and completion dates for 

the recommendations. Recommendations 8(a), 8(b) and 8(c) are resolved 

and open with corrective actions ongoing. Appendix B contains the 

agency’s response to our draft report. Appendix C contains our detailed 

evaluation of that response. 

14-P-0154 33 
 

 

 

 

 

 

 

 

 

 

 

Status of Recommendations and 

Potential Monetary Benefits 

POTENTIAL MONETARY 

RECOMMENDATIONS BENEFITS (in $000s) 

Rec. 

No. 

Page 

No. Subject Status1 Action Official 

Planned 

Completion 

Date 

Claimed 

Amount 

Agreed-To 

Amount 

1 17 Revise the NHEERL Human Research Guidance to 

include: 

a. An EPA internal review and approval process 

for significant study modifications which 

include a definition and illustrative examples of 

significant study modifications. The review and 

approval process should indicate when 

significant study modifications should be sent 

to the HSRRO for review and approval. 

b. A revised flowchart of the protocol review 

process listing the HSRRO as an independent 

reviewer and not part of the NHEERL-level 

review process and eliminating the DHRO 

review. 

O Assistant Administrator 

for Research and 

Development 

9/30/14 

2 

3 

4 

5 

17 

17 

18 

25 

Implement a procedure for documenting that human 

subjects research study investigators have met the 

requirement for continuing annual ethics education. 

Revise the NHEERL Human Research Policy to 

eliminate the DHRO position and transfer the duties 

of the DHRO to the HRPO Director. 

Develop management controls to ensure NHEERL 

management reviews and approvals are properly 

documented and follow NHEERL guidance. 

Revise NHEERL Human Research Guidance to 

include a definition for “reasonably foreseeable risks” 

including illustrative examples of the types of 

information that should be included in the consent 

forms. 

C 

O 

O 

O 

Assistant Administrator 

for Research and 

Development 

Assistant Administrator 

for Research and 

Development 

Assistant Administrator 

for Research and 

Development 

Assistant Administrator 

for Research and 

Development 

10/31/13 

9/30/14 

9/30/14 

9/30/14 

6 

7 

25 

25 

Revise NHEERL Human Research Guidance to 

include procedures for ensuring that human subjects 

research consent forms consistently present the 

risks of the pollutants to which human subjects are 

exposed, including a summary of EPA assessments 

of short-term and long-term health effects and the 

upper pollutant concentration level for the pollutant to 

which the subject will be exposed. 

Include in its consent forms any known or likely 

carcinogenic effects of pollutants that the EPA uses 

in human exposure studies, based on EPA, other 

federal health agency, or other organization’s (as 

appropriate) assessment of such risks. If EPA uses 

the work of non-federal entities, the agency should 

document the basis for using non-federal information 

as opposed to the assessments of federal health 

agencies. 

O 

O 

Assistant Administrator 

for Research and 

Development 

Assistant Administrator 

for Research and 

Development 

9/30/14 

6/30/14 

14-P-0154 34 
 

 

 

RECOMMENDATIONS 

POTENTIAL MONETARY 

BENEFITS (in $000s) 

Rec. 

No. 

8 

Page 

No.

33 

 Subject 

Revise the EPA’s NHEERL Human Research 

Guidance to: 

a. Clearly state that NHEERL has adopted UNCIRB SOP definitions and reporting timeframes 

for adverse events and unanticipated 

problems and will continue to follow the most 

updated version of the SOPs. 

b. Establish the EPA’s clinical follow-up 

responsibilities after adverse and serious 

adverse events, including general timeframes 

for clinical follow-up. 

c. Require principal investigators to include a 

summary of the agency’s clinical follow-up 

responsibilities in each study’s protocol and 

consent forms. 

Status1 

O 

Action Official 

Assistant Administrator 

for Research and 

Development 

Planned 

Completion 

Date 

9/30/14 

Claimed 

Amount 

Agreed-To 

Amount 

O = recommendation is open with agreed-to corrective actions pending 

C = recommendation is closed with all agreed-to actions completed 

U = recommendation is unresolved with resolution efforts in progress 

14-P-0154 35 

1 
 

 

 

 

Appendix A 

EPA Air Quality Index for Particle Pollution 

(with concentration levels added) 

AQI level Risk level Description of risk PM2.5 concentration 

(range) 

0-50 Good None 0 to 12 μg/m3 

51-100 Moderate Unusually sensitive people should consider reducing 

prolonged or heavy exertion. 12.1 to 35.4 μg/m3 

101-150 

Unhealthy for 

sensitive 

groups 

People with respiratory or heart disease, the elderly 

and children should limit prolonged exertion. 35.5 to 55.4 μg/m3 

151-200 

Unhealthy People with respiratory or heart disease, the elderly 

and children should avoid prolonged exertion. 

Everyone else should limit prolonged exertion. 

55.5 to 150.4 μg/m3 

201- 300 

Very unhealthy People with respiratory or heart disease, the elderly 

and children should avoid any outdoor activity. 

Everyone else should avoid prolonged exertion. 

150.5 to 250.4 μg/m3 

301-500 

Hazardous Everyone should avoid any outdoor exertion. People 

with respiratory or heart disease, the elderly and 

children should remain indoors. 

250.5 to 500.4 μg/m3 

Source: EPA’s Air Quality website (http://www.airnow.gov/?action=aqibasics.aqi and 

http://www.airnow.gov/index.cfm?action=resources.aqi_conc_calc). 

Note: The Air Quality Index range for PM2.5 is 0 to 500. 

14-P-0154 36 
 

 

 

 

Appendix B 

Agency Comments on Draft Report

[This appendix includes OIG-created footnotes regarding subsequent communications with the agency after their 

response was received concerning their planned corrective actions and completion dates.] 

February 18, 2014 

MEMORANDUM 

SUBJECT:	 Response to the Office of Inspector General (OIG) Draft Report titled, 

Improvements in Policies and Guidance Could Better Ensure Protection of 

Human Study Subjects, dated January 17, 2014 

FROM: Lek G. Kadeli, Acting Assistant Administrator 

TO: Arthur A. Elkins, Jr., Inspector General 

Office of Inspector General 

Thank you for the opportunity to respond to the OIG’s draft report, Improvements in Policies and 

Guidance Could Better Ensure Protection of Human Study Subjects, (Project No.OPE-FY13-0001). 

The Office of Research and Development (ORD) appreciates the OIG’s recognition that the EPA 

followed all applicable regulations governing our human subjects research program. This 

acknowledgement is based on the OIG’s findings, which demonstrated that EPA obtained all 

required approvals before initiating a study, obtained proper informed consents from research 

participants, complied with its clinical follow-up responsibilities for all research participants, and 

responded appropriately to adverse events, according to the latest guidelines of the University of 

North Carolina Institutional Review Board (the board of record for EPA research taking place in 

Chapel Hill). 

EPA policy decisions must be based on sound science. To that end, ORD’s human subjects 

research program generates critical and objective scientific data to inform the development of 

policies that may reduce the effects of air pollutants on human health. 

While there is a critical need for studies involving human subjects, ORD also understands that 

the research must be conducted in an ethical and vigilant manner. As documented in the OIG’s 

report, EPA has established guidelines for conducting this type of research that are far in excess 

of what is normally required by universities, industry, and other government agencies conducting 

human studies research. For example, ORD’s research program can sometimes undergo over 

eight separate levels of approvals before a research study can be initiated, which may include 

statistical and medical review of the study, IRB review, Quality Assurance Officer review, and 

review by at least three other officials, whose approvals must be documented before a study can 

begin. 

In conclusion, ORD appreciates the OIG’s efforts and thorough evaluation of our research 

program. ORD concurs with the OIG’s recommendations, which are primarily based on 

14-P-0154 37 
 

 

enhancing our internal policies and guidance to further improve the EPA’s human studies 

program. 

Provided in the table below is ORD’s response to the OIG’s recommendations. 

Rec. Recommendation Suggested Corrective Action Estimated 

No. Revisions to 

Recommendation(s) 

(If applicable) 

Completion 

Date 

1. Revise the NHEERL 

guidance to include: 

An EPA internal review 

and approval process for 

significant study 

modifications which 

include a definition and 

illustrative examples of 

significant study 

modifications. The review 

and approval process 

should indicate when 

significant study 

modifications should be 

sent to the HSRRO for 

approval. 

A revised flowchart of the 

protocol review process 

listing the HSRRO review 

as an independent 

reviewer and not part of 

the NHEERL-level review 

process and eliminating 

the DHRO review. 

A job offer has been 

made to fill the HRPO 

position. Updating the 

NHEERL guidance 

policy will be the first 

task assigned to the 

HRPO. Included in the 

updated guidance will 

be a definition of what 

constitutes a significant 

study modification and 

who must review and 

approved such 

modifications. The 

flowchart will also be 

revised as suggested by 

the IG. 

4th Quarter 

FY 2014 

(assuming 

the HRPO 

position is 

filled by the 

Spring of 

2014). 

See Appendix C, Note 1, for OIG Response 

2. Implement a procedure for 

documenting that human 

subjects research study 

investigators have met the 

requirement for 

continuing annual ethics 

education. 

A procedure was 

implemented in the fall 

of 2013. 

Completed 

See Appendix C, Note 2, for OIG Response 

14-P-0154 38 
 

 

 

 

     

                             

                              

Rec. 

No. 

Recommendation Suggested 

Revisions to 

Recommendation(s) 

(If applicable) 

Corrective Action Estimated 

Completion 

Date 

3. Revise the NHEERL 

policy to eliminate the 

DHRO position and 

transfer the duties of the 

DHRO to the HRPO 

Director. 

This will be done as part 

of the HRPO revision of 

the NHEERL 

guidelines. 

4th Quarter 

FY 2014. 

See Appendix C, Note 3, for OIG Response 

4. Develop management 

controls to ensure 

NHEERL management 

reviews and approvals are 

properly documented and 

follow NHEERL 

guidance. 

 NHEERL guidelines 

will be modified to 

specifically indicate the 

order in which reviews 

and approvals should be 

obtained. 

4th

 Quarter 

FY 2014. 

See Appendix C, Note 4, for OIG Response 

5. Revise NHEERL 

guidance to include a 

definition for “reasonably 

foreseeable risks” 

including illustrative 

examples of the types of 

information that should be 

included in the consent 

forms. 

Revise NHEERL 

guidance to include 

the federal definition 

of “minimal risk” 

including illustrative 

examples of the 

types of information 

that should be 

included in the 

consent forms.8 

NHEERL guidelines 

will be modified to 

include the federal 

definition of minimal 

risk as well as 

illustrative examples. 

4th

 Quarter 

FY 2014. 

See Appendix C, Note 5, for OIG Response 

8

In a subsequent communication, ORD agreed to implement the OIG recommendation, stating that “NHEERL

guidelines will be modified to include a definition for “reasonably foreseeable risks” including illustrative examples

of the types of information that should be included in the consent form.” The Agency also provided corrective 

actions and milestones for Recommendation 5. 

14-P-0154 39 
 

 

 

 

 

Rec 

No. 

Recommendation Suggested Revisions to 

Recommendation(s) (If 

applicable) 

Corrective Action Estimated 

Completion 

Date 

6. Revise NHEERL 

guidance to include 

procedures for ensuring 

that human subjects 

research consent forms 

consistently present the 

risks of the pollutants to 

which human subjects 

are exposed, including a 

summary of EPA 

assessments of short-term 

and long-term health 

effects and the upper 

pollutant concentration 

level for the pollutant to 

which the subject will be 

exposed. 

The HRPO will enhance 

what is currently in place 

regarding language about 

risks of pollutants to which 

subjects are exposed and 

will ensure this language be 

placed as appropriate in 

consent forms. Interim 

steps have already been 

taken to ensure that the 

communication of risk from 

studies are consistently 

presented throughout the 

consent form process. 

4th Quarter FY 

2014. 

See Appendix C, Note 6, for OIG Response 

7. Include in its consent 

forms any known 

carcinogenic effects of 

pollutants that the EPA 

uses in human exposure 

studies. 

While the current consent This recommendation will 

forms do present be implemented 

information regarding the immediately. 

present risk of pollutants the 

EPA uses in human 

exposure studies, additional 

information will be included 

for any known carcinogenic 

effects of the pollutants (as 

reported by IARC, National 

Toxicology Program’s 

Report on Carcinogens, or 

by the EPA IRIS Program). 

In general, the potential for 

carcinogenic effects is not 

anticipated to be greater than 

minimal.9 

2nd 

Quarter10 

FY 2014. 

See Appendix C, Note 7, for OIG Response 

9

At the exit conference, the OIG and ORD amended Recommendation 7 to include not only pollutants with known 

carcinogenic effects, but also those pollutants with likely carcinogenic effects. ORD proposed that we use EPA’s 

two highest cancer risk guidelines. According to EPA’s 2005 Cancer Guidelines, these are “Carcinogenic to 

Humans” and “Likely to be Carcinogenic to Humans.” In a subsequent communication, ORD agreed to the new 

language and confirmed that the corrective actions for Recommendation 7 would remain the same.

10 In a subsequent communication, the agency revised the corrective action date to June 30, 2014. 

14-P-0154 40 
 

 

 

 

   

                                   

                             

                               

       

 

Rec. 

No. 

Recommendation Suggested Revisions to 

Recommendation(s) (If 

applicable) 

Corrective Action Estimated 

Completion 

Date 

8. Revise the EPA’s 

NHEERL guidance to: 

a. Clearly state that 

NHEERL has adopted 

UNC-IRB SOP 

definitions and 

reporting timeframes 

for adverse events and 

unanticipated 

problems and will 

continue to follow the 

most updated version 

of the SOPs. 

b. Establish the EPA’s 

clinical follow-up 

responsibilities after 

adverse and serious 

adverse events, 

including general 

timeframes for clinical 

follow-up. 

c. Require principal 

investigators to 

include a summary of 

the agency’s clinical 

follow-up 

responsibilities in each 

study’s protocol and 

consent forms. 

Changes to the NHEERL 

guidance document will 

be made to ensure that 

the NHEERL policy is 

always harmonized with 

the latest IRB policy. 

The IG is aware that EPA 

physicians are not legal 

health care providers. 

The guidelines will be 

modified to describe 

EPA’s follow-up clinical 

responsibilities. 

EPA will discuss with the 

UNC IRB whether they 

want to see a summary of 

the Agency’s clinical 

followup responsibilities 

included in consent 

forms, and include it if 

the IRB recommends we 

do so.11 

4th Quarter 

FY 2014. 

11 In subsequent communications, ORD agreed to implement OIG recommendation 8(c), stating that “Changes to

the NHEERL guidance document will be made to ensure that the NHEERL policy is always harmonized with the

latest IRB policy. The NHEERL guidance document will be updated to specify EPA’s follow‐up clinical

responsibilities,” and clarified that a summary of these responsibilities would be included in study protocols and

consent forms. The agency also provided corrective actions and milestones for Recommendation 8(c). 

14-P-0154 41 
 

 

 

 

 

 

 

 

Appendix C 

OIG Evaluation of Agency Comments 

We appreciate the agency’s comments and its recognition of our efforts to understand and 

thoroughly evaluate its human research studies program. Our evaluation of the agency’s 

comments on our draft report recommendations is below. We also received a number of 

technical comments from the agency and made changes to the final report as appropriate. 

Note 1- Response to Recommendations 1(a) and 1(b): 

The agency concurred with our recommendation to revise the NHEERL guidance to include an 

EPA internal review and approval process for significant study modifications and to revise the 

flowchart to accurately portray the protocol review process. The agency provided a corrective 

action plan stating that a job offer has been made to fill the HRPO position, and that updating the 

NHEERL guidance to address the OIG’s recommendation will be the first task assigned to the 

HRPO. This includes developing guidance on what constitutes a significant study modification, 

who must review and approve such modifications, and correcting the flowchart. The agency 

explained that the NHEERL guidance will be updated by September 30, 2014, unless the vacant 

HRPO position remains unfilled. We accept the agency’s planned corrective actions in response 

to this recommendation. The recommendation is resolved and open pending the agency’s 

completion of the agreed-to corrective actions.12 

Note 2- Response to Recommendation 2: 

The agency concurred with our recommendation to implement a procedure for documenting that 

investigators have met the requirement for continuing annual ethics education and began 

implementing such a procedure in the fall of 2013. The agency provided evidence of the 

corrective action taken. We accept the agency’s corrective action in response to this 

recommendation. This recommendation is closed and no further action is required. 

Note 3- Response to Recommendation 3: 

The agency concurred with our recommendation to revise the NHEERL policy to eliminate the 

DHRO position and transfer those duties to the HRPO Director. The agency provided a 

corrective action plan stating that the transfer of these duties to the HRPO will be done as part of 

the HRPO revision of the NHEERL guidelines. The agency said this corrective action would be 

completed by September 30, 2014. We accept the agency’s planned corrective action in response 

to this recommendation. The recommendation is resolved and open pending the agency’s 

completion of the agreed-to corrective actions. 

Note 4- Response to Recommendation 4: 

The agency concurred with our recommendation to develop management controls to ensure 

NHEERL management reviews and approvals are properly documented. The agency provided a 

12 Subsequent to this action plan, NHEERL management provided us with clarification about the responsible 

party/office for recommendations 1, 3, 4, 5, 6, 7 and 8. 

14-P-0154 42 
 

 

 

 

corrective action plan stating that the NHEERL guidelines will be modified to specifically 

indicate the order in which reviews and approvals should be obtained by September 30, 2014. 

We accept the agency’s planned corrective action in response to this recommendation. The 

recommendation is resolved and open pending the agency’s completion of the agreed-to 

corrective actions. 

Note 5- Response to Recommendation 5: 

The agency proposed an alternative to Recommendation 5. The agency proposed that we change 

our recommendation that NHEERL’s guidance include a definition for “reasonably foreseeable 

risks” to recommend, instead, that the guidance include a definition of “minimal risk.” After 

discussing the proposed alternative at the exit conference, NHEERL management concurred with 

our recommendation to revise NHEERL guidance to include a definition for “reasonably 

foreseeable risks.” In a subsequent communication, the agency stated that “NHEERL guidelines 

will be modified to include a definition for “reasonably foreseeable risks” including illustrative 

examples of the types of information that should be included in the consent form.” The agency 

provided a corrective action plan stating that this corrective action would be completed by 

September 30, 2014. We accept the agency’s planned corrective action in response to this 

recommendation. The recommendation is resolved and open pending the agency’s completion of 

the agreed-to corrective actions. 

Note 6- Response to Recommendation 6: 

The agency concurred with our recommendation to revise NHEERL guidance to include 

procedures for ensuring that consent forms consistently present the risks of pollutants, including 

EPA assessments of short-term and long-term health effects and the upper pollutant 

concentration levels of the pollutants. The agency stated that they will “enhance what is currently 

in place regarding language about risks of pollutants and that they have already taken interim 

steps to ensure that the communication of risks is consistently presented through the consent 

form process.” Subsequent to providing the OIG with its action plan, NHEERL management 

clarified their proposed corrective actions for this recommendation. NHEERL said it would 

develop language to ensure that consent forms consistently present the same risks of pollutants, 

including the risks of short- and long-term health effects associated with pollutant exposure, and 

the upper-pollutant concentration. We accept the agency’s planned corrective action in response 

to this recommendation, as clarified in subsequent communications with NHEERL management. 

The recommendation is resolved and open pending the agency’s completion of the agreed-to 

corrective actions. 

Note 7- Response to Recommendation 7: 

The agency concurred with our recommendation to include in its consent forms any known 

carcinogenic effects of pollutants, and suggested that the recommendation include specific health 

institutions to ensure the credibility of a finding of carcinogenic effects. The agency also 

recommended that the International Agency for Research on Cancer, an associated body of the 

World Health Organization, be included in the list of agencies. We agree with the agency’s 

suggestion to include other federal health agencies, and acknowledge the agency’s desire to use 

other organizations such as the IARC, as appropriate. We have revised the recommendation to 

14-P-0154 43 
 

 

 

specify EPA and other federal health agencies,13 as appropriate, and to recommend the agency 

document the basis for using non-federal information as opposed to the assessments of federal 

health agencies. During the exit conference, the OIG and ORD also agreed to amend 

recommendation 7 to include not only pollutants with known carcinogenic effects, but also 

pollutants with likely carcinogenic effects. In a subsequent communication, ORD agreed to the 

amended recommendation and provided a corrective action plan stating that it would revise its 

consent forms by June 30, 2014. We accept the agency’s planned corrective action in response to 

this recommendation. The recommendation is resolved and open pending the agency’s 

completion of the agreed-to corrective actions. 

Note 8 - Response to Recommendations 8(a), 8(b) and 8(c): 

The agency concurred with our recommendation to revise the NHEERL guidance to: (a) clearly 

state that NHEERL has adopted UNC-IRB SOP definitions and reporting timeframes for adverse 

events and unanticipated problems, (b) establish the EPA’s clinical follow-up responsibilities 

after adverse and serious adverse events, and (c) require principal investigators to include a 

summary of clinical follow-up responsibilities in the study protocol and consent forms. We 

accept the agency’s planned corrective actions to ensure that NHEERL guidance is harmonized 

with the latest IRB policy and to revise the guidance to describe EPA’s clinical follow-up 

responsibilities. The agency provided a corrective action plan stating that it would revise the 

adverse events section of the NHEERL guidance to ensure that NHEERL policy is always 

harmonized with the latest IRB policy, and that it would describe EPA’s follow up clinical 

responsibilities. The agency committed to completing these actions by September 30, 2014. We 

accept the agency’s planned corrective action in response to recommendations 8(a) and 8(b). 

Recommendations 8(a) and 8(b) are resolved and open pending the agency’s completion of the 

agreed-to corrective actions. 

Regarding recommendation 8(c), the agency said that it would discuss with the UNC-IRB whether 

it wants to see a summary of the agency’s clinical follow-up responsibilities included in consent 

forms, and include it if the IRB recommends the agency do so. After discussing the agency 

response at the exit conference, NHEERL management concurred with our recommendation to 

include a summary of the agency’s clinical follow-up responsibilities in the study protocol and 

consent forms. In a subsequent communication, the agency stated that “Changes to the NHEERL 

guidance document will be made to ensure that the NHEERL policy is always harmonized with 

the latest IRB policy. The NHEERL guidance document will be updated to specify EPA’s 

follow-up clinical responsibilities.” Further, the agency clarified that a summary of these 

responsibilities would be included in study protocols and consent forms. The agency said this 

corrective action would be completed by September 30, 2014. We accept the agency’s planned 

corrective action in response to this recommendation. The recommendation is resolved and open 

pending the agency’s completion of the agreed-to corrective actions. 

13Other federal health agencies would include the National Institute of Health, National Toxicology Program, Health 

and Human Services, the Center for Disease Control, National Institute of Environmental Health Sciences, and 

Agency for Toxic Substances and Disease Registry. 

14-P-0154 44 
 

 

Appendix D 

Definitions of Adverse Events and 

Unanticipated Problems 

Event/Problem NHEERL guidance definitions 2009 UNC IRB-SOP definitions 

Serious adverse 

events 

Fatal or life-threatening; results in 

significant or persistent disability; 

require[s] or prolong[s] hospitalization; 

result[s] in a congenital anomaly/birth 

defect; or, in the opinion of the 

investigators, represents other 

significant hazards or potentially serious 

harm to research subjects or others. 

A serious adverse event is any adverse event 

temporally associated with the subject’s participation 

in research that meets any of the following criteria: 

• results in death; 

• is life-threatening (places the subject at immediate 

risk of death from the event as it occurred); 

• requires inpatient hospitalization or prolongation of 

existing hospitalization; 

• results in a persistent or significant 

disability/incapacity; 

• results in a congenital anomaly/ birth defect; or 

• any other adverse event that, based upon 

appropriate medical judgment, may jeopardize the 

subject’s health and may require medical or surgical 

intervention to prevent one of the other outcomes 

listed in this definition. 

Adverse event Undesirable and unintended, though not An adverse event or adverse experience is any 

or experience necessarily unanticipated, injury or 

physical or emotional consequence to a 

human subject. 

untoward or unfavorable medical occurrence in a 

human subject, including any abnormal sign (for 

example, abnormal physical exam or laboratory 

finding), symptom, or disease, temporally associated 

with the subject’s participation in the research, 

whether or not considered related to the subject’s 

participation in the research. Adverse events 

encompass both physical and psychological 

harms and occur most frequently in the context of 

biomedical research, although they can occur in the 

context of social and behavioral research. 

Adverse events that are not unanticipated problems 

are not required to be reported to the IRB. 

Unanticipated May or may not include specific events An unanticipated problem refers to any incident, 

problems experienced by individual subjects, but 

are developments within the research 

activity that suggest a potential for 

increased risks to subjects or others. 

experience, or outcome that: 

• is unexpected (in terms of nature, severity, or 

frequency) given (a) the research procedures that are 

described in the protocol-related documents, such as 

the IRB-approved research protocol and informed 

consent document; and (b) the characteristics of the 

subject population being studied; 

• is related or possibly related to a subject’s 

participation in the research; and 

• suggests that the research places subjects or others 

at a greater risk of harm (including physical, 

psychological, economic, or social harm) related to 

the research than was previously known or 

recognized. 

Note that for UNC reporting purposes an event that 

satisfies the first two criteria will be considered 

reportable. 

Source: The EPA NHEERL Guidance, 2010, and UNC IRB standard operating procedure Number 19.0, 2009. 

14-P-0154 45 
 

 

Appendix E 

Distribution 

Office of the Administrator 

Assistant Administrator for Research and Development 

Deputy Assistant Administrator for Science, Office of Research and Development 

Principal Deputy Assistant Administrator, Office of Research and Development 

Agency Follow-Up Official (the CFO) 

Agency Follow-Up Coordinator 

General Counsel 

Associate Administrator for Congressional and Intergovernmental Relations 

Associate Administrator for External Affairs and Environmental Education 

Audit Follow-Up Coordinator, Office of Research and Development 

14-P-0154 46