1

January 20, 2015 

EPA-HSRB-14-03 

Thomas Burke, Ph.D. 

EPA Science Advisor 

Office of the Science Advisor 

1200 Pennsylvania Avenue, NW 

Washington, DC 20460 

Subject: November 5, 2014 EPA Human Studies Review Board Meeting Report 

Dear Dr. Burke, 

The United States Environmental Protection Agency (EPA or Agency) requested that the 

Human Studies Review Board (HSRB) provide scientific and ethics reviews of two items: a new 

open pour protocol proposed by the Agricultural Handler Exposure Task Force, LLC (AHETF) 

and a pre-Rule publication by Frampton et al. (2002) concerning human exposure to nitrogen 

dioxide. The Board’s key responses to the charge questions are summarized in this letter and are 

detailed in the enclosed final meeting report. 

AHETF Protocol (AHE 170) – Open Pour Loading of Granules

Science

• The Board concludes that given the clearly defined boundaries of the protocol, if 

performed as described and as recommended by the HSRB, the AHE170 protocol is 

likely to generate scientifically reliable data that will be useful for assessing the exposure 

of those who perform open pour loading of granular pesticide products. 

Ethics

• If the research is performed as described, it is likely to meet the applicable requirements 

of 40 CFR part 26, subparts K and L. 

A Published Report by Frampton et al. (2002) on Nitrogen Dioxide Effects on Airway and

Blood Cells

Science

• The sample collection and laboratory analysis parts of this study are scientifically sound. 

However, the chamber exposure data are not reliable without more information regarding 

the actual exposure levels to nitrogen dioxide. 
2

• The statistical analysis is adequate to justify the significant differences that the authors 

identified, but is not adequate to detect all of the differences that may have occurred. This 

study may be used in a quantitative way as part of a weight-of-evidence analysis to 

support effects that might occur at the exposure levels reported (0.6 and 1.5 ppm), but 

this study (as published) is not relevant to support the existence of no effects at the 

claimed levels of exposure. 

Ethics

• The Board concluded that the published report by Frampton et al. (2002) submitted for 

review meets the applicable requirements of 40 CFR part 26 subpart Q, and that the data 

within this article may be considered acceptable for EPA’s reliance, contingent upon the 

determination of their scientific validity. 

 

Sincerely, 

Rebecca T. Parkin, PhD, MPH 

Chair 

EPA Human Studies Review Board 
3

NOTICE 

This report has been written as part of the activities of the EPA Human Studies Review Board, a 

Federal advisory committee providing advice, information and recommendations on issues 

related to scientific and ethical aspects of human subjects research. This report has not been 

reviewed for approval by the Agency and, hence, the contents of this report do not necessarily 

represent the view and policies of the Environmental Protection Agency, nor of other agencies in 

the Executive Branch of the Federal government, nor does the mention of trade names or 

commercial products constitute a recommendation for use. You may obtain further information 

about the EPA Human Studies Review Board from its website at http://www.epa.gov/osa/hsrb. 

You may also contact the HSRB Designated Federal Officer, via e-mail at ord-osahsrb@epa.gov 

In preparing this document, the Board carefully considered all information provided and 

presented by the Agency presenters, as well as information presented by public commenters. 

This document addresses the information provided and presented within the structure of the 

charge by the Agency. 
4

US ENVIRONMENTAL PROTECTION AGENCY 

HUMAN STUDIES REVIEW BOARD 

Chair 

Rebecca Parkin, Ph.D., M.P.H., Professorial Lecturer, Environmental and Occupational Health 

and Epidemiology & Biostatistics, Milken Institute School of Public Health, The George 

Washington University, Washington, DC 

Vice Chair

Jewell H. Halanych, M.D., Assistant Professor, Internal Medicine Residency Program, 

Montgomery Regional Campus, University of Alabama at Birmingham, Montgomery, AL 

Members 

Gary L. Chadwick, Pharm.D., M.P.H., C.I.P., Senior Consultant, HRP Consulting Group, Inc. 

Fairport, NY 

Liza Dawson, Ph.D., Research Ethics Team Leader, Division of AIDS National Institutes of 

Health (NIH), National Institute of Allergy and Infectious Disease (NIAID), Bethesda, MD 

George Fernandez, Ph.D., Statistical Training Specialist, SAS Institute, Statistical Training and 

Technical Services, Sparks, NV 

Kyle L. Galbraith, Ph.D., Manager, Human Subjects Protection, Carle Foundation Hospital, 

Urbana, IL 

Edward Gbur, Jr., Ph.D., Professor, Agricultural Statistics Laboratory, University of Arkansas, 

Fayetteville, AR 

Elizabeth Heitman, Ph.D., Associate Professor of Medical Ethics, Center for Biomedical Ethics 

and Society, Vanderbilt University Medical Center, Nashville, TN 

John C. Kissel, Ph.D., Professor, Department of Environmental and Occupational Health, School 

of Public Health, University of Washington, Seattle, WA 

Randy Maddalena, Ph.D., Physical Research Scientist, Indoor Environment, Lawrence Berkeley

National Laboratory, Berkeley, CA 

William Popendorf, Ph.D., M.P.H., Professor Emeritus, Department of Biology, Utah State 

University, Logan, UT 
5

Kenneth Ramos, M.D., Ph.D., Pharm.B., Associate Vice President, Precision Health Sciences, 

Professor of Medicine, Arizona Health Sciences Center, Tucson, AZ 

Suzanne M. Rivera, Ph.D., M.S.W., Vice President for Research and Technology Management, 

Case Western University, Cleveland, OH 

Jun Zhu, Ph.D., Professor of Statistics and of Entomology, Department of Statistics, University

of Wisconsin – Madison, Madison, WI

Human Studies Review Board Staff 

Jim Downing, Executive Director, Human Studies Review Board Staff, Office of the Science 

Advisor, United States Environmental Protection Agency, Washington, DC INTRODUCTION 

On November 5, 2014, the United States Environmental Protection Agency’s (EPA or Agency) 

Human Studies Review Board (HSRB or Board) met to address the scientific and ethical charge 

questions related to two agenda items: a new protocol proposed by the Agricultural Handler 

Exposure Task Force, LLC on open pouring of granules (AHE170), and a published article by

Frampton et al. (2002) regarding airway and blood cell responses to nitrogen dioxide exposures. 

REVIEW PROCESS 

The Board conducted a public meeting using Adobe Connect1, on November 5, 2014. Advance 

notice of the meeting was published in the Federal Register as “Human Studies Review Board; 

Notification of a Public Meeting” (EPA, 2014, pp. 62437-62439). 

Following welcoming remarks from Agency officials, the Board heard presentations from EPA 

for the two agenda items in sequence. This Final Report of the meeting describes the HSRB’s 

discussion, recommendations, rationale and consensus in response to each charge question. 

For each agenda item, Agency staff first presented their review of the science and the Board 

asked the Agency presenters clarifying questions. The staff then described their review of the 

ethical aspects and the Board asked clarifying questions about those. The Board solicited public 

comments and next asked Agency staff to read the Charge Questions for the document under 

consideration. The Board discussed the science questions first and then the ethics question. The 

Chair then called for a vote to confirm concurrence on a summary statement in response to each 

charge question. 

For their evaluation and discussion, the Board considered materials presented at the meeting, oral 

comments, the original protocol and published report, related materials and published articles, 

and the Agency’s science and ethics reviews. There were no public comments presented at this 

meeting. A comprehensive list of background documents is available online at 

http://www.epa.gov/hsrb/. 

 

1 Accessed at http://epa.connectsolutions.com/hsrb.
6

CHARGE TO THE BOARD AND BOARD RESPONSE 

AHETF Protocol (AHE 170) – Open Pour Loading of Granular Pesticide Products

Overview of the Study 

The Agricultural Handler Exposure Task Force, LLC (AHETF) designs and implements study

protocols to assess workers’ exposures in a variety of workplace scenarios. In this proposal, 

AHETF plans to evaluate experienced handlers’ dermal and inhalation exposures to conventional 

pesticide formulations in granular form (e.g., passable through 4-79 mesh sieves). The study will 

focus on open pour loading of pesticides from non-bulk containers into typical agricultural 

application equipment; application of these pesticides will not be evaluated in this study. EPA 

plans to use the results to estimate exposures to a variety of pesticides loaded in comparable 

conditions. 

A 7x3 study design has been selected. Seven monitoring areas (MA) across the U.S. have been 

identified to represent the various climatic, crop and geographic conditions under which the test 

substances are applied. Within each area, three monitoring units (MU) will be determined; one 

handler will comprise each MU. The sample of handlers will be identified using a two-step 

process: screening for appropriate and willing commercial growers, who have appropriately 

experienced and interested workers; potential participants will be recruited through notices 

and/or meetings. Similarity restrictions will be used to ensure purposive diversity of the study. 

Eligible participants will be provided with written informed consent materials and time to review 

them before deciding whether they will take part in the study. Participants will be given an 

opportunity in the consent process to request their individual research results. 

Growers will select one pesticide from a list of 10, known to be stable and having well-known 

analytical methods, for use on their property. During regular work days, participants will conduct

typical but partially scripted tasks; e.g., transferring 50-lb. bags into spreaders, stacking bags, 

loading and unloading bags, and/or cleaning up equipment. One of the three ranges of the 

amount of the active ingredient handled (AaiH), from 5 to 400 lbs. AaiH, will be randomly

assigned to the 4+ hour monitoring period for each MU. At least three loads will be poured per 

monitoring period. The workers will wear two layers of clothing: full-body work clothing plus a 

long-sleeved, long-legged whole body dosimeter (WBD) underneath. The under layer will be 

used to assess whole-body dermal exposures. Face/neck wipe and hand wash samples will be 

collected before exposure, before breaks and at the end of the monitoring period. Breathing zone 

air samples will be collected using standard personal air monitoring equipment (either fiber 

filters in cartridges or Occupational Safety and Health Agency (OSHA) Versatile Sampler (OVS) 

tubes with personal air sampling pumps). Fortified field samples will be used in triplicate to 

assess the stability of the active ingredient (AI) in the field, during transport and storage, and on 

sampling materials, such as face/neck wipes and the dosimeter. The study team will observe 

participants during their monitoring periods and note environmental conditions, personal 

protective equipment used, and participant activities. 
7

The primary statistical objective is to estimate exposures within three-fold of the actual 

population values. The secondary objective is to ensure that the study has at least 80% power to 

distinguish proportionality from independence between exposure and AaiH. 

Science 

Charge to the Board 

• Is this research likely to generate scientifically reliable data, useful for assessing the 

exposure of those who perform open pour loading of granular pesticide products? 

Board Response to the Charge 

HSRB Recommendations 

• The Board concludes that given the clearly defined boundaries of the protocol, if 

performed as described and as recommended by the HSRB, the AHE170 protocol is 

likely to generate scientifically reliable data that will be useful for assessing the exposure 

of those who perform open pour loading of granular pesticide products. 

HSRB Detailed Recommendations and Rationale

Overall, the Board agreed with the Agency’s scientific assessment that the AHE170 protocol 

“addresses the technical aspects of applicable exposure monitoring guidelines and is likely to 

produce scientifically valid and useful data” (Evans and Sherman, 2014, p. 2) for assessing the 

exposure of those who perform open pour loading of granular pesticide products. Some aspects 

of the study are commented on below; these provide opportunities to improve the design or 

identify points that should be considered by the Agency when interpreting and using the data. 

1. Surrogate Related Issues 

The AHETF has identified a range of pesticides to use as surrogates for all active ingredients in 

granular formulations. The use of multiple pesticides as surrogates increases the chances of 

successfully identifying representative MUs for each AaiH level in each MA. The protocol states 

that “any of the AHETF surrogates can be used for generating exposure data for this scenario” 

(Collier, 2014, p. 35). However, the validity of this assertion for a given surrogate pesticide 

depends on the persistence of that pesticide in the granular formulation over the period of time 

between the start of exposure and the collection of the dosimeter or skin washes. If a given 

surrogate pesticide is lost from the formulation by volatilization, degradation, or by irrecoverable 

sorption into the skin during the exposure period, then the resulting exposure measurement for 

that specific pesticide will be biased low and result in an underestimate of exposure. 

Differences in volatilization for each surrogate pesticide can be evaluated in part by comparing 

results of the measured vapor-phase inhalation exposure (measured with the OVS tubes) relative 

to the particle bound formulation (measured on the OVS filter). Differences in the degradation

rate on the dosimeter and air samplers for the surrogates can be determined in part by the 

recovery of spiked pesticide from the fortified samplers if volatilization is found to be negligible. 
8

The data analysis for the protocol should include, at a minimum, a qualitative assessment of the 

relative persistence of the selected surrogate pesticides over the duration of the exposure. For 

those surrogates that are not completely conserved in/on the granular formulation and/or on the 

WBD, the data analysis should include a quantitative assessment of chemical loss and how that 

loss impacts the measured exposure for the open pour loading of granules. 

Quantifying the loss of surrogate pesticide due to irrecoverable dermal sorption is more difficult 

in the absence of absorbed dose (AD) measurements, which are beyond the scope of the protocol. 

Clearly the loss of surrogate pesticide during the monitoring period by sorption into the dermal 

layer will bias the exposure measurements low, but it also has implications for the safety of the 

study and the calculation of margins of exposure (MOEs) as discussed below. The archived 

literature is rich with information about the performance of WBD (cotton long underwear) as 

dosimeters for dermal exposure, particularly as a primary outer garment (i.e., as used in the 

JazzerciseTM exposure assessment scenario2). The efficiency of WBDs is affected by the 

efficiency of the fabric in collecting and retaining particles for the duration of the scenariobased 

activity, and retaining AI in the light of volatilization and loss via transfer to the skin. It is not 

clear whether there is good information on the relationship between the WBD-based exposure 

data and absorbed dose during field studies. For example, the protocol did not include 

information about how factors such as temperature and sweat saturation of the WBD garment 

would influence the estimation of absorbed dose from a whole body dosimeter. 

2. Non-conservatism of the protocol 

Non-conservatism of study results can be produced from uncertainty in the relationship between 

1) exposure estimates based on face/neck wipe and hand wash and 2) the sorption of active 

ingredients from granules contacting the skin. For example, hands and exposed skin are to be 

washed to assess exposure. It is well-established that wash recovery efficiency declines with 

delay time (e.g., Fenske & Lu, 1994; Fenske et al., 1996). Granular formulations are typically 

about 10% AI (i.e., 100,000 ppm); this may be greater than saturation for most or all 

compound/carrier combinations. Therefore, some fraction of the AI on the outer surfaces of the 

granules will be readily available for transfer to skin and may behave, upon contact with skin, as 

pure compound. Recovery of this material would likely be incomplete even with rapid washing, 

so results will not be conservative. In addition, delay until washing will be variable and will 

introduce more variability in recovery efficiency. Particles are going to be very large so they

will likely fall from the skin prior to washing, and some AI will be absorbed into the skin beyond 

the point of recovery; therefore, using either process as a basis would underestimate the 

exposure. Furthermore, the protocol provides for use of new gloves for each worker. This avoids 

exposure to residues in gloves from activities unrelated to the study but is non-conservative in 

that gloves can be contaminated under normal working conditions and workers are very unlikely

 

2 Around 1999, EPA’s Office of Pesticide Programs decided to use transfer coefficients derived from JazzerciseTM to 

represent one hour of extreme exercise or four hours of typical activities. For more information on this, see 

“Overview of Issues Related to the Standard Operating Procedures for Residential Exposure Assessment” presented 

at the FIFRA Scientific Advisory Panel meeting on September 21, 1999 (pp. 2, 41, and 45-50). For technical 

information, see Ross, J. et al. (1990). Measuring potential dermal transfer of surface pesticide residue generated 

from indoor fogger use. Chemosphere. 20:349-360. 
9

to use fresh gloves every day. The Board recommends that these issues be considered and 

addressed in interpretation of study results. 

3. Non-conservatism of the estimate of MOEs 

The uncertainties in absorbed dose relative to the measured exposure can also lead to 

nonconservatism of stated margins of exposure and consequently of the study’s safety. For 

example, four of the proposed surrogates (chlorpyrifos, imidacloprid, pendamethalin, and 

tefluthrin) have MOEs that reflect assumed dermal bioavailabilities that are less than 100%. 

Those assumed dermal bioavailabilities are probably taken from (non-specified) studies that are 

or may be very dissimilar to conditions relevant to granular pour operations (i.e., dry WBDs 

versus sweat soaked WBDs). Dermal bioavailability is not independent of loading conditions. 

Therefore, the assumed bioavailabilities cannot be presumed to be transferable. The Agency

should review and document the loading conditions used to develop the dermal absorption 

factors for these four surrogates for their relevance to the field conditions to be tested within this 

protocol to assure that the MOEs are indeed adequate.

4. Statistical and sample design issues 

The chosen sampling design that involves stratifying a large portion of the country into 

monitoring areas and selecting monitoring units within each area is an appropriate sampling 

design based on the nature of this study. The justification of the selection of monitoring areas is 

reasonable, given practical and cost constraints. 

The limited randomness in the selection of monitoring units that would likely be achieved by the 

recruitment process as described in the documents was discussed at some length. Although the 

Board felt the proposed approach was acceptable given practical constraints, it recommends that, 

if feasible, some form of classification system be developed that would identify the mechanism 

by which each MU was recruited and in particular would identify those, if any, that might need 

to be recruited by “traditional recruitment” in order to provide an after-the-fact (future) indicator 

of the randomness of the MU selection process. 

The objective of creating a database of exposures over a range of exposure scenarios may be 

achieved with the proposed sampling design for three types of agricultural pesticides, viz.: 

herbicides, insecticides, and fungicides. The design attempts to take into account 

climate/environment (7 MAs), usage (11 crops), pesticides (10), and amount of AI handled (3 

AaiH ranges). In addition, the documents indicate that the best case scenario would be to have 

each MU within an MA be classified in a different AI range. Having at least one MU in every

one of these combinations clearly is not possible with the chosen sample size. 

It has been stated that no data currently exist for this particular exposure issue (Collier, 2014, p. 

11; Evans and Sherman, 2014, p. 3). Each MU in the study is expected to perform a variety of 

tasks, but it is not clear if all or some or only one the 10 pesticides will be handled during the 

monitoring. Since the goal is to have each MU in an area have a different AaiH level, with three 

MUs and three AaiH levels, it appears that there are no replications of AaiH levels within an 

area. As a result, the variability among humans who would be exposed to a particular scenario 

cannot be addressed using only the information from this study. 
10

Implementation of this protocol will provide quantitative information on a variety of exposure 

situations related to open pour or granule formulations. While statistical analyses are not the goal 

of this study, the information will surely be used in some way by someone for that purpose 

eventually. Lack of replication of the human variability would make it difficult or likely

impossible for any user of the database to conduct further statistical analyses. 

Ethics 

Charge to the Board 

• Is the research likely to meet the applicable requirements of 40 CFR part 26, subparts K 

and L?

Board Response to the Charge 

HSRB Recommendation 

• If the research is performed as described, it is likely to meet the applicable requirements 

of 40 CFR part 26, subparts K and L. 

HSRB’s Detailed Recommendation and Rationale 

Overall, the Board agreed with the agency's assessment that this study is ethically acceptable. 

The Board's detailed reasoning is given below. 

1. Risks to subjects 

The risks to study participants are outlined in section 2.3 of the protocol (Collier, 2014, pp. 

6267), as follows: a) risk of heat-related illness, b) risk associated with scripting of field 

activities; c) psychological risks; d) risk of exposure to surfactants and e) risk of exposure to 

surrogate chemicals. Pregnant and nursing women will be excluded. 

a. Regarding heat-related illness, the protocol states that this risk may be increased due to the 

fact that the subjects must wear an extra layer of clothing (long underwear (or WBD) for 

exposure monitoring during the manual pouring procedure). Likely, the more significant 

factors for heat-related illness would be temperature, humidity and level of activity of the 

subject. In other words, the addition of a single layer of clothing may not be significant in 

the overall risk of heat-related illness. Therefore, the statement that the use of the extra 

clothing constitutes greater than minimal risk in this study might be an overstatement. 

To minimize risk of heat-related illness, the protocol appropriately outlines steps for 

monitoring temperature and if necessary, using cycles of work and rest, moving the worker 

to a cooler environment, or conducting the testing during early morning or night when 

temperatures are lower. Given that the subjects will be individuals who are familiar with the 

manual pouring procedures and will be in good health, the risks are reasonable. 

b. The risk of scripted activities is likely to be negligible, since it involves reducing the load 

size of the material to achieve three load/apply cycles per MU. 
11

c. Likewise, the psychological risks appear to be minimal; they relate to changing clothes in the 

presence of a researcher, and for women, taking a pregnancy test, which is done in a private 

room without disclosure of test results to anyone outside the research team. 

d. The protocol describes the risk from use of a surfactant (0.01% v/v Aerosol® OT in water). 

The surfactant is used as a very dilute solution and risks of skin irritation are likely to be 

negligible. An eye wash is available in case of accidental exposure to the eyes (Collier, 

2014, pp. 64-65). 

e. However, the handling of surrogate materials itself could, in theory, constitute greater than 

minimal risk. The standard of minimal risk relates to the probability and magnitude of harm 

experienced in daily life3and this is generally interpreted as the daily life of healthy 

individuals—which could reasonably be taken to mean individuals in the general (not 

occupational) population who are not exposed to pesticides on a daily basis. In other words, 

the fact that agricultural workers are exposed to pesticides in their daily work does not 

necessarily mean this exposure would a priori be considered minimal risk using a general 

healthy person standard. 

The risks of exposure to surrogate materials are described in section 2.3.5 (Collier, 2014, pp. 

65-67). Since the study involves one day of exposure, the risks are those associated with 

acute toxicity, rather than chronic exposure. Table 1 of the protocol shows the dermal MOE 

for each surrogate (Collier, 2014, p. 66). The calculated MOEs meet or exceed the required 

minimum MOE in each case. Therefore, the risks are considered acceptable according to 

EPA standards for exposure to organophosphate pesticides (Collier, 2014, pp. 65-66). 

Whether or not the overall study constitutes minimal risk, or greater than minimal risk, is 

difficult to determine, but the risks are reasonably low and are well managed as described in 

the protocol. 

Workers engaged in the pouring procedures for the study will be experienced pesticides 

handlers, will be using appropriate personal protective equipment (PPE) and will be 

reminded of safe handling practices. 

2. Selection of subjects 

The subjects will be offered participation in the study after their employer has agreed to allow 

recruitment of his/her workers (Collier, 2014, p. 32). Workers will be informed of the study and 

have an opportunity to learn about the study without their supervisors being present. Employers 

will sign consent documents stating that they will not influence or coerce their employees to 

 

3 Minimal risk means that the probability and magnitude of harm or discomfort anticipated in the research are not 

greater in and of themselves than those ordinarily encountered in daily life or during the performance of routine 

physical or psychological examinations or tests. The definition is in the Code of Federal Regulations at 40 CFR 

26.102(i); it is available at http://www.gpo.gov/fdsys/granule/CFR-1999-title40-vol1/CFR-1999-title40-vol1-

sec26102/content-detail.html
12

participate in the study. It is not clear whether employees might be prohibited or discouraged 

from joining the study if employers did not feel it was in their (the employer’s) interest to allow 

participation. Pregnant and nursing women will be excluded. Some workers in agricultural 

settings have low levels of literacy; these individuals will not be excluded from participation. 

3. Informed consent 

Informed consent will be conducted in individual sessions using prepared materials in English or 

in Spanish. For workers who cannot read or who are not comfortable reading consent materials, 

the materials will be read aloud by the study staff in the presence of a witness, per SOP AHETF 

11.I.3 (Collier, date, pp. 261-266). These informed consent arrangements are acceptable. 

4. Privacy and confidentiality 

The main privacy issues relate to possible photographs or videos during the study activities, 

changing of clothing in the presence of researchers, and pregnancy testing. These risks are 

appropriately managed as described in section 2.3.3 of the protocol (Collier, 2014, pp. 64). 

5. Monitoring of data 

The protocol describes monitoring for heat-related illness in Section 2.3.1 (Collier, 2014, p. 62). 

Workers will be observed and reminded to use appropriate PPE during handling of the product. 

Nearby medical facilities will be identified and transportation and costs for any needed medical 

care will be provided, per section 2.3.5 (Collier, 2014, pp. 65-67). 

6. Vulnerable or disadvantaged subjects 

Given that some agricultural workers are economically disadvantaged, or may have low literacy

levels, these individuals are potentially vulnerable. It is acceptable to enroll these subjects, given 

that the study involves low levels of risk, participants are appropriately compensated for their 

time, and consent procedures require adequate oral explanation of the study for those who cannot 

read. It is important that language issues are adequately addressed—for example, that bilingual 

or Spanish speaking witnesses are available if needed. For other workers who are not fluent in 

English, or who do not read English and are not Spanish-speakers, they would need to be 

excluded from the study due to the lack of translated forms and materials. 

In summary, the study has appropriate procedures and protections in place for human subjects. 

The level of risk is low and the procedures to minimize risk are reasonable. The social value of 

conducting the research is significant and the overall risk/benefit ratio is acceptable. 

A published report by Frampton et al. (2002) of intentional human exposures to nitrogen

dioxide and their effects on airway and blood cells4 5

 

4 Chair’s note: Dr. Gary Chadwick recused himself from Board decisions and votes related to this study. 

5

Following this meeting, the EPA decided to not rely on this study. 
13

Overview of the Study 

This standard, three-period cross-over inhalation toxicity study was conducted to assess the 

“effects of nitrogen dioxide (NO2) on airway inflammation, blood lymphocyte recruitment to the 

lung, and susceptibility of airway cells to infection with influenza virus and RSV [respiratory

syncytial virus]” (Frampton et al., 2002, p. L161). Twenty-one (12 male, 9 female) adult subjects 

participated in this study; they were recruited, from the population of students and community

members around the University of Rochester. Eligible participants were evaluated individually

in an environmental chamber during 1995-1996. The participants averaged 27 years old, were 

non-smokers, had no history of cardiac or respiratory diseases, and were free of respiratory

infections for at least six weeks. They were exposed three times (to air and to 0.6 ppm and 0.15 

ppm NO2) for three hours each, during which they exercised intermittently several times. The 

exposure levels typified those found in indoor environments with unvented combustion sources 

(Frampton et al., 2002, p. L155). The three exposure days were separated by at least three weeks. 

Airway exposures were measured using standard spirometry methods, fiberoptic bronchoscopy, 

and airway brush biopsies. Blood cell counts were performed on the bronchoscopy and brush 

biopsy samples. Flow cytometry was used to assess cell differential counts, blood lymphocytes 

and other metrics. Cell cultures were used to determine the viability of respiratory and epithelial 

cells in the presence of influenza and RSV strains. 

Analysis of variance (ANOVA) methods were used to evaluate the effects related to NO2

exposures. Period, carryover, treatment effect and gender differences were examined. Residuals 

were assessed to determine whether the assumption of normality had been violated. 

Science 

Charge to the Board 

• Is this study scientifically sound, providing reliable data? 

• If so, is this study adequate for quantitative use in support of an inhalation risk 

assessment for the use of nitrogen dioxide as a medical equipment sterilant? 

Board Response to the Charge 

HSRB Recommendations 

• The sample collection and laboratory analysis parts of this study are scientifically sound. 

However, the chamber exposure data are not reliable without more information regarding 

the actual exposure levels to nitrogen dioxide. 

• The statistical analysis is adequate to justify the significant differences that the authors 

identified, but is not adequate to detect all of the differences that may have occurred. This 

study may be used in a quantitative way as part of a weight-of-evidence analysis to 

support effects that might occur at the exposure levels reported (0.6 and 1.5 ppm), but 
14

this study (as published) is not relevant to support the existence of no effects at the 

claimed levels of exposure. 

HSRB Detailed Recommendations and Rationale

At its core, this paper describes some very detailed and scientifically sound biological sample 

collection and analytical methods. However, the Board noted issues that add to the Agency’s 

review (Leshin, 2014). Specifically, the HSRB has four concerns related to the operation of the 

exposure chamber that greatly weaken the Board’s confidence in the reliability of the study’s 

data and two concerns about statistical methods (two concerns) that weaken the study’s 

reliability and constrain its potential power. 

1. Exposure chamber 

a. In regard to the chamber, on p. L156 Frampton et al. (2002) state that “For comfort, 

temperature and relative humidity were maintained at 37.1 ± 3.0°C and 21.2 ± 0.92% 

(mean ± SD), respectively.” This temperature (corresponding to 98.8°F) is outside of the 

normal comfort range and comprises a potential heat stress setting, especially when 

combined with an exercise regime “sufficient to increase the minute ventilation to 40 

L/min” for 10 of each 30 minutes for three hours (Frampton et al., 2002, p. L156). No 

recognition of this risk is mentioned within the paper (or its associated consent 

information). 

b. The authors state “The capabilities for generating and maintaining pollutant levels and 

constant temperature and humidity have been described previously (48)” (Frampton et 

al., 2002, p. L156). This citation (ref. 48) is probably in error and should be ref. 49 (and 

likely reference 49 should be reference 48).6 Reference 49 is a three-page extended 

conference abstract that describes the designed operating capabilities of this chamber: 

"This system [(S), (DX), (R) and (H)] was designed to vary the chamber temperature 

from about 10°C to 31.5°C and relative humidities [sic] from about 25 to 85%” (Utell et 

al., 1984, p. 220); these limits are repeated on page 221 in Table 1. Thus, the reported 

research by Frampton et al. (2002) seems to have been conducted outside of both the 

temperature and humidity ranges for which the chamber was designed. No comment 

regarding exceeding these limits was found within the paper. 

c. Two of the Frampton et al. (2002) claims regarding NO2 on page L156 are without any

supporting methods, data, or references but are virtually word-for-word from the aerosols 

reported in reference 49 (Utell et al., 1984, p. 220). The first claim, “This [referring to 

~0.3 atmospheric changes/min] enabled NO2 levels to reach >90% of target levels within 

4 min” (Frampton et al., 2002, p. L156), is technically incorrect. Comments elsewhere in 

reference 49 read, “A volumetric control is incorporated into the chamber air supply to 

 

6 Their reference 48 is “Utell MJ, Frampton MW, Roberts NJ Jr, Finkelstein JN, Cox C, and Morrow PE. 

Mechanisms of nitrogen dioxide toxicity in humans. Health Effects Institute Research Report 43:1–44, 1991” 

(Frampton et al., 2002, p. L165). It appears that this reference to 48 should be to reference 49 that is “Utell MJ, 

Morrow PE, Hyde RW, and Schreck RM. Exposure chamber for studies of pollutant gases and aerosols in human 

subjects: design considerations. J Aerosol Sci 15: 219–221, 1984” (Frampton et al., 2002, p. L165). Reference 49 

was probably also mistakenly cited on p. L155 of the Frampton et al. (2002) paper in reference to toxicity 

mechanisms "Because of its oxidative potential and limited solubility, NO2 is a deep lung irritant, and accidental

exposures to high concentrations can cause acute lung injury and death (13, 49)."
15

stabilize the air flow at 10 m3/min” (the same flow rate as in Frampton et al., 2002) and 

“the effective chamber volume (less instrumentation furniture, etc.) is approximately 40 

m

3

” (both quotes are from Utell et al., 1984, p. 220). Thus, the ideal (best case) air 

exchange time is 40 m3divided by 10 m3/min or the four minutes referred to by the 

authors.78 However, 2.3 air changes (at least 9 minutes) are required in order for any 

injected air component (like NO2) to reach >90% of target levels (derived from: -ln(1 –

C/Ctarget) = -ln(1 – 0.9) = -ln(0.1) = 2.3). Less than ideal mixing would have extended the 

room's actual air exchange rate by 10% to 50% of its ideal value, and thus the time to 

reach 90% of its target level could have been 10 to 15 minutes. Because the authors do 

not describe the startup sequence of the exposure chamber, the Board could not determine 

how long in advance NO2 might have been injected into the chamber and the magnitude 

of the effect of this delay on the exposure conditions experienced by the test subjects. 

However, even a 15-minute transient is a small fraction of subjects’ three-hour exposure 

periods. 

d. Another claim is potentially of much greater concern. In their Methods section, the 

authors state that "The concentrations of NO2 at the 3- and 6-ft levels within the chamber 

varied by no more than 5% of the mean" (Frampton et al., p. L156). Furthermore, in the 

Results, the authors claim that “Actual achieved NO2 concentrations were 0.61 ± 0.02 

and 1.50 ± 0.02 (SD) ppm” (Frampton et al., 2002, p. L157). However, no method of 

measuring the NO2 was mentioned; measuring NO2 concentrations is not a trivial matter, 

and its method is worthy of a description. While these oversights might just reflect the 

authors’ focus on biologic responses and these results may be a repeatable measure of 

chamber performance, it is also possible that this claim of uniformity was simply copied 

from reference 49 that states "Sampling of test aerosol concentrations at 3' and 6' levels 

(3 x 3 matrix) within the chamber has revealed that the aerosol concentrations vary by no 

more than ± 5% of the mean" (Utell et al., 1984, p. 220). In contrast to those test aerosol 

concentrations, Figure 1 of reference 49 (Utell et al., 1984, p. 220) indicates only one 

location for environmental monitoring ["EM"]. In fact, the authors state later that 

"Chamber monitoring is based on environmental factors, e.g. temperature, air flow, 

relative humidity, and the pollutant per se" (Utell et al., 1984, p. 220). This statement 

referenced by Frampton et al. to that citation “for generating and maintaining pollutant 

levels,” and the lack of a method to measure NO2 imply that the authors of Frampton et 

al. may have based their exposure values on the rate at which the 5000 ppm NO2

compressed gas was emitted into the venturi mixer and diluted into the 10 m3/min air 

supplied to the exposure chamber; i.e., the reported concentrations of NO2 may be merely 

calculated but unverified assumptions. Neither the length nor the material of the ducting 

that carried the air from the control and mixing equipment located in an adjacent 

engineering space to the exposure chamber was described within either paper. Since NO2

is fairly reactive, it is at least possible - if not likely - that some of the supplied NO2 was 

lost en route to (or within) the exposure chamber. If the test concentrations were 

 

7

 Less than ideal mixing will extend this time to more than four minutes. On p. 221 of reference 49, the authors 

mention "a wall-mounted air circulation unit which is normally kept in operation to assist in chamber mixing" (Utell 

et al., 1984). Without more specifics, such a unit would probably keep the room's actual air exchange rate within 

8 % to 50% of its ideal value.
16

calculated rather than measured, they were probably not as high as expected and reported. 

This uncertainty makes the exposure side of this study seem unreliable. 

Although there is one author (Utell) common to both Frampton et al. (2002) and 

reference 49 (Utell et al., 1984), the four concerns noted above suggest that the authors of 

the former paper may have had a limited understanding of the exposure chamber itself. 

2. Statistical issues 

The statistical design and analysis procedures are clearly stated on page L157 of Frampton et al.

(2002). The study’s three-period cross-over design with washout periods was commonly used in 

this type of situation. Based on a document containing background material, an undescribed 

prestudy power analysis apparently indicated that four subjects for each of the six treatment 

sequences would provide adequate power (Sherman, 2014, p. 45). Additional subjects were 

recruited to allow for dropouts but data from only 21 subjects are reported in the paper; this 

number of actual participants does have an effect on the study’s power. The analyses of variance 

included period and cross-over effects. Checks on model assumptions were carried out. These 

analysis procedures are standard and strengthen the acceptability of the numerical results. 

a. The first statistical concern relates to the treatment of outliers. Frampton et al. (2002) 

state that “Data means shown in RESULTS include all study subjects, even though 

statistical outliers were excluded for the ANOVA” (p. L157). Deletion of outliers is an 

acceptable practice as long as it is reported as part of the methods. However, no 

indication was given as to the criteria by which an observation was identified as an 

outlier. More importantly, there was neither mention of which variables were affected 

nor the severity of the problem. As a consequence, in the Results, parts or all of Figures 

1 through 6 do not accurately reflect the numerical results of the statistical analysis (pp. 

L159-L161). Moreover, the extent of the inaccuracies cannot be determined; Figure 4A 

(p. L160) is a good example of this issue. The authors indicate that the only significant 

effect is the NO2 main effect. However, the figure shows a very large difference between 

male and female responses at 1.5 ppm which would be indicative of a potential gender × 

NO2 level interaction. At best, this leads to potential confusion as to which effects may

actually be significant. 

b. The level of the second statistical concern depends upon the intended use of the Frampton 

et al. (2002) data. The ANOVA tests the authors used were adequate to support their 

conclusions that pulmonary responses were observed. However, in some cases the paper 

could have used more powerful statistical tests; these might have detected other 

significant responses that were not recognized. A good example is the respiratory

ventilation data that comprise the first two data columns of Table 2 (at rest and while 

exercising) (p. L158). In both columns, the mean respiratory rates progress with 

increasing dose of NO2, but the only test mentioned was Student’s t-test for total NO2

intake. The differences in ventilation at the two doses subjectively look rather large (i.e., 

the increase in rates for all but the females while exercising are in the range of +4 to 

+9%); these could have been significant if the results had been analyzed using paired 

ttests. Another statistical option might have been some form of regression analysis, but 

the authors only provided data for 0.6 to 1.5 ppm exposures, instead for the three sets of 
17

data (including 0 ppm) as was done for all other comparisons. The existence of such 

increases in ventilation rate would be consistent with the effects of such a respiratory 

irritant and with the decreases in hematocrit that were observed and presented in Figure 1 

(p. L159) for all three test levels (0, 0.6, and 1.5 ppm). 

In summary, the statistical methods used in the study do not detract from the significant 

responses that the authors identified nor from their conclusions, but the limitations to these tests 

imply that the authors may not have identified all of the statistically significant responses to these 

exposures that may be of interest to the Agency. 

Ethics 

Charge to the Board 

• Does the study meet the applicable requirements of 40 CFR part 26 subpart Q? 

Board Response to the Charge 

HSRB Recommendation

• The Board concluded that the published report by Frampton et al. (2002) submitted for 

review meets the applicable requirements of 40 CFR part 26 subpart Q, and that the data 

within this article may be considered acceptable for EPA’s reliance, contingent upon the 

determination of their scientific validity. 

HSRB’s Detailed Recommendation and Rationale

The Board agreed overall with the Agency’s assessment of the ethics of the study (Sherman, 

2014). The Frampton et al. (2002) article, published in the American Journal of Physiology of 

Lung Cellular and Molecular Physiology, is the report of a medical study intended to test the 

effects of nitrogen dioxide (NO2) on airway inflammation, blood cells, and resistance to 

respiratory viral infection in 21 normal, non-smoking adult male and female volunteers. The 

investigation was conducted at the University of Rochester, School of Medicine and was funded 

by the National Institutes of Health and the Center for Indoor Air Research. It was designed as a 

study of intentional exposure to NO2 as an environmental pollutant, but EPA identified the work 

as a potential source of data for evaluating NO2 as a sterilant of medical equipment. 

In the Methods and Study Design section of the article, the investigators report that the study was 

approved by the University of Rochester’s Research Subjects Review Board, the university’s 

institutional review board (IRB) for protection of human subjects in research (Frampton et al.,

2002, pp. L156). At that time the IRB would have been operating under regulatory standards of 

the 1996 version of 45 CFR 469and the ethical guidance of the Belmont Report9. 

 

9

Protection of Human Subjects in the U.S. Code of Federal Regulations, 45 CFR part 46. For a timeline of 

these federal regulations, go to http://history.nih.gov/about/timelines_laws_human.html 

9 Available at 

http://www.hhs.gov/ohrp/humansubjects/guidance/belmont.html
18

The article states that investigators enrolled 21 subjects (9 females, 12 males) ages 18-40 and that 

“Informed consent was obtained” (Frampton et al., 2002, p. L156). When interviewed by 

Agency staff in August 2014 and later contacted by email, the article’s first author, Dr. Mark 

Frampton, reported that none of the participants was pregnant or nursing, and that subjects were 

tested for pregnancy before each exposure (Sherman, 2014, p.30). 

In response to a request from EPA staff, the University of Rochester’s IRB provided electronic 

copies of available records on this study, which included the study’s protocol and consent 

document, and the IRB’s approval letter. It may be assumed that the IRB approved the protocol 

following the above regulatory requirements in place at the time the study was reviewed. 

 

The Board concurred with the conclusions of the OPP’s Ethics Review (Sherman, 2014) that the 

reported research does not rely on data from intentional exposure of any human subject who was 

a pregnant or nursing woman or a child. There is no evidence that the conduct of the research 

was fundamentally unethical. There is no evidence that 1) the research was conducted in a way

that placed participants at increased risk of harm (based on the knowledge available at the time 

the study was conducted) or 2) vulnerable populations were targeted. However, because 

recruitment processes focused on the University community where the study was conducted, and 

the protocol does not address additional protections for students or employees, there is a 

possibility that some participants’ ability to give informed consent may have been impaired by

coercion. 

1. Assessment of risks and benefits 

The article reports that many Americans are exposed to levels of NO2 as an indoor air pollutant 

resulting from combustion, such as when natural gas-fueled ice resurfacing machines are used in 

enclosed ice rinks (Frampton et al., 202, p. L155). The primary benefit of the study was to 

society, by generating knowledge of NO2’s health effects. The highest concentration studied was 

a level stated in the consent form to be a level observed in homes where gas stoves are used. 

Risks of exposure to NO2 are not clearly spelled out in the consent document, although the 

shortterm risks of the bronchoscopy are identified. Neither the article nor the consent form 

addresses any potential benefit to participants or the balance of risks and benefits. The 

University of Rochester IRB’s approval of this study can be interpreted as its assessment that the 

risks to participants did not outweigh the study’s anticipated benefits under ethical and regulatory 

standards in place at the time of its review. 

2. Equitable selection of study participants 

Although vulnerable populations were not targeted, it is not possible to determine whether 

selection of study participants was equitable or whether individual members of vulnerable 

populations were enrolled in the study. According to the protocol and consent document, 

participants were compensated a total of $550 for taking part in all three phases of the study, 

with progressively increased exposure to NO2, a blood draw, and bronchoscopy at each stage. 

Participants received $50 for completion of the first phase, an additional $50 for completion of 

the second phase, and additional $450 for completion of the third phase. Although the 
19

compensation schedule was structured to promote participants’ completion of all three phases, 

the amount and schedule appear do not appear to have been disproportionate or coercive. 

3. Voluntary and informed consent of all participants 

Participants were recruited from among members of the University of Rochester community in 

Rochester, New York, including students and employees of the University. Although vulnerable 

populations were not targeted for the study, it is not possible to tell whether participants from 

potentially vulnerable populations were recruited. Similarly, there is no indication that the study

protocol included mechanisms designed to minimize coercive recruitment and enrollment of 

students and employees who may have been in a subordinate position to the researchers. 

To participate in the study, subjects were required to be healthy, non-pregnant adults with no 

history of smoking. No children were enrolled. After an initial conversation with the study

coordinator by phone, potential participants met with one of the investigators to discuss the 

study. Potential participants received a copy of the consent document to take home for further 

review before deciding whether to enroll. In his communication with EPA staff, Dr. Frampton 

reported that participants signed consent documents, and documents from the IRB call for 

signatures from the participant, an investigator, and a witness. Participants were informed in the 

consent document that they were able to withdraw from the study at any time. 
20

References 

U.S. Department of Health, Education and Welfare (DHEW) (1979). The Belmont Report: 

Ethical Principles and Guidelines for the Protection of Human Subjects of Research.

Washington, DC: U.S. DHEW, The National Commission for the Protection of Human 

Subjects of Biomedical and Behavioral Research. Retrieved from 

http://www.hhs.gov/ohrp/humansubjects/guidance/belmont.html 

U.S. Department of Health and Human Services (DHHS) (1996). 45 CFR part 46. Code of 

Federal Regulations. Washington, DC: U.S. Department of Health and Human Services 

(DHHS). Retrieved December 8, 2014, from 

http://www.hhs.gov/ohrp/humansubjects/guidance/45cfr46.html (2009 version) 

Collier, R. H. (2014, July 25). Determination of Dermal and Inhalation Exposures to Workers 

During Open Pour Loading of Granules, AHETF Study No. AHE170. Agricultural 

Handler Exposure Task Force, LLC. 

Evans, J., & Sherman, K. (2014). Science and Ethics Review of AHETF Scenario Design and 

Protocol (AHE170) for Exposure Monitoring of Workers during Open Pour Loading of 

Granules. Washington, DC: U.S. Environmental Protection Agency, Office of Pesticide 

Programs. 

Fenske, R., & Lu, C. (1994). Determination of handwash removal efficiency: incomplete 

removal of the pesticide chlorpyrifos from skin by standard handwash techniques. 

American Industrial Hygiene Association Journal, 55(5), 425-432. 

Fenske, R., Schulter, C., Lu, C., & Allen, E. (1998). Incomplete removal of the pesticide captan 

from skin by standard handwash exposure assessment procedures. Bulletin of 

Environmental Contamination and Toxicolocy, 61(2), 194-201. 

Frampton, M. W., Boscia, J., Roberts, M. J., et al. (2002, January). Nitrogen dioxide exposure: 

effects on airway and blood cells. American Journal of Physiology of Lung Cellular and 

Molecular Physiology, 282, L155-L165. 

Leshin, J. (2014). Nitrogen Dioxide: Evaluation of inhalation study, Frampton et al 2002.

Washington, DC: U.S. Environmental Protection Agency, Office of Chemical Safety and 

Pollution Prevention. 

Sherman, K. (2014). Ethics Review of Human Toxicity Study with Nitrogen Dioxide.

Washington, DC: U.S. Environmental Protection Agency, Office of Pesticide Programs. 

U.S. Environmental Protection Agency (EPA). (2010, July 1). 40 CFR part 26. Code of Federal 

Regulations. Washington, DC: U.S. Government Printing Office. Retrieved December 9, 
21

2014, from http://www.gpo.gov/fdsys/pkg/CFR-2010-title40-vol1/xml/CFR-2010-

title40vol1-part26.xml 

U.S. Environmental Protection Agency (EPA). (2014, October 17). Human Studies Review 

Board; Notification of a Public Meeting. 79 FR [Federal Register], pp. 62437-62439. 

Retrieved November 2, 2014, from 

https://www.federalregister.gov/articles/2014/10/17/2014-24757/human-studiesreviewboard-notification-of-a-public-meeting 

Utell, M., Frampton, M., Jr, R. N., JN, F., Cox, C., & Morrow, P. (1991). Mechanisms of 

Nitrogen Dioxide Toxicity in Humans. Boston, MA: Health Effects Institute. 

Utell, M., Morrow, P., Hyde, R., & Schreck, R. (1984). Exposure chamber for studies of 

pollutant gases and aerosols in human subjects: design considerations. Journal of Aerosol 

Science, 15, 219-221.