1

December 3, 2015

EPA-HSRB-15-03

Thomas A. Burke, Ph.D., MPH

EPA Science Advisor 

Office of the Science Advisor 

1200 Pennsylvania Avenue, NW 

Washington, DC 20460 

Subject: October 19-20, 2015 EPA Human Studies Review Board Meeting Report 

Dear Dr. Burke,

The United States Environmental Protection Agency (EPA or Agency) requested that the 

Human Studies Review Board (HSRB) provide scientific and ethics reviews of two items:

 A Completed Study from the U.S. Department of Agriculture Describing 

Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for 

the United States Military

 Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to Support 

Use of the EPA Repellency Awareness Graphic

The Board’s key responses to the charge questions are detailed in the enclosed final 

report of the meeting.

Signed,

Liza Dawson, PhD

Chair

EPA Human Studies Review Board
2

INTRODUCTION 

On October 19-20, 2015, the United States Environmental Protection Agency’s (EPA or Agency) 

Human Studies Review Board (HSRB or Board) met to address the scientific and ethical charge 

questions related to two items:

 A Completed Study from the U.S. Department of Agriculture Describing 

Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for 

the United States Military

 Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to Support 

Use of the EPA Repellency Awareness Graphic

REVIEW PROCESS 

The Board conducted a public meeting on October 19-20, 2015. Advance notice of the meeting 

was published in the Federal Register as “Human Studies Review Board; Notification of a 

Public Meeting” (EPA-HQ-ORD-2015-0588). 

Following welcoming remarks from Agency officials, the Board began its review of the first 

item.

The Board heard presentations from EPA for each agenda items in sequence, consisting of the 

Agency’s review of scientific and ethical aspects of the two studies. This Final Report of the 

meeting describes the HSRB’s discussion, recommendations, rationale and consensus in 

response to each charge question for each of these items. 

For each agenda item, Agency staff first presented their review of the science and the Board

asked the Agency presenters clarifying questions. The staff then described their review of the 

ethical aspects and the Board asked clarifying questions about those. The HSRB solicited public 

comments and next asked Agency staff to read the Charge Questions for the publication under 

consideration. The Board discussed the science questions first and then the ethics question. The 

Chair then called for a vote to confirm concurrence on a summary statement in response to each 

charge question.

For their evaluation and discussion, the Board considered materials presented at the meeting, oral 

comments, related materials and documents provided by the study sponsors, the Agency’s 

science and ethics reviews of the studies, oral responses from a study sponsor and protocol team, 

and public comments made at the meeting. A comprehensive list of background documents is 

available online at http://www.epa.gov/hsrb/. 

CHARGE TO THE BOARD AND BOARD RESPONSE

HSRB review of a Completed Study from the U.S. Department of Agriculture Describing 

Laboratory Evaluation of Bite Protection from Repellent-Impregnated Clothing for the 

United States Military
3

Charge to the Board:

Is the research reported in the completed study sufficiently sound, from a scientific perspective, 

to be used to evaluate the bite protection level of etofenprox-treated military clothing?

Board Response: The Board concluded that the research reported in the USDA report of the 

completed study is sufficiently sound from a scientific perspective to be used for evaluation of 

etofenprox-treated military clothing

HSRB Detailed Recommendations and Rationale:

The protocol for the study Laboratory Evaluation of Bite Protection from RepellentImpregnated Clothing for the United States Military was reviewed by HSRB at the April 

2014 meeting. The study was completed in July 2015 and the final report of the completed study 

has been reviewed and approved by EPA. The original HSRB review of the protocol (April 

2014) concluded that the protocol would “likely generate scientifically reliable data for 

estimation of bite protection efficacy of impregnated clothing, if improvements in the overall 

study design enumerated by EPA (Sweeney & Sherman, 2014) and the Board are adequately 

considered.” The protocol was revised according to EPA and HSRB recommendations prior to 

the study and revisions were approved by IRB. The study was completed as planned following 

the revised protocol. 

Protocol deviations and amendments

There were documented deviations from the protocol during the completion of the study but 

none of them, in the view of the Board, had a significant adverse effect on the scientific validity 

of the study. Several of the deviations were minor changes in informed consent procedures. 

Two amendments to the protocol related to low rates of mosquito bite through. It was postulated 

by the study team that low bite through in one case was related to improper laboratory 

procedures in maintaining the mosquitoes; the batch of mosquitoes used did not exhibit sufficient 

biting pressure on the control (untreated) fabric. For this subject, repeat testing was undertaken 

with a new batch of mosquitoes. For a second subject, there was low bite through for all tests, 

and the data from that subject were not included in the study analysis; the protocol was amended, 

with IRB approval, to recruit one additional subject for testing to replace this subject.

A further amendment allowed for the inclusion of an additional set of test sleeves with 75 washes 

(increased from the protocol which originally specified a maximum of 50 washes). The protocol 

team had anticipated that 50 washes would exhaust the repellency of the treated fabric, but in 

fact this was not the case, leading to the need to test a higher number of washes.

Another protocol deviation was the concentration of etofenprox on the treated fabric. The 

protocol specified that the ratio of product to fabric would be 1.0% (weight/weight), but the 

actual ratio was 0.9%. This was deemed scientifically acceptable, since the change in amount of 

product did not affect the reliability of the study results.

One additional deviation was related to the statistical data analysis method (discussed in 

following section). 
4

Statistical analysis

There was extensive discussion of the statistical analysis of the data from this study. The 

protocol had left unspecified what statistical analysis would be used; at the previous HSRB 

review of the protocol, the Board had recommended that a generalized linear mixed model 

(GLiM) procedure be used to estimate confidence intervals (CIs). In the GLiM procedure, the 

subject to subject variation can be treated as a random effect or fixed effect. Dr. Gbur and Dr. 

Fernandez recommended using the GLiM approach with the subject level treated as a random 

effect, which is more appropriate when the aim is to generalize the results to a broader 

population. The study team chose to use the t-distribution for estimating CIs, which is a simpler 

method and does not support generalizations beyond the study population itself. Dr. Fernandez 

ran a simulation comparing the GLiM and t-test procedures using parameters based on estimates 

from the data in the study report, and found that the CIs are wider using the GLiM approach 

compared to the t-distribution approach, but the overall conclusions regarding repellency are not 

significantly different. A copy of Dr. Fernandez’ simulation results are provided in an appendix 

to this report.

There was some discussion of the fact that differing version of statistical software can produce 

different results, and that the most recent versions of software will be most effective in 

conducting these analyses. Board members recommended that the GLiM approach be used in 

this study and in future studies of this type, but the analysis used, although not ideal, was 

sufficient to allow the data to be used for decision-making in this case. 

Statisticians on the board also pointed out that the GLiM approach can be used to support further 

objectives other than simply analyzing the mean and CI for each test product. For example, the 

longevity of the product (based on number of washes) could be estimated, or other secondary 

objectives. 

Board recommendations for methodologic considerations in future studies

A number of issues arose in this study that the Board recommended be addressed in future 

protocols. First, specific criteria for adequate biting pressure for a specific assay should be 

outlined in the protocol, such that any judgment about inadequate biting pressure is made on 

objective pre-specified criteria. Along the same lines, specific conditions under which subjects 

may be replaced, or data not included in final analysis, should be outlined in advance to avoid 

subjective decisions or the possible introduction of bias into study procedures and data 

collection. 

In addition, the use of GLiM statistical tests for this type of study is recommended by all the 

statisticians on the Board. Use of t-tests for a study protocol of this type of complexity is not 

considered adequate for statistical analysis. Because in this particular case, Board members were 

able to ascertain that the study conclusions would not differ significantly with a different analysis 

(see Appendix I), it is deemed acceptable to use the data as presented in the study report. 

However, as a general matter, a more robust and reliable analysis that more closely reflects the 

manner in which the study was conducted should be used and the statistical plan should be 

detailed in advance in the protocol document.
5

Ethics

Charge to the Board:

Does available information support a determination that the Completed Study from the U.S. 

Department of Agriculture Describing Laboratory Evaluation of Bite Protection from 

Repellent-Impregnated Clothing for the United States Military was conducted in substantial 

compliance with subparts K and L of 40 CFR Part 26? 

Board Response

The information provided supports a determination that the studies were conducted in substantial 

compliance with subparts K and L of 40 CFR Part 26. 

HSRB Detailed Recommendations and Rationale:

The purpose of this study was to determine the bite protection level of the etofenprox-treated 

U.S. military Fire Resistant Army Combat Uniforms (FRACUs). The protocol for this study and 

subsequent amendments were approved by the overseeing institutional review board, the 

Western Institutional Review Board (WIRB). Because this study was initiated after April 7, 

2006, prior submission of the protocol and supporting materials to EPA was required by 40 CFR 

§26.1125. The HSRB discussed the protocol at its April 9, 2014 meeting and determined that 

with revisions as suggested by the EPA and the HSRB, the study would meet the applicable 

requirements of 40 CFR Part 26, Subparts K and L. Adult subjects were recruited from the 

general population in Gainesville, Florida. Children (under the age of 18) and pregnant or 

lactating women were excluded. Consent was obtained before participation. Subjects received 

$250 for completion of testing with the full set of 10 pairs of sleeves. Several minor deviations 

from the protocol were reported, but did not negatively affect participants’ rights or their health 

or safety.

CHARGE TO THE BOARD AND BOARD RESPONSE

HSRB review of the protocol for Testing of S.C. Johnson & Son, Inc. Personal Tick 

Repellent Products to Support Use of the EPA Repellency Awareness Graphic

Charge to the Board 

Is the protocol Testing of S.C. Johnson & Son, Inc. Personal Tick Repellent Products to 

Support Use of the EPA Repellency Awareness Graphic likely to generate scientifically 

reliable data, useful for estimating the complete protection time of various EPA-registered S.C. 

Johnson skin-applied repellents in laboratory studies using three species of tick populations? 

Board Response:

The Board concluded that the protocol, if modified according to Agency and HSRB 

recommendations, is likely to generate scientifically reliable data, useful for estimating the 
6

complete protection time as defined in the protocol, of various EPA-registered S.C. Johnson 

skin-applied repellents.

HSRB Detailed Recommendations and Rationale

The objective of the S.C.J. protocol is clearly stated and follows EPA’s Product Performance 

Test Guidelines OPPTS 810.3700: Insect Repellents to be applied to Human Skin. The field 

tests are designed to provide data to support the EPA’s Repellency Awareness Graphic. 

The objective of this study is to establish the complete protection time (CPT) of up to 18 EPAregistered repellent products (test substances) on human subjects in a laboratory setting against 

three species of ticks (Pathogen free Amblyomma americanum, Dermacentor variabilis, and

Ixodes scapularis). The First Confirmed Crossing (FCC), defined as a tick crossing into the 

treated area confirmed by another crossing within two 15-minute test periods (or within 30 

minutes) will determine repellent failure for each tick species. The CPT for ticks to be used on 

the insect repellency graphic will be from the tick species with the lowest CPT value.

The product dose will be standardized for product comparisons, so the study has no dosimetry 

phase to determine the typical consumer dose of these products when applied by human 

subjects participating in the study. Twelve subjects (ten test subjects and two alternates) will be 

selected from a pool of subjects to be recruited in the Baltimore Metro area. The number of test 

subjects for a tick repellency efficacy study should strike a balance among three critical and 

competing criteria: a) Minimization of potential hazard to test subjects, where fewer subjects is 

better; b) Statistical robustness, where more subjects results in greater precision of numeric 

estimates; c) Consistency with previous repellent efficacy studies. Current and recent practice is 

to utilize ten subjects for each test site/test product.

Several scientific aspects of the protocol require clarification and possible revision. The most 

significant scientific issue was the question of positive and negative control groups. For each 

subject enrolled, one arm is treated with repellent and the other is not. The untreated arm serves 

as a negative control; both arms are exposed to ticks. Thus the only question about this negative 

control is whether the test conditions (placing of the tick on the arm and measurement of 

subsequent behavior) are identical and unbiased under both conditions, treated and untreated. 

HSRB members expressed some concern about whether there was an objective measurement of 

tick viability and whether the removal of the tick from one arm and placement on the other arm 

might cause any change to the tick’s behavior. In discussion at the meeting, EPA staff and 

members of the protocol team described the sequential placement of the tick on the untreated and 

treated arm as a standard procedure for these repellency studies. 

A second issue discussed was whether a positive control was needed, namely, a subject tested 

using identical test procedures, treated with a known positive control repellent. Agency 

representatives have described that this is not standard practice for these repellency tests and that 

there is no particular product typically used a positive control; staff also expressed that the 

additional cost and burden of the positive control is not warranted. At the same time, it was 

acknowledged that a positive control allows for data to be compared across sites and different 

test days. Members of the HSRB pointed out that the scientific rationale for lack of positive 

control, and the adequacy of the untreated arm as a negative control, as well as other aspects of 

study design, should be supported by peer-reviewed published literature in the field, and 
7

justification for the study design should be included in the study protocol. For example, a 

number of different study designs are described in published literature, and each appears to give 

somewhat different results.1

There was extensive discussion of the pregnancy test that is required as part of the study 

protocol, since pregnancy is an exclusion criterion in the study. The study team has included 

provisions for women to take the pregnancy test up to 48 hours before the first test day, so that 

the procedures on the test day, which are already very lengthy, can be somewhat shorter in order 

to reduce burden on the participants. While the majority of the board felt this was acceptable, a 

board member questioned whether the pregnancy testing should be carried out the same day to 

eliminate the possibility of an early pregnancy (less than 48 hours) that is not detected. In an 

effort to find a compromise solution, it was proposed that women be given an option to take the 

test 48 hours in advance, or on the test day, and if they chose the early test, that they be 

counseled about avoiding sexual activity that could lead to pregnancy. However it should be 

noted that the majority of the board expressed that the 48 time window was acceptable, 

particularly given that the test products are licensed and marketed to the public with no 

regulatory or safety concerns regarding use in pregnancy, and that the likelihood of an 

unintended pregnancy in a very short time period is low. 

HSRB members commented that more information should be provided on training of medical 

personnel for the study, who would be available in case of any adverse reactions or medical 

needs of the study participants related to exposure to the repellent products. There was also 

concern that three 15-minute breaks in an 18-hour day may not be sufficient.

There was a lengthy discussion about whether the study population is representative of the 

population that uses insect repellency products. The study team justified their choice of study 

population based on Nilson survey data about use of personal repellent products, and the team’s 

intention is to include approximately 10% Hispanic study participants to match the demographics 

of the US population using these products. However the team is not planning to use Spanish 

language consent materials or have an interpreter available for those individuals who do not 

speak English. Therefore, the effort to include Hispanic participants will consist in enrolling 

individuals who are bilingual, Spanish and English speaking. Members of the HSRB 

commented that a more direct approach would be simply to recruit Hispanic individuals who 

self-identify as such, rather than using language as an indirect criterion. There was discussion 

about whether lack of Spanish language materials was ethically problematic. Diverse views 

were voiced on this point. Some members of the board maintained that Spanish language 

translation ought to be required, while others stated that in a study offering no direct benefits to 

participants, concerns about unfair exclusion of non-English speakers are significantly reduced. 

The Board also recognized that in a small study of this type there will almost no possibility of 

determining any statistically significant differences in population subgroups. In the end the 

Board did not recommend a specific change to study procedures on this point.

 

1 Dautle H, Dippel C., Wekhausen A, and Diller R. Efficacy testing of several Ixodes ricinus tick repellents: Different 

results with different assays. Ticks and Tick borne Diseases 4(2013) 256-263; Krober T, Bourquin M and Guerin 

PM. A standardised in vivo and in vitro test method for evaluating tick repellents. Pesticide Biochemistry and 

Physiology 107 (2013) 160-168.
8

While the protocol was also well developed from a statistical point of view, the Board 

recommended that a random number generator approach be used instead of the coin toss 

approach. In addition, clarification was requested regarding the methodology used for calculation 

of sample size determinations. It was suggested that additional detail be provided about the table 

and its accompanying text and data presentation. It was also recommended that the statistical 

analysis plan be more detailed and that specific plans for analyzing treatment versus control 

conditions, examining gender differences, and assessing potential confounding effects when a 

subject is tested using more than one product.

Ethics

Charge to the Board

Is the research described in the protocol “Testing of S.C. Johnson Personal Tick Repellent 

Products to Support Their Use of the EPA Repellency Awareness Graphic” likely to meet the 

applicable requirements of 40 CFR part 26, subparts K and L?

HSRB Recommendation

The study, when modified as recommended by the HSRB in this report, is likely to meet the 

applicable requirements of 40 CFR part 26, subparts K and L.

HSRB Detailed Recommendations and Rationale

This is a protocol for third-party research involving intentional exposure of human subjects to a 

pesticide. As noted above, the study involves the use of consumer products for tick repellency 

for defined periods of time with exposure to three tick species for the purpose of determining the 

duration of protection of up to eighteen EPA-registered repellent products from S.C. Johnson & 

Son, Inc. (‘Johnson’). The data is intended to be used to support these products’ use of the EPA 

repellency awareness graphic for labels in order to better inform consumers about the relative 

protection times of these products.

The following aspects of the research plan were deemed acceptable by the HSRB:

 40 CFR part 26, subpart K requires that research involving human subjects must be 

reviewed and approved by an institutional review board (IRB). This study has been 

reviewed and conditionally approved by the Schulman Associates IRB (SAIRB), with 

SAIRB’s final approval contingent upon Johnson obtaining HSRB review and approval.

 40 CFR part 26, subpart L requires that EPA must not rely on data from any research 

involving intentional exposure of any human subject who is a pregnant woman (and 

therefore her fetus), a nursing woman, or a child. The protocol, informed consent 

document, and recruitment script indicate that participants must be 18-55 years of age 

and that female subjects who are pregnant or breastfeeding will be excluded from study 

participation. The HSRB discussed changes to the pregnancy testing process for this 

study, as noted above.
9

 Risks to subjects are appropriately minimized through the following mechanisms: 

inclusion and exclusion criteria for potential participants that minimize physical risks 

related to exposure to ticks, the test products, or latex;2training provided to potential 

participants prior to testing day; access to first-aid treatment in the event of a researchrelated injury; a 24-hour telephone number for participants to contact the Study Director 

in the event of a research-related reaction after participants have left the testing facility; a 

required break of at least two calendar days between testing days, for participants who 

choose to participate in more than one testing day; and the use of supporting materials on 

which participants can rest their arms during testing procedures.

 The informed consent process includes the use of open-ended prompts to assess potential 

participants’ understanding of the study’s purpose and what study participation entails. 

Although the study materials provided to the HSRB did not include these open-ended 

prompts, the EPA Ethics Reviewer presented these prompts at the October 20, 2015 

meeting. 

Before the research begins, study materials should be revised to address the comments from EPA 

and the following concerns of the HSRB:

 To minimize the risk of exposure to tick-borne disease, the study protocol indicates that 

the study will use only laboratory-reared ticks that are not known to harbor any 

pathogens. Confirmation from the supplier will be obtained. The HSRB recommends that 

the name of the tick supplier be included in the study protocol.

 Since meals will be provided to participants on testing day, HSRB recommends that 

during training sessions, study personnel should ask potential participants about any 

dietary accommodations that must be met (for example, food allergies or religious dietary 

law) so that all participants are able to eat during the lengthy testing day.

 HSRB recommends that study personnel should either provide entertainment (such as a 

small television and DVD player) or encourage participants to bring their own on testing 

day in order to minimize participant anxiety and potential boredom during testing 

procedures.

 In accordance with 40 CFR part 26, subpart K, the HSRB reminds Johnson that study 

materials that have been revised in light of EPA and HSRB recommendations, as well as 

related correspondence with SAIRB, must be submitted to EPA. Further, Spanish 

language recruitment materials should be certified by a qualified translator or translation 

service, and both Spanish language materials and the accompanying certification should 

be submitted to EPA.

 

2 Some HSRB members question the appropriateness of excluding non-English speakers from study participation 

with no clear scientific rationale for this exclusion, as well as the use of Spanish-language recruitment materials to 

recruit bilingual subjects. However, no formal recommendations arose from this discussion. 
10

Finally as noted above in the section on scientific review, the issues of pregnancy testing was 

discussed. The board’s deliberations resulted in the following options:

 Female participants could undergo pregnancy testing at the beginning of each testing day 

or at the training session prior to testing day, provided that this training session takes 

place within 48 hours of the testing period. If female participants choose to undergo 

pregnancy testing at this training session, counseling could be provided about the need to 

avoid sexual activity leading to pregnancy or use appropriate contraception. 

CONCLUSION

This study is intended to be conducted in substantial compliance with the requirements of 40 

CFR 26 subparts K and L. The amended protocol, when approved by SAIRB, should met all 

applicable ethical standards for the protection of human subjects of research, and all 

requirements for documentation of ethical conduct of the research. If this study is determined to 

be scientifically valid and relevant, there is no regulatory barrier to EPA’s reliance on it in 

actions under FIFRA or §408 of FFDCA.
11

NOTICE

This report has been written as part of the activities of the EPA Human Studies Review Board, a 

Federal advisory committee providing advice, information and recommendations on issues 

related to scientific and ethical aspects of human subjects research. This report has not been 

reviewed for approval by the Agency and, hence, the contents of this report do not necessarily 

represent the view and policies of the Environmental Protection Agency, nor of other agencies in 

the Executive Branch of the Federal government, nor does the mention of trade names or 

commercial products constitute a recommendation for use. You may obtain further information 

about the EPA Human Studies Review Board from its website at http://www.epa.gov/osa/hsrb. 

You may also contact the HSRB Designated Federal Officer, via e-mail at ord-osahsrb@epa.gov 

In preparing this document, the Board carefully considered all information provided and 

presented by the Agency presenters, as well as information presented by public commenters. 

This document addresses the information provided and presented within the structure of the 

charge by the Agency.
12

Appendix I. Simulation Results Comparing t-Distribution and Generalized Linear Model 

(GLiM) Means and Confidence Intervals (CIs)*

* Note: Subject‐Subject Background Variation (Logit SD) = 0.3, Subject‐Subject Protection 

Variation (Logit SD) = 0, Number of simulated samples = 5000, Seed = 98135183.
13

US ENVIRONMENTAL PROTECTION AGENCY

HUMAN STUDIES REVIEW BOARD

Chair 

Liza Dawson, Ph.D., Research Ethics Team Leader, Division of AIDS National Institutes of 

Health (NIH), National Institute of Allergy and Infectious Disease (NIAID), Bethesda, MD 

Vice Chair

Edward Gbur, Jr., Ph.D., Professor, Director, Agricultural Statistics Laboratory, University of 

Arkansas, Fayetteville, AR 

Members 

Gary L. Chadwick, Pharm.D., M.P.H., C.I.P., Senior Consultant, HRP Consulting Group, Inc. 

Fairport, NY

George Fernandez, Ph.D., Statistical Training Specialist, SAS Institute, Statistical Training and 

Technical Services, Sparks, NV

Kyle L. Galbraith, Ph.D., Manager, Human Subjects Protection, Carle Foundation Hospital,

Urbana, IL 

Jewell H. Halanych, M.D., Assistant Professor, Internal Medicine Residency Program, 

Montgomery Regional Campus, University of Alabama at Birmingham, Montgomery, AL

Randy Maddalena, Ph.D., Physical Research Scientist, Indoor Environment, Lawrence Berkeley 

National Laboratory, Berkeley, CA

Kenneth Ramos, M.D., Ph.D., PharmB, Associate Vice President, Precision Health Sciences, 

Professor of Medicine, Arizona Health Sciences Center, Tucson, AZ

Suzanne M. Rivera, Ph.D., M.S.W., Vice President for Research and Technology Management, 

Case Western Reserve University, Cleveland Heights, OH

Helen H. Suh, Ph.D., Associate Professor of Health Sciences, Northeastern University, Boston, 

MA

Jun Zhu, Ph.D., Professor of Statistics and of Entomology, Department of Statistics, University 

of Wisconsin – Madison, Madison, WI

Human Studies Review Board Staff 

Jim Downing, Executive Director, Human Studies Review Board Staff, Office of the Science 

Advisor, United States Environmental Protection Agency, Washington, DC