December 30, 2008 

EPA-HSRB-08-04 

George Gray, Ph.D. 

Science Advisor 

Office of the Science Advisor 

1200 Pennsylvania Avenue, NW 

Washington, DC 20460 

Subject: October 21-22, 2008 EPA Human Studies Review Board Meeting Report 

Dear Dr. Gray:

 The United States Environmental Protection Agency (EPA or Agency) requested the 

Human Studies Review Board (HSRB) to review scientific and ethical issues addressing: (1) 

Agricultural Handlers Exposure Task Force (AHETF) pesticide handler protocols: closed-cab 

airblast scenario, with addition of chlorothalonil, (2) AHETF pesticide handler protocols: opencab airblast scenario and (3) completed Carroll-Loye Biological Research picaridin-based 

mosquito repellant efficacy study LNX-001. In addition, the Board provided comments on the 

Guidelines for Product Performance Testing of Skin-Applied Insect Repellents. The Board also 

addressed procedural issues that surfaced as points of concern at the meeting. 

Consideration to Add Sulfur and Copper as Surrogate Chemical for AHETF Airblast 

Studies

Prior to the Human Studies Review Board meeting in October, the AHETF submitted to 

the Agency a request to support the addition of copper and sulfur as surrogate chemical for the 

open and closed cab scenario designs. Specifically, background information and sample product 

risk statements were submitted to the EPA on October 10th and transmitted to the HSRB on 

October 17th, but a revised and IRB-approved protocol reflecting the addition of these two 

compounds was not available for Agency or Board review. 

 The Board recommended that copper and sulfur not be used as surrogate chemicals for 

airblast applications based on concerns regarding exposure measurements and the applicability 

of data produced by such studies for the generic database under development. The Board also 

noted that it was not provided with a specific proposal describing the rationale and procedures 

for using these two chemicals. The Board affirmed its understanding that regulations governing 

HSRB review of third-party pesticide intentional exposure studies precludes the Board from 

reviewing a protocol unless the Agency (a) provides the Board with explicit information 

regarding scientific design and human subject protections and (b) evidence that the protocol has 

been submitted to an IRB for approval. 

Following the Board’s statement to this effect, the EPA stated that copper and sulfur 

would not be used as surrogates in the protocols under review and the Board proceeded to review 

the original protocols. 

Page 1 of 28 
1. AHETF Pesticide Handler Protocols: Closed-Cab Airblast Scenario, with Addition of 

Chlorothalonil 

Science

 If the proposed closed-cab airblast application field study protocols are revised as 

suggested in the EPA review, and data collection and design weaknesses are modified as detailed 

by the HSRB in this report, then the research is likely to generate scientifically reliable data, 

useful for assessing the exposure of handlers who apply liquid pesticides using airblast 

equipment drawn by vehicles with closed cabs. However, due to the limitations posed by the use 

of purposive sampling, small sample size, reduced range of exposures and representativeness of 

participating growers noted in previous Board reports, the data will not meet the assumptions of 

most statistical analyses that rely on mean scores or continuous data. Conclusions drawn from 

these data must recognize these limitations. 

Ethics

If revised as suggested by the Board, the research described in these three protocols is 

likely to meet the applicable requirements of 40 CFR part 26, subparts K and L. 

2. AHETF Pesticide Handler Protocols: Open-Cab Airblast Scenario 

Science

 For the open-cab airblast protocols the Board reiterated its recommendations already put 

forward for the closed-cab protocols. In addition, the Board recommended that a careful 

evaluation be conducted before combining old and new open cab datasets. Finally, the Board 

concluded that a more even distribution of monitoring units across the five proposed categories 

of active ingredient handled might strengthen the design and provide greater insight into the 

question of proportionality. 

Ethics

If revised as suggested by the Board in its review, the research described in these three 

protocols is likely to meet the applicable requirements of 40 CFR part 26, subparts K and L. 

3. Completed Carroll-Loye Biological Research Picaridin-Based Mosquito Repellant 

Efficacy Study LNX-001 

Science

The Carroll-Loye Biological Research study LNX-100 is sufficiently sound, from a 

scientific perspective, to be used to assess the repellent efficacy of the tested formulations 

against mosquitoes. However, the calculation of Mean “Complete Protection Time” is 

Page 2 of 28 
inappropriate for censored data and the Agency should not rely on this aspect of the data analysis 

to assess the repellent efficacy of the tested formulations. 

Ethics

The Board concurred with the Agency’s assessment that the study submitted for review 

was conducted in substantial compliance with subparts K and L of 40 CFR 26. 

Guidelines for Product Performance Testing of Skin-Applied Insect Repellents 

While the Board was not provided a charge for consideration of the guidelines, it did 

have several comments to enhance the utility of the document. The Board noted that 

improvements in insect repellent protocols would be derived from portions of the guidelines and 

that the document was well written and clear. The Board appreciated the inclusion of landings as 

well as bites as an endpoint and encouraged the Agency to consider landings as the preferred 

endpoint. 

The Board expressed serious concerns that if the erroneous assumptions and inconsistent 

statements regarding the statistical analysis plan are not corrected prior to publication of the 

Guidelines, investigators following the guideline recommendations would develop scientifically 

unreliable protocols. The Board underscored that it would continue to evaluate protocols 

submitted for review to the HSRB based on appropriate statistical assumptions and analytic plans 

and on that basis might recommend rejection of a protocol that followed those elements of the 

Guidelines that were incorrect. Based on continued statistical concerns with the guidelines, the 

Board strongly recommended that a Board working group provide a review of the statistical 

portion of the guidelines, for subsequent consideration by the HSRB at a public meeting, prior to 

the final guidelines being released by EPA to the public. 

Clarification of Board Report Process 

Prior to the posting of and public meeting to finalize the June 2008 report, the Agency 

wrote a memo to the Board stating that the Agency had approved the implementation of AHETF 

studies based in part on their interpretation of Board discussion at the June meeting and the 

Chair’s minutes. The Board recognized the need for a more rapid turn around of its final report to 

avoid future confusion. The Board clarified its process for approving final recommendations and 

adopted a new report schedule and format. 

Sincerely, 

 Celia Fisher, Ph.D., Chair 

 EPA Human Studies Review Board

Page 3 of 28 
NOTICE

This report has been written as part of the activities of the EPA Human Studies Review 

Board, a Federal advisory committee providing advice, information and recommendations on 

issues related to scientific and ethical aspects of human subjects research. This report has not 

been reviewed for approval by the Agency and, hence, the contents of this report do not 

necessarily represent the view and policies of the Environmental Protection Agency, nor of other 

agencies in the Executive Branch of the Federal government, nor does the mention of trade 

names or commercial products constitute a recommendation for use. Further information about 

the EPA Human Studies Review Board can be obtained from its website at 

http://www.epa.gov/osa/hsrb/. Interested persons are invited to contact Paul Lewis, Designated 

Federal Officer, via e-mail at lewis.paul@epa.gov. 

 In preparing this document, the Board carefully considered all information provided and 

presented by the Agency presenters, as well as information presented by public commenters. 

This document addresses the information provided and presented within the structure of the 

charge by the Agency. 

Page 4 of 28 
U. S. ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW 

BOARD MEMBERS 

Chair

Celia B. Fisher, Ph.D., Marie Ward Doty Professor of Psychology, Director, Center for Ethics 

Education, Fordham University, Department of Psychology, Bronx, NY 

Vice Chair

William S. Brimijoin, Ph.D., Chair and Professor, Molecular Pharmacology and Experimental 

Therapeutics, Mayo Foundation, Rochester, MN 

Members 

Alicia Carriquiry, Ph.D., Professor, Department of Statistics, Iowa State University 

Snedecor Hall, Ames, IA 

Gary L. Chadwick, PharmD, MPH, CIP, Associate Provost, Director, Office for Human Subjects 

Protection, University of Rochester, Rochester, NY 

Janice Chambers, Ph.D., D.A.B.T., William L. Giles Distinguished Professor, Director, Center 

for Environmental Health Sciences, College of Veterinary Medicine, Mississippi State 

University, Mississippi, MS 

Richard Fenske, Ph.D., MPH, Professor and Associate Chair, Department of Environmental and 

Occupational Health Sciences, University of Washington, Seattle, WA 

Susan S. Fish, PharmD, MPH, Professor, Biostatistics & Epidemiology, Boston University 

School of Public Health, Co-Director, MA in Clinical Investigation, Boston University School of 

Medicine, Boston, MA 

Suzanne C. Fitzpatrick, Ph.D., D.A.B.T, Senior Science Policy Analyst, Office of the 

Commissioner, Office of Science and Health Coordination, U.S. Food and Drug Administration, 

Rockville, MD 

Dallas E. Johnson, Ph.D., Professor Emeritus, Department of Statistics, Kansas State University, 

Manhattan, KS 

Kannan Krishnan, Ph.D., Professor, Département de santé environnementale et santé au travail, 

Faculté de medicine, Université de Montréal, Montréal, Canada * 

Michael D. Lebowitz, Ph.D., FCCP, Retired Professor of Public Health & Medicine. University 

of Arizona, Tucson, AZ 

Page 5 of 28 
Lois D. Lehman-Mckeeman, Ph.D., Distinguished Research Fellow, Discovery Toxicology, 

Bristol-Myers Squibb Company, Princeton, NJ 

Jerry A. Menikoff, M.D., Director, Office of Human Subjects Research, Office of the Director, 

National Institutes of Health, Bethesda, MD 

Rebecca Parkin, Ph.D., MPH, Associate Dean for Research and Public Health Practice, School 

of Public Health and Human Services, The George Washington University, Washington, DC 

Sean Philpott, Ph.D., MS Bioethics, Science and Ethics Officer, Global Campaign for 

Microbiocides, PATH, Washington, DC 

Ernest D. Prentice, Ph.D., Associate Vice Chancellor for Academic Affairs, University of 

Nebraska Medical Center, Omaha, NE 

Richard Sharp, Ph.D., Director of Bioethics Research, Department of Bioethics, Cleveland 

Clinic, Cleveland, OH ** 

Linda J. Young, Ph.D., Professor, Department of Statistics, Institute of Food and Agricultural 

Sciences, University of Florida, Gainesville, FL 

Human Studies Review Board Staff

Paul I. Lewis, Ph.D., Executive Director, Human Studies Review Board Staff, Office of the 

Science Advisor, United States Environmental Protection Agency, Washington, DC 

* Resigned from the HSRB at the end of October 21-22, 2008 Public Meeting 

** Not in attendance at the October 21-22, 2008 Public Meeting 

Page 6 of 28 
INTRODUCTION 

On October 21-22, 2008, the United States Environmental Protection Agency’s (EPA or 

Agency) Human Studies Review Board (HSRB) met to address scientific and ethical issues 

concerning: (1) Agricultural Handlers Exposure Task Force (AHETF) pesticide handler 

protocols: closed-cab airblast scenario, with addition of chlorothalonil, (2) AHETF pesticide 

handler protocols: open-cab airblast scenario, and (3) completed Carroll-Loye Biological 

Research picaridin-based mosquito repellant efficacy study LNX-001. The Board also provided 

comments on the Agency’s draft guidelines for product performance testing of skin-applied 

insect repellents. In addition, the Board addressed procedural issues that surfaced as points of 

concern at the meeting. Each of these topics is discussed more fully below. 

Regulations Relevant to Studies Submitted for Review

EPA’s regulation, 40 CFR §26.1125, requires a sponsor or investigator to submit to EPA, 

before conducting a study involving intentional exposure of human subjects, materials describing 

the proposed human research in order to allow EPA to conduct scientific and ethics reviews. In 

addition, EPA’s regulation, 40 CFR §26.1601, requires EPA to seek HSRB review of the 

proposed research. Because the two AHETF proposed studies involves scripted exposure, it 

meets the regulatory definition of “research involving intentional exposure of a human subject” 

and thus these cited provisions of regulation apply to it. 

The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB review of an 

EPA decision to rely on the results of these studies. Carroll-Loye Biological Research has 

submitted data to support continued registration of two test materials. EPA has reviewed the 

research, applying the standard in 40 CFR §26.1705, which states: “Except as provided in 

§26.1706, in actions within the scope of §26.1701, EPA shall not rely on data from any research 

initiated after April 7, 2006, unless EPA has adequate information to determine that the research 

was conducted in substantial compliance with subparts A through L of this part...This 

prohibition is in addition to the prohibition in §26.1703. “ 

 

REVIEW PROCESS 

On October 21-22, 2008, the Board had a public face-to-face meeting in Arlington, 

Virginia. Advance notice of the meeting was published in the Federal Register “Human Studies 

Review Board: Notice of Public Meeting” (73 Federal Register 73, 53422). At the public 

meeting, following welcoming remarks from Agency officials, the Board heard presentations 

from the Agency on the following topics: (1) Agricultural Handlers Exposure Task Force 

(AHETF) pesticide handler protocols: closed-cab airblast scenario, with addition of 

Chlorothalonil, (2) AHETF pesticide handler protocols: open-cab airblast scenario, (3) 

completed Carroll-Loye Biological Research picaridin-based mosquito repellant efficacy study 

LNX-001, and (4) Agency Guidelines for Product Performance Testing of Skin-Applied Insect 

Repellents. 

Page 7 of 28 
Oral comments 

The following oral comments were presented at the meeting: 

Agricultural Handlers Exposure Task Force (AHETF) Pesticide Handler Protocols: Closed-Cab 

Airblast Scenario, with Addition of Chlorothalonil

Victor Canez, Ph.D. from BASF on behalf of the AHETF 

AHETF Pesticide Handler Protocols: Open-Cab Airblast Scenario

Victor Canez from BASF on behalf of the AHETF 

Completed Carroll-Loye Biological Research picaridin-based mosquito repellant efficacy study 

LNX-001

Scott Carroll, Ph.D., on behalf of Carroll-Loye Biological Research 

Guidelines for Product Performance Testing of Skin-Applied Insect Repellents

Tom Osimitz, Ph.D. of Science Strategies LLC on behalf of the DEET Task Force 

Scott Carroll, Ph.D., on behalf of Carroll-Loye Biological Research 

Mr. Nick Spero and Robin Todd, Ph.D. on behalf of ICR 

Written comments 

Written comments were received from: 

B. Sachau, a public citizen 

For their deliberations, the Board considered the materials presented at the meeting, 

written public comments and Agency background documents (e.g., the published literature, 

Agency data evaluation record, weight of evidence review, ethics review, pesticide human study 

protocols and Agency evaluation of the protocol or study). For a comprehensive list of 

background documents visit www.regulations.gov 

CHARGE TO THE BOARD AND BOARD RESPONSE 

Board Response to Request to Add Sulfur and Copper as Surrogate Chemicals for Closed 

and Open Cab Airblast Studies 

HSRB Recommendation

The Board decided not to consider the addition of copper and sulfur as surrogate 

chemicals for open and closed cab protocols based on concerns that the lack of a detailed and 

IRB approved protocol prevented adequate Board review regarding exposure measurements, the 

Page 8 of 28 
applicability of data produced by such studies for the generic database under development and 

human subjects protections. 

HSRB Detailed Recommendations and Rationale

 Prior to the Human Studies Review Board meeting in October, the AHETF submitted to 

the Agency a request to support the addition of copper and sulfur as surrogate pesticides for the 

open and closed Cab scenario designs discussed previously. Specifically, background 

information and sample product information were submitted to the EPA on October 10th and 

transmitted to the HSRB on October 17th, but a revised and IRB-approved protocol reflecting the 

addition of these two compounds was not available for Agency or Board review. 

 

 The Board recognized the practical considerations underlying the request, but expressed a 

number of scientific concerns. It was not clear to the Board that these compounds would be 

applied in the same manner as the current surrogate pesticides; e.g., nozzle size, droplet size. It 

was thought that the replacement of organic, semi-volatile pesticides with inorganic, non-volatile 

compounds would present significant challenges in regard to air sampling measurements. It was 

noted that the sampling methods typically used for these two categories of chemicals were 

substantially different. The Board was not convinced that measurement of copper or sulfur in air 

samples would properly represent inhalation exposures to the insecticides. It was also not clear to 

the Board that the current dermal sampling methods would be adequate to assess exposures to 

copper or sulfur. It was noted that copper and sulfur are present at much higher concentrations in 

their formulated products than are insecticides, and that they may be applied at much higher 

rates. It was not clear if the potential for breakthrough of the dermal sampling garments or the 

removal efficiency of hand wash and wipe procedures would be comparable. 

 Based on an internal review of the toxicological profiles of these two compounds, the 

Agency concluded that copper and sulfur did not pose significant additional safety concerns and 

thus supported their use as surrogate pesticides in the protocols AHE57, AHE58, and AHE59 

pending protocol amendment and IRB review. The Board expressed concern that the lack of 

revised and IRB-approved protocols and supporting documentation prevented careful 

consideration of the risks that the addition of these two compounds could pose to study 

participants. In addition the Final Human Studies rule precluded the HSRB from reviewing the 

protocols if the AHETF intended to amend the three submitted protocols to include the two 

compounds after the October 2008 HSRB meeting was over. The Board affirmed its 

understanding that regulations governing HSRB review of third-party pesticide intentional 

exposure studies precludes the Board from reviewing a protocol unless the Agency (a) provides 

the Board with explicit information regarding scientific design and human subject protections 

and (b) evidence that the protocol has been submitted to an IRB for approval. Specifically, 40 

CFR 26.1125 states that: 

Any person or institution who intends to conduct or sponsor human research covered by 

Sec. 26.1101(a) shall, after receiving approval from all appropriate IRBs, submit to EPA 

prior to initiating such research all information relevant to the proposed research 

specified by Sec. 26.1115(a). 

Page 9 of 28 
At the time of the October 2008 HSRB meeting, the IRB of record, Independent 

Institutional Review Board, Inc. of Plantation, FL (IIRB) had not received an application, 

reviewed or approved the addition of copper and sulfur to these three protocols, so the regulatory 

requirements of 40 CFR 26.1125 were not met. 

The Board thus declined to consider the addition of copper and sulfur as surrogate 

pesticides for open and closed cab protocols. Following the Board’s statement to this effect, the 

EPA stated that the copper and sulfur would not be used as surrogates in the protocols under 

review and the Board proceeded to review the original protocols. 

1. Assessment of AHETF Pesticide Handler Protocols: Closed-Cab Airblast Scenario, with 

Addition of Chlorothalonil 

Overview

The HSRB has previously considered the design and conduct of research to measure the 

levels of exposure received by workers when handling (i.e., mixing, loading, or applying) 

pesticides. Two industry Task Forces, the Antimicrobials Exposure Assessment Task Force II 

(AEATF) and the AHETF, have previously submitted materials for HSRB review. The Board 

had addressed this kind of research in its meetings in June 2006, June 2007, April 2008, and June 

2008. 

At its June 2008 meeting the Board reviewed and provided recommendations for enhancing 

the scientific validity and usefulness of two AHETF field studies on the exposure of workers 

applying liquid pesticides using closed-cab airblast equipment. At the October 2008 meeting 

EPA presented proposals from the AHETF for the three additional field studies required to fulfill 

this scenario design. These protocols are very similar to those reviewed by the Board in June, but 

involve monitoring workers making pesticide applications to different crops in different regions. 

In addition, these protocols incorporated numerous refinements agreed to by EPA and the 

AHETF shortly after the June HSRB meeting. 

 

The three new protocols involve monitoring applications of surrogate pesticides to Michigan 

cherry orchards (AHE57), California vineyards (AHE 58), and Washington apple orchards 

(AHE59). Together with the data generated under the two field study protocols reviewed by the 

Board in June 2008, these will fulfill the design of the closed-cab air blast application scenario. 

Because of the widespread use of chlorothalonil on Michigan cherries, the Agency requested 

that the Board consider a revision to the scenario design to add several formulations of this active 

ingredient to the list of permissible surrogate pesticides. 

Page 10 of 28 
Charge To The Board 

Science

If the proposed closed-cab airblast application field study protocols AHE57, AHE58, and 

AHE59 are revised as suggested in EPA’s reviews and if the research is performed as described: 

Is the research likely to generate scientifically reliable data, useful for assessing the exposure of 

handlers who apply liquid pesticides using airblast equipment drawn by vehicles with closed 

cabs? 

Board Response to the Charge 

HSRB Recommendation

 If the proposed closed-cab airblast application field study is revised as suggested in the 

EPA review and data collection and design weaknesses are modified as detailed below, then the 

research is likely to generate scientifically reliable data, useful for assessing the exposure of 

handlers who apply liquid pesticides using airblast equipment drawn by vehicles with closed 

cabs. However, due to the limitations posed by the use of purposive sampling, small sample size, 

reduced range of exposures and representativeness of participating growers, the data will not 

meet the assumptions of most statistical analyses that rely on mean scores or continuous data. 

Conclusions drawn from these data must recognize these limitations. 

HSRB Detailed Recommendations and Rationale

The Board learned at the October meeting that, following the June 2008 HSRB meeting, 

the Agency had instructed the AHETF to move forward with airblast application studies. Since 

the Board’s report for the June meeting was not finalized until the October meeting, the protocols 

that will be used for airblast studies will not necessarily incorporate some of the Board’s 

recommendations from the June meeting. Thus, the primary concerns raised by the Board at this 

meeting were similar to those presented in the HSRB June meeting report. The Board 

recommended that the Agency consider the concerns listed below when it provides its final 

response to AHETF regarding these protocols. In particular, the Board viewed the selection of 

monitoring units from separate farms to be a modification needed to ensure that the design and 

analyses are scientifically sound. 

1. The Board again noted that selecting 5 monitoring units in each cluster, each from a different 

farm is needed to ensure that the design and analyses are scientifically sound. 

2. The Board reiterated its observation that many aspects of the proposed studies were likely to 

reduce the range of exposures that would be measured with applicators under real-world 

conditions. While a reduction in the range of exposures may be unavoidable due to practical 

considerations, it should be considered by the Agency when evaluating the usefulness of the data 

produced by these studies. 

Page 11 of 28 
3. The Board once again advised the Agency to require collection of information on growers who 

do not respond or who decline to participate, such that the representativeness of participating 

growers can be evaluated. The usefulness of the data will be enhanced by statistical comparisons 

between characteristics of participating growers and those that decline. The Board expressed 

concern regarding the inadequacies of a master list for which 43% of growers were listed as 

“unreachable.” The inadequacy of the list makes the design vulnerable to selection bias that can 

seriously compromise the usefulness of the data. If the master list cannot be improved then these 

limitations need to be considered in data analysis and interpretation of results and should be 

explicitly described whenever the EPA refers to these studies. 

4. The Board also continued to recommend the selection of Local Site Coordinators with 

demonstrable training and expertise in survey implementation to ensure optimal recruiting and 

thereby enhance the usefulness of the data. 

Ethics

Charge to the Board 

 Is the research likely to meet the applicable requirements of 40 CFR part 26, subparts K 

and L? 

Board Response to the Charge 

HSRB Recommendation

If revised as suggested in the EPA review and as detailed in the specific points below, the 

research described in these three protocols is likely to meet the applicable requirements of 40 

CFR part 26, subparts K and L. 

HSRB Detailed Recommendations and Rationale

A. Clarifications Required (such clarifications would also be subject to IRB review).

1. The Local Site Coordinator, the Principal Field Investigator, the Field Facility, the 

Analytical Facility, and the Principal Analytical Investigator must be identified in the 

protocol. Any key members of the research team who will have contact with the research 

subjects or their identifiable data must receive and document their recent (not expired) 

training in human subjects protection. 

Many training programs in human subjects protection issue certificates of completion 

indicating the date of issuance. Depending on the program, and the requirements of the relevant 

IRB, there may be policies relating to the frequency of training. Many certificates “expire” after 

two or three years, and in the U.S., the “best practice” standard is to require continual training or 

retraining at least every three years. It is important for the certifications of all key research 

personnel to be current. 

Page 12 of 28 
The specific identities of key research personnel have an impact on the assessment of the 

scientific validity of the research, which in turn relates to an ethical and regulatory assessment of 

the reasonableness of risks in relation to the potential benefits of research. 

2. Revise protocols to remove the inconsistencies in listing all three surrogate chemicals 

that have been reviewed. Evaluation of risk-benefit must be informed by information on 

all surrogate chemicals involved.

3. Revise subject recruitment plan to specifically address the probability that subjects may 

also be growers. Based on data from initial studies by AHETF, subjects can be workers 

or growers. The recruitment plan is directed toward recruitment of workers. Information 

should be provided on whether any additional considerations must be given to ethical 

recruitment of growers. 

4. Remove risks of agricultural work from the listing of risks related to the research. The 

agricultural work itself is not part of the research. Thus the risks associated with that 

work are unrelated to research and should be removed from this and all future discussions 

of research risks or risk-benefit relationship. 

5. Identify risks of pesticide products that are due to scripting. Working with the pesticide 

products is not part of the research. However the scripting that is done for the purpose of 

the research may produce exposures to higher concentrations or longer durations, which 

are incremental risks of the research. These are the risks associated with the research. 

6. Revise the IRB application to indicate expected ethnic/racial distribution specific to the 

location in which the study will be conducted. The ethnic/racial distribution indicated on 

all IRB applications is identical for all studies submitted. This is unlikely as the research 

will take place at sites in widely dispersed geographic locations. 

7. Answer question 17 in the IRB application, and submit a Data Safety Monitoring Plan 

Form as required by IIRB. Question 17 asks if risk is minimal or greater than minimal. 

This question is not completed on any of the applications. If greater than minimal, the 

IRB application requires attachment of “a Data Safety Monitoring Plan Form that 

includes procedures for monitoring the progress of the trial and the safety of the subjects, 

reporting adverse events (AEs) and ensuring data accuracy and protocol compliance.” 

Because this study is greater than minimal risk (the major risk being heat exposure due to 

whole body dosimeters), such a form should be submitted to the IIRB, even if all of the 

items requested in the form are found elsewhere in the study protocols.

. 

8. Revise the description of the consent process to indicate how bilingual witnesses are 

going to be recruited and the amount of remuneration, as referenced in the recruitment 

standard operating procedures. This information is critical to assess whether or not the 

consent process is conducted in a manner to minimize potential coercion or undue 

influence of Spanish-speaking subjects. One suggestion is to recruit a bi-lingual farm 

worker advocate or other community representative to serve in this role. 

Page 13 of 28 
9. Revise the section of the recruitment SOP that relates to recruitment and enrollment of 

illiterate or low literacy subjects. 

 While an excellent attempt to address this issue has been made, there are aspects with 

which the HSRB is uncomfortable. The Board recommended that the AHETF consult with farm 

worker advocacy groups on this issue. The SOP states that “reading ability ...will be ascertained 

by the person obtaining consent by asking the potential subject to read a portion of the consent 

form out loud.” This can lead to embarrassment of a low literacy or illiterate person. In addition, 

subjects are asked to identify if they cannot read. Subjects may be hesitant to disclose illiteracy, 

and may feel shame at their condition. By relying more on the consent process than the form, the 

ability to read competently becomes less important, and then embarrassment and shame do not 

become risks associated with the research. 

 Also, the recruitment SOP states that if the researcher is not comfortable with the 

potential subject’s level of literacy then “the Study Director will also verify that the worker has 

apparently understood the materials read to and discussed with them by asking specific questions 

to assess comprehension (see Sec. 7.10.a).” The Board recommended clarity how the test of 

comprehension will be used for those who are judged literate and non-literate. Sec. 7.10.a 

suggests that all participants will undergo tests of comprehension but earlier sections are unclear 

(e.g., Sec. 6.5a and 6.5c) 

B. Clarifications needed prior to study implementation

1. Explain why the IRB review of one protocol out of three was tabled, and why 

communication from IIRB to the investigator was not in writing. IRB communication 

includes a “note to file” from the principal investigator that IRB review of AHE57 was 

initially tabled, but not AHE58 or AHE59. The Board requested that the IRB provide 

this communication in writing.

2. Describe how individual level data will be presented to the subject upon request. 

 Several issues are raised with this point. How will the data be framed or translated to the 

worker? As an example, how will the AHETF work to prevent workers from changing future 

behavior to their own detriment if their individual risk levels are lower than the average of all 

workers? The HSRB felt that consideration must be given to the ways in which data could be 

used or misused by the individual subjects. For example, a worker who finds out simply that 

his/her individual exposure levels were less than the group average may feel that s/he can be less 

cautious in the future (i.e. behavioral disinhibition), thereby putting him/herself at greater risk. 

C. Modifications to informed consent form prior to study implementation (and subject to IRB 

review)

1. Harmonize enrollment criteria (inclusion, exclusion) between the protocol and consent 

form. Providing potential subjects with the most accurate and informative information 

allows them to make the best decision about participation. 

Page 14 of 28 
2. In the consent form, replace “regular working hours” with the specific hours for the time 

zone in which the research will take place. Since the research is being conducted in 

locations that may be in different time zones than the IIRB, the term “regular working 

hours” is not informative to the subjects. 

3. Revise the consent form with the underlined word: “you may refuse medical treatment. 

However, you cannot refuse medical treatment if you get sick from too much exposure to 

pesticides.....” As is, these two sentences appear contradictory. Adding “however” 

corrects the apparent contradiction. 

4. Revise the forms (consent forms, product risk statements, recruitment materials) that have 

been translated into Spanish. Spanish translations should be written in common, correct, 

simple Spanish. 

5. Revise the consent form to identify who makes the determination that an injury is studyrelated. Providing potential subjects with the most accurate and informative information 

allows them to make the best decision about participation.

6. Revise attestation statement for witness on consent form to eliminate subject’s 

understanding or accuracy of the consent process. A witness cannot honestly attest to a 

subject’s understanding or accuracy of the consent process. The witness can only attest 

that the consent process was consistent with the consent form. 

7. Revise consent form’s attestation statement of person obtaining consent: to “ I conducted 

the private consent meeting with the worker named above and confirm that consent was 

given voluntarily after the participant was fully informed of all of the information stated 

above, including the procedures, risks, and benefits of the research.” There is more that 

the researcher should have discussed with the potential subject than benefits, risks and 

procedures. In addition, the procedures and risks should precede the benefits in the 

discussion. 

8. Confirm and clarify in informed consent that compensation for research-related injury 

includes any co-pay. Subjects in this type of study should not have to assume any out-ofpocket expenses related to treatment for research-related injury. 

D. Points to Consider

1. Consider revising Product Risk Statements to indicate which sign and symptoms are due 

to use that follows the product label, and which are due to overdose or massive spills or 

excessive exposure beyond what is expected to occur in the research. 

 There was not consensus among HSRB members as to whether or not signs and 

symptoms of overdose or massive spills or excessive exposure beyond what is expected to occur 

in the research need to be included in the Product Risk Statements. Some members felt that since 

these risks are part of the risks of the agricultural work rather than the research, they should not 

be included. Others felt that the additional information would be helpful to potential subjects, 

and that education about pesticides and their exposure risks could be considered a benefit for 

individual study participants. 

 The Board agreed with EPA’s proposal to use generic product information sheets based 

on the active chemical(s) rather than specific products. This would simplify paperwork and 

decrease the probability of error. 

Page 15 of 28 
2. Define the term “sufficient” when used in the inclusion criterion “sufficient experience”. 

Use of a subjective term in the protocol increases the probability of inconsistent subject 

populations. 

3. Revise wording of eligibility criterion that states “Confirm that you normally wear the 

personal protective equipment (PPE) listed on the Product Risk Statement. Confirm that 

you will follow label directions.” 

 By asking potential subjects to “confirm”, it is unlikely that the subject will say that they 

do not normally wear PPE or follow label directions. The inclusion criterion could be worded by 

deleting “confirm that you” and begin that criterion with “usually”. Likewise number 3 should 

delete “confirm you do”. Then in the screening process, potential subjects should be asked an 

open-ended question like “what types of protective gear do you usually wear? 

 On page 45 of the CCAB scenario design document, there are clearly articulated program 

requirements for use of minimal PPE. The CCAB scenario design document, for example, states: 

The desired PPE and clothing situation for this scenario is: 

• Long pants and long-sleeved shirt 

• Chemical-resistant gloves (new, provided by AHETF) only when contacting 

treated surfaces outside of the closed cab 

• Any footwear that are required by the label or that the workers choose to wear (as 

long as they are consistent with the WPS) 

• Any eyewear required by the label or desired by the worker (except full face 

shields that would prevent face exposure) 

• Any respiratory protection required by the label or desired by the worker (except 

full face respirators which would prevent face exposure) 

 As these are exclusionary criteria, in order to collect exposure data for workers using 

minimal PPE, Task Force researchers should be careful that they are not recruiting participants 

who agree to use less PPE than normal in order to receive payment. By asking open-ended 

questions such as “what types of protective gear do you normally wear,” researchers will be able 

to obtain an unbiased assessment of potential participant’s normal practices and exclude those 

who use PPE that would potentially bias the exposure data collected. 

4. Add to the protocol some provisions for counting and reporting the number of potentially 

eligible workers linked to each grower, the number of potential subjects attending initial 

group meetings, number attending individual consent interviews, number consenting to 

participation, number subsequently withdrawing or being withdrawn (with the reason for 

withdrawal) and number completing the study. 

 These data can provide evidence of equitable subject selection, a regulatory and ethical 

requirement. In addition, it is only through these numbers that any assessment can be made as to 

the representativeness of the study population, and to the generalizability of the data. 

Page 16 of 28 
2. Assessment of AHETF Pesticide Handler Protocols: Open-Cab Airblast Scenario

Overview

The three proposed field study protocols are similar to those the Board reviewed for the 

closed cab airblast scenario, modified to monitor additional exposure, especially to the head, 

resulting from use of open cab equipment. The Agency has concluded that a previous AHETF 

study, AHE-07A, conducted before promulgation of EPA’s human research rule, provided 15 

monitoring units that met contemporary standards of scientific validity and then-prevailing 

standards of ethical conduct. The AHETF proposes to use the data from this earlier study in its 

generic database, and to supplement it with 15 more monitoring units, obtained through three 

additional field studies. The HSRB has not been provided with details or data from this prior 

study. 

Charge to the Board 

Science 

 If proposed open-cab airblast application field study protocols AHE62, AHE63, and 

AHE64 are revised as suggested in EPA’s reviews, and if the research is performed as described: 

Is the research likely to generate scientifically reliable data, useful for assessing the exposure of 

handlers who apply liquid pesticides using airblast equipment drawn by vehicles with open cabs? 

Board Response to the Charge

HSRB Recommendation

 If the proposed open-cab airblast application field study protocols are revised as suggested 

in EPA’s reviews, and data collection and design weaknesses are modified as detailed below, 

then the research is likely to generate scientifically reliable data, useful for assessing the 

exposure of handlers who apply liquid pesticides using airblast equipment drawn by vehicles 

with open cabs. However, the Board has not reviewed the procedures used for the pre-rule 

existing data and cannot evaluate whether combining these studies and existing data is 

scientifically justified. The Board concluded that a more even distribution of monitoring units 

across the five proposed categories of active ingredient handled would strengthen the design and 

provide greater insight into the question of proportionality. The Board recommended that once 

the study is completed the Agency formally evaluate whether combining the datasets is 

appropriate. Finally, as the Board has stated throughout its review of these types of protocols, 

due to the limitations posed by the use of purposive sampling, small sample sizes, reduced range 

of exposures and representativeness of participating growers, the data will not meet the 

assumptions of most statistical analyses that rely on mean scores or continuous data. 

Interpretations of the usefulness of the data should recognize these limitations. 

Page 17 of 28 
HSRB Detailed Recommendations and Rationale

 For the open-cab airblast protocols the Board reiterated its recommendations already put 

forward for the closed-cab protocols. The Board had two additional concerns. 

1. The first concern focused on the intention to add the new data to existing data. The Board has 

not seen the pre-Rule open-cab airblast data that have already been collected, nor has it seen the 

protocols used to generate these data. The Board recommended that once the proposed protocols 

are completed, the data from the older and new studies should be evaluated to ensure that 

combining the datasets is appropriate. There may be unforeseen reasons why these data cannot 

be combined. If this were the case, the AHETF would need to collect additional open-cab 

airblast data to meet the sample size objective. 

2. The second concern focused on the AHETF plan to distribute the 15 new monitoring units 

evenly across five categories of pounds of active ingredient handled (5-9; 10-17; 18-30; 31-55; 

56-100). The existing open-cab dataset consists of 15 monitoring units: one in the middle 

category, and seven in each of the top two categories. The current plan will result in a monitoring 

unit distribution of 3, 3, 4, 10, and 10 across the five categories. The Board was not able to 

ascertain the rationale for this distribution. The Board concluded that to gain greater insight into 

the question of proportionality the final number of such units per scenario should be similar, and 

that sufficient numbers of new MUs in each scenario be collected to compare old and new. 

 

Ethics

 Charge to the Board 

 Is the research likely to meet the applicable requirements of 40 CFR part 26, subparts K 

and L? 

Board Response to the Charge 

HSRB Recommendation

If revised as suggested by the Agency review and Board suggestions, the research 

described in these three protocols is likely to meet the applicable requirements of 40 CFR part 

26, subparts K and L. 

HSRB Rationale

 The Board considered the comments and concerns that had been provided for the closed 

cab protocols highlighted earlier to be applicable to the open cab protocols as well. 

3. Assessment of Completed Carroll-Loye Biological Research Picaridin-Based Mosquito 

Repellant Efficacy Study LNX-001

Page 18 of 28 
Overview of the Study

LNX-001 was a field-based study of repellency to mosquitoes of two conditionally 

registered products (one lotion and one pump spray) containing 20% picaridin; these two 

formulations are KBR 3023 All-Family Insect Repellent Cream (EPA Reg. No. 39967-50) and 

KBR 3023 All-Family Insect Repellent Spray (EPA Reg. No. 39967-53). It was conducted by 

Carroll-Loye Biological Research of Davis, CA between May 14th and June 15th, 2008. The 

study was sponsored by LANXESS, Inc. of Pittsburgh, PA. The study was required by EPA to 

support registration of these two products. 

As submitted to the EPA, the completed study consisted of two interdependent analyses: 

1) a dosimetry study designed to determine the amount of lotion or spray that typical users would 

typically apply; and 2) an efficacy study designed to measure the effectiveness of each 

compound as repellent for those species of mosquitoes likely to be vectors for West Nile Virus 

(WNV) in the United States. The efficacy study was conducted at two environmentally distinct 

field sites in Butte and Glenn Counties, CA. Each phase included 10 participants (5 female and 5 

male). Two experienced participants (1 male and 1 female) served as untreated controls to 

measure ambient mosquito pressure. Based on the findings of the dosimetry phase, it was 

determined that the margin of exposure (MOE) for dermal toxicity at the highest dose level (seen 

with use application of lotion on the arms) was about 266. 

The efficacy of each formulation as a mosquito repellent was determined by measuring 

the ability of the formulations to prevent mosquito landings (defined as “Lite with Intent to 

Bite”; LIBe). During the field study, treated participants and untreated controls exposed their 

limbs to mosquitoes for one minute at fifteen-minute intervals, for 10 hours (40 exposure 

periods) post-treatment or until failure of efficacy, whichever occurred first. Failure of efficacy 

was defined as the two confirmed LIBes within a single exposure period, or a single LIBe within 

each of two consequtive exposure periods. Participants worked in pairs to facilitate identification 

of LIBe and to aspirate mosquitoes during exposure periods. No actual bites were reported, and 

aspirated mosquitoes were stored for later identification and arboviral testing. 

Statistical analyses performed included: Mean “Complete Protection Time” (CPT) for 

the pump spray formulation was 11.6 hours at both field sites; Mean CPT for the lotion was ≥14 

hours at the Butte County site and 13.5 hours at the Glenn County site. 

Science

Charge to the Board 

Is the CLBR study LNX-001 sufficiently sound, from a scientific perspective, to be used 

to assess the repellent efficacy of the tested formulations against mosquitoes? 

Page 19 of 28 
Board Response to the Charge 

HSRB Recommendation

The data collected in the Carroll-Loye Research study LNX-001 is sufficiently sound, 

from a scientific perspective, to be used to assess the repellent efficacy of the tested formulations 

against mosquitoes. However, the calculation of Mean “Complete Protection Time” is 

inappropriate for censored data and the Agency should not rely on this aspect of the data analysis 

to assess the repellent efficacy of the tested formulations. 

HSRB Detailed Recommendations and Rationale

1. Overall the study was conducted according to the approved protocol. There were some 

minor deviations, but these did not affect the quality or validity of the scientific findings. 

One deviation was the use of historical limb measurement data (within the past 2 years) 

instead of taking new measurements on subjects who had previously participated in similar 

studies with Carroll-Loye Biological Research; since it is unlikely that limb measurements would 

have changed significantly in this period of time for those participants who indicated that they 

had not changed weight appreciably during the interval since the previous measurements, the 

historical measurements would have been accurate. 

The second deviation was the recording of an average start time instead of a precise start time 

for each individual participant in half of the study; because the range of start times was such a 

small fraction of the total protection time, this average would not have affected the results 

substantively. 

2. Improvements to the statistical analysis would enhance the value of the results. 

When the data are heavily censored, as is often the case in arthropod repellent studies, the 

use of the Kaplan-Meier method is problematic. If censored values are replaced by the time of 

censoring, the estimated mean is biased downward, and the standard deviation is underestimated. The downward bias of the mean tends toward a more conservative result, but underestimating the standard deviation may lead one to assume more confidence in the results than 

should be given. It is not appropriate to return to the power analysis conducted prior to the study 

as a foundation for drawing conclusions. 

In its use of the data the Agency should consider the potential use of either maximum 

likelihood methods for estimation of the mean and variance in the presence of heavy censoring 

or estimation of the proportion of the population having protection times of at least a prespecified number of hours should be considered as alternatives to those currently used to analyze 

the data. The suggestions for statistical analysis provided in the June 2008 HSRB report would 

enhance the value of the results. 

Page 20 of 28 
Ethics

Charge to the Board 

Does available information support a determination that study LNX-001 was conducted 

in substantial compliance with subparts K and L 40 CFR Part 26? 

Board Response to the Charge 

HSRB Recommendation

The Board concurred with the Agency’s assessment that the study submitted for review 

was conducted in substantial compliance with subparts K and L of 40 CFR 26. 

HSRB Detailed Recommendations and Rationale

The documents provided by Carroll-Loye (Carroll 2008a, 2008b) state that each study 

was conducted in compliance the requirements of the US EPA Good Laboratory Practice 

Regulations for Pesticide Programs (40 CFR 160). Additional regulations – 40 CFR 26 subparts 

K and L; FIFRA § 12(a)(2)(P); and the California Code of Regulations Title 3, Section 6710 – 

are also applicable. The study was reviewed and approved by a commercial human subjects 

review committee, IIRB, Inc.. Documentation provided to the EPA by IIRB, Inc. indicates that it 

reviewed these studies pursuant to the standards of the Common Rule (45 CFR Part 46, Subpart 

A) and found them in compliance. 

1. The Board concurred with the conclusions and factual observations of the ethical 

strengths and weaknesses of the study, as detailed in the EPA’s Science and Ethics 

Review (Carley 2008). 

2. The Board concluded that this study met all applicable ethical requirements for research 

involving human participants, in accordance to the following criteria: 

a. Acceptable risk-benefit ratio. The risks to study participants were minimized 

appropriately and were justified by the potential societal benefits, particularly data on the 

efficacy of these new formulations as personal insect repellents. 

i. Minors and pregnant or lactating women were excluded from participation, with 

pregnancy confirmed by self-administered pregnancy testing on each “day of 

study”. The potential of stigma resulting from study exclusion was minimized by 

enrolling three ‘alternate’ participants, allowing volunteers to withdraw or be 

excluded without compromising confidentiality. 

ii. Based on toxicological data currently available for picaridin, study participants 

were unlikely to be at risk of adverse side effects with exposure. One 28-year-old 

male study participant reported severe itching around the face and neck during a 

field efficacy trials, but this was a localized reaction distance from the site of 

Page 21 of 28 
repellent application, immediately treated, and later determined by the researchers 

and IIRB, Inc. as likely unrelated to product use. 

iii. The study was designed to minimize the likelihood of insect bites, but when they 

occur are usually mild and readily treated with steroidal creams. The study 

excluded individuals with a history of severe reactions to bites 

iv. Finally, the field-based trials were conducted only in areas where known vectorborne diseases like WNV had not been detected by county and state health or 

vector/mosquito control agencies for at least one month. Mosquitoes collected 

during the field studies also were subjected to molecular analyses to confirm that 

they were free of known pathogens. 

b. Voluntary and informed consent of all participants 

• The study protocol included several mechanisms designed to minimize coercive 

recruitment and enrollment. Monetary compensation was not so high as to unduly 

influence participation. 

• Several minor protocol deviations occurred. As reported to IIRB, Inc. and to the 

Agency, these appear to have been unintentional and are unlikely to have placed 

study participants at risk or to have compromised significantly the informed 

consent process. For example, Carroll-Loye researchers failed to countersign 

promptly two informed consent documents. In both cases, participants 

appropriately were listed as screen failures and then re-screened, re-consented, 

and re-enrolled. 

• Contrary to the HSRB-reviewed (EPA HSRB 2007) and IIRB-approved protocol, 

Carroll-Loye researchers also used previously recorded limb measurements, 

rather than collect physical data from all trial participants. This deviation occurred 

inadvertently when a Carroll-Loye researcher, acting upon an EPA suggestion 

that use of archival limb measurements was scientifically valid and would 

minimize study procedure invasiveness, implemented this protocol change 

without consulting Carroll-Loye management or IIRB, Inc. The likely result of an 

error in communication, this deviation again did not place study participants at 

increased risk or compromise the informed consent process. 

Comments on Guidelines for Product Performance Testing of Skin-Applied Insect 

Repellents

Overview

In order to improve the quality and reliability of data submitted to the Agency, EPA 

issues non-binding guidance documents, referred to as “Test Guidelines,” describing the 

scientific methodology recommended by EPA to develop data required to support applications to 

register pesticide products. EPA’s test guidelines typically contain detailed information about 

Page 22 of 28 
many aspects of the study design – for example, the test material, the use of control groups, the 

nature and number of data points, and the content of study reports. 

The Agency has been working to revise its Product Performance Test Guidelines OPPTS 

810.3700: Insect Repellents for Human Skin and Outdoor Premises since it was published as a 

“public draft” in December 1999. A revised draft addressing only repellents for human skin was 

presented to and discussed by the HSRB in June, 2006. Since then the HSRB has reviewed and 

commented on numerous proposals for insect repellent efficacy studies and several reports of 

completed studies. Over the course of these reviews the Board has made many suggestions for 

strengthening the scientific and ethical conduct of this kind of research, and has encouraged EPA 

to further revise and publish its guidelines for researchers considering this type of study. 

EPA has extensively revised its insect repellent efficacy test guideline in response to 

many helpful suggestions from the HSRB and others. This new guideline of September 23, 2008 

contains many new sections addressing the ethical considerations affecting the design and 

conduct of repellent efficacy studies. EPA has also expanded and revised the sections dealing 

with scientific aspects of this kind of study. 

 EPA will soon announce in the Federal Register the availability of the new draft of the 

guideline. The Agency plans to designate the new version as an “interim guideline” for 

immediate use by investigators, but subject to further refinement in light of future comments 

from the HSRB and the public. 

HSRB Evaluation

While the Board was not provided a charge for consideration on the guidelines, it did 

have several comments to enhance the utility of the document. The Board noted that 

improvements in insect repellent protocols would be derived from portions of the guidelines and 

the document was well written and clear. The Board appreciated the inclusion of landings as well 

as bites as an endpoint. However, as noted previously by the Board, it encourages the Agency to 

consider landings as the preferred endpoint. 

The Board expressed serious concerns that if the erroneous assumptions and inconsistent 

statements regarding the statistical analysis plan are not corrected, investigators following the 

guideline recommendations would develop scientifically unreliable protocols. The Board 

underscored that it would evaluate protocols submitted for review to the HSRB based on 

appropriate statistical assumptions and analytic plans and on that basis might recommend 

rejection of a protocol that followed those elements of the Guidelines that were incorrect. Based 

on continued statistical concerns with the guidelines, the Board strongly recommended that a 

Board working group provide a review of the statistical portion of the guidelines, for subsequent 

consideration by the HSRB, prior to the final guidelines being released by EPA to the public. 

Below are sections relevant to protocol design and statistical analysis that the Board 

recommends should be revised prior to publication: 

Page 23 of 28 
a. On page 9, it was stated that “standard deviations around point estimates of 

relative protection (based on percent reduction in number of bites) are likely to 

follow a Poisson model.” The number of bites may be a Poisson distributed 

random variable, but neither the standard deviations nor the percent reduction in 

number of bites would be anticipated to have that distribution. 

b. Also on page 9, it is stated that the best manner to randomize is by use of a 

random number table. Although this is an acceptable method, it is not necessarily 

best. Random numbers can be generated on a calculator or computer. Some 

programs do this randomization. It would be better to stress the need for 

randomization without dictating the method. 

c. The statistical analysis plan section as presented on page 10 raised a question by 

the Board. If the response is the complete protection time, the distribution is 

unlikely to be normal. Although a transformation is one possible approach of 

accounting for non-normality, generalized linear mixed models could also be 

applied appropriately in this setting and should be included as a potential method 

of analysis. 

d. On page 27, it is first stated that field tests should be conducted in two distinct 

habitats where the predominant mosquito species are different. Then it was noted 

that the study could be “repeated in different locations within the same habitat, or 

in the same location more than once, on different days, using the same subjects to 

minimize variability.” It should be clarified that repeating in the same location 

does not remove the need to use two distinct habitats. 

e. In these studies, a large proportion of the observations are censored, making the 

statistical analysis more challenging, and some guidance on appropriate methods 

should be given. If censored values are replaced by the time of censoring, the 

estimated mean is biased downward, and the standard deviation is underestimated. The downward bias of the mean tends toward a more conservative 

result, but under estimating the standard deviation may lead one to assume more 

confidence in the results than should be given. Thus, it is not appropriate to set a 

confidence interval that uses the time at censoring to represent all censored 

responses, ignoring the censored nature of the data. The use of maximum 

likelihood methods that account for the censoring of the values to obtain estimates 

for the mean and standard deviation is appropriate. These are iterative methods 

that must converge to obtain the estimates of mean and variance. Convergence 

will not be obtained if none of the study participants have a confirmed bite before 

the study’s end. If one or two confirmed bites are observed, convergence may be 

obtained, but the estimates may still be unreliable. An alternative to estimating the 

mean is to estimate the proportion of the population for which the complete 

protection time exceeds the study duration. Some thought should be given to 

exactly what measure is most appropriate; mean protection time is unlikely to be 

the quantity that is of greatest interest. 

Page 24 of 28 
f. “Round Robin designs” were also suggested by the Board as a potential design 

when testing more than one product. The Round Robin designs may have all 

subjects having the same order of treatments, which would not be an appropriate 

design. Replicated Latin squares or similar designs might be useful ones to 

mention by name. 

g. Although differences in location and date should be accounted for in an analysis, 

this should not be referred to as blocking as occurred several times in the 

document. These may be referred to as stratifying variables. If the same subject 

is used in different habitats, locations, dates, or treatments, then the appropriate 

blocking variable is the subject. Because the efficacy for more than one mosquito 

species is to be evaluated, the potential differential effects of the repellant should 

be considered. 

h. Finally, in the field study guidance, it is suggested that the subjects work in pairs. 

Further, the subjects may engage in normal outdoor activities during testing. Is it 

possible for the subjects to engage in normal outdoor activities during testing and 

to still be able to accurately identify landings? Before this is accepted as part of 

the protocol, the effect of such pairings and activities on observation should be 

studied. 

The Board also made recommendations regarding improvement of sections relevant to 

human subjects protections. 

a. To improve the consistency and clarity of terms associated with bioethical principles 

articulated in the Belmont Report, the Agency should consider reframing those portions 

of the document appropriately with three headings: respect for persons, beneficience, and 

justice. 

b. On page 11, the Agency downplays the importance of alternatives to research with 

human subjects by stating, “In general, the efficacy of skin-applied repellents can only be 

tested in research involving human subjects.” The human subjects protection importance 

and scientific value of methods that do not require intentional dosing of human subjects 

with appropriate examples should be equally emphasized. 

c. The Agency should consider placing more emphasis on risk minimization for example: 

To minimize the risks posed by biting arthropods, the Agency may wish to require that 

any proposal to use bites as an endpoint in either field or laboratory studies be justified. 

On page 12, the list of proposed minimization procedures should read “and have a 

physician on call during testing.” Also, the document should set aside and/or emphasize 

the need for stopping rules, monitoring, medical management and post-study follow-up to 

monitor for delayed adverse events, particularly for studies that involved potential 

exposure to arthropod-borne illnesses. 

d. On page 13 “The target population to which the results of repellent testing should ideally 

be as generalizable as possible to the population of repellent users”, the Agency should 

Page 25 of 28 
consider recommending that sponsors include an explanation and justification 

recruitment strategies that will not include appropriate representation of adults of various 

ages, sexes and ethnicities. Convenience sampling, recruiting difficulties, etc. should not 

be considered valid justifications.

e. Consider changing the text on page 14 to read, “If it is proposed, study protocols should 

clearly justify transporting study, participants to distant locations and include a clear 

description of mechanisms to both prevent undue influence (i.e. offers of travel) and 

coercion (i.e. reluctance of participants to withdrawal once they have traveled to a distant 

site). 

f. On page 15, when discussing the issue of language translation, it is important to use 

qualified translators and also ensure translations are at an appropriate reading level. 

g. On page 21, the Agency should be explicit that any unanticipated adverse event should be 

reported, even if it is eventually determined that it is not related to product exposure or 

study participation. 

Clarification of Board Report Process 

Precipitating factor. At the June 2008 public meeting the Board discussed a series of 

recommendations for improving the scientific value of the AHETF closed and open cab studies. 

Prior to the posting of and public meeting to finalize the June 2008 report, the Office of Pesticide 

Programs (OPP) wrote a memo to the Board stating that the Agency had approved the 

implementation of the studies based in part on their interpretation of Board discussion at the June 

meeting and interpretation of the Chair’s minutes. The Board recognized the need for a more 

rapid turn around of its final report to avoid future confusion. The Board clarified its process for 

approving final recommendations and adopted a new report schedule and format. 

Clarification of Board Report Process. Since its inception, the HSRB DFO has stated that 

recommendations of the HSRB are not finalized until formal Board approval of a final report is 

completed at a public face- to- face or teleconference meeting. While Board discussion at each 

meeting serves as an important foundation for final recommendations, no formal consensus is 

taken at the meeting. The Chair’s minutes reflect a summary of meeting discussions, but it is not 

approved by the Board nor is it a substitute for Board final consensus. Board members write the 

report after each meeting and take responsibility for integrating meeting materials and Board 

discussion into a summary of issues and recommendations. The draft report is then posted for 

public review. Additions, deletions, corrections and modifications to the final report are finalized 

at a public meeting that includes further opportunity for public comments and Board discussion, 

review and approval of the document. 

Improving Report Format. At the October 2008 meeting the Board adopted steps to 

complete its reports in a more expeditious period of time while proving greater clarity of its 

recommendations. In its deliberations and subsequent report, whenever possible the Board will 

clarify which recommendations are directed to: (a) modifications that would enhance a 

Page 26 of 28 
scientifically sound design and analysis; (b) limitations that need to be considered in data 

analysis and interpretation of a scientifically sound design and; (c) modifications that are needed 

to ensure that the design and analyses are scientifically sound. The Board also agreed to make 

efforts to adopt similarly clarifying language for its ethics reviews.

Page 27 of 28 
Page 28 of 28 

REFERENCES 

Carley, J.M. 2008. Ethics Review of Completed Carroll-Loye Field Mosquito Repellent Efficacy 

Study LNX-001. Dated September 23, 2008. Unpublished document prepared by Office of 

Pesticide Programs, United States Environmental Protection Agency. 

Carroll, S. 2008a. Carroll, S. (2008) Efficacy Test of KBR 3023 (Picaridin; Icaridin)-based 

Personal Insect Repellents (20% Cream and 20% Spray) with Mosquitoes under Field 

Conditions. MRID 47506401. Dated August 5, 2008. Unpublished study prepared by CarrollLoye Biological Research. 

Carroll, S. 2008b. Supplementary Information for Mosquito Repellent Efficacy Report (Protocol 

LNX-001). Letter to John M. Carley transmitting supplemental correspondence between CarrollLoye Biological Research and Independent Investigations Review Board, Inc., concerning LNX001, with enclosures. Dated September 26, 2008. Unpublished letter prepared by Carroll-Loye 

Biological Research. 

EPA HSRB. 2007. June 27-29, 2007 EPA Human Studies Review Board Meeting Report.