Document ID: EPA-HQ-ORD-2007-0942-0038
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2008-02-08T05:00Z

January 10, 2008 

EPA-HSRB-07-04

George Gray, Ph.D.

Science Advisor

Office of the Science Advisor

1200 Pennsylvania Avenue, NW

Washington, DC 20460 

Subject:  October 24-26, 2007 EPA Human Studies Review Board Meeting
Report

Dear Dr. Gray:

	The United States Environmental Protection Agency (EPA or Agency)
requested the Human Studies Review Board (HSRB) to review scientific and
ethical issues addressing: (1) the EPA Office of Research and
Development’s document, Scientific and Ethical Approaches for
Observational Exposure Studies; (2) a sodium azide study; (3)
Carroll-Loye Biological Research Completed Field Efficacy Studies
(SCI-001 and WPC-001); (4) Carroll-Loye Biological Research Proposed
Insect Repellent Efficacy Protocols (SPC-001 and SPC-002); and (5) ICR
Proposed Repellent Efficacy Protocol (A 117).  The enclosed HSRB report
provides the Board’s response to EPA charge questions presented at the
October 24-26, 2007 meeting.

At previous HSRB meetings, the Board raised a number of science
questions related to mosquito repellent efficacy field research.  At the
Board’s request, three consultants in the field of entomology were
invited to the October 2007 HSRB meeting to discuss the frequency and
duration of exposure of subjects to potential mosquito landings.  Board
discussion of the consultants’ comments are is briefly provided.  This
HSRB report includes Board discussions on the consultants opinions
presented in the consultants’ responses and supplemental information
shared at the meeting.   Finally, the Board also appreciated the Agency
highlighting progress on issues relating to the design of sampling
strategies for handler research programs proposed by the Agricultural
Handlers Exposure Task Force and the Antimicrobials Exposure Assessment
Task Force II.  

On a general note, future protocols should include a statistical
analysis plan for subsequent Board review.  In addition, the Board
requests that revised protocols or subsequent studies submitted to the
Agency should include a response to changes as specified by the Agency. 
Such a written response would assist the Board in its review process. 
Finally, the HSRB requests expert consultants at a future meeting to
educate the Board and Agency on acceptable statistical approaches to
insect repellent studies.

A summary of the Board’s conclusions is provided below.  

Scientific and Ethical Approaches for Observational Exposure Studies

Introduction, Purpose and Scope

The Board concurred that the document is extremely well-written,
presents information accurately and clearly, and will be extremely
useful; however, the Board provided suggestions for improvements.

Expand the discussion on how observational research is distinguished
from intentional exposure research with specific illustrations,
including discussion of the need for ethical review and evaluation of
observational research.

The document presents more ethical than science approaches for
observational exposure studies.  The Board recommends that the document
be revised to present a more balanced presentation and highlight the
interplay between good science and ethics.

The Board also recommended providing examples of study questions, citing
other documents to guide researchers in alternative research designs,
and providing bulleted summary highlights at the end of each chapter.

The abstract states that the document will address chemicals and other
stressors, but focuses on chemicals, which gives the impression that
attention is not needed except for research involving chemicals.  The
language and examples used in the document should reflect the importance
of the document for both chemicals and other stressors.

Elements to Be Considered in Study Conceptualization and Planning

The Board concurred that Section 2 adequately identified the major areas
and issues where ethical considerations should be addressed in the study
conceptualization.  The Board suggested that rather than having distinct
sections and even documents on ethics and science there be just one
document, with the study design elements being a portion of the human
subjects protocol. This would more closely parallel the information
submitted to an IRB.  

Bulleted items should also include justification for sample selection
and size and sampling method, discussion of alternative designs that
were rejected to help justify the use of human subjects in the
particular way being proposed, expanded examples of conflicts of
interest, and the importance of using validated measures. Expansion of
text regarding adverse events vs. unanticipated problems, scientific
misconduct, subject attrition, reporting biases etc. were recommended.

Ensuring Protections of Vulnerable Populations

The Board recommended additional discussion on justifications for
including vulnerable populations in research as well as expansion on
discussion of who is vulnerable outside of the federal regulations (e.g.
pregnant women, prisoners, children) such as economic, educational or
social vulnerabilities; noting however that in federal regulations
vulnerability is defined in terms of susceptibility to coercion and
undue influence.

Expand examples of studies that might involve these populations.

Discuss the tension between over and under sampling these populations
including the importance of including vulnerable populations in research
to ensure that there is safety data on compounds to which they are more
likely than other populations to be exposed.

Discuss the weighing of risks and benefits in these types of studies.

Privacy, Confidentiality, and other Concerns Related to Observational
Exposure Studies

This section of the document is sensitive to many key ethical and legal
considerations relating to the safeguarding of research-subject privacy,
including the need to (1) disclose the possibility of incidental
reporting requirements to potential volunteers as part of the
informed-consent process, (2) provide advance notification of research
visits to third parties who may be residing in private environments, and
(3) consider potential harms to research volunteers who display personal
monitoring equipment in public settings.  It would be strengthened by
including specific advice on reporting and disclosure procedures when
confidential information indicates a participant or another person is in
jeopardy, the uses of the Certificate of Confidentiality and additional
references.

Creating an Appropriate Relationship Between Participant and Researcher

Most of the major areas and issues where ethical considerations should
be addressed were included in this section, with the exception of
communication/language issues.  However, in many cases the section takes
the tone of reporting what others have said, with and without comment;
and sometimes (and more appropriately) stating “it is recommended.” 
A document with clear recommendations will be more useful by researchers
than one that raises issues without giving direction.

OMB and other guidelines for payment/remuneration could be included.

Provide more examples of appropriate participant-researcher
relationships to observational studies.

The role of informed consent and the IRB or other institutional
representative who protects subjects’ rights.

Building and Maintaining Appropriate Community and Stakeholder
Relationships

Many of the major issues requiring ethical considerations were included.
However, more data to support points made and less assertion or
“theory” would strengthen this section.  It is also important to
differentiate the terms stakeholder and community as well as their
interrelationships and discussing the value of community advisory boards
and community sensitive piloting of procedures.

Successful community advisory board procedures, how the
scientist-community relationship will evolve and be monitored over time,
how the results of research are disseminated and the informational
benefits to the community should also be discussed.

The Board also cautioned about editing the section so that readers would
not erroneously conclude that the EPA is advocating that scientists
become community advocates.  

Designing and Implementing Strategies for Effective Communication  

This section had a very comprehensive and informative list of reference
and several suggestions for additional references have been noted.  
While this section is very well-written, it does not clearly focus on
the communication methods most suitable for observational exposure
studies. The use of side bars is a very effective tool to communicate
small bits of information clearly and quickly.  

The goals of the communication should also be included. In this regard,
sections on data sharing and how to address potential
scientist-community disagreements (e.g., interpretation of data) would
be helpful.

The context in which communication occurs (e.g. a participant’s home)
should also be discussed.

The importance of formative (process) evaluation and the importance of
considering in advance how the data might be used should be included.

Science and Ethics of Sodium Azide Study

The HSRB concluded that the Black study does not contain sufficient
information to be used as a point of departure to estimate a safe level
of acute and chronic exposure and is not informative as to whether human
responding confirms estimates based on animal studies. 

Based on lack of documentation in the Black et al. study, the Board was
unable to conclude that there was clear and convincing evidence that the
conduct of the study was fundamentally unethical.  

there was clear and convincing evidence that the study was significantly
deficient relative to the ethical standards prevailing at the time the
research was conducted.

Science Issues in Mosquito Repellent Efficacy Field Research

The Board concluded that more research is needed to determine biases and
adjustments in mosquito repellent efficacy research..

The Board remains unclear of what the mean of the times to the first 5
bites/(landings with intent to bite is measuring.  

The Board concluded that that it would be helpful for HSRB deliberations
if protocols contained rationales for sample size, outcome measures,
number of treatment groups and controls, why a field study is
preferable, why a specific environment was selected, how different
environments differ, and how controls for environmental shifts in
temperature or time of day are determined.  

The Board understands that the need for smaller sample sizes and the
accompanying lack of power must be balanced with subject protection, but
it is also important to understand which variables can be controlled. 
The expertise of control and treated subjects with respect to detecting
mosquito landings must be balanced and the activity of subjects also
should be controlled.

Completed Insect Repellent Efficacy Study (SCI-001) of DEET Formulations

While the Board concluded that the participation of several subjects on
the day after they had been treated with a different test repellent was
not ideal and there were errors (i.e. choice of test limb) in the study,
this did not affect the validity of the results.  

However, with only 33 subjects for 80 data points (excluding the
negative controls), the overlap, some of same subjects for different
test materials, for Sites 1 and 2, and for different dates of experiment
without proper experimental design and control, the Board concluded that
it is impossible to interpret the reported data adequately thus
rendering scientific validity of the results into question.  In
addition, the study may not have been sufficiently sound to estimate
variance within the population variances. 

Thus, the Board concluded that the study was not sufficiently sound,
from a scientific perspective, to be used to assess the repellent
efficacy of the formulations tested against mosquitoes.  

The Board also concluded that the research was conducted in a manner
that failed to meet the applicable requirements of §40 CFR 26, subparts
K and L.  

Completed Insect Repellent Efficacy Study with Oil of Lemon Eucalyptus
(WPC-001)

The Board concluded that despite problems estimating variability and
some of the same design problems found in study SCI-001, the
Carroll-Loye study WPC-001 assessing the repellent efficacy of the
formulation tested was sufficiently sound for the purposes for which it
was intended.

The majority of the Board concurred with the initial assessment of the
Agency that the study submitted for review by the Board meets the
applicable requirements of §40 CFR 26, subparts K and L.

Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies
(SPC-001)

The Board concluded that the protocol to study the efficacy of three
formulations of picaridin for repelling mosquitoes did not provide
sufficient design or a statistical analysis plan that could be evaluated
for its validity or utility.

The Board concurred with the initial assessment of the Agency that the
revised protocol submitted for review by the Board meets the applicable
requirements of §40 CFR 26, subparts K and L.

Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies
(SPC-002)

The Board concluded that the protocol assessing the efficacy of the test
substances for repelling ticks (SPC-002) appears likely to generate
scientifically reliable data, provided that the revisions suggested by
EPA are incorporated.  

The Board underscored that the statistical analysis plan was not
well-laid out and urged EPA to ensure there was a sufficient analytic
plan before the study be conducted

The Board urges EPA to consider the design of newer studies and the
designs already used for existing products to make certain that labels
reflect information of comparative value to consumers.  

The Board concurred with the initial assessment of the Agency that the
revised protocol submitted for review by the Board meets the applicable
requirements of §40 CFR 26, subparts K and L.

Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117)

The Board concluded that proposed research is generally clear and
appropriately designed, with the exception of the appropriate
statistics.  

The Board concurred that if the proposed research is revised consistent
with EPA’s recommendations and the Board’s suggestions, the study
should yield valid data regarding the efficacy of these products in
repelling Culex. 

The Board concurred with the assessment of the Agency that the protocol
ICR A117 submitted for review by the Board, if revised as suggested in
both EPA’s review and by the Board, would meet the applicable
requirements of §40 CFR 26, subparts K and L. 

In conclusion, the EPA HSRB appreciated the opportunity to advise the
Agency on the scientific and ethical aspects of human studies research
and looks forward to future opportunities to continue advising the
Agency in this endeavor. 

Sincerely,

Celia Fisher, Ph.D., Chair

		EPA Human Studies Review BoardNOTICE

This report has been written as part of the activities of the EPA Human
Studies Review Board, a Federal advisory committee providing advice,
information and recommendations on issues related to scientific and
ethical aspects of human subjects research.  This report has not been
reviewed for approval by the Agency and, hence, the contents of this
report do not necessarily represent the view and policies of the
Environmental Protection Agency, nor of other agencies in the Executive
Branch of the Federal government, nor does the mention of trade names or
commercial products constitute a recommendation for use.  Further
information about the EPA Human Studies Review Board can be obtained
from its website at http://www.epa.gov/osa/hsrb/.  Interested persons
are invited to contact Paul Lewis, Designated Federal Officer, via
e-mail at lewis.paul@epa.gov.

	In preparing this document, the Board carefully considered all
information provided and presented by the Agency presenters, as well as
information presented by public commenters.  This document addresses the
information provided and presented within the structure of the charge by
the Agency.

U. S. ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW BOARD
MEMBERS

Chair

Celia B. Fisher, Ph.D., Marie Ward Doty Professor of Psychology,
Director, Center for Ethics Education, Fordham University, Department of
Psychology, Bronx, NY 

Vice Chair

William S. Brimijoin, Ph.D., Chair and Professor, Molecular Pharmacology
and Experimental Therapeutics, Mayo Foundation, Rochester, MN  

Members 

Alicia Carriquiry, Ph.D., Professor, Department of Statistics, Iowa
State University

Snedecor Hall, Ames, IA *

Gary L. Chadwick, PharmD, MPH, CIP, Associate Provost, Director, Office
for Human Subjects Protection, University of Rochester, Rochester, NY 

Janice Chambers, Ph.D., D.A.B.T., William L. Giles Distinguished
Professor, Director, Center for Environmental Health Sciences, College
of Veterinary Medicine, Mississippi State University, Mississippi State,
MS 

Richard Fenske, Ph.D., MPH, Professor, Department of Environmental and
Occupational Health Sciences, University of Washington, Seattle, WA*  

Susan S. Fish, PharmD, MPH, Professor, Biostatistics & Epidemiology,
Boston University School of Public Health, Co-Director, MA in Clinical
Investigation, Boston University School of Medicine, Boston, MA 

Suzanne C. Fitzpatrick, Ph.D., D.A.B.T, Senior Science Policy Analyst,
Office of the Commissioner, Office of Science and Health Coordination,
U.S. Food and Drug Administration, Rockville, MD 

Dallas E. Johnson, Ph.D., Professor Emeritus, Department of Statistics,
Kansas State University, Manhattan, KS

KyungMann Kim, Ph.D., CCRP, Professor & Associate Chair, Department of
Biostatistics & Medical Informatics, School of Medicine and Public
Health, University of Wisconsin-Madison, Madison, WI  

Kannan Krishnan, Ph.D., Professor, Département de santé
environnementale et santé au travail, Faculté de medicine, Université
de Montréal, Montréal, Canada 

Michael D. Lebowitz, Ph.D., FCCP, Professor of Public Health & Medicine.
University of Arizona, Tucson, AZ 

Lois D. Lehman-Mckeeman, Ph.D., Distinguished Research Fellow, Discovery
Toxicology, Bristol-Myers Squibb Company, Princeton, NJ  

Jerry A. Menikoff, M.D., Director, Office of Human Subjects Research,
Office of the Director, National Institutes of Health, Bethesda, MD

Rebecca Parkin, Ph.D., Associate Dean for Research and Public Health
Practice, School of Public Health and Human Services, The George
Washington University, Washington, DC

Sean Philpott, Ph.D., MS Bioethics, Policy and Ethics Director, Global
Campaign for

Microbicides, Program for Appropriate Technology in Health, Washington,
DC

Ernest D. Prentice, Ph.D., Associate Vice Chancellor for Academic
Affairs, University of Nebraska Medical Center, Omaha, NE*

Richard Sharp, Ph.D., Director of Bioethics Research, Department of
Bioethics, Cleveland Clinic, Cleveland, OH

Consultants to the Board

Scientific and Ethical Approaches for Observational Exposure Studies
Session 

	

Germaine Buck-Louis, Ph.D.

Division of Epidemiology, Statistics & Prevention Research

National Institute of Children & Human Development

Rockville, MD  

Barry Ryan, Ph.D.

Department of Environmental and Occupational Health

Rollins School of Public Health 

Emory University

Atlanta, GA 

Frequency, Duration and Timing of Exposure of Subjects to Potential
Mosquito Landings Session 

Col. Raj. Gupta, Ph.D.

Director, Research Plans and Programs

Walter Reed Army Medical Center

Medical Research and Material Command

Fort Detrick, MD 

Steve Schofield, Ph.D.

Department of National Defence

Canadian Forces Health Services Group – HQ Ottawa

Force Health Protection

Communicable Disease Control Program

Ottawa, Ontario, Canada

Daniel Strickman, Ph.D.

USDA, ARS

National Program Leader

Program 104: Veterinary, Medical, and Urban Entomology

National Program Staff - APP

Beltsville, MD 

Human Studies Review Board Staff

Paul I. Lewis, Ph.D., Designated Federal Officer, United States
Environmental Protection Agency, Washington, DC 

* Not in attendance at October 24-26, 2007 Public Meeting

TABLE OF CONTENTS

  TOC \o "1-3" \h \z \u    HYPERLINK \l "_Toc182819601"  U. S.
ENVIRONMENTAL PROTECTION AGENCY HUMAN STUDIES REVIEW BOARD MEMBERS	 
PAGEREF _Toc182819601 \h  9  

  HYPERLINK \l "_Toc182819602"  INTRODUCTION	  PAGEREF _Toc182819602 \h 
13  

  HYPERLINK \l "_Toc182819604"  REVIEW PROCESS	  PAGEREF _Toc182819604
\h  21  

  HYPERLINK \l "_Toc182819605"  CHARGE TO THE BOARD AND BOARD RESPONSE	 
PAGEREF _Toc182819605 \h  23  

  HYPERLINK \l "_Toc182819606"  A.  Scientific and Ethical Approaches
for Observational Exposure Studies	  PAGEREF _Toc182819606 \h  23  

  HYPERLINK \l "_Toc182819609"  B.  Completed Oral Therapeutic Study
with Sodium Azide	  PAGEREF _Toc182819609 \h  2643  

  HYPERLINK \l "_Toc182819610"  C.  Science Issues in Mosquito Repellent
Efficacy Field Research	  PAGEREF _Toc182819610 \h  2647  

  HYPERLINK \l "_Toc182819611"  D.  Completed Insect Repellent Efficacy
Study (SCI-001) of DEET Formulations	  PAGEREF _Toc182819611 \h  2652  

  HYPERLINK \l "_Toc182819612"  E.  Completed Insect Repellent Efficacy
Study with Oil of Lemon Eucalyptus (WPC-001)	  PAGEREF _Toc182819612 \h 
2661  

  HYPERLINK \l "_Toc182819613"  F(1).  Proposed Carroll-Loye Picaridin
Insect Repellent Efficacy Studies (SPC-001)	  PAGEREF _Toc182819613 \h 
2666  

  HYPERLINK \l "_Toc182819614"  F(2).  Proposed Carroll-Loye Picaridin
Insect Repellent Efficacy Studies (SPC-002)	  PAGEREF _Toc182819614 \h 
2671  

  HYPERLINK \l "_Toc182819615"  G.  Proposed ICR Picaridin Insect
Repellent Efficacy Study (A 117)	  PAGEREF _Toc182819615 \h  2676  

  HYPERLINK \l "_Toc182819616"  REFERENCES	  PAGEREF _Toc182819616 \h 
2682  

  HYPERLINK \l "_Toc182819617"  APPENDIX A:  DISCUSSION QUESTIONS FOR
MOSQUITO REPELLENT STUDIES	  PAGEREF _Toc182819617 \h  2684  

 

INTRODUCTION

From October 24-26, 2007, the United States Environmental Protection
Agency’s (EPA or Agency) Human Studies Review Board (HSRB) met to
address scientific and ethical issues concerning:

A.  Scientific and Ethical Approaches for Observational Exposure Studies
  

Scientists at the U.S. Environmental Protection Agency’s (EPA’s),
Office of Research and Development’s National Exposure Research
Laboratory (NERL) have conducted observational exposure measurement
research for several decades to understand how and why people come into
contact with chemicals and other stressors in their everyday lives.
These studies are performed to determine what chemicals people are
exposed to, the concentrations of the chemicals, the most important
sources contributing to people’s exposures, the routes and pathways of
exposure, and the factors that have the biggest impact on exposure. 

EPA strives to follow the most up-to-date approaches in designing and
performing observational exposure studies to ensure that these studies
are based on sound science and meet the highest ethical standards. To
meet that goal, researchers in NERL have prepared a draft document that
identifies key scientific and ethical issues and provides information
and resources to assist researchers as they plan and implement
observational exposure studies. The document is not meant to represent
an official Agency “guidance document.” Moreover, it recognizes that
researchers will work with others – EPA’s Human Subjects Research
Review Official, Institutional Review Board (IRB) members, the
participants and their community, and other stakeholders – to identify
and address all of the relevant issues for any specific study to ensure
that all participants are respected and protected. 

Review material:  EPA provided the following materials to the HSRB
relevant to this topic: 

1. External review draft document titled Scientific and Ethical
Approaches for Observational Exposure Studies

2. Charge Questions 

3. Report on the Workshop to Discuss State-of-the-Science Approaches for
Observational Exposure Measurement Studies, dated January 25, 2007. The
report provides background on the framework for the document and topic
areas recommended by an expert panel. 

B.  Completed Oral Therapeutic Study with Sodium Azide

In its registration program EPA reexamines the safety of pesticides
being proposed for new or amended registration.  The Agency is currently
reviewing an application for registration of the active ingredient,
sodium azide (NaN3), as a limited replacement for the fumigant, methyl
bromide.  The application seeks to register sodium azide for commercial
production of ornamental cut flowers and pre-plant application via drip
tape irrigation on beds under plastic mulch; for sod farms with
pre-plant application to soil with tarping after application; and for
golf course turf area renovation with pre-plant application and
immediate tarping. 

Sodium azide also has been used for many years as a laboratory reagent
and as a raw material for production of azide-containing compounds.  It
has been used as a pharmaceutical intermediate and as a preservative of
blood, laboratory reagents, and biological fluids.  It has been used as
a gas generant in automotive airbags, and was commonly used in early
inflator designs.  During the 1990s, however, airbag propellants
containing NaN3 were phased out in favor of more efficient, less
expensive and less toxic alternatives.  In the past, NaN3 was also used
as a pharmaceutical to treat high blood pressure and as an
anti-neoplastic agent.

EPA has identified a study published in 1954 in which human subjects
received oral doses of sodium azide to assess its potential for lowering
blood pressure.  The Agency intends to use this study in its hazard
assessment to derive a “point of departure” (POD) for assessing
acute and chronic toxicity resulting from both acute and chronic
exposures to this chemical.  

The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB
review of an EPA decision to rely on the results of any study if the
research was “initiated before April 7, 2006, and the research was
conducted for the purpose of identifying or measuring a toxic effect.”
 EPA has reviewed the study, applying the standards in 40 CFR
§§26.1703 and 26.1704.  Those provisions state:

§26.1703 Prohibition of reliance on research involving intentional
exposure of human subjects who are pregnant women (and therefore their
fetuses), nursing women, or children.

Except as provided in §26.1706, in actions within the scope of
§26.1701 EPA shall not rely on data from any research involving
intentional exposure of any human subject who is a pregnant woman (and
therefore her fetus), a nursing woman, or a child.

§26.1704 Prohibition on reliance on unethical research with
non-pregnant, non-nursing adults conducted before April 7, 2006

Except as provided in §26.1706, in actions within the scope of
§26.1701, EPA shall not rely on data from any research initiated before
April 7, 2006, if there is clear and convincing evidence that the
conduct of the research was fundamentally unethical (e.g., the research
was intended to seriously harm participants or failed to obtain informed
consent), or was significantly deficient relative to the ethical
standards prevailing at the time the research was conducted.  This
prohibition is in addition to the prohibition in §26.1703.

The Agency’s reviews concluded that the data were scientifically sound
and that there was no clear and convincing evidence that the conduct of
the research was fundamentally unethical or significantly deficient
relative to the ethical standards prevailing at the time the research
was conducted.   Nor was there evidence to show that the subjects
included nursing or pregnant women or children.

Review materials. EPA provided the following materials to the HSRB
relevant to the 

completed oral therapeutic study with sodium azide:

MRID 47221401 Black et al 1954

Black, M.; Zweifach, B.; Speer, F. (1954) “Comparison of Hypotensive
Action of Sodium Azide in Normotensive and Hypertensive Patients.”  In
Proceedings of the Society for Experimental Biology and Medicine, Jan
1954, pp. 11-16.  MRID 47221401. 

MRID 47221401 Data Evaluation Record 

EPA WOE Sodium Azide 9-18-07

Memorandum from Nancy McCarroll to Jack Housenger, Associate Director
Health Effects Division, “Human Studies Review Board: Weight of
Evidence Discussion for Sodium azide (NaN3).” September 18, 2007.

EPA Ethics Review MRID 47221401 9-27-07

C.  Science Issues in Mosquito Repellent Efficacy Field Research

Currently, EPA requires all pesticide products that claim to repel
mosquitoes to provide data on the duration of efficacy under field
conditions at two biologically distinct sites. These data are derived
from human research with subjects who have been treated with the
repellent formulations in the field. The Agency evaluates the duration
of repellent efficacy for a subject by calculating the time from
application of the repellent to the occurrence of an event indicating an
efficacy failure. Historically, for field studies of mosquito
repellency, EPA has used the “first confirmed bite” as an indication
of efficacy failure on a test subject. Several recent studies have
shifted to the “first confirmed landing with intent to bite;” EPA
has accepted this alternative endpoint. A “confirmed landing” on a
test subject is a mosquito landing followed by a second landing on the
same subject within a specified period of time (usually 30 minutes)
after the initial landing. 

Field studies typically involve 6 – 10 subjects who have been treated
with a defined amount of the test material. Each subject is then
regularly and repeatedly exposed to ambient mosquito populations for a
fixed interval of time until the subject experiences an efficacy failure
followed by a confirmation with the specified period of time. Mosquito
landing pressure (representing intent to bite) at a site is monitored by
concurrently exposing untreated subjects to mosquito landings. A study
is considered valid only if there are at least a specified minimum
number of mosquito landings on untreated subjects during each exposure
interval. 

On October 25, 2007, the HSRB discussed scientific aspects of the design
of field studies to assess the efficacy of mosquito repellents. Prior to
the meeting, the Board requested consultants to provide specialized
information or assistance to the Board.  The Board was particularly
interested in the frequency, duration and timing of exposure of subjects
to potential mosquito landings. The Board requested each consultant to
respond briefly to the series of questions below.  

• What do data show about the variability of the time intervals
between first and subsequent landings in mosquito repellent field
trials? 

• What is the current scientific understanding of how factors other
than repellent efficacy could affect the likelihood that an initial
event—a mosquito landing or mosquito bite—would be “confirmed”
by another similar event within 30 minutes? Please address at least
these factors: 

o Characteristics of mosquito populations 

o Characteristics of test sites 

o Characteristics of test subjects 

o Characteristics of test methods 

• Can the impact of such factors on the likelihood or timing of an
initial and confirming event be predicted? Can it be quantified? 

Review materials. EPA provided the following materials to the HSRB
relevant to the science issues in mosquito repellent efficacy field
research:

Consultant Responses to Discussion Questions

1.  Col. Raj Gupta’s Responses to Discussion Questions 

2.  Dr. Steve Schofield's Responses to Discussion Questions  

3.  Dr. Daniel Strickman’s Responses to Discussion Questions  

D.  Completed Insect Repellent Efficacy Study (SCI-001) of DEET
Formulations 

In its January 2007 meeting, the HSRB reviewed and commented on
materials related to a comparative insect repellent efficacy protocol
from Carroll-Loye Biological Research, submitted by Dr. Scott Carroll. 
The proposal, identified as SCI-001, described a study to evaluate the
efficacy of four repellent formulations containing the active ingredient
DEET.  (Note:  One formulation included two other active ingredients as
well.)  The study was designed to measure the efficacy against
mosquitoes under field conditions of three test formulations as compared
to one “comparison article”—the US military standard repellent. 
The HSRB offered comments on the protocol at its January 2007 meeting. 
Following that meeting, Dr. Carroll revised the protocol to address
comments from the HSRB.  Dr. Carroll conducted the research in July
2007, and has submitted the results to EPA for review.  EPA presented
the results of this testing at the October 2007 HSRB meeting.

Although the protocol SCI-001 was executed only once, the results are
presented in three separate volumes, each one addressing a single test
formulation as compared to the military standard repellent.  Most of the
material presented in each report is duplicated in the other two
reports, but there are unique elements in each volume.  

The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB
review of an EPA decision to rely on the results of these studies.  The
sponsor has submitted applications for amendment of two of the test
materials citing these data, but the third test material (LipoDEET 3434)
is not registered, nor is it the subject of any application.  EPA has
reviewed the research, applying the standard in 40 CFR §26.1705.  That
provision states:

§ 26.1705 Prohibition on reliance on unethical research with
non-pregnant, non-nursing adults conducted after April 7, 2006

Except as provided in §26.1706, in actions within the scope of
§26.1701, EPA shall not rely on data from any research initiated after
April 7, 2006, unless EPA has adequate information to determine that the
research was conducted in substantial compliance with subparts A through
L of this part . . . This prohibition is in addition to the prohibition
in §26.1703.

	Dr. Carroll conducted the research covered by SCI-001 at the same times
and at the same locations as the research covered by protocol WPC-001,
described below.  Because these two protocols were executed
concurrently, in the same field locations, with the same untreated
controls, and with overlapping sets of treated subjects, EPA believed
that the conduct of WPC-001 study may affect the results of SCI-001, and
vice versa.   Thus EPA conducted a single ethics review addressing both
studies.  

The Agency’s science review raised questions about whether the data
are scientifically sound.  In addition, EPA’s ethics review raised
questions about whether the research under SCI-001 was conducted in
substantial compliance with the requirements of subparts K and L of
EPA’s final rule establishing Protections for Subjects in Human
Research—the only subparts of the rule which apply to third-party
research.  EPA requested the Board’s advice on whether the research
data are scientifically sound and whether the available information
supports a determination of “substantial compliance” with the
applicable rules.  If the Board concluded that the data are
scientifically sound and the research substantially complied with the
applicable requirements, EPA would rely on these data in support of
applications for new or amended registration of the test materials. 

Review materials. EPA provided the following materials to the HSRB
relevant to the completed repellent efficacy study with four DEET
formulations (SCI-001):  

a. 	EPA Ethics Rvw SCI-001 & WPC-001 9-26-07

This review addresses both this study and the concurrently conducted
WPC-001.

b. 	MRID 47211901 SCI-001.1 LipoDEET 302

Carroll, S. (2007) Test of Dermaegis LipoDEET 302 Personal Insect
Repellent: EPA Reg. #82810-1.  Unpublished study prepared by
Carroll-Loye Biological Research under Project No. SCI-001.1.  219 p.  

c.	MRID 47208401 SCI-001.2 LipoDEET 3434

Carroll, S. (2007) Test of Dermaegis LipoDEET 3434 Personal Insect
Repellent.  Unpublished study prepared by Carroll-Loye Biological
Research under Project No. SCI-001.2.  222 p.  

d. 	MRID 47211801 SCI-001.3 Coulston’s Duranon

Carroll, S. (2007) Test of Coulston’s Duranon Personal Insect
Repellent (EPA Reg. #50404-8).  Unpublished study prepared by
Carroll-Loye Biological Research under Project No. SCI-001.3.  217 p.  

e. 	CLBR Supplement Re LipoDEET 3434

Carroll-Loye Biological Research’s September 24, 2007 response to
EPA’s request for additional information about LipoDEET-3434 and the
rationale for the amendment by which it became one of the test
repellents

f. 	EPA Protocol Review SCI-001 12-20-06

g. 	4-16-07 HSRB Report of Jan 07 discussion of SCI-001

h. 	SCI-001 Science Review 9-27-07

E.  Completed Insect Repellent Efficacy Study with Oil of Lemon
Eucalyptus (WPC-001)

In the June 2007 HSRB meeting, the Board reviewed and commented on
materials relating to an insect repellent efficacy protocol from
Carroll-Loye Biological Research, submitted by Dr. Scott Carroll.  The
protocol described proposed research to evaluate the efficacy of a
conditionally registered repellent product containing the active
ingredient Oil of Lemon Eucalyptus (OLE).  The protocol, identified as
WPC-001, described a field study of efficacy of the test formulation
against mosquitoes.  

Following the June meeting, Dr. Carroll revised the protocols to address
comments from the HSRB, conducted the study, and submitted the results. 

The Agency’s regulation, 40 CFR §26.1602, requires EPA to seek HSRB
review of an EPA decision to rely on the results of these studies.  EPA
has reviewed the study, applying the standard in 40 CFR §26.1705.  That
provision states:

§ 26.1705  Prohibition on reliance on unethical research with
non-pregnant, non-nursing adults conducted after April 7, 2006

Except as provided in §26.1706, in actions within the scope of
§26.1701, EPA shall not rely on data from any research initiated after
April 7, 2006, unless EPA has adequate information to determine that the
research was conducted in substantial compliance with subparts A through
L of this part . . . This prohibition is in addition to the prohibition
in §26.1703.

	As noted above, the principal investigator conducted the research
covered by WPC-001 at the same times and at the same locations as the
research covered by protocol SCI-001.  Because these two protocols were
executed together, EPA questioned whether and how this fact affects the
review of the separate reports.  

The Agency’s science review raised questions about whether the data
were scientifically sound.  In addition, depending on whether the study
covered by WPC-001 was considered separate from the study covered by
SCI-001, EPA’s ethics review (discussed above under the heading of
SCI-001) raised a question about whether the research under WPC-001 was
conducted in a manner that substantially complies the requirements of
subparts K and L of EPA’s final rule establishing Protections for
Subjects in Human Research—the only subparts of the rule which apply
to third-party research.  The Agency requested the Board’s advice on
whether the research was scientifically sound and whether the available
information supports a determination of “substantial compliance”
with the applicable rules.  If the Board concluded that the data are
scientifically sound and the research substantially complied with the
applicable requirements, EPA would rely on these data to satisfy the
data requirement imposed as part of the conditional registration of this
product.  

Review materials. EPA provided the following materials to the HSRB
relevant to the 

completed repellent efficacy studies of  oil of eucalyptus:

MRID 47217601 WPC-001 OLE

Carroll, S. (2007) Test of an Oil of Lemon Eucalyptus-Based Personal
Insect Repellent: EPA Reg. #305-62.  Unpublished study prepared by
Carroll-Loye Biological Research under Project No. WPC-001.  225 p.  

b. 	CLBR Supplement Re: Consent Documentation

Carroll-Loye Biological Research’s September 20, 2007 response to
EPA’s request for additional information concerning which subjects
signed which version(s) of the consent document on what date(s)

c.	EPA Protocol Review WPC-001 3-13-07

d.	6-13-07 HSRB Report of Apr 07 discussion of WPC-001

e.	WPC-001 Science Review 9-27-07

F.  Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies
(SPC-001 & SPC-002)

EPA requires data from efficacy studies using appropriate insect species
to support claims of greater efficacy than have previously been
approved.  

EPA’s regulation, 40 CFR §26.1125, requires the sponsor or
investigator to submit to EPA, before conducting a study involving
intentional exposure of human subjects, materials describing the
proposed human research in order to allow EPA to conduct scientific and
ethics reviews.  In addition, EPA’s regulation, 40 CFR §26.1601,
requires EPA to seek HSRB review of the research proposal.  

In previous meetings the HSRB has reviewed and commented favorably on
several proposed insect repellent efficacy protocols to be conducted by
Carroll-Loye Biological Research, submitted by Dr. Scott Carroll.  Dr.
Carroll has submitted proposals for new research to evaluate the
efficacy of two registered repellent sprays containing the active
ingredient picaridin, as well as one lotion formulation including both
picaridin and a sunscreen, for which an application for registration is
pending.  The first research protocol, identified as SPC-001, describes
a field study of the efficacy of the test formulations against
mosquitoes.  The second research protocol, identified as SPC-002,
describes a laboratory study of the efficacy of the test formulations
against ticks.  Both proposals bear many similarities to protocols that
the HSRB had previously reviewed favorably.  

EPA has concluded that, with some refinements, these protocols appear
likely to generate scientifically sound, useful information and to meet
the applicable provisions of the EPA regulations in 40 CFR part 26,
subparts K and L.  

Review materials. EPA provided the following materials to the HSRB
relevant to the 

proposed Carroll-Loye picaridin insect repellent efficacy studies
(SPC-001 and SPC-002):

IIRB Minutes 7-17-2007  

This single document addresses IIRB review of both protocols.

SPC-001: Field test of mosquito repellency

Carroll-Loye Protocol SPC-001 7-13-07

EPA Science & Ethics Review SPC-001 9-24-07

SPC-002: Laboratory test of tick repellency

Carroll-Loye Protocol SPC-002 7-10-07

EPA Science & Ethics Review SPC-002 9-24-07

G.  Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117)

EPA requires data from efficacy studies with human subjects to support
claims of efficacy of a new pesticide product intended to repel insects
that transmit human diseases.  

EPA’s regulation, 40 CFR §26.1125, requires the sponsor or
investigator to submit to EPA, before conducting a study involving
intentional exposure of human subjects, materials describing the
proposed human research in order to allow EPA to conduct scientific and
ethics reviews.  In addition, EPA’s regulation, 40 CFR §26.1601,
requires EPA to seek HSRB review of the research proposal.  

Dr. Niketas Spero has submitted a proposal for new research to evaluate
the efficacy of two registered products containing picaridin, to be
conducted by Insect Control & Research, Inc. (ICR).  The research
protocol, identified by Protocol ID G0590607001A117 describes a
laboratory study of the efficacy of the test formulations against
mosquitoes of the genus Culex. 

EPA has reviewed ICR’s protocol and has concluded that, with a number
of required revisions, it appears likely to generate scientifically
sound, useful information and to meet the applicable provisions of the
EPA regulations in 40 CFR part 26, subparts K and L.  The sponsor wishes
to submit the data to EPA later this year in support of an application
to amend the registration of these picaridin products in order to claim
specifically that the products are effective at repelling the mosquito
species that transmit West Nile Virus.  In the interest of providing a
thorough and timely decision on such applications, and since EPA found
the protocol can meet applicable scientific and ethical standards, EPA
presented this protocol for review at the October 2007 HSRB meeting.

Review materials. EPA provided the following materials to the HSRB
relevant to the 

Insect Control & Research Inc. Repellent Efficacy Protocol A117:

a.	ICR Protocol A117 Transmittal 8-8-07

b.	ICR Protocol A117 8-8-07

This protocol proposes a laboratory test of repellency of Culex spp.
mosquitoes by two formulations containing picaridin

c.	EPA Science & Ethics Review  ICR A117 9-24-07

  This report transmits the HSRB’s comments and recommendations from
its October 24-26, 2007 meeting.        

REVIEW PROCESS

From October 24-26, 2007, the Board had a public face-to-face meeting in
Arlington, Virginia.  Advance notice of the meeting was published in the
Federal Register “Human Studies Review Board: Notice of Public Meeting
(72 Federal Register 187, 54908).  At the public meeting, following
welcoming remarks from Agency officials the Board then heard
presentations from the Agency on the following topics: 

EPA’s draft document Scientific and Ethical Approaches for
Observational Exposure Studies.  The document, prepared by researchers
in EPA’s National Exposure Research Laboratory, identifies the types
of issues that should be considered in planning and implementing
observational human exposure studies and provides information and
resources to assist EPA researchers in these studies.

A published report of a completed clinical trial measuring the effects
of single and repeated treatments with sodium azide on blood pressure in
human subjects.  Sodium azide is a pesticidally active ingredient being
proposed as a replacement for the fumigant methyl bromide.

An overview of the discussion questions related to the Science Issues in
Mosquito Repellent Efficacy Field Research. 

A research proposal from Carroll-Loye Biological Research to evaluate
the field efficacy in repelling mosquitoes of three registered products
containing picaridin.  

A research proposal from Carroll-Loye Biological Research to evaluate
the laboratory efficacy in repelling ticks of three registered products
containing picaridin.  

A research proposal from Insect Control & Research, Inc. to evaluate the
laboratory efficacy in repelling mosquitoes of the genus Culex of two
registered products containing picaridin.

A report of a completed field study by Carroll-Loye Biological Research
of the mosquito repellent efficacy of a registered product containing
Oil of Lemon Eucalyptus.

Three closely related product-specific reports from a single completed
field study by Carroll-Loye Biological Research of the mosquito
repellent efficacy of four pesticides, all containing DEET.  

Design of sampling strategies for handler research programs proposed by
the Agricultural Handlers Exposure Task Force and the Antimicrobials
Exposure Assessment Task Force II.  

 The following oral comments were presented at the meeting: 

 

Judith Hauswirth, Ph.D., and Mr. Douglas Richards  - representing
American Pacific Corporation and addressing the Completed Oral
Therapeutic Study with Sodium Azide.

Thomas Osimitz, Ph.D., and M. Keith Kennedy, Ph.D., - representing
Science Strategies and addressing the Science Issues in Mosquito
Repellent Efficacy Field Research.

Scott Carroll, Ph.D. - representing Carroll-Loye Biological Research and
addressing: (1)  Science Issues in Mosquito Repellent Efficacy Field
Research; (2) Completed Field Efficacy Studies by Carroll-Loye
Biological Research: SCI-001 and WPC-001; and (3) Proposed Insect
Repellent Efficacy Studies SPC-001 and SPC 002. 

Mr. Niketas Spero and Robin Todd, Ph.D. - representing ICR, Inc. and
addressing the ICR Repellency Efficacy Protocol A117.

	For their deliberations, the Board considered the materials presented
at the meeting, written public comments and Agency background documents
(e.g., the published literature, Agency data evaluation record, weight
of evidence review, ethics review, pesticide human study protocols and
Agency evaluation of the protocol).   For a comprehensive list of
background documents visit the www.regulations.gov, Docket ID No. 
EPA-HQ-ORD-2007-0942, or EPA’s HSRB website at   HYPERLINK
"http://www.epa.gov/osa/hsrb/oct-24-26-2007-public-meeting.htm" 
http://www.epa.gov/osa/hsrb/oct-24-26-2007-public-meeting.htm .

CHARGE TO THE BOARD AND BOARD RESPONSE

A.  Scientific and Ethical Approaches for Observational Exposure Studies
  

Charge to the Board

1.   One of the goals of the document is identify the major scientific
and ethical areas and issues that researchers should address in the
design and implementation of observational human exposure measurement
studies, with the emphasis on the areas requiring ethical
considerations.  Does each section identify the major areas and issues
where ethical considerations should be addressed? 

2.   The document is intended to serve as a reference and resource of
information that researchers can use in the design and implementation of
observational exposure studies. For each section, are there additional
sources of information that should be considered for inclusion? 

3.   Is the information presented accurately and clearly in each
section? 

Board Response

Board discussion focused on responding to the three charge questions
together by section (Section 1 to 7) of the EPA draft document.  

Section 1:  Introduction, Purpose and Scope 

Strengths

The document received nearly universal praise from the Board for the
clarity of its writing and its thoughtful consideration of the many
issues involved in observational studies of toxicant exposure in human
subjects.  The introduction set the tone for this excellent document. 
In particular, the Board found that it did a superb job of explaining
the purpose of the document and defining its scope. It gave attention to
the proper goals of such studies and the nature of the data to be
collected. Most importantly, it outlined the range of ethical issues in
the conduct of such studies.  And finally, it accomplished exactly what
an introduction must do by clearly indicating the organization and
general contents of the several sections to follow in the main body of
the document.

Document Enhancement

	The document can be enhanced by:

Expanding the discussion on how observational research is distinguished
from intentional exposure research with specific illustrations,
including discussion of the need for ethical review and evaluation of
observational research.

Revising the document to present a more balanced presentation and
highlight the interplay between good science and ethics.

Providing examples of study questions, citing other documents to guide
researchers in alternative research designs, and providing bulleted
summary highlights at the end of each chapter.

HSRB Consensus and Rationale

	Section 1 is well written and sets the tone for ethical consideration
of observational exposure studies. The document could be enhanced by
examples distinguishing observational from intentional exposure studies,
introducing the appropriate balance between scientific and ethical
concerns, and providing study questions and bullets.

The abstract states that the document will address chemicals and other
stressors, but focuses on chemicals, which gives the impression that
attention is not needed except for research involving chemicals.  The
language and examples used in the document should reflect the importance
of the document for both chemicals and other stressors.  Finally, while
the Agency does define and differentiate observational versus
intentional exposure, such a definition should be more prominently
displayed in the body of the report versus as a footnote.  

Section 2:  Elements to be Considered in Study Conceptualization and
Planning

Strengths

Section 2 is very strong in its consideration of the overall
conceptualization of study planning, especially the ethical component
often insufficiently conceptualized by the scientists in their initial
approach to the study.  The planning and scoping of the study
conceptualization includes both the science and the ethics, as does the
review process (in which each component reviews the other component as
well).  The strong emphasis on the ethical issues is beneficial. 
Recognizing some of the needs and scope of exposure sciences is
noteworthy, as well. The text boxes are particularly helpful, as they
summarize many of the important points from the text quite succinctly.

Document enhancements

	The Board made many constructive comments with the intent of making a
good document better.  As with other chapters, Section 2 has a strong
emphasis on ethics, to the apparent lack of emphasis on scientific
aspects.  In addition, a primary concern is that separating the study
design document from the human subjects protocol will potentially lead
to inconsistencies between the documents (Figure 2-1).  The Board
suggests that there be just one document, with the study design elements
being a portion of the human subjects protocol.  Science cannot easily
be separated from ethics, so any single document should contain elements
of both study design and implementation, as well as other elements that
may be required for an IRB to assess whether all regulatory requirements
have been met.

The major area of deficiency, given the purpose of the document, is in
the paucity of information, materials, and references regarding the
purpose, design and conduct of exposure studies.  Even the initial
paragraphs place more emphasis on the ethical issues (covered
extensively elsewhere) than on the scientific ones; there should be more
of a balance. In addition there are contradictions starting in Section
1.1 as to whether epidemiologic studies are included or not.

Specific Suggestions to the Document

There is little explanation or coverage of exposure study designs and
methods, and their different attributes.  Well-recognized study designs
such as cross-sectional, case-control, and cohort (both prospective and
retrospective) designs should be fully described.  Sufficient examples
are not provided and references to such discussions, as well as to such
studies, are inadequate.  There are excellent sources for such material
to guide researchers, starting with EPA documents (e.g., actual examples
and references for TEAM, PTEAM, NHEXAS, the pesticide studies, etc.). 
There are also excellent NRC/NAS & WHO documents that could be utilized.
 There are many lessons learned from such that could be culled to
provide research guidance.  Some of these sources include the NRC/NAS
1991 report on Human Exposure Assessment for Airborne Pollutants, the
WHO EHC 27 (1983) report on Guidelines on Studies in Environmental
Epidemiology, and the WHO/EURO & EC/EU documents on Exposure Assessment
from the ECEHs & EU Lab.  Additional suggestions are provided below. 

1.  Section 2.1.1 does not define the types of study problems and
questions scientifically (or specific EPA & other references where such
can be found).

2.  Section 2.1.2 does not provide any basis or criteria for
justification of the science component (or specific EPA & other
references where such can be found), only the ethical component.

3.  In Text Box 2-1 Elements to be Considered in Justifying a Study,
there should be a bullet added “A discussion of alternative designs,
alternative models, or alternative populations”

4.  Page 21, line 4: “considers” should be changed to
“considered”

5.  Section 2.2 does not outline the steps in planning the study
scientifically (or specific EPA & other references where such can be
found).  Section 2.2.1 has only one sentence about the scientific
aspects, and does not really discuss “innovative” scientific
aspects.  Text Box 2-2, Study Elements that Could Affect People’s
Behavior, is the closest the section comes to delineating the components
of the study that are relevant also to its planning.

6.  Section 2.2 Planning and Scoping:  The latter term is jargon and can
be eliminated, or if not, it should at least be defined. 

7.  Section 2.2.1 is entitled Innovative Study Designs, but is actually
about adding direct benefit (such as educational materials) for research
subjects in studies for which there is not direct benefit. Innovative
designs might instead include computer modeling or Bayesian designs.
Either the section title or the content needs to be changed.  In
addition, the section’s current content could be said more directly
and succinctly.

8.  Page 23, line 42 “…community’s perspective” . Sentence
should read, “…community’s perspective better, the
researcher…”

9.  Section 2.2.4 states that conflicts due to project funding are
“the most likely to occur.” Please verify that this is the case in
EPA related to observational exposure studies, or change the sentence.
Unless this type of research is unique, it is likely that financial
conflicts are not necessarily the most common, but rather they are the
easiest to identify and manage.

10.  Text Box 2-3 (Elements That May Be Included in a Study Design) and
its corresponding section list elements to be included in a study
design. One bullet lists items to describe technical approach and
conceptual model. The Board recommended adding to the list that endpoint
or outcome measures should include a description of their accuracy and
precision. Survey instruments and questionnaires should include a
description of whether they have been previously validated and, if not,
how they will be validated prior to use within a study. A source for
additional elements that might be included in a study description can be
found at PLoS Medicine, http://medicine.plosjournals.org, volume 4,
number 10, October 2007 in two articles concerning STROBE:   HYPERLINK
"http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.
1371%2Fjournal.pmed.0040296" \o "Read Open Access Article"  The
Strengthening the Reporting of Observational Studies in Epidemiology
(STROBE) Statement: Guidelines for Reporting Observational Studies  (von
Elm E et al., 2007) and   HYPERLINK
"http://medicine.plosjournals.org/perlserv/?request=get-document&doi=10.
1371%2Fjournal.pmed.0040297" \o "Read Open Access Article" 
Strengthening the Reporting of Observational Studies in Epidemiology
(STROBE): Explanation and Elaboration  (Vandenbroucke et al., 2007)

11.  Section 2.3 did not provide some of the more important specifics of
the scope and technical approaches (or specific EPA and other examples
and references where such can be found). Sec. 2.3.1 does not include all
the important  appropriate questions regarding scientific feasibility
– its one “bullet” does not even include questions about the
feasibility of measurement methods.   The sub-sections also did not
include monitoring of observer errors and biases, participant reporting
biases and reliability, inappropriate (as well as inadequate) selection
criteria, representativeness (refusals and withdrawals), etc. (or
specific EPA and other examples and references where such can be found).
 Enrollment criteria (inclusion and exclusion) need to be included, and
a discussion of the ethical issues of subject selection should be added.

12.  Section 2.3.1.1 Sample Size Determination:  The authors are to be
commended for including this section.  However, the section should be
written in consultation with a biostatistician. There are, in fact, many
references on this topic, and to state that there is “surprisingly
small amount of literature” is inaccurate.  In addition, the
methodology cited may not be optimal. One needs to quote and reference
statistical specifics as provided in the documents previously mentioned,
specific survey statistical textbook questions approaches, and also more
from other statistics books), and the issue of design factors (e.g.
Clickner book and other books) as well as expected refusals and expected
losses that need to be taken into account.  Issues of intra- and
inter-participant and observer variability are not discussed.  

13.  A section on obtaining an appropriate sample or a representative
sample in order to derive generalizable data should be written. 
Descriptions of sampling methods with their relevant strengths and
weaknesses are critical, and would help investigators enormously.

14.  Text Box 2-5, Potential Topics in a Human Subjects Research
Protocol, lists potential topics for a submission to an IRB.  

Item #6 should be changed to “Affirmation of Belmont Principles…”
rather than Belmont Report.  

Item #13 should be changed to “Sample size/power and statistical
analysis plan.”  

Item #22 should either be changed to include “unanticipated
problems” or a separate item should be added.  

Item #32 should also include a comment concerning unforeseen uses, if
appropriate.  

Item #42 should be changed to state “Procedures for preventing
falsification of data” with the emphasis on prevention rather than on
what to do if falsification occurs. 

HSRB Consensus and Rationale

The Board concurred that Section 2 adequately identified the major areas
and issues where ethical considerations should be addressed in the study
conceptualization.  The Board suggested that rather than having distinct
sections and even documents on ethics and science there be just one
document, with the study design elements being a portion of the human
subjects protocol. This would more closely parallel the information
submitted to an IRB.  Bulleted items should also include justification
for sample selection and size and sampling method, discussion of
alternative designs that were rejected to help justify the use of human
subjects in the particular way being proposed, expanded examples of
conflicts of interest, and the importance of using validated measures.
Expansion of text regarding adverse events vs. unanticipated problems,
scientific misconduct, subject attrition, reporting biases etc were
recommended.

Section 3:  Ensuring Protection of Vulnerable Groups

Strengths

Section 3 is of high quality and does a very good job in addressing the
major areas and issues related to vulnerable subjects. The section
presently devotes the bulk of the discussion to issues relating to
children and women.  It might be helpful to present a somewhat more
balanced discussion, with added material devoted to discussing issues
relating to other vulnerable groups.

Specific Suggestions to the Document

1.  Section 3.1, Identification of Vulnerable Groups, it might be
helpful to more specifically point out the differences between the rules
under which the EPA is operating with regard to identification of
vulnerable groups (primarily, the Common Rule), as compared to the
concepts of vulnerability that the lay public might have.  In addition,
a discussion of how vulnerability can be context-dependent could be
helpful.

2.  In Section 3.2, the current discussion puts an emphasis on the
“special justification” needed for inviting vulnerable subjects to
participate in research, without expanding on how that concept plays out
in observational studies.  In fact, many of the observational studies
with regard to which this document will apply are studies that are
intended to collect important health information regarding the
well-being of various vulnerable groups (such as children and pregnant
women).  Moreover, unlike many other studies (such as interventional
studies), observational studies generally do not impose more than
minimal risks on subjects.  Those circumstances—the possible large
benefits to the vulnerable populations from gaining the information to
be learned, and the minimal risk regarding including them in the studies
–combine to often make a strong ethical case for including such groups
in these studies.  Thus, it would be appropriate for this section to
also discuss the harms from inappropriately excluding various groups of
vulnerable subjects from this category of studies, and how those harms
would often be to those very groups.  This point is already made in
Section 3 in some of the discussions of specific subject groups (e.g.,
in the discussion in Section 3.4 with regard to children), but it would
be helpful to highlight it as a general proposition in the analysis of
the general issue of the justification for including vulnerable
subjects.

3.  It might be helpful to have a more expanded discussion of how to
mitigate risks to vulnerable subjects in these studies.  This point is
particularly important given the substantial reasons (as noted in item 2
above, and already discussed in portions of Section 3) for including
such vulnerable subjects in these studies.  This discussion could go
beyond the risks created or imposed by the inclusion of the subjects in
the study (which are often minimal), and also address the extent to
which it is appropriate for researchers to be standing by and observing
while vulnerable groups are exposed to risks not created by the study. 

4.  It could be helpful to add a subsection dealing specifically with
workplace-based studies. The types of vulnerabilities of workers who
participate in those studies are somewhat unique, and can vary depending
on the specific type of study.

5.  Given the very substantial literature on the topic of vulnerable
subjects, the list of references in Section 3.7 might be expanded.   

HSRB Consensus and Rationale

The Board determined that Section 3 is of high quality, accurate and
written clearly. 

The Board recommended additional discussion on justifications for
including vulnerable populations in research as well as expansion on
discussion of who is vulnerable outside of the federal regulations (e.g.
pregnant women, prisoners, children) such as economic, educational or
social vulnerabilities; noting however that in federal regulations
vulnerability is defined in terms of susceptibility to coercion and
undue influence.

The Board also recommended the document: (a) expand examples of studies
that might involve these populations; (b) discuss the tension between
over and under sampling these populations including the importance of
including vulnerable populations in research to ensure that there is
safety data on compounds in which they are more likely than other
populations to be exposed; and (c) discuss the weighing of risks and
benefits in these types of studies

Section 4:  Privacy, Confidentiality, and Other Concerns Related to
Observational Exposure Measurement Studies

Strengths

Section 4 provides a good discussion of several privacy concerns
associated with observational studies, particularly those conducted in
private or semi-private places such as homes and schools.  Although
research volunteers allow study staff to access these environments,
others who may be residing in these environments may object or feel that
their privacy is not being appropriately respected.  Additionally,
during the course of collecting research data, study staff may observe
illegal or immoral activities that may challenge their ability to
maintain the confidentiality of research volunteers or others who may be
living in these environments.

	Document Enhancement

 Considerations relating to the protection of the vulnerable groups,
where more active intervention by study staff may be appropriate should
be strengthen.  For example, Section 4 suggests that researchers develop
a plan for responding to the incidental observation of illegal behaviors
such as child or elder abuse.  It does not recommend that research staff
be trained with regard to the recognition of such behaviors, which is
essential for avoiding both missed opportunities for intervention and
inappropriate accusations of abuse.  Similarly, this section provides
little guidance with respect to the observation of environmental
situations associated with imminent harm, such as observations of
combustible materials near an open flame, a child playing unattended by
a pool, a firearm placed near young children, etc.  Section 4 could be
strengthened by including advice for addressing such situations or
offering general guidance for determining when members of the research
team should act in circumstances involving imminent harm.  It would also
be helpful to reinforce the point that local and state reporting
requirements may vary considerably.

Section 4 does not consider the potential risks to members of the
research staff who may find themselves in situations involving illegal
activities such as drug use or sales, nor does it consider the potential
burdens that research staff members may feel when presented with
behaviors that they find morally objectionable.  In contrast to several
of the other sections in the document, the citations to relevant
literature provided in this section are lacking (e.g., no recent review
articles are cited, nor are there any references to important privacy
documents such as HIPAA).

Specific Suggestions

Section 4 could be strengthened by including advice for addressing
environmental situations associated with imminent harm and offering
guidance for determining when members of the research team should act in
circumstances involving imminent harm.  

Purpose and use of the Certificate of Confidentiality should be
included.  Such certificates do not diminish the need to protect
personally identifiable information and does not relieve the
requirements for reporting illegal behaviors.  While the Agency’s
document states that certificates should be used for “sensitive
matters,” it does not define sensitive matters.  Finally, Federal
regulations exist that clearly define populations needing certificates
of confidentiality.  Such regulations should be noted in the document.  

HSRB Consensus and Recommendations

This section of the document is sensitive to many key ethical and legal
considerations relating to the safeguarding of research-subject privacy,
including the need to (1) disclose the possibility of incidental
reporting requirements to potential volunteers as part of the
informed-consent process, (2) provide advance notification of research
visits to third parties who may be residing in private environments, and
(3) consider potential harms to research volunteers who display personal
monitoring equipment in public settings.  It would be strengthened by
including specific advice on reporting and disclosure procedures when
confidential information indicates  a participant or another person is
in jeopardy, the uses of the Certificate of Confidentiality and
additional references.

Section 5:  Creating an Appropriate Relationship Between the Participant
and Researcher 

Strengths

The Board also commented that this section, like other sections of the
document is excellent; it is readable and useful. In general, Section 5
accurately and clearly discusses the ethical considerations in the
relationship between investigator and participant.  A major strength of
this work is its focus on researcher responsibilities and guidance for
researchers.  A strength is that it includes consideration of the
context of the participant’s community, etc. (One assumes that
“community” includes all the socio-cultural aspects in which the
investigators are competent and respectful.)  In relation to these
strengths, it would help if the document stressed the importance on
researcher training in human subject protection, with emphasis on
observational techniques and community-based research,

The sub-section relating to remuneration (payment) is highly appropriate
and well written. Likewise, Section 5.3 on rights is well done.  Section
5.4, Creating a Supportive Environment for Research and Interaction , as
defined, is very useful and well stated also.

Section 5.5 provides good discussion of equitable selection and of IRB
guidelines (Text Box 5-3) for selecting sub-populations for study from
the ethical standpoint (see below re: scientific standpoints). Section
5.6 on retention issues and ideas, especially in longitudinal studies,
is useful and generally well done.

Document Enhancements

One of the areas that is important not only to recruitment, but to the
successful conduct of studies is language.  The regulations require
information be presented in a language that is “understandable” to
the subjects/potential subjects.  That has always been interpreted to
mean two things; at a literacy level (oral and reading) that is
appropriate for the target population and, in a language that the
listener/reader speaks/reads.  Because various populations to be studied
in the U.S. tend to include persons who speak/read little, if any,
English, it is very important that researchers address how
communications will be handled.  At a minimum, written documents, such
as consent forms, advertising flyers, instruction sheets, etc must be
developed in more than just English.  Such written materials must be
reviewed and approved by an IRB prior to use.  Those review bodies
should obtain independent back-translations to ensure that the
communication accurately conveys the information in the English
version(s).  That is only part of the issue, however, because during the
conduct of the research, investigators and the research team must be
able to communicate orally with subjects, including those who may not
speak English.  Thus researchers must address in protocols how planned
and ad hoc translations will be accomplished.  It should be stressed in
this document that untrained persons (e.g., co-workers etc.) generally
do not meet the ethical requirement for facilitating full understanding
and protecting subject welfare.

Too many discussions/descriptions/examples are correct, but the examples
given are for/applicable to research in general.  Section 5 would be
more useful to researchers conducting observational studies if the
discussion were more focused on the special needs/considerations in that
type of research.  In a related vein, Section 5 makes some global
statements, such as “additional considerations arise …” or “a
number of issues have been identified …,” but there is no
expansion/explanation; so there is no “teaching point.”  These
generalities should be removed or used to start a description of the
concern or issue; Text Boxes 5-2, 5-3 and Appendix C are good examples
of getting the key points into this section without repeating the source
verbatim. 

There are additional sources of information that should be considered
for inclusion in the section, specifically the NRC/NAS 1991 report on
Human Exposure Assessment for Airborne Pollutants and the WHO EHC 27
(1983) report on Guidelines on Studies in Environmental Epidemiology,
statistical survey sampling textbooks, the Board’s discussions of
“purposive” sampling, EPA and NIEHS documents on environmental
equity/justice, and other references contained in this document on
community and Community Advisory Boards (CAB) involvement.  Also, the
literature on observational studies, such as the developing community
based participatory research literature may be informative.  With a
caution about keeping the focus on observational research, there is also
a wealth of “good clinical practice” (GCP) references which could be
used to expand some issues (such as payment and other incentives in
section 5.2.1). 

Specific Suggestions to the Document 

Section 5.1.1 seems to include descriptions of consent elements that are
of general application, but this section would be more useful to
researchers conducting observational studies if the discussion was
focused on the special needs/considerations in that type of research.

Section 5.5, Recruiting Strategies, does not address the scientific
necessities of sometimes including over-sampling (via stratified/cluster
methods) of sub-populations, including the underrepresented and the
overexposed.  Thus, there should be a similar delineation of when such
sampling and recruitment are necessary, similar to Text Box 5-3, which
discusses those needs from an exposure science standpoint. For instance,
it is not  inappropriate to study minority/poor children when evaluating
exposures to lead, pesticides, etc., under existing laws/rules/statutes.
 Some of the “environmental equity/justice” issues need to be
discussed.  CABs need to be involved in approving recruitment materials.

Section 5.5 should discuss consideration of the limitations, especially
statistical issues (representativeness & generalizability) and
non-random sampling (i.e., the scientific problems inherent in
“convenience,” “purposive,” etc. sampling).

There should be a parallel section to Section 5.7 that discusses the
benefits of longitudinal follow-up for the participants, communities,
and responsible agencies and the decrease of risks that may be so
obtained.

The way some references are used (e.g., Grady (page 58, lines 13-21)
seems more suited to a journal article than a review document; that is,
“what does this mean for the researcher” and “what is the
application to observational research” is lost. 

Additional Specific Suggestions

The Board had a few suggestions for possible additional information that
might be included in the chapter:

  Page 51, lines 37-39 makes the comment that the consent process must
explain risks, but in order for subjects to make decisions about
participation that reflect their individual concerns, it would be well
to state that an accurate and realistic description of possible benefits
must be included as well.  See lines 41-44.

    

  Page 52, lines 2-3 refers the reader to three sources for additional
information.  It would be more useful for readers if key points made in
these publications were listed or summarized.  That allows the point to
be made or the issue to be raised.

  Page 52, lines 36-39 discusses one interpretation of “language
understandable to subjects” (native tongue), but it also should be
pointed out that this has a readability requirement too.  Page 53, lines
1-4 and Page 55 lines 1-2 seem to relate this requirement just to a
description of purpose.  This should be fixed.

  Page 53, line 16 makes a reference to a court case; its relevance
needs to be explained.

  Page, line 24 uses the term “informed consent” incorrectly to
imply the form, not the process of information exchange needs to be
presented to parents. (Note the rest of that bullet is fairly dense and
could be revised to make it more readable.)

  Page 55, line 9 uses the term “administration procedure” when
“consent procedure” would be more appropriate.

  Page 55, line 28 uses the term “study elements” when “study
characteristics” would be better.

  Page 55, line 29 is the first use of the word “remuneration” in
this section.  This term clouds the reason subjects are offered payment
for participation in research.  It is not payment for services in the
way employment is.  It is an “inducement,” hopefully not unduly
large, that encourages participation.  The word “remuneration”
should be changed to “inducement” throughout the section and the
document (search and replace) so that the ethical issue regarding
payments is not lost.

  Page 57, line 5 is the first use of the word “compensation” in
this section.  This term is a “regulatory word” and is therefore
associated with “compensation for injury.” It would be clearer and
more accurate to use the term “payment” throughout the section and
the document – for example in the Executive Summary on page 5 -
(search and replace) so that the ethical issue regarding payments is not
clouded.

Page 61, lines 4-5 indicates that community input can be obtained from
research team members.  While true, to be truly representative of
community attitudes, non-research team members should be sought out as
they should be free from any potential scientific biases.

	HSRB Consensus and Rationale

Section 5 identifies most of the major areas and issues where ethical
considerations should be addressed, with the exception of
communication/language issues.  However, in many cases the section takes
the tone of reporting what others have said, with and without comment;
and sometimes (and more appropriately) stating “it is recommended.” 
A document with clear recommendations will be more useful by researchers
than one raising issues without giving direction. The Board raised three
overall questions in reviewing this section of the document.   The
questions were as follows:

1. Can the authors define “a strong relationship”, and what is a
strong scientific relationship?

2. Are there specific OMB guidelines on remuneration that could be
included?

3. Should the participant grievance procedures include also any
component of EPA or the IRB(s) who approved the study?

Section 6:  Building and Maintaining Appropriate Community and
Stakeholder Relationships

Strengths

Section 6 identifies many important areas and issues that need to be
considered in addressing ethical aspects of observational exposure
studies.  This section addresses the rationale for and the complexity of
building and maintaining trustworthy and effective relationships with
communities and stakeholders who are either directly or indirectly
affected by observational exposure studies.  The authors are to be
commended for identifying the key components of this complex and
multi-faceted process.  

This section identifies options for researchers with regard to community
involvement including CABs, which have been shown to be quite successful
in several observational epidemiologic studies in which exposures are
quantified in relation to human health effects.   The composition of
CABs is not well defined.  Such boards should be representative of the
community.  This section also does not clearly articulate the advantages
and disadvantages of engaging a community of research, nor does it
clearly describe best practices.  For example, investigators should be
advocates for the data, not for the community and a CAB is not a
substitute for ethical decisions.  

Section 6 carefully defines important concepts and operationalizes them
for researchers.   Definitions, by design, are relatively broad offering
flexibility to meet specific study objectives tailored for particular
communities.  

Document Enhancements

This section identifies and provides some of the supporting
documentation for key components required for building relationships;
e.g., within an overall conceptual framework that builds largely upon
the diffusions of innovations literature (Rogers, 1995a, 1995b). 
However, much of the presentation is in the form of assertions rather
than supported by sound science.  While many of the assertions are
quoted from references, there may be no hard data in the references to
back up the statements.  The Board recommended that the final report
include evidence-based discussions and/or tables that show better
compliance, retention, and data quality when community involvement is
obtained compared to when it was not.  Scientific rigor is essential in
the exposure science field and therefore should be attended to carefully
in this document.

Both moral and “best practice” considerations are included as
ethical aspects of these studies.  (For example, moral principles with
aspirational results are listed on p. 67, lines 12-15, while Text Box
6-1 lists “best practice” considerations.)  Clearer distinction of
the types of ethical issues and the better use of text box listings to
highlight the differences would strengthen this section [and document]. 
Further, clarification would make it easier for the reader to understand
whether “ethical issues” in this section refers to moral principles,
“best practices,” or both.

Examination of the many ethical issues raised in Section 6 suggests that
the section [and document] may benefit from a text box or table (i.e., a
box that would be parallel to Table 1-3) summarizing the ethical
principles, which are essential for observational exposure studies.  The
advantage of such a summary would be to pull out the principles that
resonate throughout the document; i.e., the ones that are more than
single mentions.  [This summarization may be more effective on a
section-by-section basis, rather than across the entire document.]

Specific Suggestions to the Document

Important points that may merit emphasis in text boxes or other
mechanisms in Section 6 include the underlying principles essential for
effective relationships (now spread throughout Section 6): 

Respect, fairness, beneficence, honesty, openness, trust, commitment,
confidentiality, and responsiveness [Note that justice and other
principles are listed on pages 2 and 5 (etc.).  Perhaps a
cross-reference to earlier, more complete text or supporting materials
is needed in Section 6.]

As appropriate for the contexts, early and active community involvement,
meaningful community roles and responsibilities, and two-way
communications

Effective representation of the community’s diverse views

Dignity, veracity, sustainability, justice and community (p. 69)

Establishing a relationship with the community before a study (p. 71)

Build community capacity (p. 73)

Researchers need to be forthright with communities (e.g., p. 73)

Research relevant to communities (p. 74)

Introductory paragraphs

 

Community-based studies involve numerous ethical issues that are
fundamentally different in important ways from clinical studies (p. 67,
lines 19-21).  Some of these issues are addressed in the earlier
sections of the document, but not discussed in Section 6.  The one issue
highlighted here is two-way communications (p. 67, line 30), but this
emphasis is not consistently presented throughout Sections 6 and 7.

Section 6.1

  

This section is quite good and includes a compendium of ways to approach
the community and get the community involved with a study.  Such
information is invaluable to the neophyte researcher attempting to
gather data in a new community.  

However, some of the reasoning presented in Section 6.1.1 and
sub-sections borders on philosophical, is convoluted and occasionally
circular: ethical action requires trust and trust is essential in
ethical action.  Still, the points are of interest and should be
developed.  Exposure researchers typically have “hard science”
backgrounds, and therefore need both guidance in this area and
substantial evidence supporting the points (e.g., those cited from the
EPA November 2007 workshop on observational exposure studies).  Perhaps
the use of examples would be a better teaching tool and would serve
EPA’s needs better.  

Sections 6.1.1.1 and 6.1.1.2

 

	The sections are not balanced in their discussions of community and
stakeholders, giving less attention to the latter.  The reason for this
imbalance is not apparent and may confuse researchers about who
stakeholders are and what their roles are.

Who is to be involved in community-based studies and how involves a
variety of ethical concerns.  The distinction between “community”
and “stakeholder” should be made clearer (see pp. 69-70).  The
“stakeholder” descriptions on p. 70 (lines14-17) and p. 74 (lines
19-20) could be improved; e.g., stakeholders can physically speak for
communities but may not be seen as legitimate spokespersons for
community interests.  The key issue is whether the community has
actually or officially delegated any of its representation or speaking
rights to stakeholders (whether stakeholders are also members of the
community or not); this seems to be the intent of p. 70, lines 21-23,
27-29, and 38-39.  Perhaps minor editing of this page will clarify and
strengthen the Agency’s guidance, or – more likely – a major
reconsideration of the definitions and differences between
“community” and “stakeholder” may be needed for use throughout
the document.  

Section 6.1.1.4 

Reading comprehension needs to be stressed more and, possibly,
operationalized further.  While there is at least one comment to strive
for an 8th grade reading level, many IRBs require materials prepared for
a 5th grade level.  Also, if translation is required, many IRBs require
a certified translating service and increasingly are requiring back
translation when certificated services are not available.  In the
advisory spirit, some reference may be needed with regard to
translation.

Additionally, data collection instruments can be designed so that
reading comprehension is objectively measured.  To this end, researchers
could be advised to formally evaluate grade level reading and
comprehension while capturing data relevant for exposures.

Sections 6.1.1.4 and 6.1.1.5 

 

A new section should be included since means of communication (sometimes
referred to as channels of communication) are not discussed here.  Just
as the level and type of language used is important, means of
communication should align with communities’ preferences. 
Understanding and using the ways in which communities want to receive
and share information are essential ways of demonstrating respect for
communities’ interests and showing that their input makes a
difference.  The Board suggests that a brief section about this issue be
added between Sections 6.1.1.4 and 6.1.1.5.

Section 6.1.1.5 

Cultural differences should include race/ethnicity but, minimally, also
religious beliefs or other unique lifestyles.  Currently, the text only
pertains to the former and thus is unbalanced and potentially
misleading.  The Board recommended that the authors consider adding
language about providing study results to the communities (and
stakeholders) before publication.

Section 6.1.1.7

  

The role of the researcher as an advocate (in addition to capacity
building) for the community is alluded to in the document (e.g., p. 73,
lines 73-75), but is noticeably absent here.   This is an important
aspect for this section but one that is strongly resisted by many
researchers.  The Agency needs to consider this section very carefully
as it will embark on a new expected path for many EPA researchers -
advocacy – in addition to researcher.

Section 6.2 

This section could also provide advice about a reasonable length of time
for communicating study values (including personal exposure results) to
participants.  Ideally, such times should be determined upfront with
community input.  For example, a reasonable time might be within 6-12
months upon quantification of biospecimens.  

Perhaps, attention should be given to the planned inclusion for health
alerts in real time.  Some observational epidemiologic studies have
health alerts built into web-based data management structures so that
researchers and participants can be alerted to any value requiring
further attention.  This approach needs formalization and guidelines for
implementation in the overall study protocol.  

This section also should address the lessons learned from various
observational epidemiologic studies that have utilized CABs or other
forms of participatory research.  For example, a wealth of information
has been learned from the environmentally oriented studies involving
adults and children or from the Long Island Breast Cancer research
(Gammon et al., 2002).

Section 6.3

The use of “other stakeholders” (p. 74) is confusing to the reader;
what types of stakeholders are “other?”  If the definition of
“stakeholder” is broad, then the concept of “other” becomes
moot.  The text of this section is not fully aligned with the
stakeholder concepts in Section 6.1.1.1 (p. 70, lines 13-14) or the
Glossary (p. 107) and therefore needs to be reconsidered.

Further, the stakeholder discussion in Section 6.3 is given short shrift
when compared with the Community Involvement discussion (Sections 6.1
and 6.2).  Buy-in from stakeholders is often as important as buy-in from
the community even if for the reason that stakeholders often have more
fiscal resources to draw upon.  Without their buy-in, this would limit
financial resources, potentially hampering the ability for the study to
be conducted.  Stakeholders may offer alternative points of view
involving jobs for the community versus the views of some groups in the
community that may want “no pollution and no risk” without regard to
fiscal and social costs.  More comprehensive information about
stakeholders, particularly the needs of local government and business
stakeholders, should be included. 

The importance of continually attending to the dynamics of relationships
is not recognized in this section.  Beyond identifying stakeholders (p.
74, lines 25-29), an additional benefit of Mitchell et al’s (Mitchell
et al, 1997) framework is that it offers a means for ongoing assessment
of the shifts in power, legitimacy and urgency that occur during studies
and that affect views of who is which type of “stakeholder” (as
defined in their article) and how they should be involved.  Further,
Michell et al (1997)  describe different ways in which organizations may
interact with different types of stakeholders throughout a dynamic
process (such as occurs in observational studies).  Recognition and
guidance about anticipating and strategically managing relationships
throughout observational studies would strengthen this section.

It is important to be sure the text boxes (not just in this section) are
fully documented in the text in order to avoid careless use of the
points without regard to the broader issue of community and/or
stakeholder involvement.  Ordering of bulleted items should reflect
logical steps in the process or importance; nonetheless, the rationale
for ordering should be stated.

Editorial Suggestions

Page	Line/s	Section	Comment

67	5	intro	The word exposure is missing   “…observational exposure
research…”

67	36	intro	Add c after 2003. [See reference on page 76.]

68	5	6.1	Consider revising this sentence to read “…nature of the
research itself and affected (or targeted) community…”

68	12	6.1	Do you really mean all qualified members?  Possibly add text
to say how to select among qualified members if the number is high
(e.g., randomly select, established rotations, etc…)

68 

to 

69	47 

to

30	6.1	The section gets a bit far a field in its discussion of new forms
for Institutional Review Boards.  Perhaps a new section or sub-section
is warranted at this point.

69	7-8	6.1	Ensure that these lines reflect the author’s intent.  As
now written they could be misinterpreted.  “CAB members

…have to be educated” sounds directive and could be read as
condescending; is that what the author said?  [The point made on p. 69,
lines 24-26 is related.]

…should represent their communities honestly…” sounds judgmental,
as if the presumption is that CABs will not do so.  Is that what the
author said?

…need to be willing to interact…” involves the same issues as the
bullet immediately preceding this one.

69	28-30	6.1	What is the evidence base for the conclusion made here?  If
none, consider rephrasing this point.

70	13-14	6.1.1.1	This definition of stakeholder could overlap with the
definition of community found on p. 69, lines 39-41.  Inserting a
well-accepted definition for “stakeholder” here and in the Glossary
is recommended.

70	29-32	6.1.1.2	Delete this sentence.  It doesn’t add to the
document.

70	46-47	6.1.1.2	Is this a “should” or “must” issue?  The latter
implies that it is a critical issue, while the former does not.  If the
agency believes that reaching out to multiple organizations is
essential, this sentence would benefit from rephrasing.  There are other
should/must issues in the document; these also may benefit from similar
reconsideration by Agency authors.

71	4-6	6.1.1.3	Delete this sentence.  It doesn’t add to the document.

71	20-21	6.1.1.3	This sentence could be misinterpreted as being
directive to communities, rather than advisory to researchers. 
Rephrasing is recommended.

72	10	6.1.1.4	“Materials distributed” implies written and/or visual
tools only.  This suggests that oral/aural and interactive forms of
communication are excluded; the Board doubts that the authors intended
to exclude such forms of communication here and suggest rephrasing.

72	19	6.1.1.4	“Explanation” is a limited concept in light of the
document’s earlier discussion of the importance of two-way
communications.  Listening to communities’ feedback about the research
should be coupled here to complement the explanation function noted.

72	30-32	6.1.1.5	The key points from this reference should be included
here.  Without them, this sentence is of little use to the reader and
does not provide a level of information about ethical issues comparable
to the rest of the paragraph.

72	35-39	6.1.1.6	This sentence uses the terms “relational paradigm”
and “societal context.”  These terms stand in sharp contrast to the
urgings earlier to present information in a manner understandable to
subjects in a study. These words are jargon and are not clear; this
sentence should be re-worked.

73	6-7	6.1.1.5	An important ethical issue not mentioned here is the need
to attend to the dynamics of relationships; these may change whether one
or more parties judge pre-established roles and responsibilities as
acceptable.  An ongoing process of assuring mutual understanding and
acceptability is often needed because community research environments
are inherently complex and dynamic.

73	17-19	6.1.1.7	Same issue as on p. 71, lines 20-21.

HSRB Consensus and Rationale

In large part, the information is presented accurately and clearly in
each section; however, suggestions for improvements are noted.  Section
6 addresses many of the major issues requiring ethical considerations
but lacks the level of scientific rigor needed by exposure scientists. 
Major suggestions by the Board included:

More data to support points made and less assertion or “theory”
would strengthen this section.

It is important to differentiate the terms stakeholder and community as
well as their interrelationships and discussing the value of community
advisory boards and community sensitive piloting of procedures.

Successful community advisory board procedures, how the
scientist-community relationship will evolve and be monitored over time,
how the results of research are disseminated and the informational
benefits to the community should be discussed.

The Board cautioned about editing the section so that readers would not
erroneously conclude that the EPA is advocating that scientists become
community advocates.  

Section 7:  Designing and Implementing Strategies for Effective
Communication

Strength

Section 7 discusses principles for designing and implementing effective
communication strategies between all affected stakeholders in an
observational exposure study.  One of the important messages coming from
this section is that an effective communication strategy should be a
structured, formal plan and should be given extensive forethought prior
to beginning the study.  Additionally the communication strategy plan
should be a living document that is constantly reexamined and updated as
the study progresses.  It should be stressed that this involves a
two-way communication strategy and an intentional process.  It is as (or
more) important for the researchers to listen to the stakeholders as it
is for them to give information to the stakeholders. 

Document Enhancements

However, Section 7 emphasizes one-way, media-directed, and crisis
communications, which conflicts with the rest of the document.  These
aspects of communications may be part of a comprehensive, strategic
communication program but often are not appropriate as the major
emphasis for community-based, observational exposure studies.  Instead,
community concerns about potentially being stigmatized by research
participation and/or results are often important issues.

In Section 7.1, the importance of formative evaluation should be noted. 
Ongoing evaluation is a means to identify community needs and issues,
thereby permitting researchers the opportunity not only to improve
conduct of the study but also to actively demonstrate respect to
participants, showing that their input makes a difference.

Section 7.2 states it is essential to engage all stakeholders early and
often in the process.  One area that has been commented on for previous
sections is the definition of “stakeholder”.   On page 74 of the
document (Section 6.3) it states “stakeholders may include business,
industry, and local or state governments or agencies with jurisdiction
over the community.”  In contrast, Section 7 includes the community as
one of the stakeholders.  It is important upfront to define the term
“stakeholder” and what groups it encompasses and to use that
definition consistently throughout the document.  The community from
which  the subjects are to be drawn has traditionally been considered as
one of the stakeholders.

While it is critical to identify all pertinent stakeholders, there is a
danger that the group would become so large that it becomes
unmanageable.  In a group such as this there is the potential for having
strong divergent interests.  This could generate enough conflict that it
could impede the research from moving forward in a timely manner.  It is
therefore advisable to add as references some articles on managing
conflict with a diverse population.  A sidebar with some salient points
on managing conflict might help.

Specific Suggestions to the Document

Section 7.3 discusses communication timetables.  It encourages
researchers to begin the dialogue with the community as soon as
possible, learn from the participants and the community,  and continue
exchanges of data and information through the reporting of the study
results and beyond.  It suggests using press releases as an important
tool to engage the community.  Section 7 states that observational
studies should also be announced to stakeholders and the public via
media, community interactions, or other means well in advance of study
implementation.  It is strongly recommended that there be some
“buy-in” by the community before any public announcement.  

Section 7.4 talks about the importance of developing communication
materials at different levels of science literacy.  It stresses that all
materials be written in “plain language.”  However, there are no
follow-up references on available “plain language” strategies and
tools.  Below is the link to the Department of Health and Human Services
Plain Language webpage.  Other empirically-based references could also
be included in the document along with a sidebar on “plain language”
tips.

http://www.health.gov/communication/literacy/plainlanguage/PlainLanguage
.htm

While comprehension is correctly identified as an issue, testing of
tools is under-emphasized here and at the end of Section 7.5.. Empirical
testing of communication methods and content is known to be essential
and demonstrates respect for communities (cf. Health Canada’s
strategic risk communication handbook available at
http://www.hc-sc.gc.ca/ahc-asc/activit/ris-comm/index_e.html).  

Section 7.5 discusses the development of communication materials.  An
example of a pediatric assent document is given.  The point that should
be stressed with this example is that it must be written at the level of
understanding of the reader.  The use of generic communication materials
for several different stakeholders would not be appropriate, contrary to
what the document might suggest, because each stakeholder will be unique
in its understanding, level of involvement, and connection to the
research.  Researchers should be strongly encouraged to gather data
about individual stakeholder needs and priorities prior to developing
communication materials geared towards that population.  Again two way
communication is essential because stakeholders can best define their
strengths and limitations in understanding.

Another communication tool emphasized in this section is the use of the
internet to communicate with study subjects and with interested
stakeholders.  One important point that should be stressed here is that
a communication tool is only useful if it is readily available and
readily understood by the target stakeholder population.  For example,
if the researcher chooses to communicate by internet, he/she should be
certain that everyone in the stakeholder population has ready access to
the internet and has the requisite computer skills to navigate the
research site.  Researchers will have to be even more creative in
developing communication strategies with stakeholder groups that include
 populations of illiterate members or those which do not speak English.

The real emphasis should be on listening carefully to and the learning
from the stakeholders and looking carefully at the communication needs
and limitations of all the pertinent stakeholders before beginning the
development of communication materials.

The end of this section brings up the important issue of conflicting
ethical values and study elements.  However, the final lines only
address remuneration and communication materials (see comments above);
two elements of a study that are not tools for addressing conflicting
values and elements.  The text should segue into comments about methods
appropriate for addressing conflicting values and study disputes.

Section 7.6 states that researchers need to make a commitment to
communicating with and educating the study participants and the
community on the purpose of the study.  Some discussion should be
included about the issue of whether behavior will be adversely changed
as a result of educating the participants.  If too much education
results in a behavior change, the goals of the research will be
compromised.  

Section 7.7 discusses that approaches for reporting results to the
participants but does not note that these approaches should be developed
with community input.   In communicating the results, it is vital to
make sure the study participants clearly understand the meaning of the
results.  Additionally the study participants and the community should
be directed to sources of additional information, resources, and
counselors where they can turn to for additional information or
follow-up information in case of additional questions or concerns.  

Participants should also be given the option of not receiving any
results.  Their options about receiving results should be clearly laid
out and discussed prior to initiation of the study.

The community and stakeholders should be notified prior to the
publication of any study results or any press releases relating to the
publications.  These interested parties have a right to know when data
are going to be made public but do not have the right to change the
science or the researcher’s interpretation of the science in the
publication.  They can, however, dispute the interpretations in public. 

It is unclear why part of this Section focuses on crisis communications
and “responding.”  There is no rationale to help researchers
understand why this discussion is included, or how this advice for a
specific type of communication would fit in a comprehensive risk
communication strategy for observational exposure studies. However,
judging people’s perceptions as “accurate” is not appropriate;
perceptions are what they are whether someone else agrees with them or
not.  Perceptions may differ among individuals and groups; perhaps the
intent here is that lay people may have perceptions that differ from
that of experts (see Morgan et al. 2002. Risk communication: a mental
models approach.) .  Section 7.9 deals with communicating with the
interested stakeholders when a dispute in interpretation of data results
occurs.   This is an important area to stress so that researchers
discuss strategies to deal with this as they develop their
implementation plan.  An additional section on how to deal with
litigation, should it occur, might be helpful to researchers should they
find themselves in this situation.

HSRB Consensus and Rationale

Section 7 covered most of the important issues.  This section had a very
comprehensive and informative list of reference and several suggestions
for additional references have been noted.   While this section is very
well-written, it does not clearly focus on the risk communication
methods most suitable for observational exposure studies.  The use of
side bars is a very effective tool to communicate small bits of
information clearly and quickly.  Other areas that need additional
discussion include: (a) the goals of communication; (b) data sharing and
how to address potential scientist community disagreements; (c) the
context in which communication occurs; and (d) the importance of
formative evaluation.

B.  Completed Oral Therapeutic Study with Sodium Azide

Charge to the Board

1.  The Agency has concluded that this study contains information
sufficient for assessing human risk resulting from potential acute and
chronic exposure.  Please comment on whether the study is sufficiently
sound, from a scientific perspective, to be used as the point of
departure to estimate a safe level of acute and chronic exposure to
sodium azide.  

Board Response to the Charge

  The Black et al. (1954) study, however, suggested increased
sensitivity in several subjects with chronicity, even though the basis
of that sensitivity is undefined.  Whether that was due to additional
blood pressure-lowering effect or another adverse effect is really not
clear. In its discussions, the Board noted that it may also not be
relevant to make direct comparisons of these human data with animal
data, without body surface correction or appropriateness of the endpoint
measured.  The specific concerns of the Board are summarized below: 

Dose: The information provided by Black et al. (1954) suggested that
doses of 0.65 and 1.3 mg were given three to five times to individuals
with no known body weight gender, race, and age.  In the case of chronic
study, the doses were given during 5 days to more than 2 years.  In page
15 of the report, the authors refer to decreasing the dose from 0.5 to
0.25 mg in 20 patients.  However in  Table I which lists 30 out of the
39 patients, only 2 patients received 0.5 mg dose.  This kind of
reporting does not facilitate confident determination of the point of
departure for establishing a safe level of acute and chronic exposure to
sodium azide.  

Toxicity: The study looked for some side effects (liver, kidney) and
clinical measures (no description of methodology or control values) at
the therapeutic doses given to people.  There was no focus on the
critical organ or critical toxicological effect.  Further, toxicity was
evaluated only in 3 individuals and it did not involve any time course
analysis.  The rationale regarding why, when and what was evaluated –
is missing.  One out of three subjects reported pounding of the head
after taking sodium azide, and this level of information is not adequate
to determine a scientifically-defensible NOAEL or LOAEL for acute and
chronic exposures to sodium azide.  

Subjects: There was no baseline/pre-treatment information about the
patients, other than pre-treatment blood pressure, limiting the value of
the results presented in Black et al. (1954).  It is also unclear
whether there was an overlap of patients for the acute and chronic
effects of sodium azide.

The blood pressure data for the 30 hypertensive patients given in Table
I appeared to be included in the data presented in Figures 2 and 3 for
the acute blood pressure change for 35 patients, but only partially.  In
Table I, there are 19 patients with pre-treatment systolic blood
pressure (SBP) of > 190 mmHg and 11 patients with pre-treatment SBP of 
≥ 140 but  ≤ 190 mmHg among 30 hypertensive patients.  However, in
Figure 2, there are only 13 patients with pre-treatment SBP of > 190
mmHg, 13 patients with pre-treatment SBP of  ≥ 140 but  ≤ 190 mmHg,
and 9 normotensive patients with pre-treatment SBP of < 140 mmHg. The
gap in the data cannot be accounted for, and as a result, the veracity
of the data is being called into question.  As shown in the figure
below, the plots of the acute systolic and diastolic blood pressure
changes based on the data presented in Table I for 30 hypertensive
patients in contrast to Figures 2 and 3 from Black et al. (1954).

n=35

n=30

n=35

n=29

HSRB Consensus and Rationale

Based on the inadequacies in the design, methodology and reporting, the
Board concluded that the Black et al study was not sufficiently sound,
from a scientific perspective, to be used as a point of departure to
estimate a safe level of acute and chronic exposure to sodium azide.  In
addition, it was not informative as to whether human responding confirms
estimates based on animal studies. 

 

Charge to the Board

2. Please comment on the following:

Is there clear and convincing evidence that the conduct of the study was
fundamentally unethical?

Is there clear and convincing evidence that the conduct of the study was
significantly deficient relative to the ethical standards prevailing at
the time the research was conducted?

Board Response

The Board noted the lack of documentation pertaining to the ethical
conduct of the research reported by Black and colleagues (Black, et al.,
1954).  It was not unusual for clinical reports of this era to be vague
with respect to matters related to subject recruitment and informed
consent.  The publication by Black and colleagues does not provide many
details with regard to the protection of human volunteers but does hint
at the following: (1) the public-health rationale for the study was
supported by prior research, (2) medical monitoring was in place to
minimize potential risks to research volunteers, and (3) initial dosing
schedules were set at levels believed to be substantially below acutely
toxic levels.  A full assessment of these issues is not possible,
however, due to the limited information available. Even if more
information were available, the Board’s analysis is complicated by the
absence of widely agreed upon standards for the ethical conduct of
research during the time this study was conducted. 

The report makes just a single statement that is related directly to the
adequacy of the process used to obtain informed consent from research
volunteers.  That statement is that the study drug “was administered
without informing the patient of either the nature of the drug, or the
change to be expected.”  Contemporary standards of informed consent
would suggest that this was ethically worrisome because key facts
affecting the assessment of the study’s risk-to-benefit ratio were
withheld from volunteers.  Without a more detailed description of the
set of procedures used to present the study, however, it is difficult to
conclude that the researchers deliberately sought to deceive research
subjects about the nature of the research—which might be seen as a
reason for regarding the study as significantly deficient with regard to
the prevailing ethical standards of the mid 1950s.  For example,
subjects might have been informed that, to avoid a possible placebo-like
effect, they would be “blinded” to the nature of the study drug or
the clinical features being studied.  If subjects were informed about
other pertinent aspects of the study, and agreed to participate, then
that might be viewed as consistent with prevailing ethical standards of
the time.  Alternatively, the researchers conducting the study may have
conceptualized this experimental intervention as “innovative care”
and not felt it appropriate to present the intervention as a research
study.

In summary, two factors combine to limit the ability to assert that the
study was “significantly deficient relative to the ethical standards
prevailing at the time the research was conducted”: (1) lack of
additional detail regarding the procedures used to recruit research
subjects and obtain their consent to participate in the Black study, and
(2) lack of unambiguous ethical standards for determining what
information must be disclosed to potential research volunteers.  

HSRB Consensus and Rationale

Based on lack of documentation in the Black et al. study, the Board was
unable to conclude that there was clear and convincing evidence that the
conduct of the study was fundamentally unethical.   The Board was also
unable to assert conclude that there was clear and convincing evidence
that the study was significantly deficient relative to the ethical
standards prevailing at the time the research was conducted.

C.  Science Issues in Mosquito Repellent Efficacy Field Research

	Appendix A provides background and discussion questions for Board
member and consultant consideration concerning mosquito repellent
studies.  Board responses to the questions are provided below.

Board Response

Issue 1: Factors Affecting Repellent Efficacy

Presentations by EPA and consultants indicated that factors which may
affect the variability of initial and confirmed landings include biotic
factors such as characteristics of the mosquito population (genus and
species distribution, level of ambient mosquito pressure) or abiotic
factors, which are related to characteristics of the test site such as
season, time of day/brightness, or the microclimate (including
temperature, humidity, and wind speed and direction).  Other factors
that may affect landings include characteristics of the test subjects
(such as differences in mosquito attractiveness; use of alcohol,
tobacco, or scented products; or behavior).  Characteristics of the test
methods used that may affect variability include the pattern and
duration of exposure, area of skin treated, method used to determine the
amount of test material applied, and number of concurrent treatments per
subject.

Variable factors that affect first or second landing or bite include
characteristics of the subjects’ skin, temperature, age, hair on the
skin surface, and the density of the mosquito population at the test
site.  Usually, mosquitoes will bite within 5 to 15 minutes after the
subject enters the area; after that, the mosquitoes have acclimated to
the subject’s presence and bite much less frequently.  Temperature
plays a major role in landing or biting activity; an increase in skin
temperature is associated with a decrease in repellency.  Skin color or
type does not have a major effect on landing or biting, and hair on the
skin surface has a small effect.  The density of the insect population
has a major effect on landing or biting activity.  

Factors affecting landing or biting activity also include
characteristics of the mosquito population and the test sites.  There is
significant disagreement among researchers concerning the most
appropriate mosquito population to use for testing.  Density of the
population can affect landing or biting activity.  The age of the
mosquito population also plays a role, with mosquitoes between 5 to 15
days of age being the most avid biters.  Susceptibility of the test
subjects to mosquito bites also can affect landing and biting, although
most people are susceptible to bites.  Selection of test sites
critically affects landing and biting.  Weather, particularly
temperature, can significantly affect biting behavior; repellency
decreases by 8 minutes for every 1 degree Celsius (°C) increase in
ambient temperature.  Human skin temperature (30-32°C) is considered to
be constant.  Wind can significantly affect the efficacy of vapor
repellents because the wind will quickly remove the vapors; increased
wind speed is associated with decreased repellency.  Humidity interferes
with evaporation and repellency.  Light does not appear to have a
significant impact because different mosquito populations exist that are
active either during the day or at night.  The local fauna or flora can
impact repellency, especially if the local fauna are the mosquitoes’
preferred hosts.

Test subject attraction for mosquitoes is variable.  Skin chemistry may
have an impact, but the delivery mechanism used to apply the repellent
to the skin has a larger effect.  Skin temperature is constant and thus
has no impact.  Skin permeability affects repellency, but it is
dependent on the formulation of the repellent.

HSRB Consensus and Rationale

Based on the thorough and highly informative presentations by the
consultants the Board concluded that in general a protocol needs to
elucidate environmental, insect, and subject factors that would create
variability and then provide a rationale for the design, sampling
method, number of subjects based on those factors.  Efficacy relevant
information that might be included: a description and rationale for the
activity level of the subject population, the proximity of subjects to
one another (or to a partner), the expected density of the insect
population, comparison of insect density and other environmental factors
(e.g. temperature, wind) across sites. The Board also concluded that
while replication is important it is difficult to define across
different studies.

Issue 2:  Designing for length based sampling 

The rationales for different designs should be available by reference
and the effects of different designs should be noted.  It is unknown
what, if any temporal protocol for exposure is standard (classical). 
That is, how were the designs derived and who accepted them as standard.
Each design has different length based sampling.  One rationale for
using first confirmed intent to bite (LIBe) rather that first confirmed
bite (FCB), as stated by Dr. Carroll, is that using LIBe minimizes the
probability that a subject is actually bitten by a foraging mosquito
[that might contain a pathogen or produce an irritant or allergic
reaction, etc.].

Exposures in one study were during one-minute periods of exposure in
15-minute intervals, and in another study 5 minutes every  30-minutes. 
The time between first and second (or subsequent) landings is likely to
be very variable. Thirty minutes is usually suggested as a long enough
interval to allow for any mosquitoes in the area to land.  

The Board raised the following general questions.   Were these periods
at the beginning, the middle, or the end of the interval, or were their
timing randomly determined, and what impact did these choices have? 
What impact is there of using 1 or 5 minute periods, and/or of 15 or
30-minute intervals, compared to other period intervals? 

has to be followed by another within a period of time (e.g., “one-half
hour, i.e., in either of the subsequent two exposure periods” for the
LIBe).  Dr. Carroll has indicated but not confirmed that LIBe is
identical to that of ‘First Confirmed Bite’, which was classically
used in measures of repellency to biting insects. 

Dr. Carroll has indicated that it is most likely that a second LIBe (or
bite - FCB) would occur within that 30-minute period.   However, from an
analytic perspective, caution is warranted where FCB is calculated on
the basis of non-continuous field exposure because it reduces biting
pressure by a factor equivalent to: 1/(1-proportional decrease in
exposure).  The upshot (where RP < 100%) is a decrease in the
probability of receiving a first and confirmatory bite in any 30 min
period.

Dr. Carroll’s  modeling of continuous versus. two intermittent
exposure scenarios indicates that the intermittent exposures can
substantially reduce the probability of seeing a FCB at a given RP”,
though this has not been validated experimentally.

  This implies that continuous exposure approaches have been used widely
in field studies.  To make intermittent exposure protocols standard
should require comparison testing for experimental validation.  Board
consultants acknowledge that a 30-minute or one-hour design is more
common in the literature, but does not state whether such designs are
continuous or intermittent exposure designs but imply (from other
statements) that is continuous exposure.  It is never really accurate to
do a repellent trial continuously for more than about an hour or two
because the avidity of the mosquito population will start to change
significantly. The best designs treat people the appropriate number of
hours before the peak biting time and then expose all subjects
simultaneously. 

 do various types of continuous periods of measurements. The military
may do numbers of landings in 20-minute exposures (challenges) with a
criterion of 1 mosquito per minute (and compare treated versus 
untreated leg), with exposures (challenges) at 2, 4, 6, 8, 10 and 12
hours post-application.  Or they can have a protocol with staggered
treatment times during the day (e.g., 800, 1200 or 1600) with specified
continuous exposures at two times during the day (e.g., 1800, 2000). 
USDA may do 6-hour duration testing starting at 0730 and 1345.

There are further concerns regarding such protocols and their effects on
CPT.  A set of concerns has to do with situational factors within the
field setting.  What  differential effect is there within such studies
from light intensity, such as obtained during days in which dusk was
included?  Dr. Schofield states there is an [unspecified] effect, as
there are effects due to variations in temperature (partially related to
the mosquito species in that area as well), and the impact of
subject-activity patterns.  One Board member believed that there must be
also effects of field variations in relative humidity, barometric
pressure, wind speed, and smoke pollution, etc., on mosquito behavior. 
Such variations probably affect species-specific and general mosquito
behavior and may be relevant to CPT calculations. 

The Board consultants added that it might be possible to adjust for wind
and temperature, if the studies have been done for that particular area
and those species of mosquitoes. A much more serious problem is that the
biting activity of mosquitoes varies systematically with time of day.
Variability is due to subjects, location, date and time, and one would
not obtain a normally-distributed (Gaussian) result.  Lab variability
has been related to biotic factors (in subjects and mosquitoes) and
abiotic factors (temperature, relative humidity, light, dose and
exposure time).  These are likely to be variability factors in the
field.  

  For example, we can reasonably predict that moderate-level exercise
will attenuate product performance thereby resulting in a
(probabilistic) systematic decrease in the time of the first and
subsequent bites.  Similarly, we can design a set of protocols to
evaluate the phenomena and to quantify (a posteriori) the effect for
that specific suite of experimental circumstances. However, given the
lack of standardization between studies, the paucity of research
specifically directed towards evaluation of the above-mentioned factors
and the complexity of the test system, elaboration of anything more than
a very basic general model is not possible. 

HSRB Consensus and Rationale 

	The Board noted that the choice of intermittent vs. continuous exposure
designs depends upon the goal of the study (e.g. relative or complete
protection) as well as other factors. The Board concluded that more
research is needed to determine the relative benefits of these designs
and concurred with EPA that future guidance on this matter would be
helpful.

Issue 3: Complete Protection Time

	Dr. Matt Kramer, a USDA statistician, has suggested that the precision
of CPT estimates in repellent testing could be significantly increased
by defining failure of efficacy as the mean time from treatment to a
series of several [e.g., five] landings or bites.  The Board consultants
were asked to determine whether this approach would markedly increase
the precision of CPT estimates without requiring additional subjects
and, if so, whether the increased precision would justify
the incremental risk to the subjects resulting from their exposure to
mosquitoes.  The consultants also were asked to consider the
practicalities involved in testing long-lasting repellents to the point
of five landings.

as the time of repellent application to the first bites/LIBe’s with or
without the confirmation of a second bites/LIBe’s.  It is unclear what
the mean of the times to the first 5 bites/LIBe’s is measuring.  A
discussion on the precision seems irrelevant when one is not so sure
what is being measured.  Besides, these 5 times are all correlated
within each subject, and so the efficiency gain is not so obvious due to
lack of independence among measurements.  With the censoring of efficacy
failure seen in a number of mosquito repellency studies, it is
unrealistic to be able to observe, say first 5 bites or LIBe’s, and
probably impractical to test to the point of 5 bites/LIBe’s.

Perhaps more critical is the artifact of taking average of times to the
first five bites/LIBe’s, which is to overestimate the time to efficacy
failure.  This seems to make sense if one is interested in understanding
the average mosquito biting/LIBing behavior.  However, it seems
unsuitable if one is interested in characterizing the “efficacy” of
repellent products.

The rationale for this idea of measuring times to five bites/LIBe’s
became very clear during the consultant’s presentation.  If the
population for statistical inference is the population of mosquitoes and
the statistical inference is on the mosquito biting behavior, it makes
sense.  However, that is not the case in these mosquito repellency
efficacy studies.  The population for statistical inference is the
population of repellent users, not mosquitoes, and the statistical
inference is on the time from application of repellent to failure of its
efficacy.  

	

HSRB Consensus and Rationale

Based upon information provided by the consultants, the Board remains
unclear of what the mean of the times to the first 5 bites/(landings
with intent to bite) would measure of relevance to EPA determinations of
efficacy.  Therefore The Board concluded that precision of CPT estimates
in repellent testing would not be significantly increased by defining
failure of efficacy as the mean time from treatment to a series of
several [e.g., five] landings or bites.

	The Board concluded it would be helpful to their deliberations if
protocols submitted to the HSRB included rationales for sample size,
outcome measures, number of treatment groups and controls, why a field
study is preferable, why a specific environment was selected, how
different environments differ, and how controls for environmental shifts
in temperature or time of day are determined.  

D.  Completed Insect Repellent Efficacy Study (SCI-001) of DEET
Formulations

Charge to the Board

Is this study sufficiently sound, from a scientific perspective, to be
used to assess the repellent efficacy of the formulations tested against
mosquitoes?  Please comment specifically on:

Whether participation in field testing by several subjects on the day
after they had been treated with a different test repellent is likely to
have affected the validity of the results for those subjects on those
days.  

Board Response

The active ingredient DEET formulated in three forms was tested for its
ability to repel mosquitoes from the legs of volunteers by the protocol
presented and modified by Carroll-Loye.  The protocol had been modified
based on the suggestions and input of EPA and the HSRB.  The results
were reported in SCI.001.1, LipoDEET 302; SCI.001.2, LipoDEET 3434; and
SCI.001.3 Coulston’s Duranon, all compared to Ultrathon (approved by
the military). All experiments were conducted using Good Laboratory
Practices.

The dosimetry for the three products was done in the laboratory on July
3-5, 2007.  The field tests were conducted at Site 1 in Butte County, a
grassland habitat, on July 7, 8 and 15, and at Site 2 in Glenn County, a
forest habitat, on July 12, 13 and 14.  Different mosquito species
composition occurred at the two sites. Ten subjects were used for the
dosimetry tests.  Ten subjects were used for each of the three products.
Some of the subjects participated for all three of these test articles
as well as for the Ultrathon. The subjects were required to be above 18
years of age and no more than 55 years of age, active in rural outdoor
settings, and having used no repellent on the day before the test. Only
legs were tested in this study because of greater biting pressure on
legs than arms. There were two experienced persons serving as negative
controls (i.e., without any repellent product) to confirm mosquito
biting pressure (and biting pressure was maintained throughout the
period of the study, defined as at least one Landing with Intent to
Bite, LIBe, per min at Site 1 and the second site selected to be Site
2). Experimental subjects, in pairs, monitored LIBe’s during a one
minute interval each 15 minutes, until the First Confirmed LIBe (FCLIBe)
could be determined. Stopping rules were employed.  No evidence of West
Nile Virus (WNV)  was present in either test site. Mosquitoes landing
were taken to the laboratory for later identification, and for screening
for WNV, Western Equine Encephalitis, and St. Louis Encephalitis Virus,
and all mosquitoes were negative.  All subjects wore Tyvek coverall,
head nets and surgical gloves, and they worked in pairs.  There was a
one minute observation period during each 15 minute interval, starting
150-210 minutes post application.  Toward the end of the day, controls
were reduced from 2 to 1 subject to limit exposure to mosquitoes.
Complete protection time (CPT) was measured, defined as the time to the
First Confirmed Landing with Intent to Bite (FCLIBe). Adequate LIBing
pressure was maintained throughout the test period. 

LipoDEET 302 is 30% DEET on lipid spheres designed to improve the
durability and to improve the cosmetic properties. Six females and four
males were tested for dosimetry, 7 females and 3 males at Site 1, 3
females and 7 males at Site 2.  The MOE was 522.  It yielded a CPT of
10.3 ± 1.3 hr in Site 1 and 9.5 ± 1.8 hr in Site 2.

LipoDEET 3434 is DEET at 34.34%; this protocol SCI.001.2 was amended to
test a different test article than the original protocol described  Five
females and five males were tested at Site 1, 4 females and 6 males at
Site 2.  The MOE was 492.  It yielded a CPT of 10.6 ± 1.3 hr in Site 1
and 10.4 ± 1.9 hr in Site 2.

Coulson’s Duranon is 20% DEET in microscopic protein spheres to
reduced skin absorption of DEET, improve cosmetic properties and inhibit
evaporation.  Five females and five males were tested at Site 1, 3
females and 7 males at Site 2.  The MOE was 856.  It yielded a CPT of
8.4 ± 1.9 hr in Site 1 and 9.5 ± 1.3 hr in Site 2.

Data were presented for the comparison article, Ultrathon, CPT of 10.1
± 2.3 hr for Site 1 and 10.0 ± 2.2 hr for Site 2.  While this
comparison may have been of interest to the sponsor, it is unclear why
this information was provided in this report.

The report was clearly written and a greater attention to statistical
analysis was provided than has been provided in the past by Carroll-Loye
Biological Research.  The study was justified in that additional insect
repellents that are more efficacious and/or more acceptable cosmetically
to the public would be an advantage from both the standpoint of health
(to reduce the chances of contracting a mosquito-borne disease) and of
comfort.  The information should be generalizable to the public,
although the exclusions, which were highly appropriate, excluded some
subpopulations that would likely use insect repellents.  The experiment
was necessary to determine the field efficacy of these test
formulations, and the experiments were set up to meet the study
objective.  Measurements taken were appropriate for the objective and
quality assurance considerations were in place.

 despite the fact that the protocol stated “Use of insect repellent
within one day of preceding the study” was an exclusion criterion. 
Additionally, the tests on any given repellent (including the repellent
used in study WPC-001) were not conducted on a single day and the
statistical treatment of the data did not account for different days of
testing.  

The deviation of not allowing a day of non-repellent use before testing
was of concern to the Board because of potential persistence of the
repellent of one day’s test into the test of the following day. 
However, information provided by the Carroll-Loye Biological Research
via public comments indicated that the repellent was carefully washed
off at the end of the experiment both with soap and water and with
alcohol.  The public comments also indicated that the reason for the one
day wait in the protocol was not related to concerns about the
persistence of the repellent.  Therefore this deviation did not affect
the validity of the results obtained.  In addition, the public comments
indicated that the reason for the compression of the tests on these
several test materials into a short span of time was because of the
reports that West Nile virus-contaminated mosquitoes were moving close
to the test area and that this test region would have been unsafe in a
matter of just a few more days. This explanation was adequate to explain
why the testing days were set up as they were.  It would have been
advisable to have had this deviation and the rationale for it presented
in the report. 

Charge to Board

b.	The effects of changes to the experimental design resulting in
evaluation of repellents using fewer than ten subjects per treatment per
day, followed by pooling of results by site for statistical analysis.

Board Response

Strengths

 of the study is the inclusion of the positive control treatment with
3M’s Ultrathon.

Weaknesses

SCI-001 studies are very troubling from a statistical design point of
view.  There is some indication that there was no desire to compare the
various repellents to one another, and yet each of reports compared a
repellent to a positive control, the Ultrathon repellent.  The
statistical analysis assumes that there are 20 different subjects on
each repellent over two sites, but many of the subjects wore both
repellents and a few subjects were used in both sites.  Subjects that
were used at both sites were subjects:  8, 13, 14, 15, 37, 40, 46, 52,
53, 60, 61, 62, 63, 67, 71, 72, and two control subjects.  Almost all
subjects wore more than one repellent.  This is not an inherent flaw in
the design, although if subjects are going to be used more than once,
there should have been some counterbalancing of the repellents to the
subjects.  The major flaw is in the statistical analysis, particularly
when repellents are being compared to one another as is done in the
reports by comparing each repellent to Ultrathon.  The analysis assumes
that there are 20 different subjects involved in each of the three
write-ups.  That is, the analysis assumes that the experimental design
is a parallel subjects design and it is clearly not.

In the combined  experiments, there were only 33 unique subjects,
representing 80 data points, excluding the negative controls.  Subjects
do not appear to have been randomly allocated to four test materials,
between Sites 1 and 2, and over five days.  This lack of statistical
independence of data between different test materials, Sites 1 and 2,
and different dates of experiment renders the analysis incorrect.  For
example, due to overlap of subjects between Sites 1 and 2, one cannot
pool the results by site for statistical analysis.  Also there is
substantial overlap of subjects among and between test materials, the
comparison among and between test materials cannot be made.  This
renders the analysis scientifically invalid.  

In SCI-001, there is 0% censoring.  This differs to previous studies
conducted and reported by Carroll-Loye Biological Research, Inc and
reviewed by the Board.  As an example, there was 40% censoring in
EMD-004.1, 10-30% censoring in EMD-004.2, and 90-100% censoring in
EMD-004.3.  The Board supports the EPA’s evaluation that “Further
clarification is needed to verify the accuracy of the data generated.”

The data analysis (pairwise comparison between three test materials and
the positive control Ultrathon) is inadequate and is inconsistent with
the experimental design and conduct of the study in which subjects were
allocated to four treatment groups.  As was suggested by the HSRB in its
January 2007 meeting, the appropriate analysis would have been an
analysis of variance type.  Wilcoxon rank sum test in the presence of
censored data is known as Gehan’s test, and is known to be inefficient
for comparison of Kaplan-Meier estimates.  A more appropriate test
comparison of censored time to event data is the log rank test or better
yet a regression analysis for censored time to event data based on Cox
proportional hazards regression models.   The analysis of the number of
LIBe’s is inappropriate as it ignores the duration of follow-up during
which LIBe’s are recorded.  An appropriate analysis would have been
Poisson analysis.

Having ten subjects spread over  five days and combining the results
into the current analysis has a potential to confound the effect of
repellent with the different mosquito pressures on different days. This
would have been better if it were planned in advance.  The whole idea is
that the experimenter has  control over the experiment by using random
mechanism for subject selection and for allocation of subjects to
repellents, days and sites. One solution would be to perform the
analysis of variance as was done when the results for two sites were
pooled, i.e., by including the day as the main effect and the two-way
repellent by day interaction (repellent effect modification by day
effect) in the analysis of variance, in Table 5, on page 17 of 217, of
the report for SCI-001.3.  Of course, this requires making the
assumption that the experimenter had adequate control of the experiment
as described above and that observations are statistically independent,
which is not the case.  With the proper experimental design using site
and day of experiment as blocking factors, the analysis of variance
would allow evaluation of the three main effects of repellent, day and
site, along with the two-way and three-way interactions.  There is no
inherent problem with pooling across studies.  Study site can be
considered as a blocking factor, and as such it would generally be
appropriate to pool across the two blocks.  This assumes independence of
data from sites, namely, no overlap in subjects between the sites.  This
was not the case for these experiments, and as a consequence the data
cannot be pooled across the two sites.

Pooling of results by site for statistical analysis is generally
preferable if the other experimental design aspects are the same, which
appears to be the case with the reported studies.  The main advantage of
a  pooled analysis versus   marginal analyses  for each site is
two-fold: 1) by doubling the sample size for the analysis, you increase
the statistical efficiency by a factor of 1.41 in the point estimation
of the CPT exactly or in the analysis of variance approximately (as you
lose one degree of freedom for site effect) and 2) you can assess the
effect of repellent more accurately by accounting for the potential
effect of site difference in mosquito pressure.  However, this assumes
that the data from two sites are independent, which is again obviously
not the case.  There are three subjects (40, 18 and 62) used in both
sites with LipoDEET 302, two subjects (53 and 46) used in both sites
with Ultrathon, and three subjects (40, 8 and 63) used in both sites
with Duranon.

HSRB Consensus and Rationale

 in the study, this did not affect the validity of the results.  

 thus rendering scientific validity of the results into question.  In
addition, the study may not have been sufficiently sound to estimate 
within  population variances. 

Thus, the Board concluded that the study was not sufficiently sound,
from a scientific perspective, to be used to assess the repellent
efficacy of the formulations tested against mosquitoes.  

Charge to Board

2.  Does available information support a determination that this study
was conducted in substantial compliance with subparts K and L of EPA
regulations at 40 CFR part 26?  Please comment specifically on:

a.	The decision to use a different test formulation in place of one of
the test materials described in the protocol reviewed by the IRB, EPA
and the HSRB.

b.	How to assess the ethical conduct of an insect repellency study
involving multiple test formulations when there is an ethical deficiency
in the conduct of the study with respect to one of the test
formulations.  If the ethical deficiency warrants not relying on the
results of the testing with regard to one test formulation, under what
circumstances (if any) does the ethical deficiency affect the
acceptability of the results from testing the other formulations?  

Board Response

Brief Overview of the Study

The basic protocol for these studies (SCI-001) was initially reviewed at
the January 2007 meeting of the HSRB, at which time the Board concluded
that the study would meet the requirements established in the
Environmental Protection Agency’s final human studies rule (40 CFR
Part 26) pending minor revision.  In particular, the Board was concerned
about several matters relevant to subject recruitment and the overall
conduct of the study.  For example, the protocol did not describe how
untreated controls would be recruited, but instead implied that controls
will be recruited in the same manner as subjects in the “exposure”
arm—via “word-of-mouth” and a Volunteer Data Base maintained by
the Principal Investigator.  The Board recommended that the protocol
clarify how untreated controls will be recruited, and that the IRB of
record (Independent Investigational Review Board [IIRB], Inc. of
Plantation, FL) review any materials used for recruiting purposes,
including any telephone scripts fliers, emails, letters, or local ads. 
The Board also felt that the risks associated with DEET exposure during
the course of the study are mischaracterized, and that the
informed-consent document was also structured in a manner that did not
apply to unexposed control subjects.  In light of these deficiencies,
the Board recommended that the informed-consent document be redesigned
and re-reviewed by IIRB (EPA HSRB 2007a). 

The documents provided by Carroll-Loye (Carroll, 2007a; Carroll, 2007b;
Carroll, 2007c) specifically state that each study was conducted in
compliance with the requirements of the U.S.  EPA Good Laboratory
Practice Regulations for Pesticide Programs (40 CFR 160); 40 CFR 26
subparts K and L; FIFRA § 12(a)(2)(P); and the California State EPA
Department of Pesticide Regulations for study monitoring (California
Code of Regulations Title 3, Section 6710).  Each study was also
reviewed and approved by a commercial human subjects review committee,
IIRB.  Documentation provided to the EPA by IIRB indicated that it
reviewed these studies pursuant to the standards of the Common Rule (45
C.F.R. Part 46, Subpart A) and determined them to be in compliance with
that Rule.

As submitted to the Agency, each completed study consists of two
interdependent analyses: 1) a dosimetry study designed to determine the
amount of an insect-repelling compound (30% or 34% DEET in liposomal
capsules, or 20% DEET in protein capsules) that typical users would
typically apply when provided with a lotion formulations; and 2) an
efficacy study designed to measure the effectiveness of each compound as
a mosquito repellent.  For each dosimetry and efficacy studiesstudy,
commercially available and obtained 3M Ultrathon (34.34% polymerized
DEET) was used as a comparator.  Ultrathon is the principal insect
repellent used by American military forces and is considered to be one
of the most effective insect repellents available.

Dosimetry and efficacy studies for personal insect repellents containing
LipoDEET 302 and 3434 (30% and 34% DEET in liposomal capsules,
respectively) and Duranon (20% DEET in protein capsules) were conducted
from July 3 through August 2, 2007 (Carroll 2007a; Carroll 2007b;
Carroll 2007c).  All studies (one using each test compound) were
performed simultaneously at a laboratory site in Davis, California, and
at field sites in Butte and Glenn Counties, California, by researchers
at Carroll-Loye Biological Research.  The study sponsor was Scientific
Coordination, Inc., of Rockville, Maryland.  The studies were conducted
using products from two manufacturers:  LipoDEET 302 and 3434 was
manufactured and supplied by DermAegis, Inc. of Rockford, Illinois;
Duranon was manufactured and supplied by Sawyer Products of Safety
Harbor, Florida.  It is important to note, however, that the protocol
originally reviewed and approved at the January 2007 HSRB meeting listed
a different set of test compounds; at the sponsor’s request
Carroll-Loye Biological Research changed one of the test compounds from
Insect Guard II (EPA Reg. No. 54287-8) to LipoDEET 3434 (an unregistered
compound) at the sponsor’s request.

The dosimetry study enrolled a total of 10 individuals, each of who
tested all four formulations (the three test compounds and the
comparator).  Each efficacy study enrolled 10 subjects for each
formulation at each of the two field sites.  Many volunteers
participated in multiple analytic phases.  For example, six of the 10
participants in the dosimetry study also participated in one or more of
the field tests.  In total, 39 volunteers participated in at least one
analytic phase of SCI-001.1, SCI-001.2 or SCI-001.3.  Fifteen of these
volunteers also participated in the dosimetry and/or efficacy phase of
Carroll-Loye study WPC-001 described below.  In addition to these 41
volunteers, two controls (described as “experienced personnel” and
who were untreated with repellent) also participated to determine
ambient LIBe pressure at field sites. On any given day at a field site,
the same pair of participants served as controls for SCI-001.1,
SCI-001.2, SCI-001.3 and the concurrently run WPC-001 study.  Five
individuals served (repeatedly) as controls during each of the six field
tests; two also participated in the repellent efficacy tests, to give a
cumulative total of 45 volunteers.  In addition, three alternate
participants were enrolled to: 1) replace any individual who withdrew;
and 2) protect the confidentiality of any participant excluded from the
study as a result of pregnancy or other potentially stigmatizing
condition, as described below.

Critique 

The Board concurred with the factual observations of the ethical
strengths and weaknesses of the study, as detailed in the EPA’s
Science and Ethics Review (Carley, 2007).  The risks to study
participants, in general, were minimal and were justified by the likely
societal benefits, including data on the efficacy of these new
formulations (30% or 34% DEET in liposomal capsules, or 20% DEET in
protein capsules) as personal insect repellents.  Higher concentrations
of DEET are commercially available and have been used as repellents for
years; the subjects enrolled in this study were thus unlikely to be at
increased risk of experiencing adverse side effects upon exposure. 
Reactions to mosquito bites are usually mild and easily treated with
over-the-counter steroidal creams.  The study also excluded individuals
who have a history of such severe skin reactions to further minimize the
risk of a participant experiencing a severe physical reaction to a
mosquito bite.  In addition, the study protocol was designed
specifically to minimize the likelihood that a mosquito will bite,
through the use of clear stopping rules, limited exposure periods, and
paired observation; no side effects or adverse events were reported.  To
minimize the risk that study participants will would be exposed to
illnesses like WNV, the study protocol called for field tests of
repellent efficacy to be conducted only in areas where known
vector-borne diseases have not been detected by county and state health
or vector/mosquito control agencies for at least one month.  Mosquitoes
collected during the field studies also were subjected to serologic or
molecular analyses to confirm that they were free of known pathogens. 
Finally, the study protocol also included several mechanisms designed to
minimize coercive recruitment and enrollment, compensation was not
considered to be so high as to unduly influence participation, and
minors and pregnant or lactating women were explicitly excluded from
volunteering (pregnancy being confirmed by requiring all female
volunteers to undergo a self-administered over-the-counter pregnancy
test on the day of the study).  The potential stigmatization resulting
from study exclusion was minimized by the use of so-called
“alternate” participants, allowing for volunteers to withdraw or be
excluded from participating without unduly compromising their
confidentiality.

Regrettably, several serious protocol deviations occurred during the
conduct of these studies. Most notably, due to the study
investigator’s failure to obtain IRB approval for fundamental changes
to one of the study protocols, the research described in SCI-001.1,
SCI-001.2 and SCI-001.3 does not comport with the applicable
requirements of 40 CFR Part 26, subparts K and L. 

 was also made without submitting the protocol for approval or informing
study participants of this change, and may represent a departure from
accepted review and approval practices.  There was some disagreement
among Board members, however, as to whether the phrase “one day
preceding the study” specifically excluded participants who had used
or been exposed to an insect repellent on the day preceding a field
trial, or simply prior to consent and enrollment in the study overall. 
Dr. Carroll submitted a response to the EPA that implied the former
(Carroll, 2007d), but the Board nevertheless recognized the uncertainty
surrounding this issue.

Second, to minimize the risk that study participants will would be
exposed to arthropod-borne illnesses like WNV, for example, during its
review of another Carroll-Loye protocol at the April 2007 HSRB meeting
(EPA HSRB, 2007b) the Board recommended conducting serologic or
molecular analyses to confirm that mosquitoes collected during the field
studies were free of known pathogens.  Study investigators did conduct
such analyses in the context of these studies, but did so without
submitting the protocol for approval or informing study participants of
these analyses in the informed consent document.  Although this change
is likely to have actually reduced the risks faced by study
participants, it is nevertheless a substantial departure from accepted
review and approval practices.

Finally, and most seriously, at the sponsor’s request Carroll-Loye
Biological Research changed one of the test compounds from Insect Guard
II (EPA Reg. No. 54287-8) to LipoDEET 3434 (an unregistered compound)
without IRB or HSRB review or approval.  Dr. Carroll has justified the
decision to substitute a test material by stating that, in conversations
that he has previously had with the IRB and the California EPA, that IRB
review and approval is needed only for “proposed changes … that are
likely to increase risk to participants” (Carroll 2007d, 3).  Because
the substituted material (LipoDEET 3434) was nearly identical to another
compound approved for testing (LipoDEET 302), and contained a
concentration of DEET considerably less than that in already approved
and marketed personal insect repellents, Dr. Carroll felt that the
substitution posed no increase in risk to study participants and fell
within the “latitude” given to study investigators with respect to
minor protocol changes.  The Board disagreed, feeling that a change of
test materials was not simply a minor protocol change; although it is
unlikely that study participants were subjected to greater risk, the
substitution of LipoDEET 3434 for Insect Guard II was a protocol change
of sufficient magnitude to warrant IRB notification and review.  More
importantly, the failure to seek IRB review and approval is a
significant and serious departure from accepted review and approval
practices, as well as a violation of the letter and intent of the
Agency’s Final Human Studies Rule.

Federal guidelines from the Office of Human Research Protections (OHRP)
clearly state that all proposed protocol changes must be reviewed by the
IRB of record at convened meetings, in accordance with HHS regulations
at 45 CFR 46.108(b), although institutions may adopt policies describing
the types of minor changes in previously approved research that can be
approved under an expedited review procedure in accordance with HHS
regulations at 45 CFR 46.110(b)(2).  Except in cases when necessary to
prevent imminent harm to study participants, an investigator should
never institute a protocol change without IRB review.  Federal
regulations regarding review and approval of human subjects research,
for example, explicitly prohibit investigators from implementing any
protocol changes without prior IRB approval unless such changes are
necessary to prevent immediate, serious harm to study participants.  It
is never an investigator’s prerogative to determine which protocol
changes warrant IRB review and which do not; only the IRB (or some
authority other than the investigator) has that authority. 

The failure of Carroll-Loye Biological Research to 1) obtain IRB
approval of the revised protocol; and 2) report these protocol
deviations to the Independent Investigational Review Board in a timely
manner are serious regulatory breaches.  The failure of Dr. Carroll to
notify the EPA and the HSRB of protocol deviations such as the
substitution of LipoDEET 3434 for Insect Guard II in submitted study
documents for studies SCI-001.1 and SCI-001.3 is also troubling.  The
Board recommended the Carroll-Loye Biological Research report these
deviations to the IIRB as soon as possible and work with that
organization to develop and implement a corrective course of action. 

Because of the serious nature of these deviations, the research
described in SCI-001.2 using the unapproved pesticide formulation failed
to comport with the applicable requirements of 40 CFR Part 26, subparts
K and L, and the Board recommended that the Agency not accept for
regulatory decision-making purposes any of the data obtained during the
conduct of that particular study.  Furthermore, this particular ethical
deficiency also affects the acceptability of the results from testing
the other two formulations. SCI-001.1, SCI-001.2 and SCI-001.3 were run
concurrently, and study participants enrolled in all three studies ran
the risk of being randomized to receive treatment with the unapproved
investigational compound, LipoDEET 3434.  It is thus impossible to
separate the data collected in SCI-001.1 and SCI-001.3 from that
collected in SCI-001.2, and the Board recommended that the Agency not
utilize any the data obtained during the conduct of these three studies.

HSRB Consensus and Rationale

Based on a careful review of the study documents provided, the Board
concluded that the research was conducted in a manner that failed to
meet the applicable requirements of §40 CFR 26, subparts K and L.  The
study investigator failed to obtain IRB approval for fundamental changes
to one of the study protocols and, as a result, the data collected from
these three concurrently run studies cannot should not be used by the
Agency for regulatory decision-making purposes. 

E.  Completed Insect Repellent Efficacy Study with Oil of Lemon
Eucalyptus (WPC-001)

Charge to the Board

1.  Is the research conducted under WPC-001 sufficiently sound, from a
scientific perspective, to be used to assess the repellent efficacy of
the formulation tested against mosquitoes?  Please comment specifically
on whether participation in field testing by several subjects on the day
after they had been treated with a different test repellent is likely to
have affected the validity of the results for those subjects on those
days.

Board Response

The active ingredient Oil of Lemon Eucalyptus (OLE) formulated as a pump
spray was tested for its ability to repel mosquitoes from the legs of
volunteers by a study presented by Carroll-Loye.  All experiments were
conducted using GLP. This study was conducted similarly to and in
conjunction with the DEET efficacy studies of SCI-001, and the
descriptions of the experimental conditions and design of SCI-001,
above, are appropriate for this study and will not be repeated here.

The test product had 30% OLE in a pump spray. Seven females and three
males were tested at Site 1, 3 females and 7 males at Site 2.  It
yielded a CPT of 6.1 ± 1.5 hr in Site 1 and 4.2 ± 0.8 hr in Site 2.

, with the most serious being the lack of at least a one day wait for
some of the subjects before they were tested with another product.  The
Board’s analysis of this deviation and design of this experiment was
discussed above for SCI-001, and is pertinent to this experiment also
and will not be repeated here. 

  Treatment of subjects on successive days is not intellectually
satisfying, but it was unlikely to have a significant effect.  

HSRB Consensus and Rationale

Charge to Board

2.  Does available information support a determination that the research
covered by WPC-001 was conducted in substantial compliance with subparts
K and L of EPA regulations at 40 CFR part 26?  If the conduct of any
part of SCI-001 is deemed not to substantially comply with the
requirements of subparts K and L, please comment specifically on how to
assess the ethical conduct of research conducted under WPC-001 in light
of the fact that it was conducted at the same times and at the same
places as the research covered under protocol SCI-001.  

Brief Overview of the Study

The protocol for this study was initially reviewed at the April 2007
meeting of the Human Studies Review Board, at which time the Board
concluded that the study would meet the requirements established in the
Environmental Protection Agency’s final human studies rule (40 CFR
Part 26) pending minor revision.  In particular, the Board was concerned
about several matters relevant to subject recruitment and the overall
conduct of the study.  To minimize the risk that study participants
would be exposed to arthropod-borne illness, for example, the Board
recommended that investigators trap landing mosquitoes or other vectors
for pooled serologic or nucleic acid-based testing and alert research
participants if they had been inadvertently exposed to vector-borne
pathogens.  Secondly, the Board expressed concerns about plans to
recruit research subjects in Florida, as these recruitment procedures
were not described adequately in the protocol and supporting materials. 
Finally, the Board raised questions about the informed consent
procedures for control subjects, suggesting that the informed consent
procedures should be modified to more clearly explain the risks to
control (untreated) research subjects.  In light of these deficiencies,
the Board recommended that the protocol and informed-consent document be
revised and submitted to re-review by IIRB (EPA HSRB, 2007b).  The
protocol and informed consent documents subsequently were revised in
accordance with Board and EPA recommendations, submitted to IIRB for
re-review, and approved (Carley, 2007). 

The documents provided by Carroll-Loye (Carroll, 2007e) specifically
state that each study was conducted in compliance with the requirements
of the U.S. EPA Good Laboratory Practice Regulations for Pesticide
Programs (40 CFR 160); 40 CFR 26 subparts K and L; FIFRA § 12(a)(2)(P);
and the California State EPA Department of Pesticide Regulations for
study monitoring (California Code of Regulations Title 3, Section 6710).
 Each study was also reviewed and approved by a commercial human
subjects review committee, IIRB.  Documentation provided to the EPA by
IIRB indicated that it reviewed these studies pursuant to the standards
of the Common Rule (45 CFR Part 46, Subpart A) and determined them to be
in compliance with that Rule.

As submitted to the EPA, the completed study consists of two
interdependent analyses: 1) a dosimetry study designed to determine the
amount of oil of lemon eucalyptus-based personal insect repellent that
typical users would typically apply when provided with a pump spray
formulation; and 2) an efficacy study designed to measure the
effectiveness of the pump spray formulation as a mosquito repellent. 
The dosimetry and efficacy studies were conducted from July 10 through
August 1, 2007 (Carroll, 2007e).  The studies werey was performed
simultaneously at a laboratory site in Davis, California, and at field
sites in Butte and Glenn Counties, California, by researchers at
Carroll-Loye Biological Research.  The study sponsor was WPC Brands,
Inc. of Bridgeton, Missouri.  The studies werey was conducted using
product manufactured and supplied by ChemRite CoPac, Inc. of Lannon,
Wisconsin.

The dosimetry study enrolled a total of 10 individuals.  Each efficacy
study enrolled 10 subjects at each of the two field sites.  Many
volunteers participated in multiple analytic phases. For example, six of
the 10 participants in the dosimetry study also participated in one of
the field tests.  In total, 23 volunteers participated in at least one
analytic phase of WPC-001. Fifteen of these volunteers also participated
in the dosimetry and/or efficacy phase of Carroll-Loye studies
SCI-001.1, SCI-001.2 or SCI-001.3 described previously (Carroll, 2007a;
Carroll, 2007b; Carroll, 2007c).  In addition to these 23 volunteers,
four controls (described as “experienced personnel” and who were
untreated with repellent) also participated to determine ambient LIBe
pressure at field sites.  On any given day at a field site, the same
pair of participants served as controls for WPC-001 and the
concurrently-run SCI-001.1, SCI-001.2, and SCI-001.3 studies.  Three
individuals served (repeatedly) as controls during each of the three
field tests; all three also participated in the repellent efficacy
tests, giving a cumulative total of 27 volunteers. In addition, three
alternate participants were enrolled to: 1) replace any individual who
withdrew; and 2) protect the confidentiality of any participant excluded
from the study as a result of pregnancy or other potentially
stigmatizing condition, as described below.

Critique

The Board concurred with the factual observations of the ethical
strengths and weaknesses of the study, as detailed in the EPA’s
Science and Ethics Review (Carley, 2007). In general, the research
described in WPC-001 comports with the applicable requirements of 40 CFR
Part 26, subparts K and L.  The risks to study participants were minimal
and were justified by the likely societal benefits, including data on
the efficacy of this formulation as a personal insect repellent.  Based
on toxicological data currently available, the subjects enrolled in this
study were unlikely to be at increased risk of experiencing adverse side
effects upon exposure.  Reactions to mosquito bites are usually mild and
easily treated with over-the-counter steroidal creams.  The study also
excluded individuals who have a history of such severe skin reactions to
further minimize the risk of a participant experiencing a severe
physical reaction to a mosquito bite.  In addition, the study protocol
was designed specifically to minimize the likelihood that a mosquito
would bite, through the use of clear stopping rules, limited exposure
periods, and paired observation; no side effects or adverse events were
reported.  To minimize the risk that study participants will be exposed
to illnesses like WNV, the study protocol called for field tests of
repellent efficacy to be conducted only in areas where known
vector-borne diseases have not been detected by county and state health
or vector/mosquito control agencies for at least one month.  Mosquitoes
collected during the field studies also were subjected to serologic or
molecular analyses to confirm that they were free of known pathogens. 
Finally, the study protocol also included several mechanisms designed to
minimize coercive recruitment and enrollment, compensation was not
considered to be so high as to unduly influence participation, and
minors and pregnant or lactating women were explicitly excluded from
volunteering (pregnancy being confirmed by requiring all female
volunteers to undergo a self-administered over-the-counter pregnancy
test on the day of the study).  The potential stigmatization resulting
from study exclusion was minimized by the use of so-called
“alternate” participants, allowing for volunteers to withdraw or be
excluded from participating without unduly compromising their
confidentiality.

 comments are warranted on certain events that took place during the
conduct of the study.  In particular, as noted in the EPA’s review of
these studies, several significant protocol deviations occurred.

 from accepted review and approval practices.

 to IIRB in a timely manner are serious regulatory breaches.  The Board
recommends the Carroll-Loye Biological Research report these deviations
to the IIRB as soon as possible and work with that organization to
develop and implement a corrective course of action. 

HSRB Consensus and Rationale

 were ethical violations. However while a majority of the Board agreed
that these violations did not rise to the level of substantial
noncompliance, some Board members believed it did meet that level.  Thus
a majority of the Board concurred with the initial assessment of the
Agency that the study submitted for review by the Board meets the
applicable requirements of §40 CFR 26, subparts K and L. 

	

F(1).  Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies
(SPC-001)

Charge to the Board

1.  If the proposed research described in Protocol SPC-001 from
Carroll-Loye Biological Research is revised as suggested in EPA’s
review, does the research appear likely to generate scientifically
reliable data, useful for assessing the efficacy of the test substances
for repelling mosquitoes? 

Board Response

The active ingredient Picaridin in three formulations will be tested in
the field for its ability to repel mosquitoes by the Carroll-Loye
company.  The active ingredient will be formulated inttested aso 1) a 7%
pump spray, with results bridged to  that will also be used for a 10%
pump spray, an aerosol and a 5.75% towelette; 2) a 15% pump spray, with
results bridged to  that will also be used for a 15% aerosol and a 12%
towelette; and 3) a 15% lotion that will also contain a sunscreen.  All
experiments will be conducted using GLP.  A dosimetry experiment with 10
subjects, each applying all three test formulations, will be performed
to determine the typical amount of product that would be utilized by
consumers using the product as directed; a single dosimetry experiment
will provide this information for both SPC-001 and SPC-002.   The grand
mean dose of all subject mean doses for each test formulation will be
applied by technicians to the subjects in the efficacy experiment.  A
dosimetry experiment with 10 individuals will be performed to determine
the amount of product that would be utilized by people using the product
as directed.  The experiment will be a field study.  Two locations in
California would be used.

Legs and/or arms will be tested.  There will be two experienced persons
serving as negative controls (i.e., without any repellent product) to
confirm mosquito biting/landing pressure. Experimental subjects, in
pairs, will monitor LIBe’s during a one minute interval each 15
minute, until the First Confirmed LIBe (FCLIBe) can be determined.
Stopping rules will be employed. The Complete Protection Time (CPT) will
be determined.

With respect to the pertinent science criteria established earlier by
the HSRB for completed studies: 

The scientific question was stated (i.e., to test the efficacy of
Picaridin in several formulations to repel mosquitoes).

Because existing data were not adequate to answer the question of
efficacy, new studies involving human subjects are necessary.

The risks are minimal because the active ingredient is of very low
toxicity, the other formulation ingredients are of very low toxicity,
the mosquitoes will be aspirated before they have an opportunity to
bite, and the regions selected will not have evidence of West Nile or
encephalitis viruses.

Deficiencies related to information about the lotion formulation have
been addressed.  

Study Design Criteria

The purpose of the study is clearly defined (i.e., efficacy testing).

There are specific objectives (i.e., to determine the CPT that Picaridin
in three formulations displays as a mosquito repellent).

The sample size will be 10 individuals per product along with 2
experienced individuals to confirm mosquito biting pressure.  A
dosimetry experiment prior to the field experiment will quantify the
amount of repellent being used.

The Board had some concern that with respect to data interpretation, a
stronger solution would be used to estimate CPT for the lower
concentration towelettes.  However, it was explained during the meeting
that the concentration quoted for the towelette also included the weight
of the fabric, and the picaridin solution was identical to the spray. 

 

Participation Criteria:

The participants will be representative of a subset  of the population
of concern; however, there are others in the population unlike these
participants who are likely to use these products, but it would either
be unethical to test them or would be less appropriate to test them. 
The participating population is considered appropriate and reasonable.

Measurement Criteria

 On the other hand, if the dosimetry phase demonstrates the towelette
administers approximately the same dose, extrapolation would be
acceptable. Quality assurance will be a part of the experimental plan. 
If the products will not be compared, randomization is less of an issue
if the experiments can be performed on separate days, however
randomization and allocation of subjects to treatments needs to be
fully specified.

The measurements will be accurate and reliable.

The measurements will be appropriate to the question being asked.

Quality assurance will be a part of the experimental plan.

Laboratory and Field Conditions

Laboratory experiments are not proposed, except for the dosimetry

Field experiments will be appropriate.

The study will include a stop rule plan, medical management plan, and a
safety monitor.

	HSRB Consensus and Rationale

Charge to the Board

b. If the proposed research described in Protocol SPC-001 from
Carroll-Loye Biological Research is revised as suggested in EPA’s
review, does the research appear to meet the applicable requirements of
40 CFR part 26, subparts K and L?  

Board Response

Brief Overview of the Study

The research is to be conducted by Carroll-Loye Biological Research, a
private laboratory in Davis, California by using healthy volunteers and
a controlled environment. The study sponsor is Spectrum Brands, Inc. of
Bridgeton, Missouri, a division of United Industries Corporation.  The
study protocol was reviewed and approved by a commercial human subjects
review committee, IIRB of Plantation, Florida. Minutes of IIRB meetings
were provided to the EPA as a separate document, and documentation
previously provided to the EPA by IIRB indicates that it reviewed this
protocol pursuant to the standards of the Common Rule (45 CFR Part 46,
Subpart A).

The research protocol submitted consisted of two interdependent studies:
1) a dosimetry study designed to determine the amount of an
insect-repelling compound, known as picaridin, that normal subjects
would typically apply when provided with one of three compound
formulations (7% Picardin Pump Spray [data will be bridged to 10% Pump
Spray and Aerosol and 5.75% Towelette],15% Picaridin Pump Spray [data
will be bridged to 15% Aerosol and 12% Towelette], and 15% Picaridin
Lotion formulated with sunscreen); and 2) an efficacy study designed to
measure the effectiveness of picaridin as a mosquito repellent. 
Dosimetry will be determined either by passive patch dosimetry (spray
formulations) or by direct measurement of compound application (lotion
formulation).  The efficacy of each formulation as a mosquito repellent
will be determined by measuring the ability of the formulations to
prevent mosquito landings (defined as “Lite with Intent to Bite”;
LIBe) under field conditions. Mosquitoes will be aspirated mechanically
prior to biting; prior to initiation of the efficacy study, all
volunteers will be trained, using laboratory-raised, pathogen-free
mosquitoes in a controlled laboratory setting, both to recognize a
mosquitoes landing with the intent to bite and to remove such mosquitoes
with an aspirator using laboratory-raised, pathogen-free mosquitoes in a
controlled laboratory setting.  During the field studies, participants
will work in pairs to facilitate identification and aspiration of LIBing
mosquitoes during brief exposure periods.  The strengths and weaknesses
of each study design are described above.

The dosimetry study will enroll a total of 10 subjects, each of whom
will test all three formulations.  The efficacy study will enroll 10
subjects per test formulation per field site, for a total of 30
subjects. Efficacy will be tested independently at two different sites,
representing two different environments.  Two controls (described as
“experienced personnel” and who were untreated with repellent) will
also participated to determine ambient LIBe pressure at each field site,
giving a cumulative total of 28 volunteers.  Subjects may participate in
either or both studies, making the total number of volunteers enrolled
no less than 32 but no greater than 74.  In addition, three alternate
subjects will be enrolled to: 1) replace any subject who withdraws from
participating; and 2) protect the confidentiality of any subject
excluded from the study as a result of pregnancy or other potentially
stigmatizing condition, as described below.

Critique

The Board concurred with the factual observations of the strengths and
weaknesses of the study, as detailed in the EPA’s Science and Ethics
Review (Carley and Sweeney, 2007a). The proposed research described in
Protocol SPC-001 comports withmeets the applicable requirements of 40
CFR Part 26, subparts K and L. 

The risks to study participants are minimal and justified by the likely
societal benefits, including data on the efficacy of picaridin as a
mosquito repellent.  The risks to study participants are three-fold: 1)
reaction to test materials themselves; 2) exposure to biting arthropods;
and 3) possible exposure to arthropod-borne diseases.

 and is present at similar concentrations in other EPA-registered
products; specifically, picaridin is registered and marketed as an
insect repellent in the United States under the registered trade name
BayrepelTM and the brand name Autan.  As volunteers with known allergic
reactions to insect repellents and common cosmetics are excluded from
participating in this study, enrolled participants are unlikely to be at
increased risk of experiencing adverse side effects upon exposure. 
Clear stopping rules also have been developed, as have plans for the
medical management of any side effects or adverse events associated with
product exposure. 

Reactions to mosquito bites are usually mild and easily treated with
over-the-counter steroidal creams.  The study also excluded individuals
who have a history of such severe skin reactions to further minimize the
risk of a participant experiencing a severe physical reaction to a
mosquito bite.  In addition, the study protocol was designed
specifically to minimize the likelihood that a mosquito will bite.  All
volunteers will be trained both to recognize LIBe’s mosquitoes landing
with intent to bite and to remove such mosquitoes with an aspirator
prior to bitingbefore they bite. Risk of bites is further minimized
through the use of clear stopping rules, limited exposure periods, and
paired observation.

To minimize the risk that study participants will be exposed to
illnesses like WNV, the study protocol calls for field tests of
repellent efficacy to be conducted only in areas where known
vector-borne diseases have not been detected by county and state health
or vector/mosquito control agencies for at least one month.  Mosquitoes
will be collected during the field studies and subjected to serologic or
molecular analyses to confirm that they were free of known pathogens. 

Finally, the study protocol also includes several mechanisms designed to
minimize coercive recruitment and enrollment, compensation does not
appear to be so high as to unduly influence participation, and minors
and pregnant or lactating women are explicitly excluded from
volunteering (pregnancy being confirmed by requiring all female
volunteers to undergo a self-administered over-the-counter pregnancy
test on the day of the study).  In reviewing similar protocols submitted
by Carroll-Loye Biological Research at previous HSRB meetings, for
example, the Board has expressed concern about the potentially coercive
nature of study subject recruitment.  Although the study is to be
conducted by Carroll-Loye Biological Research, a private research
laboratory in Davis, California, the Principal Investigator of the study
and Co-Owner of the research laboratory, Dr. Scott P. Carroll, also is
an adjunct faculty member of the Department of Entomology at the
University of California, Davis.  As the majority of research
participants will be recruited from the University’s student
population, including from Dr. Carroll’s own department, the Board
previously recommended that the protocol and consent documents be
altered to define clearly the mechanisms in place to prevent coercion. 
The current protocol acceptably minimizes the potential for coercive
subject recruitment or enrollment through includes several mechanisms,
including the exclusion of any student or employee of the Study
Director, a substantial waiting period between recruitment and study
enrollment, and an interview by Dr. Carroll, designed to minimize
coercive subject recruitment and enrollment.

In accordance with the EPA’s final human studies rule, 40 CFR §§
26.1701-17043,  minorschildren and pregnant or nursing women are
explicitly excluded from participation.  , the latter being confirmed by
requiring aAll female volunteers are required to undergo a
self-administered over-the-counter pregnancy test on the day of the
study.  In reviewing previous protocols, the Board has previously raised
concerns about the potentially stigmatizing nature of a positive test,
and recommended that investigators develop additional protections to
ensure that the results of over-the-counter pregnancy tests would be
kept private.  The use of so-called “alternate” subjects is one
suchan appropriate  safeguard to minimize the risks of stigmatization
resulting from a positive pregnancy test;; that study participants may
be designated as alternate subjects and automatically thereby excluded
from participation allows for potentially volunteers found to be
pregnant volunteers to withdraw without compromising their
confidentiality.

HSRB Consensus and Rationale

The Board concurred with the initial assessment of the Agency that the
revised protocol submitted for review by the Board meets the applicable
requirements of §40 CFR 26, subparts K and L. 

F(2).  Proposed Carroll-Loye Picaridin Insect Repellent Efficacy Studies
(SPC-002)

Charge to Board

1.  If the proposed research described in Protocol SPC-002 from
Carroll-Loye Biological Research is revised as suggested in EPA’s
review, does the research appear likely to generate scientifically
reliable data, useful for assessing the efficacy of the test substances
for repelling ticks? 

Board Response

The active ingredient Picaridin in three formulations will be tested in
the laboratory for its ability to repel ticks by the Carroll-Loye
company. The active ingredient will be formulated intotested in 1) a 7%
pump spray, with results bridged to  that will also be used for a 10%
pump spray, an aerosol and a 5.75% towelette; 2) a 15% pump spray, with
results bridged to  that will also be used for a 15% aerosol and a 12%
towelette; and 3) a 15% lotion that will also contain a sunscreen. 
These are the same formulations as would be tested in protocol SPC-001
(to test repellency against mosquitoes). All experiments will be
conducted using GLP..  A dosimetry experiment with 10 subjects, each
applying all three test formulations,individuals will be performed to
determine the typical amount of product that would be utilized by people
consumers using the product as directed; a single dosimetry experiment
will provide this information for both SPC-001 and SPC-002.   this The
grand mean dose of all subject mean doses for each test formulation will
amount will be applied by technicians to the subjects in the efficacy
experiment.  Subjects will be trained in the procedures.  One of each
subject’s arms will serve as a negative control to validate each
experimental tick for its questing behavior.

Ticks will be laboratory reared American dog ticks (Dermacentor
variabilis) and deer ticks (Ixodes scapularis).  Because these are
laboratory reared, the ticks are anticipated to be disease-free.

This protocol is very similar to protocols on tick repellency submitted
by this investigator in the past, using the metric of repellency from
the treated area on the arms of the subjects during a 3 minute
observation period during each 15 minutes of the test.  The Complete
Protection Time from treatment until the First Confirmed Crossing will
be calculateddetermined for each subject in the efficacy phase.. 

With respect to the pertinent science criteria established earlier by
the HSRB for completed studies: 

The scientific question was stated (i.e., to test the efficacy of
Picaridin in several formulations to repel ticks).

Because existing data were not adequate to answer the question of
efficacy, new studies involving human subjects are necessary.

It is likely that the benefits would be realized because repellent
efficacy will be determined in carefully designed laboratory
experiments.

The risks are minimal because the active ingredient is of very low
toxicity, the other formulation ingredients are of very low toxicity,
the ticks should be removed before they have an opportunity to bite, and
the laboratory reared ticks will not possess any diseases.

Study Design Criteria

The purpose of the study is clearly defined (i.e., efficacy testing for
tick repellency).

There are specific objectives (i.e., to determine the Complete
Protection Time that Picaridin in three formulations displays as a tick
repellent).

The sample size will be 10 individuals per product. A dosimetry
experiment prior to the field experiment will quantify the amount of
repellent being used. 

It is anticipated that the findings from this study can be generalized
beyond the study sample.

Participation Criteria

The participants will be representative of some of the population of
concern, but are not representative of the entire population, as
acknowledged by the Investigator; however, since it would either be
unethical to test them or would be less appropriate to test them, the
participating population is considered appropriate and reasonable.

Measurement Criteria

The measurements of tick crossing will be accurate and reliable.

The measurements will be appropriate to the question being asked.

Quality assurance will be a part of the experimental plan.

Laboratory and Field Conditions

Laboratory experiments are proposed.

Field experiments are not proposed.

The EPA science review did not identify any questions or concerns that
needed to be addressed by the HSRB. As was true for protocol SPC-001,
the Board had some concern that, with respect to data interpretation, a
more concentrated product appeared to be proposed to estimate CPT for
the lower concentration towelettes.  However, it was explained during
the meeting that the concentration quoted for the towelette also
included the weight of the fabric, and the Picaridin solution was
identical to the spray. 

There were a number of concerns from a statistical perspective. The
protocol states in section 8.3.1 that “[s]ubjects will be assigned to
the treatment groups on the basis of a randomly assigned subject
number,” 1-10 to Lotion, 11-20 to 7% Pump, and 21-30 15% Pump. 
However, it also states in section 8.3.2 that “individual subjects may
test more than one repellent, on separate days.”  These two statements
are inconsistent as the former statement implies 30 distinct and unique
subjects, whereas the latter implies an overlap, which may render the
statistical interpretation of the data very difficult, depending on how
the experiment is carried out.  If there is going to an overlap of
subjects among test groups, an appropriate experimental design needs be
employed to allow proper statistical inference.  It is also unclear what
is meant by the sentence “The experiment will be partially randomized
by subjects” in section 8.2 on experimental design.

As for the method of analysis, the statement that “Kaplan-Meier
analyses provide median estimates with substantially reduced error
estimates” is incorrect as it can go either way.  One cannot make a
direct comparison between the median and its 95% confidence interval
based on the Kaplan-Meier estimate of survival function and the mean and
the 95% confidence interval based on the normal theory.  Also the
statement that the median based on Kaplan-Meier method is “less
sensitive to data censoring” is incorrect.

 on the following items:

Quantification of efficacy of the test materials is inappropriate as the
normality assumption is inadequate and due to potential censoring.

Discussion of the statistical power is irrelevant as there is no
statistical test of hypotheses is involved.

Justification of the sample size for the repellency phase is not
scientific and thus inadequate.

The Carroll-Loye Protocol SPC-002 7/10/07 document contained the Site
Questionnaire (pp 63-66 of 70) and the Study Specific Instructions (p 67
of 70) for SPC-001, which must be in error, as they refer to mosquitoes,
instead of ticks. There appears to be discrepancies in the Material
Safety Data Sheets (MSDS) in the above document: On pp 59 and 61 the EPA
Registration Number is identical (121-92), but Product Item Numbers are
different (53667 on p 59 and 53661 on p 61) for the identical
formulation. There did not seem to be a MSDS included for the
formulation with sunscreen.

The IRB is universally understood as an acronym for “Institutional
Review Board.”  Therefore, reference to “Independent Investigational
Review Board” as “Independent IRB” is inappropriate and
misleading.

HSRB Consensus and Rationale

  

The Board urges EPA to consider the design of newer studies and the
designs already used for existing products to make certain that labels
reflect information of comparative value to consumers.  If there are
currently inconsistencies in the information used to authorize label
information on efficacy, the Board urges EPA to develop new guidelines
or revisions to the current guidelines that will balance accuracy of
data collected, safety to human subjects and consistency in the ultimate
labels produced.

Charge to Board

2.  If the proposed research described in Protocol SPC-002 from
Carroll-Loye Biological Research is revised as suggested in EPA’s
review, does the research appear to meet the applicable requirements of
40 CFR part 26, subparts K and L?  

Board Response

Brief Overview of the Study

The research is to be conducted by Carroll-Loye Biological Research, a
private laboratory in Davis, California by using healthy volunteers and
a controlled environment.  The study sponsor is Spectrum Brands, Inc. of
Bridgeton, Missouri, a division of United Industries Corporation.  The
study protocol was reviewed and approved by a commercial human subjects
review committee, Independent Investigational Review Board, Inc. (IIRB)
of Plantation, Florida. Minutes of IIRB meetings were provided to the
EPA as a separate document, and documentation previously provided to the
EPA by IIRB indicates that it reviewed this protocol pursuant to the
standards of the Common Rule (45 CFR Part 46, Subpart A).

 that moving insects would migrate  into the treated area.  The
strengths and weaknesses of each study design are described above.

The dosimetry study will enroll a total of 10 subjects, each of whom
will test all three formulations. The efficacy study will enroll 10
subjects per test formulation.  Each subject will serve as his/her own
control.  Participants in the dosimetry study may or may not participate
in the efficacy study, and participants in the efficacy study may also
test different formulations on different days; this makes the total
number of volunteers enrolled in both the dosimetry and efficacy studies
no less than 10 but no greater than 40. In addition, three alternate
subjects will be enrolled to: 1) replace any subject who withdraws from
participating; and 2) protect the confidentiality of any subject
excluded from the study as a result of pregnancy or other potentially
stigmatizing condition, as described below.

Critique

The Board concurred with the factual observations of the strengths and
weaknesses of the study, as detailed in the EPA’s Science and Ethics
Review (Carley and Sweeney, 2007b). Once the recommended changes
outlined therein are incorporated into the protocol, the proposed
research described in Protocol SPC-002 should comport withmeet the
applicable requirements of 40 CFR Part 26, subparts K and L. 

The risks to study participants are minimal and justified by the likely
societal benefits, including data on the efficacy of these
picaridin-based formulations as a tick repellent. The risks to study
participants are three-fold: 1) reaction to test materials themselves;
2) exposure to biting arthropods; and 3) possible exposure to
arthropod-borne diseases.

and is present at similar concentrations in other EPA-registered
products; specifically, picaridin is registered and marketed as an
insect repellent in the United States under the registered trade name
BayrepelTM and the brand name Autan.  As volunteers with known allergic
reactions to insect repellents and common cosmetics are excluded from
participating in this study, enrolled participants are unlikely to be at
increased risk of experiencing adverse side effects upon exposure. 
Clear stopping rules also have been developed, as have plans for the
medical management of any side effects or adverse events associated with
product exposure. 

The risks of bites are negligible and minimized by the study design;
tick questing and biting behavior is slow, and study participants are
trained to remove ticks from their forearms prior to biting.

The ticks used for the study are bred and raised in a laboratory
environment and are considered to be pathogen-free, minimizing the risk
of vector-borne disease.  The Agency noted in their review (2007b) that
the tick-borne rickettsial illness Rocky Mountain Spotted Fever (RMSF)
has been passed within tick colonies through a transovarian transmission
mechanism, so the laboratory colonies in question should either be
tested for RMSF prior to initiation of SPC-002, or the consent form and
protocol altered accordingly to reflect the low but not negligible risk
posed to trial participants.

The study protocol also includes several mechanisms, similar to those
described above in the Board’s review of Study Protocol SPC-001,
designed to minimize coercive recruitment and enrollment, compensation
does not appear to be so high as to unduly influence participation,
minors children and pregnant or lactating women are explicitly excluded
from volunteering (pregnancy being confirmed by requiring all female
volunteers are required to undergo a self-administered over-the-counter
pregnancy test on the day of the study), and the use of so-called
“alternate” subjects allows for volunteers to withdraw or be
excluded without compromising their confidentiality. The Board’s only
concern about subject recruitment was the unjustified exclusion of study
participants older than 55 years of age; most Board members felt that a
clear rationale for excluding older volunteers should be provided, or
the exclusion criteria changed.

HSRB Consensus and Rationale

The Board concurred with the initial assessment of the Agency that the
revised protocol submitted for review by the Board meets the applicable
requirements of §40CFR26, subparts K and L. 

G.  Proposed ICR Picaridin Insect Repellent Efficacy Study (A 117)

Charge to the Board

1.  If the proposed research described in ICR’s proposed picaridin
protocol is revised as suggested in EPA’s review, does the research
appear likely to generate scientifically reliable data, useful for
assessing the efficacy of the test substances for repelling mosquitoes
of the genus Culex? 

Board Response

This protocol presented by ICR was designed to determine in laboratory
studies whether two Picaridin-containing products will repel mosquitoes
of the genus Culex. Thisese products are is already registered and the
study, if showing efficacy of the products in repelling Culex, would
allow claims for repellency of this genus to be added to the label. 
This is important because Culex is one of the major carriers of West
Nile virus, and efficacy against this genus would be of interest to
consumers.  The particular products to be tested are Avon Skin-So-Soft
SSS Bug Guard Plus Picaridin Insect Repellent and Avon Skin-So-Soft SSS
Bug Guard Plus Picaridin Insect Repellent spray.  The hypothesis was
that 8 hours of protection against Culex quinquefasciatus, a species
known to be a vector for West Nile virus, would be demonstrated by the
two products.

The protocol was clearly written.  Laboratory reared insects that were
disease free would be used. The protocol proposed the use of the
guideline amount of repellent, not an amount determined by a dosimetry
study, and would monitor bites, not landings, in order to be consistent
with the methods used to obtain the label currently borne by these
products.  The negative control would use only landings, avoiding bites
as much as possible, and would be used to guarantee that a landing rate
of at least 5 in 60 seconds was maintained; if the landing rate dropped
below this, more mosquitoes would be added to the cages. 

This protocol appeared to follow guideline procedures, and data from a
study such as this would be necessary to modify label claims to include
efficacy against Culex, particularly if Culex was not represented or was
not represented well in the original field studies, which presumably was
the case. The procedures seem to be straightforward.  Pre-testing the
subjects for attractiveness to Culex should yield more consistent
results than if such pretesting was not done. 

There were a few points of confusion.  The reference to dose range
finding (pg. 27) was not clear, particularly when the standard guideline
dose level was proposed. It is unclear whether confounding from the
subject’s other arm containing a different product, or the presence of
the arms of the second subject in the cage would occur.  However both
subjects would use the same two products and both products would contain
the same active ingredient, i.e., Picaridin, so there is probably
little, if any, confounding likely. There was a question as to whether
the laboratory temperature and humidity would be similar to that
occurring in the field.  There was also a question of how, with an 8
hour limit to the test, an average of 8 hours efficacy with a 2 hour
deviation might be obtained.  It was suggested that a Q test was not the
most appropriate statistical method, and that the Kaplan Meier test
might be the better approach.

Two of the aspects of this protocol are at variance with what the Board
had previously concluded regarding other insect repellency protocols,
and these deserve specific discussion.  One is the use of the standard
guideline dosage instead of a dosimetry test to determine the dosage to
be used.  While in a totally new study, the dosimetry test would yield a
dosage more representative of the amount of product that the consumer
would use, in this situation, this study is supposed to match a
previously conducted field study with the same products.  It is
therefore appropriate to use the same dosage as was used in the earlier
field study.  Likewise, the previous field study used bites. While the
Board has been impressed with the added safety to participants imparted
by the landing endpoint over the bite endpoint, these data would need to
match the data obtained in the previous field study, which used bites as
the endpoint.  In addition, the consultants indicated that landings did
not necessarily predict bites, leading to even more support for the use
of bites in order to match the earlier study with bites.

	HSRB Consensus and Rationale

The Board concluded that this study would be appropriate to confirm
efficacy against Culex, was designed to meet the current EPA guidelines,
and was generally clear and adequately designed, with the exception of
the appropriate statistics.  If revised consistent with EPA’s
recommendations and the Board’s suggestions, the study should yield
valid data regarding the efficacy of these products in repelling Culex. 

Charge to Board

2.  If the proposed research described in ICR’s proposed picaridin
protocol is revised as suggested in EPA’s review, does the research
appear to meet the applicable requirements of 40 CFR part 26, subparts K
and L?  

Board Response

Brief Overview of the Study

The proposed study (Spero, 2007) would evaluate the efficacy of two
different skin-applied lotion formulations of picaridin-based insect
repellents under laboratory conditions; these two formulations are Avon
Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent (EPA Reg. No.
806-29) and Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect
Repellent Spray (EPA Reg. No. 806-31.

The research is to be conducted by ICR, a commercial organization based
in Baltimore, Maryland; ICR provides testing and regulatory consulting
services for companies developing and marketing pesticides and
insecticides in the United States and Canada.  The study is managed by
toXcel, LLC of Gainesville, Virginia.  The sponsor of this study is Avon
Products, Inc. of New York, New York.  The submitted documents assert
that the study will be conducted in accordance with the ethical and
regulatory standards of 40 CFR 26, Subparts K and L, as well as the
requirements of FIFRA §12(a)(2)(P), and the U.S. EPA’s GLP Standards
described at 40 CFR 160. The protocol was reviewed and approved by an
independent human subjects review committee, Essex Investigational
Review Board (EIRB), Inc., of Lebanon, New Jersey prior to submission to
the Agency. 

Efficacy of the two picaridin-based formulations will be evaluated by
using healthy volunteers. The study will be performed at ICR’s
laboratory in Baltimore, Maryland, and test the effectiveness of the two
compounds as mosquito repellents by measuring the ability of each
formulation to prevent mosquito bites under laboratory conditions.  The
strengths and weaknesses of the study design are described above.

The effectiveness study will enroll a total of 13 subjects.  Of the 13
participants, twelve will be treated and test the effectiveness of the
two picaridin-based repellent formulations.  The compounds will be
applied to 250 cm2 patches bands of skin on the forearms of each study
participant; one compound will be applied to the right forearm and one
to the left forearm, with the effectiveness of each formulation
simultaneously evaluated.  Treated skin will be exposed to mosquitoes
for five minutes at half-hour intervals by having study participants
insert their forearms into 8 ft.3 test “cages” containing 100
laboratory-reared, pathogen-free female C. quinquefasciatus.  The
efficacy of each repellent formulation will be ascertained by measuring
the time from application to "breakdown" of repellency, with
“breakdown" defined in the protocol as an initial bite confirmed by
either two confirmed bites a second bite within a singlethe same
five-minute exposure period, or by a second bite in the next one bite in
each of two consecutive exposure periods.  Treated study participants
will work in pairs, observing mosquito landings and alerting attendant
ICR staff of potential bites; ICR staff will determine whether or not
these are confirmed indeed bites. Probes (i.e., "bites" where the
mosquito punctures the skin but does not collect blood) and bites from
mosquitoes that do not fully alight (i.e., all six legs) on the surface
of the treatedexposed skin) will not be considered as confirmedcounted
as bites.  The cycle of intermittent exposures will continue for each
subject until Once breakdown is reached for both arms.  Once the
repellent on one arm has reached breakdown has occurred for a particular
repellent formulation, no subject, no further exposure of thate
subject's arm treated skin will occur.  Exposures of the subject’s
other arm will continue until breakdown of the other repellent.  The
study protocol justifies the enrollment of twelve treated participants
by stating that ten volunteers are needed to obtain statistical
validity; an additional two participants will be enrolled as alternates
to “allow for drop outs” (Spero and Gaynor, 2007).

One study participant, chosen by lottery (a “coin toss”), will
remain untreated and will be monitored to determine mosquito-biting
pressure under laboratory conditions.  A 250-cm2 patch band of untreated
skin will be exposed in each test cage for five minutes at half-hour
intervals, with the ambient biting pressure determined by measuring the
number of mosquitoes landing on the skin.  A minimum rate of 5 landings
per minutewithin one minute is necessary for the laboratory trial to be
conducted or continued.  Landing mosquitoes will be “shaken” away by
the study participant while ICR staff counts the number of landings.  If
the untreated control subject experiences fewer than 5 landings in one
minute, fresh mosquitoes will be added to the cage.  

Critique

The supporting documentation provided by the study investigators,
sponsor and EIRB, as submitted to the Agency, appears to meet the
regulatory requirements of 40 CFR 26.1115a and 40 CFR 26.1125. A
description of EIRB procedures was provided to the EPA with a claim of
confidentiality, so were was not available for review by the HSRB.
Agency staff, however, reviewed the documentation provided by EIRB and
determined these procedures and policies to be in compliance with the
applicable standards of the Common Rule (40 45 CFR 246, Subpart KA). The
protocol as submitted to the Agency thus is substantially compliant with
the regulatory requirements of review and documentation, minor
deficiencies not withstanding.

The Board concurred with the factual observations of the strengths and
weaknesses of the study, as detailed in the EPA’s Science and Ethics
Review (Carley and Sweeney, 2007c). Specifically, the Board agreed with
the Agency’s recommendations that ICR: 1) revise the current protocol
discussion of risks and benefits; 2) provide more information about
ICR’s subject recruitment and enrollment processes; and, 3) alter the
data collection form to remove all identifiable information.  Most Board
members disagreed, however, with the EPA’s recommendation that study
investigators provide a clearer justification for relying on time to FCB
as a measure of repellent efficacy.  Previous field studies using time
to FCB as an endpoint for these particular repellents efficacy have
already be submitted to and approved by the Agency.  These additional
laboratory studies are to be conducted solely for the purpose of
justifying additional label claims as per Agency requirements.  For
consistency, use of time to FCB, using pathogen-free colonies of
mosquitoes under controlled conditions, is justified. 

Once these changes are incorporated into the protocol, the proposed
research described in ICR A117 should comport withmeet the applicable
requirements of 40 CFR 26, subparts K and L. In brief, the risks to
study participants are minimal and justified by the likely societal
benefits, including data on the efficacy of these picaridin-based
formulations as ain repellent for ing one of the key mosquito genera
known to transmit WNV in the United States.  At first glance, tThe
potential risks to study participants are three-fold: 1) reaction to
test materials themselves; 2) exposure to biting arthropods; and, 3)
exposure to arthropod-borne diseases.

These two picaridin-based repellent formulations are commercially
available in the United States, and their non-repellent ingredients have
been used widely as in cosmetic and personal care products with little
evidence of toxic effects.  Volunteers with known allergic reactions to
insect repellents and common cosmetics are excluded from participating
in this study, and the amount of skin treated with picaridin is limited,
so enrolled participants are unlikely to be at increased risk of
experiencing adverse side effects upon exposure to the test materials. 
Clear stopping rules also have been developed, as have plans for the
medical management of any side effects or adverse events associated with
product exposure.  The Board did recommends, however, that the informed
consent documents be modified to more accurately represent the known
toxic risks associated with acute picaridin exposure, particularly via
ocular, oral and respiratory routes of exposure.

The endpoints of the study protocol requires a confirmed bite—i.e., an
initial bite confirmed by another bite within 30 minutes—to  two
confirmed mosquito bites to document breakdown of repellent
effectiveness.  Since two repellents will be tested concurrently, each
subject can expect to receive at least 4 bites.  Reactions to mosquito
bites are usually mild and easily treated with over-the-counter
steroidal creams; such a cream, in addition to CalomineTM and rubbing
alcohol, will be provided to study participants to alleviate minor
symptoms associated mosquito bites.  The study excludes individuals who
have a history of such severe skin reactions to further minimize the
risk of a participant experiencing a severe physical reaction to a
mosquito bite. 

The mosquitoes used for the study are bred and raised in a laboratory
environment and are considered to be pathogen-free, minimizing the risk
of vector-borne disease.  In the Agency’s Scientific Science and
Ethics review of the protocol (Carley and Sweeney, 2007b), a concern was
raised that the risk of arthropod-borne illnesses is not adequately
discussed in the informed consent document; the Agency recommended that
a discussion of these risks be added.  The Board agreedagrees.

The differential risks to the untreated control subjects (chosen by
lottery rather than via separate enrollment of more experienced study
participants) are listed in both the protocol and the informed consent
document.  The Board did noted, however, a misleading statement in the
consent document, which states that in which mosquitoes will be
“brushed [off] by ICR staff” from the untreated control subject’s
arms prior to biting.  The protocol statesd that the untreated controls
subject will “shake” landing mosquitoes off.  This inconsistency
must be corrected.

Furthermore, it is clear from the protocol that all study volunteers, be
they untreated controls or treated volunteers, will be asked to expose
both an untreated their arms, before they are treated, to the mosquito
colony before the efficacy study begins, in order to test
“attractiveness.”  The attractiveness component of the study is not
mentionedlisted in the informed consent document, and must be added. 
The informed consent document also lists one of the societal benefits of
the study as bringing a “new repellent to market.”  The study as
described, however, is designed to simply allow a labeling change.  The
repellent formulations under evaluation are already on the market.

A more detailed explanation of study recruitment is also needed; the
exact procedures for recruiting study participants are unclear.  As
currently written, however, compensation for study participation is not
so high as to unduly influence enrollment, and employees and
contractors—or family members of employees or contractors— of ICR,
toXcel and the sponsor (as well as family members) are excluded from
participation.  In accordance with the newly promulgated provisions in
the EPA’s final human studies rule (40 CFR §§ 26.17031-1704),
childrenminors and pregnant or nursing women are explicitly excluded
from participation., the latter being confirmed by requiring Aall female
volunteers are required to undergo a self-administered over-the-counter
pregnancy test on the day of the field study.  The use of two potential
“alternate” subjects allows for volunteers to withdraw or be
excluded without compromising their confidentiality. 

Finally, the Board was concerned that an overlooked risk of study
participation might be the strain associated with the physical
requirements imposed upon volunteers.  Each study participant will be
asked to spend a full day in a warm, humidified laboratory environment.
Although the likelihood of an adverse event precipitated by the physical
strains of study participation is small, these risks nevertheless should
be incorporated intomentioned in the informed consent document, and a
clear plan for medical monitoring and treatment should be articulated.

HSRB Consensus and Rationale

The Board concurred with the assessment of the Agency that the protocol
ICR A117 submitted for review by the Board, if revised as suggested in
both EPA’s review and by the Board, would meet the applicable
requirements of 40 CFR 26, subparts K and L. 

REFERENCES

Carley, J.M. 2007. Ethics Review of Reports of Completed Carroll-Loye
Field Efficacy Studies SCI-001 and WPC-001 for Mosquito Repellents.
Dated September 26, 2007.  Unpublished document prepared by Office of
Pesticide Programs, United States Environmental Protection Agency.

Carley, J.M., and K. Sweeney. 2007a. Science and Ethics Review of
Protocol for Human Study of Mosquito Repellent Performance. Dated
September 24, 2007.  Unpublished document prepared by Office of
Pesticide Programs, United States Environmental Protection Agency.

Carley, J.M., and K. Sweeney. 2007b. Science and Ethics Review of
Protocol for Human Study of Tick Repellent Performance. Dated September
24, 2007.  Unpublished document prepared by Office of Pesticide
Programs, United States Environmental Protection Agency.

Carley, J.M., and K. Sweeney. 2007c. Science and Ethics Review of
Protocol for Human Study of Mosquito Repellent Performance. Dated
September 24, 2007.  Unpublished document prepared by Office of
Pesticide Programs, United States Environmental Protection Agency.

Carroll, S. 2007a. Test of DermAegis LipoDEET 302 Personal Insect
Repellent: EPA Reg. #82810-1. Date unknown. Unpublished study prepared
by Carroll-Loye Biological Research under Project No. SCI-001.1.

Carroll, S. 2007b. Test of DermAegis LipoDEET 3434 Personal Insect
Repellent. Date unknown. Unpublished study prepared by Carroll-Loye
Biological Research under Project No. SCI-001.2.

Carroll, S. 2007c. Test of Coulston’s Duranon Personal Insect
Repellent (EPA Reg. #50404-8). Date unknown. Unpublished study prepared
by Carroll-Loye Biological Research under Project No. SCI-001.3.

Carroll, S. 2007d. Response to ‘Ethics Review of Reports of Completed
Carroll-Loye Field Efficacy Studies SCI-001 and WPC-001 for Mosquito
Repellents.’ Dated October 19, 2007. Unpublished document prepared by
Carroll-Loye Biological Research.

Carroll, S. 2007e. Test of an Oil Of Lemon Eucalyptus-Based Personal
Insect Repellent (EPA Reg. # 305-62). Date unknown. Unpublished document
prepared by Carroll-Loye Biological Research.

Carroll, S. 2007f. Efficacy Test Protocol SPC-001: Efficacy Test of
Picaridin-Based Personal Insect Repellents with Mosquitoes Under Field
Conditions. Dates July 13, 2007. Unpublished protocol prepared by
Carroll-Loye Biological Research.

Carroll, S. 2007g. Efficacy Test Protocol SPC-002 Efficacy Test of
Picaridin-Based Personal Tick Repellents. Dates July 10, 2007.
Unpublished protocol prepared by Carroll-Loye Biological Research.

EPA HSRB. 2007a. Draft Final HSRB Report on SCI-001 proposed studies.
Washington, DC: Environmental Protection Agency.

EPA HSRB. 2007b. Draft Final HSRB Report on WCI-001 proposed studies.
Washington, DC: Environmental Protection Agency.

Gammon, MD, Neugut, AI, Santella, RM, et al.  2002.  The Long Island
Breast Cancer Study Project:  Description of a Multi-Institutional
Collaboration to Identify Environmental Risk Factors for Breast Cancer. 
Breast Cancer Res Treat 74:235-54.

Mitchell, RK, Agle, BR, Wood, DJ.  1997.  Toward a Theory of Stakeholder
Identification and Salience:  Defining the Principle of Who and What
Really Counts.  Academy of Management Review 22(4):853-886.

Rogers, EM.  1995a.  Diffusion of Innovations, 4th ed. New York:  Free
Press, 1995.

Rogers, EM.  1995b.  Lessons for Guidelines from the Diffusion of
Innovations.  Joint Comm J Qual Improv 21:324-8.

Spero, N., and W. Gaynor. 2007. Protocol for Evaluating the Efficacy of
Personal Repellents Against Mosquitoes in the Laboratory Including
Supporting Materials Satisfying 40 CFR §26.1125 for Avon Skin-So-Soft
SSS Bug Guard Plus Picaridin Insect Repellent (EPA Reg. No. 806-29) and
Avon Skin-So-Soft SSS Bug Guard Plus Picaridin Insect Repellent Spray
(EPA Reg. No. 806-31). Dated August 8, 2007. Unpublished document
prepared by Insect Control & Research, Inc. 

Vandenbroucke JP, von Elm E, Altman DG, Gøtzsche PC, Mulrow CD, et
al. (2007) Strengthening the Reporting of Observational Studies in
Epidemiology (STROBE): Explanation and Elaboration. PLoS Med 4(10): e297
  HYPERLINK "http://dx.doi.org/10.1371/journal.pmed.0040297" 
doi:10.1371/journal.pmed.0040297 

von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, et al.
(2007) The Strengthening the Reporting of Observational Studies in
Epidemiology (STROBE) Statement: Guidelines for Reporting Observational
Studies. PLoS Med 4(10): e296   HYPERLINK
"http://dx.doi.org/10.1371/journal.pmed.0040296" 
doi:10.1371/journal.pmed.0040296 



The Human Studies Review Board (HSRB or Board) has discussed and
provided advice to EPA on scientific and ethical issues related to the
conduct of field studies to evaluate the efficacy of mosquito repellent
products.  The HSRB has reviewed both proposals for new field studies
and the results of completed studies.  The HSRB has noted that, although
there are many similarities across studies, not all studies employ the
same study design.  The HSRB has identified several methodological
issues for which additional background information would assist the
Board in its evaluation of such studies.  

BACKGROUND	

Currently, EPA requires all pesticide products that claim to repel
mosquitoes to provide data on the duration of efficacy under field
conditions at two biologically distinct sites.  These data are derived
from human research with subjects who have been treated with the
repellent formulations in the field.  The Agency evaluates the duration
of repellent efficacy for a subject by calculating the time from
application of the repellent to the occurrence of an event indicating an
efficacy failure.  Historically, for field studies of mosquito
repellency, EPA has used the “first confirmed bite” as an indication
of efficacy failure on a test subject.  Several recent studies have
shifted to the “first confirmed landing with intent to bite;” EPA
has accepted this alternative endpoint.  A “confirmed landing” on a
test subject is a mosquito landing followed by a second landing on the
same subject within a specified period of time (usually 30 minutes)
after the initial landing.  

Field studies typically involve 6 – 10 subjects who have been treated
with a defined amount of the test material.  Each subject is then
regularly and repeatedly exposed to ambient mosquito populations for a
fixed interval of time until the subject experiences an efficacy failure
followed by a confirmation with the specified period of time. Mosquito
landing pressure (representing intent to bite) at a site is monitored by
concurrently exposing untreated subjects to mosquito landings.  A study
is considered valid only if there are at least a specified minimum
number of mosquito landings on untreated subjects during each exposure
interval. 

On October 25, 2007, the HSRB will discuss scientific aspects of the
design of field studies to assess the efficacy of mosquito repellents. 
For this meeting the Board has requested consultants to provide
specialized information or assistance to the Board.   The Board is
particularly interested in the frequency, duration and timing of
exposure of subjects to potential mosquito landings.  The Board requests
each consultant to respond briefly to the series of questions below. 
Please send the responses to the HSRB Chair and Designated Federal
Official (DFO) at least one week before the meeting—i.e., by no later
than October 18.  All responses will subsequently be provided to the
other consultants, the HSRB members, and EPA staff for their review, and
will be posted on   HYPERLINK "http://www.regulations.gov" 
www.regulations.gov  under docket ID number, EPA-HQ-ORD-2007-0942.  HSRB
consultants will be available at the meeting to discuss their responses
and address questions from the Board.  The questions for Board
consultant
consider瑡潩⁮牡⁥牰癯摩摥戠汥睯›ࠍ䐍卉啃卓佉⁎啑
卅䥔乏൓

	What do data show about the variability of the time intervals
between first and subsequent landings in mosquito repellent field
trials?

	What is the current scientific understanding of how factors other
than repellent efficacy could affect the likelihood that an initial
event—a mosquito landing or mosquito bite—would be “confirmed”
by another similar event within 30 minutes?  Please address at least
these factors:

o	Characteristics of mosquito populations

o	Characteristics of test sites

o
Characteristi獣漠⁦整瑳猠扵敪瑣൳९桃牡捡整楲瑳捩⁳
景琠獥⁴敭桴摯൳

	Can the impact of such factors on the likelihood or timing of an
initial and confirming event be predicted?  Can it be quantified?   

At its June 27 - 29, 2007 meeting the Board learned that different
designs with different “length-biased” sampling for mosquito
repellent field studies are in use.  One design exposes subjects to
potential mosquito landings for one minute of every 15 minutes; another
design exposes subjects to potential mosquito landings for five minutes
of every 30 minutes.  The DFO is separately providing a CD containing
the background materials for the June 27 – 29, 2007 HSRB meeting.  The
protocols are loaded on the CD.  These designs have different
“length-biased” sampling.  

	What is the methodological rationale for the two different designs? 

	Which design is used more widely in the field? Why? 

	Can potential effects of variation in the pattern of intermittent
exposure on the results of efficacy testing be isolated from the effects
of other variables?  If so, can the direction or magnitude of the
effects be predicted?  How might these influences be analyzed and
accounted for in collecting, reporting and analyzing repellent efficacy
data?

Dr. Matt Kramer, a USDA statistician who has served as a consultant, has
suggested that the precision of estimates of Complete Protection Time
(CPT) in repellent testing could be significantly increased by defining
a failure of efficacy as the mean time from treatment to a series of
several landings or bites.  He has stated:

The precision of CPT increases when it is estimated beyond time to
[First Confirmed Bite] FCB or FCLanding.  How well CPT can be estimated
depends on the distribution of so many bites beyond FCB.  The number of
mosquitoes that will bite (n) will determine results of the test.  Each
person in the field should be his/her own control; that way it is
possible to know n per person, and reduce person-to-person variability.

If using the mean time to the first 5 bites, the SE will decrease
proportionally as n increases (n = 5 in this case).  That is equivalent
to an increase in the power of the test of 5 times.  This method allows
for detecting formulation differences near the CPT.

	Does this approach, indeed, increase the precision of estimates of
CPT markedly without requiring additional subjects?

	If so, would this increased precision justify the incremental risk
to the subjects resulting from their exposure to a great?

Is it practical to test long-lasting repellents to the point of five
landings?

Proposed Final Draft v.1 Dated January 10, 2008 Do Not Cite or Quote

Edits and comments by OPP February 5, 2008

  PAGE  5  of   NUMPAGES  86 

This change makes the second and third bullets parallel and tracks the
standard in the applicable regulation.

Please clarify relevance of this observation to this review.

There seems to be some confusion in the description of this term.  A
“confirmed” event is an event – such as a “landing with intent
to bite” (LIBE), bite, or crossing onto treated skin – followed by
another comparable confirming event within a specified period of time,
typically 30 minutes.  

Intermittent exposure in tests to establish FCB has become common in
recent years, as WNV has grown in importance as a public health concern.
 Testing to FCB remains uncommon in studies conducted for public health
purposes, as distinct from  EPA’s regulatory purposes.

OPP doesn’t think this came up at the meeting.  OPP is not familiar
with ICR’s full body of work, but their recently submitted proposed
protocols involved continuous exposures.

OPP is unclear about the import of this sentence.  Our understanding is
that there are no existing scientific data, much less models, available
to quantify or predict the impact of most if not all of the identified
variables on the likelihood and timing of mosquito bites under field
conditions.  We agree that it would be possible to design research to
develop such data and models. 

See comment W3.

See comment W3.

Please explain the relationship between sample size and the number of
bites per subject.

Please clarify these points.

OPP suggests the edit to make this part of the report more consistent
with the conclusion stated below on lines 18-19 that this deviation did
not affect the validity of the results.

In the interest of fairness, it might be worth mentioning others.

Please clarify

This conclusion is much narrower than is suggested by the previous
discussion of the many deficiencies of the statistical design and
analysis.  It would be helpful if the statistical discussion
acknowledged that the Board reviewed this protocol, generally favorably,
in January 07, and focused more on how the design and/or analysis
deviated from expectations.

OPP thinks that the chronology of events is clear.  The protocol and
consent form were revised in response to EPA’s review and approved by
IIRB in January.  These revised versions were provided to the HSRB for
review.  The protocol was not further revised after the HSRB meeting
until just days before it was executed.  These amendments were not
submitted to or approved by the IIRB

Is this a carryover from the protocol review?  There’s no discussion
of strengths above, and the discussion of weaknesses is focused
exclusively on statistics.

The issue, addressed below, is with the interpretation of these
criteria.  In light of that discussion, it is probably better not to say
here that the criteria are “clear”.

The Board’s conclusion reported above is that the use of subjects in
the study who had been treated on the previous day did not adversely
affect the scientific validity.

This is much too strong wording.  There was no decision to
disregard—as is acknowledged below, Dr. Carroll’s interpretation was
that the exclusion criterion was to be applied to uncontrolled exposures
to repellents during the day before beginning the test.  That
interpretation is plausible.  The Board’s recognition of uncertainty
is not the same as the investigator’s “decision to disregard
previously established exclusion criteria.”

See comment J11.

This explanation leads to an apparent inconsistency.  With respect to
SCI-001, on p. 54, lines18-19, the report says indicates that the
treatment of subjects on successive days “did not affect the validity
of the results.”  This is the same conclusion as is reached for
WPC-001, and cannot be the basis for saying one is scientifically
unacceptable, but the other is not.  As OPP understood the Board’s
discussion, SCI-001 is unacceptable because the execution of the
protocol produced results that could not be analyzed to evaluate CPT.  

See comment J21.

See comment  J16.

It would be better to use the language of the regulation here—i.e.,
“was in substantial compliance”

The deleted criterion is applicable only to pre-rule study review.  This
is a post-rule study, and the criterion of acceptability is
“substantial compliance with subparts A through L”.

All changes to the protocol subsequent to the HSRB review were reviewed
and approved by the IIRB

Please clarify this sentence.  The record contains Dr. Carroll’s
explanation that he interpreted this exclusion criterion to apply to
uncontrolled repellent exposures before the first day of the test.  In
both science reviews the Board has concluded that this didn’t affect
the results.  Even if it were concluded to have been a deviation from
the protocol, it does not seem to be a “significant change to the
study protocol.”

EPA knows of no such failure.  See comments J26 & J27.

Please clarify what protocol deviations you mean.  If this criticism is
retained, consider whether a similar comment should be made about
SCI-001.

Criterion is “substantial compliance with Subparts A through L”. 
This applies only to pre-rule research.

This is a very serious allegation and is very forcefully expressed.  OPP
thinks it should be deleted because we know of no failure in this case
to obtain IRB approval.  

-

.

8

<

?

R

ø

µ

¶

摧缁

摧矞sԀ

y

摧ᔟ¿

$

„@

„Ð^„@

 hqj

  hqj

ÿᔀ

j;

j¸

j/

ༀ킄㜂$␸䠀$葞ː摧ᔟ¿	㜀$␸䠀$摧ᔟ¿

萑ː葠ː摧ᔟ¿

 hÏN

j¬

萑ː葠ː摧ᵾÙ

F

F

F

:

;

ú

 h`

	



态ú摧ᔟ¿

ᄀ킄㜂$␸䠀$葠ː摧᭦x	㜀$␸䠀$摧ᔟ¿

ༀ킄㜂$␸䠀$葞ː摧ᔟ¿

+

,

-

.

/

0

M

N

T

Y

€

ƒ

.

0

M

N

ƒ

î

ô

ú

-

%脈栖ᔟ¿栗➐

桤뮵曁ē脈䠄栅뮴曁栖➐

%脈栖ᔟ¿栗➐

萑ː葠ː摧牪e

)脈栖ᔟ¿栗➐

桤뮰曁/脈栕放Ð栖ᔟ¿栗➐

桤뮰曁ė脈䠄栅뮯曁栖➐

/脈栕放Ð栖ᔟ¿栗➐

ė脈䠄栅뮮曁栖➐

/脈栕放Ð栖ᔟ¿栗➐

桤뮲曁ė脈䠄栅뮲曁栖➐

)脈栖ᔟ¿栗➐

桤뮮曁/脈栕放Ð栖ᔟ¿栗➐

桤밀曁ĝ脈䠄栅뮱曁栕放Ð栖➐

/脈栕放Ð栖ᔟ¿栗➐

桤뮱曁ė脈䠄栅뮱曁栖➐

)脈栖ᔟ¿栗➐

桤뮰曁/脈栕放Ð栖ᔟ¿栗➐

ė脈䠄栅뮰曁栖➐

/脈栕༙(栖ᔟ¿栗➐

È

É

ē脈䠄栅뮵曁栖➐

%脈栖ᔟ¿栗➐

桤뮶曁ē脈䠄栅뮶曁栖➐

%脈栖烠;栗➐

ē脈䠄栅뮴曁栖➐

%脈栖ᔟ¿栗➐

桤뮴曁ē脈䠄栅뮳曁栖➐

%脈栖ᔟ¿栗➐

T

d

‰

“

Å

Ç

ê

ë

ē脈䠄栅뮷曁栖➐

%脈栖ᔟ¿栗➐

ì

hÆ

,脈栖㽨

hÆ

态ú摧ᔟ¿

฀࿾뒄㜀$␸䠀$葝ﻠ葞´摧ᔟ¿
฀䲄࿿뒄ᄀᲄ㜂$␸䠀$葝ｌ葞´葠Ȝ摧ࢷÆ

  hÇ

ༀ梄㜁$␸䠀$葞Ũ摧ᔟ¿	㜀$␸䠀$摧ᔟ¿

ༀ뒄㜀$␸䠀$葞´摧ᔟ¿

ഀ׆Āːༀᲄ㜂$␸䠀$葞Ȝ摧ᔟ¿

ༀ梄㜁$␸䠀$葞Ũ摧ᔟ¿

ༀ蒄ᄃ预㟾$␸䠀$葞΄葠ﺘ摧ᔟ¿

态ú摧ᔟ¿

hÆ

hÆ

hÆ

hÆ

hÆ

hÆ

Mis at odds with the Board’s earlier discussions, in which it has
repeatedly stated that subjects were not put at risk.

See comments J26, J27, W31 & W32.

Since all test products are already registered, the results of the study
can only be to permit current products to remain in the market.

OPP notes that this information is unlikely to change from one study to
the next, and that there is only limited information available to
characterize the impacts of these variables on test measures (see
comment W6 and accompanying text).  Therefore we question whether it is
necessary for inclusion in a protocol?

Other comments suggest this “problem” has been resolved.

Please clarify

In OPP’s view, this protocol is very similar to earlier protocols from
the same investigator which the Board found acceptable.  Therefore we do
not understand the basis for the Board’s position.

All three repellent formulations are currently registered.

See comment J 39.  The results of the study can only be to permit
current products to remain in the market.

The protocol explains that the mean will be provided because EPA calls
for it in the guideline, and that K-M analysis will also be used to
better deal with censored data.  Does the Board think this is a problem?
 Please clarify.

This seems inconsistent with the HSRB consensus stated below.

It is clear in the protocol (p. 31 of 70) that “no statistical
comparisons are planned.”  What is the Board asking for here?  Also
see p. 72 lines 42-43.

No measurement of speed or distance is proposed.  Measurements are of
crossings, defined as crossing and remaining at least 3 cm into the
treated area of skin within 3 minutes, or repulsions.

See comment J 39.