Document ID: EPA-HQ-ORD-2007-0942-0022
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2007-10-24T04:00Z

October 16, 2007 

MEMORANDUM 

SUBJECT: 	Consultant response to United States Environmental Protection
Agencies (US EPA) Human Studies Review Board (HSRB). 

FROM: 	Steve Schofield, PhD. 

Senior Advisor - Public Health Entomology, 

National Defence, Canada.

 

TO: 		US EPA HSRB

		c/o Paul Lewis, PhD.

		Executive Director,

		HSRB

		Office of the Science Advisor

		US EPA  

BACKGROUND:

The HSRB of the EPA has identified several methodological issues related
to mosquito repellent testing and has requested inputs from independent
consultants1.  Specifically, the HSRB is considering the outcome measure
first confirmed bite (FCB), defined as one bite confirmed by another
within a 30 minute period2, and its correlate, first confirmed landing
with intent to bite. 

The following responses are specific, that is they are concerned only
with the questions posed by the HSRB and not with wider issues such as:
the relative merit of using FCB versus other repellent outcome measures
like relative protection; or the applicability of regulatory estimates
of repellent performance to real-world use scenarios.  Further,
responses provided herein represent the opinion of the consultant, and
do not necessarily reflect those of the Canadian Department of National
Defence. 

HSRB QUESTIONS/CONSULTANT RESPONSES:

Section 1 – Time intervals between first and second bites.

Q1. What do data show about the variability of the time intervals
between first and subsequent landings in mosquito repellent field
trials?

A1. FCB is not the most common endpoint used in peer-reviewed and
published field studies on repellent efficacy.  Further, even where FCB
is used, non-confirmed bites are not always reported.  Consequently,
there is a relative paucity of data available for critical analyses of
this question.

In our own recent studies3,4, the initial bite on a repellent-treated
subject was usually confirmed by another bite within 30 min.  Is this
representative of what normally would be expected?  To the extent that
bites tend to be temporally aggregated – it likely is. However, the
specifics of such clustering are not expected to be constant, but rather
should vary based on factors such as the interplay between biting
pressure and the relative protection (RP) afforded by the product.  In
our work, because biting pressure was high (e.g., 10+ bites/min) at the
time of the first bite, expectation is for a confirming bite to occur
within 30 min (also see A4). Such may not be the case under a different
set of experimental conditions. 

Q2. What is the current scientific understanding of how factors other
than repellent efficacy could affect the likelihood that an initial
event—a mosquito landing or mosquito bite—would be “confirmed”
by another similar event within 30 minutes?  Please address at least
these factors:

Characteristics of mosquito populations

Characteristics of test sites

Characteristics of test subjects

Characteristics of test methods

A2. A variety of factors can influence estimation of repellent efficacy.
 These include, but are not necessarily limited to, characteristics of:
mosquito populations, test sites, test subjects and test methods. A
brief discussion of each of the above-listed factors is presented below.
If required, additional commentary is available in several recent
publications that have tackled this subject area 5,6. 

Mosquito populations. Substantial intergeneric, interspecific and
intraspecific variability has been demonstrated in the response of
mosquitoes to repellents. Certain genera (Anopheles)7,8 and species (An.
albimanus9) are less “sensitive” to deet than are others; and
variability between species strains has been demonstrated10,11. Other
mosquito based-factors that might influence outcomes from repellent
tests include nutritional status, endogenous activity cycles, age/parity
status and mosquito population density5,6.    

Test sites. Variability between test sites can influence repellent
performance estimates.  For example, differences in mosquito population
density or species make-up (as discussed above) may substantially affect
outcomes. Further, climatic influences such as wind speed, temperature,
humidity and ambient light5,6 might impact mosquito activity and
behaviour and/or the persistence of repellent products on test subjects.
 

Test subjects. Variability between test subjects can have a profound
impact on estimates of repellent performance3,5,6. In some of our recent
work, despite virtually identical test circumstances, FCB estimates
varied by approximately 40% among subjects3 and moderate-intensity
exercise was associated with an approximately 50% reduction in product
performance4. Although causal relationships remain elusive,
subject-based variability likely involves the interplay of factors such
as: intrinsic variability in the release of cues (e.g., semiochemicals)
that affect mosquito behaviour and/or subject-based differences in
evaporative, absorptive or physical loss of repellent. 

Test methods. Potential impacts of test method on repellent performance
estimates are numerous. They include: simultaneous use of the same
subject (or even different subjects12) for biting pressure and repellent
performance estimates5,6, variable performance outcome measures (e.g.,
FCB versus RP), intermittency of exposure, inconsistent data analyses,
varied subject-activity patterns, various emphasis on product versus
subject based variability, different approaches to product dosing or
area treated, etc. 

Summary: Evaluation of how (or whether) all of the above-listed factors
would affect estimation of FCB, or more specifically the likelihood of
confirmation within a 30 min period has not been systematically
reviewed. Certainly, field13 and laboratory studies14 have suggested a
relationship between mosquito biting pressure or density and time to
first bite or FCB, respectively.  By extension, similar effects are
expected for the likelihood of a confirmatory bite within any 30 min
period (also see A4). Likewise, Barnard’s15 observation that parity
and age interact to increase the likelihood of biting at the point of
FCB suggests that mosquito based factors might also influence these
probabilities.  

Q3 Can the impact of such factors on the likelihood or timing of an
initial and confirming event be predicted?  Can it be quantified?

A3. The impact on the above-listed factors on the likelihood and timing
of a first and confirming bite can be predicted and quantified.  For
example, we can reasonably predict that moderate-level exercise will
attenuate product performance thereby resulting in a (probabilistic)
systematic decrease in the time of the first and subsequent bites. 
Similarly, we can design a set of protocols to evaluate the phenomena
and to quantify (a posteriori) the effect for that specific suite of
experimental circumstances. 

However, given the lack of standardization between studies, the paucity
of research specifically directly towards evaluation of the
above-mentioned factors and the complexity of the test system,
elaboration of anything more than a very basic general model is not
possible. 

Section 2 – Validity of intermittently sampling to determine FCB

At its June 27 - 29, 2007 meeting the Board learned that different
designs with different “length-biased” sampling for mosquito
repellent field studies are in use.  One design exposes subjects to
potential mosquito landings for one minute of every 15 minutes; another
design exposes subjects to potential mosquito landings for five minutes
of every 30 minutes.  The DFO is separately providing a CD containing
the background materials for the June 27 – 29, 2007 HSRB meeting.  The
protocols are loaded on the CD.  These designs have different
“length-biased” sampling.  

Q4. What is the methodological rationale for the two different designs?

Periodic exposure to mosquito attack is a procedural norm in
peer-reviewed and published field studies that use RP as an endpoint. 
Alternative approaches include continuous exposure to elaborate RP
and/or FCB, or various approaches using survivorship analyses.  To my
knowledge, intermittent exposure of treated subjects to establish FCB
has not been used widely in field studies. 

From a guidelines perspective, the recent draft US EPA guideline on
repellent testing would seem to allow for intermittent exposure in the
field even where FCB is the endpoint2. However, previous EPA doctrine as
well as other relevant guidelines17,18 indicate exposure in the field is
to be continuous or suggest RP as the appropriate endpoint19. 

I can only speculate on the methodological rationale for use of
intermittent exposure where FCB is the endpoint.  It might provide
logistic advantage where multiple subjects are being screened, or could
reflect an extension from laboratory approaches where intermittent
exposure appears to be more generally accepted2,16,17. 

From an analytic perspective, caution is warranted where FCB is
calculated on the basis of non-continuous field exposure because it
reduces biting pressure by a factor equivalent to: 1/(1-proportional
decrease in exposure). The upshot (where RP < 100%) is a decrease in the
probability of receiving a first and confirmatory bite in any 30 min
period. In other words, intermittent exposure should systematically
overestimate FCB compared against continuous exposure.  To illustrate
this point, I have modeled the likelihood of FCB failure during any 30
min period under several scenarios:

continuous exposure of treated subjects to mosquito attack

intermittent exposure for 10 min/30 min (i.e. two 5-min periods)

intermittent exposure for 3 min/30 min (i.e. three 1-min periods)

It is assumed that: biting pressure is at the EPA minimum of one
bite/min2,16 or higher, RP is 75% or 95%; the probability of a given
mosquito biting is 1-RP; 0 mosquitoes have bitten at time 0; and biting
probability is binomially distributed (NOTE: using a Poisson-based
approach yields virtually identical results). 

Outcomes are shown in the figure (pg. 10) and demonstrate that, within
the model construct, intermittent exposure can substantially reduce the
probability of seeing a FCB at a given RP. For example, at a RP of 75%
(well below the EPA RP standard of 95%2,16) and a biting pressure of one
bite/min, the probability of a FCB is <20% with intermittent exposure
totaling 3 min over 30 min compared to a probability approaching 100%
for continuous exposure. Similarly, at the higher RP of 95%, the
probability of a FCB can be more than 10 times less during intermittent
compared against continuous exposure. 

It should be pointed out that the above model represents a simple
approach to analyzing and representing concerns about FCB and sampling
interval.  While useful conceptually, it has not been validated
(experimentally or statistically) or elaborated (e.g., for periods > 30
min) and hence should not be construed to represent anything more the
simple illustrative model intended. 

  

Q5. Which design is used more widely in the field? Why? 

A5. As indicated in Q4, these designs have not been widely used in
peer-reviewed and published field studies evaluating repellent
performance. 

Q6. Can potential effects of variation in the pattern of intermittent
exposure on the results of efficacy testing be isolated from the effects
of other variables?  If so, can the direction or magnitude of the
effects be predicted?  How might these influences be analyzed and
accounted for in collecting, reporting and analyzing repellent efficacy
data?

A6. Outside of theoretical probability models, the present peer-reviewed
and published database for repellent testing is not sufficient to allow
for elaboration of general quantitative models describing the impact of
intermittent exposure on the results of efficacy tests. Specific
characterizations, for example by comparing in the same field procedure
the estimates of FCB derived through intermittent versus continuous
exposure, are possible (but, to my knowledge, have not been done).  

Given the absence of a robust framework to characterize the impact of
intermittent exposure on FCB-based outcomes, development of specific
advice on how to analyze and account for the impact of intermittent
exposure on estimates of FCB is not possible. General approaches that
might be appropriate include: internal standardization within a given
experiment by benchmarking intermittent FCB results against those
derived from continuous exposure; adjust upwards minimum biting pressure
requirements to offset reduced exposure intervals, and; abandon the
approach and instead evaluate performance based on intermittent or
continuous exposure for RP and/or continuous exposure for FCB. 

Section 3 – Alternative endpoints to FCB

Dr. Matt Kramer, a USDA statistician who has served as a consultant, has
suggested that the precision of estimates of Complete Protection Time
(CPT) in repellent testing could be significantly increased by defining
a failure of efficacy as the mean time from treatment to a series of
several landings or bites.  He has stated:

The precision of CPT increases when it is estimated beyond time to
[First Confirmed Bite] FCB or FCLanding.  How well CPT can be estimated
depends on the distribution of so many bites beyond FCB.  The number of
mosquitoes that will bite (n) will determine results of the test.  Each
person in the field should be his/her own control; that way it is
possible to know n per person, and reduce person-to-person variability.

If using the mean time to the first 5 bites, the SE will decrease
proportionally as n increases (n = 5 in this case).  That is equivalent
to an increase in the power of the test of 5 times.  This method allows
for detecting formulation differences near the CPT.

Q7. Does this approach, indeed, increase the precision of estimates of
CPT markedly without requiring additional subjects?

A7. The above proposal is difficult to judge in the absence of a more
detailed description.  Nevertheless, the following comments are
provided: 

The proposal is not directly comparable to FCB unless the suggestion is
for an endpoint of 5 versus 2 bites within a 30-min period.  This change
would actually decrease sensitivity for detecting product differences
over at least part of the range of possible biting pressures.  

Presuming the intent is to have an endpoint of 5 cumulative bites over x
period, performance might be expressed as: timing of the 5th bite;
arithmetic mean time for all 5 bites; or another measure such as mean
timing of bites based on survivorship analyses. The latter two
approaches likely would improve precision, at least when compared
against a measure solely based on time to first bite. Whether or not
this in turn would yield useful additional statistical power cannot be
determined on the basis on the information provided. 

If 5 cumulative bites over x period is the measure of interest, then the
proposed approach seems similar to measuring RP, albeit where outcomes
are censored on the basis of a maximum number of bites.  In this vein,
it is informative that estimates of RP do not appear to be especially
sensitive where detection of product-based differences is the goal.
Admittedly, most evaluations to date have been limited to conventional
statistical analyses. More robust experimental design or analytic
techniques might yield greater sensitivity.

The above discussion begs the question: what level of difference in
protection time is biologically, operationally, or economically
meaningful? We3 have previously argued that substantial advantage might
not be gained by designing experiments to be able to detect small
variations in repellent performance.  Rather, procedures that are
adequately powered to detect rather large differences (e.g., 20% or
greater) in performance might be sufficient, or even preferable.  

Q8. If so, would this increased precision justify the incremental risk
to the subjects resulting from their exposure to a greater number of
mosquito landings?

A8. This question is difficult to answer because risks and benefits
change with experimental, epidemiological and societal context.  For
example, exposure to several additional mosquito bites during a
repellent test in North America likely does not appreciably/meaningfully
change the risk that an individual will be infected with an arbovirus. 
At the same time, a marginal increase in precision in a repellent trial
or protection time estimate is unlikely to provide
appreciable/meaningful benefit to society. 

Q9. Is it practical to test long-lasting repellents to the point of five
landings?

A9. Yes.  Indeed, multiple bites or landings are the norm where RP is
the endpoint. 

REFERENCES

HSRB. 2007. Discussion questions for mosquito repellent studies. October
4, 2007.

EPA. 2006. OPPTS 810.3700. Product performance of skin-applied
repellents of insects and other arthropods. Draft Guideline.

Schofield S, Tepper M, Gadawski R. 2007. Field evaluation against
mosquitoes of regular and polymer-based deet formulations in Manitoba,
Canada, with comment on methodological issues. J Med Entomol. 44:457-62.

Schofield S, Tepper M, Gadawski R. In press. Laboratory and field
evaluation of the impact of exercise on the performance of regular and
polymer-based deet repellents. J Med Entomol.

Barnard DR. 2000. Repellents and toxicants for personal protection.
Position paper. World Health Organization – Pesticide Evaluation
Scheme (pgs 26-27).    HYPERLINK
"http://whqlibdoc.who.int/hq/2000/WHO_CDS_WHOPES_GCDPP_2000.5.pdf" 
http://whqlibdoc.who.int/hq/2000/WHO_CDS_WHOPES_GCDPP_2000.5.pdf 

Insect Repellents. Principles, Methods and Use. 2007. CRC Press, New
York (Chapters 5, 6, 12).

Frances SP, Waterson DG, Beebe NW, Cooper RD.  2004. Field evaluation of
repellent formulations containing deet and picaridin against mosquitoes
in Northern Territory, Australia. J Med Entomol. 41:414-7. 

Frances SP, Cooper RD, Popat S, Beebe NW. 2001. Field evaluation of
repellents containing deet and AI3-37220 against Anopheles koliensis in
Papua New Guinea. J Am Mosq Control Assoc. 17:42-4.

Klun JA, Strickman D, Rowton E, Williams J, Kramer M, Roberts D, Debboun
M. 2004. Comparative resistance of Anopheles albimanus and Aedes aegypti
to N,N-diethyl-3-methylbenzamide (Deet) and
2-methylpiperidinyl-3-cyclohexen-1-carboxamide (AI3-37220) in laboratory
human-volunteer repellent assays. J Med Entomol. 41:418-22. 

Rutledge LC, Gupta RK, Piper GN, Lowe CA. 1994. Studies on the
inheritance of repellent tolerances in Aedes aegypti. J Am Mosq Control
Assoc. 10:93-100.

Rutledge LC, Moussa MA, Lowe CA and Sofield RK. 1978. Comparative
sensitivity of representative mosquitoes (Diptera: Culicidae) to
repellents. J Med Entomol. 14:536-541.

Moore SJ, Davies CR, Hill N, Cameron MM. 2007. Are mosquitoes diverted
from repellent-using individuals to non-users? Results of a field study
in Bolivia. Trop Med Int Health. 12:532-9.

Granett P. 1940. Studies of mosquito repellents, I. Test procedure and
method of evaluating test data. J Econ Entomol. 33: 563-65

Barnard DR, Posey KH, Smith D, Schreck CE. 1998. Mosquito density,
biting rate and cage size effects on repellent tests. Med Vet Entomol.
12:39-45.

Barnard DR. 1998. Mediation of deet repellency in mosquitoes (Diptera:
Culicidae) by species, age, and parity. J Med Entomol. 35:340-3.

EPA. 1999. OPPTS 810.3700. Product performance test guidelines –
insect repellents for human skin and outdoor premises. 

Pest Management Regulatory Agency, Health Canada. 1995.  Draft Efficacy
Assessment Guidelines: Personal repellents. Ottawa.

American Society for Testing and Materials. E 939-94 (Reapproved 2006).
Field testing topical applications of compounds as repellents for
medically important and pest arthropods (including insects, ticks and
mites): 1. Mosquitoes. 

WHOPES. 1996. Report of the WHO informal consultation on the evaluation
and testing of insecticides (96.1). Geneva.

 

 

 PAGE   

 PAGE   7