1

 

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY

WASHINGTON D.C., 20460

OFFICE OF CHEMICAL SAFETY AND 

POLLUTION PREVENTION

December 15, 2015

MEMORANDUM

SUBJECT: Science Review of Field Testing of S.C. Johnson Personal Mosquito Repellent 

Products to Support Their Use of the EPA Repellency Awareness Graphic.

FROM: Eric W. Bohnenblust, Ph.D., Entomologist

Invertebrate-Vertebrate Branch 2

Registration Division (7505P) 

Office of Pesticide Programs

Kevin J. Sweeney, Senior Entomologist

Invertebrate-Vertebrate Branch 1

Registration Division (7505P) 

Office of Pesticide Programs

TO: Marietta Echeverria, Chief,

Invertebrate-Vertebrate Branch 1

Registration Division (7505P)

Office of Pesticide Programs

REF: Palm, J. (2015) Field Testing of S.C. Johnson Personal Mosquito Repellent 

Products to Support Their Use of the EPA Repellency Awareness Graphic –

Unpublished document prepared by S. C. Johnson & Son, Inc., 1525 Howe Street 

Racine, WI 53403. September 24, 2015. 149 p. (MRID 49750101) (D430002).

ACTION REQUESTED 

Conduct a science review of a completed field study testing the efficacy against mosquitoes of an 

insect repellent formulated as a pump spray containing diethyl toluamide (DEET). Determine the 

adequacy of the methods employed and the scientific validity of the reported data. These data are 

required to establish the median complete protection time (CPT) against mosquitoes for the EPA

Repellency Awareness Graphic on the label of EPA Reg. No. 4822-276, Mark-3, Maximum 
2

Strength Pump Spray Deep Woods Off! (98.25% DEET). The protocol used to conduct this study 

was previously reviewed and accepted by EPA and the HSRB with comment on April 23, 2014. 

The protocol used in this study was amended to address EPA and HSRB recommendations.

CONCLUSIONS 

The EPA assessed the scientific aspects of the research in relation to the recommendations of the 

EPA §810.3700 product performance testing guideline and the Human Studies Review Board.

Study MRID 49750101 was conducted in accordance with Good Laboratory Practices as 

described in 40 CFR §160 (with one minor exception) and provides scientific data that are 

acceptable. The Human Studies Review Board will be asked to comment on this study. 

SCIENCE REVIEW 

Study objective: The objective of this study is to establish, for the EPA Repellency Awareness 

Graphic, the median complete protection time of MARK-3 in the field against populations of wild 

mosquitoes using human volunteer subjects. This is a guideline study designed to fulfill the 

requirements of OPPTS Series §810.3700 product performance guideline, Insect Repellents to be 

Applied to Human Skin. This study was conducted in accordance with EPA, FIFRA (Federal 

Insecticide, Fungicide and Rodenticide Act), Good Laboratory Practice Standards (GLP); 40 CFR, 

Part 160 (October 1989). (p. 3 of 149).

Identification of the test system: In this study, landings of wild mosquitoes on replicate human 

subjects were used to evaluate the repellency of an insect repellent product (98.25% DEET) 

applied to human skin. Mosquitoes were used because they are one of the insect pest groups 

repelled by the product, and one of the insect groups represented on the EPA Repellency

Awareness Graphic. The registrant conducted tests in two field locations, one in Florida and one 

in Wisconsin. The following mosquito genera were collected and identified, Coquillettidia, 

Psorophora, Aedes, Mansonia, and Wyeomyia (Tables 1, 2; §10.1, p. 11 of 149).

Table 1. Wisconsin Site Mosquito Species collected - July 29, 2015 (revised table, §10.1, p. 11 

of 149)

Species Number 

Collected % of Total

Coquillettidia perturbans 1 1.4%

Psorophora ferox 3 4.2%

Aedes vexans 2 3%

Aedes trivittatus 63 90%

Aedesstimulans 1 1.4%

Total 70 100%
3

Table 2. Florida Site Mosquito Species collected - September 10,2015 (revised table, §10.1, 

p. 11 of 149)

Species Number 

Collecte % of Total

Anopheles atropos 9 3%

Anopheles crucians 4 1.3%

Aedes atlanticus 95 31.4%

Aedes infirmatus 137 45.4%

Aedestaeniorhynchus 7 2.3%

Coquillettidia perturbans 2 0.7%

Culex erraticus 4 1.3%

Mansonia dyari 4 1.3%

Mansonia titillans 4 1.3%

Psorophora ferox 30 10%

Wyeomyia mitchellae 6 2%

Total 302 100%

Experimental design: This is a field study conducted with human subjects at two

geographically and ecologically distinct field sites, one in Kenosha County, Wisconsin on July 

29, 2015, and one in Collier County, Florida on September 10, 2015. At each site, the 

experimental treatment groups of 10 different subjects treated subjects and the untreated control 

group consisted of two untreated subjects. Subjects at each site were selected from a pool of 

trained participants (Table 3; §12.4, pp. 15-16 of 149). At each site, testing was conducted during 

the course of a single day (§13.1, p. 20 of 149).

Table 3. Summary of Test Subject Participation in GLP 865E1 (Revision of Table 7 in the Study 

§12.4, pp. 15-16 of 149).

GLP 865E1 - 4822-276 Wisconsin Florida Total

Number of Subjects Enrolled 22 22 44

Number of No-Shows to 

training and/or testing

No-shows for training: 0

No-shows for testing: 1

No-shows for training: 7

No-shows for testing: 0

8 

*(Note to HSRB: In 

the study, the table 

cited 7)

Number Assigned as Test 

Subjects 12 12 24

Number Assigned as 

Alternates or Extras

4 alternates (1 was used)

5 extras

3 alternates

0 extras

12

(Note to HSRB: in the 

study, the table cited 

13)
4

Number of Test Subjects 

Withdrawn Voluntarily

1 on training day, 

2 on test day 

(post-treatment)

1 on test day 

(post-treatment)

4

(1 during training, 3 

on test day) (Note to 

HSRB: in the study, 

the table cited 2)

Number of Test Subjects 

Withdrawn Involuntarily 0 0 0

Number of Test Subjects 

Completed Research 10 11 22

In Wisconsin: 12 test 

subjects (10 treated test 

subjects and 2 controls) and 

4 alternates were randomly 

selected as described in the 

report out of the pool of 22

trained participants.

In Florida: Only 15 

trained participants 

were available on this 

test day. 12 test 

subjects (10 treated and 

2 controls) and 3 

alternates were selected 

as described in the

report out of the pool of 

15 trained participants.

*No-show from WI test date was not previously included in the total

The experiment was partially randomized, the test subjects were randomly selected from a pool 

of potential subjects that met the inclusion criteria found in §12.3, pp. 14-15 of 149. Using an 

allocation table, treatments were assigned to the limbs of each subject. If the subject was 

assigned an odd number, the left limb was treated and if assigned an even number the right arm 

was treated. Treated subjects were blinded to the identity of the test substance (§13.3, p. 20 of 

149). Two untreated control subjects were used at each site to determine landing rate and aspirate 

mosquitoes. The control subjects chose which arm to expose to mosquitoes so the dominant hand 

could be used to aspirate mosquitoes (§13.7, p. 24 of 149).

At the testing sites, study staff prepared subjects for testing in a tent enclosure to protect them 

from mosquitoes during preparation. Subjects washed their limbs before dressing in bug suit 

pants and jackets, and gloves to protect untreated body parts from mosquito bites. To treat the 

forearm, the selected forearm of the bug suit was rolled up and the arm treated as discussed in 

the Test Substance Application Rate and Treatment section below (§13.2-13.5, pp. 20-21 of 

149).

The unit of measure for determining repellent effects (Complete Protection Time) in this 

experiment was mosquito landings, similar to previous skin applied repellent evaluations where 

the “Landings” measure is used and efficacy is measured as CPT. To assess CPT, subjects were 

grouped into pairs, and the untreated control subjects used to confirm that mosquito populations 

were adequate to test the repellency of the product were paired together. Pairs were separated from 

each other by at least twenty feet. Starting three hours after application of the repellent product, 

subjects observed their limbs for five minutes for landing mosquitoes. To reduce the exposure of 

untreated control subjects to mosquitoes, untreated subjects covered their exposed limb after five 

landings in the five-minute exposure period. Five landings in the five-minute exposure period were 

considered the minimum necessary to ensure the mosquito population was large enough to 

determine repellency of the tested product. Mosquitoes that landed were aspirated by the subjects 
5

or their partner, and if necessary, headlamps were used at night so subjects could see to aspirate 

mosquitoes. After the five-minute exposure period, subjects reported the number of mosquito 

landings to study staff who recorded the number for each subject, and aspirated mosquitoes were 

labeled and kept to be identified. The time to reach five landings on the untreated control subjects 

was also recorded by study staff for each five-minute exposure period. 

The five-minute exposures were conducted every thirty minutes until repellent failure occurred. 

Repellent failure (i.e., first confirmed landing) is the exposure period in which two or more 

mosquito landings occurred, or when one land occurred in an exposure period and another landing 

occurred in the subsequent exposure period. If the repellent failed on a subject, the test was finished 

for the subject. However, the repellent did not fail on all subjects. In the event subjects withdrew 

from the study, they were not replaced with an alternate subject, instead the exposure period at 

which the subjects withdrew was used as their CPT endpoint. If landing pressure on the untreated 

control was lower than five landings per exposure period during 10% of the exposure periods, the 

study was ended because landing rates were not high enough to assess efficacy. At the Wisconsin 

site, the study was ended after 20 exposure periods (12.5 h post treatment) because landings on an

untreated control were lower than five landings in five minutes on one subject following an 

exposure period during which neither untreated control subject observed five landings on their 

exposed arm. At the discretion of the study director, the study was ended at the Florida site after 

19 exposure periods (12 h post-treatment) because the median CPT of 12 was the same as the 

median CPT from the Wisconsin site (§13.6 – 13.7, pp. 21-23 of 149).

The duration of repellency to appear on the Repellency Awareness Graphic on the product label 

will be based on the EPA Repellency Awareness Guidance for Skin-Applied Insect Repellent 

Products, which states, “The mosquito claim should be calculated using the most conservative (i.e.,

lowest) CPT from all available studies (In this study - Florida was the same as Wisconsin).” “And 

“The number of hours of protection appearing on the repellency awareness graphic should be 

expressed as a whole number. If the most conservative calculated CPT is not a whole number, 

EPA recommends rounding down to the nearest whole number.”(§4.1, p. 8 of 149).

Test Substance Application Rate and Treatment: The target application rate was 0.5 g per 600 

cm2

. Formula 1 (below) was used to calculate the amount of the test substance to apply to each 

subject. Prior to applying the test substance, each subject washed the limb to be treated with water 

and unscented soap, dried the limb, and then wiped the limb in a 70% solution of isopropanol and 

allowed the limb to dry. After drying, the treated area was marked with a felt-tipped pen. Adhesive 

surgical tape and adhesive bandage were wrapped around the treated area so the treated area was 

the only exposed skin. Study staff weighed the test substance container, tared the balance, and then

removed the target amount of the test substance from the container using a pipette and distributed

the tested substance onto the forearm of the treated subject. After the test substance was placed on 

the forearm, to ensure complete coverage, study staff spread the repellent around the forearm of 

the subject using two fingers of a gloved hand. The amount applied to each subject is found in 

Table 4 (§13.7, p. 23 of 149).

Formula 1. Weight of test substance to apply (g) = [Area of limb cm2

/600 cm2] * 0.5 g

Table 4. Amount of Test Substance Applied (§13.7, p. 23 of 149).
6

Site

Test

Subject

No.

Date Limb

Treated

Target 

Amount 1

(g)

Actual 

Amount

(g)

% of

Target

Wisconsin 26 7/29/15 Right Arm 0.50 0.51 102%

Wisconsin 27 7/29/15 Left Arm 0.40 0.41 103%

Wisconsin 28 7/29/15 Right Arm 0.38 0.37 97%

Wisconsin 29 7/29/15 Left Arm 0.45 0.45 100%

Wisconsin 32 7/29/15 Right Arm 0.45 0.45 100%

Wisconsin 37 7/29/15 Left Arm 0.45 0.45 100%

Wisconsin 38 7/29/15 Right Arm 0.54 0.56 104%

Wisconsin 41 7/29/15 Left Arm 0.39 0.39 100%

Wisconsin 43 7/29/15 Left Arm 0.49 0.49 100%

Wisconsin 45 7/29/15 Left Arm 0.54 0.55 102%

Florida 241 9/10/15 Left Arm 0.46 0.47 102%

Florida 242 9/10/15 Right Arm 0.48 0.50 104%

Florida 243 9/10/15 Left Arm 0.51 0.51 100%

Florida 244 9/10/15 Right Arm 0.48 0.49 102%

Florida 245 9/10/15 Left Arm 0.57 0.58 102%

Florida 253 9/10/15 Left Arm 0.39 0.40 103%

Florida 254 9/10/15 Right Arm 0.42 0.41 98%

Florida 255 9/10/15 Left Arm 0.34 0.35 103%

Florida 256 9/10/15 Right Arm 0.55 0.56 102%

Florida 257 9/10/15 Left Arm 0.40 0.40 100%

Average 101%

Min 97%

Max 104%

1

Target amount was based upon forearm surface area calculation.

Protocol amendments and deviations: The approved protocol was dated June 26, 2015. There 

were no amendments to the protocol (§20.2 p. 31 of 149). 

Two protocol deviations occurred during the study (Appendix B, p. 112 – 114 of 149).

Neither of the deviations affected the study.

Results and analyses:

Test Systems: Test subjects collected mosquitoes representing three genera (Coquillettidia, 

Psorophora, and Aedes) and five species in Wisconsin, and seven genera (Coquillettidia,

Mansonia, Psorophora, Wyeomyia, Anopheles, Culex, and Aedes) and eleven species in Florida 

for a total of seven genera and 14 species across the two sites. Aedes spp. mosquitoes, a mosquito 
7

genus containing numerous important vectors of human diseases, represented 94% of the 

mosquitoes collected in Wisconsin, and 77% of the mosquitoes collected in Florida (Tables 1, 2; 

§10.1 p. 11 of 149).

Duration of Repellency: At the Wisconsin site, at least five mosquitoes landed on the untreated 

control subjects in all but one exposure period (exposure period 14 on subject 47) through the 

18th exposure period. Five mosquito landings did not occur on either untreated control subject 

during exposure period 19, and five landings only occurred on one untreated control subject 

during the 20th exposure period. For exposure periods in Wisconsin where five landings were 

observed on the untreated control subjects, the time to count five mosquito landings on control 

subjects ranged from 17 seconds to 3 minutes and 45 seconds.

At the Florida site at least 5 mosquitoes landed on the untreated control subjects in all of the fiveminute exposure periods, and the time to count five mosquito landings on control subjects ranged 

from 18 seconds to 4 minutes and 57 seconds.

Median complete protection time and 95% confidence limits (lower confidence limit [LCL] and 

upper confidence limit [UCL]) were calculated by Kaplan Meier analysis using PROC 

LIFETEST in SAS, which employs a generalization of the Brookmeyer and Crowley (1982) 

method under a log-log transformation. The median duration of repellency for the tested product

was 12.0 hours in Wisconsin and Florida (Table 5, 6; §4.1, pp. 8-9 of 149).

At the Wisconsin site, two treated subjects withdrew before the end of the test; test subject 29 

withdrew after exposure period 19 and subject 38 withdrew after exposure period 4 (Table 6; §4.1, p. 

9 of 149). At the Florida site, one treated subject withdrew before the decision was made to end the 

test; subject 243 withdrew after exposure period 19 which was the last exposure period for testing 

(Table 6; §4.1, p. 9 of 149). At the Florida site, subject 258, one of the untreated control subjects 

also withdrew after exposure period 19, but this did not affect the study because the study ended after 

exposure period 19.

Table 5. MARK-3 (98.25% DEET pump spray) Repellency Duration Results Summary, Hours, 

Sample size = 10 Wisconsin site, Sample size = 10 Florida Site (§4.1, p. 8 of 149).

Measure Wisconsin

Site

Florida

Site

Median 12.0 12.0

95% LCL 6.0 8.5

95% UCL 12.0 12.0

Range 4.5 - 12.0 8.5 - 12.0

Table 6. MARK-3 (98.25% DEET pump spray) Repellency Duration Results, Hours (Revised 

table, §4.1, p. 9 of 149).

Wisconsin

Site

Florida

Site
8

Subject

No.

Repellency

Duration

Subject

No.

Repellency

Duration

26 12.5 241 12.0

27 12.5 242 12.0

28 6.0 243 12.02

29 12.03 244 12.0

32 7.0 245 12.0

37 11.0 253 9.0

38 4.51 254 8.5

41 12.5 255 8.5

43 12.5 256 12.0

45 8.0 257 11.0

1

Subject withdrew at 4.5 hrs post treatment.

2

Subject withdrew at 12.0 hrs post treatment.

3

Subject withdrew at 12.0 hrs post treatment.

Conclusions:

The methods used in this study were adequate to produce scientifically reliable results. The 

methods were based on the protocol reviewed and accepted by the EPA and HSRB on April 23, 

2015 as amended to incorporate EPA and HSRB recommendations before testing began. The 

data in the study are acceptable to support a median CPT of 12.0 hours against mosquitoes for 

the EPA Repellency Awareness Graphic on the label for EPA Reg. No. 4822-276, Mark-3, 

Maximum Strength Pump Spray Deep Woods Off! (98.25% DEET).

Note:

The limb measurement forms should have been included with the raw data.

HSRB Comments and Science Recommendations on the Protocol with S.C. Johnson’s 

Responses from the April 2015 Meeting Report Dated June 23, 2015:

Product application rate: The Board understands that the data from this protocol will be used to 

calculate median CPT values across all subjects from each of two sites rounded down to the 

nearest integer and the lowest value will be used for the product graphic (p. 11 of 39, Sweeney 

and Sherman). Given the stated use of the data, the Board agrees with the use of a standardized 

dose of 1 gram product per 600 cm2 treated skin. However, we strongly suggest changing the 

language from standardized dose to standardized application rate and reserve “dose” to describe 

how much active ingredient is applied. 

The protocol specifies that the actual amount of product applied will be recorded. The percent 

active ingredient in each product is also available so the Agency can in fact assess the 

relationship between dose (active ingredient) and efficacy (or CPT) as a quality assurance check 

of the data. An assessment of the relationship between CPT and dose (mass active ingredient per 

treated area) can provide an indication of the quality of the data or point out data that might be 
9

suspect because it does not fall along the expected dose-response trend. The Board recommends 

that the data analysis include a dose-response comparison for all products where multiple 

concentrations are available (e.g., DEET content in the different products ranges from 5% to 

98.25%) to help assess data quality. In addition, the Agency might consider normalizing the CPT 

results to better represent the expected application rates derived from earlier dosimetry studies 

when calculating the final graphic number.

S.C. Johnson Response: The word “dose” was changed to “application rate” throughout. A 

dose-response comparison was not included in the data analysis because each study only tested a 

single product. 

Product application method: The protocol proposes to use a variety of application methods 

including using pipettes for transferring liquid contents from pump sprays and spatulas for 

lotions, while aerosol sprays are applied directly. In each case, the product is applied and spread 

on the subject’s skin by a staff member. The Board was concerned that these application methods 

were not representative of actual application methods but ultimately concluded that the need for 

consistency outweighs the need for the protocol to be representative of consumer behavior. 

However, the Board stresses the importance of accurately reporting the application rate (mass of 

product per area of skin) for each subject. 

A particular concern with the aerosol application method was that the iterative procedure leaves 

open the possibility to repeatedly apply more than the target 1 gram. Some means should be 

integrated within the protocol to limit or preclude the potential to bias the average dose upward. 

One (and perhaps not the best) way to preclude such a bias would be to place some upper bound 

on the highest level above 1 gram that would be allowed to proceed to the field testing phase. 

Such an upper bound (or similar restriction) could be applied to all application methods, but 

over-exposures seem most likely to occur with the aerosol application method.

S.C. Johnson Response: The test substance (pump spray formulation) was applied using a 

pipette, and S.C. Johnson reported the exact amount of the test substance applied to each subject.

The limit on the upper dose was not implemented because the test substance was applied by 

pipette.

Use of sites outside the U.S: The protocol specifies the use of two established and ecologically 

distinct field sites in the United States for testing, and the Board agrees that this will provide 

sufficient representation for determining CPT, but the Board shares the concerns expressed by 

the Agency about using sites located outside of the U.S. The study sponsor has described that in 

addition to the two established sites in the U.S., there is at least one established site in Australia 

that could be used. The protocol needs to provide more information on what constitutes an 

“established site” either in the U.S. or another country (i.e., climate, mosquito species present, 

other hazards such as other mosquito borne diseases, presence of cell phone service, 

representativeness of local demographics) and more importantly should describe how the data 

collected at the alternate sites outside the U.S. will be related to the U.S. consumer demographics 

and the expected mosquito populations in the U.S.
10

S.C. Johnson Response: The protocol only states that sites will be qualified by confirming 

mosquito populations are adequate to achieve the minimum landing pressure (5 mosquitoes on 

an untreated control in five minutes) required to evaluate efficacy. In addition, the protocol was 

revised to include the requested information (e.g., mosquito species present, climate, etc.) for the 

proposed site in Australia. However, testing was not conducted outside of the U.S., so this 

information is not relevant to the submitted study. 

Potential for cross contamination: There are a number of places in the protocol that provide 

opportunities for insecticide to be inadvertently either lost or gained from/to the treated area on 

subjects. Simple precautions can be taken to alleviate this issue, but the protocol should specify 

steps that will be taken to insure that the treated area on subjects is not impacted by activities that 

take place before or during the experiment (i.e., rubbing sleeve or pant leg across the treated 

area).

S.C. Johnson Response: Subjects were not transported using a vehicle after the test substance 

was applied, and subjects were reminded not to touch or contact the treated skin in any manner. 

Any inadvertent contact with the treated area was reported to the study staff and documented in 

the raw data.

Potential for “carryover”: No justification was provided for the adequacy of separating 

multiple participations by any test subject by a minimum of one day (Science Response #6 in 

S.C. Johnson letter of 17 April 2015). It is important to verify that no carryover effect is present 

on subjects used on multiple days. The protocol suggests that a day between treatments will be 

sufficient when the same subject is used a second time, but justification or references are needed 

to support this. If a subject is treated with 98.25% DEET, is there any residual effect after 24 

hours that might affect a low dose treatment (application of 5.6% DEET wipe)?

S.C. Johnson Response: Subjects washed before and after each test, and a minimum of two 

days was required to pass before subjects could participate in another test. There is no indication 

that test subjects participated in multiple tests so we assume all subjects did not previously 

participate in other tests and therefore there was no “carryover” effect.

Landing pressure: The protocol includes untreated control subjects with each test to insure that 

there is sufficient landing pressure to provide valid results. However, the landing pressure is not 

measured in quantitative terms, only whether it is sufficient or not (five landings in five 

minutes). Discussions during the meeting seemed to imply that landing pressure will influence 

the measured CPT. If in fact the landing pressure can influence the resulting median CPT and 

products tested on different days are subjected to different landing pressures, then it would be 

important to collect quantitative information on landing pressure that could be used to correct, 

normalize, or at least interpret the resulting CPT values. The Board recommends that the Agency 

and S.C. Johnson consider how a quantitative estimate of landing pressure can be determined 

without increasing the likelihood of bites if landing pressure is excessive (e.g., recording the time 
11

of each landing, the time to reach 5 landings, or the total landings in 5 minutes) and how that 

information can be used to normalize or interpret CPTs measured under different landing 

pressure conditions.

S.C. Johnson Response: Study staff recorded the time to reach five landings if less than five 

minutes. Because all studies were performed at the same two sites, landing pressure appears to 

be fairly consistent across sites, therefore normalization is not required.

Delayed start: The Board recognizes the advantages of delaying the exposure to mosquitoes for 

subjects treated with products that are known from previous experience to last for a long time. 

However, the protocol needs to provide more information about the criteria used to determine 

how long to wait before starting the test cycles (5 minute exposure at 30 minute intervals). 

Regardless of how long the subject’s exposure is delayed, the protocol should require a 

minimum number of completed cycles to insure valid results. For example, following a delayed 

exposure, the subject should complete at least three exposure cycles before getting a confirmed 

landing.

S.C. Johnson Response: Exposures were delayed until three hours after application. The test 

substance contained 98.25% DEET as the active ingredient and according to the protocol 

exposures were to be delayed by three hours for products containing more than 16% DEET. This 

deviation did not affect the study, and all subjects were exposed to mosquitoes for at least three 

exposure cycles before a confirmed landing was recorded. In addition, delaying exposure periods 

reduced the exposure of the subjects to bites from mosquitoes.

Experimental design: The design as presented tests all ten subjects assigned to a product on a 

single day. The downside of this design is that it does not allow results to be easily generalized to 

a range of environmental conditions that may affect the attractiveness of a subject (e.g., sweating 

due to temperature and humidity levels). An alternative would be to test each product on several 

days; e.g., five subjects on each of two days or three subjects on each of two days and the 

remaining four subjects on a third day. Each day would form a block for the analysis of that 

product’s data. Such a design would allow testing of multiple products on a given day. If it were 

of interest to compare product formulations and/or application methods, the combined data could 

be analyzed as a block design with multiple replications of each product within each block (day).

These alternative designs that utilize blocking (e.g., by individual test subject) could be 

considered to account for known sources of variation (e.g., individual effects).

S.C. Johnson Response: The two different sites in different areas of the country should allow 

for generalizing results to a greater range of environmental conditions than a single site. Also, 

the main goal of the study is to determine the CPT for an individual product, not to compare 

multiple products.
12

Randomization: The randomization mechanism should be described in more detail and rationale 

should be given for any given choice of randomization within the protocol. For example, it is not 

clear whether/how cross-substance relations are to be evaluated in the data analysis and why 

randomization among test substances is needed. An explanation of this would be helpful. In 

addition, when the conditions support use of arm rather than leg for exposure, then it may be 

more important to consider handedness when selecting what arm to treat, rather than randomly 

assigning to left or right hand, so the subject can have their dominant hand to remove landing 

mosquitoes before they bite. 

S.C. Johnson Response: One test substance was tested on each day, therefore randomizing the 

treatment was not necessary. The mechanism for randomizing the arm was not provided in the 

study; however, after discussions with S.C. Johnson, they indicated that the mechanism for 

randomizing the arm to be treated was based on the random selection of test ID numbers. 

Subjects assigned odd numbers had their left arm treated, and subjects with even numbers had 

their right arms treated. In addition, the protocol notes that aspirating mosquitoes is not difficult 

even with a non-dominant hand. The untreated control subjects were allowed to choose which 

arm to expose.

Sample size determination: A sample size calculation would be useful here to inform the power 

of testing and the width of confidence intervals. Power and sample size calculation can be 

implemented using existing SAS procedures. Information about appropriate sample size 

calculations is included in the EPA document “Product performance Test Guidelines 

OPPTS810.3700: Insect Repellents to be Applied to Human Skin.” 

S.C. Johnson Response: After discussions with S.C. Johnson, they indicated that they did not 

conduct a sample size calculation or power analysis but the table which summarizes the effect of 

sample size on the width of the confidence interval for median CPT presented in the protocol for 

testing repellents against ticks to the Human Studies Review board at the October meeting would 

apply to these studies because both studies use 10 subjects.

Sources of variation: Multiple sources of variation including, for example, site selection, 

treatment dosage, application rate, mosquito type/age/condition, and landing pressure can impact 

the results. For the most part, however, they are not accounted for explicitly within the study, and 

when the source of variation is not controlled (e.g. as it is using a standard application rate) then 

the contribution to variance should be acknowledged or discussed. The protocol does not 

currently specify the conditions that might cause the CPT data from the two sites to differ; 

however, the researchers should consider collecting information to explain any large and 

potentially significant differences in the CPT values between otherwise matched studies 

conducted at two different sites.

S.C. Johnson Response: The researchers identified mosquitoes to species, recorded habitat 

characteristics, climatic conditions, and the time to five landings on the untreated control 

subjects, information which could explain large and significant differences in CPT. Note the 

CPTs at both sites were similar.
13

EPA Comments and Science Recommendations on the Protocol with S.C. Johnson’s 

Responses from the April 2015 Meeting Review Dated March 31, 2015:

EPA Comment: Change “mosquito biting pressure” to “mosquito landing rate” as subject bites are 

not counted or recorded in this study. 

S.C. Johnson Response: The term “mosquito biting pressure” was changed to “mosquito landing 

rate” throughout the protocol except in §16.0 p. 27 of 149 and on the data sheet for subject 28, 

where the term “bite” was used. S.C. Johnson confirmed these are written/typographical errors

and landings were evaluated.

EPA Comment: Describe how the data will be analyzed if the number of test subjects at the end of 

the test is less than ten. In other words, what if subjects withdraw? If alternates replace them, how 

will Johnson account for this change of subjects in the data analysis? 

S.C. Johnson Response: At the Wisconsin site, two treated subjects withdrew before the end of the 

test; test subject 29 withdrew after exposure period 19 and subject 38 withdrew after exposure period 

4. Exposure period 20 was not included in the statistical analysis. At the Florida site, one treated 

subject withdrew before the decision was made to end the test; subject 243 withdrew after exposure 

period 19 which was the last exposure period for testing. For subjects that withdrew before receiving 

their first confirmed landing, their CPT was considered to be the time at which they withdrew. These 

“censored” subjects are indicated by a + on the survival curve graphs. At the Florida site, subject 

243, one of the untreated control subjects also withdrew after exposure period 19, but this did not 

affect the study because the study ended after exposure period 19. 

EPA Comment: The protocol states that up to 10% of the exposure periods in a test may have less 

than the minimum landing (biting in the protocol) pressure of five mosquitoes landing in five minutes 

or less. Will treatment exposures occur during periods of insufficient landing pressure? If treatment 

data are collected during these periods, how will they be used in CPT calculation? If they are not 

used, how will the lack of data points be considered in the K-M analysis and calculation of Median 

CPT?

S.C. Johnson Response: Landing pressure at the Wisconsin site was below the minimum 

landing pressure required during the last two exposure periods (#19, 20); therefore, the study was 

ended after exposure period 20. Exposure period 20 was not included in the Kaplan Meier

analysis for this site. Landing pressure at the Florida site reached the minimum landing pressure 

for all exposure periods therefore this comment does not apply to this site.

EPA Comment: State/justify why no positive control substance is to be used.

S.C. Johnson Response: A positive control was not used because the Agency did not provide 

information on how positive control data would be used to normalize the data across sites. 

Therefore exposing additional subjects to repellent products and mosquitoes was not justified.
14

EPA Comment: Product application is not fully described. After weighing the set dose, how is the 

product applied to the limb for pump sprays and lotions? For instance, is the required amount left in 

the container and the pump used to spray it on the limb? For lotions, the amount to be applied is 

removed with a spatula instead of a larger syringe so transfer to the subject might be easier? For 

aerosols, Johnson could estimate the delivery of the prescribed amount of product by counting the 

number seconds needed to deliver the dose to the limb and determine the amount applied per second 

of spraying to more closely estimate the application amount? How does this compare to the product’s 

label directions? Will study staff be spreading the lotion with a gloved hand?

S.C. Johnson Response: After weighing, study staff pipetted the test substance onto the exposed 

skin and spread the substance evenly over the treatment area using a gloved hand.

EPA Comment: Appendix III – Land Data Form. Identification of which limb was treated needs to 

be added to this data sheet. 

S.C. Johnson Response: A line was added to the data form to identify the treated limb.

EPA Comment: Data compilation and processing. Little detail is provided in the protocol on how 

the data from these sheets will be compiled and processed before entry into Excel, JMP, or SAS, etc. 

S.C. Johnson Response: Median confidence limits were calculated by Kaplan Meier analysis

using PROC LIFETEST in SAS, which employs a generalization of the Brookmeyer and 

Crowley (1982) method under a log-log transformation.