Wednesday, December 16, 2009

EPA-HSRB-09-03

Kevin P. Teichman, PhD Acting Science Advisor
Office of the Science Advisor 1200 Pennsylvania Avenue, NW Washington, DC 20460

Subject: October 20-21, 2009 EPA Human St udies Review Board Meeting Report

Dear Dr. Teichman,

The United States Environmental Protecti on Agency (EPA or Agency) requested the
Human Studies Review Bo ard (HSRB) to review two completed intentional studies examining the effects of intentional exposure of human s ubjects to pesticides containing pyrethrins/ pyrethroids. The Agency proposes to rely on thes e two studies, conducted prior to publication of the EPA’s expanded final rule for protection of subjects in human research (40 CFR 26) on February 6, 2006 (71 Federal Regi ster 24, 6137), for regulatory acti ons under the pesticide laws. The Agency asked the HSRB to advise the Agency on a range of scientific and ethics issues regarding how the studies shoul d be assessed against the prov isions in 40 CFR sections 26.1701

– 26.1704 of the final human studies rule.

The Agency also requested the HSRB to provi de scientific and et hics reviews of two proposed human studies: a proposed Carroll-Loye Biological Research, Inc. (CLBR) insect repellent efficacy study (LNX-003); and a proposed Antimicrobial Exposure Assessment Task Force II (AEATF II) aerosol spray application protocol (AEA04).

The enclosed report provides the Board’s res ponse to EPA charge questions presented at the October 20-21, 2009 meeting.

Assessment of Completed Research Study: Newton, J., Breslin, A. (1983) Asthmatic reactions to a commonly used aerosol insect killer. Medical Journal of Australia 1:378-380.

Science

• Because of the use of a complex mixture repres ented by an insecticide product rather than a

specific chemical in this study, the Board conc luded that the Newton and Breslin study was
not relevant when consider ing the asthmatic or allerg ic respiratory response for pyrethrins/pyrethroids as a class. The Ne wton and Breslin study was relevant when considering the asthmatic or al lergic respiratory response only for the specific insecticide product that was tested.

Page 1 of 26• Considering the substantial limitations of the study, the Board conclude d that the study was of limited utility. The Board recommended that the Agency be cautious in its use of these data, limiting it to a qualitative contribution to the overall weight of evidence analysis.

Ethics

• Given the limited information available about this study, the Board concluded that there

neither was clear and convincing evidence that the study was fundamentally unethical, nor clear and convincing evidence that the study was significantly deficient relative to the ethical standards prevailing at th e time the research was conducted.

Assessment of Completed Research Study : Lisi, P. (1992) Short Communication: Sensitization risk of pyrethroid insecticides. Contact Dermatitis 26:349-350.

Science

• The Board concluded that the Lisi study was lik ely scientifically sound, but the brevity of

the report and limited details as to how the study was c onducted prevent an accurate assessment of its reliability.

• If carried out according to accepted guidelines and criteria, the Lisi study likely provides

hazard assessment data that are releva nt to assessing whether exposures to pyrethrins/pyrethroids may be associated with allergic contact dermatitis or sensitization

responses.

• Unanswered questions about participant history and sele ction, study procedures, and

outcome definitions should be taken into accoun t when considering these data in the overall analysis.

Ethics

• Given the limited information available about this study, the Board concluded that there

neither was clear and convincing evidence that the study was fundamentally unethical, nor clear and convincing evidence that the study was significantly deficient relative to the ethical standards prevailing at th e time the research was conducted.

Assessment of Proposed Carroll-Loye Biological Research Study LNX-003: Efficacy Test of KBR 3023 (Picaridin; Icaridin) - Based Person al Insect Repellent s (20% Cream and 20% Spray) with Ticks Under Laboratory Conditions.

Science

• The Board concluded that the protocol submitted for review, if modified in accordance

with Agency recommendations and conducted a ccordingly, will likely yield scientifically valid results on the efficacy of these two pi caridin-based insect repellent formulations against ticks.

Page 2 of 26Ethics

• The Board concluded that the protocol submitted for review, if modified in accordance

with Agency recommendations and conducted acco rdingly, is likely to meet the applicable requirements of 40 CFR 26, subparts K and L.

Assessment of Proposed AEATF II Scenario and Protocol AEA04: Research on Exposure of Janitorial Works Applying Pestic ides Formulated as Aerosol Sprays.

Science

• The Board concluded that the protocol submitted for review, if modified in accordance

with Agency and HSRB recommendations, will like ly generate scientifically reliable data, useful for assessing the exposure of handlers w ho apply antimicrobial pe sticides formulated as aerosol sprays.

• The Board provided a number of recommenda tions for how the study could be improved,

including: clarifying th e criteria for exclusion of subj ects who deviate grossly from the protocol; acknowledging that systematic differen ces in the air sampling results are likely to occur between the two methods used; and, cons idering carefully other variables that may influence exposure measurements.

Ethics

• The Board concluded that the protocol submitted for review, if modified in accordance

with Agency and HSRB recommendations and conducted accordingly, is likely to meet the applicable requirements of 40 CFR 26, subparts K and L.

• The Board recommended that the protocol be revised to address certain concerns,

including: ensuring that documents in Spanis h are reviewed by someone familiar with the dialect written and spoken in the target community; design and posting of community notification flyers appropriate for guests stay ing at the motels where study procedures are likely to take place; and, exclusion of volunt eers who may be at greater risk of productrelated harm due to immunodeficien cy or other underlying conditions.

Sincerely,

Sean Philpott, PhD, MSBioethics
Chair
EPA Human Studies Review Board

Page 3 of 26NOTICE

This report has been written as part of th e activities of the EPA Human Studies Review Board, a Federal advisory committee providing advice, information and recommendations on
issues related to scientific and ethical aspects of human subjects research. This report has not been reviewed for approval by the Agency a nd, hence, the contents of this report do not necessarily represent the view a nd policies of the Environmental Protection Agency, nor of other agencies in the Executive Branch of the Fede ral government, nor does the mention of trade names or commercial products constitute a re commendation for use. You may obtain further
information about the EPA Human Studies Review Board from its website at http://www.epa.gov/osa/hsrb

. You may also contact the HSRB Designated Federal Officer, via
e-mail at phre@epa.gov

In preparing this document, the Board caref ully considered all information provided and presented by the Agency presenters, as well as information presented by public commenters. This document addresses the information provide d and presented within the structure of the charge by the Agency.

Page 4 of 26US ENVIRONMENTAL PROTECTION AGENCY
HUMAN STUDIES REVIEW BOARD

Chair

Sean Philpott, PhD, MSBioethics, Director for Re search Ethics, Assistant Professor of Bioethics, The Bioethics Program of Union Graduate Coll ege and the Mount Sina i School of Medicine, Schenectady, NY

Vice Chair

Janice Chambers, PhD, DABT, William L. Giles Di stinguished Professor, Director, Center for Environmental Health Sciences, College of Vete rinary Medicine, Mississ ippi State University, Mississippi State, MS

Members

Suzanne C. Fitzpatrick, PhD, DABT, Senior Science Policy Analyst, Office of the Commissioner, Office of Science and Health Coordination, US Food a nd Drug Administration, Rockville, MD

Vanessa Northington Gamble, MD, PhD, University Professor of Medical Humanities, Professor of Health Policy and American Studies, The George Washington University, Washington, DC

Sidney Green, Jr., PhD, Fellow of the ATS, Prof essor, Department of Pharmacology, Howard University College of Me dicine, Washington, DC

Dallas E. Johnson, PhD, Professor Emeritus, Depart ment of Statistics, Kansas State University, Manhattan, KS

Michael D. Lebowitz, PhD, FCCP, Retired Professor of Public Health & Medicine, University of Arizona, Tucson, AZ

Lois D. Lehman-Mckeeman, PhD, Distinguish ed Research Fellow, Discovery Toxicology, Bristol-Myers Squibb Company, Princeton, NJ

Jerry A. Menikoff, MD, JD, Director, Office of Human Subjects Research, Office of the Director, National Institutes of Health, Bethesda, MD

Rebecca Parkin, PhD, MPH, Associate Dean for Re search and Public Health Practice, School of Public Health and Human Services, The Geor ge Washington Universi ty, Washington, DC

William Popendorf, PhD, MPH, Professor, Depa rtment of Biology, Utah State University, Logan, UT

Page 5 of 26*Ernest D. Prentice, PhD, Associate Vice Chan cellor for Academic Affairs, University of Nebraska Medical Center, Omaha, NE

Linda J. Young, PhD, Professor, Department of Statistics, Institute of Food and Agricultural Sciences, University of Fl orida, Gainesville, FL

Human Studies Revi ew Board Staff

**Paul I. Lewis, PhD, Executive Director, Huma n Studies Review Board Staff, Office of the Science Advisor, United States Environm ental Protection Agency, Washington, DC

* Not in attendance at Oct ober 20-21, 2009 Public Meeting

**Service ending date was November 21, 2009. As of November 22, 2009, the DFO is Jim Downing, Office of the Science Advisor, Unit ed States Environmental Protection Agency, Washington, DC

Page 6 of 26INTRODUCTION

From October 20-21, 2009, the United States Environmental Protection Agency’s (EPA or Agency) Human Studies Review Board (HSRB) met to address scientific and ethical issues concerning: two completed human toxicity stud ies involving one class of pesticide active ingredients – pyrethrins/pyrethroi ds– conducted prior to publicati on of the EPA’s expanded final rule for protection of subjects in human research. In accordance with 40 CFR 26.1602, EPA sought HSRB review of these completed studies. E ach of these completed studies is discussed more fully below.

In addition, the Agency submitted two prot ocols for conducting new research involving human subjects: one study measuring the efficacy of two registered insect repellents containing picaridin against ticks under la boratory conditions; and one study measuring levels of exposure received by janitorial workers when applying a commercially-avail able antimicrobial pesticide formulated as an aerosol spray. In accord ance with 40 CFR 26.1601, EPA sought HSRB review of these two proposed protocols. Each of thes e protocols is discussed more fully below.

REVIEW PROCESS

On October 20-21, 2009, the Board conducted a public face-to-face meeting in Arlington, Virginia. Advance notice of the meeting was publ ished in the Federal Register as “Human Studies Review Board; Notice of Public Meeting” (74 Federa l Register 106, 26861).

Following welcoming remarks from Agency offi cials, the Board heard presentations from EPA on the following topics: two completed stud ies involving intentional human exposure to pyrethrin/pyrethroid pesticides (Newton and Breslin [1983], and Lisi [1992]); a proposed Carroll-Loye Biological Research, Inc. (CLBR) insect repellent efficacy study (LNX-003); and a proposed Antimicrobial Exposure Assessment Task Force II (AEATF II) aerosol spray application protocol (AEA04).

The Board also asked clarifying questions of several study sponsors and/or research investigators, including:

Dr. Scott Carroll, Principal, Ca rroll-Loye Biological Research Dr. Jeffery Driver, Principal, infoscientific.com, Inc. Mr. Shawn King, Director of Operations , Carroll-Loye Biological Research Dr. Bryce Landenberger, Dow AgroSciences Dr. Sami Selim, President, Golden Pacific Laboratories

Oral comments were provided by:

Dr. Scott Carroll, Principal, Ca rroll-Loye Biological Research Mr. Stephen McFadden, Independent Advocate Dr. Lawrence Plumlee, Independent Advocate

No written public comments were provided.

Page 7 of 26For their deliberations, the Board considered the materials presented at the meeting, oral comments, and Agency background document s (e.g., published literature, sponsor and investigator research reports, study protocols, data evaluation r ecords, and Agency science and ethics reviews of proposed protocols and co mpleted studies). A comprehensive list of background documents is available online at http://www.regulations.gov.

CHARGE TO THE BOARD AND BOARD RESPONSE

Assessment of Completed Research Study: Newton, J., Breslin, A. (1983) Asthmatic reactions to a commonly used aerosol insect killer. Medical Journal of Australia 1:378-380.

Overview of the Study

In the Newton and Breslin (1982) study, seven subjects diagnosed with asthma and a history of chest tightness were evaluated for airway narrowi ng and chest tightening before and after a controlled exposure to aerosol insecticide sprays. Subjects were exposed to a commercially-available aerosol insecticide (Mortein Pressure Pak insect killer containing 3.0 g/kg pyrethrins, 0.9 g/ kg tetramethrin, 15 g/kg piperonyl butoxide,
7.5 g/kg N-octyl-bicycloheptene dicarbons, propellants [chlorofluorocarbons and hydrocarbons] and solvents [non-water based]). The aim of the study was three-fold: to study the response of asthmatic to exposure to insecticides containing pyrethrins and tetramethrin; to study the time-course of exacerbation of asthma following exposure to the insecticide; and to evaluate bronchial r eactivity to histamine af ter exposure to the insecticide. Data were reported as simple counts or proportions across exposure groups.

Science

Charge to the Board

Is the Newton and Breslin (1983) study sc ientifically sound, pr oviding reliable data?

Board Response to the Charge

In light of the substantial limitations of the study, the Board concluded that the study was of limited utility. The Board recommended that the Agency be cautious in its use of these data, limiting it to a qualitative contribution to th e overall weight of evidence. The use of these
data should also be limited to a ssessments of the specific insecticide product tested rather than pyrethrins/pyrethroids as a class of pesticides.

Page 8 of 26Charge to the Board

Is the Newton and Breslin (1983) study relevant to an assess ment of the proposition that exposures to pyrethrins/pyrethroids may be associ ated with asthmatic or allergic respiratory responses?

Board Response to the Charge

The Board concluded that the Newton and Breslin study was relevant only when considering the asthmatic or alle rgic respiratory response for the specific insecticide product that
was tested. It was not relevant when considering the asthmatic or allergic respiratory response for pyrethrins/pyrethroids as a class because of the use of a complex mixture represented by an insecticide product rather than a specific chemical.

Charge to the Board

What limitations of the Newton and Breslin (1983) study should be taken into account by EPA in assessing the proposition that exposures to pyrethrins/pyrethr oids may be associated with asthmatic or allergic respiratory responses?

Board Response to the Charge

The Board cautioned against using the Ne wton and Breslin study for assessing the proposition that exposures to pyret hrins/pyrethroids may be associat ed with asthmatic or allergic respiratory responses. The use of a complex mixtur e represented by an ins ecticide product rather than a specific chemical was of considerable concern, as was the more robust response for respiratory irritation rather than asth ma or allergy reported in the study.

HSRB Detailed Recommendations and Rationale

The Newton and Breslin (1982) article provide d an incomplete record of the conduct of the study and analysis of the data, such that a number of import ant data gaps were identified.

One primary concern was that the tested ma terial contained a mixt ure of ingredients, some of those ingredients at hi gher concentrations than the py rethrins. One such ingredient, piperonyl butoxide, has been iden tified previously as a potential respirator y allergen (c.f. FDA warnings on pediculicide pr oducts, 21 CFR § 358.601). Accordingl y, the Board concluded that the data obtained by Newton and Breslin are direct ly applicable only to the product tested, and are not broadly applicable to pyrethrins/pyrethroids as a pesticide class.

Other major issues and limitations note d during the Board’s discussion included:

1. Limited and likely inac curate exposure data.

a. Exposures to the test material were not di rectly measured, but were calculated based on the assessment of the amount of product deli vered per unit time. Study subjects were

Page 9 of 26likely to have been exposed to the experimental product, but the lack of data on the actual exposure rate or concentration precludes any quantit ative evaluation.

b. The Newton and Breslin articl e provides a description that is inadequate to assess whether the facility was appr opriate for this type of te sting. The test chamber was
described as a 7-m 3 room with large windows on one side for observation purposes and
an exhaust fan that “removed fumes between sprays.” There is no indication, however, that there was a standardized protocol that governed spraying procedures or cleaning requirements between exposures. There also was no indication whether the air exchange rate of the room or cleaning procedures between exposures we re sufficient to ensure that the exposure levels intended by the investig ators were delivered. Thus, there may be a high degree of error in the estimated exposur es, as compared to the actual exposures.

c. There was no mention of factors that may have affected actual exposures, such as the distance between participant and spray nozzle, spray droplet size, or any odorants or deodorants used to “mask” the product spray from the placebo.

2. Small sample size.

a. Although study subjects were noted to have a predisposition for asthma, and thus likely to be potentially susceptible to showing an effect, only seven subjec ts were evaluated.

b. There is insufficient descriptive informa tion for the seven subjects chosen, such as parameters of respiratory functi on and disease state. It was al so noted that the two oldest subjects were both male, but no considerati on whether age, gender or other participant characteristics may have affected the study results.

3. Incomplete results and data analysis.

a. Two subjects did not complete the pretest histamine challenge, and three subjects did not complete the post-exposure challenge. Ma ximum mid-expiratory flow rate (MMEFR) was not measured for one subject. More notable is the fact that th e only subject with a significant decrease in forced expiratory volum e in one second (FEV1) did not complete either histamine challenge evaluation. No expl anation for subjects’ failure to complete these study procedures and tests was provided.

b. There are limited details on how FEV1 and MME FR values were reported. It was noted during the Board’s discussion that standard cl inical practice is to report the maximum values, but it is unclear if this was done in the Newton and Breslin study.

c. There was no indication that baseline FEV1 or MMEFR values were within the normal ranges for these subjects, nor any additional detail on participant characteristics that could affect these values, such as age, gender, body size, or underlying disease.

Page 10 of 26d. There was no specific designation of the sympto ms that each patient experienced. It is not clear which patient was the person not c ounted among the “6 of 7” who experienced chest tightness with other symptoms.

e. There was no specific indication whether a nd how the observation of respiratory irritation correlated with changes in FEV or MMEFR values.

f. It is unclear which subjects were tested at which levels of exposure (e.g. 10 versus 20 seconds), how many times each person was expos ed at each exposure duration, or which volunteers received the placebo on day 2.

4. Potential information bias.

a. Subjects were not blinded to treatment, and there was no description of how symptoms were self-diagnosed and reported by voluntee rs. Without the use of standardized definitions as a reference, there is no way to know whether each subject had the same interpretation and reporting practices fo r self-diagnosed respiratory symptoms.

Ethics

Charge to the Board

Is there clear and convincing evidence that the conduct of the Newt on and Breslin (1983) study was fundamentally unethical, or significantly de ficient relative to th e standards of ethical research prevailing when it was conducted?

Board Response to the Charge

Given the limited information available abou t this study, the Board concurred with the Agency’s assessment (Sherman 2009a) that there was neither clear and convincing evidence that the study was fundamentally unethical, nor clear and convincing evidence that the study was significantly deficient relative to the ethical st andards prevailing at the time the research was
conducted.

HSRB Detailed Recommendations and Rationale

The Newton and Breslin study was conducted in Australia in 1981 or 1982. The Board concurred with the Agency’s assessment that mo st widely recognized ethi cal standard at that time was the 1975 Declaration of Helsinki, but also noted the importance of placing ethical standards within a specific na tional context. The Australian National Health and Medical Research Council (NHMRC) -- the la rgest public supporter of research in Australia -- issued guidelines as early as 1966 that required all NHMRC-funded research to be reviewed by an independent ethics committee. A 1976 supplementary note to these guidelines, prompted by and explicitly referencing the Declaration of Helsi nki, called upon all Australi an research facilities
to establish committees to review all research involving human , NHMRC-funded or otherwise.

Page 11 of 26Limited information is available abou t how the Newton and Breslin study was conducted. The Agency attempted to obtain additional information about the conduct of this study, but was unable to do so. The only available information is from the published article,
which states that the written informed consent of all subjects was obt ained. The article also indicates that all subjects were adults, and th at none of the female volunteers were pregnant. Although the article does not describe the type of ethical oversight of this research that took place, it is assumed that independent ethics review occurred in accordance with NHMRC guidelines.

Considering the information available, the Board concurred with the conclusions and factual observations of the ethical strengths a nd weaknesses of the study, as detailed in the Agency’s Ethics Review (Sherman 2009a). The Board also concluded th at this study met all applicable ethical requirement s for such research involvi ng humans, as required by the Agency’s expanded human studies rule, according to the following criteria:

a. Not fundamentally unethical . When determining whether or not a study is fundamentally unethical, the Board’s standard approach is to decide if the res earch was intended to seriously harm human subjects, if it failed to obtain informed consent, or if it was fundamentally unethical for other reasons. In this case, there was no evidence that the study was intended to seriously harm subjects. The published article al so failed to provide evidence supporting a conclusion that the informed consent of subjects was not obtained. Given lack of clear and convincing evidence th at for any other reasons it might have been fundamentally unethical, the Board concluded that it was not funda mentally unethical.

b. Not significantly deficient . With regard to determining whether a study was significantly
deficient relative to the ethical standard s prevailing at the time the research was conducted, the Board’s standard is to determin e whether or not any ethical deficiencies
identified could have resulted in serious harm to human subjects, based on knowledge
available to researchers at the time the st udy was conducted, or whether the information provided to study subjects coul d seriously impair informed consent. As noted above, there is limited information available rega rding this study. Only the published article provides any information, and the informa tion contained therei n fails to support a conclusion that the study was si gnificantly deficient relative to the standards at the time.

Assessment of Completed Research Study : Lisi, P. (1992) Short Communication: Sensitization risk of pyrethroid insecticides. Contact Dermatitis 26:349-350.

Overview of the Study

In the Lisi (1992) study, 230 volunteers we re patch tested with each of seven pyrethroids: allethrin, cypermethrin, deltame thrin, fenothrin (assumed phenothrin), fenvalerate, permethrin, resmethrin. The goal was to establish the irritation and sensitization potential of these widely used pyrethroids. Of the volunteers who participated, 162 were men and 68 were women. Subjects ranged from 19 to 78 years in age. Approximately 35% of the subjects were agricultural workers, 12% had previously

Page 12 of 26worked on a farm, and the remaining 53% were engaged in nonagric ultural activities. Fifty-four subjects had irritant or allergic contact dermatitis of the hands, 18 of which were correlated with agricu ltural activities. Further, 176 subjects had non-allergic skin disorders, and 16 subjects were also atopic. Simple counts and proportions were used to compare irritation and sens itization effects across the three occupational groups.

Science

Charge to the Board

Is the Lisi (1992) study scientific ally sound, providing reliable data?

Board Response to the Charge

The Board concluded that the Lisi study wa s likely sound, but the brevity of the report and limited details as to how the study was c onducted prevent an accurate assessment of its reliability.

Charge to the Board

Is the Lisi (1992) study relevant to an a ssessment of the proposition that exposures to pyrethrins/pyrethroids may be associated with allergic contact dermatitis or sensitization responses?

Board Response to the Charge

If the Lisi study was carried out according to accepted guidelines and criteria, it likely provides hazard assessment data that are relevant to assessing whether exposures to pyrethrins/pyrethroids may be associated with allergic contact dermatitis or sensitization

responses.

Charge to the Board

What limitations of the Lisi (1992) study should be taken into account by EPA in assessing the proposition th at exposures to pyrethrins/pyrethroids may be associated with allergic contact dermatitis or sensitization responses?

Board Response to the Charge

A lack of experimental detail and subject history represent two major limitations of the Lisi study. The Board also noted that the allergic /irritant response rate was extremely low in the study. The low response rate, along with concerns about participant history and selection, study procedures, and outcome definiti ons, should be taken into account when considering these data in the overall weight of evidence analysis.

Page 13 of 26HSRB Detailed Recommendations and Rationale

The extremely brief nature of the report rais ed a number of questi ons about how the study was conducted. Because the study was reportedly perf ormed according to standards and criteria established by the Inte rnational Contact Dermatitis Re search Group (ICDRG), the Board assumed that standard testing procedures were used.

Major issues and limitations noted duri ng the Board’s discussion included:

1. Poorly defined participant population.

a. It was unclear how the 230 were chosen for study.

b. Subjects were identified as “agricultural work ers,” “ex-agricultural workers,” or “other,” but there was no explanation for cr eating and using these categories.

c. Participant compliance with study procedur es was not reported. There was no indication as whether any withdrew or were otherwise not observed, nor did th e report provide any detail whether all test patches were evaluated for all on days 2 and 3.

2. Lack of protocol detail.

a. ICDRG protocols may have been used, but the report provides few details on how investigators distinguished allergic versus irritant responses or any information on how they graded response severity.

3. Low response rate.

a. The low response rate (less than 2% of all study) suggests that the seven compounds tested pose little risk of alle rgic contact dermatitis or sk in sensitization. However, these data could also be indicative of problems in study design and conduct that invalidate the results. Because the appropriate positive and negative controls were not included in the study design, the Board felt that it could not rule out this al ternative explanation for the low response rate.

Ethics

Charge to the Board

Is there clear and convincing evidence th at the conduct of the Lisi (1992) study was fundamentally unethical, or signifi cantly deficient relative to th e standards of ethical research prevailing when it was conducted?

Board Response to the Charge

Given the extremely limited information available about this study, the Board concurred with the Agency’s assessment (She rman 2009b) that there was neither clear and

Page 14 of 26convincing evidence that the study was fundame ntally unethical, nor clear and convincing evidence that the study was significa ntly deficient relative to the ethical standards prevailing at
the time the research was conducted.

HSRB Detailed Recommendations and Rationale

The Lisi study was conducted in Italy some time in 1990 or 1991. The Board concurred with the Agency’s assessment that the most wide ly recognized ethical standard at that time was
the 1989 Declaration of Helsinki , but again noted the importance of placing ethical standards within a specific national context. Italy ha d established a National Bioethics Committee in 1990. That Committee may have developed directives applicable to this study. Nevertheless, as explained below, the potential existence of thos e national standards does not alter the Board’s analysis.

Extremely limited information is availabl e about how the Lisi study was conducted. The EPA attempted to obtain additional informati on about the design and conduct of the study, but was unable to do so. The only available informa tion is from the published article, which does
not state whether written informed consent of the subjects was obtaine d. The published article also does not address what type of ethical ov ersight took place (e.g. review by an independent ethics committee). Study subjects were reported to be adults, a nd there is no indication that any of the subjects were pregnant or nursing. Although the EPA ethics re view states that “there is no evidence suggesting that any subjects came from an especially vulnera ble group,” several of were current and former agricultural workers. Su ch may be socioeconomically or educationally disadvantaged, and thus may be vulnerable. The Board recommended that the Agency more clearly articulate or standardiz e its definition of vulnerability.

Considering the information available, the Board concurred with the conclusions and factual observations of the ethical strengths a nd weaknesses of the study, as detailed in the Agency’s Ethics Review (Sherman 2009b). The Board also concluded th at this study met all applicable ethical requirements for such research involving huma n, as required by the Agency’s expanded human studies rule, accord ing to the following criteria:

a. Not fundamentally unethical . When determining whether or not a study is fundamentally unethical, the Board’s standard approach is to decide if the res earch was intended to seriously harm, if it failed to obtain inform ed consent, or if it was fundamentally unethical for other reasons. In this case, there was no evidence that the study was intended to seriously harm. The published ar ticle also failed to provide evidence supporting a conclusion that the informed cons ent of was not obtained. Given the lack of clear and convincing evidence that for any other reasons it might have been fundamentally unethical, the Board concluded that it was not funda mentally unethical.

b. Not significantly deficient . With regard to determining whether a study was significantly
deficient relative to the ethical standard s prevailing at the time the research was conducted, the Board’s standard is to determin e whether or not any ethical deficiencies
identified could have resulted in serious ha rm to subjects, based on knowledge available to researchers at the time the study was c onducted, or whether the information provided

Page 15 of 26to study subjects could seriously impair info rmed consent. Only the published article provides any information, and the informa tion contained therei n fails to support a conclusion that the study was si gnificantly deficient relative to the standards at the time.

Assessment of Proposed Carroll-Loye Biological Research Study LNX-003: Efficacy Test of KBR 3023 (Picaridin; Icaridin) - Based Person al Insect Repellent s (20% Cream and 20% Spray) with Ticks Under Laboratory Conditions.

Overview of the Study

The protocol describes a study to measure the e ffectiveness of picaridin as a tick repellent when used in one of two compound formulations (20% picardin KBR 303 2 All-Family Insect Repellent Cream and 20% picari din KBR 3023 All-Family Insect Repellent Spray). Dosimetry data accumulated in previous Carroll-Loye studies (LNX-001 a nd LNX-002) would be used for dose selection. The efficacy of pica ridin as a tick repellent will be determined in a controlled laboratory setting by placing bot h Western black-legged ticks ( Ixodes pacificus) and American dog ticks ( Dermacentor variabilis) on picaridin-treated and untrea ted forearms and measuring the speed and distance that m oving ticks would penetrate into the treated area at 15-minute intervals. A total of 20 subjects will be enrolled.

Science

Charge to the Board

If the proposed laboratory tick repellenc y study protocol LNX-003 is revised as suggested in EPA’s review and if the research is performed as described, is the research likely to generate scientifically reliable data, useful for assessing the efficacy of the tested materials in repelling ticks?

Board Response to the Charge

The Board concluded that the protocol submitted for review, if modified in accordance with Agency recommendations and conducted accord ingly, will likely yield scientifically valid results on the efficacy of these two picaridin-based insect repellent formulations against ticks.

HSRB Detailed Recommendations and Rationale

Protocol LNX-003 from Carroll-Loye Biologi cal Research (Carroll 2009a, 2009b) will be conducted using methods similar to those presen ted to and commented on by the Board in the past. Although the study protocol was overly long and includes redundant or unnecessary text, it was relatively clear and addresse d adequately a number of key scientific issues, including: scientific jus tification, objectives, and data collection and compilation methods.

The proposed methods largely follow EPA’s guidelines, with the one notable exception being the use of ten volunteers per study aim, ra ther than the Agency’s existing recommendation

Page 16 of 26of six volunteers per study aim. The greater number of study subject s should yield more useful information than might otherwise be obtained. The protocol also incorporated the use of dosimetry-generated data, which will likely genera te data representative of real-world use by

consumers.

As has been pointed out previously in Boar d reviews of other repellency protocols, the proposed statistical approach fails to account for censoring of data and the calculation of complete protection time is not the best end-use of the study data . Calculating the proportion of individuals protected for a given time may be a be tter way to report this ty pe of data and should be considered by the Agency.

Ethics

Charge to the Board

If the proposed laboratory tick repellenc y study protocol LNX-003 is revised as suggested in EPA’s review and if the research is performed as described, is the research likely to meet the applicable requirements of 40 CFR part 26, subparts K and L?

Board Response to the Charge

The Board concluded that the proposed labor atory tick repellency study protocol LNX003, if modified in accordance with EPA (She rman and Sweeney 2009) recommendations, and performed as described, will likely meet the a pplicable requirements of 40 CFR 26, subparts K
and L.

HSRB Detailed Recommendations and Rationale

The submitted documents assert that the st udy will be conducted in accordance with the ethical and regulatory standards of 40 CFR 26, Subparts K and L, as well as the requirements of US EPA’s GLP Standards described at 40 CFR 160, and the California Department of Pesticide Regulation study monitoring (California Code of Regulations Title 3, Section 6710) (Carroll 2009a, 2009b). Requirements of FIFRA §12(a)(2)(P) al so apply. The protocol was reviewed and approved by an independent human subjects re view committee, Indepe ndent Investigational Review Board, Inc. (IIRB, Inc.), of Plantation, FL, prior to submission. Minutes of IIRB, Inc. meetings and a copy of IIRB, Inc. policies and procedures were provided to the EPA as a separate document (IIRB, Inc. 2009) . These documents indicate that IIRB, Inc. reviewed this protocol pursuant to the standards of th e Common Rule (45 CFR Part 46, Subpart A).

1. The Board concurred with the conclusions and factual observations of the ethical strengths and weaknesses of the study, as de tailed in the EPA’s Ethics Review (Sherman and Sweeney 2009). The proposed study is likely to meet the applicable ethical requi rements for research involving human subjects, in accordance with the following criteria:

a. Acceptable risk-benefit ratio . The risks as noted in the st udy protocol are fivefold: 1) allergic reaction to test materials themselves ; 2) exposure to biting arthropods; 3) possible

Page 17 of 26exposure to arthropod-borne diseases; 4) physic al stress from the test conditions; and 5) psychological stress and/or breach of confid entiality for pregnancy test results. These risks are minimized appropriately and are ju stified by the potential societal benefits, particularly data on the efficacy of these ne w formulations as personal tick repellents.

· Based on toxicological data currently av ailable for picaridin, coupled with appropriate exclusion criteria, study subjects are unlikely to be at risk of adverse side effects with exposure.

· The risk of bites is negligible and mini mized by the study design; tick questing and biting behavior is slow, a nd study subjects are trained to remove ticks from their forearms prior to biting. Study subjects will be trained in proper tick observation and handling techniques.

· The ticks used for the study are bred and ra ised in a laboratory environment and are considered to be pathogen-free, minimizing the risk of vector-borne disease. Tick
colonies and their rabbit hosts are also screened regularly for known tick-borne diseases, including the ricketts ial illness Rocky Mountain S potted Fever that has been observed in the past to be transmitted within laboratory tick colonies through a transovarian mechanism.

· The potential risks to subjects from phys ical stress are minimized. Although the 12­ hour duration of the study pr otocol raises some concerns about physical stress and exhaustion, the study investigators attest that similar protocols of equivalent length have never been seen as unduly stressfu l by study subjects. Appropriate stopping rules and medical management procedures are in place. Subjects are also given frequent breaks and can withdraw fr om the study at any time should the investigational procedures prove too strenuous.

· Minors and pregnant or l actating women are excluded from participation, with pregnancy either confirmed by over-the-count er pregnancy testi ng on the day of study or by opt-out. The potential stigma resulting from study exclusion due to pregnancy is also appropriately minimized.

b. Voluntary and informed consent of all

· The study protocol includes several mech anisms designed to minimize coercive recruitment and enrollment. For example, a lthough many of the rese arch subjects will be recruited from the University of Calif ornia at Davis student population, where Dr. Carroll holds an adjunct appointment, stude nt and employees of the Study Director are excluded from participation. Additi onal mechanisms designed to minimize coercive recruitment, developed in response to earlier HSRB concerns and recommendations (c.f. EPA HSRB 2006a; 2006b) are also in place.

· Monetary compensation is not so high as to unduly influence study subjects.

Page 18 of 26c. Equitable selection of study

· The majority of research subjects will be recruited from the University of California at Davis student population. Study subjects are likely to reflect the ethnic and racial diversity of individuals in the City of Da vis, but the use of this convenience sample may limit the broad applicability of the study results to the general population. The investigators have noted th is fact in the protocol.

Assessment of Proposed AEATF II Scenario and Protocol AEA04: Research on Exposure of Janitorial Workers Applying Pestic ides Formulated as Aerosol Sprays.

Overview of the Study

AEATF II aerosol spraying scenario is de signed to measure a typical occupational handler’s daily exposure to an antimicrobial spra y packaged in a pressurized aerosol spray can. These data will be used generically to estimat e dermal and inhalation exposures and risks for other antimicrobial ingredients where the product is packaged in a pressu rized aerosol spray can. The Task Force expects these data to be useful for estimating exposures for various types of aerosol spray products, different types of surfaces, different room configur ations, different types of buildings, different handler s, and various antimicrobial active ingredie nts. Eighteen professional janitors will be enrolled in the st udy, and asked to apply (spray, but not wipe) the product at one of three motels in the Fresno, CA area. Study subjects will be randomized to apply different amounts of product, from 1 to 4 can s of product in 1/2-can increments (i.e., 1 to
1.5 cans, 1.5 to 2 cans, up to 3.5 to 4 cans). Dermal and inhalation exposure will be measured using whole-body dosimeters (inner and outer), hand and face washes, and personal air monitors.

Science

Charge to the Board

If the proposed AEATF II aerosol applicati on scenario and field study protocol AEA04 is revised as suggested in EPA’s review and if th e research is performed as described, is the research likely to generate sc ientifically reliable data, useful for assessing the exposure of handlers who apply antimicrobial pesticid es formulated as aerosol sprays?

Board Response to the Charge

The Board concluded that this protocol, if modified in accordance with EPA (Leighton, Walls and Sherman 2009) and HSRB recommendations will likely generate scientifically reliable data, useful for assessing the exposure of handler s who apply antimicrobial pesticides formulated as aerosol sprays.

The Board also recommended a number of prot ocol modifications, as listed below.
Additional Board review of th e protocol is not required prior to study implementation.

Page 19 of 26HSRB Detailed Recommendations and Rationale

The Board concluded that the protocol (A EATF II 2009a, 2009b) a nd supplemental SOPs (AEATF II 2009b) address adequately a number of key scientific issues, including : scientific objective, quantification of the test material s, data collection and compilation methods and summary of test results, justification for se lection of the test substances, and QA/QC requirements. The process used to select the pr oduct to be tested seems rational. The AEATF II has described in detail their sampling design fo r the aerosol scenario, and has incorporated
random elements where feasible. The aerosol scenar io is well defined, and the diversity of daily exposures under the aerosol scenario as defined in this proposal is likely to describe adequately a typical occupational handler’ s daily exposure to this an timicrobial application. The environmental monitoring, exposure monitoring and an alytic techniques appear to be adequate. The variation in exposure should be sufficien t to determine a distribution of exposures.

The Board did raise a number of concerns about perceived inadequacies in the study design, as summarized below:

1. Use of the results : EPA plans to use the data genera ted from the proposed aerosol study generically to estimate dermal and inhalation exposures and risks for other antimicrobial ingredients where the product is packaged in a pressurized aerosol spray can. However, other variables can affect rates of exposure, incl uding different nozzle si zes, spray and ejection rates, the size of the particles generated, a nd the generation of nonvolat ile active ingredients. Inhalation versus dermal exposure may also vary with these variables.

It is also unclear if occupa tional handler exposure can be used to estimate exposures of nonprofessionals using similar c onsumer products. However, for risk assessment purposes it was felt that exposure among occupational users is likely to exceed that of consumers; the higher frequency of exposures among occupational users is likely to exceed a plausibly higher individual but less frequent exposure among nonprofessional users.

2. Sample Size & Analysis : Raw and descriptive data will be provided, but no statistical analyses have been planned. Sample size adeq uacy cannot be judged without a statistical analysis plan. Placing 18 subjec ts into three clusters of si x each requires consideration of cluster effects, and may complicate analysis of the results. Assumed constants related to exposure to the active ingredient may also prove to be incorrect, and additional subjects may
need to be enrolled.

3. Application of spray : The target number of cans sprayed in each location by each worker on each day could be specified better within the protocol. For example, does a worker need to continue spraying a room if the target applic ation is met before completion of that room? Additional protocol clarificati ons needed include: addressing the issue of the interval between sampling at different s ites (currently, it appears that no two monitoring events can occur within the same building within one week); clarifying the issue of using empty apartments rather than motel rooms; and clarif ying the effect of exhaust vent effects on exposure, which could be different than indicated in the protocol.

Page 20 of 264. Quality assurance and control (QA/QC): QA/QC can be improved by:

a. Determining likely differences in the ai r sampling results depending on the method used (e.g. depending on the air sampling equipment us ed, differences in the orifice diameters and air flow sampling rates could affect ae rosol collection efficiencies and effects).

b. Setting the minimal spike to two to four times that of the limit of quantitation (LOQ; cf. AEATF II 2009b: 42, 46).

c. Considering other variables that may infl uence the measurement of exposure, including whether the surface being treated is dry or wet at the time of the next spray application, and whether the applicant accidentally wiped the surface after spraying. Data accumulated during gross deviations from the protocol should also be excluded. Finally, the protocol should indicate that there w ill be a maximum of two workers on any given day in the same location.

Ethics

Charge to the Board

If the proposed AEATF II aerosol applicati on scenario and field study protocol AEA04 is revised as suggested in EPA’s review and if th e research is performed as described: Is the research likely to meet the applicable requi rements of 40 CFR part 26, subparts K and L?

Board Response to the Charge

The Board concluded that the protocol submitted for review, if modified in accordance with EPA (Leighton, Walls and Sherman 2009) and HSRB recommendations, is likely to meet the applicable requirements of 40 CFR 26, subparts K and L.

HSRB Detailed Recommendations and Rationale

This is a protocol for third-party resear ch involving intentio nal exposure of human subjects to a pesticide, with the intention of submitting the resulting data to EPA under the pesticide laws (AEATF II 2009a, 2009b, 2009d). Thus, it is subject to the ethical and regulatory standards of 40 CFR 26, Subparts K and L. The re quirements of FIFRA § 12(a)(2)(P) relating to informed, voluntary consent and those of the Ca lifornia Department of Pesticide Regulation study monitoring (California Code of Regulations Title 3, Section 6710) also apply. The protocol was reviewed and approved by an independent human subjects review committee, Independent Investigational Review Board, Inc. of Plantation, Florida, prior to submission (AEATF 2009b, 2009c).

1. Except as noted below, the Board concurred with the conclusions and factual observations of the ethical strengths and weakne sses of the study, as detailed in the EPA’s Ethics Review (Leighton, Walls and Sherman 2009). The proposed study is likely to meet the applicable ethical requirements for research involving human subjects, in accordance with the following
criteria:

Page 21 of 26a. Acceptable risk-benefit ratio . The risks from participa tion in the study include: 1) discomfort and heat-related illness from wearing two layers of clothing; 2) discomfort from wearing an air sampling device; 3) react ions from exposure to the test material (including accidental exposure) or to solven ts used to obtain residues from hands, face and neck; 4) embarrassment from disrobing in th e presence of a research technician; and 5) psychological stress and/or breach of confidentiality for pregnancy test results. These risks are minimized appropriately and are ju stified by the potential societal benefits, particularly data for new exposure assessmen ts for antimicrobial products applied with pressurized aerosol cans.

· The surrogate materials consist of a co mmonly used commercial antimicrobial product, which is known to have low risks. The subjects will only be exposed to concentrations of the surrogate co mpound at accepted exposure thresholds.
Candidates who are known to be sensitive to that compound, or who are in poor health, or who have broken skin on thei r hands, face, or neck will be excluded.

· Risk of heat-related illness is minimized appropriately. Testing will take place in a temperature-controlled environment, and th e heat index will be monitored with associated stopping rule. There will be a lim ited time of exposure with rest periods at 30-minute intervals, or more frequently if requested. There also will be close observation of subjects, with subjects alerted to the si gns and symptoms of heat

stress.

· Subjects will be reminded about safe ha ndling practices and procedures, wear appropriate personal protective equipment (PPE), and will be monitored for any accidental or unintended product exposure.

· Minors and pregnant or l actating women are excluded from participation, with pregnancy either confirmed by over-the-count er pregnancy testi ng on the day of study or by opt-out. The potential stigma resulting from study exclusion due to pregnancy is appropriately minimized.

· The purpose of the study is to develop more accurate information on worker exposures to liquid antimicrobial products applied using pressu red aerosol cans for indoor surface disinfecting. A lthough there is no clear bene fit to study participation, Subjects may benefit by requesting their result s and using these results to change their working practices. Benefits and risks of pa rticipation are clearl y articulated in the protocol and informed consent documents.

· Possible subjects for this study may be undocumented immigrants. Recruitment materials should state that government-i ssued identification is necessary for enrollment in this study.

· A valid government-issued form of identification is necessary for enrollment. Volunteers who lack proper id entification are excluded fr om study participation but

Page 22 of 26no other action is taken, minimizing any so cial risks to undocumented workers from
disclosure of their status to law- enforcement or immigr ation authorities.

b. Voluntary and informed consent of all

· The informed consent documents clearly state that study subject can withdraw at any time without penalty.

· The Board disagrees with the EPA commen t that “no potential subjects are from a vulnerable population” (Leight on, Walls and Sherman 2009: 11) . In fact, subjects in this study might represent particularly vulnerable populations and thus might be susceptible to coercion or undue influence. Some subjects may be educationally or economically disadvantaged, or may not be native speakers of English. However, the
risk of coercion or undue influence is appropriately minimized as follows:

o Monetary compensation is roughly equiva lent to the average daily wage of janitorial workers in Fr esno, California, and so is not so high as to unduly influence the subjects.

o Spanish translations of all inform ed consent documents, informational packets, and recruitment flyers are av ailable. If necessary, the informed consent process will be conducted in Sp anish by a member of the research team fluent in that language.

o The subjects communicate dir ectly with the researchers about their interest in enrolling in the study. Recruitment is not conducted through employers.

c. Equitable selection of study

· The study is designed to recru it an appropriately diverse population of subjects who represent professional janito rial workers in the Fresno County area. The researchers plan to place recruitment notices in three newspapers that cater to the general population, the African-American popul ation, and the Latino population.

2. The Board recommended that the study protocol be modified to address the few concerns noted in the EPA’s Ethics Review (Leight on, Walls and Sherman 2009). The Board also made the following additional recommendations:

· The Board recommended that the consent form be reviewed to make sure that it is at an appropriate reading level for the proposed participant population. In some instances, there appear to be word choices that could be simplified (e.g., wording such as “non-porous surfaces”, “pressurized aerosol can” instead of “spray can”, and assuming that the volunteer knows what a Material Safety Data Sheet is).

Page 23 of 26· The Board agrees with the EPA ethics revi ew that the informed consent form should read, “We will pay for needed medical treat ment that is not paid for by your own
insurance or the insura nce of a third party under which you are covered.”

· As some possible subjects for this study may be undocumented immigrants, recruitment materials should more explic itly state that a valid government-issued
form of identification is necessary for enrollment.

· The Board recommended that the Task Force ensure that documents in Spanish are reviewed by someone familiar with the di alects written and spoken in the target community. The National Standards for Cu lturally and Linguistically Appropriate Services in Health developed by the Depart ment of Health and Human Services, the Office of Minority Health has stressed the im portance of treating medical patients in a “culturally and linguistically appropriate manner” (Office of Minority Health 2001, 3). The same practice should be extended to subjects in EPA- or third partyconducted research, with materials such as informed consent forms and recruitment flyers written in ways that reflect “the dialectic and cultural nuances as well as the acculturation, educational, and literacy leve ls of the local target population” (Office of Minority Health 2001, 80).

· The protocol provides for use of a comm unity notification flyer. However, one neglected community includes persons who might be staying in the hotels where the study is conducted. The flyer should be revi sed so that it communicates the goals of the study and risks to that group. These flyers, in both English and Spanish, should also be posted in locations so that hotel guests are likely to see it. Alternatively, the researchers should consider conducting the res earch in areas away from hotel guests.

· The exclusion criteria should be revised to eliminate some groups that might be at higher risk of physical harm but are not presently excluded. This might include subjects who might be immunosuppressed for a variety of reasons, those with severe diabetes, and those with other cond itions that pose a health risk.

Page 24 of 26REFERENCES

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009a. AEATF II Aerosol Study Vol. 1 Primary Documentati on: Aerosol Application Scenario Design. Dated August 4, 2009. Unpublished protoc ol prepared by the AEATF II.

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009b. AEATF II Aerosol Study Vol. 2 Primary Documenta tion: Study Protocol and IIRB Approval and Documentation. Dated August 4, 2009. Unpub lished protocol and documentation prepared by the AEATF II.

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009c. AEATF II Aerosol Study Vol. 3 Primary Documenta tion: IIRB Communicati ons. Dated August 4, 2009. Unpublished documentation prepared by the AEATF II.

Antimicrobial Exposure Assessment Task Force II (AEATF II). 2009d. AEATF II Aerosol Study Vol. 4 Standard Operating Procedures for a Multi-Year Antimicrobial Chemical Exposure Monitoring Program. Dated August 4, 2009. Unpublished documentation prepared by the AEATF II.

Carroll, S. 2009a. Efficacy Test of KBR 3023 (Picaridin; Icaridin)-Based Personal Insect Repellents (20% Cream and 20% Spray) with Ticks under Labor atory Conditions. Dated July 30, 2009. Unpublished protocol prepared by Carroll-Loye Biological Research. MRID

47836801.

Carroll, S. 2009b. Carroll-Loye Study LNX- 003 Proposed Amended Language. Dated September 18, 2009. Unpublished protocol amendm ent prepared by Carroll-Loye Biological
Research.

EPA HSRB. 2006a. June 27-30, 2006 EPA Human St udies Review Board Meeting Report.

EPA Human Studies Review Board. 2006b. Octobe r 18-19, 2006 Human Studies Review Board Meeting Report.

Independent Institutional Review Board, Inc. (IIRB, Inc.). 2009. Supplemental IIRB, Inc., Documentation Package. Unpublishe d materials prepared by IIRB.

Leighton, T., C. Walls and K. Sherman. 2009. Scie nce and Ethics Review of AEATF II Aerosol Scenario Design and Protocol for Expos ure Monitoring. Dated September 21, 2009. Unpublished document prepared by Office of Pes ticide Programs, United States Environmental Protection Agency.

Lisi, P. 1992. Sensitization Risk of Pyrethroid Insecticides. Contact Dermatitis 26:349-350.
MRID 47852801.

Page 25 of 26Newton, J.G. and A.B. Breslin. 1983. Asthmatic R eactions to a Commonly Used Aerosol Insect
Killer. Medical Journal of Australia 1:378-380. MRID 47686201.

Office of Minority Health, Department of Hea lth and Human Services. 2001. National Standards for Culturally and Linguistically Appropriate Servi ces in Health Care: Fi nal Report. Washington, DC: U.S. Department of Health and Human Services.

Sherman, K. 2009a. Ethics Review of Pre-Rule Intentional Exposure Human Toxicity Study (Newton and Breslin 1983). Dated Septembe r 16, 2009. Unpublished document prepared by Office of Pesticide Programs, United St ates Environmental Protection Agency.

Sherman, K. 2009b. Ethics Review of Pre-Rule Intentional Exposure Human Toxicity Study (Lisi 1992). Dated September 16, 2009. Unpublishe d document prepared by Office of Pesticide Programs, United States Envi ronmental Protection Agency.

Sherman, K., and K. Sweeney. 2009. Science and Et hics Review of Protocol for Human Study of Tick Repellent Performance. Dated Septem ber 21, 2009. Unpublished document prepared by Office of Pesticide Programs, United St ates Environmental Protection Agency.

Page 26 of 26