Document ID: EPA-HQ-OPP-2009-0056-0046
Agency: epa
Document Type: Supporting & Related Material
Title: 
Posted Date: 2017-05-25T04:00Z

UNITED STATES ENVIRONMENTAL PROTECTION AGENCY
                         WASHINGTON, D.C. 20460      

                                                                     	OFFICE OF
                                                           CHEMICAL SAFETY AND 
                                                           POLLUTION PREVENTION
	

MEMORANDUM

Date:  December 29, 2016

SUBJECT:	Updated Literature Review on Neurodevelopment Effects & FQPA Safety Factor Determination for the Organophosphate Pesticides
 
PC Code:  See Below
DP Barcode:  437043
Decision No.: 
Registration No.: None
Petition No.: None
Regulatory Action: None
Risk Assessment Type: None
Case No.: None
TXR No.: 0057252
CAS No.: See Below
MRID No.: None
40 CFR: None
		              									
	          	
FROM:	Ashlee Aldridge
	Health Effects Division (7509P)
	Office of Pesticide Programs

              Anna Lowit, Ph.D., Senior Science Advisory
              Immediate Office
              Office of Pesticide Programs (7501P)
              
              Virginia C Moser, PhD, DABT, Fellow ATS, Toxicologist 
              Toxicity Assessment Division
              Office of Research and Development
		
THROUGH:	Dana Vogel, Director
	Health Effects Division (7509P)
		
TO:	Nicole Zinn, Chemical Review Manager
	Neil Anderson, Branch Chief
	Risk Management and Implementation Branch 1	
              Pesticide Re-Evaluation Division (7508P)	

This paper supports the use of the 10X FQPA Safety Factor in the individual organophosphate human health risk assessments.  This paper updates the September, 2015 paper based on comments from the public and addition of new epidemiology papers from the open literature. No additional studies have been added to the laboratory animal toxicology sections of the literature review.
Chemical
PC Code
CAS No. 
Dicrotophos
035201
141-66-2
Fosthiazate
129022
98886-44-3
Coumaphos 
036501
56-72-4
Terbufos
105001
13071-79-9
Profenofos
111401
41198-08-7
Bensulide
009801
741-58-2
Diazinon
057801
333-41-5
Ethoprop
041101
13194-48-4
Dimethoate
035001
60-51-5
Malathion
057701
121-75-5
Phosmet
059201
732-11-6
Chlorethoxyfos
129006
54593-83-8
Acephate/
Methamidiphos
103301/ 101201
30560-19-1/ 10265-92-6
Pirimiphos-methyl
108102
29232-93-7
TCVP
083701
961-11-5
Tribufos
074801
78-48-8
Phorate
057201
 298-02-2
Phostebupirim
129086
96182-53-5
DDVP
084001
62-73-7
Naled
034401
300-76-5
Trichlorfon
057901
52-68-6
Fenamiphos
100601
22224-92-6
AZM
058001
86-50-0
Methidathion
100301
950-37-8
Propetamphos
113601
31218-83-4
ODM
058702
301-12-2
Disulfoton
032501
298-04-4
Methyl parathion
053501
298-00-0
Temephos
059001
3383-96-8
Chlorpyrifos-methyl
059102
5598-13-0
		

                                       
                                       
Updated Literature Review on Neurodevelopment Effects & FQPA Safety Factor Determination for the Organophosphate Pesticides 
                                       
                         Office of Pesticide Programs
                      US Environmental Protection Agency
                                       
                                       
                               December 29, 2016

Table of Contents

1.0	Introduction and Background	6
2.0	Literature Review	8
2.1	Developmental Neurotoxicity (DNT) Research on OPs other than Chlorpyrifos:  Laboratory Animal Studies	8
2.1.1	Literature Search Strategy & Results	9
2.1.2	Integration of Literature:  Neurobehavioral Domains	11
2.1.3	Integration with AChE Inhibition	22
2.1.4	Summary of Findings from the Developmental Neurotoxicity (DNT) Guideline Studies	23
2.1.5	Conclusions on In Vivo Laboratory Animal Studies	24
2.2	Epidemiology Research on OPs other than Chlorpyrifos	25
2.2.1	Overview of Literature Reviews: 2012/2014, 2015, and updated in 2016	25
2.2.2	Literature Search Methodology	26
2.2.3	Breadth and Depth of the 2015/2016 Literature Review	27
Table 2.2-a	Study Designs, Exposure Measurement Methods, and Outcome Measurement Methods Used across the 2015/2016	31
Table 2.2-b	Detailed Neurological Outcome Measurement Methods Used across the  2015/2016 Review Studies	33
2.2.4	Study Designs & Considerations for Study Quality Evaluation	35
2.2.4.1	Study Designs	35
2.2.4.2	Considerations for Study Quality	37
Table 2.2.4-1	Study Quality Considerations	38
2.2.4.2.1	Exposure Measures	39
Table 2.2.4-2	CDC Table of Organophosphate Pesticides and Their Dialkyl Phosphate Metabolites (2008)	41
2.2.4.2.2	Neurological and Other Outcome Measures	42
2.2.4.2.3.	Statistical Analysis	42
2.2.4.2.4	Confounding	43
2.2.4.2.5	Risk of Bias	43
2.2.5	Review of Quality Results	44
2.2.5.1	"High" Quality Group	44
Table 2.2.5.1-1.	High Quality Studies from 2015 and 2016 reviews: Summary of Study Design Elements Impacting Study Quality Assignment	45
Table 2.2.5.1-2	High Quality Studies from 2012/2014 literature search of Children's Environmental Health Epidemiology Studies	49
2.2.5.2	"Moderate" Quality Group	64
Table 2.2.5.2-1.	Moderate Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment	65
2.2.5.3	"Low" Quality Group	68
2.2.6	Assessment of Epidemiological Studies for Relevance to Analysis	68
2.2.7	Birth Outcome Epidemiologic Studies	70
2.2.8	Neurodevelopment Outcome Epidemiologic Studies	71
2.2.8.1	CCCEH, CHAMACOS, & Mt. Sinai Cohorts	72
2.2.8.2	HOME Cohort	77
2.2.8.3	Meta-analysis (Engel et al, 2015)	78
2.2.8.4	Other Neurodevelopmental Studies	79
3.0	Weight of Evidence Analysis:  Integration Across Multiple Lines of Evidence	80
3.1	Dose-Response Relationships & Temporal Concordance	82
3.2	Strength, Consistency & Specificity	83
3.2.1	Strength	83
3.2.2	Consistency	84
3.2.3	Specificity	89
3.3	Biological Plausability & Coherence	91
4.0	10X FQPA Safety Factor for Infants and Children	92
5.0	References	94
6.0	Appendices	106
Appendix 1.	Table of In Vivo Developmental Neurotoxicity Studies of OPs.	107
Appendix 2.	Summary of Guideline DNT Studies Submitted to the Agency for OPs other than Chlorpyrifos.	112
Appendix 3.	Low Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment	114
Appendix 4.	Table of Systematic Review Analysis:  Initial Search Research & Initial Exclusion Criteria.	118
Appendix 5.	Table of Systematic Review Analysis:  Second Tier Exclusion Criteria.	157
Appendix 6.	Plausible hypotheses on MOA/AOP for neurodevelopmental outcomes (Extracted from Section 4.4.3. RHHRA for chlorpyrifos)	171

Literature Review on Neurodevelopment Effects & FQPA Safety Factor Determination for the Organophosphate Pesticides 

1.0	Introduction and Background

Organophosphate pesticides (OPs), widely used in agricultural and household pesticidal applications, act by inhibiting acetylcholinesterase (AChE) in nerve cells.  OPs share the ability to inhibit AChE via phosphorylation of the active site of the enzyme leading to accumulation of acetylcholine and ultimately neurotoxicity, this class of pesticides is subject to assessment of cumulative risk (USEPA, 1999; 2006).  Historically the agency has used inhibition of AChE as the point of departure for OP human health risk assessments (HHRAs).  

Newer lines of research on OPs in the areas of potential modes of action/adverse outcome pathways (MOAs/AOPs), 0F in vivo animal studies, and notably epidemiological studies in mothers and children, have raised uncertainty about the agency's risk assessment approach with regard to the potential for neurodevelopmental effects in fetuses and children.  Many of these studies focus on chlorpyrifos and have been the subject of review by the agency since 2008 (See Appendix 6).  

The agency has taken a stepwise, objective and transparent approach in evaluating, interpreting, and characterizing the strengths and uncertainties associated with all of the available lines of scientific information related to the potential for adverse neurodevelopmental effects in infants and children.  The stepwise evaluation began with the September 2008 FIFRA Scientific Advisory Panel (SAP) meeting involving a preliminary review of the epidemiology studies on three children's cohorts, with a particular focus on women and children and exposure to chlorpyrifos (USEPA, 2008), followed by the draft "Framework for Incorporating Human Epidemiologic & Incident Data in Health Risk Assessment" for integration of epidemiology with other types of experimental data (USEPA, 2010; FIFRA SAP 2010a,b). In December, 2016, OPP's Framework for Incorporating Human Epidemiologic & Incident Data in
 Risk Assessments for Pesticides (USEPA, 2016c) was finalized based on input from the 2010 FIFRA SAP, public comment and experience gained over the last few years.

In 2012, the agency convened another meeting of the FIFRA SAP focused on chlorpyrifos which incorporated the newest experimental data related to AChE inhibition and both cholinergic and non-cholinergic adverse outcomes, including neurodevelopmental studies on behavior and cognition effects (FIFRA SAP 2012). Similarly, the agency also performed a more in-depth analysis of the biomonitoring data and of epidemiological studies from three major children's health epidemiology cohort studies in the U.S., as well as plausible hypotheses on MOAs/AOPs leading to neurodevelopmental outcomes (USEPA 2012; Appendix 6).  Following the 2012 SAP meeting, the agency solicited additional input from federal experts in the areas of Magnetic Resonance Imaging (MRI) and neurobehavioral testing in children to further clarify results obtained by examination of the epidemiological studies.1F  In December, 2014, the agency released "Chlorpyrifos: Revised Human Health Risk Assessment for Registration Review" which included the use of a physiologically-based pharmacokinetic/pharmacodynamic (PBPK-PD) model to derive human PODs, which obviated the need for the animal to human extrapolation factor, and refined intra-species factors for some lifestages (USEPA 2014).  The chlorpyrifos 2014 revised HHRA also included retention of the 10X FQPA Safety Factor due to uncertainty regarding the degree of protection the endpoint of AChE inhibition provides for potential neurodevelopmental effects (USEPA, 2014).  

A review of the scientific literature on potential MOA/AOP leading to effects on the developing brain was conducted for the 2012 FIFRA SAP meeting (USEPA, 2012) and updated for the 2014 chlorpyrifos revised HHRA (USEPA 2014; Summarized in Appendix 6).  In short, multiple biologically plausible hypotheses and pathways are being pursued by researchers including: AChE as a morphogen; cholinergic system; endocannabinoid system; reactive oxygen species; serotonergic system; tubulin, microtubule associated proteins and axonal transport.  However, no one pathway has sufficient data to be considered more plausible than the others.  Among the available studies, there are effects which are either as sensitive as or more sensitive than AChE inhibition.  The fact that there are, however, sparse data to support the in vitro to in vivo extrapolation, or the extrapolation from biological perturbation to adverse consequence, significantly limits their quantitative use in risk assessment.  The SAP concurred with the agency in 2008 and 2012 about the lack of definable key events in a MOA/AOP leading to developmental neurobehavioral effects.  Since the 2014 literature review, there have been no substantive changes in the ability to define and quantify steps in an MOA/AOP leading from exposure to effects on the developing brain.  The lack of an established MOA/AOP makes quantitative use of the epidemiology study in risk assessment challenging, particularly with respect to dose-response, critical duration of exposure, and window(s) of susceptibility.  The agency will continue to monitor the scientific literature for studies on the AOP for neurodevelopmental effects but this document does not include an updated literature review on this line of evidence.
This document (Section 2.0) provides the literature review of in vivo laboratory animal studies and epidemiology studies for OPs other than chlorpyrifos to support the single chemical HHRAs.  It also provides an integrated weight of evidence (WOE) analysis for all the OPs to support retention of the 10X FQPA Safety Factor (Section 3.0).  Section 4.0 states that the 10X FQPA Safety Factor is being retained for all the OPs listed in the table on page 2 above.

2.0	Literature Review

In recent years, the National Academy of Sciences has encouraged the agency to move towards systematic review processes to enhance the transparency of scientific literature reviews that support chemical-specific HHRAs to inform regulatory decision making2F.  The NRC defines systematic review as "a scientific investigation that focuses on a specific question and uses explicit, pre-specified scientific methods to identify, select, assess, and summarize the findings of similar but separate studies".3F  Consistent with NRC's recommendations, EPA's Office of Chemical Safety and Pollution Prevention (OCSPP) is currently developing systematic review policies and procedures.  In short, OCSPP employs fit-for-purpose systematic reviews that rely on standard methods for collecting, evaluating and integrating the scientific data supporting the agency's decisions.  The literature review described here uses concepts consistent with systematic review such as detailed tracking of search terms and which literature have been included or excluded.  

2.1	Developmental Neurotoxicity (DNT) Research on OPs other than Chlorpyrifos:  Laboratory Animal Studies

The literature on neurobehavioral effects of developmental exposure to chlorpyrifos was summarized and discussed at the 2012 FIFRA SAP.  More recent studies were added to this summary for the 2014 chlorpyrifos revised HHRA (USEPA, 2014).  At that time, the conclusions were that the animal studies clearly showed neurobehavioral outcomes following developmental exposure to chlorpyrifos, but there were inconsistencies in the types of effects reported (neurological domain altered, direction of change, gender specificity).  Furthermore, the studies were conducted with doses that most likely produced at least some amount of AChE inhibition at some time during the exposure based on results of guideline studies submitted for registration.  The impact of these observations has lead the agency to evaluate whether or not these conclusions extend to other OP pesticides.  In this review, the studies of a number of OP pesticides are summarized. 

The search aimed to focus on rodent studies involving prenatal/perinatal exposure to OPs in which the offspring were evaluated with in vivo neurobehavioral tests.  The search methods and analytical scope for this analysis are consistent with the chlorpyrifos analysis from the 2012 FIFRA SAP and 2014 HHRA.  Preweaning measurements of behavioral development were noted but not compiled, since these could reflect effects of current exposure to the pesticide rather than long-term neuronal changes.  Information on AChE inhibition in either fetuses/pups or dams during this exposure period was evaluated where available.  Sections 2.1.1-2.1.3 describe the studies from the open scientific literature.  Section 2.1.4 summarizes relevant results from the DNT guidelines studies submitted for pesticide registration (US EPA guideline 870.6300 and/or OECD guideline 426).  

2.1.1	Literature Search Strategy & Results

To review and evaluate the developmentally neurotoxic effects of other OPs, a search of the open literature was undertaken.  Due to the limited number of studies available, the agency did not limit the search to currently registered OPs.  The agency is aware that some OPs listed below are no longer registered for use in the US.  In addition, the data evaluation records (DERs) for existing guideline DNT studies were collected from OPP files and summarized.  The literature search strategy was developed and conducted by a US EPA reference librarian.  Databases searched were PubMed, Web of Science (WoS) and ScienceDirect using key words described below.  Duplicates were eliminated after the total database was generated.

1. PubMed (751 results)
(((((organophos* OR Cholinesterase Inhibitors OR acetylcholinesterase inhibitor))) AND ((prenatal OR neonatal OR perinatal OR in utero OR fetal OR foetale OR newborn OR infant* OR postnatal OR gestational OR pregnancy[MeSH Terms])))) AND ((neurodevelop* OR attention OR birth outcome* OR health outcome* OR cognitive OR cognition OR developmental disability* OR fetal growth OR fetal growth OR foetal development OR fetal development OR sex OR social OR intelligence OR memory OR neurological functioning OR psychomotor OR neurotoxicity OR neurobehavior OR behavior OR learning OR Nervous System[MeSH Terms]) OR Neurotoxicity Syndromes[MeSH Terms]) AND ((((((guinea pigs[MeSH Terms]) OR rabbits[MeSH Terms]) OR mice[MeSH Terms]) OR rats[MeSH Terms]) NOT fishes[MeSH Terms])

2.  Web of Science (427 results)
#5	#4 AND #3 AND #2 AND #1 
DocType=All document types; Language=All languages; 
#4	TS=(guinea pig* OR rabbit* OR mice OR mouse OR rat* OR rodent*) NOT TS=(fish*)
DocType=All document types; Language=All languages; 
#3	TS=(neurodevelop* OR attention OR birth outcome* OR health outcome* OR cognitive OR cognition OR developmental disabilit* OR fetal growth OR fetal growth OR foetal development OR fetal development OR sex OR social OR intelligence OR memory OR neurological functioning OR psychomotor OR neurotoxicity OR neurobehavior OR behavior OR learning OR Nervous System OR Neurotoxicity Syndromes) 
DocType=All document types; Language=All languages; 
#2	TS=(prenatal OR neonatal OR perinatal OR in utero OR fetal OR foetale OR newborn OR infant* OR postnatal OR gestational OR pregnan*) 
DocType=All document types; Language=All languages; 
#1	TS=(organophos* OR Cholinesterase Inhibitors OR acetylcholinesterase inhibitor OR chlorpyrifos) 
DocType=All document types; Language=All languages; 

3. Science Direct (19 results)
(ALL(organophos* OR Cholinesterase Inhibitors OR acetylcholinesterase inhibitor OR chlorpyrifos) and ALL(prenatal OR neonatal OR perinatal OR in utero OR fetal OR foetale OR newborn OR infant* OR postnatal OR gestational OR pregnan*)) AND (neurodevelop* OR attention OR birth outcome* OR health outcome* OR cognitive OR cognition OR developmental disabilit* OR fetal growth OR fetal growth OR foetal development OR fetal development OR sex OR social OR intelligence OR memory OR neurological functioning OR psychomotor OR neurotoxicity OR neurobehavior OR behavior OR learning OR Nervous System OR Neurotoxicity Syndromes) AND (ALL(guinea pig* OR rabbit* OR mice OR mouse OR rat* OR rodent) and not ALL(fish*)). 

This broad literature search identified 1012 potential papers, which were reviewed individually.  Specific criteria were applied to select suitable studies, as was previously done with chlorpyrifos.  Since the literature on chlorpyrifos has been previously reviewed, those papers were excluded here.  This resulted in 19 relevant papers with the following specifications:
 Exposure occurred during gestation and/or the postnatal time frame, ending no later than weaning.  
 Dosing included maternal and/or pup administration.  
 Dosing was via oral or subcutaneous injection.  One paper with intracisternal injection was excluded.  
 Behavioral testing of the offspring occurred after weaning and/or into adulthood.  
 Studies involved only single-chemical exposure, and where two or more chemicals were administered together, only the single-chemical data were included in the summaries.  
 Test subjects were rats or mice.  Several papers in pigs and rabbits were excluded due to the lack of comparative database for those species.  
 The test measures of interest were neurobehavioral endpoints.  At least two studies involved only electrophysiological measures, and those were excluded.  No neurochemical, genomic, or other molecular endpoints were included.  

The OPs examined, and the number of papers for each, are listed below.  Of particular interest are studies from one laboratory (Duke University) that included parathion and diazinon, and can be directly compared to studies with chlorpyrifos using similar experimental designs.  The majority of studies used rats (13), and exposures periods varied about evenly between gestational and postnatal stages.

 Parathion (5) 
 Diazinon (5)
 Methyl parathion (3) 
 Methamidophos (2) 
 Chlormephos (1)
 Dichlorvos (1)
 Fenitrothion (sumithion) (1)
 Oxydemeton-methyl (demeton-S-methyl, metasystox-R) (1)

These papers dated back to 1968, and there was a wide range in study quality.  Shortcomings were noted in almost all papers, including cursory methodological information and presentation of results, inappropriate statistical analyses, contradictory statements, and problematic interpretation of the data.  Regardless, the literature is summarized below in terms of the functional domains organized by each neurobehavioral evaluation.  

Below are study descriptions and summaries organized by neurological domain.  Appendix 1 is an overall summary for each chemical, presenting each endpoint and outcome. 

2.1.2	Integration of Literature:  Neurobehavioral Domains

Cognition

Fifteen studies measured some aspect of cognition: tests included mazes (radial arm maze, Lashley maze, T-maze spontaneous alternation, M-maze), conditioned response (passive avoidance, conditioned avoidance, operant responding, T-maze), and recognition (novel object).  Most of these showed adverse effects of OP exposure, although not always in a consistent or dose-responsive manner.

Rats treated with diazinon (0.5, 2 mg/kg/d, postnatal day (PND) 1-4) showed no differences alternating in a T-maze (Timofeeva et al., 2008a).  The same rats were tested several months later in a radial arm maze, and showed increased working memory errors (both males and females), but only at the low dose (0.5 mg/kg/d) with no effect on reference memory performance.  Diazinon (1 mg/kg/d) was given to rats on gestational day (GD) 15-18 or PND1-4, and there was no change in the trials to criterion in a passive avoidance test; however, there was clearly decreased step-down latency when tested 24 hr later (Vatanparast et al., 2013).  This finding suggests a change in memory but not learning.  After in utero exposure this effect was only seen in females.  In contrast, both genders (greater effect in males) were affected following postnatal exposure.  Using a novel object test, male mice (females not tested) exposed postnatally (PND8-11) to diazinon (0.5, 5 mg/kg/d) showed less exploration and discrimination of the new object (Win-Shwe et al., 2013).  This was significant at both doses (but no dose-response) when tested at PND49, and only the high dose group showed effects at PND84.  Mice exposed to diazinon (0.18, 9 mg/kg/d) throughout gestation were tested in a Lashley III maze, with no changes in the number of errors, suggesting no effect on learning (Spyker and Avery, 1977).  Thus, these data on diazinon suggest an effect on learning and/or memory (radial arm maze, passive avoidance, novel object recognition) but no changes in learning a maze task.  There was a lack of dose-response across studies.

Parathion exposure in rats (0.1, 0.2 mg/kg/d, PND1-4) produced no changes in T-maze spontaneous alternation (Timofeeva et al., 2008b).  These rats showed decreased working memory errors, indicative of improvement, at the low dose only (both males and females), and no changes in reference memory errors when tested at about 3 months of age.  However, Levin et al. (2008) tested littermates from the Timofeeva study beginning at 14 months, and reported increased working memory errors in male rats treated with the low dose only. Interestingly, there is a difference in direction of change and gender specificity compared to the Timofeeva data.  Reference memory errors were also increased at both doses (but no dose-response) in male rats only.  When the rats were tested again at 17 months, working memory errors were increased at both doses (but no dose-response), again only in males.  There was no effect on reference errors at 17 months, and no change in either parameter when the rats were again tested at 19 months.  In a study by Stamper et al. (1988), rats (only males were tested) exposed postnatally to parathion (1.3, 1.9 mg/kg/d, PND5-20) showed decreased spontaneous alternations in a T-maze at both doses (dose-response evident), and increased working memory errors in a radial arm maze at both doses (but no dose-response).  Reference memory errors were not altered.  Al-Hachim and Fink (1968) administered parathion (3 mg/kg/d) during either the first, second or third week of gestation in mice, and reported no effect on conditioned avoidance (sex not mentioned) following any exposure period.  Overall, the radial arm maze showed effects of parathion in several studies, but the direction of change, specificity of errors, and gender selectivity differed.  The results with the T-maze were contradictory, with one study out of two reporting effects.  

Radial arm maze performance was affected by methyl parathion given directly to pups PND1-21 using an incrementally increasing dose schedule (Johnson et al., 2009).  The middle (0.2 to 0.6 mg/kg/d) and high (0.3-0.9 mg/kg/d) dose groups increased both working and reference memory errors.  The lowest dose group (0.2 mg/kg/d throughout dosing) also increased reference memory errors; however, on this measure all dose groups had similar averages.  Only males were affected on all measures.  Rats dosed with methyl parathion (1 mg/kg/d, GD7-15) were trained to go to a specific side in T-maze, and the correct side reversed five times (Crowder et al., 1980).  Treated rats had more trials to criterion only on second and fifth switch (note, the text claimed there was an effect on the 4[th] switch, but the figure does not show it as significant).  Effects on only certain reversals is difficult to interpret, and may be a reflection of the multiple t-tests used to analyze the data.  Males and females were tested but data were not provided for each sex separately.  There were no effects on passive or active avoidance in rats (no mention of gender) exposed to methyl parathion (1, 1.5 mg/kg/d, GD6-20) (Gupta et al., 1985).  The low-dose group only showed slower latency to bar press during operant shaping, and more days to asymptote; however, details of operant training and schedule were not provided, the sample size was extremely small (n=4/group), and high variability was mentioned.  Thus, the most consistent effect was seen in the radial arm maze (even though there was no dose-response), and other tests were either negative or the data were inconclusive.

Several other OPs were tested in different cognitive tasks, but there is no more than one report for any specific pesticide.  After single-trial passive avoidance training, male rats (females not tested) treated with dichlorvos (8 mg/kg/d, GD6-15) showed faster latency to cross when tested 7 days later, suggesting delayed retention (Lazarini et al., 2004).  With fenitrothion, male rats (females not tested) exposed gestationally (5, 10, 15 mg/kg/d, GD7-15) were conditioned to climb a pole to escape shock (Lehotzky et al., 1989).  The mid and high dose groups (dose-response evident) showed more escapes, were faster, and reached criterion faster than controls.  This same pattern seen with reacquisition after a period of extinction, during which time there were no group differences.  There was no effect of oxydemeton methyl (0.5, 1.5, 4.5 mg/kg/d, GD6-15) on M-maze learning or memory in rats (Clemens et al., 1990).  There was also no effect on retention of single trial passive avoidance in mice exposed to methamidophos (1 mg/kg/d, PND3-9); however, the retention trial occurred at 3 hr instead of the more standard 24 hr or greater (Lima et al., 2013).

Table 2.1.1 Summary of Cognitive Outcomes

Early gestation
~GD1-7, ~GD8-14, GD6-15
GD7-15
Late gestation
~GD15-21, GD6-20, GD15-18
Gestation
GD1-birth
Early postnatal
PND1-4,
PND3-9
Late postnatal
PND8-11, PND5-20
Postnatal 
PND1-21
Radial arm maze

Diazinon: cognitive deficit  -  rat, low dose only, M & F[1] 

Parathion: improved cognition  -  rat, low dose only, M & F[2]

Parathion: cognitive deficit  -  rat, M not F[3]
Parathion: cognitive deficit  -  rat, no dose-response, only M tested[4]
Methyl parathion: cognitive deficit  -  rat, dose-response, M only[5]
T-maze spontaneous alternation

Diazinon: no effect  -  rat, M & F[1] 

Parathion: no effect  -  rat, M & F[2] 
Parathion: cognitive deficit  -  rat, dose-response, only M tested[4] 

T-maze learning
Methyl parathion: cognitive deficit  -  rat, sex not specified[6]

Lashley III maze

Diazinon: no effect  -  mouse, sex not specified[7] 

M-maze
Oxydemeton methyl: no effect  -  rat, M & F[8] 

Active avoidance
Fenitrothion: improved cognition  -  rat, dose-response, only M tested[9] 

Parathion: no effect  -  mouse, sex not specified[11] 
Methyl parathion: no effect  -  rat, sex not specified[10] 

Parathion: no effect  -  mouse, sex not specified[11]

Passive avoidance
Dichlorvos: cognitive deficit  -  rat, only M tested[12] 
Diazinon: cognitive deficit  -  rat, F not M[13] 

Methyl parathion: no effect  -  rat, sex not specified[10] 

Diazinon: cognitive deficit  -  rat, M & F[13] 

Methamidophos: no effect  -  mouse, sex not specified[14] 

Novel object recognition

Diazinon: cognitive deficit  -  mouse, no dose-response, only M tested[15]

Operant responding

Methyl parathion: cognitive deficit  -  rat, sex not specified[10]

[1] Timofeeva et al., 2008a
[2] Timofeeva et al., 2008b
[3] Levin et al., 2008
[4] Stamper et al., 1988
[5] Johnson et al., 2009
[6] Crowder et al., 1980
[7] Spyker and Avery, 1977
[8] Clemens et al., 1990

[9] Lehotzky et al., 1989
[10] Gupta et al., 1985
[11] al-Hachim and Fink, 1968
[12] Lazarini et al., 2004
[13] Vatanparast et al., 2013
[14] Lima et al., 2013
[15] Win-Shwe et al., 2013

Motor Activity

Despite being a commonly tested measure in these studies, only a few of the tested OPs produced changes in motor activity.  Rats exposed to dichlorvos (8 mg/kg/d, GD6-15) showed decreased open field locomotion at weaning (males not females), and decreased locomotion and increased immobility as adults (age not specified, only males tested) while rearing was not affected at either age (Lazarini et al., 2004).  Male rats exposed to fenitrothion (5, 10, 15 mg/kg/d, GD7-15) showed decreased horizontal activity in the high-dose group only on PND104, with an apparent but not significant effect at PND26, but no effect at PND36 (Lehotsky et al., 1989).  Open field testing of rats exposed to methyl parathion (1 mg/kg/d, GD7-15) showed what appeared to be increases only on PND23 and 54, with no differences on PND30, 44, 65 (also not PND18); however, the data are not compelling since statistics are not provided, and the text refers to the data as "a possible change" (Crowder et al., 1980).  Methyl parathion-treated rats (1 mg/kg/d, GD6-20) showed a decrease in locomotor activity "accommodation" (apparently the period that is 15-30 min into the activity session) in the low dose group only (Gupta et al., 1985).  

Several OPs consistently produced no changes on motor activity, regardless of exposure or test species.  No effects were seen with diazinon in rats exposed gestationally (1 mg/kg/d, GD15-18, Vatanparast et al., 2013) or postnatally (0.5, 2 mg/kg/d, PND1-4, Timofeeva et al., 2008a, or 1 mg/kg/d, PND1-4, Vatanparast et al., 2013), or in mice exposed gestationally (0.18, 9 mg/kg/d, GD1-birth, Spyker and Avery, 1977).  As with diazinon, there were no motor activity changes following parathion exposure postnatally in rats (0.1, 0.2 mg/kg/d, PND1-4, Timofeeva et al., 2008b, or 1.3, 1.9 mg/kg/d, PND5-20, Stamper et al., 1988) or in mice treated during either the first, second, or third week of gestation (3 mg/kg/d, Al-Hachim and Fink, 1968).  There was no effect on measures of activity reported in rats treated with methamidophos (1 mg/kg/d, GD6-15), although high variability of the measures was discussed (deCastro et al., 2000).  Methamidophos (1 mg/kg/d, PND3-9) also produced no effect in mice (Lima et al., 2013).  Finally, there was no effect of oxydemeton methyl exposure (0.5, 1.5, 4.5 mg/kg/d, GD6-15) on open field activity (Clemens et al., 1990).

In this review, effects on activity are only considered in tests designed specifically for that purpose.  During the course of other behavioral tests, e.g., radial arm or T-maze, speed or latency is often measured.  These ancillary activity measures were sometimes altered by treatment, but are not included in this domain, since they are not designed to specifically target motor activity.
Table 2.1.2. Summary of Motor Activity Outcomes

Task/Test Apparatus
Early gestation
~GD1-7, ~GD8-14, GD6-15
GD7-15
Late gestation
~GD15-21, GD6-20, GD15-18
Gestation
GD1-birth
Early postnatal
PND1-4,
PND3-9
Late postnatal
PND8-11, PND5-20
Open field
Dichlorvos: decreased activity  -  rat, only M tested[1]

Fenitrothion: decreased activity  -  rat, dose-response, only M tested[2]

Methamidophos: no effect  -  rat, sex not specified[3] 

Methyl parathion: increased activity  -  rat, sex not specified[4] 

Oxydemeton methyl: no effect  -  rat, M & F[5]

Parathion: no effect  -  mouse, sex not specified[6] 
Diazinon: no effect  -  rat, M & F[7] 

Parathion: no effect  -  mouse, sex not specified[6]
Diazinon: no effect  -  mouse, sex not specified[8] 
Diazinon: no effect  -  rat, M & F[7] 

Methamidophos: no effect  -  mouse, sex not specified[9] 
Parathion: no effect  -  rat, only M tested[10] 
Figure-Eight 

Diazinon: no effect  -  rat, M & F[11]

Parathion: no effect  -  rat, M & F[12] 

Donut 

Methyl parathion: decreased activity  -  rat, no dose-response, sex not specified[13]

1 Lazarini et al., 2004
[2] Lehotzky et al., 1989
[3] deCastro et al., 2000
[4] Crowder et al., 1980
[5] Clemens et al., 1990
[6] al-Hachim and Fink, 1968
[7] Vatanparast et al., 2013
[8] Spyker and Avery, 1977
[9] Lima et al., 2013
[10] Stamper et al., 1988
[11] Timofeeva et al., 2008a
[12] Timofeeva et al., 2008b
[13] Gupta et al., 1985

Anxiety/Depression

In a series of tests in rats, Roegge et al. (2008) showed that early postnatal exposure to diazinon (0.5, 2 mg/kg/d, PND1-4) produced behaviors suggesting higher anxiety at the high dose (decreased open arm time in an elevated plus maze), lower fearfulness (decreased time to start eating in novel environment) in both dose groups, and anhedonia (decreased chocolate milk preference) at the low dose only, but not depression (forced swim test).  These effects occurred only in males and did not show a clear dose-response for the novelty eating and chocolate milk preference tests.  Using the same tests (same laboratory; Timofeeva et al., 2008b), parathion exposure (0.1, 0.2 mg/kg/d, PND1-4) in rats (high dose, both sexes) increased time in the open arm in an elevated plus maze (suggesting decreased anxiety); however, there was also an increase in center crossings, indicating hyperactivity, that may confound overall interpretation.  These same rats showed no changes in novelty-suppressed feeding or in chocolate milk preference.  Thus, in these two studies the pesticides appear to have different effects on this functional domain.

There are only a few reports of these behaviors with other pesticides.  Rats treated with methyl parathion (1, 1.5 mg/kg/d, GD6-20) showed faster emergence from a cage, interpreted by the authors as lowered anxiety, in the low dose group only (Gupta et al., 1985).  With chlormephos exposure (~0.06, 0.6 mg/kg/d in drinking water, one week premating to weaning), adult mice (both males and females) in the high-dose group showed decreased time spent in the open arms and increased time in the closed arms (no change in latency) of an elevated plus maze, suggesting increased anxiety (Ceh et al., 2012).  Mice exposed to methamidophos (1 mg/kg/d, PND3-9) showed no differences in time spent in either arm of an elevated plus maze, but spent less time in center (Lima et al., 2013).  This was interpreted by the authors as effect on choosing arms, which they say is a cognitive effect; however, this measure is often interpreted to reflect only activity levels.  The same mice showed increased immobility in forced swim test, suggesting depressive-like behaviors.  Overall, these results are varied and did not consistently show a dose-response.  

Table 2.1.3. Summary of Anxiety/Depression Outcomes

Late gestation
~GD15-21, GD6-20,
GD15-18
Perinatal
premating-weaning
Early postnatal
PND1-4,
PND3-9
Elevated plus maze

Chlormephos: increased anxiety  -  mouse, M & F[1] 
Diazinon: increased anxiety  -  rat, dose-response, M not F[2] 

Methamidophos: no effect  - mouse, sex not specified[3] 

Parathion: decreased anxiety  -  rat, dose-response, M & F[4] 
Chocolate milk preference

Diazinon: increased anhedonia  -  rat, no dose-response, M not F[2] 

Parathion: no effect  -  rat, M & F[4] 
Novelty suppressed feeding

Diazinon: decreased fearfulness  -  rat, dose-response, M not F[2] 

Parathion: no effect  -  rat, M & F[4] 
Forced swim

Diazinon: no effect  -  rat, M & F[2] 

Methamidophos: increased despair  -  mouse, sex not specified[3] 
Open field behaviors
Methyl parathion: decreased anxiety  -  rat, no dose-response, sex not specified[5] 

[1] Ceh et al., 2012
[2] Roegge et al., 2008
[3] Lima et al., 2013
[4] Timofeeva et al., 2008b
[5] Gupta et al., 1985

Social Behavior

Only one study has used tests of social behavior using these OPs.  With exposure to fenitrothion (5, 10, 15 mg/kg/d, GD7-15), rats in mid and high dose (dose-response) spent more time actively interacting in conspecific pairs (Lehotsky et al., 1989).  The scarcity of data on this measure prevents any conclusions across OPs.

Sensory Function

In a study measuring response to a tactile stimulus, with and without an acoustic prepulse, male (but not female) rats treated with diazinon (0.5, 2 mg/kg/d, PND1-4) showed less prepulse inhibition at both doses; however, no dose-response was evident (Timofeeva et al., 2008a).  Using the same paradigm, rats treated with parathion (0.1, 0.2 mg/kg/d, PND1-4) showed a different pattern: lower response to the stimulus alone (high dose, both sexes), but no change in the inhibition produced by the prepulse (Timofeeva et al., 2008b).  Mice treated with diazinon through gestation showed no change in response to noise (auditory startle) or smell (olfactory orientation), but did show change in visual cliff behavior (more steps off a "cliff") which occurred only in females in the low dose group (Spyker and Avery, 1977).  Additional studies are needed for general conclusions regarding the effects of these pesticides on sensory function.

Table 2.1.4. Summary of Sensory Outcomes

Gestation
GD1-birth
Early postnatal
PND1-4,
PND3-9
Auditory
Diazinon: no effect  -  mouse, sex not specified[1] 

Tactile with or without prepulse

Diazinon: decreased sensory gating  -  rat, no dose-response, M not F[2] 

Parathion: decreased startle response  -  rat, dose-response, M & F[3] 
Visual
Diazinon: decreased function  -  mouse, no dose-response, F not M[1] 

Olfactory
Diazinon: no effect  -  mouse, sex not specified[1] 

[1] Spyker and Avery, 1977
[2] Timofeeva et al., 2008a
[3] Timofeeva et al., 2008b

Neuromotor Function

One study assessed neuromotor function in mice exposed to diazinon (0.18, 9 mg/kg/d, GD1-birth), and reported some changes in motor abilities measured at about 2 months of age; however, these were not consistent in dose or direction of change (Spyker and Avery, 1977).  Increased ability was suggested by a longer time to cling to a rod (both doses, no dose-response).  In contrast, there was less ability to stay on an increasingly inclined plane (both doses, no dose-response) or perhaps on a rotarod (group means not significant due to large variability, suggestive of effect at both doses).  

Rats in the high-dose group exposed to fenitrothion (5, 10, 15 mg/kg/d, GD7-15) fell off a rotarod faster on PND26 and PND104, but not PND36.  Only males were tested (Lehotsky et al., 1989).

There were no neuromotor changes in terms of rotarod performance in rats following exposure to parathion (1.3, 1.9 mg/kg/d, PND5-20) (Stamper et al., 1988) or methyl parathion (1, 1.5 mg/kg/d, GD6-20) (Gupta et al., 1985).  Overall, there is little support for conclusions of neuromotor outcomes following these pesticides, but more studies are needed.

Table 2.1.5. Summary of Neuromotor Outcomes

Early gestation
~GD1-7, ~GD8-14, GD6-15
GD7-15
Late gestation
~GD15-21, GD6-20,
GD15-18
Gestation
GD1-birth
Late postnatal
PND8-11, PND5-20
Rotarod 
Fenitrothion: decreased  performance  -  rat, dose-response, only M tested[1] 
Methyl parathion: no effect  -  rat, sex not specified[2]
Diazinon: no effect  -  mouse, sex not specified[3] 
Parathion: no effect  -  rat, only M tested[4] 
Inclined plane

Diazinon: decreased performance  -  mouse, dose-response, sex not specified[3] (Spyker)

Rod cling

Diazinon: increased performance  -  mouse, no dose-response, sex not specified[3] 

[1] Lehotzky et al., 1989
[2] Gupta et al., 1985
[3] Spyker and Avery, 1977
[4] Stamper et al., 1988

2.1.3	Integration with AChE Inhibition

There are some data with which to compare effective doses in these DNT studies with doses producing AChE inhibition.  A number of studies in this literature review included AChE inhibition (brain and/or blood) in their measurements.  Information on diazinon and parathion can be taken from a separate (non-behavioral) study (Slotkin et al., 2006) conducted in the same laboratory with the same dosing paradigm used in several studies (Levin et al., 2008; Roegge et al., 2008, Timofeeva et al., 2008a, 2008b).

Neurobehavioral effects of diazinon were reported in rats at doses of 0.5-2 mg/kg/d (Roegge et al., 2008; Timofeeva et al., 2008a).  Slotkin et al. (2006) reported that a dose of 0.5 mg/kg/d produced some (<10%, statistically significant) brain AChE inhibition when measured the day after the last dose.  At 2 hr after a higher dose (2 mg/kg/d; lower doses were not tested at 2 hr), there was greater brain inhibition (25-30%) compared to 24 hr (10-20%).  Thus it is probable that for diazinon, inhibition during and shortly after the dosing period (i.e., within hours) was greater at lower doses.  While there is no direct AChE data following diazinon exposure at 1 mg/kg/d (Vataparast et al., 2013), it can be assumed from the Slotkin data that this dose also inhibited brain AChE at some time during/after dosing.  No AChE activity was measured in mice by Spyker and Avery (1977), but the high dose of 9 mg/kg/d resulted in depressed weight gain, a sign of maternal toxicity.  There is no mention of toxicity at the lower doses (0.5, 5 mg/kg/d) used in mice by Win-Shwe et al. (2013).  
      
The parathion studies showing effects at 0.1 and 0.2 mg/kg/d (Timofeeva et al., 2008b; Levin et al., 2008) are also informed by the AChE inhibition presented in Slotkin et al. (2006) in which a dose of 0.1 mg/kg/d inhibited brain AChE 5-15% on the day after the last dose.  There are no AChE data available for a higher dose, 0.2 mg/kg/d, but Timofeeva noted 5% mortality in that dose group.  Much higher doses (1.3, 1.9 mg/kg/d in rats) were reported to produced 35, 68% brain inhibition on the day after the last dose (PND5-20), with 26, 36% inhibition persisting a week later (Stamper et al., 1988).

The methyl parathion postnatal incrementing dose paradigm in rats used by Johnson et al. (2009) produced brain AChE inhibition in all dose groups at the end of dosing, persisting for 10 to 20 days later; recovery was evident 30-40 days later.  The low dose (0.2 mg/kg/d throughout) produced 13-15% inhibition.  During gestational exposure to higher doses of methyl parathion at 1, 1.5 mg/kg/d, the dams had 20, 60% brain inhibition at the end of dosing (GD6-20) (Gupta et al., 1985).  At the high dose, cholinergic signs and increased resorptions were noted.  Furthermore, the offspring (fostered to control dams) showed brain inhibition as high as 50% in both dose groups when measured at birth and also PND7, 14, 21, and 28.  The low dose showed recovery at PND28, but not the high dose.

In the study of rats treated with fenitrothion, there was postnatal mortality of 16-17.5% at all doses (5, 10, 15 mg/kg/d, GD7-15), compared to 5% in controls (Lehotsky et al., 1989).  In separate study of rats, 5, 25 mg/kg on GD19 produced 40, 80% brain AChE inhibition in dams, and fetal brains showed about 90% inhibition (no dose response) (Sochaski et al., 2007).  Assuming similar responses, the postnatal mortality observed in the Lehotsky study could be at least partly due to this high degree of AChE inhibition in both dams and fetuses. 

In rats, nonpregnant females dosed methamidophos 1 mg/kg/d for 10 days showed 16% plasma AChE inhibition, but brain AChE was apparently not measured.  This suggests that dams treated at the same dose in a study by deCastro et al. (2000) most likely experienced some plasma inhibition.  Mouse pups dosed with methamidophos 1 mg/kg/d PND3-9 showed ~36, 46% brain inhibition 1, 4 hr after first dose, 53, 61% inhibition at 1, 4 hr after last dose, and ~19% brain inhibition the day after last dose (Lima et al., 2013).  The mice therefore experienced considerable brain AChE inhibition throughout dosing.

In the oxydemeton methyl study, dams showed 22-68% brain AChE inhibition on the day after the last dose (0.5-4.5 mg/kg/d, dose response), and 5 days later (GD20) there was 20-54% brain AChE inhibition.  Fetal brains taken the day after dosing showed no inhibition (Clemens et al., 1990); however, there were no fetal AChE tissues collected during or shortly after the dosing period when AChE inhibition would be greatest.

Overall, in the studies for which there are direct or comparable data, it is clear that the dosing paradigms produced AChE inhibition and in some cases maternal toxicity.  Indeed, there are no studies reporting or even suggesting a lack of AChE inhibition in the dam and/or fetus/pup at any time during dosing.  Thus, it is not known whether exposure paradigms that do not inhibit AChE would produce any neurobehavioral effects.

2.1.4	Summary of Findings from the Developmental Neurotoxicity (DNT) Guideline Studies

DNT studies have been submitted for 20 OPs, summarized in Appendix 2.  These studies follow the US EPA guideline 870.6300 and/or OECD guideline 426 which require testing of motor activity, acoustic startle response, learning and memory, and brain morphometrics in the offspring around weaning and also in adulthood.  In general, these studies provide exposure during development either via diet or oral gavage dosing, including direct dosing of the pup preweaning.  As with the literature studies, these submitted studies have shortcomings such as inappropriate statistical analyses, limited methodological information and presentation of results.  Many measures tend to show high variability, which reduces their interpretability and utility.

In order to compare the submitted guideline and published studies under the scope identified under Section 2.1 and to be consistent with the chlorpyrifos 2012/2014 review, only changes that occurred after dosing had ended (i.e., shortly after weaning or as adults) were considered here.  Across the seven submitted studies that reported effects, there are mostly changes in acoustic startle reactivity, cognitive function, and to a lesser extent, motor activity.  Some OPs altered multiple domains, others only one.  There are both submitted guideline studies and literature studies for only four OPs: diazinon, methyl parathion, methamidophos, and dichlorvos.  Diazinon produced cognitive changes but no effects on motor activity or acoustic startle in the DNT: these results are generally in agreement with published studies.  Male offspring in the high dose group (~33 mg/kg/d in the dam diet) showed increased errors and longer latency in Biel maze performance at both PND24 and PND62, and similar effects were seen in females but only in the middle dose group (~3.4 mg/kg/d via diet) at PND24.  Following methamidophos exposure, female rats in the middle and high dose groups (~1.7, 5.2 mg/kg/d in the dam diet) showed decreased peak amplitudes of the startle response, which was statistically significant at PND38 and apparent but not significant at PND60.  In contrast to the literature reports of cognitive and motor effects of methyl parathion, there were no reported changes in the submitted guideline study.  The submitted study of dichlorvos was uninterpretable due to high pup mortality in all groups, including control.
 
In these studies, AChE activity was assessed as part of the DNT itself, or by means of a separate study comparing the response in pups and adults (comparative cholinesterase, or CCA, studies).  Thus, in almost all guideline DNT studies there are adequate data describing AChE inhibition in the pups at some time during development.  It is clear from these DNT studies that the doses used did produce AChE inhibition in the offspring, sometimes at all doses tested.  It was noted that in these studies, most of the reported effects occurred before weaning, which is the period during which there was likely to be ongoing AChE inhibition.

Thus, as with the literature studies, there are scant data that could inform potential neurodevelopmental changes occurring at doses lower than those needed to inhibit AChE.  Furthermore, there is little consistency in patterns of effects across studies or chemicals.  Thus, there is uncertainty as to whether lower, non-inhibiting exposures are developmentally neurotoxic; this uncertainty was described in the chlorpyrifos reviews and remains applicable for the available data for other OPs as well.
 
2.1.5	Conclusions on In Vivo Laboratory Animal Studies

For chlorpyrifos, there are >30 papers on developmental neurotoxicity; for the remaining OPs, the literature is sparse with very few studies for each OP (including DNT guideline studies).  The studies span over decades, and many of the lower quality studies were the earlier ones; however, some very recent papers also have significant deficits.  Methodological detail is lacking, inappropriate statistical analyses are applied, results are cursorily described and/or inaccurately presented, and interpretation of some behavioral changes is faulty.  Overall, most studies have significant shortcomings and/or are of low quality.

The most commonly tested behaviors considered aspects of cognition.  In the majority of studies, some sort of cognitive deficit was detected, especially with working memory performance (radial arm maze) and conditioned response retention (passive avoidance).  However, in many cases there was no dose-response, there was some gender specificity which did not replicate in multiple studies, and cognitive improvement instead of deficit was noted in a few papers.  Changes in motor activity in offspring were generally not reported, and the direction of change differed in the papers reporting such effects.  There is generally not enough information to make definitive statements about OP effects on other types of neurological disorders.

Few published papers included AChE measurements of the dams and/or offspring, but where measured, all doses used inhibited AChE to some degree.  Some papers even reported overt maternal and fetal toxicity.  This was also the case in the guideline studies, most of which included concurrent or supplemental data on AChE inhibition.  Since there are no studies with low doses that definitively do not inhibit AChE, there is no information in the animal literature that shows whether or not there would be developmentally neurotoxic outcomes at those lower exposures.

2.2	Epidemiology Research on OPs other than Chlorpyrifos

2.2.1	Overview of Literature Reviews: 2012/2014, 2015, and updated in 2016

In April 2012, EPA presented to the FIFRA Scientific Advisory Panel (SAP) its review and assessment of several epidemiological investigations of the potential adverse neurodevelopmental outcomes of in utero and early life exposure to chlorpyrifos. In this effort, EPA limited its review to studies conducted within three major US based prospective birth cohort studies: 1) Mother's and Newborn Study of North Manhattan and South Bronx conducted by Columbia University, referred to in this document as CCCEH 2) Mount Sinai Inner-City Toxicants, Child Growth and Development Study, or the "Mount Sinai Study/Cohort;" and 3) Center for Health Assessment of Mothers and Children of Salinas Valley (CHAMACOS) conducted by the University of California Berkeley, or "CHAMACOS Study/ Cohort." The conclusion of EPA's evaluation, supported by the FIFRA SAP (2008, 2012), was that "chlorpyrifos likely played a role in the neurodevelopmental outcomes observed in these studies." 

In 2015, the agency expanded its consideration of the epidemiological data to include studies of any OP pesticide; several different types of development and neurological, neurodevelopmental, and neurobehavioral health outcomes; studies performed in non-U.S. countries as well as US based studies; and non-cohort studies.  This 2016 updated literature review includes edits made in response to public comments on the 2015 literature review and addition of new epidemiology papers:  Yolton et al. (2013), Cartier et al. (2015), Rauh et al (2015), Ranaan et al, (2015), Engel et al. (2015), Fiedler et al. (2015), Harley et al (2016), Stein et al (2016), Rauh et al (2015), Ranaan et al, 2015; Engel et al. (2015), and Donauer et al (2016).The agency has developed individual study reviews for each study in the 2015 and 2016 updated literature review; these study reviews contain details about study design, results, strengths, uncertainties, and interpretation.  The study reviews can be found in USEPA, 2015b, USEPA, 2016, and USEPA,2016b.  This literature review document summarizes key information, strengths and uncertainties and provides a concise, coherent weight of evidence analysis.  

The 2012/2014 literature review was limited to studies from CCCEH, Mt. Sinai, and CHAMACOS; all studies from these cohorts are considered high quality.   In the CCCEH, Mt. Sinai, and CHAMACOS studies, mother-infant pairs were recruited for the purpose of studying the potential health effects of environmental exposures during pregnancy on subsequent child development. Importantly, each of these cohorts evaluated the association between prenatal chlorpyrifos or OP exposure with adverse neurodevelopmental outcomes in children through age 7 years.  For those studies in 2012/2014 literature review detailed summaries and evaluation, associated strengths and limitations, and accompanying detailed evidence table for the CCCEH, CHAMACOS, and Mt. Sinai cohorts can be found in the white paper for the 2012 SAP review, and the 2014 chlorpyrifos revised HHRA.  Limited summary information is provided here for consideration with the studies identified in the 2015 and updated 2016 literature reviews.

Publication bias is a type of bias associated with the kinds of academic research likely to be published in the open literature; generally, this bias tends towards lack of publishing of null findings.  However, in the case of the neurodevelopmental epidemiology studies on OPs, as noted by commenter, there are already significant amount of null findings reported in this body of evidence.  Moreover, these studies are derived from existing cohorts of mothers and children which are well-known to the science community; these kinds of cohorts are expensive and resource intensive to develop and maintain making the number of studies from other groups unlikely.  Given the amount of null findings already reported in the literature, the agency believes that publication bias with regard to lack of publishing of null findings does not impact the interpretation of the existing studies on neurodevelopmental outcomes associated with OPs.

The findings from the birth outcome studies are summarized in 2.2.7.  The major findings of CCCEH, Mt. Sinai, and CHAMACOS on neurodevelopmental outcomes are briefly summarized in Section 2.2.8.1. Results from other studies on neurodevelopmental outcomes are summarized in 2.2.8.2.   

2.2.2	Literature Search Methodology

To identify the epidemiological investigations of the association between OP exposure and adverse neurological, neurodevelopmental or neurobehavioral effects, EPA scientists queried PubMed/Medline and Web of Science directly on January 21 and January 23, 2015. In this literature search, emphasis was placed upon identification of all possible epidemiological studies available, and the ability to use the identical search string in both PubMed/Medline and Web of Science. The following search string was utilized:

      ((Chlorpyrifos OR Organophosphates) AND (prenatal OR neonatal OR perinatal OR in utero OR fetal OR foetal OR newborn OR infant* OR infancy OR preschool OR child* OR maternal OR mother* OR pregnan*) AND (neurodevelop* OR attention OR birth outcome* OR health outcome* OR birth height OR birth weight OR birth length OR cephalometry OR head circumference OR child development OR cognitive OR cognition OR developmental disability* OR fetal growth OR foetal growth OR foetal development OR fetal development OR intelligence OR memory OR neurological functioning OR psychomotor) AND (human)) English language only((Chlorpyrifos OR Organophosphates) AND (prenatal OR neonatal OR perinatal OR in utero OR fetal OR foetal OR newborn OR infant* OR infancy OR preschool OR child* OR maternal OR mother* OR pregnan*) AND (neurodevelop* OR attention OR birth outcome* OR health outcome* OR birth height OR birth weight OR birth length OR cephalometry OR head circumference OR child development OR cognitive OR cognition OR developmental disability* OR fetal growth OR foetal growth OR foetal development OR fetal development OR intelligence OR memory OR neurological functioning OR psychomotor) AND (human)) English language only((Chlorpyrifos OR Organophosphates) AND (prenatal OR neonatal OR perinatal OR in utero OR fetal OR foetal OR newborn OR infant* OR infancy OR preschool OR child* OR maternal OR mother* OR pregnan*) AND (neurodevelop* OR attention OR birth outcome* OR health outcome* OR birth height OR birth weight OR birth length OR cephalometry OR head circumference OR child development OR cognitive OR cognition OR developmental disability* OR fetal growth OR foetal growth OR foetal development OR fetal development OR intelligence OR memory OR neurological functioning OR psychomotor) AND (human)) (Filter: English language only)
      
With the aid of an EPA reference librarian, EPA also searched the following databases: PsycInfo, Agricola, Biosis, Embase, Enviroline, Gale Health and Wellness, Global Health, Pascal and Pollution Abstracts. EPA used similar, but modified search terms as listed above. Upon identification of the final set of relevant articles (n=38), limited hand-searching of the reference lists and citation mapping (Science citation index) of articles deemed to be most relevant to the review question was performed.
In 2015, EPA identified 299 articles across these several biomedical search engines. The determination of relevance to the study question was made by two EPA epidemiologists who agreed by consensus as to article disposition in the 2015 literature search.  Removing duplicates (56), there were 243 articles, and 79 were determined to be epidemiological investigations of potential relevance.  The 164 studies excluded from the analysis comprised 57 exposure only studies; 51 review articles; 33 reports of acute OP intoxication; 20 studies in non-human systems; and 3 were otherwise not relevant (See Appendix 4). Among the 79 potentially relevant epidemiologic studies, 41 were excluded; 17 articles were previously reviewed in 2012; 16 were epidemiological methods papers including exposure validation studies without an original epidemiological risk estimate; and 8 were otherwise not relevant for various reasons. Among the 40 remaining studies, 2 were additionally excluded (one was a duplicate study published a second time; the other did not make a measure of an OP pesticide. Therefore, 38 articles are included in the 2015 narrative literature review (referred to herein as "2015 Literature Review/Studies"). (See Appendix 5). 
No additional formal literature search was conducted prior to the development of this 2016 update.  However, the agency maintains active inquiry of the open scientific literature on issues related to OPs and the agency believes the added studies represent the available, relevant information.  Of the 9 new epidemiology studies, two are meta-analyses.  
The following sections provide the results of this literature review. Section 2.2.3 describes the breadth and depth of the 2015/2016 literature review, with Section 2.2.4 summarizing the approach for assigning a quality ranking and Section 2.2.5 providing the results of this quality ranking.  In Section 2.2.6, these studies were further analyzed with focus on identification of the most appropriate exposure assessment and relevant outcomes for this assessment.  Studies focusing solely on birth outcomes are discussed in Section 2.2.7.  However, the emphasis in this assessment is on those studies focusing on neurodevelopmental outcomes, which are discussed in detail in Section 2.2.8 and summarized in Section 2.2.9.  

2.2.3	Breadth and Depth of the 2015/2016 Literature Review

Key features of each of the 47 articles in the 2015/2016 literature review are summarized in Table 2.2.5.1-1, 2.2.5.1-2, and 2.2.5.2-1 as well as Appendix 3. These articles cover wide range of study designs, study locations and time periods, and exposure and outcome measurement approaches and are listed in Table 2.2-a. The majority of the studies utilized a cross-sectional or a prospective birth cohort study design.
 Mothers and Newborn Study of North Manhattan and South Bronx conducted by Columbia University, New York, NY (See tables 2.2.5.1-1 & 2.2.5.1-2)
 Mount Sinai Inner-City Toxicants, Child Growth and Development Study, New York, NY (See tables 2.2.5.1-1 & 2.2.5.1-2)
 Center for Health Assessment of Mothers and Children of Salinas Valley (CHAMACOS) conducted by the University of California Berkeley (See tables 2.2.5.1-1 & 2.2.5.1-2)
 Denmark, Birth Cohort (Andersen et al. 2015)
 Saint Peter's University Hospital, New Brunswick, New Jersey, Birth Cohort (Barr et al. 2010)
 Early Life Exposures in Mexico to Environmental Toxicants (ELEMENT), Mexico City, Mexico, Birth Cohort (Fortenberry et al. 2014, 2014a)
 EcoSalud Project, Cayambe-Tabacundo region, Ecuador, Infant and Young Child Cohort (Handal et al. 2007, 2007b, 2008)
 University Hospital of Heraklion, Crete, Greece, Birth Cohort (Koutroulakis et al. 2014)
 Children's Pesticide Survey (CPS), Yuma County, Arizona (Lizardi et al. 2008)
 Infancio y Medio Ambiente (INMA) (Environment and Childhood), Spain, Birth Cohort (Llop et al. 2013)
 Perturbateurs endocriniens: Étude Longitudinale sur les Anomalies de la Grossesse, l'Infertilité et l'Enfance (PELAGIE), Brittany, France, Birth Cohort (Petit et al. 2010; Cartier et al, 2015)
 Health Outcomes and Measure of the Environment (HOME), Cincinnati, OH, Birth Cohort (Rauch et al. 2012; Yolton et al, 2013; Donauer et al, 2016; Harley et al, 2016; Engel et al, 2015)
 Embilipitiya Base Hospital, Southern Sri Lanka, Birth Cohort (Samarawickrema et al. 2008)
 Ontario Farm Family Health Study, Ontario, Canada, Birth Cohort (Savitz et al. 1997)
 Childhood Autism Risks from Genetics and the Environment (CHARGE), California, Child Cohort (Shelton et al. 2014)
 Secondary Exposure to Pesticides Among Infants, Children and Adolescents (ESPINA), Pedro Moncayo County, Pichincha, Ecuador, Child Cohort (Suarez-Lopez et al. 2012, 2013, 2013a)
 Shenyang, China, Birth Cohort (Zhang et al. 2014)
 Rice and aquaculture farming regions outside of Bangkok, Thailand (Fiedler et al, 2015)
Exposures were assessed via environmental samples, biomarkers, and/or proxy methods.
Only one study used environmental samples as an exposure measure - methyl parathion in household wipe samples (Ruckart et al., 2004).
Among the studies, 11 biomarkers were used to assess exposure. One study utilized a direct measure of OP pesticide  -  chlorpyrifos (CPF) in maternal and cord serum (Barr et al., 2010). Several studies looked at specific OP parent metabolites (IMPY, 3,5,6-trichloro-2-pyridinol [TCPy], PNP, MDA). The majority of studies used non-specific OP biomarkers: dialkyl phosphates (DAPs), diethyl phosphates (DEPs), and dimethyl phosphates (DMPs). Two effect biomarkers, AChE and butyl cholinesterase (BuChE), were used as the exposure metric. 
Proxy exposure methods included questionnaire and non-questionnaire approaches. For example, questionnaire-based exposures included maternal and or paternal self-report of living with an exposed worker, occupational exposure/ employment, and home pesticide use and child outdoor play exposure. Non-questionnaire-based exposures included Community/area of residence, distance to treated area/farm, percent of area treated with pesticide, level of urbanization, pounds OP pesticide used/year, and pesticide spray season. 
Likewise, there were numerous outcome measures examined across the studies, falling into six broad categories: birth characteristics, autonomic nervous system (ANS) effects, Attention Deficit Hyperactivity Disorder (ADHD)/attention problems, autism, general neurodevelopment (cognitive, behavioral, IQ), and physiological effects. The most common outcome measures were birth characteristics (with birth weight, birth length, head circumference and gestational age being the most frequent) and neurodevelopment tests and test batteries. Table 2.2-b lists the many specific neurodevelopment tests employed in the studies. Most studies utilized more than one test, and few tests were utilized in more than one study.

Other features that varied widely among the studies include:
Study periods ranged from 1991 to 2012 (reports were published from 1997 to 2016).
Study sizes varied from 25 to 3,159 participants.
Children's ages ranged from newborns to age 15 years.
 Pre-natal exposures were assessed in 23 reports, post-natal exposures in 14 reports, and both pre- and post-natal exposures in 7 reports.

Table 2.2-a	Study Designs, Exposure Measurement Methods, and Outcome Measurement Methods Used across the 2015/2016 
                                 Study design
                               (# studies used)
                             Exposure measurement
                               (# studies used*)
                              Outcome measurement
                               (# studies used*)

Prospective Cohort (16)
Retrospective Cohort (3)
Case-control (2)
Cross-sectional (16)
Ecological (1)
Biomarkers
AChE - acetyl cholinesterase (maternal 1, child 2)
BuChE - butyl cholinesterase (child 1)
CPF -chlorpyrifos parent (maternal 1, cord serum 1)
DAP  -  dialkyl phosphate (maternal 9, child 10, amniotic fluid 1)
DCCA - dimethylvinylcyclopropane carboxylic acid (child 1)
DEP  -  diethyl phosphate (maternal 3, child 8, amniotic fluid 1)
DETP  -  diethylthiophosphate (maternal 2)
DEDTP  -  diethyldithiophosphate  (maternal 2)
DMP  -  dimethyl phosphate (maternal 8, child 8, amniotic fluid 1)
DMTP  -  dimethylthiophosphate (maternal 1)
DMDTP  -  dimethyldithiophosphate (maternal 2)
DZN  -  diazinon parent (1)
IMPY  -  diazinon metabolite (child 1)
MAL - malathion parent (0)
MDA - malathion metabolite (maternal 1)
OP - organophosphate (cord blood 1)
PNP - para-nitrophenol (child 2)
TCPy - chlorpyrifos metabolite (maternal 1, child 2)

Environmental
Methyl parathion - Household wipe (1)

Proxy (questionnaire)
 Maternal occupational exposure/ employment (6)
 Paternal occupational exposure/ employment (2)
 Living with exposed worker (1)
 Home and outdoor play exposure (1)
   
Proxy (non-questionnaire)
 Community/area of residence (3)
 Distance to treated area/farm (2)
 Percent of area treated with pesticide (1)
 Level of urbanization (1)
 Pounds OP used/year (1)
 Pesticide spray season (1)
Birth characteristics (10)
    Birth Weight/LBW/FGR (records 7, NP 1, report 2)
    Birth Length (records 3)
    Head circumference (records 3, NP 1)
    Abdominal circumference (records 1)
    Gestational age (GA)/preterm (report 2, records 3)
    Ponderal index (records 1)
    Placental maturity index (1)
    Spontaneous abortion/miscarriage (report 1)
    Altered sex ratio (report 1)

Autonomic Nervous System (4)
   

ADHD/attention problems (2)
    Diagnosis DISC-IV (1)
    Use of medication (1)
    Screening (CPRS-R, CPT, BASC-PRS) (1)

Autism (2)
    CA Department of Developmental Services (CDDS) reports
    US Individuals with Disabilities Education Act (IDEA) reports
    Autism spectrum disorders (ASD)
    Autism Diagnostic Observation Schedule (ADOS) combined with ADI-R.

Neurodevelopmental (ND) test/battery  (19)  -  see Table 2.2-2b for details

IQ (57) - see Table 2.2-2b for details

Physiological (3)
    AChE activity, child
    BuChE activity, maternal
    Antioxidant status: superoxide dismutase (SOD) activity
    Fetal oxidative stress: malondialdehyde (MDA) concentrations)
    Fetal DNA fragmentation: electrophoresis
    Respiratory symptoms, child

*A single study may include biomarkers at multiple endpoints and / or multiple outcome measurements; thus the total number of studies used may exceed the total number of studies in this manuscript

Table 2.2-b	Detailed Neurological Outcome Measurement Methods Used across the  2015/2016 Review Studies
Outcome measurement
Neurodevelopmental (ND) test/battery  (USA-based studies)
 Pediatric Environmental Neurobehavioral Test Battery (PENTB) : cognitive, motor, sensory, and affect domains
       Developmental Test of Visual-Motor Integration (VMI)
       Kaufman Brief Intelligence test (K-BIT)
       Purdue Pegboard
       Story Memory and Story Memory-Delay from Wide Range Assessment of Memory and Learning 
       Trail-Making test, Part A and Part B
       Verbal Cancellation test
 Children's Memory Scale (CMS)
 Bayley Scales of Infant Development, Second Edition (Bayley-II)
    Mental Developmental Index (MDI)
    Psychomotor Developmental Index
    Clinical Evaluation of Language Fundamentals-Preschool, Second Edition
    Archimedes spirals
    NICU Network Neurobehavioral Scale (NNNS)
 Behavioral measures:
       The Child Behavior Checklist/4-18 
       The Teacher Report Form
 Developmental delay (DD):
       Mullen Scales of Early Learning (MSEL)
       Vineland Adaptive Behavioral Scale (VABS)
       Reciprocal social interaction: Social Responsiveness Scale
ND test/battery  (Non-USA-based studies)
 Bayley Scales of Infant Development - mental and psychomotor development
 Behavioral Assessment and Research System (BARS): Memory and attention, response speed and coordination, visual memory, attention, divided attention, recall and recognition memory, dexterity, hand-eye coordination
 Figure drawing task: child's perception and dexterity
 Long-term memory test
 Ages and Stages Questionnaire (ASQ)  -  communication, fine motor, gross motor, problem solving, and personal - social skills
 Strengths and Difficulties Questionnaire, parent version (SDQ): behavioral problems
 NEPSY-II test (trained examiners): general assessment battery:
       attention and executive functioning 
       language
       memory and learning
       sensorimotor (visuomotor precision), 
       visuospatial processing
       Statue and Knock Tap
 Neonatal Behavioral Neurological Assessment (NBNA)
       Behavior
       Passive Tone
       Active Tone
       Primary Reflexes
       General Assessment
 Reach-and-grasp, bi-manual coordination: Prehension abilities
 UC Berkeley Preferential Looking Test Cards: Visual acuity skills
 Visual Motor Integration (VMI), Beery-Buktenica, 4th Ed.
 Finger Tapping test: Manual motor speed
 Catsys equipment: Simple reaction time
 Conners' Continuous Performance Test II (CPT II, v5): Attention
 Woodcock-Johnson III Tests of Cognitive Abilities (WJ-III) Verbal Comprehension test: Long-term memory and language function
 Visuospatial performance and memory functions:
       Raven's Colored Progressive Matrices 
       Stanford - Binet Copying Test, 4th ed
 Physical examination (social response, spontaneous motility, involuntary movements, Romberg's sign, walking straight line, standing on one leg, number hops, biceps and patellar reflexes, finger opposition, diadochokinesis, finger - nose coordination, hearing, vision)
 Gesell Development Schedule (GDS): motor, adaptive, language, and social
 Santa Ana Form Board: Motor coordination
 Child's developmental delay - Parent interview
IQ
 Wechsler Intelligence Scale for Children-Revised (WISC-R) Digit Span Test, Card sorting Test
    Wechsler Intelligence Scale for Children, 4th edition (WISC-IV)
    Wechsler Preschool and Primary Scale of Intelligence, Third Edition
 Stanford-Binet Memory for Sentences and Digit String test
 Recall and recognition test

2.2.4	Study Designs & Considerations for Study Quality Evaluation

2.2.4.1	Study Designs
Besides the two meta-analyses, four basic study designs were used in the literature reviewed for this document: cohort study, case-control study, cross-sectional study, and ecologic study. The first of these two constitute the two basic types of observational (i.e. non-interventional) studies used to evaluate relative incidence of health and disease outcomes by exposure status. The latter two are generally considered descriptive or hypothesis generating study designs, though they too can be used to test hypotheses regarding relative prevalence of health outcomes and, under certain conditions, incidence as well. 

Cohort Study

A commonly used design in this literature was the cohort study (See Tables 2.2.5.1-1, 2.2.5.1-2, and Appendix 3 for examples). In a typical cohort study, individuals are classified according to exposure status (i.e., presence, absence, or magnitude of exposure), and then followed over time to quantify and compare the development (i.e., incidence) of the health outcome of interest by exposure group. Conceptually, the non-exposed comparison group in a cohort study provides an estimate of the incidence of the outcome among the exposed, had they, counter-to-fact, not been exposed. Apart from chance variations, a valid cohort study comparing exposed individuals to non-exposed individuals provides an estimate of the relative risk (or rate) of the disease associated with exposure. Ideally, the exposed and non-exposed groups are exchangeable, in the sense that switching the exposed to non-exposed, and non-exposed to exposed would yield the same measure of association (e.g., relative risk). If this were the case then, apart from chance, a cohort study would yield a measure of association equivalent to that produced in a corresponding (intervention) study where exposure status was randomly assigned. 

The chief advantage of the cohort study design is that it affords the investigator the opportunity to avoid and/or adjust for potential biases (i.e., selection bias, information bias, and confounding). Cohort studies also allow for discernment of the chronological relationship between exposure and outcome, and can be particularly efficient for studying uncommon exposures. The primary disadvantage of the cohort study design is logistical inefficiency with respect to the necessary time, expense, and other resources needed to conduct them. Cohort studies are particularly inefficient for evaluating associations with rare outcomes and diseases with long induction or latency periods. Though prospective studies are often logistically less efficient relative to other study designs (e.g., the case-control study), these logistical concerns can be minimized in cohort studies of short duration, such as those used to evaluate prenatal OP pesticide exposure effects on birth outcomes or other outcomes of neonatal development assessed shortly after birth. 

Two sub-categories of cohort studies  -  prospective and retrospective - are often applied to distinguish between studies in which the health outcome has occurred (retrospective study), or has not occurred (prospective study) at the time the investigators initiate the study. This distinction is important primarily as it pertains to the potential differences in the quality (e.g., completeness, accuracy, and precision) of information that can be ascertained by the investigators, and also as it relates to potential sources of bias. Although not always true, the prospective study design is considered the preferable of the two, as investigators can potentially have more choices in determining how exposure, outcome, and covariate information is collected. In a retrospective study conducted to evaluate the same hypothesis, by contrast, the investigators would have to rely on exposure information such as maternal self-report. Such reporting is subject to (human) errors in recall. Moreover, the outcome status of the child (i.e., whether a child has developmental delays that are known to the mother) may influence the recall of prenatal OP pesticide exposure by the mother.

Case-control Study
In a typical case-control study (see, for example, Dawbrowski et al., 2003 in Appendix 3 and Shelton et al. 2014 in Table 2.2.5.2-1), individuals are classified according to their outcome status (i.e., cases who have developed the outcome of interest, and controls who represent the population from which the cases arise). The relative odds of exposure are then compared between cases and controls. The primary advantage of case-control studies is that they are logistically efficient relative to cohort studies. In fact, properly conducted case-control study can be conceptualized as a cohort study with efficient sampling of exposure among the cohort, yet they can often be conducted at a fraction of the cost, in a fraction of the time as a corresponding cohort study. Case-control studies can be used to examine associations between multiple exposures and a given health outcome. They are particularly efficient for evaluating rare outcomes but are inefficient for studying uncommon exposures. The primary weakness of the case-control study is the potential for selection bias, which arises if the exposure distribution among the control subjects is not representative of the exposure distribution among the population that gave rise to the cases. Case-control studies that rely on self-reported exposure measures are also susceptible to information bias. 

Cross-sectional study
Cross-sectional studies (see, for example, Suarez-Lopez et al. in Table 2.2.5.2-1and Grandjean et al. 2006 in Appendix 3) are used to evaluate associations between exposure and outcome prevalence in a population at a single point in (or period of) time. The primary advantage of a cross-sectional study is logistical efficiency; they are relatively quick and inexpensive to conduct, as a long period of follow-up is not required, and exposure and outcome assessments occur simultaneously. Cross sectional studies have three primary potential disadvantages: 1) potential difficulty in discerning the temporal relationships (i.e., whether the exposure precedes the outcome); 2) estimating outcome prevalence rather than incidence of the outcome; and 3) the possible overrepresentation of cases of the outcome with long duration relative to the average in the population, and often with a better prognosis. 

Ecological study
Ecological studies are used to evaluate associations between exposures and outcomes using population-level rather than individual-level data. For example, Nevison (2014, Appendix 3) uses annual estimates of pesticides applied to crops and population level autism prevalence to assess the association between OP pesticide exposure and autism. The primary advantages of ecological studies are related to logistical efficiency, as they often rely on pre-existing data sources and require no individual-level exposure, outcome, or covariate assessments. The primary weakness of the ecologic study is the potential for confounding and resultant inappropriate extrapolation of associations observed on the aggregate-level to associations on an individual level. The mistaken belief that associations observed at the population level exist at the individual level is referred to as the ecological fallacy. 

In judging an individual study's contribution to the strength of evidence in the epidemiologic literature base, the following hierarchy of observational study designs was considered (from most to least preferred): prospective cohort study, retrospective cohort study, case-control study, cross-sectional study, ecological study. It is important to note, however, that this hierarchy of study designs reflects the potential for the collection of high quality information (related to exposure, outcome, confounders, and effect modifiers) and potential for efficient and valid estimation of the true association. Thus, in deliberating on quality, care has been taken to consider the circumstances and particulars of each individual study. For example, a well-conducted case-control study of a rare outcome can provide much higher quality evidence vis-à-vis the association of interest than a poorly conducted prospective cohort study of the same relationship. For this report, the placement of the study design in the aforementioned hierarchy of observational study designs was but one facet of the judgment of study design quality. Additional consideration was given to whether the study was well conducted, independent of study design type. The particulars of a study's design, specifically the design elements employed to minimize and adjust for biases, were also considered. Finally, the relevance of each study with respect to the association of primary interest in this initiative, namely the relationship between prenatal (and early life) OP pesticide exposures and fetal and child neurodevelopment was considered.

2.2.4.2	Considerations for Study Quality

This section summarizes how specific study characteristics factored in overall quality category.  [Note:  these study quality considerations are specific to issue of relevance to this document, namely potential for neurodevelopmental effects of OPs.  These considerations are considered `fit for purpose' under this context and could differ in another regulatory or scientific context.]   

The literature base evaluated is heterogeneous, as noted in Section 2.2.3. Pesticide exposure assessments variously relied on, for example, exposure biomarkers, maternal self-reports, and other proxy indicators of OP pesticide exposures. Outcome assessments were similarly varied, relying, for example, on biomarkers of biological effects, birth records, maternal self-reports, and clinical instruments designed to evaluate neurocognitive and neurobehavioral development. These design elements have potential impacts on study quality and relevance to this document. Each study was therefore judged to be of high, moderate, or low quality in five domains effecting study quality: exposure assessment, outcome assessment, confounder control, statistical analysis, susceptibility to bias (See Table 2.2.4-1 for general considerations under each domain).

Table 2.2.4-1	Study Quality Considerations
Parameter
High

Moderate

Low

Exposure assessment
Exposure assessment includes information on specific OP a.i.'s (e.g., CPF, MAL), or urinary metabolite (TCPy, IMPy,), or high quality questionnaire based chemical specific exposure assessment during relevant exposure window (pre-natal, early life)
Non-specific biomarker of exposure (DAP), or effect (AChE/BuChE), or questionnaire based individual level information on the OP class, or sub-class
Low quality questionnaire based exposure assessment, or ecologic exposure assessment, with or without validation 
Outcome Assessment
Standardized tool, validated in study population; or, medical record review with trained staff (birth characteristics)
Standardized tool, not validated in population, or screening tool; or, medical record review, methods unstated (birth characteristics)
Selected sections of test, or maternal report, other; or, maternal/paternal self-report (birth characteristics)
Confounder control
Good control for important confounders relevant to OP-ND question, and standard confounders
Moderately good control confounders, standard variables, not all variables for OP-ND question
Multi-variable analysis not performed, no adjustments
Statistical Analysis
Appropriate to study question and design, supported by adequate sample size, maximizing use of data, reported well (not selective)
Acceptable methods, questionable study power (especially sub-analyses), analytic choices that lose information, not reported clearly 
Minimal attention to statistical analyses, comparisons not performed or described clearly 
Risk of (other) bias (selection, differential misclassification, effect size magnification, other)
Major sources of other potential biases not likely present, present but analyzed, unlikely to influence magnitude and direction of the risk estimate
Other sources of bias present, acknowledged but not addressed in study, may influence magnitude but not direction of estimate
Major study biases present, unacknowledged or unaddressed in study, cannot exclude other explanations for study finding
(Adapted from Munoz-Quezada et al. 2013)

2.2.4.2.1	Exposure Measures
There were three major categories of exposure assessment employed in this literature: exposure biomarkers, participant-reported proxy exposure via questionnaire, and objectively obtained proxy indicators of exposure. Although one study included environmental wipe sampling results in the exposure assessment, urinary biomarker measures were also included and no differentiation of the two approaches was presented, so this exposure assessment category has not been included. The merits and the disadvantages of the three primary exposure assessment strategies are discussed below.  For this evaluation, studies employing exposure assessments that quantified biomarkers of specific OP pesticides (e.g., chlorpyrifos, malathion, diazinon), or urinary metabolites of these pesticides (e.g., TCPy, malathion dicarboxylic acid, 2-isopropyl-4-methyl-6-hydroxypyrimidine), or high-quality chemical-specific exposure quantitation during relevant exposure-time windows (i.e. pre-natal, early life) were given the highest weight. Studies that quantified levels of non-specific biomarker of OP pesticide exposure (e.g., DAPs), or exposure effects (e.g., AChE, BuChE,) were given a moderate weight. Studies relying on high-quality survey-based individual-level information on pesticide exposure were also assigned a moderate weight. Exposure assessments that only crudely or subjectively classified pesticide exposures and ecologic and other proxy measures of exposure were assigned a low weight.

Many studies used questionnaire-based exposure assessments in which study participants (typically mothers) self-reported their exposures, in addition to, or instead of, quantifying OP pesticide biomarkers in samples of biological media. These exposure assessments typically include querying OP pesticide exposure directly, or asked study participants to report on behaviors and conditions associated with pesticide use (e.g., occupation, tasks). Such reporting likely misclassifies actual OP pesticide exposure. If conducted as part of a prospective exposure assessment, these errors are likely to be non-differential with respect to the outcome(s) of interest. In the context of a retrospective assessment in which the mother has knowledge of the outcome status of the child, these errors may be differential or non-differential.

Several studies used proxy measures (including ecological indicators) of pesticide exposure. These included, for example questions about occupational use and exposure to pesticides, distance from residence to fields where pesticides were applied, the proportion of land in a specified area dedicated to agricultural uses, and occurrence of pregnancy during the pesticide application season. Again, substantial non-differential exposure measurement error/misclassification of exposure is likely.

The CHARGE study (Shelton et al., 2015) used a different method for exposure assessment.  This study used geospatial analysis to focus on the residential proximity to OP exposure and the association of this exposure with autism spectrum disorders.  OP exposure was assessed by Shelton et al  (2015) using data from the California Department of Pesticide Regulation, with five OPs accounting for a total of 73% of the exposure and each accounting for 10% or more of the exposure (chlorpyrifos, acephate, diazinon, bensulide, and dimethoate); eight OPs accounting for a total of 25% of the exposure and each accounting for 1% or more of the exposure (malathion, methyl parathion, azinphos-methyl, phosmet, oxydemeton-methyl, ethephon, naled, and methidathion); and eight OPs accounting for a total of 2% of the exposure and each accounting for 0.1% or more of the exposure (methamidophos, phorate, disulfoton, fenamiphos, coumaphos, parathion, ethoprop, and sulfotep).

Most of the studies reviewed herein assessed biomarkers of exposure quantified in samples of biological media (most often urine, but also blood, serum, and breast milk). These biomarkers were of three types: 1) OP pesticide residues, 2) metabolites of specific OP pesticides, and 3) non-specific OP metabolites.  

Two studies focusing on neurodevelopmental outcomes measured OP exposure using both DAPs and malathion dicarboxylic acid (a metabolite of malathion) (Eskenazi et al., 2007; Engel et al., 2007), with TCPy exposure also being measured in one of these studies (Eskenazi et al., 2007).  Additionally, two studies focusing on birth outcomes measured OP exposure by testing for specific OPs, with Whyatt et al. (2004) measuring chlorpyrifos and diazinon; and Eskenazi et al. (2004) testing for DAPs and for seven pesticide specific metabolites (MDA - derived from malathion; PNP - derived from methyl parathion, parathion, and other nonpesticide chemicals; TCPy - from chlorpyrifos, chlorpyrifos methyl, and triclopyr; DEAMPY - from pirimiphos methyl; IMPY - from diazinon; CMHC - from coumaphos and coumaphos methyl; CIT - from isazophos and isazophos methyl).   Finally, several method validation studies tested for specific OP pesticides, but did not evaluate the association between these exposures and specific adverse health outcomes (Whyatt et al., 2007; Whyatt et al., 2009; Bradman et al., 2003).

The most commonly measured biomarkers were urinary DAPs. These non-specific markers are easily quantified using gas chromatography/mass spectrometry and related methods. Though objective, use of urinary DAPs as biomarkers for OP pesticide exposures has limitations, including substantial temporal variability, often varying substantially over short time scales (i.e., day-to-day). Quantification of DAPs in a single urine sample may not represent an individual's usual exposure to OP pesticides over the time period of interest (e.g., pregnancy) in their utility as biomarkers of OP pesticide exposure. Urinary DAP metabolite levels may also reflect exposure to ambient metabolites in addition to exposure to OP parent compounds.  For example, Chen et al (2012) and Zhang et al (2008) have shown that direct exposure to DAPs can occur from consumption of DAPs in food.  Lu et al. (2005) observed that OP pesticides in fortified fruit juice samples degraded into DAPs.  In this literature, errors in DAP as a biomarker of OP pesticide exposure are likely to be non-differential with respect to outcomes.  Epidemiologists often distinguish between two mechanisms or types of misclassification  -  those that are non-differential (or random) and those that are differential (non-random).

Total DAPs is a non-specific measure of OP exposure and is the sum of six separate molecules - three dimethyl alkylphosphate (DMAP) molecules of DMP, DMTP, DMDTP, and three diethyl alkylphosphate (DEAP) molecules of DEP, DETP, and DEDTP.  Each metabolite is a breakdown product from multiple OPs (Table 2.2.4-2).  Specifically, DMP, DMTP, and DMDTP are associated with 18, 13, and 5 OPs, whereas DEP, DETP, and DEDTP are associated with 10, 10, and 4 OPs, respectively.  Thus, using DAPs as an exposure measure, it is not possible to separate the exposure and associated effects for single, specific OPs.  For studies evaluating TCPy (e.g., Fortenberry et al., 2014; Eskenazi et al., 2007; Whyatt et al., 2009), this molecule is a metabolite of chlorpyrifos, chlorpyrifos-methyl, and the herbicide triclopyr and thus is not entirely specific to chlorpyrifos.  TCPy can be found on directly food.  Studies focusing solely on chlorpyrifos could assess exposure to only this OP (i.e., measure chlorpyrifos directly) (e.g., Lovasi et al., 2011; Whyatt et al., 2004; Rauh et al., 2011).

Table 2.2.4-2	CDC Table of Organophosphate Pesticides and Their Dialkyl Phosphate Metabolites (2008)
Pesticide 
DMP 
DMTP 
DMDTP 
DEP 
DETP 
DEDTP 
Azinphos methyl
|
|
|

Chlorethoxyphos

|
|

Chlorpyrifos

|
|

Chlorpyrifos methyl
|
|

Coumaphos

|
|

Dichlorvos (DDVP)
|

Diazinon

|
|

Dicrotophos
|

Dimethoate
|
|
|

Disulfoton

|
|
|
Ethion

|
|
|
Fenitrothion
|
|

Fenthion
|
|

Isazaphos-methyl
|
|

Malathion
|
|
|

Methidathion
|
|
|

Methyl parathion
|
|

Naled
|

Oxydemeton-methyl
|
|

Parathion

|
|

Phorate

|
|
|
Phosmet
|
|
|

Pirimiphos-methyl
|
|

Sulfotepp

|
|

Temephos
|
|

Terbufos

|
|
|
Tetrachlorviphos
|

Trichlorfon
|

DMP = dimethylphosphate; DEP = diethylphosphate; DMTP = dimethylthiophosphate; DMDTP = dimethyldithiophosphate; DETP = diethylthiophosphate; DEDTP = diethyldithiophosphate.

2.2.4.2.2	Neurological and Other Outcome Measures
With some exceptions, the outcomes assessed in this literature fall into three broad categories: 1) neurobehavioral and/or neurodevelopmental status; 2) birth outcomes; and 3) neurodevelopmental diseases and/or disorders (see Table 2.2-b).

There is a broad body of literature available on the use and interpretation of instruments designed to quantify neurological status that is beyond the scope of this summary. Many instruments were used in this literature to assess infant neurodevelopment and neurobehavior (see Table 2.2-b). Importantly, performance on some of these tests can be influenced by the administrator of the assessment. Also of concern is whether these assessments are sensitive enough to distinguish potentially subtle effects of OP pesticide exposure. Some degree of error in the assessment of neurological outcomes is likely in all of the studies, though the errors were unlikely to be related to exposure status in the well conducted studies. 

The assessment of birth outcomes in this literature was primarily conducted by reviewing of medical records or birth certificates; these assessments are likely to have minimal errors, and errors that do arise are almost certainly non-differential with respect to exposure status. In some studies, however, birth outcomes were reported by the mother and in these instances, differential misclassification is possible.

The studies that evaluated specific diagnoses, autism spectrum disorders or developmental delay, for example, relied on existing medical records, with some effort to validate diagnoses in a subset of the investigations.

Studies that relied on standardized, validated instruments to assess neurodevelopment, medical records of birth outcomes, or validated diagnosis of disease states were weighted highly in the judgment of study quality. Those that used standardized instruments that had not been validated in the relevant population or screening assessments were given a moderate weight, while studies relying on selected sections of neurodevelopment assessment tools, maternal report of outcome status, or aggregated (ecological) outcome measures were given the lowest weight.
2.2.4.2.3.	Statistical Analysis 
Statistical analyses that were appropriate to the study question and study design, supported by adequate sample size, maximized the use of available data, and were well characterized in the report were weighted most highly. Acceptable statistical methods, moderate study power, and analytic choices that resulted in the loss of information or that were not clearly reported were given moderate weight. Reports with only minimal attention paid to the conduct and reporting of the statistical analyses were given the lowest weight. 

2.2.4.2.4	Confounding
Risk factors for early life neurodevelopment perturbations that are associated with OP pesticide exposure, but not caused by pesticide exposure, are potential confounders in this literature. Socioeconomic determinants of child development (including, for example, maternal education and access to prenatal and early life health care), quantity and quality of parent-child interactions, and exposure to other environmental toxicants (e.g., lead, PCBs, other pesticides) are difficult to measure and were either not accounted for, or inadequately accounted for in many studies. That said, some studies were relatively homogeneous with respect to these factors and thus limited confounding by design, while others attempted to quantify these factors and adjust for them. Other important potential confounders, such as the child's sex, are easy to identify and adjust for analytically. 

2.2.4.2.5	Risk of Bias
The internal validity of the studies reviewed was judged by noting the design strategies and analytic methods used in each study to constrain or eliminate selection bias, information bias, and confounding. Selection bias occurs when the sampling of the population by the investigator yields a study population that is not representative of the exposure and outcome distributions in the population sampled. Put simply, selection bias occurs if selection of the study sample yields a different estimate of the measure of association than that which would have been obtained, had the entire target population been evaluated. Although there are numerous sources of selection bias, there are several mechanisms that may have induced selection bias in the studies reviewed: less than 100% participation rates of eligible individuals due to non-responsiveness or refusal (self-selection bias); loss to follow-up (i.e. failure to retain all study participants initially enrolled in the study); and, in a case-control study, control selection bias arising because the exposure distribution in the control sample does not represent the exposure distribution of the study base (i.e., the population that gave rise to the cases or more formally, the person-time experience of that population).

Information bias (also referred to as observation bias) arises when study participants are incorrectly categorized with respect to their exposure or outcome status, or when errors arise in the measurement of exposure or outcome, in the case of continuously distributed measures. Epidemiologists often distinguish between two mechanisms or types of misclassification  -  those that are non-differential (or random) and those that are differential (non-random). Non-differential misclassification of exposure (or non-differential exposure measurement error) occurs when the probability or magnitude of error in the classification or measurement of exposure is independent of the outcome status of the study participants. Similarly, non-differential misclassification of outcome (or outcome measurement error) occurs when the probability or magnitude of error in the assignment of outcome status or level is independent of exposure status. In contrast, differential exposure misclassification (or measurement error) occurs when the error in the exposure assignment is not independent of the outcome status. 

The mechanisms that cause non-differential misclassification in this literature include errors in the medical records, laboratory errors, sampling of biospecimens for biomarker assays, and errors in recall. The mechanisms that induce differential misclassification include recall bias, and interviewer/observer bias. Note that mismeasurement of confounders can result in residual confounding of the association of interest, even when adjustment for that confounder has been conducted in the analysis. 

Studies in which major sources of potential biases were not likely to be present, or in which potential sources of bias were present but effectively addressed and analyzed to maximize the study validity, and those in which sources of bias were unlikely to influence the magnitude and direction of the risk estimate were given a high weight. Studies where sources of bias were present and acknowledged by the authors but not addressed in the study and yet may influence the magnitude, but not direction of the association estimate received a moderate weight. A low weight was given to studies in which major biases were present and yet were not acknowledged or addressed in the study, such that they cannot be excluded as an alternative explanation for the study finding.

2.2.5	Review of Quality Results
Each of the studies in the 2015 review was judged to be of high, moderate, or low quality in each of five domains of study design and methodology effecting study quality as discussed above in Section 2.1. The results of the quality assessment are presented separately below for each group.  The quality categories represent to the total evaluation.  In Section 2.2.6, further evaluation of the study design and exposure assessment of the medium and high quality studies. This further evaluation led to additional studies being removed from the final analysis, with these excluded studies not being considered further in the remaining sections of this document.

2.2.5.1	"High" Quality Group
Fifteen articles assigned a high quality rating are shown in Tables 2.2.5.1-1 and 2.2.5.1-2. In general, these were prospective birth cohort studies with moderate to high sample size; exposure assessment was based on an objective biomarker measure, the outcome measurement(s) utilized standardized tests and trained data collectors, appropriate statistical analyses were performed, considering relevant covariates, and risks of bias were minimized to the extent possible. All of the studies from CCCEH, Mt. Sinai, CHAMACOS, and HOME cohorts are assigned a high ranking.  Fortenberry et al. (2014, Table 2.2.5-1) reported on findings in the ELEMENT study population of 187 mother-child pairs, assessed third trimester maternal urinary TCPy as a biomarker of prenatal chlorpyrifos exposure, and employed a trained and experienced research team to administer a battery of validated ADHD/psychometric assessments (Conner's' Parental Rating Scales-Revised, Conner's' Continuous Performance Test, and the Behavior Assessment System for Children-Parental Rating Scales) that had been translated into Spanish.
Table 2.2.5.1-1.	High Quality Studies from 2015 and 2016 reviews: Summary of Study Design Elements Impacting Study Quality Assignment
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Barr et al. (2010)
New Brunswick, New Jersey
Mother who underwent elective Cesarean delivery at term and their newborns, Saint Peter's University Hospital
Cross sectional birth cohort, modest sample numbers
N=150
Mother-infant pairs 
Objective measure, prenatal - parent CPF in maternal and cord serum at birth - exposures do not necessarily precede outcome
Birth outcomes from medical records
Generally appropriate; missing a few, e.g., maternal age, SES, nutrition, pre-natal care, race
Appropriate multivariate analysis
Convenience sample  -  deemed low probability of selection bias; low potential for exposure or outcome misclassification 
Bouchard et al. (2010)
U.S. National Population
National Health and Nutrition Examination Survey (NHANES)
Age 8-15 years
Cross-sectional, large sample size
N=1139

Objective measure, postnatal  -  single non-OP specific child urinary DAPs
ADHD from DISC-IV diagnosis or medication in children 8-15 years old, standard protocol, trained interviewers
Appropriate: also included blood lead
Appropriate, accounted for NHANES multistage probability sampling
Low probability of selection bias; some potential for non-differential misclassification of exposure
Donauer et al. (2016).

Cincinnati, Ohio, USA
Health Outcomes and Measure of the Environment (HOME)
Prospective Birth Cohort Study  -  large sample size.
N=327
Mother-infant pairs
Metabolite concentrations (total DAP, DM, and DE) of prenatal OP pesticide exposure quantified in two maternal spot urine samples provided at 16 and 26 weeks of gestation
For children, Bayley-II, MDI, and Psychomotor Development Index examinations at 1-3 years of age, and Clinical Evaluation of Language Fundamentals-
Preschool, Second Edition, and the Wechsler Preschool and Primary Scale of Intelligence, at ages 4 and 5 were used.
Appropriate. Included household income, maternal intelligence, child's sex, maternal intelligence quotient, and maternal education; house inventory score, maternal age at birth, alcohol consumption during pregnancy, parity, and maternal lead levels
Appropriate. Bivariate and multivariable regression
Lack of frequent urine spot sampling, short half-life of DAP metabolites, and the sporadic exposure to DAP may have led to non-differential exposure misclassification; inability to determine exposure toxicity.

Engel et al. (2015)
Salinas Valley, California; Brown, Butler, Clermont, Hamilton, or Warren, Ohio; South Bronx, New York City; Manhattan, New York City
Pooled analysis of the CHAMACOS (1999-2000), Columbia (2000-2001), Mt. Sinai (1998-2002), and HOME (2003-2006) birth cohorts 
Birth Cohort Study  -  large sample size
N= 936 women (377 from CHAMACOS study, 265 from HOME study, 60 from Columbia study, and 234 from Mount Sinai study)
Mother-infant pairs
Metabolite concentrations (total DAP, DMP, and DEP) of prenatal OP pesticide exposure quantified in two maternal spot urine samples provided at 13 and 26 weeks and 16 and 26 weeks of gestation (CHAMACOS and HOME studies) and in one maternal spot urine sample at 31 and 32 weeks (Mt. Sinai and Columbia studies)
Blood samples from the mother and child (via the umbilical chord) were also collected and genotyped for the PON1 genotype
For each child, Bayley Scales of Infant Development II
(BSID-11) to generate a Mental Development Index (MDI) and a Psychomotor Development Index (PDI) to assess neurodevelopment in early childhood.
Appropriate.  Included maternal education, marital status, and age at delivery, alcohol use during pregnancy, race/ethnicity, smoking and drug use during pregnancy, child sex, HOME environment score quartile, breastfeeding at least 3 months after birth, metabolite x center interaction terms, race/ethnicity, smoking and drug use during pregnancy.
Linear models with interaction analysis.
Lack of frequent urine spot sampling, short half-life of DAP metabolites, and the sporadic exposure to DAP may have led to non-differential exposure misclassification; inability to determine exposure toxicity; and hetergoenity of the study populations of the 4 cohorts and exposures may have effected the results
Furlong et al. (2014)
New York City, 
U.S.
Mount Sinai Children's Environmental Health Study
Prospective Cohort
N=136
Age 7-9 years
Objective biomarker, prenatal OP pesticide exposure (DAP) quantified in single maternal spot urine sample provided during the 3rd trimester, simple imputation < lowest level of detection (LLOD)
Reciprocal social impairment at age 7-9 years assessed using the Social Responsiveness Scale, completed by mothers. Designed to assess reciprocal social behaviors in evaluating ASDs, used here as general indicator of impaired social responsiveness.
Appropriate. Included maternal and child demographic and SES indicators (age, education) and ETS (not other environmental toxicants).
Appropriate multivariate analysis
Selection bias possible due to considerable loss to follow-up; Residual confounding likely small in magnitude; some potential for non-differential misclassification of exposure possible explanation for null findings). 
Fortenberry et al. (2014)
Mexico City, Mexico 
Early Life Exposures in Mexico to Environmental Toxicants (ELEMENT) study (3 cohorts 1994-1997 (cohort 1), 1997-2000 (cohort 2), and 2001-2005 (cohort 3))
Prospective Birth Cohort Study  -  modest sample size
N=187 
Mother-infant pairs
Objective measure - TCPy concentration was measured in third trimester maternal urine samples. In a subset of women randomly selected urinary TCPy in samples collected during all three trimesters of pregnancy was measured. 

ADHD - LP - Conner's' Parent Rating Scales-Revised (CPRS-R), Conner's' Continuous Performance Test (CPT), and Behavior Assessment System for Children-Parental Rating Scales (BASC-PRS)  -  These are screening tools, not diagnostic tools. Standardization and quality control checks were conducted by reviews of videotaped evaluations.
Appropriate. Included continuous maternal IQ, education, socioeconomic status and blood lead one month after delivery, breast feeding (yes/no), child's sex, continuous age at testing, birth length and head circumference at birth.
Appropriate. Multivariable linear regression
History of maternal and paternal exposure to pesticides not included, use of a single urinary measure to estimate exposure, potential for differential exposure misclassification possible as mothers were likely aware of the neurobehavioral status of their children
Fortenberry et al. (2014a)
Mexico City, Mexico
Early Life Exposures in Mexico to Environmental Toxicants (ELEMENT) study
Prospective Birth Cohort Study  -  large sample size
N=591 
Mother-infant pairs
Objective measure - blood samples from mother - child pairs were analyzed for PON1.  OP exposure was not explicitly assessed.   
ADHD - LP - Conner's' Parent Rating Scales-Revised (CPRS-R), Conner's' Continuous Performance Test (CPT), and Behavior Assessment System for Children-Parental Rating Scales (BASC-PRS) - These are screening tools, not diagnostic tools.
Appropriate: Included continuous maternal IQ, education, socioeconomic status and blood lead one month after delivery, breast feeding (yes/no), child's sex, continuous age at testing, birth length and head circumference at birth.
Appropriate. Multivariable linear regression
Potential for differential exposure misclassification possible as mothers were likely aware of the neurobehavioral status of their children.
Harley et al. (2016)
Salinas Valley, California; Brown, Butler, Clermont, Hamilton, or Warren, Ohio; South Bronx, New York City; Manhattan, New York City
Pooled analysis of four CHAMACOS, Columbia, Mt. Sinai, and HOME birth cohorts
Birth Cohort Study  -  large sample size
N= 1,235 women (484 from CHAMACOS study, 328 from HOME study, 82 from Columbia study, and 341 from Mount Sinai study)
Mother-infant pairs
Metabolite concentrations (total DAP, DMP, and DEP) of prenatal OP pesticide exposure quantified in two maternal spot urine samples provided at 15.9 and 26.4 weeks of gestation (CHAMACOS and HOME studies) and in one maternal spot urine sample at 31.8 and 33.3 weeks (Mt. Sinai and Columbia studies)
Blood samples from the mother and child (via the umbilical chord) were also collected and genotyped for the PON1 genotype
For each child, head circumference, birth weight, and birth length were obtained from medical records 
Appropriate. Included sex, race/ethnicity, cohort, country of origin, marital status, maternal education, smoking during pregnancy, parity, maternal age at delivery, and gestational age.  
Appropriate. Multivariable linear regression and mixed-effect models to test the effect modification for sex, maternal education, race/ethnicity, and child PON1 genotype.
Lack of frequent urine spot sampling, short half-life of DAP metabolites, and the sporadic exposure to DAP may have led to non-differential exposure misclassification; inability to determine exposure toxicity; and hetergoenity of the study populations of the 4 cohorts and exposures may have effected the results 
Quirós-Alcalá et al. (2011)
Salinas Valley, California, USA 
Center for the Health Assessment of Mothers and Children of Salinas (CHAMACOS)
Prospective Birth Cohort Study Outcomes assessed  at: 
N=149 (6 months) 
N=149 (1 year)
N=97 (3.5 years) 
N= 274 (5 years)
Objective biomarker, prenatal OP pesticide exposure (DAP) quantified in two maternal spot urine samples at 14 and 27 weeks gestation (on average), and in single child urine sample provided at time of each outcome assessment
Autonomic nervous system dysregulation at ages 6 months, 1 year, 3.5 years, and 5 years; standard protocol, trained, bilingual research staff, appropriate to age of child. May not be sensitive to resolve subtle effects of OP pesticide exposure.
Appropriate. Included maternal and child demographic and SES indicators (age, education).
Appropriate multivariate analysis
Selection bias possible due to considerable loss to follow-up; Residual confounding likely small in magnitude; some potential for non-differential misclassification of exposure (possible explanation for null findings). 
Raanan et al. (2015)
Salinas
Valley, California
CHAMACOS (Mothers and their children. Children were followed from through age 7)
Prospective Birth Cohort

N = 364 children
Diethyl (DE) and dimethyl(DM) phosphate metabolites and other dialkyl phosphate (DAP) metabolites of OP pesticides measured in mother's urine during pregnancy and from children during childhood
Mother's report of child's respiratory symptoms and child's exercise-induced coughing at 5 and 7 years of age
Models were adjusted for child's sex, child's age, maternal smoking during pregnancy, secondhand tobacco smoke exposure during early childhood, season of birth, mean daily fine particulate concentration during first 3 months of life (PM2.5), breastfeeding, mold and cockroaches in home, and home distance from highway in first year.
Generalized estimating equations (GEE) with repeated measures of respiratory symptoms
Risk of misclassification of exposure, due to nonspecificity of DAP metabolite used as biomarker which may reflect exposure to parent compound (pesticide) or preformed DAPs in environment, and also due to potential for daily fluctuations in DAP levels due to high variability in OP pesticide exposures.

                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                         Confounder /
Covariate Contro
                             Statistical Analysis
                             Risk of (other) Bias
Rauch et al. (2012)
Cincinnati, Ohio,  USA 
Health Outcomes and Measure of the Environment (HOME)
Age: newborns
Prospective Birth Cohort Study  -  large sample size.
N=306

Objective biomarker of prenatal OP pesticide exposure (DAP) quantified in two maternal spot urine samples provided at 16 and 26 weeks gestation 
Abstracted birth weight from medical records, calculated gestational age from mother's self-reported date of last menstrual period. When gestational age not available, results of an ultrasound was used or a Ballard examination performed just after delivery.
Appropriate. Included income, education, maternal depressive symptoms, maternal IQ, insurance status, area of residence, prenatal care, PON1 genotype, and gestational exposure to alcohol, lead, and tobacco. 
Appropriate. multivariable regression
Unable to rule out the possibility that differences in DAP concentrations partially reflect individual variation in metabolism, Recall error
Rauh et al. (2015)
New York City, New York
Columbia Center for Children's
Environmental Health (CCCEH) (Children of Black and Dominican mothers, aged 9  -  13.9 years old) 
Prospective Birth Cohort Study
N = 271

Chlorpyrifos (CPF) concentrations in sample of umbilical cord blood collected at delivery
Chlorpyrifos (CPF) concentrations in sample of maternal blood collected within 2 days postpartum
Neurodevelopmental assessment via  Archimedes spirals, a drawing exercise, to test for tremor
Models were adjusted for sex, exact age at testing, ethnicity, and medications
Chi-square was to assess crude associations between high CPF exposure and presence of mild to moderate tremor. 
Binary logistic regression was used to estimate associations between high exposure level and presence of tremor
Risk of misclassification of exposure, particularly due to lack of CPF exposure biomarkers to assess postnatal and childhood exposure. Misclassification may also occur in the outcome assessment due to the use of a clinical scale, based on observer's rating to assess tremor
Stein et al. (2016)
Salinas Valley, California area
CHAMACOS Study (Pregnant women and 7 year-old children)
Prospective longitudinal cohort study
N = 329
Mother-infant pairs
Biomarker of prenatal OP pesticide exposure (DAPs) quantified in two maternal spot urine sample collected at 13 and 26 weeks of gestation.

Mothers were also interviewed following birth at 6 months, 1, 2, 3.5, 5, and 7 years to collect data on early adversities and assess the home environment using the HOME test
Children's IQ was analyzed using the Wechsler Intelligence Scale for Children
Appropriate. All models were adjusted for maternal IQ score at 6m assessed by Peabody Picture Vocabulary Test.  Full-Scale and Verbal Comprehension IQ models were also adjusted for language of neurological assessment.  Models in which adversity measures did not include H.O.M.E. subscores were adjusted for long term H.O.M.E. score average; models in which adversity measures did not include parental educational attainment were adjusted for maternal education.

Appropriate. Multivariate linear regression model and three-way interaction model
Selection bias probable; adversity measures calculated via a priori hypotheses, instead of incorporating using more advanced statistical methods; lack of frequent maternal urine spot sampling and due to the short half-life of DAP metabolites may have led to exposure misclassification; low concentrations of DAP detected in the urine; lack of study details regarding the urine collection methods
Wolff et al. (2007)
New York City, U.S.
Mount Sinai Children's Environmental Health Study
Prospective Cohort   -  moderate sample size
N=404
Mother-infant pairs
Objective biomarker, prenatal OP pesticide exposure (DAP), malathion (MDA) quantified in single maternal spot urine sample provided during the 3rd trimester
Birth outcomes from computerized perinatal database
Appropriate: different covariates for each outcome, e.g., weight: maternal age, race/ethnicity, maternal BMI*pregnancy weight gain, infant sex, gestational age, creatinine
Appropriate PROC GLM, with varying covariates
Selection bias unlikely; some potential for non-differential misclassification of exposure
Yolton et al. (2013)
Cincinnati, Ohio, USA
Health Outcomes and Measure of the Environment (HOME)
Prospective Birth Cohort Study  -  large sample size.
N=350
Mother-infant pairs
Metabolite concentrations (total DAP, DM, and DE) of prenatal OP pesticide exposure quantified in two maternal spot urine samples provided at 16 and 26 weeks of gestation
For children, a neurobehavioral assessment was conducted via the NICU Network Neurobehavorial Scale (NNNS) at ~ 5 weeks of age
Appropriate. Included sex, birthweight, infant weight change from birth to exam, parity, maternal age at delivery, marital status, education, employment during pregnancy, household income, body mass index (BMI) at 16 weeks gestation, weight gain per week during pregnancy, moderate to severe depression (score > 13) measured on the Beck Depression Inventory-II during pregnancy and at 5 weeks post-delivery, reported marijuana and alcohol use during pregnancy, whole blood lead and folate during pregnancy, serum cotinine, and reported fruit and vegetable consumption during pregnancy
Appropriate. Bivariate and multivariable regression
Low concentrations of DAP detected in the urine; lack of frequent urine spot sampling during pregnancy.

Table 2.2.5.1-2	High Quality Studies from 2012/2014 literature search of Children's Environmental Health Epidemiology Studies 

                                Author, Year, 
                                  Sample Size
                                   Outcome 
                                   Assessed
                             Exposure Measurement
                       Potential confounders considered
                                Primary Result
                          Conclusions/Uncertainties 
                           Strengths and Weaknesses 
 
 
 
 
 
 
 
Whyatt et al. (2004) Columbia U. (N=314)
Birth length, Birth weight, head circumference
4 cord plasma chlorpyrifos exposure groups and 4 chlorpyrifos and diazinon exposure groups. Chlorpyrifos only categories, Group 1: levels below LOD (32% of participants); Group 2: lowest 1/3 of detectable levels (20 %); Group 3: middle 1/3 (24%), Group 4: highest 1/3 (25%.)  
Chlorpyrifos and diazinon together:  Group 1: 26%, Group 2: 22 %, Group 3: 26%, Group 4: 26%.
Gestational age, maternal pre-pregnancy weight, maternal net pregnancy weight gain, gender of newborn, parity, race/ethnicity, ETS in home, season, cesarean section
For each log unit increase in cord plasma chlorpyrifos levels, birth weight decreased by 42.6 g (95% CI:  - 81.8 to  - 3.8) and birth length decreased by 0.24 cm (95% CI:  - 0.47 to  - 0.01). Birth weight averaged 186.3 g less (95% CI:  - 375.2 to  - 45.5) among newborns with the highest compared with lowest 26% of exposure levels (p = 0.01).
Associations between birth weight and length and cord plasma chlorpyrifos were statistically significant (p <= 0.007) among newborns born before the January 2001 policy change. Among newborns born after January 2001, exposure levels were substantially lower, and no associations with fetal growth outcomes were observed (p > 0.8). 
Strengths: prospective nature of the study; direct measurement of chlorpyrifos in cord blood and personal air samples, rather than non-specific markers of organophosphate pesticide exposure; consideration of other pesticides and environmental contaminants as covariates in the multivariate models. 
Limitations: single exposure sampling period; the authors did not present nor discuss regression diagnostics to assess the degree to which their models met or violated the assumptions implicit in linear models. 
Berkowitz et al.(2004)           
Mt. Sinai (N=404)
Birth length, birth weight, head circumference, gestational age
LOD: 11 ug/L (57% <LOD TCPy)
Race/ethnicity, infant sex, and gestational age. The authors also controlled for birth weight or birth length in their assessment of head circumference and pesticide exposure.
Mean levels of birth weight, length, head circumference, and gestational age did not differ between those with urinary pesticide metabolite levels below and above the level of detection.

Similarly, no statistically significant associations were observed between reported pesticide exposure and mean indices of fetal growth and gestational age. 
PON1 activity also predictor of smaller head circumference; creatinine corrected
Very well conducted study with numerous strengths and very few weaknesses. 

The questionnaire-based pesticide exposure questions are subject to imperfect recall. Errors would, on average, attenuate associations between these exposure metrics and fetal development. 

Recall-based exposure assessments were fortified by objective measures of pesticides/pesticide metabolites.

A metabolite specific for chlorpyrifos (TCPy) was assessed. 

Statistical analysis was appropriate. 

Observed mean reductions in the outcome parameters appear to be small in magnitude and may be of little clinical significance. 

Assessment and control for confounding were appropriate. However, confounding by unmeasured (and mismeasured) risk factors for abnormal growth that are related to pesticide exposure would bias the results in this study. Such factors may be related to socioeconomic status of the study participants, a cofactor which is difficult to define, no less measure in an epidemiologic study.

Limited external validity (generalizability) due to the particular study population recruited and the numerous exclusion criteria applied.
Eskenazi et al. (2004)          CHAMACOS (N=488)
Birth length, birth weight, head circumference, Gestation Length, Ponderal index
Total DAPs: median 136 nmol/L (range: 10 - 6,854); DEP: median 22 nmol/L (range: 2 - 680 nmol/L); TCPy: median 3.3 nmol/L (range: 0.2 - 56.1nmol/L) (76% >LOD)
Gestational age, gestational age squared, maternal age, pregnancy weight gain, week of initiating prenatal care, parity, infant sex, mother's country of birth, body mass index, family income, poverty level, smoking, alcohol, illicit drug use, environmental tobacco smoke, caffeine, history of low birth weight, and history of pre-term delivery.
Decreases in gestational duration associated with two measures of in utero pesticide exposure: levels of metabolites of dimethyl phosphate pesticide compounds and whole blood ChE.
Maternal urine collection averaged weeks 14, 26, not creatinine-corrected
Strengths in the study design include the longitudinal design, the use of multiple exposure biomarkers, including quantification of non-specific (DAPs), chlorpyrifos-specific (TCPy) metabolites, and other environmental co-exposures. A reasonable set of exclusion criteria were applied.The selection of the CHAMACOS population, which consists mostly of children from low-income families, served to increase the relative statistical efficiency of the study, as this population is at high risk of neurodevelopmental deficits, compared to the general population.The statistical analysis used to assess the associations between the markers of exposure and neurodevelopment were appropriate.Errors in the assignment of exposure in this prospective study will likely have resulted in attenuation of observed associations.
Harley et al. (2011)              CHAMACOS (N=329)
Birth length, birth weight, head circumference, gestational age 
The geometric mean for the DAP concentrations during pregnancy (for the average of the two sampling periods) was 146 nmol/L (95% CI: 133, 160); of this, a larger proportion was DMP metabolites (GM = 109 nmol/L; 95% CI =98, 120) than DEPs (GM= 23 nmol/L; 95% CI = 21, 25). Allele frequencies: PON1 192 Q allele= 50%; PON1 -108 T allele= 46%.Mean arylesterase activity: For infants: 33.6 U/mL (SD = 16) For mothers: 136.6 U/mL (SD = 44). Mean paraoxonase activity: For infants: 256.6 U/L (SD = 165); For mothers: 989.0 U/L (SD = 616).
Maternal intelligence (Peabody Picture Vocabulary Test (PPVT)), measures of how stimulating the environment is, and known or suspected neurotoxins were measured prenatally.  To measure the quality and extent of stimulation available to a child in the home environment, the Infant-Toddler HOME (Home Observation for Measurement of the Environment) inventory was completed at the 6-month, 1, 2, 3.5, 5, and 7 year visits; known or suspected neurotoxicants, polybrominated diphenyl ethers (PBDEs), polychlorinated biphenyls (PCBs), p,p'-dichlorodiphenyltrichlorethane (DDT), p,p'- dichlorodiphenyltrichlorethylene (DDE), and lead. 
The authors observed evidence of an association between prenatal exposure to OP pesticides as measured by urinary DAP metabolites in women during pregnancy, is associated with decreased cognitive functioning in children at age 7.
Infants whose PON1 genotype and enzyme activity levels suggested that they might be more susceptible tothe effects of OP pesticide exposure had decreased fetal growth and length of gestation. PON1 may be a contributing factor to preterm or low birth weight birth.
This study has many strengths, the longitudinal design, the measurement of urinary DAP at multiple time points and following children to age seven when tests of cognitive function are reportedly more reliable. The authors were able to adjust for or consider many factors related to cognitive function, such as prenatal exposure to other environmental agents, socioeconomic indicators, maternal intelligence and education, and child stimulation. The cohort had a relatively homogenous socioeconomic profile, reducing the potential for uncontrolled confounding. 
Engel et al. (2007)  Mt. Sinai (N=311)
Brazelton Neonatal Behavioral Assessment Scale (BNBAS), primitive reflexes (neurological integrity) measured before hospital discharge.
DEP: 24.7 nmol/L; Total DAP: 82 nmol/L
Maternal age, race, marital status, education, cesarean delivery, delivery anesthesia, infant age at examination, infant gender, infant jaundice, smoking (yes/no), alcohol consumption, caffeine consumption, illicit drug use during pregnancy, and the examiner. 
No adverse associations were found for DAPs and any measured behavior. Relative to the first quartile, quartiles 2 - 4 of total DEPs, DMPs, and DAPs were associated with an increased proportion of abnormal reflexes, although the associations did not increase monotonically and varied in their strength and precision. 
Used non-specific biomarker DEP/DAP
This was a well conducted prospective study conducted in a young, predominantly minority population. The study design, analytic approach, and statistical analyses were appropriate. Pesticide metabolites evaluated are not specific for chlorpyrifos. The BNBAS was administered before hospital discharge only on a subset of children in the cohort (n =311/404).  Factors related to weekend delivery (e.g., fewer inductions) would be underrepresented among the tested subjects, and may induce bias, reduce the degree of precision with which associations were estimated, and limit the generalizability of the study findings.The statistical analysis was largely appropriate. Imputing of missing data may affect precision of association estimates and result in attenuated effect estimates as a result of exposure measurement error. Assessment and control for confounding were appropriate. However, confounding by unmeasured (and mis-measured) risk factors for abnormal growth that are related to pesticide exposure would bias the results in this study. Such factors may be related to socioeconomic status of the study participants, a cofactor which is difficult to define, no less measure in an epidemiologic study.
Young et al. (2005) CHAMACOS (N=381)
Neurodevelopment, Brazelton Neonatal Behavioral AssessmentScale (BNBAS), abnormal reflexes
DAP (average during pregnancy): median 222nmol/L (range: 7 - 21,867 nmol/L); DEP (average during pregnancy): median 21 nmol/L (range: 2 - 680 nmol/L)
Maternal age, BMI, any smoking/alcohol/drug use during pregnancy, gestational age at which prenatal care was initiated, total number of prenatal care visits, mean pregnancy blood pressure, parity, method of delivery, general anesthesia used during delivery, breastfeeding initiated after delivery, poverty level, infant sex, age in days at BNBAS, minutes since last feed at BNBAS, and BNBAS examiner.
Among the >3 day old infants, increasing average prenatal urinary metabolite levels were associated with both an increase in number of abnormal reflexes (total DAP: adjusted beta = 0.53, 95% CI = 0.23, 0.82; dimethyls: adjusted beta = 0.41, 95% CI = 0.12, 0.69; diethyls: adjusted beta = 0.37, 95% CI = 0.09, 0.64), and the proportion of infants with more than three abnormal reflexes (total DAP: adjusted OR = 4.9, 95% CI = 1.5, 16.1; dimethyls: adjusted OR = 3.2, 95% CI = 1.1, 9.8; diethyls: adjusted OR = 3.4, 95% CI = 1.2, 9.9).
Associations seen pre-natal OP, not post-natal OP exposure, Maternal urine collection averaged weeks 14, 26
Strengths: Longitudinal design, measurement and consideration of many confounders, the prenatal exposure measures were, with some exceptions, the average of two measurements, and thus may better reflect chronic exposure during the pregnancy
Weaknesses: Potential for residual confounding, potential for exposure misclassification, potential selection bias (not all children had the outcome measures), study population not generalizable to the whole US.
Rauh et al. (2006)    Columbia U. (N=254)
Neurodevelopment: The Bayley Scales of Infant Development II (BSID-II), Mental Development Index (MDI) and Psychomotor Development Index (PDI) at 12, 24, and 36 months of age.  :: Behavior:  Child Behavior Checklist (CBCL) at  12, 24, and 36 months.  
:: Quality of the child-care environment: The Home Observation for Measurement of the Environment (HOME) 
Exposure levels were categorized as low (<  6.17 pg/g) or high (>6.17 pg/g) 
Data were collected regarding lead exposure, demographics, education and occupational history, income, active and passive smoking, alcohol and drug use during pregnancy, and residential pesticide use.  Final models included prenatal environmental tobacco smoke (ETS) exposure, gender, ethnicity, gestational age at birth, quality of home care-taking environment, maternal education, and maternal IQ.
At the 36 month milestone, the likelihood of highly exposed children developing mental delays were 2.4 times greater (95% CI: 1.12-5.08, p = 0.02) and motor delays were 4.9 greater (95% CI: 1.78-13.72; p = 0.002) than those with lower prenatal exposure. The GLM analysis for PDI scores showed a significant effect of chlorpyrifos exposure over time with an estimated deficit of approximately 7 points by age 36 months (p = 0.01).  
The authors summarize three main findings:  1) by age 3, the more highly exposed children demonstrated mental and motor delays; 2) the observed developmental trajectories for PDI and MDI scores confirmed that the adverse impact on cognitive and motor development increased over time; and 3) by age 3, highly exposed children were more likely to demonstrate clinically significant attention problems.
:: Only 53% of the children reached the three year milestone with study data collected.  It is unclear what percentage of these children did not survive, were lost to follow-up, or too sick to participate.  
:: Reliance on a single exposure level (prenatal/cord blood.) 
:: No control for exposure over the subsequent 3 years :: Creation of a dichotomous exposure variable brings limitations due to the amount of within-group variation.   
:: Limitations of the sensitivity and predictive validity of the developmental tests, especially among children less than 3 years of age.   
:: No discussion of whether this 7-point deficit is clinically relevant.  
:: Due to the pervasive, non-specific nature of neurological effects, it is difficult to attribute causality.  
Lovasi et al. 2011      Columbia U. (N=266)
Bayley scores (MDI/PDI) 12 months, 24 months, 36 months
N/A
Neighborhood characteristics: The percentage of housing units without complete plumbing, the percentage of vacant housing units, the percentage of residents below the federal poverty line, the percentage of residents older than 25 years of age who completed high school, the percentage of households receiving public assistance, the percentage of housing units with one or more residents per room, racial composition, the percentage of residents born outside the United States, the percentage of Spanish-speaking residents, and the percentage of residents who were linguistically isolated
Neighborhood characteristics did not confound the observed association between chlorpyrifos levels and cognitive development.
Hierarchal regression analysis of potential confounding by SES
Direct measurement of chlorpyrifos.The statistical analyses were generally appropriate. Missing data on covariates were estimated using multiple imputation, and the variance estimates presented appropriately reflect the degree of uncertainty caused by missing covariate data. Robust standard errors were used.The setting of the investigation in a sample drawn from low-income African American and Dominican communities is both a strength (increases the power, restriction of confounders) and a limitation of the study (reduced generalizability). 
Engel et al. (2011)         
Mt. Sinai (N=276)
Bayley scores (MDI/PDI) at 12 months, 24 months.
DEP: 24.7 nmol/L; Total DAP: 82 nmol/L (same as Engel 2007)
Maternal age, race/ethnicity, marital status, education, breast-feeding, child sex, alcohol, smoking, or drug use during pregnancy, maternal IQ, a score based on assessment of the home environment (HOME), season of urine collection, language spoken in the home, age at testing, examiner, and urinary creatinine level.
An observed association between prenatal total dialkylphosphate metabolite level and a decrement in mental development at 12 months among blacks and Hispanics. 
Used non-specific biomarker DEP/DAP; some evidence of effect modification by PON1 genotype
Limitations include use of non-specific markers of chlorpyrifos pesticide exposure (DAPs), use of only a single (third-trimester) urine sample, and the large proportion of loss to follow-up. Statistical analysis was appropriate.  Imputing of missing data may affect precision of association estimates and result in attenuated effect estimates as a result of exposure measurement error, although these are offset by the further categorization of the exposure levels (at the median). However, binning of exposure levels reduces precision, relative to a continuously distributed measure of exposure. Assessment and control for confounding were appropriate. However, confounding by unmeasured (and mismeasured) risk factors for abnormal growth that are related to pesticide exposure would bias the results in this study. Such factors may be related to socioeconomic status of the study participants, a cofactor which is difficult to define, no less measure in an epidemiologic study.
Eskenazi et al. (2007)   CHAMACOS (N=372)
Neurodevelopment, Bayley Index (MDI, PDI), Maternal behavior checklist at 6, 12, and 24 months
DEP:  geom. Mean in mother 18.1 nmol/L (95% CI = 16.7 - 19.7); DEP geometric mean in child at 24 months 10.5 nmol/L (95% CI =8.8 - 12.6); TCPy median 3.54 ug/l
Psychometrician, location of assessment, exact age at assessment, sex, breast-feeding duration (months), HOME score, and household income, parity, maternal PPVT, maternal age, education, depressive symptoms, active/passive smoking exposure during pregnancy, regular alcohol use during pregnancy, marital status, father's presence in home, housing density, maternal work status, >= 15 hours out-of-home childcare/week, birth weight, gestational age, abnormal reflexes, PCBs, lead, DDT, β-hexachlorocyclohexane, and hexa-chlorobenzene 
DAP metabolite levels during pregnancy, particularly from dimethyl phosphate pesticides, may be negatively associated at 24 months with mental development (MDI) on the Bayley Scales and an increase in risk of maternally reported PDD. 
No strong associations identified with DE or TCPy, Maternal urine collection averaged weeks 14, 26
Strengths: Longitudinal design, measurement and consideration of many confounders (including other environmental chemicals); the prenatal exposure measures were, with some exceptions, the average of two measurements, and thus may better reflect chronic exposure during the pregnancy

Weaknesses: Potential for residual confounding, potential for exposure misclassification, potential selection bias (not all children had the outcome measures), study population not generalizable to the whole US.
Eskenazi et al. (2010)         CHAMACOS (N=371)
Neurodevelopment, Bayley Index (MDI, PDI), Maternal behavior checklist at 6, 12, and 24 months, PON1 gene and enzyme levels
The geometric mean for the DAP concentrations during pregnancy (for the average of the two sampling periods) was 146 nmol/L (95% CI: 133, 160); of this, a larger proportion was DMP metabolites (GM = 109 nmol/L; 95% CI =98, 120) than DEPs (GM= 23 nmol/L; 95% CI = 21, 25). Allele frequencies: PON1 192 Q allele= 50%; PON1 -108 T allele= 46%.Mean arylesterase activity: For infants: 33.6 U/mL (SD = 16) For mothers: 136.6 U/mL (SD = 44). Mean paraoxonase activity: For infants: 256.6 U/L (SD = 165); For mothers: 989.0 U/L (SD = 616).
Psychometrician, location of assessment, exact age at assessment, sex, breast-feeding duration (months), HOME score, and household income, parity, maternal PPVT, maternal age, education, depressive symptoms, active/passive smoking exposure during pregnancy, regular alcohol use during pregnancy, marital status, father's presence in home, housing density, maternal work status, >= 15 hours out-of-home childcare/week, birth weight, gestational age, abnormal reflexes, PCBs, lead, DDT, β-hexachlorocyclohexane, and hexa-chlorobenzene 
Decrease MDI (24 months) PON1108TT  - 5.7 ( - 9.0 to  - 2.5) 9=0.01; Decrease PDI (24 months) PON1108TT  - 2.8 ( - 5.7 to 0.2) p=0.07; increased odds PDD 2.0 (0.8 to 5.1) p=0.14; no association PON1192; no association PON1 activity measured newborn, 2 years, maternal and MDI, PDI, PDD. Evidence of decreasing MDI score by number of PON1108 variant alleles: PON1108CC  - 3.2 ( - 9.8 to 3.5), CT  - 3.7 ( - 8.0 to 0.6), TT  - 5.5 ( - 11.1 to 0.1), p-interaction 0.98.
In this study population, evidence PON1 may influence MDI score, but not PDI or PDD risk at two-years. Non-significant evidence of decreasing MDI score by increasing DAP levels across strata of the number of PON1108 variant alleles, interaction non-significant. Similar trend with prenatal DEP levels and MDI, PDI by PON108 alleles, but less pronounced. Overall, limited, non-definitive evidence of effect modification by PON1 status in the relation between mental and psychomotor effects and prenatal DAPs. 
Strengths: Longitudinal design, measurement and consideration of many confounders (including other environmental chemicals); the prenatal exposure measures were, with some exceptions, the average of two measurements, and thus may better reflect chronic exposure during the pregnancy

Weaknesses: Potential for residual confounding, potential for exposure misclassification, potential selection bias (not all children had the outcome measures), study population not generalizable to the whole US. Study may be under-powered to evaluate effect modification by PON1 status.
Marks et al. (2010)          CHAMACOS (N=348)
CBCL; K-CPT; ADHD confidence index; Hillside behavioral rating scale; composite ADHD indicator
DAP (geometric mean) pregnancy 109.0 nmol/L; DEP 17.7 nmol/L
Psychometrician, exact age at assessment, sex, maternal education, depressive symptoms, PPVT (continuous), >= 15 hr out-of-home child care/week, breast feeding duration (months), maternal age, parity, marital status, active/passive smoking exposure and regular alcohol use during pregnancy, presence of father in home, maternal work status, and household income
Prenatal DAPs were non-significantly associated with maternal report of attention problems and ADHD at age 3.5 years, but were significantly related at age 5 years [CBCL attention problems: β = 0.7 points; 95% confidence interval (CI), 0.2-1.2; ADHD: β = 1.3; 95% CI, 0.4-2.1]. 
Marked effect modification by gender: 11-fold increase ADHD composite indicator in boys, less than 2-fold in girls, however unstable estimates; weak evidence of association DAPs at 3.5, 5 years and attention 
Strengths: Longitudinal design, measurement and consideration of many confounders (including other environmental chemicals); the prenatal exposure measures were, with some exceptions, the average of two measurements, and thus may better reflect chronic exposure during the pregnancy

Weaknesses: Potential for residual confounding, potential for exposure misclassification, potential selection bias (not all children had the outcome measures), study population not generalizable to the whole US.
Rauh et al. (2011)    Columbia U. (N=265)
:: Wechsler Scales of Intelligence for Children (WISC-IV) :: Child Behavior Checklist (CBCL).
:: Chlorpyrifos levels in umbilical cord blood samples, N=256 newborns
:: If no cord blood (12% of subjects), levels were imputed from mothers' values. 
:: Values for samples with non-detectable chlorpyrifos levels (N=115, 43%) were imputed by using assay-specific limit of detection (LOD) values to impute an approximate level.  
Data were collected regarding lead exposure, demographics, education and occupational history, income, active and passive smoking, alcohol and drug use during pregnancy, and residential pesticide use.  Final models included prenatal environmental tobacco smoke (ETS) exposure, gender, ethnicity, gestational age at birth, quality of home care-taking environment, maternal education, and maternal IQ.
Full-Scale IQ:  (B) of -0.003, CI = 0.006, 0.001, p= 0.064 Working Memory Index: (B) of -0.006, CI = 0.009, 0.002, p<0.001.  The investigators articulated these results as showing that a 1 pg/g increase in chlorpyrifos exposure was associated with a 0.006 point decrease in the log-transformed Working Memory score and a 0.003 point decrease in the log-transformed Full-Scale IQ score.  The investigators concluded that for each standard deviation increase in exposure (4.61pg/g) there is a 1.4% reduction in Full-Scale IQ and a 2.8% reduction in Working Memory.
For each standard deviation increase in exposure (4.61pg/g) there is a 1.4% reduction in Full-Scale IQ and a 2.8% reduction in Working Memory.
Strengths
:: Direct assessment of chlorpyrifos levels using maternal serum and cord blood. 
:: Analysis using a continuous CPF level, which, in contrast to dichotomous CPF levels, provides a more meaningful look at potential threshold effects and dose-response trends.
:: The investigators rigorously evaluated their methods for imputing values for undetectable CPF levels which in the end, were validated.
:: The authors describe an elegant and methodologically sound statistical analysis, addressing many of the potential shortcomings of their exposure data and covariates.

Weaknesses: The use of a single snapshot of prenatal chlorpyrifos exposure may not be an accurate surrogate for full prenatal exposure levels.  
:: There is no control for exposure over the subsequent 7 years which may be critical, especially as the process of neurocognitive development is fluid and rapid during these early childhood years.   :: Possibility of that an increased awareness of the risks of pesticide exposures could disproportionately affect postnatal exposure behavior.:: Complicating this analysis is the pervasive, non-specific nature of neurological effects and the difficulty in attributing causal pathways. :: when closely reviewed, the 95% CI for Full Scale IQ for both techniques contain 0 (LASSO: -0.006, 0.001, p=0.064; fully-adjusted: -0.006, 0.001, p=0.048:: The authors do not address the clinical relevance of the 1.4% and 2.8% reductions and how this may impact a child or his/her psychological or educational plans.    
Engel et al. (2011)                             Mt. Sinai (N=169)
Wechsler Preschool and Primary Scale of Intelligence, 3rd edition (WPPSI-III) at ages < 7 years; Wechsler-IV Intelligence Scale (verbal comprehension; perceptual reasoning, working memory, processing speed, full scale intelligence) at age 7-9 years
DEP: 24.7 nmol/L; Total DAP: 82 nmol/L (same as Engel 2007)
Maternal age, race/ethnicity, marital status, education, breast-feeding, child sex, alcohol, smoking, or drug use during pregnancy, maternal IQ, a score based on assessment of the home environment (HOME), season of urine collection, language spoken in the home, age at testing, examiner and urinary creatinine level.
At age 6-9 years, non-statistically significant reductions in full scale IQ, perceptual reasoning, verbal comprehension, working memory and processing speed with increasing DAP, more profound with DEP than DMP 
Used non-specific biomarker DEP/DAP; some evidence of effect modification by PON1 genotype
Limitations include use of non-specific markers of chlorpyrifos pesticide exposure (DAPs), use of only a single (third-trimester) urine sample, and the large proportion of loss to follow-up.

The statistical analysis was largely appropriate. Imputing of missing data may affect precision of association estimates and result in attenuated effect estimates as a result of exposure measurement error, although these are offset by the further categorization of the exposure levels (at the median).

Assessment and control for confounding were appropriate. However, confounding by unmeasured (and mis-measured) risk factors for abnormal growth that are related to pesticide exposure would bias the results in this study. Such factors may be related to socioeconomic status of the study participants, a cofactor which is difficult to define, no less measure in an epidemiologic study.
Bouchard et al. (2011)    CHAMACOS (N=329)
Wechsler-IV Intelligence Scale (verbal comprehension; perceptual reasoning, working memory, processing speed, full scale intelligence) measured at age 7 years
Total DAPs (quintiles): Q1 (39 nmol/L); Q2 75 nmol/L; Q3 126 nmol/L; Q4 221 nmol/L; Q5 508 nmol/L. Geometric mean DAP 131 nmol/L
Maternal intelligence, measures of how stimulating the environment is, and known or suspected neurotoxins were measured prenatally. Maternal intelligence was assessed via the Peabody Picture Vocabulary Test (PPVT). To measure the quality and extent of stimulation available to a child in the home environment, the Infant-Toddler HOME (Home Observation for Measurement of the Environment) inventory was completed at the 6-month, 1, 2, 3.5, 5, and 7 year visits; known or suspected neurotoxicants, polybrominated diphenyl ethers (PBDEs), polychlorinated biphenyls (PCBs), p,p'-dichlorodiphenyltrichlorethane (DDT), p,p'- dichlorodiphenyltrichlorethylene (DDE), and lead. 
The authors observed evidence of an association between prenatal exposures to OP pesticides as measured by urinary DAP metabolites in women during pregnancy, and decreased cognitive functioning in children at age 7.
Prenatal measures taken later half of pregnancy more significantly associated intelligence than early; little evidence post-natal OP exposure associated with intelligence; 7 point reduction in full scale intelligence DAP Q5/Q1 (SS)
Strengths: the longitudinal design, the measurement of urinary DAP at multiple time points and following children to age seven when tests of cognitive function are reportedly more reliable. The authors were able to adjust for or consider many factors related to cognitive function, such as prenatal exposure to other environmental agents, socioeconomic indicators, maternal intelligence and education, and child stimulation. The cohort had a relatively homogenous socioeconomic profile, reducing the potential for uncontrolled confounding. 
Whyatt et al. (2007)      Columbia U. (N=102)
None
Geometric mean, 6.9 +- 17.0 ng/m[3]; range < 0.4 - 171 ng/m[3]. Personal air monitor: median 2.8 ng/m[3], mean 6.2 +- 11.1 ng/m[3], range < 0.4 - 83.4 ng/m[3]
N/A
There was little within-home variability and no significant difference in air concentrations within homes over time (p >= 0.2); between-home variability accounted for 88% of the variance in the indoor air levels of propoxur, 92% in chlorpyrifos, 94% in diazinon, and 62% in piperonyl butoxide (p < 0.001). Indoor and maternal personal air insecticide levels were highly correlated (r = 0.7 - 0.9, p < 0.001). 
Indoor and maternal personal air insecticide levels were highly correlated (r = 0.7 - 0.9, p < 0.001). 
Strengths: study design and exposure assessment techniques, Limitations: only those cohort participants enrolled after 2011 were included in the analysis (most likely due to the lack of serial data from the earlier years.) 
Whyatt et al. (2009)      Columbia U. (N=102)
None
The limit of detection (LOD) of chlorpyrifos in blood samples was 0.5 - 1 pg/g plasma. The LOD of TCPy in urine samples was 0.26 ng/mL urine. The LOD for TCPy in meconium was 0.2 ng based on a sample weighing 0.5 g.  Exposure marker levels below the LOD were given a value of half the level of detection, and were then log10 transformed.
N/A
Meconium TCPy concentrations were positively correlated with chlorpyrifos in maternal and cord blood (r = 0.25 - 0.33, p < 0.05) and with TCPy in maternal urine (r = 0.31, p < 0.01).
TCPy in maternal urine samples was not reliable, but the maternal and cord blood chlorpyrifos as well as the TCPy levels in meconium were reliable measures of exposure
Comprehensive exposure assessment including actual blood chlorpyrifos levels, the repeated sampling, and the environmental sampling.

Weaknesses: only included participants recruited in the post-cancellation period, use of nonparametric, rank-based statistics is appropriate but the large number of observations below the level of detection receiving equal rank, may be problematic; no dietary assessment 
Rauh et al.(2012), (n=40)
Morphological change in the pediatric brain in regions of the brain known to be associated with learning, cognition and social behavior
Tertile 3 ( >=4.39 pg/g), compared to Tertiles 0, 1, 2 (<4.39 pg/g, including those not
exposed to CPF)
Age, sex
Authors report differences in brain structure (regional cerebral size and thickness) by CPF exposure groups, and the differences (high>low CPF) in regional brain size is likely due to enlargement of underlying white matter. Statistical interaction by gender reported.
Authors concluded that the evidence from the study illustrated changes in brain morphology in association with higher CPF exposure, and that changes observed were in areas of the brain that subserve those learning, cognition and social behavioral, supported by previous observational and experimental literature.
Study supports general hypothesis of CPF influence on brain morphology, but lacks specific hypotheses regarding particular areas of the cerebrum affected; limited and somewhat unbalanced depiction of the available rodent experimental data; statistical methods appropriate, correction for multiple statistical comparisons a strength; MRI image readers blinded to exposure status enhances study validity; lack of information on other validation practices; small sample size, pilot study, low statistical power; external validity limited; one time measure of pre-natal exposure

 

2.2.5.2	"Moderate" Quality Group

Fifteen articles were assigned a moderate quality rating, as shown in Table 2.2.5.2-1. In general, these were cross-sectional or prospective cohort studies with small to high sample size; exposure assessment was based on a non-specific biomarker measure or current self-report, the outcome measurement(s) utilized standardized tests or screening tools, appropriate statistical analyses were performed, considering some but maybe not all relevant covariates, and risks of bias were minimized to some extent. For example, Guodong et al. (2012) cross-sectionally evaluated the relationship between DAPs concentrations in urine sampled from 301 young children as an objective, non-specific marker of prenatal OP pesticide exposure and Developmental Quotients based on the Gesell Developmental Schedules adapted for a Chinese population.

Table 2.2.5.2-1.	Moderate Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Andersen et al.(2015)
Denmark
School age children living in Denmark
Prospective Birth Cohort, small sample size
N=177 
(112 maternal occupational pesticide exposure during pregnancy; 65 without) 
Age: 6-11 years 
Prenatal occupational pesticide exposure ascertained by maternal interview at enrollment
Objective standardized clinical exam; neurophysiological status (heart rate variability); Previously validated, neuropsychological testing with demonstrated sensitivity to environmental pollutants; administered and scored by single neuropsychologist
Appropriate. Self-reported by mother. Included age, maternal demographics and risk factors, SES indicator (broad categories), maternal smoking and alcohol use. Possibility of recall errors (residual confounding). Self-reported (by mother)
Appropriate multivariate analysis. Evaluated numerous hypotheses without adjustment for multiple comparisons. Also constructed structural equation model of interaction between child sex and prenatal pesticide exposure on general intellectual ability (parameterized as a latent variable)
Selection bias possible due to loss to follow-up; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure
Guodong et al. (2012)
Shanghai, China
2-year old children visiting community hospitals
Cross-sectional, large sample size
N=301
Age 23-25 months
Objective biomarker, OP exposure (DAP) quantified in single child spot urine sample, simple imputation <LLOD; OP exposure also assessed via questionnaire administered to mothers after delivery
Developmental Quotients based on Gesell Developmental Schedules, adapted for Chinese population
Appropriate. Assessed via mother report via questionnaire, included child sex, maternal demographics and risk factors, SES indicators (maternal education, household income), maternal smoking and alcohol use; Possibility of recall errors
Appropriate multivariate analysis. Assumed linear relationship between log-transformed DAP level and DQ Scores
Selection bias unlikely; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure
Handal et al. (2007)
Cayambe-Tabacundo region, Ecuador
EcoSalud Project: infants and young children 3 to 61 months of age in lower-altitude communities A&B dominated largely by cut-flower production and in more traditional rural, higher altitude community C.
Cross-sectional, modest sample size from census
N=283 
Age 3-61 months
- 154 high exposure
- 129 low exposure
Objective proxy measure: Community of residence
Ages and Stages Questionnaire (ASQ), adapted into local vernacular, 2 trained testers  -  considered a screening tool
Appropriate: child health status (anemia, stunting) and other characteristics of the home environment (stimulation by 2 methods)
Appropriate: Multiple linear regressions to evaluate associations between community of residence and delayed development, Pairwise t-tests and chi-square to assess mean difference in ASQ score; Effect size Cohen's d
Selection bias unlikely; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure
Handal et al. (2008) 
Cayambe-Tabacundo region, Ecuador
EcoSalud Project: Children 3 to 23 months of age in lower-altitude communities A&B dominated largely by cut-flower production and in more traditional rural, higher altitude community C.
Cross-sectional, modest sample size
N:121
Age 3-23 months
Proxy: Distance home to farm, parental employment, pesticide use on domestic crops & within home, child play activities
Ages and Stages Questionnaire (ASQ) (ages 24-61 months), a screening tool; Visual Motor Integration (VMI) Test (ages 48-61 months); two trained testers.
Appropriate: As above
Appropriate: As above
Selection bias unlikely; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure

Table 2.2.5-2.	Moderate Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment (continued)
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Handal et al.(2007B)
Cayambe-Tabacundo region, Ecuador
EcoSalud Project: Children 24 to 61 months of age in lower-altitude communities designated A & B were dominated largely by cut-flower production than in more traditional rural, higher altitude community C.
Cross-sectional, modest sample size
N:
ASQ - 142 
Age 24-61 months
VMI - 57 
Age 48-61 months
Proxy: Maternal employment during pregnancy, child plays outdoors, Pesticide use on domestic crops , inside home
Ages and Stages Questionnaire (ASQ) (3-23 months)  -  screening tool; Reach-and-grasp, UC Berkeley Preferential Looking Test Cards; trained tester
Appropriate: As above
Appropriate: As above;
Prehension- Logistic regression models, generalized estimation equations (GEE) to account for between-test dependence
Selection bias unlikely; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure
Llop et al. (2013)
Spain
INMA (Environment and Childhood) Project
Prospective Birth Cohort, large sample size
N=1980
Age 14 months
Maternal self-report of prenatal and postnatal indoor pesticide use (pesticide spray or use of a plug-in device assessed via questionnaire)
Mental and psychomotor development at 14 months assessed using validated instrument (Bayley Scales of Infant Development) 
Appropriate. Self-reported by mother. Included maternal demographics and risk factors (Age, BMI) SES indicators (maternal education, occupation), maternal smoking and alcohol use. Also childcare behaviors (breast feeding, number of siblings, day care) Possibility of recall errors (residual confounding)
Appropriate multivariate analysis 
Selection bias unlikely; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure
Oulhote and Bouchard  (2013)
Canadian National Population
Canadian Health Measures Survey (CHMS; cycle 1, 2007 - 2009);
Cross-sectional Study, large national sample size
n = 779 children
Age 6-11 years
Child urinary DAP, DMP, DEP collected within 2 weeks of survey questionnaire completion by the parents.
Behavioral problems in children based on the parent version of the Strengths and Difficulties Questionnaire (SDQ) (Goodman 1997) - SDQ is a validated screening questionnaire and accepted by parents. 
Appropriate: Included sex, age, race/ethnicity, family income, parental education, blood lead levels, maternal smoking during pregnancy, birth weight, maternal age at child's birth, child's BMI, and fasting status (fasting duration at urine collection > 10 hour/<= 10 hour)
Appropriate: Logistic Regression
Selection bias unlikely; Study population not mixed, single urine sample 
Petit et al. (2010)
Brittany, France
Newborn children in the PELAGIE Study.
Prospective Birth Cohort, large sample size
N= 3,159 
Age: newborns
Ecological, proxy indicator of exposure (proportion of municipality devoted to agricultural activity
Objectively measured birth outcomes assessed using hospital records
Appropriate. Maternal report via questionnaire. Included maternal demographics (age, BMI) and pregnancy risk factors (gestational age, hypertension, diabetes, season of pregnancy) SES indicators (district of residence, maternal education, occupation), maternal smoking and alcohol use. Also childcare behaviors (breast feeding, number of siblings, day care); Possibility of recall errors
Appropriate multivariate analysis. Multiple comparisons were conducted for this evaluation, and the authors did not adjust for multiple testing
Selection bias unlikely; Residual confounding likely, small in magnitude; potential for substantial non-differential misclassification of exposure; Misclassification of outcomes unlikely

Table 2.2.5-2.	Moderate Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment (continued)
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Ruckart et al. (2004)
Mississippi, USA (29 counties), Ohio (one multi-family facility)
Children who were 6 years or younger in MS and OH when homes sprayed with MP
Retrospective cohort, moderate sample size in 2 states, participating/ nonparticipating similar in sex and age
MS: N=365 (147 exposed, 218 unexposed)
OH: N=287 (104 exposed, 183 unexposed)
Age: 1.9-12.5 years old at testing
Specific OP measures: Household methyl parathion (wipe) by approved labs, OR highest urinary PNP level in household, analyzed by CDC
Variety of neuro measures: PENTB, Parenting Stress Index (PSI), Personality Inventory for Children (PIC), Vineland Adaptive Behavior Scales (VABS)
Appropriate: Income, race, site term, ethnicity, mother's use of chemicals at work, mother health/pregnancy conditions, report that child had lead or mercury poisoning (MS only); Raw scores were child age-adjusted
Appropriate: Linear regression (continuous scores), logistic regression (dichotomous scores)
Selection bias possible due to loss of follow-up; Residual confounding likely; substantial potential for differential exposure misclassification (time between spraying and testing, frequency and duration unknown)
Shelton et al. (2014)
California, USA
CHARGE Study (3 year-old children)
Case-control, large sample size
N = 970
Age 3 years
N=486 (ASD)
N=168 (DD)
N=316 (Controls)
Proxy indicator of prenatal OP pesticide exposure (residential proximity to agricultural pesticide applications defined using ecological pesticide use data)
Clinical outcomes  -  validated within the study
Appropriate adjustment for demographics (place of birth, race), SES indicators (paternal education) and vitamin intake during pregnancy
Appropriate multivariate analysis. 
Selection bias probable; Residual confounding likely; substantial potential for differential misclassification of exposure. Outcome misclassification unlikely.
Suarez-Lopez et al. (2012)
Pedro Moncayo County, Pichincha, Ecuador
Secondary Exposure to Pesticides Among Infants, Children and Adolescents (ESPINA) Study, plus new volunteers
Cross-sectional, moderate sample size, from prior census
N=277
Exposed: (n=158)
Unexposed: (n=119)
Age 4-9 years
Proxy: Cohabitation with flower worker >1yr, Adult questionnaire
Child AChE level using commercial kit
Appropriate: Sex, age, height-for-age, hemoglobin concentration, income, pesticide use within household lot, pesticide use by neighbors, examination date, residence, distance to nearest flower plantation
Appropriate: Multiple linear regression (continuous) and logistic regression (polychotomous variables), adjusted models
Selection bias possible due to loss but deemed low; Residual confounding likely small in magnitude; potential for differential exposure misclassification
Suarez-Lopez et al. (2013a)
Pedro Moncayo County, Pichincha, Ecuador
Secondary Exposure to Pesticides Among Infants, Children and Adolescents (ESPINA) Study, plus new volunteers
Cross-sectional, moderate sample size, from prior census
N=307
Age 4-9 years
Non-specific measures: 
Child AChE level using commercial kit; 
Proxy: Cohabitation with flower worker >1yr, Adult questionnaire
Heart rate (HR), Systolic blood pressure (SBP), Diastolic blood pressure (DBP) by standard methods
Appropriate: Sex, age, height-for-age, hemoglobin concentration, income, pesticide use within household lot, pesticide use by neighbors, examination date, residence, distance to nearest flower plantation
Appropriate: Multiple linear regression (continuous), adjusted.
Selection bias possible due to loss but deemed low; Residual confounding likely small in magnitude; potential for differential exposure misclassification
Suarez-Lopez et al. (2013)
Pedro Moncayo County, Pichincha, Ecuador
Secondary Exposure to Pesticides Among Infants, Children and Adolescents (ESPINA) Study, plus new volunteers
Cross-sectional, moderate sample size, from prior census
N=271
Age 4-9 years
Non-specific measures: 
Child AChE level using commercial kit
NEPSY-II test (trained examiners): general assessment battery
Model defined a priori: hemoglobin, age, sex, race, height-for-age z score, household income, maternal education, and flower worker cohabitation status
Logistic models (dichotomous and polychotomous) and linear regression models, adjusted; Effect modification according to sex among significant associations; Some imputed values
Selection bias possible due to loss but deemed low; Residual confounding likely small in magnitude; potential for differential exposure misclassification
Wang et al. (2012)
Shanghai, China
Pregnant women and newborn children
Cross-sectional Birth Cohort Study
N=187
Age: newborns
Objective biomarker of prenatal OP pesticide exposure (DAP) in single maternal spot urine sample provided at onset of labor. Simple imputation of observations < LLOD. OP pesticide exposure also assessed via questionnaire administered after delivery
Gestational age and pre-term delivery appropriately defined and assessed using medical records
Appropriate. Included maternal anthropomorphic, demographic, and SES indicators (income, occupation), predictors of high-risk pregnancy (pregnancy weight gain, gestational age)
Appropriate multivariate analysis
Selection bias unlikely; Residual confounding likely small in magnitude; potential for differential misclassification of exposure
Zhang et al. (2014)
Shenyang, China
Newborn children in a birth cohort study
Prospective Cohort, n=249 
mother-infant pairs
Age: newborns
Biomarker of prenatal OP pesticide exposure (DAP) quantified in single maternal spot urine sample provided at delivery. Many observations < LLOD
Neonatal neurodevelopment assessed using validated instrument (Neonatal Behavioral Neurological Assessment) by trained examiners
Appropriate. Included maternal demographic and SES indicators (age, education), predictors of high-risk pregnancy (BMI, gestational age) and environmental toxicant exposure (cord blood lead); Results stratified by sex
Appropriate multivariate analysis
Selection bias unlikely; Residual confounding likely small in magnitude; considerable potential for non-differential misclassification of exposure (unlikely to account for non-null findings)

2.2.5.3	"Low" Quality Group
Seventeen articles were assigned a low quality rating, as shown in Appendix 3. In general, these were small or pilot studies; exposure assessment was based on a proxy measure(s), outcome measurement(s) utilized screening tools or self-report, limited statistical analyses were performed, relevant covariates were not included or discussed, or risks of bias were possible. For example, Acosta-Maldonado et al. (2009) conducted a cross-sectional pilot study among 54 women, only nine of whom were considered to have had prenatal exposure to pesticides (defined based on either an exposure history profile or AChE level in blood sampled at the time of admission into the hospital for delivery. The outcome assessed in this study was a standardized but partially subjective assessment of placental maturity (the Placental Maturity Index, PMI). Covariates adjustment of estimated associations between prenatal exposure and PMI in this study was minimal and limited to placental characteristics.   Cartier et al (2016) is from the PELAGIE prospective cohort study and used DAP biomarkers as the exposure biomarkers.  However, despite these strengths, the urine samples were collected within the home instead of a laboratory or clinic and were returned via the local mail, which could have potentially led to several sampling errors due to inadequate collection and storage methods. 
The remaining sections of this document do not discuss further studies identified in the `low' category.  Due to limitations in these studies, they do not provide reliable information evaluating associations between OP exposure and neurodevelopmental outcomes.
2.2.6	Assessment of Epidemiological Studies for Relevance to Analysis
Using the criteria summarized in Section 2.2.4, a total of 47 literature articles were identified in the 2015 literature review and were judged as high, moderate, or low quality.  Overall, 15 articles, 15 articles, and 17 articles were judged to be of high, moderate, or low quality, respectively.  For the 30 high and moderate quality studies, additional evaluation was conducted as described in this section.

While all of the moderate quality studies had strengths including sample design and outcome assessment, six of these moderate quality studies did not have sufficient exposure assessment methods to determine whether exposure to OPs actually occurred.  These studies were conducted on study populations in Spain (Llop et al., 2013), Ecuador (Handal et al., 2007; 2007b; 2008), Denmark (Andersen et al., 2015), and France (Petit et al., 2010).  In these studies, participants were considered exposed or unexposed to pesticides based on non-specific exposure measures, such as self-reported occupational exposure, home pesticide spraying, and proportion of municipality devoted to agricultural activity.  For all of these proxy exposure assessments, the pesticides used may have included not only OPs, but also pyrethroids, fungicides, and growth regulators.  Given the uncertainty about whether OP exposure actually occurred in these studies and whether observed outcomes are associated with OP exposure or with other pesticides, these studies are excluded from further analysis.

The focus of this epidemiological literature review is on the neurodevelopmental outcomes from exposures to low levels of OPs (i.e. below exposures which would result in 10% or more AChE inhibition).  Three studies conducted in Ecuador focused on child AChE inhibition and the potential association of AChE inhibition with other measures including parental occupation (Suarez-Lopez et al., 2012), as well as clinical autonomic nervous system (ANS) outcomes such as blood pressure and heart rate (Suarez-Lopez et al., 2013a), and neurodevelopmental outcomes (Suarez-Lopez et al., 2013).  The range of AChE activity levels are lower in the first tertile (range of 1.44 to 2.93 U/mL) compared to the third tertile (range from 3.33 to 4.69 U/mL).  Therefore, due to the outcomes assessed and the potentially toxic cholinergic effects that were associated with these outcomes, these studies are not considered to be relevant to this review and are not discussed further here.

In a retrospective cohort study of children exposed to methyl parathion (MP) before age 6 years in Mississippi and Ohio (Ruckart et al. 2004), as assessed by household urinary PNP or wipe samples, exposed children performed worse than unexposed children on a few of numerous neurobehavioral development tests conducted.  Specifically, participants classified as having had MP exposure had more difficulty with short term memory and attention tasks, and parents reported more behavioral and motor skill problems, relative to unexposed children.  However, upon closer inspection of the results across the MS and OH study sites, these neurobehavioral outcomes are not seen consistently.  These inconsistencies may be due to differences in how the exposure occurred across the sites, including the fact that MS participants and OH participants were tested 2.5 and 4.5 years after MP spraying in the home.  When comparing exposed and unexposed children using general intelligence testing, integrated visual and motor skills testing, and multistep processing, they did not see any differences.  The exposure scenario associated with these observations is a critical element in assessing the utility and reliability of this study.  Samples were collected in locations from OH and MS where illegal spraying of methyl parathion is known to have occurred during the 1994-1996 time period.  Based on the "Revised Organophosphorous Pesticide Cumulative Risk Assessment" (USEPA, 2006), methyl parathion is known to be among the more potent OPs.  It is unknown whether study participants were exposed to MP levels that would have induced cholinergic effects.  Therefore, given the uncertainty around this illegal use combined with the high potency for cholinergic toxicity, the agency is not emphasizing this study further in the analysis.

In addition, two high quality studies from the CHAMACOS birth cohort (Quirós-Alcalá et al. 2011; Ranaan et al., 2015) assessed the association between DAPs and autonomic nervous system (ANS) outcomes and respiratory symptoms, respectively.  The ANS outcomes assessed included heart rate and respiratory sinus arrhythmia.  Overall, while there was some evidence of ANS dysregulation for infants at 6 months, these results were not consistently observed for the other assessed child (1 year, 3.5 years, and 5 years) and maternal OP exposures.  Ranaan et al (2015) reported that total DAPs and DEAP metabolites in urine from the second half of pregnancy were significantly associated with increased odds of respiratory symptoms in children.  Ranaan et al. l (2015) also reported exercise-induced coughing at 5 and 7 years of age was significantly associated with total DAPs, DEAPs, and DMAPs in children's urine collected between the ages of 6 months and 5 years (AUC). The authors concluded that early-life exposure to OP pesticides was associated with respiratory symptoms consistent with possible asthma in childhood. There is not a body of literature to compare these results against, making it difficult to put them into context.  Furthermore, these studies did not focus on neurodevelopmental outcomes, which is the focus of this analysis.  Consequently, Quirós-Alcalá et al. (2011) and Raanan et al. (2015) are not being evaluated further.

2.2.7	Birth Outcome Epidemiologic Studies

Four identified studies, three high quality and one moderate quality, focused solely on OP exposure and adverse birth outcomes related to fetal growth (Barr et al., 2010; Rauch et al., 2012; Wang et al., 2012; Wolff et al., 2007).  Harley et al (2016) is a meta-analysis of data from the HOME, Mt. Sinai, CCCEH, and CHAMACOS cohorts which allowed for a better assessment of race and genetic variability given the larger sample size in the pooled analysis.  

The birth outcomes assessed included birth weight, birth length, head circumference at birth, and gestational age.  The exposure assessment for these studies was conducted using objective measures or biomarkers such as maternal urinary DAPs (Rauch et al., 2012; Wang et al., 2012; Wolff et al., 2007), maternal urinary malathion (MDA) (Wolff et al., 2007), or maternal and cord blood levels for specific OPs (Barr et al., 2010).  The results from these studies are generally inconsistent, with some studies documenting statistically significant associations between OP exposure and birth outcomes whereas others do not.  For example, maternal and cord blood serum levels of TCPy were inversely associated in univariate analyses conducted among participants in a prospective cohort study conducted in New Jersey (Barr et al. 2010), though the associations did not persist after adjusting for gravidity, maternal pre-pregnancy BMI, infant sex, and gestational age.  Chlorpyrifos levels were near the lower limit of detection in this study, though detectable in 98.6% of maternal serum and 62.8% of cord serum samples.  Overall, birth length was not associated with third-trimester DAP in the Mount Sinai Cohort (Wolff et al. 2007).  However, among those with slow-activity paraoxonase-1 (PON1) or PON192, urinary total DMP (but not total DAP or DEP) was statistically significantly associated with shorter birth length (p= 0.032).  Birth length was also not associated with maternal urinary DAP sampled at delivery in the cross-sectional investigation conducted in Shanghai, China (Wang et al. 2012).  Similarly, inconsistent results were observed across these studies for birth weight, gestational age, and head circumference at birth.

For the pooled analysis in Harley et al (2016), no evidence of association was observed between OPs exposure via the ∑DAP, ∑DEP, and ∑DMP urinary metabolites and birth outcomes for birth weight, birth length, and head circumference, as all of the 95% CIs encompassed the null value of 0 and all of the p-values were > 0.05 (p-values ranged from 0.46 to 0.98), respectively.  For the interaction analysis, no evidence of a statistically significant interaction between the four cohorts and exposure to urinary metabolites (∑DAP, ∑DEP, and ∑DMP) relative to the birth outcomes (birth weight, birth length, and head circumference) were observed since all of the p for interaction values were above > 0.10, respectively.  A statistically significant interaction was observed between the PON1-108 genotype (specifically the PON1-108CC gene) and the urinary metabolite ∑DEP relative to pooled birth weight (p = 0.08), and between the PON1-108 genotype (specifically the PON1-108CC genes) and the urinary metabolites ∑DAP and ∑DMP relative to pooled birth length (p for interaction = 0.07; p = 0.02) among children.  With the PON1192 genotype, a statistically significant interaction was observed between the urinary metabolite ∑DAP relative to pooled head circumference (p for interaction = 0.05).  For mothers, a statistically significant interaction was observed between the PON1-108 genotype and the urinary metabolites ∑DAP and ∑DMP relative to pooled head circumference (p for interaction = 0.09 for both), respectively.  No additional statistically significant interactions were observed for any other genotypes among children and mothers, and no statistically significant interactions were observed for race, sex or maternal education.

Overall, in this 2015/2016 literature review, inconsistent evidence of OP exposure and association with adverse birth outcomes/fetal growth was observed (Barr et al., 2010; Rauch et al., 2012; Wang et al., 2012; Wolff et al., 2007).  This lack of consistency in the literature was also observed for birth outcomes in the recent chlorpyrifos HHRA (USEPA, 2014) which notes that researchers from CCCEH, Mt. Sinai, and CHAMACOS also investigated the possible role of prenatal OP exposure and fetal growth. These results were not consistent across these cohorts. Authors with CCCEH observed evidence of an inverse association, i.e., increasing cord blood chlorpyrifos was associated with decreased measures of birth weight and length, while authors with the Mt. Sinai and CHAMACOS cohorts reported either no association, or evidence of a positive relationship, respectively (Berkowitz et al., 2004; Eskenazi et al., 2004; Whyatt et al., 2004). Inconsistent results may be due to differences across study groups in exposure profiles as well as dissimilar methods of prenatal OP exposure assessment (Needham, 2005). 

Given the lack of consistency among cohorts for the fetal growth metrics, the proposed link between fetal growth and OP exposure is tenuous.  Therefore, consistent with previous evaluations for chlorpyrifos, EPA is focusing the remainder of this document on neurodevelopmental outcomes.  Although the agency is not evaluating these birth outcome studies further at this time, the agency will continue to monitor the scientific literature for advances in this line of research.

2.2.8	Neurodevelopment Outcome Epidemiologic Studies 

From an initial total of 30 high or moderate quality studies, with the exclusion of six studies for insufficient exposure assessment, three studies with measurable AChE inhibition and potential cholinergic toxicity which are outside the scope of this analysis, one study from an illegal use of a highly potent OP (MP) where cholinergic toxicity cannot be ruled out, five studies with birth outcome as the only assessment, two with outcomes not related to neurodevelopmental outcomes, and one study assessing only PON1 genotype expression and neurobehavioral outcomes (Fortenberry et al, 2014a), a total of twelve studies from the 2015/2016 literature review focusing on neurodevelopmental outcomes remain to be evaluated. Eight are considered high quality (Bouchard et al., 2010; Fortenberry et al., 2014; Furlong et al., 2014; Donauer et al. (2016), Stein et al. (2016), Rauh et al, (2015), Engel et al (2015), and Yolton et al. (2013). Four are considered moderate quality studies (Guodong et al., 2012; Oulhote and Bouchard, 2013; Zhang et al., 2014; Shelton et al., 2014).  

Furlong et al (2014) is from the Mt. Sinai cohort study; Stein et al (2016) is from CHAMACOS, and Rauh et al (2015) is from CCCEH--- thus these studies share many of the same strengths and uncertainties as other studies from these cohorts.    

Most of these studies used biomarker measures for their exposure assessment, including child or maternal urinary DAPs (Bouchard et al., 2010; Furlong et al., 2014; Guodong et al., 2012; Oulhote and Bouchard, 2013; Zhang et al., 2014; Yolton et al, 2013; Donauer et al, 2016; Stein et al, 2016, Engel et al, 2015), child urinary DMP and DEPs (Oulhote and Bouchard, 2013; Zhang et al., 2014; Guodong et al., 2012), or maternal TCPy (Fortenberry et al., 2014).  Shelton et al. (2014) used pesticide use data from the California Department of Pesticide Regulation and geospatial methods to map the specific pesticide use pattern to the participant residence.  Engel et al. (2015) reports on the results of a pooled analysis from HOME, CCCEH, CHAMACOS, and Mt. Sinai to evaluate the association between prenatal urinary DAPs and neurodevelopmental outcomes at 24 months only.

The study populations include national in scope (Bouchard et al., 2010; Oulhote and Bouchard, 2013); mainly urban (Fortenberry et al., 2014; Fortenberry et al., 2014a; Furlong et al., 2014; Guodong et al., 2012; Zhang et al., 2014; Rauh et al, 2015, Shelton et al., 2014); suburban (Yolton et al, 2013; Donauer et al, 2016); and agricultural (Stein et al, 2016).  

2.2.8.1	CCCEH, CHAMACOS, & Mt. Sinai Cohorts  

The CCCEH, Mt. Sinai, and CHAMACOS studies reflect different types of exposed groups in the total population which strengthens the weight of the evidence considerations regarding this stream of information. The CCCEH Mother's and Newborn study and the Mt. Sinai Child Growth and Development study participants were likely exposed to OPs through the diet and through residential use of the pesticide for indoor pest control. In the residential setting, study populations were most likely exposed through indoor residential use of the pesticide during the study time period and additionally exposed to OPs via the oral route through ingesting residues in the diet and from hand-to-mouth contact with in-home surfaces, as well as possible dermal or inhalation exposure through contact with treated areas in the home environment (Berkowitz et al., 2003; Whyatt et al., 2003; Whyatt et al., 2009; Whyatt et al., 2007). In contrast, CHAMACOS cohort participants were employed as farm laborers or were residing in homes with farm laborers. The CHAMACOS study participants likely experienced exposure to OPs through the diet and from occupational exposure (primarily inhalation and dermal routes), as well as probable indirect take-home exposures (the "tracking in" of pesticide residues through shoes and clothing, augmented by poor hygiene practices) (Bradman et al., 2007).  In each of these three US children's health cohorts, the biological measurements in these cohorts were comparable to the general population NHANES.  

These cohort studies each enrolled pregnant women during roughly the same time period, measured both environmental exposure to the pesticide during pregnancy and also measured biomarkers representing internal dose during pregnancy and at delivery, and prospectively assessed associations in their newborns and young children through age 7 years. Each study includes several hundred (approximately 100-400) mother-infant pairs; these sample sizes are sufficient to perform statistically valid analyses.  Investigators from each study cohort utilized a similarly strong study design (prospective birth cohort); measured pesticide exposure using several different methods including environmental indicators as well as specific and non-specific biomarkers of OPs; ascertained developmental outcomes using validated assessment tools well-established in both clinical and research settings; and, measured, analyzed, selected  and statistically adjusted for potentially confounding variables including socio-economic status and other environmental exposures using reasonable and appropriate methods. Limitations exist as well. These studies utilized a one-time measure (or the average of two measures) of chlorpyrifos or OP exposure to assess prenatal pesticide exposure throughout the gestational period, were unable to assess the influence of mixtures (co-occurring exposures in the relevant biological time window), and reflect a small sample size to fully evaluate the effect of more than one simultaneous exposure on neurodevelopment, i.e., evidence of effect modification.

As noted, two major uncertainties in environmental epidemiology studies are the accurate and reliable measurement of exposure and potential confounding variables such as the influence of mixtures.  The researchers with each of the three cohorts have provided supplemental methodological research to address these areas to the extent possible. Across the three children's health cohorts, study authors measured biomarkers of OP exposure. There is uncertainty as to the extent measurement of non-specific metabolites of OP or chlorpyrifos accurately reflects OP exposure; CCCEH and Mt. Sinai studies do not estimate post-natal exposure to chlorpyrifos among child participants, therefore the influence of early life and childhood OP exposure is unaccounted for in these analyses. The CHAMACOS cohort measured urinary levels of DAPs in young children and with the exception of pervasive developmental disorder (PDD) [per 10-fold increase in prenatal and post-natal DAP [prenatal: odds ratio (OR) = 2.3, p = 0.05; child DAPs OR = 1.7, p = 0.04]. did not observe negative significant associations in relation to neurodevelopment from post-natal exposure (Eskenazi et al., 2007; Marks et al, 2010). The CHAMACOS cohort investigators also measured AChE and butyl ChE as supplemental indicators of OP exposure.

Potential confounding bias is another major uncertainty within environmental epidemiology studies. Confounding variables, exposures that could be related to OP exposure and neurodevelopmental outcomes such as blood lead, may result in an incorrect epidemiological risk estimate. Across these cohort studies, investigators collected relevant information concerning demographic characteristics and other environmental exposures, and were, to the extent possible with the existing information, able to effectively hold constant the influence of these other variables when estimating the association between prenatal chlorpyrifos and adverse neurodevelopmental outcomes. Control of these variables is important to reduce the chances of a false positive study result. Overall, statistical analyses were judged to be appropriate and reasonable (not overly large number of statistical model variables) to the research question by EPA and expert Panel reviews (FIFRA SAP 2008 and 2012).

Researchers with both the Mt. Sinai and CHAMACOS cohorts evaluated neonatal neurological functioning in association with prenatal OP exposure; CCCEH did not conduct these measurements. To measure indices of abnormal neonatal behavior and/or neurological integrity authors used outcome measures derived from the Brazelton Neonatal Behavioral Assessment Scale (BNBAS), a neurological assessment of 28 behavioral items and 18 primitive reflexes. This tool was administered to infants 2-5 days post-partum by trained neonatologists in the hospital setting using similar environmental conditions. The authors with both study groups observed an increased number of abnormal reflexes in relation to increasing measures of OP exposure (Engel et al., 2007; Young et al., 2005). Among the other 27 measures in the BNBAS, neither study group reported evidence of any other positive associations. The authors also observed evidence of potential effect modification by PON1 activity level in the relation between DAPs and neonatal neurodevelopment in which infants of mothers who are slower metabolizers have greater risk of abnormal reflexes (Young et al. 2005; Engel et al. 2007). However, EPA notes these studies are likely under-powered to make a statistically robust estimate of this statistical interaction.

Researchers across the three children's health cohorts utilized the Bayley Scales of Infant Development II (BSID-II) to generate a Mental Development Index (MDI) and a Psychomotor Development Index (PDI) to assess neurodevelopment in early childhood. In the CCCEH Mothers and Newborn study, Rauh et al. (2006) investigated MDI and PDI at 12, 24, and 36 months of age.  Children were categorized as having either high (>6.17pg/g) or low (<6.17pg/g) prenatal chlorpyrifos exposure, using categories informed by results of the previous study on birth characteristics (Whyatt et al., 2004).  Authors reported that the difference in MDI scores was "marginally significant" (p = 0.06) between the "high" and "low" exposed groups; the high exposed group scoring an average of 3.3 points lower than the low exposed (Rauh et al., 2006).  Regarding the PDI score (motor skills), none of the 12 or 24 month PDI scores showed significant effects, but the 36 month score was significantly related to chlorpyrifos exposure. Researchers noted that the effects were most pronounced at the 36 month testing period. 

CCCEH study authors (Rauh et al., 2015) evaluated the relationship between prenatal chlorpyrifos exposure and motor development/movement among 271 of the cohort participants who had reached the age of approximately 11 years.  When comparing children in the upper tertile of exposure (>6.17 pg/g; N=43) to those in the lower tertiles (N=228), they observed statistically significant associations between prenatal chlorpyrifos exposure and
mild to moderate tremor in the dominant arm, both arms, either arm, and a marginally statistically significant association in the non-dominant arm. The specific OR calculated associated with these elevated risks of arm tremor are as follows: dominant arm (OR=3.2; 95% CI=l .3-8. l; p=O.O 15); both arms (OR=3.3 ; 95% CI=l .1-9.4; p=0.027); either arm (OR=2.2; 95% CI=l .1-4.6; p=0.028); and non-dominant arm (OR=2. l; 95% CI=0.99-4.3; p=0.055). These associations were observed even after controlling for potential confounding factors such as medication, sex, and ethnicity.

Within the 36 month testing period, the likelihood of highly exposed children developing mental delays were significantly greater (MDI: 2.4 times greater (95% CI: 1.12-5.08, p = 0.02) and PDI: 4.9 times greater (95% CI: 1.78-13.72; p = 0.002)) than those with lower prenatal exposure (Rauh et al., 2006).  Within the Mt. Sinai study, authors administered the BSID-II to participating children at 12 and 24 months and observed that prenatal total DAP metabolite level was associated with a decrement in mental development at 12 months among blacks and Hispanic children; however, these associations either attenuated or were non-existent at the 24-month visit (Engel et al., 2011). In the CHAMACOS cohort, Eskenazi et al. (2007) observed that prenatal DAP levels were adversely associated with MDI, and at 24 months of age these associations reached statistical significance. In this study, neither prenatal DAPs nor maternal TCPy were associated with PDI (motor skills), nor did authors observe evidence of different risk by PON1 status (Eskenazi et al., 2010).

With respect to the findings related to the autism spectrum, from CCCEH, Rauh et al. (2006) reported a large odds ratio for PDD (OR=5.39; 95% CI: 1.21-24.11) when comparing high to low chlorpyrifos exposure groups.  Among 7-9 years old children in the Mount Sinai Cohort (Furlong et al. 2014), there was no overall statistically significant association between maternal third trimester urinary DAP metabolite levels and reciprocal social responsiveness.  However, some evidence of modification of the association between prenatal OP pesticide exposure and impaired social responsiveness in early childhood was observed by both race/ethnicity and child sex, with an association between DEAP and poorer social responsiveness observed among black participants and boys.   No association was observed among whites or Hispanics, among girls, or for DAP or DMAP biomarker levels.  In the CHAMACOS cohort, Eskenazi et al. (2010) reported non-significant, but suggestive, increased odds of PDD of 2.0 (0.8 to 5.1; p=0.14), whereas Eskenazi et al (2007) reported a statistically significant association between total DAP exposure and increased odds of PDD.

With respect to attention problems, Rauh et al. (2006) also investigated 36-month child behavior checklist (CBCL) (behavioral) scores. Significant differences were observed between the high and low chlorpyrifos exposure groups in the general category of attention-problems (p=0.010), and in the more specific DSM-IV scale for ADHD problems (p = 0.018).  The CHAMACOS cohort also investigated attention problems in early childhood using three different assessment tools: maternal report of child behavior at 3.5 and 5 years of age; direct assessment of the child at 3.5 and 5 years; and by a psychometrician's report of the behavior of the child during testing at 5 years. In this study population, higher concentrations of OP metabolites in the urine of pregnant women were associated with increased odds of attention problems and poorer attention scores in their children at age 5 years (Marks et al, 2010).

To measure intelligence among school aged children, authors from each of the three children's health cohorts used the Wechsler Intelligence Scale for Children, 4th edition (WISC-IV). The instrument measures four areas of mental functioning: the Verbal Comprehension Index, the Perceptual Reasoning Index, the Working Memory Index, and the Processing Speed Index. A Full-Scale IQ score combines the four composite indices. WISC-IV scores are standardized against U.S. population-based norms for English and Spanish-speaking children. In the CCCEH Mothers and Newborn Study, Rauh et al. (2011) evaluated the relationship between prenatal chlorpyrifos exposure and neurodevelopment among 265 of the cohort participants who had reached the age of 7 years and had a complete set of data including prenatal maternal interview data, prenatal chlorpyrifos marker levels from maternal and/or cord blood samples at delivery, postnatal covariates, and neurodevelopmental outcome data (Rauh et al., 2011).  While models were developed using continuous measures of both prenatal chlorpyrifos exposure and Wechsler scores, for ease of interpretation, investigators reported that for each standard deviation increase in exposure (4.61 pg/g) there is a 1.4% reduction in Full-Scale IQ and a 2.8% reduction in Working Memory. In the Mt. Sinai study, prenatal maternal DEP urinary metabolite concentrations were associated with slight decrements in Full Scale Intelligence Quotient (FSIQ), Perceptual Reasoning, and Working Memory between the ages of 6 and 9 years, and difference in intelligence measures by putative PON1 status were also noted (Engel et al., 2011). Similarly, in the CHAMACOS cohort, Bouchard et al. (2011) observed evidence of an association between prenatal exposures to OPs as measured by urinary DAP (total DAP, DEP, and DMP) metabolites in women during pregnancy, and decreased cognitive functioning in children at age 7. In this study, children in the highest quintile of maternal DAP concentrations had a statistically significant 7-point difference in IQ points compared with those in the lowest quintile.

To determine if early childhood social adversities modify the association between maternal prenatal OP exposure and cognition in children (measured using the WISC-IV results), Stein et al. (2016) showed negative associations between maternal prenatal exposure to DAP metabolites and cognition in children were observed to be stronger among children who reported more adversity, relative to children who experienced less adversity.  For example, for boys who experienced a more adverse learning environment, a statistically significant decrease was observed between prenatal DAP concentrations and Full-Scale IQ (β = -13.3; 95% CI: -19.9; -6.7; p < 0.01), relative to boys who lived in a less adverse learning environment (β = 4.2; 95% CI: -4.2; 12.5; p < 0.01).  A similar observation was reported for perceptual reasoning and processing speed (β = -9.8; 95% CI: -17.4; -2.2; p < 0.01; β = -12.7; 95% CI: -19.5; -6.0; p = 0.01), respectively.  For girls who experienced more economic adversity, a stronger effect on the association between prenatal DAP concentrations and Full-Scale IQ was observed, relative to girls who experienced less economic adversity (β = -8.5; 95% CI: -16.7, -0.4 vs. β = -4.7 95% CI: -12.8, 3.4; p = 0.18).  This was also observed for working memory for girls (β = -5.9; 95% CI: -13.7, -1.8 vs. β = -2.6 95% CI: -10.0, 4.8; p = 0.05). In addition, similar findings were observed for girls whose mother's had more maternal adversity (β = -11.5; 95% CI: -18.5, -4.4 vs. β = -1.5 95% CI: -7.9, 4.9; p = 0.07).  

To ascertain whether observed differences in neurodevelopment after prenatal chlorpyrifos exposure may be explained by differences in brain morphology between exposed groups, investigators compared MRI brain images between high and low chlorpyrifos exposed child study participants (Rauh et al., 2012). Authors determined there were distinct morphological differences in brain areas associated with these neurodevelopmental outcomes. The pilot study included 40 child participants due to strict inclusion and exclusion criteria, and the high cost of performing the imaging studies on each child. EPA convened a Federal Panel of experts to perform a written peer-review of this study.4F The Federal Panel concurred with the authors' conclusions in general; however, the Federal Panel also noted that significantly greater and more sophisticated MRI imaging studies would be needed to link the morphological changes indicated in this study with specific functional outcomes noted in the CCCEH IQ study. Therefore, while generally supportive of the epidemiologic findings, additional study is needed to make specific links with areas of brain development change.

In sum, across these three children's environmental health studies, authors consistently identified associations with neurodevelopmental outcomes in relation to OP exposure.  There is evidence of delays in mental development in infants (24-36 months), attention problems and autism spectrum disorder in early childhood, and intelligence decrements in school age children who were exposed to chlorpyrifos or OPs during gestation. Investigators reported strong measures of statistical association across several of these evaluations (odds ratios 2-4 fold increased in some instances), and observed evidence of exposures-response trends in some instances, e.g., intelligence measures.

EPA has considered the strengths and limitations of these studies, and believes that random or systematic errors in the design, conduct or analysis of these studies were unlikely to fully explain observed positive associations between in utero OP exposure and adverse neurodevelopmental effects observed at birth and through childhood (age 7 years). EPA believes these are strong studies which support a conclusion that OPs likely played a role in these outcomes.

2.2.8.2	HOME Cohort

Two studies from the HOME cohort on neurodevelopmental outcomes have been reported (Yolton et al, 2013; Donauer et al, 2016).  The HOME study is a prospective cohort study conducted from March 2003 to February 2006, and the study population included pregnant women living in the Cincinnati, Ohio area who successfully delivered live-born infants.  The HOME cohort is a more recent US cohort compared to CHAMACOS, Mt. Sinai and CCCEH and represents exposure from post mitigation activities on many OPs which occurred during tolerance reassessment.   Thus, the exposure pattern to the mothers in the HOME cohort and the study demographics (i.e. suburban, middle class study population) are likely different relative to studies from CHAMACOS, Mt. Sinai and CCCEH. Moreover, the demographic data reported in the HOME cohort indicate that mothers tended to have a higher socioeconomic status and were more likely to eat and live a healthier lifestyle relative to mothers involved in CHAMACOS, Mt. Sinai and CCCEH.  As a result, these life habits were potentially protective towards the OP exposures during pregnancy.  
Exposure was assessed twice during pregnancy -- at 16 +- 4 weeks and 26 +- 4 weeks of gestation -- via spot urine samples that were stored and later laboratory tested.  The urine samples were measured for six DAPs.  
In the Yolton et al (2013) study, a neurobehavioral assessment, the NICU Network Neurobehavioral Scale (NNNS), was conducted at approximately 5 weeks of age, and each infant was scored based on several categories (or subscales) of the NNNS.  A total of 350 mother/ infants were included within this study, and a statistically significant association (p < 0.05) between maternal metabolite urinary concentrations (DE and DAP) and the following NNNS subscales for infant performance was observed: attention, lethargy, hypotonia, and autonomic stress.   After adjusting the model for covariates, evidence of an association was observed between increased mean urinary DE metabolites and increased attention (β = 0.066, p < 0.05), with no statistically significant covariates.  Decreased lethargy and hypotonia were also associated with increased urinary DE metabolites at 16 weeks of gestation (β = -0.069, p = 0.04; β = -0.101, p = 0.03), with black race and birthweight as statistically significant covariates (β = -0.462, p = 0.03; β = -0.044, p = 0.01).  For DAP metabolite concentrations, evidence of an association was observed between increased DAP concentrations at 26 weeks of gestation and decreased autonomic stress (β coefficient = -0.010, p = 0.01), with birthweight of infants and maternal blood lead levels during gestation as statistically significant covariates (β = -0.003, p = 0.02; β = 0.031, p = 0.02), respectively.   
In Donauer et al, 2016, offspring were assessed through neurodevelopmental examinations conducted yearly, at ages 1-5 years.  The Bayley-II assessed mental and motor advancement for children ages 1-3 years, and the MDI and PDI were used to analyze cognitive, language, and motor development.  For 4 year-old children, language skills were assessed by the Clinical Evaluation of Language Fundamentals-Preschool, and the WPPSI was used to assess intelligence (IQ) including verbal, performance, and full-scale for children 5 years of age.  Of the 327 maternal-child cases, evidence of a significant association between maternal levels of urinary metabolites (total DAP and total DM) and verbal IQ of the child from the Wechsler Preschool and Primary Scale of Intelligence test was observed in both the unadjusted and adjusted models at the p < 0.2 level (unadjusted model: β = 0.005, p = 0.034, β = 0.003, p = 0.144; adjusted model: β = 0.003, p = 0.179, β = 0.003, p = 0.144), respectively.  Although evidence of a positive association was observed for total DAP and total DM metabolites and full-scale IQ (unadjusted model: β = 0.004, p = 0.111; β = 0.004, p = 0.124, when the model was adjusted, no evidence of a positive association was observed (adjusted model: β = 0.001, p = 0.494; β = 0.002, p = 0.441).  No evidence of a positive association was observed between performance IQ of the child and total DAP, DM, and DE, respectively.  For Bayley-II MDI, evidence of a positive association was observed for maternal exposure to total DM and total DE at the child's 2-year visit (unadjusted model: β = 0.003, p = 0.16; β = 0.003, p = 0.16); however, when the model was adjusted, no evidence of a positive association was observed for total DM and total DE (adjusted model: β = 0.002, p = 0.28; β = 0.002, p = 0.37).  No evidence of a positive association was observed between maternal exposure (for total DAP, total DM, and total DE) and the Bayley-II Psychomotor Developmental Index for children aged 1-3 years.
2.2.8.3	Meta-analysis (Engel et al, 2015) 

Engel et al. (2015) reports on the results of a pooled analysis from four cohorts (N=936) to evaluate the association between prenatal DAPs and neurodevelopmental outcomes at 24 months only. In addition, researchers in this study assessed the impact on these associations of the specific cohort, race/ethnicity, and the PON I genotype of study participants. Researchers across the four children's health cohorts utilized the BSID-11 to generate MDI and PDI to assess neurodevelopment in early childhood. The four cohorts include CHAMACOS (N=377), HOME (N=265), Mt. Sinai (N=234), CCCEH (N=60), It is noted that the CCCEH participants included in this analysis are from women enrolled in 2000 to 2001, a time period which is during the phase out of chlorpyrifos in residential settings.
The results of this pooled study are relatively consistent with those seen in the individual cohorts at 24 months. After controlling for race/ethnicity, smoking, and drug use during pregnancy, a statistically significant association was observed in the pooled population between total DAPs exposure and MDI decrements, but not with PDI decrements. Consistent with the results from Eskenazi et al. (2007), the strongest evidence of an association was observed for the CHAMACOS cohort, with statistically significant associations for both total DAPs and total DMAPs exposure and MDI decrements. No significant associations were seen within the Mt. Sinai and CCCEH cohorts, a result which is basically consistent with the previous observations at 24 months in these cohorts (Engel et al., 2011; Rauh et al.,2006). The study authors observed significant heterogeneity from combining the cohorts, especially with regards to race/ethnicity, and noted that impacts on specific subpopulations may be lost when looking at the pooled results.

2.2.8.4	Other Neurodevelopmental Studies 

In the first of two Chinese studies focusing on generic neurodevelopmental outcomes, Zhang et al. (2014) investigated prenatal exposure to OPs and neurobehavioral development of neonates in a birth cohort study in Shenyang, China. The authors reported that consistent statistically significant associations were observed between all of the quantified urinary biomarkers of prenatal OP pesticide exposure and neonatal neurodevelopment deficits assessed 3 days after birth.  A 10-fold increase in total DAPs concentration was associated with an average decrease in Neonatal Behavioral Neurological Assessment (NBNA) summary scores of 1.78 points (95% CI: -2.12 to -1.45). No evidence of departure from linearity of the exposure-response relationship between maternal DAP concentrations and NBNA scores was observed.  In the second Chinese study focusing on neurodevelopmental outcomes, Guodong et al. (2012) conducted a cross-sectional study, and did not identify any statistically significant associations between the children's urinary DAP metabolite levels and any of the DQ (Developmental quotients) scores.  The authors mentioned that their results should be interpreted with caution since OP exposure was quantified in single spot urine sample from children, and should be followed up with a longitudinal study with repeated measurement of exposure levels in urine samples.
Two studies focused on impaired social responsiveness, autism spectrum disorders, or developmental delays (Furlong et al., 2014; Shelton et al., 2014).  Among 7-9 years old children in the Mount Sinai Cohort (Furlong et al. 2014), there was no overall statistically significant association between maternal third trimester urinary DAP metabolite levels and reciprocal social responsiveness (a measure linked to many neuropsychiatric conditions that involve impaired social functioning (Constantino and Gruber 2005)).  However, some evidence of modification of the association between prenatal OP pesticide exposure and impaired social responsiveness in early childhood was observed by both race/ethnicity and child sex, with an association between DEP and poorer social responsiveness observed among black participants and boys.  No association was observed among whites or Hispanics, among girls, or for DAP or DMP biomarker levels.
Shelton et al. (2014), investigated autism spectrum disorders and developmental delay (DD) in relation to gestational residential proximity to agricultural pesticide applications utilizing the California population-based Childhood Autism Risks from Genetics and Environment (CHARGE) study. The investigators reported that children with autism spectrum disorder were 60% more likely to have OPs applied near the home [1.25 km distance; adjusted OR (aOR) = 1.60; 95% CI: 1.02 - 2.51] than mothers of normally developing children. They added that as the buffer distance grew larger, these associations became lesser, indicating an exposure-response effect. The authors also mentioned that each 100-lb (45.4 kg) increase in the amount applied over the course of pregnancy (within 1.5 km of the home) was associated with a 14% higher prevalence of autism spectrum disorder (aOR = 1.14; 95% CI: 1.0, 1.32), but no association was identified with DD.
A total of three studies focused on OP exposure and behavioral, memory, or attention/ADHD outcomes (Oulhote and Bouchard, 2013; Bouchard et al., 2010; Fortenberry et al., 2014).  In a national cross-sectional study of Canadian children 2007-2009 data for age 6-11 years (Oulhote and Bouchard, 2013), there were no overall statistically significant associations observed between child urinary DAP, DMP, or DEP metabolite levels and parentally reported behavioral problems.  In contrast, Bouchard et al. (2010), looking at U.S. children age 8-15 years in the 2000-2004 National Health and Nutrition Examination Survey (NHANES),5F observed a positive association between attention and behavior problems and DAPs and DMPs, but not DEPs.  For example, even after controlling for potential confounders such as sex, age, ethnicity, and creatinine concentration, they found that a 10-fold increase in DMAP concentration was associated with a 55 to 72% increased odds of ADHD.

Fortenberry et al. (2014) evaluated the relationship between pesticide exposure and ADHD in school age Mexican children, recruiting 187 mother-child pairs from a prospective birth cohort, ELEMENT (Early Life Exposures in Mexico to Environmental Toxicants). The authors reported that, there were no statistically significant associations between tertiles of maternal third trimester urinary TCPy and measures of attention and hyperactivity in children.  However, there was suggestive evidence for increases in the ADHD index in relation to TCPy tertiles among boys (the highest TCPy tertile was associated with an ADHD index score that was 5.55 points higher than children in the lowest tertile; p-value = 0.06). 

3.0	Weight of Evidence Analysis:  Integration Across Multiple Lines of Evidence

OPP's 2016 "Framework for Incorporating Human Epidemiologic & Incident Data in Risk Assessments for Pesticides" provides the foundation for evaluating multiple lines of scientific evidence (U.S. EPA, 2016c).  OPP uses a WOE analysis for evaluating epidemiology and human incident data, such that conclusions are made on the preponderance of the information rather than relying on any one study.  OPP uses the modified Bradford Hill criteria like those in the MOA/human relevance framework as a tool for organizing and integrating information from different sources (Hill, 1965; U.S. EPA, 1999, 2005; Sonich-Mullin et al., 2001; Meek et al., 2003; OECD AOP Wiki Users Handbook6F).   It is important to note that the Bradford Hill Criteria are not intended as a check box approach but instead are points to consider when evaluating the totality of evidence.  In addition, the availability of a fully elucidated MOA/AOP is a not requirement for using epidemiology studies in human health risk assessment.  The Bradford Hill Criteria explicitly considers such concepts as strength, consistency, dose response, temporal concordance and biological plausibility in a weight of evidence analysis; sections 3.1-3.3 below summarize the available evidence based on these principles.

The agency's 2002 guidance on "Determination of the Appropriate FQPA Safety Factor(s) in Tolerance Assessment" contains a discussion of the relative weighing of animal and human data with respect to pre- and post-natal toxicity, dose-response, toxicokinetics, and mode of action.  The guidance specifically states that 

      "potential for pre- and postnatal toxicity can be determined from human and animal studies.  Although human studies are seldom available, human data are the most relevant data for assessing potential health risks. When sufficient human data are available to judge that an adverse developmental outcome is related to exposure, the degree of concern increases (p. 33, emphasis added)."

Table 1 of the 2002 FQPA Safety Factor guidance notes that when effects are found in humans related to exposure that these data receive "Increasing Weight".  This table lists the following factors as receiving increasing weight in the WOE analysis with respect to pre- and post-natal toxicity:  

::	Effects found in humans related to exposure
::	Same types of effects seen in more than one species
::	Effects of a different type with greater potential consequences in young compared to adults
::	Persistence or relatively longer recovery of effects in young compared to adults

The agency notes that each of these factors applies to the database of studies for OPs.  As described in detail above, epidemiology studies from multiple investigators representing multiple locations and different populations show associations between OP exposure and adverse neurodevelopmental outcomes.  Regarding multiple laboratory species in the database of laboratory studies, a range of effects on the developing nervous system has been shown in mouse (e.g., Braquenier et al., 2010; Venerosi et al., 2015), rat (e.g., Carr et al., 2015; Vatanparas et al., 2013), and guinea pig (e.g., Mamczarz et al., 2016).  Although very high levels of OP exposure can lead to coma, paralysis, or even death, the 10% AChE point of departure represents a precursor event believed to be protective of this downstream cholinergic neurotoxicity.  The 10% AChE PoD is much lower than these exposure levels and thus is believed to be health protective of cholinergic neurotoxicity.  In contrast, neurodevelopmental effects such as reduced IQ, autism spectrum disorder, and ADHD are adverse health outcomes of significant public health consequence.  Moreover, these types of neurodevelopment effects are likely to persist through life.  Further evidence of the long term consequence of such exposure is found in Rauh et al (2015) who have shown even at age 11 children exposed to chlorpyrifos in the home environment were more likely to exhibit mild or mild to moderate arm tremor.

3.1	Dose-Response Relationships & Temporal Concordance

Since the MOA(s)/AOP(s) is/are not established for neurodevelopmental outcomes (USEPA, 2012, 2014), it is not possible to describe the concordance in key events or biological steps leading to neurodevelopmental outcomes.  As such, the quantitative linkages between molecular initiating events (MIE)s, intermediate steps, and ultimately the adverse outcome (i.e., neurodevelopmental effects) cannot be determined. (See Appendix 6)

With respect to the timing of exposure, across the epidemiology database of studies the maternal urine, cord blood and other (meconium) measures provide evidence that exposure did occur to the fetus during gestation, but the actual level of such exposure during the critical window(s) of susceptibility is not known.  While significant uncertainties remain about the actual exposure levels experienced by mothers and infant participants in the children's health cohorts, it is unlikely that these exposures resulted in AChE inhibition.  As part of the CHAMACOS study, Eskenazi et al. (2004) measured AChE activity and showed that no differences in AChE activity were observed.  The biomarker data from the Columbia University studies are supported by the agency's dose reconstruction analysis using the PBPK-PD model.  Following the recommendation of the FIFRA SAP (2012), the agency conducted a dose reconstruction analysis of residential uses available prior to 2000 for pregnant women and young children inside the home (USEPA, 2014).  Based on the output from the PBPK-PD model, for the highest exposure considered (i.e., indoor broadcast use of a 1% chlorpyrifos formulation), <1% RBC AChE inhibition in pregnant women would be expected.  While uncertainty exists as to actual OP exposure at (unknown) critical windows of exposure, EPA believes it is unlikely individuals in the epidemiology studies experienced RBC AChE inhibition.

Preamble to the Integrated Science Assessments (ISAs) which serve as a scientific foundation for the review of EPA's National Ambient Air Quality Standards (NAAQS) notes that strong evidence for causality can be provided through "natural experiments" when a change in exposure is found to result in a change in occurrence or frequency of health (USEPA, 2015c).   Within the Columbia University epidemiology studies, the relationship in time between prenatal chlorpyrifos exposure and adverse neurodevelopmental outcomes is concordant. The time period under study within the Columbia University (CCCEH) study, spanned the point in time in which pesticide manufacturers voluntarily cancelled the use of chlorpyrifos in the home environment, and researchers were able to show the change in exposure before (high use period) and after (low/no use period) the period of removal of chlorpyrifos products from the residential marketplace. Moreover, prior to the voluntary cancellation there were >80% detectable levels of chlorpyrifos in cord blood but in the time period after the cancellation only 16% of the measured values were greater than the level of detection (LOD); there was only one child born in the time period subsequent to the voluntary cancellation of chlorpyrifos in the residential marketplace for whom the cord blood chlorpyrifos level was in the upper-tertile of pre-cancellation exposure levels. The significantly reduced proportion of measured values greater than the LOD as well as the observation of an absence of an association between prenatal chlorpyrifos exposure among infants born after the voluntary cancellation of chlorpyrifos and neurodevelopmental effects support the hypothesis that chlorpyrifos is related to these outcomes. However, as noted by study authors, EPA and the FIFRA SAP (2012), this could also be due to inadequate sample size to detect a small to modest effect among the group of infants born after the voluntary cancellation.  It is notable that epidemiology studies from other research groups have not included analyses across different years of exposure.

3.2	Strength, Consistency & Specificity 

In making a weight-of-evidence analysis, it is important to consider the strength of the statistical measures of association between OP exposure and adverse neurodevelopmental outcomes through childhood (epidemiology) and possibly into adulthood (animal studies). It is also important to consider the strength of the integrated qualitative and quantitative evidence, the consistency of the observed associations across epidemiology studies and considering both animal and human data support the conclusion that chlorpyrifos plays a role in adverse neurodevelopmental outcomes. 

3.2.1	Strength

As noted in the 2016 epidemiology and incident framework, findings of large, precise risks increases confidence that the association is not likely due to chance, bias, or other factors (USEPA, 2016c).  In the case of the CCCEH study, there are  a number of notable associations. Regarding infant and toddler neurodevelopment, the CCCEH authors reported statistically significant deficits of 6.5 points on the PDI at 3 years of age when comparing high to low exposure groups (Rauh et al., 2006). Notably these decrements in PDI persist even after adjustment for group and individual level socioeconomic variables (Lovasi et al., 2011). These investigators also observed increased odds of mental delay (OR=2.4; 95% CI: 1.1-5.1) and psychomotor delay (OR=4.9; 95% CI: 1.8-13.7) at age three when comparing high to low exposure groups (Rauh et al., 2006). Rauh et al (2006) also reported extremely large odds ratios for attention disorders (OR=11.26; 95% CI: 1.79-70.99), ADHD (OR=6.50; 95% CI: 1.09-38.69), and PDD (OR=5.39; 95% CI: 1.21-24.11) when comparing high to low chlorpyrifos exposure groups (Rauh et al., 2006). EPA notes that the magnitude of these results are so large that they are unlikely to be affected by residual confounding although limited sample sizes resulted in imprecise estimates. 

From the CHAMACOS cohort, Eskanazi et al (2007) reported that both prenatal and postnatal DAPs were associated with risk of PDD (per 10-fold increase in prenatal DAPs: OR = 2.3, p = 0.05; child DAPs OR = 1.7, p = 0.04). Marks et al (2010) reported prenatal DAPs were associated with scores on the K-CPT ADHD Confidence Index > 70th percentile (OR = 5.1; 95% CI: 1.7 - 15.7 and with a composite ADHD indicator of the various measures (OR = 3.5; 95% CI: 1.1 - 10.7). Some outcomes exhibited evidence of effect modification by sex, with associations found only among boys.  Children's concurrent total DAP and DMP metabolite levels at 3.5 years and 5 years were unrelated to attention outcomes, and but child DEP concentrations at 5 years were adversely associated with the composite measure of attention (OR = 2.0; 95% CI:1.1 - 3.6).  

In a recent evaluation by Stein et al (2016) from the CHAMACOS cohort, boys who experienced a more adverse learning environment, a statistically significant decrease was observed between prenatal DAP concentrations and Full-Scale IQ (β = -13.3; 95% CI: -19.9; -6.7; p < 0.01), relative to boys who lived in a less adverse learning environment (β = 4.2; 95% CI: -4.2; 12.5; p < 0.01).  A similar observation was reported for perceptual reasoning and processing speed (β = -9.8; 95% CI: -17.4; -2.2; p < 0.01; β = -12.7; 95% CI: -19.5; -6.0; p = 0.01), respectively.  For girls who experienced more economic adversity, a stronger effect on the association between prenatal DAP concentrations and Full-Scale IQ was observed, relative to girls who experienced less economic adversity (β = -8.5; 95% CI: -16.7, -0.4 vs. β = -4.7 95% CI: -12.8, 3.4; p = 0.18).  This was also observed for working memory for girls (β = -5.9; 95% CI: -13.7, -1.8 vs. β = -2.6 95% CI: -10.0, 4.8; p = 0.05).

3.2.2	Consistency

The Preamble to the ISA's note that an inference of causality is strengthened when a pattern of elevated risks is observed across several independent studies. The reproducibility of findings constitutes one of the strongest arguments for causality. Moreover, statistical significance is not the sole criterion by which the presence or absence of an effect is determined (USEPA, 2015c). 

Published and submitted laboratory animal studies have been reviewed for OPs.  The >30 papers on chlorpyrifos provide evidence of long-lasting neurodevelopmental disorders in rats and mice; however, there was no clear consistency in terms of pattern, timing, or dose response for these effects.  The additional toxicological literature and guideline DNT studies with the other OPs provide more evidence for the same conclusions, with again the same caveats and uncertainties.  While overall cognitive function and motor activity appeared to be altered the most often, it is apparent that these behaviors were also the most often evaluated.

Across the various epidemiology studies within the scope of this analysis and with adequate quality in design and exposure assessment, it is important to note that these studies used study methods that were highly variable, including different exposure measurement, outcome assessment, study design, and geographical location.  These differences make it challenging to compare the results across studies. Among the epidemiology studies, two of the cohorts (CCCEH and ELEMENT) have focused on chlorpyrifos whereas the other studies (Mt. Sinai cohort, CHAMACOS cohort, HOME cohort, CHARGE study, Bouchard et al., 2010) have focused on less specific biomarkers (i.e., DAPs) and are not specific to any particular OP. When considered in concert, the epidemiology studies provide consistent findings for some outcomes.  

In comparison to the studies from CCCEH, Mt. Sinai, and CHAMACOS, results from the ELEMENT and CHARGE studies provide some supportive evidence for the findings.  However, the results from the HOME cohort show different results; namely positive associations between neurodevelopmental outcomes and maternal DAPs (Yolton et al, 2013; Donauer et al, 2016).  The HOME study represents a different demographic profile (higher socioeconomic status, suburban, middle class) and different time period (2003-2006) compared to CCCEH, Mt. Sinai, and CHAMACOS which are made up of women with much lower socioeconomic status and who experienced different exposure patterns. 

In the two Chinese studies, EPA does not know how the OP exposures from these studies relate to the currently registered use pattern for OPs used in the U.S./North America.  In the case of these two Chinese studies (Guodong et al., 2012; Zhang et al., 2014), there may be differences in the study population and outcome measurements that may account for the observed differences in study results, with Zhang et al. (2014) documenting statistically significant associations for total DEAPs, total DMAPs, and total DAPs and Guodong et al. (2012) observing no association with these exposures.  The Zhang et al. (2014) study was conducted in Shenyang, with a study population reported as 87% urban and 13% rural, whereas the Guodong et al. (2012) study was conducted in Shanghai with a 99% urban and 1% suburban study population.  Given the higher percentage of study participants from rural areas, the study participants from Zhang et al. (2014) may have had different pattern and magnitude of OP exposures compared to those from Guodong et al. (2012).  In addition, it is noted that different outcome measurements were made in these studies, with Guodong et al. (2012) assessing 23-25 month old children using a developmental quotient score and Zhang et al. (2014) assessing 3 day old infants using a Neonatal Behavioral Neurological Assessment.  Given the different outcome assessments, exposure potential, study designs (cohort vs. cross-sectional), and ages of the participants in these two studies, it is difficult to draw conclusions on how these study results compare.  It is notable that the results from the Zhang et al. (2014) study focusing on neonates are consistent with those from other studies which reported statistically significant associations between delayed neurological development measured in newborns and total DEAP, total DMAP, and total DAP exposure (Engel et al., 2007; Young et al., 2005).  However, it is noted that neurological development was measured within a few days of birth for Zhang et al. (2014) and Engel et al. (2007), whereas Young et al. (2005) conducted their measurements within two months of birth.  

The CHAMACOS and Mt. Sinai cohorts that measured neurological effects at birth (the Brazelton index), observed a putative association with OPs (Engel et al., 2007; Young et al., 2005). Yolton et al (2013) used the NNNS tool at approximately 5 weeks of age in the HOME cohort and reported evidence of a positive association between increased mean urinary DE metabolites and increased attention and decreased autonomic stress. 

Researchers across CCCEH, Mt. Sinai, CHAMACOS, and HOME studies utilized the Bayley Scales of Infant Development II (BSID-II) to generate a MDI and a PDI to assess neurodevelopment in early childhood. In the CCCEH study, Rauh et al. (2006) investigated MDI and PDI at 12, 24, and 36 months of age.  Children were categorized as having either high (>6.17pg/g) or low (<6.17pg/g) prenatal chlorpyrifos exposure, using categories informed by results of the previous study on birth characteristics (Whyatt et al., 2004).  Authors reported that the difference in MDI scores was "marginally significant" (p = 0.06) between the "high" and "low" exposed groups; the high exposed group scoring an average of 3.3 points lower than the low exposed (Rauh et al., 2006).  Regarding the PDI score (motor skills), none of the 12 or 24 month PDI scores showed significant effects, but the 36 month score was significantly related to chlorpyrifos exposure. 

Within the 36 month testing period, the likelihood of highly exposed children developing mental delays were significantly greater (MDI: 2.4 times greater (95% CI: 1.12-5.08, p = 0.02) and PDI: 4.9 times greater (95% CI: 1.78-13.72; p = 0.002)) than those with lower prenatal exposure (Rauh et al., 2006).  Within the Mt. Sinai study, authors administered the BSID-II to participating children at 12 and 24 months and observed that prenatal total DAP metabolite level was associated with a decrement in mental development at 12 months among blacks and Hispanic children; however, these associations either attenuated or were non-existent at the 24-month visit (Engel et al., 2011). In the CHAMACOS cohort, Eskenazi et al. (2007) observed that prenatal DAP levels were adversely associated with MDI, and at 24 months of age these associations reached statistical significance. In this study, neither prenatal DAPs nor maternal TCPy were associated with PDI (motor skills), nor did authors observe evidence of different risk by PON1 status (Eskenazi et al., 2010). In contrast to the findings of CCCEH, Mt. Sinai, and CHAMACOS, the HOME study (Donauer et al, 2016) reported evidence of a positive association for MDI was observed for maternal exposure to total DM and total DE at the child's 2-year visit; however, when the model was adjusted, no evidence of a positive association was observed for total DM and total DE (adjusted model (β = 0.002, p = 0.28; β = 0.002, p = 0.37).  No evidence of an association was observed between maternal exposure (for total DAP, total DM, and total DE) and the Bayley-II Psychomotor Developmental Index for children aged 1-3 years.

With respect to attention problems, Rauh et al. (2006) also investigated 36-month child behavior checklist (CBCL) (behavioral) scores. Significant differences were observed between the high and low chlorpyrifos exposure groups in the general category of attention-problems (p=0.010), and in the more specific DSM-IV scale for ADHD problems (p = 0.018).  The CHAMACOS cohort also investigated attention problems in early childhood using three different assessment tools: maternal report of child behavior at 3.5 and 5 years of age; direct assessment of the child at 3.5 and 5 years; and by a psychometrician's report of the behavior of the child during testing at 5 years. In this study population, higher concentrations of OP metabolites in the urine of pregnant women were associated with increased odds of attention problems and poorer attention scores in their children at age 5 years (Marks et al, 2010).

Of the four new studies focusing on ADHD and OP exposure, three found statistically significant associations, with only Oulhote and Bouchard (2013) finding no association with total DMAP, DEAP, or total DAP exposure.  In contrast, Bouchard et al. (2010) observed an association with total DMAP and total DAP exposure and ADHD.  Fortenberry et al. (2014) found suggestive, but not statistically significant, evidence of an association with TCPy and ADHD in boys.  Overall, the Fortenberry and Bouchard study results are consistent with that of earlier studies from the CCCEH (Rauh et al., 2006) and CHAMACOS (Eskenazi et al., 2007; Marks et al., 2010).  Specifically, statistically significant associations were observed by Rauh et al. (2006) with chlorpyrifos exposure and ADHD, Eskenazi et al. (2007) with total DMAPs and total DAPs and ADHD, and Marks et al. (2010) with total DEAP, DMAP, and total DAP exposure.

It is important to put into context the specific outcome measures used in the assessment of attention and neurobehavioral problems.  For example, Bouchard et al. (2010) identified statistically significant associations between OP exposure and ADHD/behavioral problems, whereas Oulhote and Bouchard (2013) did not.  It is valuable to compare these studies given that they are both cross-sectional studies using large population level datasets with biomarker information, with Oulhote and Bouchard (2013) using a Canadian dataset and Bouchard et al (2010) using a U.S. dataset.   Bouchard et al. (2010) used criteria for ADHD from DSM-IV, whereas Oulhote and Bouchard (2013) used a "Strengths and Differences Questionnaire (SDQ)," with the SDQ being a more generic assessment of mental health status than the DSM-IV criteria.  When Oulhote and Bouchard (2013) compared their results to Bouchard et al. (2010), they noted that their outcome measurements may not have been as sensitive and that this may account for the difference in study results.

Across epidemiology studies looking at ADHD/behavioral problems, a suggestive or statistically significant positive association was observed in multiple studies between OP exposure and these neurobehavioral outcomes (Bouchard et al., 2010; Fortenberry et al., 2014; Rauh et al., 2006, Eskenazi et al., 2007).  While these studies have differences in the years that the exposure occurred, study design, exposure assessment, and outcome assessment, the commonality in their results is striking.

Several studies have now documented suggestive or positive associations between OP exposure and autism spectrum disorders (Rauh et al., 2006; Shelton et al., 2014; Furlong et al., 2014; Eskenazi et al., 2007; Eskenazi et al, 2010).  Specifically, Furlong et al. (2014) reported suggestive, but not statistically significant, evidence of an association between total DEAP exposure and reciprocal social responsiveness among black participants and boys.  These results are consistent with previous studies conducted on the CCCEH and CHAMACOS cohorts, with these studies also showing statistically significant associations between OP exposure and autism spectrum disorders (Rauh et al., 2006; Eskenazi et al., 2007).7F  Specifically, Eskenazi et al. (2007) reported a statistically significant association between PDD and total DAP exposure, whereas Rauh et al. (2006) showed a significant association between PDD and specifically chlorpyrifos exposure.  Both PDD and reciprocal social responsiveness are related to the autism spectrum disorder.  Using a different exposure assessment method (geospatial analysis and residential proximity to total OP exposure), Shelton et al. (2014) also documented statistically significant associations between total OP exposure and autism spectrum disorders.  While these studies vary in the magnitude of the overall strength of association, they have consistently observed a positive association between OP exposure and autism spectrum disorders.

Finally, CCCEH, Mt. Sinai, and CHAMACOS observed an inverse relation between the respective prenatal measures of OPs and intelligence measures at age 7 years.  To measure intelligence among school aged children, authors from each of the three children's health cohorts used the WISC-IV. Rauh et al. (2011) evaluated the relationship between prenatal chlorpyrifos exposure and neurodevelopment among 265 of the cohort participants who had reached the age of 7 years investigators reported that for each standard deviation increase in exposure (4.61 pg/g) there is a 1.4% reduction in Full-Scale IQ and a 2.8% reduction in Working Memory. In the Mt. Sinai study, prenatal maternal DEP urinary metabolite concentrations were associated with slight decrements in FSIQ, Perceptual Reasoning, and Working Memory between the ages of 6 and 9 years, and difference in intelligence measures by putative PON1 status were also noted (Engel et al., 2011). Similarly, in the CHAMACOS cohort, Bouchard et al. (2011) observed evidence of an association between prenatal exposures to OPs as measured by urinary DAP (total DAP, DEP, and DMP) metabolites in women during pregnancy, and decreased cognitive functioning in children at age 7. In this study, children in the highest quintile of maternal DAP concentrations had a statistically significant 7-point difference in IQ points compared with those in the lowest quintile.  The CHAMACOS results with the WISC-IV were furthered in Stein et al (2016) who evaluated potential impact from adverse learning environment. In contrast, to the findings of CCCEH, Mt. Sinai, and CHAMACOS, the HOME study (Donauer et al, 2016) reported evidence of a significant positive association between maternal levels of urinary metabolites (total DAP and total DM) and verbal IQ of the child from the WPPSI was observed in both the unadjusted and adjusted models. Although evidence of a positive association was observed for total DAP and total DM metabolites and full-scale IQ (unadjusted model: β = 0.004, p = 0.111; β = 0.004, p = 0.124, when the model was adjusted, no evidence of a positive association was observed (adjusted model: β = 0.001, p = 0.494; β = 0.002, p = 0.441).  

As stated in the EPA neurotoxicity guidelines8F, direct extrapolation of developmental neurotoxicity results from laboratory animals to humans is limited by the lack of knowledge about underlying toxicological mechanisms and the relevance of these results to humans.  EPA notes consistencies across the databases of in vivo laboratory animal studies and epidemiology studies, although challenges of making a direct comparison between neurodevelopmental domain inter-species remain. It can be assumed that developmental neurotoxicity effects in animal studies indicate the potential for altered neurobehavioral development in humans, although the specific types of developmental effects seen in experimental animal studies may not be the same as those that may be produced in humans.  However, considering the toxicological and epidemiological data in the context of three major neurodevelopmental domains (specifically, cognition, motor control, and social behavior), insights can be gained.  Previously reviewed studies of chlorpyrifos in rats and/or mice reported impaired cognition, changes in locomotor activity levels, altered social interaction, and to a lesser extent, changes in neuromotor function (FIFRA SAP 2012; USEPA, 2014).  While there are fewer studies for all the other OPs, behavioral effects in the same functional domains were again reported.  The most commonly reported outcome in the laboratory animal studies was cognitive dysfunction, and although it was overall consistent there were again differences in cognitive specificity, gender differences, or dose response.  Quite a few studies also report changes in motor activity and sensory function in offspring, but there generally fewer laboratory animal studies that assess social interactions for OPs other than chlorpyrifos.  It is notable that the laboratory animal studies vary in experimental designs such as species, strain, gender, dosing regimens (age, routes, vehicle), and test parameters (age, protocol).  Likewise, observational epidemiology studies vary by population characteristics (race/ethnicity, SES, and pesticide use/exposure profile), co-exposures (mix of chemicals, windows of exposure), and method of exposure and outcome assessment. Given the differences across laboratory animal and epidemiology studies, the qualitative similarity in research findings is striking.

In contrast, quantitatively, there are notable differences between animals and humans.  Specifically, in animals, the doses most often used in these studies are sufficient to elicit >=10% brain and RBC inhibition depending on the study design, age of the animal, and sampling time.  In the epidemiology studies, based on the comparisons with biomonitoring data, reported AChE data from CHAMACOS and the results of the chlorpyrifos dose-reconstruction analysis, it is unlikely that RBC AChE would have been inhibited by any meaningful or measurable amount, if any at all, and most likely none in the brain.  This key difference in dose response between the experimental toxicology and epidemiology studies poses challenges in interpreting such data.  There are a number of possible hypotheses such as:  1) limitations of experimental laboratory studies which have limited statistical power due to relatively small sample sizes; 2) humans display a broader array of behaviors and cognitive abilities than rats, thus limiting the sensitivity of the rat studies; and 3) in the epidemiology studies, the timing of OP application and blood collections are not coupled -- thus higher levels of blood OPs were likely missed.

3.2.3	Specificity
In considering the FQPA 10X Safety Factor, the statute indicates that both pre- and post-natal toxicity and exposure be considered.  As such, it is appropriate to consider the degree specificity of with regard to lifestage.  There are numerous animal studies in the literature which vary in their study design but all involve gestational and/or early postnatal dosing with behavioral evaluation from adolescence to adulthood.  The data provide support for the susceptibility of the developing mammalian brain to chlorpyrifos exposure through gestation and early in life, with adverse outcomes in several neurological domains including cognitive, anxiety and emotion, social interactions, and neuromotor function.  The studies have not shown that any specific developmental period is critical overall to the long-term outcomes, since similar effects are observed with different exposure periods.  For example, cognitive changes in one laboratory using a radial arm maze were observed following gestational and early, but not late, postnatal exposure (Aldridge et al., 2005; Icenogle et al., 2004; Levin et al., 2002; Levin et al., 2001), whereas other laboratories cognitive deficits in a Morris water maze were reported following both gestational and late-postnatal exposure (Billauer-Haimovitch et al., 2009; Turgeman et al., 2011).  Likewise, some changes in anxiety and social behaviors were reported at both gestational and postnatal exposure periods.  Overall, these data do not clearly show specific critical periods of exposure but support a conclusion that early life (pre- and post-natal) represent susceptible lifestages.  

As discussed above, numerous epidemiological investigations have observed a link between prenatal exposure to chlorpyrifos or OPs (measured as chlorpyrifos, TCPy, or DAPs) and adverse effects on neurodevelopment through age seven years, with additional more limited evidence up through approximately age eleven.  As noted previously, for these epidemiology studies chlorpyrifos was only assessed directly in the CCCEH study, with the Mexican cohort study (Fortenberry et al., 2014) assessing the chlorpyrifos metabolite TCPy, and the CHAMACOS, HOME, and Mt. Sinai cohorts focusing on the DAPs.  

The majority of epidemiological studies investigated only prenatal exposures.  Specifically, with respect to biomarkers representing prenatal exposures, CCCEH, Mt. Sinai, and CHAMACOS each reported evidence of impaired mental and psychomotor development, albeit not consistent by age at time of testing (ranging from 6 month to 36 months across the three cohorts).  Statistically significant or suggestive associations between chlorpyrifos or DAPs exposure and attentional problems/ADHD were reported by multiple prospective cohorts (Rauh et al., 2006; Eskenazi et al., 2007; Marks et al., 2010; and Fortenberry et al., 2014) with additional support from a case control study, Bouchard et al. (2010).  In addition, the three US cohorts and the CHARGE study have reported suggestive or positive associations between OP exposure and autism spectrum disorders (Rauh et al., 2006; Shelton et al., 2014; Eskenazi et al., 2007; Furlong et al., 2014).  While the studies from CCCEH, Mt. Sinai, and CHAMACOS studies vary in the magnitude of the overall strength of association, they have consistently observed a positive association between OP exposure and autism spectrum disorder.  Several studies have also documented statistically significant association between prenatal DAP exposure and abnormal motor development in neonates (reflexes, Brazelton score or similar measure; Young et al., 2005; Engel et al., 2007; Zhang et al., 2014).  Finally, CCCEH, Mt. Sinai and CHAMACOS have reported an inverse relation between the respective prenatal measures of chlorpyrifos or DAPs and intelligence measures at age 7 years (Rauh et al., 2011; Engel et al., 2011; Bouchard et al., 2011). 

With regards to effects pre-natal compared to post-natal, a small number of studies have assessed postnatal exposure to DAPs.  Postnatal exposure to DAPs has been assessed in the CHAMACOS cohort (Eskenazi et al., 2007; Young et al., 2005; Bouchard et al., 2011) and three cross-sectional studies (Guodong et al., 2012; Bouchard et al., 2010; Oulhote and Bouchard, 2013).  With the exception of Bouchard et al. (2010) and findings of PDD increase in post-natal DAP from CHAMACOS (Eskenazi et al., 2007; Marks et al, 2010), no adverse neurodevelopmental associations were found between postnatal urinary metabolite levels and any of the developmental outcomes.  Bouchard et al. (2010) looked at U.S. children age 8 - 15 years in the 2000 - 2004 National Health and Nutrition Examination Survey (NHANES), and observed a positive association between attention and behavior problems and total DAPs and DMAPs, but not DEAPs. Postnatal exposures have not been assessed in the CCCEH and Mt. Sinai studies (Rauh et al., 2011; Engel et al., 2011); as such, there are no studies included in this analysis which directly assessed the potential for postnatal chlorpyrifos exposure and associations with neurodevelopmental effects.  However, given that the major source of exposure (residential use) was cancelled partway through the CCCEH study which substantially reduced and largely removed chlorpyrifos from the home environment, this limits the ability of the CCCEH study to inform the impacts of long-term postnatal exposure to chlorpyrifos on neurodevelopment from the current uses of chlorpyrifos. 

In sum, given that the extensive experimental laboratory animal database suggests that the post-natal period is a potential susceptible time, the lack of postnatal exposure assessment in the CCCEH and Mt. Sinai studies is a source of uncertainty in the epidemiology database 

3.3	Biological Plausability & Coherence

EPA's cancer guidelines (2005) includes guidance which are also applicable to this current evaluation of OPs.  In fact, the Guidelines indicate:

"evaluation of the biological plausibility of the associations observed in epidemiologic studies reflects consideration of both exposure-related factors and toxicological evidence relevant to identification of potential modes of action (MOAs). Similarly, consideration of the coherence of health effects associations reported in the epidemiologic literature reflects broad consideration of information pertaining to the nature of the biological markers evaluated in toxicologic and epidemiologic studies. [p. 39]."

The Cancer Guidelines further state that "lack of mechanistic data, however, is not a reason to reject causality [p. 41]."  

At this time, a MOA(s)/AOP(s) has/have not been established for neurodevelopmental outcomes.  This growing body of literature does demonstrate, however, that OPs are biologically active on a number of processes that affect the developing brain.  Multiple in vitro studies on endpoints relevant to the developing brain have identified outcomes in picomole concentrations, including concentrations lower than those that elicit AChE inhibition in vitro.

      ::	1 pM chlorpyrifos oxon decreased axon length (~50%) in superior cervical ganglion cell cultures (AChE inhibition starting at 1000pM in same cell system): Howard et al., 2005 
      ::	0.03 pM oxon (30 fM) increased in CREB levels (~50%) primary cortical neuron cultures (AChE inhibition starting at 100 pM in same cell system): Schuh et al., 2002 
      ::	10 pM oxon decreased axon length (~40%) in dorsal root ganglion cell cultures (AChE inhibition starting at 100 pM in same cell system): Yang et al., 2008

With regard to coherence, there is a large body of in vivo laboratory studies which show long-term behavioral effects from early life exposure, albeit at doses which cause AChE inhibition.  EPA considers the results of the toxicological studies relevant to the human population, as qualitatively supported by the results of epidemiology studies. The lack of established MOA/AOP pathway does not undermine or reduce the confidence in the findings of the epidemiology studies.  When all the evidence is considered together, there are uncertainties with respect to a number of factors such as exposure assessment, lack of ability to make strong causal linkages, and unknown window(s) of susceptibility.  The epidemiology studies reviewed in the 2012/2014 and 2015/2016 literature reviews represent different investigators, locations, points in time, exposure assessment procedures, and outcome measurements.  Despite all these uncertainties and differences in study design, multiple investigators have identified associations with neurodevelopmental outcomes such as ADHD/behavioral problems and autism spectrum, in relation to OP exposure. There is evidence of delays in mental development in infants (24-36 months), attention problems and autism spectrum disorder in early childhood, and intelligence decrements in school age children who were exposed to OPs during gestation. Investigators reported strong measures of statistical association across several of these evaluations (odds ratios 2-4 fold increased in some instances), and observed evidence of exposures-response trends in some instances, e.g., intelligence measures. Thus, with respect to biological plausibility and coherence, although uncertainties remain, these uncertainties are diminished in the context of the qualitative similarity between the epidemiology studies.

4.0	10X FQPA Safety Factor for Infants and Children

As section 408(b)(2)(C) of the FFDCA instructs EPA, in making its "reasonable certainty of no harm" finding, that in "the case of threshold effects, an additional tenfold margin of safety for the pesticide chemical residue and other sources of exposure shall be applied for infants and children to take into account potential pre- and postnatal toxicity and completeness of data with respect to exposure and toxicity to infants and children." Section 408 (b)(2)(C) further states that "the Administrator may use a different margin of safety for the pesticide chemical residue only if, on the basis of reliable data, such margin will be safe for infants and children."  Given the totality of the evidence, there is sufficient uncertainty in the human dose-response relationship for neurodevelopmental effects which prevents the agency from reducing or removing the statutory 10X FQPA Safety Factor.  For the human health risk assessments for the OPs, a value of 10X will be applied.  Similarly, a database uncertainty factor of 10X will be retained for occupational risk assessments.  The agency will continue to evaluate the epidemiology studies and pursue approaches for quantitative or semi-quantitative comparisons between doses which elicit AChE inhibition and those which are associated with neurodevelopmental outcomes prior to a revised human health risk assessment.  The FQPA 10X Safety Factor or database uncertainty factor will be retained for OPs for the population subgroups that include infants, children, youths, and women of childbearing age for all exposure scenarios.

5.0	References

      Acosta-Maldonado, Brenda, Sánchez-Ramírez, Blanca, Reza-López, Sandra, and Levario-Carrillo, Margarita. Effects of exposure to pesticides during pregnancy on placental maturity and weight of newborns: A cross-sectional pilot study in women from the Chihuahua State, Mexico. Human and Experimental Toxicology 28(8) 451 - 459, 2009.
      Aldridge, J. E., Levin, E. D., Seidler, F. J., & Slotkin, T. A. (2005). Developmental exposure of rats to chlorpyrifos leads to behavioral alterations in adulthood, involving serotonergic mechanisms and resembling animal models of depression. Environ Health Perspect, 113(5), 527 - 531.
      al-Hachim, G.M., Fink, G.B., 1968a. Effect of DDT or parathion on open-field behavior of offspring from DDT- or parathion-treated mothers. Psychol Rep 22, 1193-1196.
      al-Hachim, G.M., Fink, G.B., 1968b. Effect of DDT or parathion on condition avoidance response of offspring from DDT or parathion-treated mothers. Psychopharmacologia 12, 424-427.
      Andersen, Helle R., Debes, Fróði, Wohlfahrt-Veje, Christine, Murata, Katsuyuki, Grandjean, Philippe. Occupational pesticide exposure in early pregnancy associated with sex-specific neurobehavioral deficits in the children at school age. Neurotoxicology and Teratology 47:1-9, 2015.
      Ankley, G. T., Bennett, R. S., Erickson, R. J., Hoff, D. J., Hornung, M. W., Johnson, R. D., et al. (2010). Adverse outcome pathways: a conceptual framework to support ecotoxicology research and risk assessment. Environ Toxicol Chem, 29(3), 730-741.
      Barr DB, Ananth CV, Yan X, Lashley S, Smulian JC, Ledoux TA, Hore P, Robson MG. Pesticide concentrations in maternal and umbilical cord sera and their relation to birth outcomes in a population of pregnant women and newborns in New Jersey. Sci Total Environ. 2010;408:790 - 795. 
      Berkowitz GS, Wetmur JG, Birman-Deych E, Obel J, Lapinski RH, Godbold JH, Holzman IR, Wolff MS. In Utero Pesticide Exposure, Maternal Paraoxonase Activity, and Head Circumference. Environmental Health Perspectives. 2004;112:388 - 391.
      Billauer-Haimovitch, H., Slotkin, T. A., Dotan, S., Langford, R., Pinkas, A., & Yanai, J. (2009). Reversal of chlorpyrifos neurobehavioral teratogenicity in mice by nicotine administration and neural stem cell transplantation. Behav Brain Res, 205(2), 499 - 504.
      Boobis, A. R., Cohen, S. M., Dellarco, V., McGregor, D., Meek, M. E., Vickers, C., et al. (2006). IPCS framework for analyzing the relevance of a cancer mode of action for humans. Crit Rev Toxicol, 36(10), 781-792.
      Boobis, A. R., Doe, J. E., Heinrich-Hirsch, B., Meek, M. E., Munn, S., Ruchirawat, M., et al. (2008). IPCS framework for analyzing the relevance of a noncancer mode of action for humans. Crit Rev Toxicol, 38(2), 87-96.
      Bouchard M.F., Bellinger D.C., Wright R.O., Weisskopf M.G., (2010). Attention-Deficit/Hyperactivity Disorder and Urinary Metabolites of Organophosphate Pesticides. PEDIATRICSDOI:10.1542/peds.2009-3058.
      Bouchard, M. F., Chevrier, J., Harley, K. G., Kogut, K., Vedar, M., Calderon, N., et al. (2011). Prenatal exposure to organophosphate pesticides and IQ in 7-year-old children. Environ Health Perspect, 119(8), 1189-1195.
      Braquenier J-B, Quertemont E, Tirelli E, Plumier J-C. Anxiety in adult female mice following perinatal exposure to chlorpyrifos. 2010. Neurotoxicol Teratol. 32:234-239, 2010
      Carr RL, Armstrong NH, Buchanan AT, Eells JB, Mohammed AN, Ross MK, Nail CA. Decreased anxiety in juvenile rats following exposure to low levels of chlorpyrifos during development. 2015. Neurotoxicol. 2015, http://dx.doi.org/10.1016/j.neuro.2015.11.016,
      Cartier, C., Warembourg, C., Le Maner-Idrissi, G., Lacroix, A., Rouget, F., Monfort, C., ... & Chevrier, C. (2016). Organophosphate Insecticide Metabolites in Prenatal and Childhood Urine Samples and Intelligence Scores at 6 Years of Age: Results from the Mother - Child PELAGIE Cohort (France). Environmental health perspectives, 124(5), 674.
      Ceh, K., Jezek, D., Zuzek, M.C., Cebulj-Kadunc, N., Majdic, G., 2012. Effects of organophosphate chlormephos on development and function of testes and brain. Slovenian Veterinary Research 49, 41-49.
      Chen L, Zhao T, Pan C, Ross JH, Krieger RI.  (2012) Preformed biomarkers including dialkylphosphates (DAPs) in produce may confound biomonitoring in pesticide exposure and risk assessment. J Agric Food Chem. 2012 Sep 12;60(36):9342-51. doi: 10.1021/jf303116p. Epub 2012 Sep 4.
      Clemens, G.R., Hartnagel, R.E., Bare, J.J., Thyssen, J.H., 1990. Teratological, neurochemical, and postnatal neurobehavioral assessment of METASYSTOX-R, an organophosphate pesticide in the rat. Fundam Appl Toxicol 14, 131-143.
      Crowder, L.A., Lanzaro, G.C., Whitson, R.S., 1980. Behavioral effects of methyl parathion and toxaphene exposure in rats. J Environ Sci Health B 15, 365-378.
      Dąbrowski, Sławomir, Hanke, Wojciech, Polańska, Kinga, Makowiec-Dąbrowska, Teresa, and Sobala, Wojciech. Pesticide exposure and birthweight: An epidemiological study in Central Poland. International Journal of Occupational Medicine and Environmental Health 16(1): 31-39, 2003.
      de Castro, V.L., Chiorato, S.H., Pinto, N.F., 2001. Relevance of developmental testing of exposure to methamidophos during gestation to its toxicology evaluation. Toxicol Lett 118, 93-102.
      Driver JH, Li Y, Ross JH, Krieger RI.  Dialkylphosphates (DAPs) in fruits and vegetables may confound biomonitoring in organophosphorus insecticide exposure and risk assessment.   J Agric Food Chem. 2008 Nov 26;56(22):10638-45. doi: 10.1021/jf8018084. 1,
      
      Donauer, S., Altaye, M., Xu, Y., Sucharew, H., Succop, P., Calafat, A. M., ... & Yolton, K. (2016). An Observational Study to Evaluate Associations Between Low-Level Gestational Exposure to Organophosphate Pesticides and Cognition During Early Childhood. American Journal of Epidemiology, 184(5), 410-418.
      Engel SM, Berkowitz GS, Barr DB, Teitelbaum SL, Siskind J, Meisel SJ, Wetmur JG, Wolff MS. Prenatal Organophosphate Metabolite and Organochlorine Levels and Performance on the Brazelton Neonatal Behavioral Assessment Scale in a Multiethnic Pregnancy Cohort. American Journal of Epidemiology. 2007;165:1397-1404.
      Engel SM, Wetmur J, Chen J, Zhu C, Barr DB, Canfield RL, Wolff MS. Prenatal Exposure to Organophosphates, Paraoxonase 1, and Cognitive Development in Childhood. Environmental Health Perspectives. 2011;119:1182-1188. 
      Engel SM, Bradman A, Wolff MS, Rauh VA, Harley KG, Yang JH, Hoepner LA, Barr DB, Yolton K, Vedar MG, Xu Y, Hornung RW, Wetmur JG, Chen J, Holland NT, Perera FP, Whyatt RM, Lanphear BP, Eskenazi B. 2015. Prenatal organophosphorus pesticide exposure and child neurodevelopment at 24 months: an analysis of four birth cohorts. Environ Health Perspect 124:822-830;  http://dx.doi.org/10.1289/ehp.1409474.
      Eskenazi B, Harley K, Bradman A, Weltzien E, Jewell NP, Barr DB, Furlong CE, Holland NT. Association of in Utero Organophosphate Pesticide Exposure and Fetal Growth and Length of Gestation in an Agricultural Population. Environmental Health Perspectives. 2004;115:792 - 798.
      Eskenazi B, Marks AR, Bradman A, Harley K, Barr DB, Johnson C, Morga N, Jewell NP. Organophosphate Pesticide Exposure and Neurodevelopment in Young Mexican-American Children. Environmental Health Perspectives. 2007;115:792 - 798.
      Eskenazi, B., Huen, K., Marks, A., Harley, K. G., Bradman, A., Barr, D. B., & Holland, N. (2010). PON1 and neurodevelopment in children from the CHAMACOS study exposed to organophosphate pesticides in utero. Environ Health Perspect, 118(12), 1775-1781.
      Fiedler, N., Rohitrattana, J., Siriwong, W., Suttiwan, P., Strickland, P. O., Ryan, P. B., ... & Robson, M. G. (2015). Neurobehavioral effects of exposure to organophosphates and pyrethroid pesticides among Thai children. Neurotoxicology, 48, 90-99.
      FIFRA Scientific Advisory Panel. (2008b). "The Agency's Evaluation of the Toxicity Profile of Chlorpyrifos." Report from the FIFRA Scientific Advisory Panel Meeting of September, 2008. FIFRA Scientific Advisory Panel, Office of Science Coordination and Policy, Office of Prevention, Pesticides and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. Available: http://www.epa.gov/scipoly/sap/meetings/2008/index.html
      FIFRA Scientific Advisory Panel. (2010). February 2 - 4, 2010: Incorporation of Epidemiology and Human Incident Data into Human Risk Assessment.
      FIFRA Scientific Advisory Panel. (2012). Transmittal of the Meeting Minutes from the FIFRA Scientific Advisory Panel Meeting of April 10-12, 2012. FIFRA Scientific Advisory Panel, Office of Science Coordination and Policy, Office of Prevention, Pesticides and Toxic Substances, U.S. Environmental Protection Agency. Washington, DC. Available: http://www.epa.gov/scipoly/sap/meetings/2012/041012meeting.html
      Fortenberry G.Z., Meeker J.D., Sanchez B.N., et al., (2014). Urinary 3,5,6-tichloro-2-pyridinol (TCPY) in pregnant women from Mexico City: Distribution, temporal variability, and relationship with child attention and hyperactivity. International Journal of Hygiene and Environmental Health 217 (2014) 405-412.
      Fortenberry, GZ, Meeker, JD, Sánchez, BN, Barr, DB, Peterson, K, Bellinger, D, Schnaas, L, Solano-González, M, Ettinger, AS, Hernandez-Avila, M, Hu, H, Tellez-Rojoj, MM. Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. (2014a.) Environmental Research 132:342 - 349.
      Furlong, Melissa A., Engel, Stephanie M., Boyd Barr, Dana, Wolff, Mary S. Prenatal exposure to organophosphate pesticides and reciprocal social behavior in childhood. 2014. Environment International 70:125-131.
      Grandjean, Philippe, Harari, Raul, Barr, Dana B., Debes, Frodi. Pesticide exposure and stunting as independent predictors of neurobehavioral deficits in Ecuadorian school children. 2006. Pediatrics 117(3):e546-e556.
      Guodong D., Pei W., Ying T., Jun Z., et al., (2012). Organophosphate Pesticide Exposure and Neurodevelopment in Young Shanghai Children. Environ Sci. Technol. 2012, 46, 2911-2917. 
      Gupta, R.C., Rech, R.H., Lovell, K.L., Welsch, F., Thornburg, J.E., 1985. Brain cholinergic, behavioral, and morphological development in rats exposed in utero to methylparathion. Toxicol Appl Pharmacol 77, 405-413.
      Handal A.J., Lozoff B., Breilh J., Harlow S.D., (2007). Effect of community of residence on neurobehavioral development in infants and young children in a flower-growing region of Ecuador. Environ Health Perspect. 2007 Jan;115(1): 128-33.
      Handal A.J., Lozoff B., Breilh J., Harlow S.D., (2007B). Neurobehavioral Development in Children with Potential Exposure to Pesticides. Epidemiology May 2007; 18: 312-320.
      Handal A.J., Lozoff B., Breilh J., Harlow S.D., (2008). Occupational Exposure to Pesticides During Pregnancy and Neurobehavioral Development of Infants and Toddlers. Epidemiology, Volume 19, number 6, November 2008.
      Harari, Raul, Julvez, Jordi, Murata, Katsuyuki, Barr, Dana, Bellinger, David C., Debes, Frodi, and Grandjean, Philippe. Neurobehavioral Deficits and Increased Blood Pressure in School-Age Children Prenatally Exposed to Pesticides. 2010. Environmental Health Perspectives 118(6): 890-896.
      Harley KG, Huen K, Schall RA, Holland NT, Bradman A, et al. Association of Organophosphate Pesticide Exposure and Paraoxonase with Birth Outcome in Mexican-American Women. PLoS ONE 2011; 6(8)1-10. 
      Harley KG, Engel SM, Vedar MG, Eskenazi B, Whyatt RM, Lanphear BP, Bradman A, Rauh VA, Yolton K, Hornung RW, Wetmur JG, Chen J, Holland NT, Barr DB, Perera FP, Wolff MS. 2016. Prenatal exposure to organophosphorous pesticides and fetal growth: pooled results from four longitudinal birth cohort studies. Environ Health Perspect 124:1084-1092; http://dx.doi.org/10.1289/ehp.1409362.
      Hill AB (1965). The Environment and Disease: Association or Causation? Proc R Soc Med. May 1965; 58(5): 295 - 300.
      Howard, A. S., Bucelli, R., Jett, D. A., Bruun, D., Yang, D., & Lein, P. J. 2005. Chlorpyrifos exerts opposing effects on axonal and dendritic growth in primary neuronal cultures. Toxicol Appl Pharmacol, 207(2), 112-124.
      Icenogle, L. M., Christopher, N. C., Blackwelder, W. P., Caldwell, D. P., Qiao, D., Seidler, F. J., et al. (2004). Behavioral alterations in adolescent and adult rats caused by a brief subtoxic exposure to chlorpyrifos during neurulation. Neurotoxicol Teratol, 26(1), 95 - 101.
      Johnson, F.O., Chambers, J.E., Nail, C.A., Givaruangsawat, S., Carr, R.L., 2009. Developmental chlorpyrifos and methyl parathion exposure alters radial-arm maze performance in juvenile and adult rats. Toxicol Sci 109, 132-142.
      Kofman, Ora, Berger, Andrea, Massarwa, Ali, Friedman, Alon, and Abu Jaffar, Abed. Motor Inhibition and Learning Impairments in School-Aged Children Following Exposure to Organophosphate Pesticides in Infancy. Pediatric Research. 60:88-92, 2006.
      Koutroulakis, D., Sifakis, S., Tzatzarakis, M.N., Alegakis, A.K., Theodoropoulou, E., Kavvalakis, M.P., Kappou, D., Tsatsakis, A.M. Dialkyl phosphates in amniotic fluid as a biomarker of fetal exposure to organophosphates in Crete, Greece; association with fetal growth. Reproductive Toxicology 46 (2014) 98 - 105.
      Lazarini, C.A., Lima, R.Y., Guedes, A.P., Bernardi, M.M., 2004. Prenatal exposure to dichlorvos: physical and behavioral effects on rat offspring. Neurotoxicol Teratol 26, 607-614.
      Lehotzky, K., Szeberenyi, M.J., Kiss, A., 1989. Behavioral consequences of prenatal exposure to the organophosphate insecticide sumithion. Neurotoxicol Teratol 11, 321-324.
      Levin, E. D., Addy, N., Nakajima, A., Christopher, N. C., Seidler, F. J., & Slotkin, T. A. (2001). Persistent behavioral consequences of neonatal chlorpyrifos exposure in rats. Brain Res Dev Brain Res, 130(1), 83 - 89.
      Levin, E. D., Addy, N., Baruah, A., Elias, A., Christopher, N. C., Seidler, F. J., et al. (2002). Prenatal chlorpyrifos exposure in rats causes persistent behavioral alterations. Neurotoxicol Teratol, 24(6), 733 - 741.
      Levin, E.D., Timofeeva, O.A., Yang, L., Petro, A., Ryde, I.T., Wrench, N., Seidler, F.J., Slotkin, T.A., 2009. Early postnatal parathion exposure in rats causes sex-selective cognitive impairment and neurotransmitter defects which emerge in aging. Behav Brain Res 208, 319-327.
      Lima, C.S., Dutra-Tavares, A.C., Nunes, F., Nunes-Freitas, A.L., Ribeiro-Carvalho, A., Filgueiras, C.C., Manhaes, A.C., Meyer, A., Abreu-Villaca, Y., 2013. Methamidophos exposure during the early postnatal period of mice: immediate and late-emergent effects on the cholinergic and serotonergic systems and behavior. Toxicol Sci 134, 125-139.
      Lizardi, Patricia Sanchez, O'Rourke, Mary Kay, and Morris, Richard J. The effects of organophosphate pesticide exposure on Hispanic children's cognitive and behavioral functioning. Journal of Pediatric Psychology 33(1): 91 - 101, 2008.
      Llop S., Julvez J., Marina L.S., Vizcaino E., et al., (2013).  Prenatal and postnatal insecticide use and infant neuropsychological development in a multicenter birth cohort study. Environment International 59(2013) 175-182.
      Lovasi GS, Quinn JW, Rauh VA, Perera FP, Andrews HF, Garfinkel R, Hoepner L, Whyatt R, Rundle A. Chlorpyrifos Exposure and Urban Residential Environment Characteristics as Determinants of Early Childhood Neurodevelopment. American Journal of Public Health. 2011;101(1):63-70. doi: 10.2105/AJPH.2009.168419. Epub 2010 Mar 18.
      Lu, et al. Assessing the association between pesticide exposure and cognitive development in rural Costa Rican children living in organic and conventional coffee farms. Int J Adolesc Med Health. 2009; 21(4): 609-621
      Mamczarz J, Pescrille JD, Gavrushenko L, Burke RD, Fawcett WP, DeTolla Jr, LJ, Chen H, Pereira EFR, Albuquerque EX. 2016. Spatial learning impairment in prepubertal guinea pigs prenatally exposed to the organophosphorus pesticide chlorpyrifos: Toxicological implications. Neurotoxicol. 56:17-28, 2016
      Marks AR, Harley K, Bradman A, Kogut K, Barr DB, Johnson C, et al. Organophosphate Pesticide Exposure and Attention in Young Mexican-American Children: The CHAMACOS Study. Environmental Health Perspectives. 2010;118:1768-1774.
      Meek ME, Boobis A, Cote I, Dellarco V, Fotakis G, Munn S, Seed J, and Vickers, C.  2014. New developments in the evolution and application of the WHO/IPCS framework on mode of action/species concordance analysis.  J Appl Toxicol.  Jan;34(1):1-18.
      Meek ME, Boobis A, Cote I, Dellarco V, Fotakis G, Munn S, Seed J, Vickers C.  2014. New developments in the evolution and application of the WHO/IPCS framework on mode of action/species concordance analysis.  J Appl Toxicol. 2014 Jan;34(1):1-18.
      Moreno-Banda, G., Blanco-Muñoz, J., Lacasaña, M., Rothenberg, S.J., Aguilar-Garduño, C., Gamboa, R., Pérez-Méndez, O. Maternal exposure to floricultural work during pregnancy, PON1 Q192R polymorphisms and the risk of low birth weight. Science of the Total Environment 407: 5478 - 5485, 2009.
      Muñoz-Quezada MT, Lucero BA, Barr DB, Steenland K, Levy K, Ryan PB, Veronica Iglesiase V, Alvaradoe S, Conchaf C, Rojas E, Catalina Vega C.  Neurodevelopmental effects in children associated with exposure to organophosphate pesticides: A systematic review.  Neurotoxicology 39: 158-168.  
      Naksen W, Prapamontol T, Mangklabruks A, Chantara S, Thavornyutikarn P, Srinual N, Panuwet P, Ryan PB, Riederer AM, Barr DB. Associations of maternal organophosphate pesticide exposure and PON1 activity with birth outcomes in SAWASDEE birth cohort, Thailand. Environmental Research142: 288 - 296, 2015.
      Nevison. A comparison of temporal trends in United States autism prevalence to trends in suspected environmental factors. Environmental Health. 2014; 13(73): 16 pp  
      Oulhote, Youssef and Bouchard, Maryse F. (2013) Urinary Metabolites of Organophosphate and Pyrethroid Pesticides and Behavioral Problems in Canadian Children. Environmental Health Perspectives. 121:1378-1384   
      Petit, Claire, Chevrier, Cécile, Durand, Gaël, Monfort, Christine, Rouget, Florence, Garlantezec, Ronan, Cordier, Sylvaine. Impact on fetal growth of prenatal exposure to pesticides due to agricultural activities: a prospective cohort study in Brittany, France. Environmental Health 9:71-83, 2010.
      Quiros-Alcala L., Alkom A.D., Boyce W.T., Lippert S., et al., (2011). Maternal prenatal and child organophosphate pesticide exposures and children's autonomic function. NeuroToxicology 32 (2011) 646-655.
      Racke J.D. Degradation of organophosphorous insecticidesin environmental matrices. In: Organophosphates Chemistry, Fate and Effects. Academic Press, Inc., San Diego, CA, 1992, pp 47 - 72.
      Raanan R, Harley KG, Balmes JR, Bradman A, Lipsett M, and Eskenazi B. (2015) Early-life Exposure to Organophosphate Pesticides and Pediatric Respiratory Symptoms in the CHAMACOS Cohort. Environmental Health Perspectives: 123(2), 179-185.
      Rauch, Stephen A., Braun, Joe M., Boyd Barr, Dana, Calafat, Antonia M., Khoury, Jane, M., Montesano, Angela, Yolton, Kimberly, and Lanphear, Bruce P. Associations of Prenatal Exposure to Organophosphate Pesticide Metabolites with Gestational Age and Birth Weight. Environmental Health Perspectives. 120:1055 - 1060 (2012).
      Rauh VA, Garfinkel R, Perera FP, Andrews HF, Hoepner L, Barr DB, Whitehead R, Tang D, Whyatt RW. Impact of Prenatal Chlorpyrifos Exposure on Neurodevelopment in the First 3 Years of Life Among Inner-City Children. Pediatrics. 2006;18(6): 1835-1859.
      Rauh V, Arunajadai S, Horton M, Perera F, Hoepner L, Barr DB, et al. 2011. Seven-Year Neurodevelopmental Scores and Prenatal Exposure to Chlorpyrifos, a Common Agricultural Pesticide. Environmental Health Perspectives. 2011;119:1196-1201.
      Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., . . . Peterson, B. S. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. Proc Natl Acad Sci U S A, 109(20), 7871-7876. 
      Rauh VA.; Garcia WE.; Whyatt RM.; et al. (2015) Prenatal exposure to the organophosphate pesticide chlorpyrifos and childhood tremor. Neurotoxicology: 51, 80-86.
      Roegge, C.S., Timofeeva, O.A., Seidler, F.J., Slotkin, T.A., Levin, E.D., 2007. Developmental diazinon neurotoxicity in rats: later effects on emotional response. Brain Res Bull 75, 166-172.
      Rohlman, Diane S., Arcury, Thomas A., Quandt, Sara A., Lasarev, Michael, Rothlein, Joan, Travers, Rachelle, Tamulinas, Alys, Scherer, Jennifer, Early, Julie, Marín, Antonio, Phillips, Jacki, McCauley, Linda. Neurobehavioral performance in preschool children from agricultural and non-agricultural communities in Oregon and North Carolina. Neurotoxicology 26:895-598, 2005.
      Ruckart, et al. Long-Term Neurobehavioral Health Effects of Methyl Parathion Exposure in Children in Mississippi and Ohio. Environmental Health Perspectives. 2004; 112(1):46-51.
      Samarawickrema, Nirma, Pathmeswaran, Arunasalam, Wickremasinghe, Rajitha, Peiris-John, Roshini, Karunaratna, Madawa, Buckley, Nick, Dawson, Andrew, and De Silva, Janaka. Fetal effects of environmental exposure of pregnant women to organophosphorus compounds in a rural farming community in Sri Lanka. Clinical Toxicology 46:489 - 495, 2008. 
      Savitz, David A., Arbuckle, Tye, Kaczor, Diane, and Curtis, Kathryn M. Male pesticide exposure and pregnancy outcome. American Journal of Epidemiology 146(12):1025-1036, 1997.
      Schuh RA, Lein PJ, Beckles RA, Jett DA.  2002. Noncholinesterase mechanisms of chlorpyrifos neurotoxicity: altered phosphorylation of Ca2+/cAMP response element binding protein in cultured neurons. Toxicol Appl Pharmacol. 2002 Jul 15;182(2):176-85
      Shelton, Janie F., Geraghty, Estella M., Tancredi, Daniel J., Delwiche, Lora D., Schmidt, Rebecca J., Ritz, Beate, Hansen, Robin L., and Hertz-Picciotto, Irva. Neurodevelopmental disorders and prenatal residential proximity to agricultural pesticides: The CHARGE Study. Environmental Health Perspectives 122:1103 - 1109, 2014.
      Simon TW, Simons SS Jr, Preston RJ, Boobis AR, Cohen SM, Doerrer NG, Fenner-Crisp PA, McMullin TS, McQueen CA, Rowlands JC; RISK21 Dose-Response Subteam.  2014.  The use of mode of action information in risk assessment: Quantitative key events/dose-response framework for modeling the dose-response for key events. Crit Rev Toxicol. 2014 Aug;44 Suppl 3:17-43.
      Slotkin, T.A., Tate, C.A., Ryde, I.T., Levin, E.D., Seidler, F.J. 2006. Organophosphate insecticides target the serotonergic system in developing rat brain regions: Disparate effects of diazinon and parathion at doses spanning the threshold for cholinesterase inhibition.  Environ Health Perspec 114, 1542-1546.
      Sochaski, M.A., McManus, B.M., Struve, M.F., Wallace, D.G., Dorman, D.C.  2007. Inhibition and recovery of maternal and foetal cholinesterase enzymes following fenitrothion administration in CD rats. Xenobiotica 37, 19-29.
      Sonich-Mullin C, Fielder R,  Wiltse J,  et al.  2001.  IPCS conceptual framework for evaluating a mode of action for chemical carcinogenesis.  Regul Toxicol Pharmacol.  34:146-152.
      Spyker, J.M., Avery, D.L., 1977. Neurobehavioral effects of prenatal exposure to the organophosphate Diazinon in mice. J Toxicol Environ Health 3, 989-1002.
      Stamper, C.R., Balduini, W., Murphy, S.D., Costa, L.G., 1988. Behavioral and biochemical effects of postnatal parathion exposure in the rat. Neurotoxicol Teratol 10, 261-266.
      Stein, L. J., Gunier, R. B., Harley, K., Kogut, K., Bradman, A., & Eskenazi, B. (2016). Early childhood adversity potentiates the adverse association between prenatal organophosphate pesticide exposure and child IQ: The CHAMACOS cohort. NeuroToxicology, 56, 180-187.
      Suarez-Lopez J. R., Jacobs D.R., Himes J.H., Alexander B.H., Lazovich D., et al., (2012). Lower acetylcholinesterase activity among children living with flower plantation workers. Environmental Research 114 (2012) 53-59.
      Suarez-Lopez J. R., Jacobs D.R., Himes J.H., Alexander B.H., (2013). Aectylcholinesterase activity and neurodevelopment in boys and girls. PEDIATRICS Volume 132, number 6, December 2013.
      Suarez-Lopez J. R., Jacobs D.R., Himes J.H., Alexander B.H., (2013a). Acetycholinesterase activity, cohabitation with floricultural workers, and blood pressure in Ecuadorian Children. Environmental Health Perspectives, volume 121, number 5, May 2013.
      Timofeeva, O.A., Roegge, C.S., Seidler, F.J., Slotkin, T.A., Levin, E.D., 2008a. Persistent cognitive alterations in rats after early postnatal exposure to low doses of the organophosphate pesticide, diazinon. Neurotoxicol Teratol 30, 38-45.
      Timofeeva, O.A., Sanders, D., Seemann, K., Yang, L., Hermanson, D., Regenbogen, S., Agoos, S., Kallepalli, A., Rastogi, A., Braddy, D., Wells, C., Perraut, C., Seidler, F.J., Slotkin, T.A., Levin, E.D., 2008b. Persistent behavioral alterations in rats neonatally exposed to low doses of the organophosphate pesticide, parathion. Brain Res Bull 77, 404-411.
      Turgeman, G., Pinkas, A., Slotkin, T. A., Tfilin, M., Langford, R., & Yanai, J. (2011). Reversal of chlorpyrifos neurobehavioral teratogenicity in mice by allographic transplantation of adult subventricular zone-derived neural stem cells. J Neurosci Res, 89(8), 1185 - 1193.
      U.S. Environmental Protection Agency. (1999). Guidelines for carcinogen risk assessment. Risk Assessment Forum. SAB review draft. Washington, DC: U.S. Environmental Protection Agency.  www.epa.gov/ncea/raf/crasab.htm.
      U.S. Environmental Protection Agency. (2002).  Determination of The Appropriate FQPA Safety Factor(S) In Tolerance Assessment.  Office of Pesticide Programs, U.S. Environmental Protection Agency, Washington, DC, February 28, 2002.
      U.S. Environmental Protection Agency. (2005). Guidelines for Carcinogen Risk Assessment.  Risk Assessment Forum, U.S. Environmental Protection Agency.  Washington, DC.  EPA/630/P-03/001F.  
      U.S. Environmental Protection Agency. (2006). Revised Organophosphorous Pesticide Cumulative Risk Assessment; July 31, 2006. Office of Pesticide Programs, U.S. Environmental Protection Agency. Washington, D.C. Available http://www.epa.gov/pesticides/cumulative/rra-op.
      U.S. Environmental Protection Agency.  (2008). Draft Science Issue Paper: Chlorpyrifos Hazard and Dose Response Characterization. http://www.epa.gov/scipoly/sap/meetings/2008/091608_mtg.htm
      U.S. Environmental Protection Agency. (2010). Draft Framework for Incorporating Human Epidemiologic and Incident Data in Health Risk Assessment, January 7, 2010.
      U.S. Environmental Protection Agency. (2011a). Chlorpyrifos: Preliminary Human Health Risk Assessment for Registration Review. June 30, 2011. D388070.
      U.S. Environmental Protection Agency. (2011b). Chlorpyrifos:  Occupational and Residential Exposure Assessment, EPA Barcode: D388165,. In Office of Pesticide Programs (Ed.). Washington D.C.
      U.S. Environmental Protection Agency.  (2012). Draft Issue Paper: Scientific Issues Concerning Health Effects of Chlorpyrifos. http://www.regulations.gov/#!documentDetail;D=EPA-HQ-OPP-2012-0040-0002 
      U.S. Environmental Protection Agency. (2014). Chlorpyrifos: Revised Human Health Risk Assessment for Registration Review. December 29, 2014. D424485.
      U.S. Environmental Protection Agency. (2015) Literature Review on Neurodevelopment Effects & FQPA Safety Factor Determination for the Organophosphate Pesticides. September 15, 2015.
      U.S. Environmental Protection Agency. 2015b. Swe, K. and Holman, E. Summary Reviews from _2015 Epidemiological Literature Review for Organophosphates. September 15, 2015.
      U.S. EPA (U.S. Environmental Protection Agency).  2015c. Preamble to the Integrated Science Assessments.  National Center for Environmental Assessment, RTP Division, Office of Research and Development, USEPA. https://yosemite.epa.gov/sab/sabproduct.nsf/0/33E1AD305287588F85257D20006BE8CC/$File/ISA_PREAMBLE_FINAL2015.PDF
      U.S. Environmental Protection Agency. 2016. Holman, E. and Swe, K. Summary Reviews for Additional Epidemiological Literature Studies from Prospective Birth Cohort Studies. D432184. March 25, 2016
      U.S. Environmental Protection Agency. 2016b. Aldridge, A. Summary Reviews for Additional Epidemiological Literature Studies from Prospective Birth Cohort Studies. D432184. December XXX, 2016
      U.S. Environmental Protection Agency. 2016c. Framework for Incorporating Human Epidemiologic and Incident Data in Health Risk Assessment, December 28, 2016.
      Vatanparast, J., Naseh, M., Baniasadi, M., Haghdoost-Yazdi, H., 2012. Developmental exposure to chlorpyrifos and diazinon differentially affect passive avoidance performance and nitric oxide synthase-containing neurons in the basolateral complex of the amygdala. Brain Res 1494, 17-27.
      Venerosi A, Tait S, Stecca L, Chiarotti F, DeFelice A, Cometa MF, Volpe MT, Calamandrei G, Ricceri L. Effects of maternal chlorpyrifos diet on social investigation and brain neuroendocrine markers in the offspring  -  a mouse study. Environ. Health 14:32-42, 2015
      Wang, Pei, Tian, Ying, Wang, Xiao-Jin, Gao, Yu, Shi, Rong, Wang, Guo-Quan, Hu, Guo-Hua, Shen, Xiao-Ming. Organophosphate pesticide exposure and perinatal outcomes in Shanghai, China. 4 Environment International 2:100 - 104, 2012.
      Whyatt, R. M., Barr, D. B., Camann, D. E., Kinney, P. L., Barr, J. R., Andrews, H. F., . . . Perera, F. P. (2003). Contemporary-use pesticides in personal air samples during pregnancy and blood samples at delivery among urban minority mothers and newborns. Environ Health Perspect, 111(5), 749-756. 
      Whyatt RM, Rauh V, Barr DB, Camann DE, Andrews HF, Garfinkel R, Hoepner LA, Diaz D, Dietrich J, Reyes A, Tang D, Kinney PL, Perera FP. Prenatal Insecticide Exposures and Birth Weight and Length among an Urban Minority Cohort. Environmental Health Perspectives. 2004;112:1125 - 1132.
      Whyatt RM, Garfinkel R, Hoepner LA, Holmes D, Borjas M, Williams MK, et al. Within- and Between-Home Variability in Indoor-Air Insecticide Levels during Pregnancy among an Inner-City Cohort from New York City. Environmental Health Perspectives 2007;115:383-389.
      Whyatt RM, Garfinkel R, Hoepner LA, Andrews H, Holmes D, Williams MK, et al. A Biomarker Validation Study of Prenatal Chlorpyrifos Exposure within an Inner-City Cohort during Pregnancy. Environmental Health Perspectives. 2009;117:559-567. 
      Wickerham, Erin L., Lozoff, Betsy, Shao, Jie, Kaciroti, Niko, Xia, Yankai, Meeker, John D. Reduced birth weight in relation to pesticide mixtures detected in cord blood of full-term infants. Environment International 47:80 - 85, 2012.
      Win-Shwe, T.T., Nakajima, D., Ahmed, S., Fujimaki, H., 2012. Impairment of novel object recognition in adulthood after neonatal exposure to diazinon. Arch Toxicol 87, 753-762. 
      Wolff, MS, Engel, S, Berkowitz, G, Teitelbaum, S, Siskind, J, Barr, DB, Wetmur, J. Prenatal Pesticide and PCB Exposures and Birth Outcomes. 2007. Pediatric Research 61(2):243-250.
      Yang, D., Howard, A., Bruun, D., Ajua-Alemanj, M., Pickart, C., & Lein, P. J. 2008. Chlorpyrifos and chlorpyrifos-oxon inhibit axonal growth by interfering with the morphogenic activity of acetylcholinesterase. Toxicol Appl Pharmacol, 228(1), 32-41.
      Yolton, K., Xu, Y., Sucharew, H., Succop, P., Altaye, M., Popelar, A., ... & Khoury, J. C. (2013). Impact of low-level gestational exposure to organophosphate pesticides on neurobehavior in early infancy: a prospective study. Environmental health, 12(1), 1.
      Young JG, Eskenazi B, Gladstone EA, Bradman A, Pedersen L, Johnson C, Barr DB, Furlong CE, Holland NT. Association Between In Utero Organophosphate Pesticide Exposure and Abnormal Reflexes in Neonates. NeuroToxicology. 2005;26:199 - 209.
      Zhang Y., Han S., Liang D., Shi X., Wang F., et al., (2014). Prenatal exposure to organophosphate pesticides and neurobehavioral development of neonates: A birth cohort study in Shenyang, China. Plos ONE 9(2):e88491. doi:10.1371/journal.pone.0088491. 
      Zhang X, Driver JH, Li Y, Ross JH, Krieger RI. (2008) Dialkylphosphates (DAPs) in fruits and vegetables may confound biomonitoring in organophosphorus insecticide exposure and risk assessment.  J Agric Food Chem. 2008 Nov 26;56(22):10638-45. doi: 10.1021/jf8018084. 
      
6.0	Appendices

 Table of In Vivo Developmental Neurotoxicity Studies of OPs.
 Summary of Guideline DNT Studies Submitted to the Agency for OPs other than Chlorpyrifos.
 Low Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment
 Table of Systematic Review Analysis:  Initial Search Research & Initial Exclusion Criteria
 Table of Systematic Review Analysis:  Second Tier Exclusion Criteria
   6. 	Plausible hypotheses on MOA/AOP for neurodevelopmental outcomes (Extracted from Section 4.4.3. RHHRA for chlorpyrifos)
Appendix 1.	Table of In Vivo Developmental Neurotoxicity Studies of OPs.
(Effects described are only those measured after weaning.  Bold indicates functional domains that were reported to show treatment effects.)

OP
Study
Species & strain
Dose, route, vehicle
Dosing period
ChE inhibition 
Domain
Age of testing
Outcomes
NOEL, LOEL
Notes & Study Problems
Chlormephos
Ceh et al., 2012
mouse BALB/c
3.5, 0.35 ug/ml in drinking water of dams
~ 0.6, 0.06 mg/kg/d (@ 5ml/d, 30 g)
7 day premating to weaning
No
Anxiety & Emotion
PND70-80
Increased time in closed arms & decreased time in open arms in elevated plus maze, ~0.6 mg/kg/d, M&F 
NOEL=0.35 ug/ml in water, ~0.06 mg/kg/d
Litter not unit of statistical analysis
No pup allocation described but had to have used some littermates
M & F responses appear similar but not statistically compared 
Diazinon
Roegge et al., 2008
rat Sprague Dawley
0.5, 2 mg/kg/d sc DMSO 
PND1-4
Cites Slotkin et al. (2006): 0.5 mg/kg/d produced <10% brain inhibition on PND5, 2 mg/kg/d produced 25-30% brain inhibition 2 hr after dose on PND4, and 10-20% inhibition on PND5
Anxiety & Emotion 
PND52-56
Decreased time in open arms in elevated plus maze, 2 mg/kg/d, M only
No NOEL
LOEL=0.5 mg/kg/d
Pups & dams redistributed daily
No effect in F
Accepts p<0.1 as significant for interactions
No dose-response in feeding or milk preference studies
Abstract misstates sex differences

PND64-67, 78-79
Decreased latency to eat in novelty suppressed but not home-cage feeding, 0.5 & 2 mg/kg/d, M only

PND73-74
Decreased chocolate milk preference, 0.5 mg/kg/d only, M only

PND86-87
No effect on forced swim test

Diazinon
Spyker and Avery, 1977
mouse F2 hybrid (NCTR cross bred)
0.18, 9 mg/kg/d in peanut butter
GD1-birth
No
Sensory

PND38
Increased errors on visual cliff, 0.18 mg/kg/d only, F only
No effect acoustic startle or olfactory responses
No NOEL
LOEL=0.18 mg/kg/d
No pup allocation described but had to have used littermates
Not clear when both sexes tested and/or compared
Statistics not described
Maternal weight gain lowered at both doses
Weight gain of high dose pups decreased
Preweaning testing: decreased contact placing, 0.18 mg/kg/d only
No dose-response for some measures
Twice as many controls as treated
Looks like decreased rotarod endurance PND65, not significant due to high variability

Neuromotor
PND50, 60, 65,75
No effect swimming ability
Increased rod cling endurance, 0.18 & 9 mg/kg/d, sex not specified
Decreased inclined plane performance, 0.18 & 9 mg/kg/d, sex not specified 

Activity

PND75-76
No effect open field

Cognition
PND87
No effect errors in Lashley maze 

Diazinon
Timofeeva et al., 2008
rat Sprague Dawley
0.5, 2 mg/kg/d sc DMSO 
PND1-4
Cites Slotkin et al. (2006): 0.5 mg/kg/d produced <10% brain inhibition on PND5, 2 mg/kg/d produced 25-30% brain inhibition 2 hr after dose on PND4, and 10-20% inhibition on PND5
Activity
PND28-42
No effect in figure-8 chamber, M&F 

No NOEL
LOEL=0.5 mg/kg/d
Littermates of those used in Roegge et al. 2008
Pups & dams redistributed daily
Accepts p<0.1 as significant for interactions
No mention of sex effects in T-maze 
No dose-response for some measures

Sensory
PND77-84
Decreased prepulse inhibition, 0.5 & 2 mg/kg/d, M only

Cognition
PND28-35, 91-126
No effect on T-maze spontaneous alternation, M&F
Increased working memory errors, 0.5 mg/kg/d only, M&F 

Diazinon
Vatanparast et al., 2013
rat Wistar
1 mg/kg/d sc DMSO 
GD15-18 or PND1-4
No
Activity
PND60
Gestational: No effect in open field, M&F 
Postnatal: No effect in open field, M&F
No NOEL
LOEL=1 mg/kg/d
1 M & 1 F per litter but sex not nested within litter in statistics
F only affected with gestational exposure, both sexes affected with postnatal, looks like M more affected
Large effect sizes
Number of dams not mentioned
Discrepancy in text on pup sample sizes

Cognition
PND60-63
Gestational: Decreased latency to cross and increased time spent in dark side on retention trial, no effect acquisition in passive avoidance, F only 
Postnatal: Decreased latency to cross and increased time spent in dark side on retention trial, no effect acquisition in passive avoidance, M&F

Diazinon
Win-Shwe et al., 2013
mouse C3H/HeN
0.5, 5 mg/kg/d sc DMSO
PND8-11
No
Cognition
PND46-49, 81-84
PND46-49: Decreased novel object exploration and discrimination, 0.5 & 5 mg/kg/d
PND81-84: Decreased novel object exploration and discrimination, 5 mg/kg/d only 
No NOEL
LOEL=0.5 mg/kg/d
M only tested
Separate mice at two test times
Litter allocation to dose group not described
Assumes that ChE inhibition reported by Slotkin in rats would be same as in mice
Larger sample size at later age
Dichlorvos
Lazarini et al., 2004
rat Wistar
8 mg/kg po (dilution of technical product); vehicle from formulation
GD6-15
No 
Activity
PND21, "adult"
PND21: Decreased locomotion open field, M only
"Adult": Decreased locomotion and increased immobility, only M tested 
No NOEL
LOEL=8 mg/kg/d
Data analyzed as litter but sex not nested within litter in statistics
No effect on physical and reflex preweaning development
Only M tested as after PND21
Adult age not given

Cognition
"Adult"
Decreased latency to cross on retention trial in passive avoidance

Fenitrothion (sumithionR, 50% ai)
Lehotzky et al., 1989
rat Lati
5, 10, 15 mg/kg/d po
sunflower oil
GD7-15
No
Neuromotor
PND26, 36, 104
Decreased latency to fall off rotarod PND26, 104, not 36, 15 mg/kg/d
NOEL=5 mg/kg/d
LOEL=10 mg/kg/d
Postnatal mortality at all doses (16-17.5%) 
No pup allocation described but had to have used some littermates
Only M tested
Statistics not described
Measured startle, righting, contact placing on PND22 but no results given
Shorter latency in cognitive task hard to interpret
Nonsignificant decrease in activity at PND26

Activity
PND26, 36, 104
Decreased activity in open field PND104, 15 mg/kg/d

Cognition
PND42, 104
Shorter escape latency in conditioned response during acquisition, 10 & 15 mg/kg/d

Social behavior
PND62
Increased time in social interaction, 10 & 15 mg/kg/d

Methamidophos
deCastro et al., 2000
rat Wistar
1 mg/kg/d po water
GD6-15
Pilot in nonpregnant F dosed for 10 d gives 17% plasma inhibition at 1 mg/kg/d
Activity
PND40
No effect in open field, sex no mentioned
NOEL=1 mg/kg/d
Used 2 pups/litter but no mention of sex, apparently used as independent observations
No effect on preweaning swimming performance
Decreased immobility time PND 14 only 
Open field measures with really high variability, not reliable
Methamidophos
Lima et al., 2013
mouse Swiss
1 mg/kg/d sc DMSO 
PND3-9
Pilot showed for 1 mg/kg/d: ~19% brain inhibition on PND10; ~36%, 46% brain inhibition 1, 4 hr after dosing on PND3; ~53, 61% brain inhibition 1, 4 hr after dosing on PND9; no brain inhibition in PND60 adults 
Activity
PND61
No effect in open field, sex not mentioned 
No NOEL
LOEL=1 mg/kg/d
1 M & 1 F per litter
No data for M & F separately or mention of statistical differences
Dosing by litter
High variability especially with passive avoidance 

Anxiety & Emotion
PND60-61
Increased immobility time in forced swim, sex not mentioned
No effect on elevated plus maze, sex not mentioned

Cognition
PND63
No effect on passive avoidance, sex not mentioned 

Methyl parathion
Crowder et al., 1980
rat Sprague Dawley
1 mg/kg/d po corn oil
GD7-15
No
Activity 
PND23, 30, 44, 54, 65
Increased activity in open field, only PND23 and 54, sex not mentioned
to criterion 
No NOEL
LOEL=1 mg/kg/d
Preweaning testing: possibly decreased wire cling time (not analyzed), no effect on righting, startle, placement response
Increased postnatal mortality (30%)
Littermates used
Only 3 litters used 
Small sample size for maze testing
Statistics not mentioned except for maze transfer test, just used t-test
Sex not mentioned except for maze transfer test, data not given for M & F
Methods & results cursory

Cognition
>PND68
Slower transfer on 1st, 4th direction change in T-maze learning transfer, sex not mentioned

Methyl parathion
Gupta et al., 1985
rat Wistar-Furth F mated with F344 M
1 mg/kg/d in peanut butter, 1.5 mg/kg/d po peanut oil
GD6-20
Dams on GD19 show 20, 60% brain inhibition at 1, 1.5 mg/kg/d
Pups show brain inhibition up to 50% on  PND1, 7, 14, 21, 1 & 1.5 mg/kg/d; on PND28 only 1.5 mg/kg/d 
Cognition
PND60
No effect on passive avoidance
No effect on shuttle box avoidance
Slower latency to bar press & increased days to asymptote on operant task (no schedule given), 1 mg/kg/d only, sex not mentioned
No NOEL
LOEL=1 mg/kg/d but no effects at 1.5 mg/kg/d
High dose dams had cholinergic signs, increased resorptions
Pups moved to foster mothers at 24 hr
No effect on preweaning reflexive behaviors
Pup allocation not described
Statistics barely described
M & F apparently tested but data for each not shown or mentioned 
Methods cursory
No dose-response for behavior but there is dose-response for ChE inhibition
Only 4/dose for operant testing

Neuromotor
PND60
No effect on rotarod

Activity
PND60
Decreased activity, 1 mg/kg/d only, sex not mentioned

Anxiety & Emotion
PND60
Faster cage emergence, 1 mg/kg/d only, sex not mentioned 

Sensory
PND120
No effect on acoustic startle response

Methyl parathion
Johnson et al., 2009
rat Sprague Dawley
Incrementing doses: low 0.2 mg/kg/d throughout; mid 0.2, 0.4, 0.6 mg/kg/d every 5-6 d; high 0.3, 0.6, 0.9 mg/kg/d every 5-6 d
PND1-21 
Low dose showed 13-15% brain inhibition and high dose showed 63, 20, 18% brain inhibition on PND20, 30, 40; all doses recovered by PND50
Cognition
PND29-60
Increased working memory errors, mid and high dose, M only
Increased reference memory errors, all doses, M only
No NOEL
LOEL=0.2 mg/kg/d
Split-litter dose design
Included litter as random effect in statistics
No effect on preweaning measures of reflex development
No effect in F

Oxydemeton methyl (metasystoxR, 91% ai)
Clemens et al., 1990
rat CD
0.5, 1.5, 4.5 mg/kg/d water
GD6-15
Dams on GD16 show 30, 54, 72%  plasma inhibition (RBC similar) & 22, 52, 68% brain inhibition at 0.5, 1.5, 4.5 mg/kg/d
Dams on GD20 show 20, 39, 54% brain inhibition at 0.5, 1.5, 4.5 mg/kg/d, 40% RBC inhibition at 4.5 mg/kg/d, and no plasma inhibition
Fetuses on GD20 show no brain inhibition 
Cognition
PND25, 26, 35
No effect on M-maze
NOEL=4.5 mg/kg/d
High dose dams had tremors
1 M & 1 F per litter
No data for M & F separately or mention of statistical differences
Statistics barely described
No effect on preweaning reflex or sensory tests

Activity

No effect in open field

Parathion
Al-Hachim & Fink, 1968
mouse CF1
3 mg/kg/d po
Corn oil
3 dosing times: 1st, 2nd, or 3rd trimester
No
Cognition
PND30-37
No effect on conditioned avoidance learning 
NOEL=3 mg/kg/d
Very similar experiment as other papers, maybe same study
Pup allocation not clear but littermates probably used
Inadequate statistics
No mention of sex
Parathion
Al-Hachim & Fink, 1968
mouse CF1
3 mg/kg/d po
Corn oil
3 dosing times: 1st, 2nd, or 3rd trimester
No
Activity
PND60-66
No effect in open field 
NOEL=3 mg/kg/d
Very similar experiment as other papers, maybe same study
Pup allocation not clear but littermates probably used
Inadequate stats
No mention of sex
Parathion
Levin et al., 2010
rat Sprague Dawley
0.1, 0.2 mg/kg/d sc DMSO
PND1-4
Cites Slotkin et al. 2006: 0.1 mg/kg/d produced 5-15% brain inhibition on PND5, no data for 0.2 mg/kg/d
Cognition
PND420, 510, 570
PND420: Increased working memory errors in radial arm maze, 0.1 mg/kg/d, M only; increased reference memory errors, 0.1 & 0.2 mg/kg/d, M only 
PND510: Increased working memory errors in radial arm maze, 0.1 & 0.2 mg/kg/d, M only
PND570: No effect in radial arm maze 
No NOEL
LOEL=0.1 mg/kg/d
Littermates of those used in Timofeeva 2008
Pups & dams redistributed daily
5% mortality high dose
No effect in F
Accepts p<0.1 as significant for interactions
No dose-response for several measures
Parathion
Stamper et al., 1988
rat Long Evans
1.3, 1.9 mg/kg/d sc corn oil
PND5-20
35, 68% brn inh PND21; 26, 36% brn in PND28
Activity 
PND24
No effect in open field 
No NOEL
LOEL=1.3 mg/kg/d
Split-litter dose design
Pup allocation not clear but littermates probably used
High dose produced cholinergic signs, says doses are 33 and 50% of LD50 in PND5 rat
Decreased weight gain with both doses
Preweaning, increased cliff avoidance latency, no effect righting, negative geotaxis, open field
M only tested
No post-hoc comparison of groups when significant, but looks like effects in both doses
No dose-response in working memory errors

Neuromotor
PND24
No effect on rotarod

Cognition
PND24, PND 35-37
Decreased alternation rate in T-maze, 1.3 & 1.9 mg/kg/d, only M tested
Increased working memory errors, 1.3 & 1.9 mg/kg/d, only M tested

Parathion
Timofeeva et al., 2008
rat Sprague Dawley
0.1. 0.2 mg/kg/d sc DMSO
PND1-4
Cites Slotkin et al. 2006: 0.1 mg/kg/d produced 5-15% brain inhibition on PND5, no data for 0.2 mg/kg/d
Activity
PND58-61
No effect in figure-8 chambers 
No NOEL
LOEL=0.1 mg/kg/d
Pups & dams redistributed daily
5% mortality high dose
Accepts p<0.1 as significant  for interactions
All radial arm maze effects only in low dose

Cognition
PND35-45
PND112-182
No effect in T-maze spontaneous alternation
Decreased working memory errors in radial arm maze, 0.1 mg/kg/d only, M&F 

Anxiety & Emotion
PND50-53
PND64-72
PND81-94
Increased time in open arms in elevated plus maze, 0.2 mg/kg/d, M&F
No effect on novelty suppressed feeding 
No effect on chocolate milk preference 

Sensory
PND78-81
Decreased tactile startle, 0.2 mg/kg/d, M&F
No effect on prepulse inhibition

Appendix 2.	Summary of Guideline DNT Studies Submitted to the Agency for OPs other than Chlorpyrifos.
(Only changes that were observed after exposure had ended (post weaning, adult) are listed.  `X' indicates no significant changes on tests for each domain.)

Chemicals
Cognition
Motor activity
Acoustic startle
Neuromotor (FOB)
Notes
Acephate
X
X
X
X
                                                                               
Azinphos-methyl
X
X
X
X
                                                                               
Coumaphos
X
X
X
X
                                                                               
Diazinon
Biel maze: increase errors & latency high dose (~33.1 mg/g,/d, dam diet) M, PND24 & PND62; also mid dose (~3.4 mg/kg/d, dam diet) F, PND24
X
X
X
                                                                               
Dichlorvos[1]
 --
 --
 --
 --

Dicrotophos
X
X
X
X
                                                                               
Dimethoate
X
X
X
X
                                                                               
Disulfoton
X
X
X
X
                                                                               
Ethoprop
M maze: increase trials to criterion high dose (~29.3 mg/kg/d, dam diet) M, PND60
X
X
X
                                                                               
Fenamiphos
X
X
X
X
                                                                               
Malathion
X
increased rearing in FOB open field mid dose (50 mg/kg/d to dams & pups), F, significant at PND45 only (maybe also PND60); no change automated motor activity
increased peak amplitude later blocks (perhaps habituation effect) all doses (5, 50, 150 mg/kg/d to dams & pups), F only, PND23; increased peak amplitude without prepulse low dose only, F only, PND60
altered gait mid & high dose (50, 150 mg/kg/d to dams & pups), M & F, PND60 but not earlier

Methamidophos
~X
X
decreased peak amplitude early blocks mid & high dose (~1.65, 5.2 mg/kg/d dam diet), F only, significant at PND38, looks same but not significant at PND60
X
PA PND24: report says decreased latency but nothing significant & table is questionable; M maze PND60: report says increased trials to criterion (M) and increased (M) or decreased (F) errors, but nothing significant and table is questionable
Methyl parathion
X
X
X
X
                                                                               
Naled
X
X
increased peak amplitude & decreased latency middle blocks low dose only (0.4 mg/kg/d to dams & pups), F only, PND60; report says decreased amplitude but not significant all blocks high dose (10 mg/kg/d to dams & pups), M only, PND23 & PND60
X
only swimming time in Y maze reported, varied significances, no mention of errors or other performance measures
Phorate
M maze: decrease number reaching criterion with relearning, low & mid dose (0.03 & 0.1 mg/kg/d to dams & pups), M only significant, PND30; not seen in second study with higher dose
X
decreased peak amplitude all blocks high dose (0.1 mg/kg/d to dams & pups), M only, PND60; not seen in second study with higher dose
X
combined two studies; one with low & mid dose, other with high dose; some data didn't agree
Profenofos
X
X
X
X
                                                                               
Terbufos
X
X
X
X
                                                                               
Tetrachlorvinphos
X
X
X
X
                                                                               
Tribufos
X
X
X
X
                                                                               
Trichlorfon
PA: decreased latency to enter on retention high dose (~205.1 mg/kg/d dam diet), M only, PND29; M maze: increased average errors second trial high dose, F only, PND60
decreased activity middle blocks (maybefaster habituation) mid dose (~76.2 mg/kg/d dam diet) only, F only, PND60
decreased peak amplitude all blocks high dose (~205.1 mg/kg/d dam diet), early blocks mid dose (~76.2 mg/kg/d dam diet), M & F, PND22; decreased amplitude middle blocks (not consistent) all doses (~23.3, 76.2, 205.1 mg/kg/d dam diet), F only, PND38; for M decreased amplitude apparent but not significant high dose PND38 & PND60
X

[1] high pup mortality in all groups, including control, negated any valid neurotoxicity assessments of dichlorvos

Appendix 3.	Low Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Acosta-Maldonado et al. (2009)
Chihuahua, Mexico
Singleton pregnancies
Cross-sectional, small pilot study  -  only 9 women exposed, participant selection and exclusion not detailed
N=54 mothers
(9 exposed, 45 comparison mothers)
Proxy indicator of exposure - Residence in agricultural community were pesticides had been applied; or home located < 5 km from a pesticide application zone; or cohabitating with worker exposed to pesticides or agricultural labor 

Also, AChE activity  -  Objective biomarker of exposure/altered function 
Standardized but partially subjective assessment of placental maturity

Minimal. Adjustment for placental characteristics. 
Appropriate multivariate analysis; Corrected hypothesis test results for multiple comparisons.
Selection bias unlikely; Residual confounding likely; substantial potential for differential misclassification of exposure.
Dawbrowski et al. (2003)
Lodz, Poland
Newborn children among Polish farmers
Case-control, large sample size
N=389
Age: newborns
Prenatal pesticide exposure assessed via questionnaire (retrospective self-report). Site visit after delivery to evaluate pesticide exposure
Pregnancy outcomes assessed using birth records
Appropriate. Included maternal demographics, predictors of high-risk pregnancy (duration, maternal weight) and environmental toxicant exposure (ETS). No adjustment for SES indicators
Appropriate multivariate analysis.
Selection bias unlikely; Residual confounding likely small in magnitude; considerable potential for non-differential misclassification of exposure (unlikely to account for non-null findings). Minimal misclassification of outcome. 
Cartier et al. (2016)
Brittany, France
PELAGIE study (French population; pregnant women and children)
Longitundinal birth cohort

n = 231

Mother-infant pairs

Metabolite concentrations (total DAP, DM, and DE) of prenatal OP pesticide exposure quantified in two maternal spot urine samples provided at 16 and 26 weeks of gestation

Maternal IQ and the family environment were also assessed using the Wechsler Adult Intelligence Scale and the HOME test
Wechsler Intelligence Scale administered to children to determine IQ

All models were adjusted for HOME score, breastfeeding duration, mothers' IQ, school, maternal education level, psychologist testing the child, and creatinine levels of mother and child
Appropriate: Linear regression and multivariate models
Urine collection methods and storage (urine samples collected at home and mailed back to laboratory); lack of frequent urine spot samling and due to the short half-life of urinary metabolites could have led to exposure misclassification
Fielder et al. (2015)

Bangkok, Thailand area
Thai population (Children aged 6 to 8, who were from two farming regions (aquaculture v. rice)
Cohort, small sample size
n = 54
Age 6- 8 years
Child Exposure  -  Total DEAP, DAP, and DMAP metabolites measured in spot urine samples, collected at three, separate time points (every 6 months) 
Parental education was assessed via a vocabulary test; the home environment was scored via the HOME test
BARS examination administered to children to determine neurobehavorial performance.  

All models were adjusted for age and the HOME scale 
Appropriate: Multivariate linear regression model and mix model linear regression to determine interaction
Small sample size (n = 54); errors in exposure classification methods, specifically, urine collection methods and storage (urine samples collected at home); short half-life of urinary metabolites could have led to exposure misclassification
Grandjean et al.(2006)
Tabacundo, Ecuador
Healthy 2nd and 3rd grade children
Cross-sectional, small sample size
N=72 
Age <9 years
Prenatal occupational exposure assessed via questionnaire; Also recent child exposure biomarker assessment (DAP).
Objective anthropometric and other clinical outcomes; Numerous neurobehavioral outcomes evaluated using easy-to-administer screening instruments; Age appropriate. May be insensitive to subtle effects of OP pesticide exposure effects.
Appropriate. Homogenous population limited confounding by design; Included child demographics (age, sex, weight); SES indicators (maternal race, housing, running water, sewage), diet (meals/day), environmental toxicants (maternal alcohol and smoking) and medical history. 
Appropriate multivariate analysis; Numerous hypotheses evaluated without correction for multiple comparisons.
Selection bias unlikely; Residual confounding likely small in magnitude; considerable potential for non-differential misclassification of exposure (likely to account for null findings). Potential misclassification of outcome. 

                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Harari et al. (2010)
Tabacundo, Ecuador (Andean plateau north of Quito)
6- to 8-year-old children in the two lowest grades (called second and third) of one of two schools
Cross-sectional, small sample size
n = 81
Age 6- 8 years
Child Current Exposure - DMP metabolites measured in a spot urine samples. A blood sample was analyzed for AChE. 

Maternal Exposure  -  Interview by skilled interviewers
Blood pressure and neurophysiologic measures - the instruments used for neurophsychologic measures were validated to avoid cross-cultural influences.

Appropriate: child's sex, age, BMI, number of daily meals (only in current exposure), stunting, hematocrit, school grade, having repeated one grade, maternal education level, family living in a traditional house, drinking water supply, and paternal education and employment
Appropriate: Standard parametric tests and logistic regression
Errors in exposure classification status since it was based on the maternal self-report, mothers likely being aware of the neurobehavioral status of their children, exposure assessment based on a spot urine sample 
Kofman et al. (2006)
Israel (Negev region)
Bedouin population (Children aged 6 to 12 at the time of the study, who were victims of poisoning before age three)
Retrospective cohort study, small sample size
N = 52
9-Exposed to OP; 17-Exposed to Kerosene/paint thinner
26-Controls 
Aged 6-12 years 
OP poisoning was confirmed by low serum butyrylcholinesterase activity based on hospital records
Neuropsychological evaluation and structured interview of parents. Errors in assessment minimized as psychologists were qualified individuals, language and cultural differences taken into consideration, each child tested on same day and in same place as matched control.
Age, sex, background (cultural and demographic)
Difference in means
Errors is outcome classification likely since psychologists who administered the tests knew which children were exposed, small sample size 
Koutroulakis et al. (2014)
Crete, Greece
Women with singleton pregnancies, permanent residents for at least two years, referral for amniocentesis to the Fetal-Maternal Unit, Department of Obstetrics and Gynecology, University Hospital of Heraklion
Prospective Cohort Study  -  large sample size, ethical issues
n = 415
Age: newborns
Objective. DAP measurement in a single amniotic fluid samples collected at either 16th or 20th weeks of gestation  -  Novel biomarker. Questionnaire was also used.
Birth weight and head circumference. Unclear how the outcome information was obtained.
Neonatal sex, maternal age, agricultural activities, and gestational age at amniocentesis  -  Rationale for confounder selection not provided
Appropriate: multiple linear regression
Comparing exposure measurements in AF against known biomarker (e.g., OP metabolite levels in urine) for validation of AF not conducted. Smoking status of participants before/during pregnancy, high risk pregnancies, gestational diabetes, PON1 enzyme activity in the fetuses not considered.

Appendix 3.	Low Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment (continued)
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Lizardi et al. (2008)
Yuma County, Arizona, USA
Children's Pesticide Survey (CPS)
Cross-sectional, small sample size
N=48
Age 7 years
Objective Biomarker of prenatal OP pesticide exposure (DAP) quantified in single child spot urine sample provided at the time of the cognitive assessment. 
Cognitive assessment using battery of test instruments considered valid and reliable in similar populations. Spanish translation as necessary. 
None
Correlation coefficients (type unspecified). Statistically significant confounders were not robust due to influential outliers
Selection bias unlikely; Residual confounding likely; considerable potential for non-differential misclassification of exposure Potential misclassification of outcome.
Lu et al. (2009)
Cota Brus, Costa Rica
4-10 yr. old children whose parents worked in organic coffee farm (La Amistad) and conventional coffee farms (Las Mellizas)
Cross-Sectional (pilot), low sample numbers
N=35:
17 Organic farm
18 Conventional farms
Age 4-10 years old
Good measure (urinary PNP, IMPY, TCPy), but no major differences between exposure groups 
Good measure (CBARS) but different SES and demographic characteristics for exposure groups
Limited number (group, age, sex, handedness, grade)
Appropriate: Linear mixed effects for significant test outcomes from paired t-test analyses
Somewhat high - Convenience sample with different recruitment methods; exposure misclassification
Moreno-Banda et al. (2009)
Mexico (Villa Guerrero, Coatepec de Harinas, Tenancingo (Mexico); Cuernavaca, Cuautla, Jiutepec, Temixco, (Morelos)
Newborn children of floricultural workers and families
Cross-sectional, large sample size
N=328
Age: newborns
Proxy indicator of prenatal occupational OP pesticide exposure (self-reported floricultural occupation)
Objectively measured birth outcomes assessed using birth certificate (partial). Self-reported by mother if birth certificate unavailable.
Appropriate, though minimal  -  history of adverse reproductive outcomes, infant sex, maternal smoking and alcohol use during pregnancy.
Appropriate multivariate analysis
Selection bias likely; Residual confounding likely small in magnitude; considerable potential for non-differential misclassification of exposure (likely accounts for null findings). Potential outcome misclassification (self-report in subset of participants).
Nevison (2014)
U.S. National population
Children with birth years 1970 - 2005 in 1) California Department of Developmental Services (CDDS) reports 2) Individuals with Disabilities Education Act (IDEA) reports
Ecological (time trend)
N=NP (national database)
Non-specific, proxy OP exposure measure (lbs/yr)
Two large reporting DBs: CA Department of Developmental Services (CDDS), US Individuals with Disabilities Education Act (IDEA)
Limited: differences in autism definitions, changes in diagnostic criteria
Appropriate: ratio of age-resolved snapshot; tracking trend slopes; correlation coefficient between temporal trend and composite autism prevalence curve
Selection bias unlikely; do not know individual exposure

Appendix 3.	Low Quality Studies: Summary of Study Design Elements Impacting Study Quality Assignment (continued)
                                     Study
                                Study Location
                 Cohort Name / Description of Study Population
                                 Study Design
                              Exposure Assessment
                              Outcome Assessment
                        Confounder /
Covariate Control
                             Statistical Analysis
                             Risk of (other) Bias
Rohlman et al. (2005)
Oregon and North Carolina, USA
Latino children of immigrant parents living in Oregon or North Carolina
Cross-Sectional, small sample size
N= 78 
Age 48-71 months
Proxy indicator of chronic OP pesticide exposure - Residence in highly agricultural communities
Battery of neurocognitive development; Screening tools; Some instruments likely not appropriate for use in study population - Not all participants able to complete all evaluations. Poor administration. Tests administered twice; only performance on 2[nd] evaluation considered in analysis. 
Self-reported covariate information collected via questionnaire. Adjustment for age, SES indicator (maternal education). No environmental toxicants. 
One-sided hypothesis tests. Numerous hypotheses evaluated without correction for multiple comparisons.
Selection bias unlikely; Residual confounding likely; substantial potential for differential misclassification of exposure and outcome. Inadequate presentation of study results. Seemingly post-hoc evaluation of effect modification. 
Samarawickrema et al. (2008)
Southern Sri Lanka
Pregnant women delivering at Embilipitiya Base Hospital
Cross-sectional Birth Cohort, small sample size
N=41 end of two pesticide spraying seasons;
N=25 at beginning of spraying season
Proxy indicator of prenatal OP pesticide exposure (delivery during pesticide spray season); Objective pesticide biomarkers assessed (OP pesticide residues), but detected in only two subject's specimens  -  not evaluated
Objective biomarkers of early biological effect outcomes - Maternal and fetal butyrylcholinesterase (BuChE) activity; antioxidant status;
fetal oxidative stress;
fetal DNA fragmentation
No adjustment for potential confounders (though comparison groups were considered to be relatively homogeneous).
Largely appropriate. Assumptions of some statistical tests likely violated.
Selection bias unlikely; Residual confounding likely; substantial potential for differential misclassification of exposure; Seemingly post-hoc evaluation of effect modification.
Savitz et al. (1997)
Ontario, Canada
Newborn children in the Ontario Farm Family Health Study; 
Retrospective Birth Cohort, large sample size
N= 1,898 couples; 3,984 
Age: newborns
Proxy indicator of pre-conception paternal para-occupational OP pesticide exposure (self-reported male farm activities in 3-month period prior to conception.
Objectively measured birth outcomes assessed by maternal self-report
Appropriate. Included family/child demographics (sex, weight, maternal age, ethnicity); SES indicators (maternal and paternal education and occupation, per capita income) race, housing, running water, sewage), diet (meals/day), pregnancy risks (maternal caffeine, alcohol and smoking) and medical history.
Inappropriate multivariate regression analysis. Likely misspecification of true variance; No adjustment for multiple comparisons.
Selection bias probable; Residual confounding likely; substantial potential for differential misclassification of exposure and outcome.
Wickerham et al. (2012)
Zhejiang Province,
China
Newborn children delivered at the Fuyang Maternal and Children's hospital
Cross-sectional, small pilot study
n=116
Age: Full term infants
Objective biomarker of pesticide exposure (pesticide residues in cord blood)  -  parameterized as number of pesticide residues detected. Methods unlikely suitable for detecting low levels.
Birth weight assessed using birth records and maternal report
Appropriate. Assessed using questionnaire (maternal self-report) and medical records.
Appropriate multivariate analysis.
Selection bias unlikely; Residual confounding likely; substantial potential for differential misclassification of exposure. Some outcome misclassification unlikely .
Naksen et al (2015)
Fang district, Chiang Mai province, Thailand
Pregnant women delivering at Fang Hospital
Prospective Cohort, small pilot study
n=52
Age: newborns
Objective biomarkers of pesticide exposure (DAPs).  Also AChE, BChE, and PON1 genotype expression measurement.  Maternal blood and urinary samples taken, plus cord blood.  Questionnaire to assess other exposures and covariates.     
Birth outcomes (Body weight and length, and head circumference) abstracted from medical records.  
Appropriate.  Assessed using questionnaire (maternal self-report) and medical records.  
Some errors in statistical analysis were identified; e.g., for gestation age, log total DEAP at 32 weeks of pregnancy, a 0.7 beta was reported, but the confidence interval is reported as (-0.1, -1.4)].  No adjustment for multiple comparisons.
Inadequate presentation of study results. Selection bias possible due to loss to follow-up; Residual confounding likely, small in magnitude; potential for non-differential misclassification of exposure.

Appendix 4.	Table of Systematic Review Analysis:  Initial Search Research & Initial Exclusion Criteria.

Initial search yielded 299 studies.  Removing duplicates (56), there were 243 articles, and 79 were determined to be epidemiological investigations of potential relevance.  The 164 studies excluded from the analysis comprised 57 exposure only studies; 51 review articles; 33 reports of acute OP intoxication; 20 studies in non-human systems; and 3 were otherwise not relevant). 

                                    Number
                                   Citations
Category (EPI-epi only; EXPO- exposure only; ACR - acute case report or case series; TOX-animal study; REV - review or commentary, no original data; OTH - other, non-English
                                       1
Abdallah, M. A. E., & Covaci, A. (2014). Organophosphate flame retardants in indoor dust from Egypt: implications for human exposure. [Journal article]. Environmental Science &amp; Technology, 48, 4782-4789.
                                     EXPO
                                       2
Acosta-Maldonado, B., Sanchez-Ramirez, B., Reza-Lopez, S., & Levario-Carrillo, M. (2009). Effects of exposure to pesticides during pregnancy on placental maturity and weight of newborns: a cross-sectional pilot study in women from the Chihuahua State, Mexico. Hum Exp Toxicol, 28(8), 451-459.
                                      EPI
                                       3
Ali, P., Anwer, A., Bashir, B., Jabeen, R., Haroon, H., & Makki, K. (2012). Clinical pattern and outcome of organophosphorus poisoning. [Article]. Journal of the Liaquat University of Medical and Health Sciences, 11, 15-18.
                                      ACR
                                       4
Alizadeh, A. M., Hassanian-Moghaddam, H., Shadnia, S., Zamani, N., & Mehrpour, O. (2014). Simplified acute physiology score II/Acute physiology and chronic health evaluation ii and prediction of the mortality and later development of complications in poisoned patients admitted to intensive care unit. [Article]. Basic and Clinical Pharmacology and Toxicology, 115, 297-300.
                                      ACR
                                       5
Andersen, H. R., Debes, F., Wohlfahrt-Veje, C., Murata, K., & Grandjean, P. (2015). Occupational pesticide exposure in early pregnancy associated with sex-specific neurobehavioral deficits in the children at school age. Neurotoxicology and Teratology, 47, 1-9.
                                      EPI
                                       6
Attfield, K. R., Hughes, M. D., Spengler, J. D., & Lu, C. (2014). Within- and between-child variation in repeated urinary pesticide metabolite measurements over a 1-year period. [Article]. Environmental Health Perspectives, 122, 201-206.
                                     EXPO
                                       7
Attfield, K. R., Hughes, M. D., Spengler, J. D., & Lu, C. S. (2014). Within- and between-child variation in repeated urinary pesticide metabolite measurements over a 1-year period. [Journal article]. Environmental Health Perspectives, 122, 201-206.
                                   DUPLICATE
                                       8
Babina, K., Dollard, M., Pilotto, L., & Edwards, J. W. (2012). Environmental exposure to organophosphorus and pyrethroid pesticides in South Australian preschool children: A cross sectional study. [Article]. Environment International, 48, 109-120.
                                   DUPLICATE
                                       9
Babina, K., Dollard, M., Pilotto, L., & Edwards, J. W. (2012). Environmental exposure to organophosphorus and pyrethroid pesticides in South Australian preschool children: A cross sectional study. [Article]. Environment International, 48, 109-120.
                                     EXPO
                                      10
Balakumar, K., Misha, K., & Milind, K. (2013). Increased fetal endocardial echogenicity mimicking endocardial fibroelastosis following maternal organophosphorus poisoning and its complete regression in utero. [Article]. Indian Journal of Radiology and Imaging, 23, 262-265.
                                      ACR
                                      11
Baltazar, M. T., Dinis-Oliveira, R. J., de Lourdes Bastos, M., Tsatsakis, A. M., Duarte, J. A., & Carvalho, F. (2014). Pesticides exposure as etiological factors of Parkinson's disease and other neurodegenerative diseases -- A mechanistic approach. Toxicology Letters, 230, 85-103.
                                      REV
                                      12
Barr, D. B., Ananth, C. V., Yan, X., Lashley, S., Smulian, J. C., Ledoux, T. A., et al. (2010). Pesticide concentrations in maternal and umbilical cord sera and their relation to birth outcomes in a population of pregnant women and newborns in New Jersey. Sci Total Environ, 408(4), 790-795.
                                      EPI
                                      13
Béchaux, C., Zetlaoui, M. l., Tressou, J., Leblanc, J.-C., Héraud, F., & Crépet, A. l. (2013). Identification of pesticide mixtures and connection between combined exposure and diet. [Article]. Food and Chemical Toxicology, 59, 191-198.
                                     EXPO
                                      14
Bedi, J. S., Gill, J. P. S., Aulakh, R. S., Kaur, P., Sharma, A., & Pooni, P. A. (2013). Pesticide residues in human breast milk: Risk assessment for infants from Punjab, India. Science of The Total Environment, 463 - 464, 720-726.
                                     EXPO
                                      15
Bellinger, D. C. (2012a). A strategy for comparing the contributions of environmental chemicals and other risk factors to neurodevelopment of children. [Article]. Environmental Health Perspectives, 120, 501-507.
                                      REV
                                      16
Bellinger, D. C. (2012b). Comparing the population neurodevelopmental burdens associated with children's exposures to environmental chemicals and other risk factors. [Article]. NeuroToxicology, 33, 641-643.
                                      REV
                                      17
Berkowitz, G. S., Wetmur, J. G., Birman-Deych, E., Obel, J., Lapinski, R. H., Godbold, J. H., et al. (2004). In utero pesticide exposure, maternal paraoxonase activity, and head circumference. Environ Health Perspect, 112(3), 388-391.
                                      EPI
                                      18
Berton, T., Mayhoub, F., Chardon, K., Duca, R.-C., Lestremau, F., Bach, V., et al. (2014a). Development of an analytical strategy based on LC-MS/MS for the measurement of different classes of pesticides and theirs metabolites in meconium: Application and characterisation of foetal exposure in France. Environmental Research, 132, 311-320.
                                     EXPO
                                      19
Berton, T., Mayhoub, F., Chardon, K., Duca, R.-C., Lestremau, F., Bach, V., et al. (2014a). Development of an analytical strategy based on LC - MS/MS for the measurement of different classes of pesticides and theirs metabolites in meconium: Application and characterisation of foetal exposure in France. Environmental Research, 132, 311-320.
                                   DUPLICATE
                                      20
Berton, T., Mayhoub, F., Chardon, K., Duca, R.-C., Lestremau, F., Bach, V., et al. (2014b). Development of an analytical strategy based on LC - MS/MS for the measurement of different classes of pesticides and theirs metabolites in meconium: Application and characterisation of foetal exposure in France. Environmental Research, 132, 311-320.
                                   DUPLICATE
                                      21
Berton, T., Mayhoub, F., Chardon, K., Duca, R.-C., Lestremau, F., Bach, V., et al. (2014b). Development of an analytical strategy based on LC--MS/MS for the measurement of different classes of pesticides and theirs metabolites in meconium: Application and characterisation of foetal exposure in France. [article]. Environmental research (New York, N.Y. : Print), 132, 311-320.
                                   DUPLICATE
                                      22
Bettegowda, C. (2012). Pesticides perturb prenatal brain. [Note]. Science Translational Medicine, 4.
                                      REV
                                      23
Betts, K. S. (2013). Lasting impacts: Pre- and postnatal PBDE exposures linked to IQ deficits. [Note]. Environmental Health Perspectives, 121.
                                      REV
                                      24
Bhagavath, P., Monteiro, F. N. P., & Gnanadev, N. C. (2012). Epidemiology of intentional self poisoning. [Article]. Medico-Legal Update, 12, 52-54.
                                      ACR
                                      25
Bhaskar, R., & Mohanty, B. (2014). Pesticides in mixture disrupt metabolic regulation: In silico and in vivo analysis of cumulative toxicity of mancozeb and imidacloprid on body weight of mice. General and Comparative Endocrinology, 205, 226-234.
                                      TOX
                                      26
Bhatnagar, S., Das, U. M., & Bhatnagar, G. (2012). Comparison of oral midazolam with oral tramadol, triclofos and zolpidem in the sedation of pediatric dental patients: An in vivo study. [Article]. Journal of Indian Society of Pedodontics and Preventive Dentistry, 30, 109-114.
                                      OTH
                                      27
Biello, D. (2012). Bad for bugs and brains? A common pesticide may interfere with a child's brain development. [Article]. Scientific American, 307, 22.
                                      REV
                                      28
Bouchard, M. F., Bellinger, D. C., Wright, R. O., & Weisskopf, M. G. (2010). Attention-deficit/hyperactivity disorder and urinary metabolites of organophosphate pesticides. [Research Support, N.I.H., Extramural]. Pediatrics, 125(6), e1270-1277.
                                      EPI
                                      29
Bouchard, M. F., Chevrier, J., Harley, K. G., Kogut, K., Vedar, M., Calderon, N., et al. (2011). Prenatal exposure to organophosphate pesticides and IQ in 7-year-old children. Environ Health Perspect, 119(8), 1189-1195.
                                      EPI
                                      30
Bradman, A., Kogut, K., Eisen, E. A., Jewell, N. P., Quirós-Alcalá, L., Castorina, R., et al. (2013). Variability of organophosphorous pesticide metabolite levels in spot and 24-hr urine samples collected from young children during 1 week. [Article]. Environmental Health Perspectives, 121, 118-124.
                                     EXPO
                                      31
Brown, J. S., Jr. (2013). Psychiatric Effects of Organic Chemical Exposure: BLACKWELL SCIENCE PUBL, OSNEY MEAD, OXFORD OX2 0EL, UK.
                                      REV
                                      32
Bulgaroni, V., Lombardo, P., Rivero-Osimani, V., Vera, B., Dulgerian, L., Cerban, F., et al. (2013). Environmental pesticide exposure modulates cytokines, arginase and ornithine decarboxylase expression in human placenta. [Article]. Reproductive Toxicology, 39, 23-32.
                                      TOX
                                      33
Burdorf, A., Brand, T., Jaddoe, V. W., Hofman, A., Mackenbach, J. P., & Steegers, E. A. (2011). The effects of work-related maternal risk factors on time to pregnancy, preterm birth and birth weight: the Generation R Study. Occup Environ Med, 68(3), 197-204.
                                      EPI
                                      34
Burns, C. J., McIntosh, L. J., Mink, P. J., Jurek, A. M., & Li, A. A. (2013). Pesticide exposure and neurodevelopmental outcomes: Review of the epidemiologic and animal studies. [Article]. Journal of Toxicology and Environmental Health - Part B: Critical Reviews, 16, 127-183.
                                      REV
                                      35
Butt, C. M., Congleton, J., Hoffman, K., Fang, M. L., & Stapleton, H. M. (2014). Metabolites of organophosphate flame retardants and 2-ethylhexyl tetrabromobenzoate in urine from paired mothers and toddlers. [Journal article]. Environmental Science &amp; Technology, 48, 10432-10438.
                                     EXPO
                                      36
Butt, C. M., Congleton, J., Hoffman, K., Fang, M., & Stapleton, H. M. (2014). Metabolites of organophosphate flame retardants and 2-ethylhexyl tetrabromobenzoate in urine from paired mothers and toddlers. Environ Sci Technol, 48, 10432-10438.
                                   DUPLICATE
                                      37
Camann, D. E., Schultz, S. T., Yau, A. Y., Heilbrun, L. P., Zuniga, M. M., Palmer, R. F., et al. (2013). Acetaminophen, pesticide, and diethylhexyl phthalate metabolites, anandamide, and fatty acids in deciduous molars: Potential biomarkers of perinatal exposure. [Article]. Journal of Exposure Science and Environmental Epidemiology, 23, 190-196.
                                     EXPO
                                      38
Cappiello, A., Famiglini, G., Palma, P., Termopoli, V., Lavezzi, A. M., & Matturri, L. (2014). Determination of selected endocrine disrupting compounds in human fetal and newborn tissues by GC-MS. [Article]. Analytical and Bioanalytical Chemistry, 406, 2779-2788.
                                     EXPO
                                      39
Carmichael, S. L., Yang, W., Roberts, E. M., Kegley, S. E., Wolff, C., Guo, L., et al. (2013). Hypospadias and residential proximity to pesticide applications. [Article]. Pediatrics, 132, e1216-e1226.
                                      EPI
                                      40
Carmichael, S. L., Yang, W., Roberts, E., Kegley, S. E., Padula, A. M., English, P. B., et al. (2014). Residential agricultural pesticide exposures and risk of selected congenital heart defects among offspring in the San Joaquin Valley of California. [Article]. Environmental Research, 135, 133-138.
                                      EPI
                                      41
Carpenter, D. O. (2013). Intellectual Developmental Disability Syndromes and Organic Chemicals: BLACKWELL SCIENCE PUBL, OSNEY MEAD, OXFORD OX2 0EL, UK.
                                      REV
                                      42
Castorina, R., Bradman, A., McKone, T. E., Barr, D. B., Harnly, M. E., & Eskenazi, B. (2003). Cumulative organophosphate pesticide exposure and risk assessment among pregnant women living in an agricultural community: a case study from the CHAMACOS cohort. Environ Health Perspect, 111(13), 1640-1648.
                                     EXPO
                                      43
Cecchi, A., Rovedatti, M. G., Sabino, G., & Magnarelli, G. G. (2012a). Environmental exposure to organophosphate pesticides: Assessment of endocrine disruption and hepatotoxicity in pregnant women. [Article]. Ecotoxicology and Environmental Safety, 80, 280-287.
                                      EPI
                                      44
Cecchi, A., Rovedatti, M. G., Sabino, G., & Magnarelli, G. G. (2012b). Environmental exposure to organophosphate pesticides: Assessment of endocrine disruption and hepatotoxicity in pregnant women. Ecotoxicology and Environmental Safety, 80, 280-287.
                                   DUPLICATE
                                      45
Cequier, E., Ionas, A. C., Covaci, A., Marcé, R. M., Becher, G., & Thomsen, C. (2014). Occurrence of a broad range of legacy and emerging flame retardants in indoor environments in Norway. [Journal article]. Environmental Science &amp; Technology, 48, 6827-6835.
                                     EXPO
                                      46
Cequier, E., Marcé, R. M., Becher, G., & Thomsen, C. (2014). A high-throughput method for determination of metabolites of organophosphate flame retardants in urine by ultra performance liquid chromatography-high resolution mass spectrometry. [Article]. Analytica Chimica Acta, 845, 98-104.
                                     EXPO
                                      47
Chen, W.-Q., Zhang, Y.-Z., Yuan, L., Li, Y.-F., & Li, J. (2014). Neurobehavioral evaluation of adolescent male rats following repeated exposure to chlorpyrifos. [Article]. Neuroscience Letters, 570, 76-80.
                                      TOX
                                      48
Chen, Y., Garcia, G. E., Huang, W., & Constantini, S. (2014). The involvement of secondary neuronal damage in the development of neuropsychiatric disorders following brain insults. [Article]. Frontiers in Neurology, 5 MAR.
                                   DUPLICATE
                                      49
Chen, Y., Garcia, G. E., Huang, W., & Constantini, S. (2014). The involvement of secondary neuronal damage in the development of neuropsychiatric disorders following brain insults. [Article]. Frontiers in Neurology, 5 MAR.
                                      REV
                                      50
Cole, T. B., Li, W.-F., Co, A. L., Hay, A. M., MacDonald, J. W., Bammler, T. K., et al. (2014). Repeated Gestational Exposure of Mice to Chlorpyrifos Oxon Is Associated with Paraoxonase 1 (PON1) Modulated Effects in Maternal and Fetal Tissues. [Article]. Toxicological Sciences, 141, 409-422.
                                      TOX
                                      51
Crane, A. L., Abdel Rasoul, G., Ismail, A. A., Hendy, O., Bonner, M. R., Lasarev, M. R., et al. (2013). Longitudinal assessment of chlorpyrifos exposure and effect biomarkers in adolescent Egyptian agricultural workers. [Article]. Journal of Exposure Science and Environmental Epidemiology, 23, 356-362.
                                     EXPO
                                      52
Dabrowski, S., Hanke, W., Polanska, K., Makowiec-Dabrowska, T., & Sobala, W. (2003). Pesticide exposure and birthweight: an epidemiological study in Central Poland. Int J Occup Med Environ Health, 16(1), 31-39.
                                      EPI
                                      53
Dahlgren, J. G., Takhar, H. S., Ruffalo, C. A., & Zwass, M. (2004). Health effects of diazinon on a family. J Toxicol Clin Toxicol, 42(5), 579-591.
                                      ACR
                                      54
Das, S., Chatterjee, K., Sarkar, N., Aich, B., & Dolui, S. (2013). Cholinergic crisis, intermediate syndrome and delayed polyneuropathy following malathion poisoning. [Article]. Journal of Pediatric Intensive Care, 2, 137-141.
                                      ACR
                                      55
De Cock, M., Maas, Y. G. H., & De Bor, M. V. (2012). Does perinatal exposure to endocrine disruptors induce autism spectrum and attention deficit hyperactivity disorders? Review. [article]. Acta paediatrica (Oslo), 101, 811-818.
                                      REV
                                      56
de Joode, B. V., Barraza, D., Ruepert, C., Mora, A. M., Cordoba, L., Oberg, M., et al. (2012). Indigenous children living nearby plantations with chlorpyrifos-treated bags have elevated 3,5,6-trichloro-2-pyridinol (TCPy) urinary concentrations. [Article]. Environmental Research, 117, 17-26.
                                     EXPO
                                      57
Detweiler, M. B. (2014). Organophosphate intermediate syndrome with neurological complications of extrapyramidal symptoms in clinical practice. [Article]. Journal of Neurosciences in Rural Practice, 5, 298-301.
                                      REV
                                      58
Deziel, N. C., Ward, M. H., Bell, E. M., Whitehead, T. P., Gunier, R. B., Friesen, M. C., et al. (2013). Temporal variability of pesticide concentrations in homes and implications for attenuation bias in epidemiologic studies. [Article]. Environmental Health Perspectives, 121, 565-571.
                                     EXPO
                                      59
Dhakal, A. K., Shrestha, D., Shakya, A., Shah, S. C., & Shakya, H. (2014). Clinical profile of acute poisoning in children at a teaching hospital in lalitpur. [Article]. Journal of Nepal Paediatric Society, 34, 100-103.
                                      ACR
                                      60
Ding, G. D., Wang, P., Tian, Y., Zhang, J., Gao, Y., Wang, X. J., et al. (2012). Organophosphate Pesticide Exposure and Neurodevelopment in Young Shanghai Children. [Article]. Environmental Science & Technology, 46(5), 2911-2917.
                                      EPI
                                      61
Ding, G., & Tian, Y. (2014). Organophosphate pesticide exposure and child health in China. [Article]. Environmental Science and Pollution Research, 21, 759-760.
                                      REV
                                      62
Ding, G., Han, S., Wang, P., Gao, Y., Shi, R., Wang, G., et al. (2012). Increased levels of 8-hydroxy-2′-deoxyguanosine are attributable to organophosphate pesticide exposure among young children. [Article]. Environmental Pollution, 167, 110-114.
                                     EXPO
                                      63
Dulaurent, S., Gaulier, J. M., Blanc-Lapierre, A., Imbert, L., & Lachâtre, G. (2013). Urinary determination of 2-isopropyl-4-methyl-6-hydroxypyrimidine in case of non fatal poisoning with diazinon. [Article]. Forensic Science International, 228, e20-e24.
                                      ACR
                                      64
Engel, S. M., Berkowitz, G. S., Barr, D. B., Teitelbaum, S. L., Siskind, J., Meisel, S. J., et al. (2007). Prenatal organophosphate metabolite and organochlorine levels and performance on the Brazelton Neonatal Behavioral Assessment Scale in a multiethnic pregnancy cohort. Am J Epidemiol, 165(12), 1397-1404.
                                      EPI
                                      65
Engel, S. M., Wetmur, J., Chen, J., Zhu, C., Barr, D. B., Canfield, R. L., et al. (2011). Prenatal exposure to organophosphates, paraoxonase 1, and cognitive development in childhood. Environ Health Perspect, 119(8), 1182-1188.
                                      EPI
                                      66
Eskenazi, B., Chevrier, J., Rauch, S. A., Kogut, K., Harley, K. G., Johnson, C., et al. (2013). In Utero and Childhood Polybrominated Diphenyl Ether (PBDE) Exposures and Neurodevelopment in the CHAMACOS Study. [Article]. Environmental Health Perspectives, 121(2), 257-262.
                                      EPI
                                      67
Eskenazi, B., Chevrier, J., Rauch, S. A., Kogut, K., Harley, K. G., Johnson, C., et al. (2013). In utero and childhood polybrominated diphenyl ether (PBDE) exposures and neurodevelopment in the CHAMACOS study. [Note]. Environmental Health Perspectives, 121, 257-262.
                                   DUPLICATE
                                      68
Eskenazi, B., Harley, K., Bradman, A., Weltzien, E., Jewell, N. P., Barr, D. B., et al. (2004). Association of in utero organophosphate pesticide exposure and fetal growth and length of gestation in an agricultural population. Environ Health Perspect, 112(10), 1116-1124.
                                      EPI
                                      69
Eskenazi, B., Huen, K., Marks, A., Harley, K. G., Bradman, A., Barr, D. B., et al. (2010). PON1 and neurodevelopment in children from the CHAMACOS study exposed to organophosphate pesticides in utero. Environ Health Perspect, 118(12), 1775-1781.
                                      EPI
                                      70
Eskenazi, B., Kogut, K., Huen, K., Harley, K. G., Bouchard, M., Bradman, A., et al. (2014a). Organophosphate pesticide exposure, PON1, and neurodevelopment in school-age children from the CHAMACOS study. Environmental Research, 134, 149-157.
                                      EPI
                                      71
Eskenazi, B., Kogut, K., Huen, K., Harley, K. G., Bouchard, M., Bradman, A., et al. (2014b). Organophosphate pesticide exposure, PON1, and neurodevelopment in school-age children from the CHAMACOS study. [Article]. Environmental Research, 134, 149-157.
                                   DUPLICATE
                                      72
Eskenazi, B., Marks, A. R., Bradman, A., Harley, K., Barr, D. B., Johnson, C., et al. (2007). Organophosphate pesticide exposure and neurodevelopment in young Mexican-American children. Environ Health Perspect, 115(5), 792-798.
                                      EPI
                                      73
Eskenazi, B., Rosas, L. G., Marks, A. R., Bradman, A., Harley, K., Holland, N., et al. (2008). Pesticide toxicity and the developing brain. Basic Clin Pharmacol Toxicol, 102(2), 228-236.
                                      REV
                                      74
Estevan, C., Fuster, E., Del Río, E., Pamies, D., Vilanova, E., & Sogorb, M. A. (2014). Organophosphorus pesticide chlorpyrifos and its metabolites alter the expression of biomarker genes of differentiation in D3 mouse embryonic stem cells in a comparable way to other model neurodevelopmental toxicants. [Article]. Chemical Research in Toxicology, 27, 1487-1495.
                                      TOX
                                      75
Fenske, R. A., Lu, C., Negrete, M., & Galvin, K. (2013). Breaking the take home pesticide exposure pathway for agricultural families: Workplace predictors of residential contamination. [Article]. American Journal of Industrial Medicine, 56, 1063-1071.
                                     EXPO
                                      76
Ferreira, J. D., Couto, A. C. z., Pombo-de-Oliveira, M. S., & Koifman, S. (2013). In utero pesticide exposure and leukemia in Brazilian children < 2 years of age. [Note]. Environmental Health Perspectives, 121, 269-275.
                                      EPI
                                      77
Feyzİoğlu, B. r., Özdemİr, M., Kutlu, N. O., Baykan, M., & Baysal, B. (2014). Pulmonary infection caused by chryseobacterium indologenes in a patient with a diagnosis of organophosphate poisoning. [Article]. Anatolian Journal of Clinical Investigation, 8, 137-139.
                                      ACR
                                      78
Fortenberry, G. Z., Meeker, J. D., Sanchez, B. N., Barr, D. B., Panuwet, P., Bellinger, D., et al. (2014). Urinary 3,5,6-trichloro-2-pyridinol (TCPY) in pregnant women from Mexico City: Distribution, temporal variability, and relationship with child attention and hyperactivity. [article]. International journal of hygiene and environmental health, 217, 405-412.
                                   DUPLICATE
                                      79
Fortenberry, G. Z., Meeker, J. D., Sanchez, B. N., Barr, D. B., Panuwet, P., Bellinger, D., et al. (2014). Urinary 3,5,6-trichloro-2-pyridinol (TCPY) in pregnant women from Mexico City: distribution, temporal variability, and relationship with child attention and hyperactivity. Int J Hyg Environ Health, 217(2-3), 405-412.
                                      EPI
                                      80
Fortenberry, G. Z., Meeker, J. D., Sánchez, B. N., Barr, D. B., Panuwet, P., Bellinger, D., et al. (2014). Urinary 3,5,6-trichloro-2-pyridinol (TCPY) in pregnant women from Mexico City: Distribution, temporal variability, and relationship with child attention and hyperactivity. International Journal of Hygiene and Environmental Health, 217, 405-412.
                                   DUPLICATE
                                      81
Fortenberry, G. Z., Meeker, J. D., Sánchez, B. N., Barr, D. B., Panuwet, P., Bellinger, D., et al. (2014). Urinary 3,5,6-trichloro-2-pyridinol (TCPY) in pregnant women from Mexico City: distribution, temporal variability, and relationship with child attention and hyperactivity. [Journal article]. International Journal of Hygiene and Environmental Health, 217, 405-412.
                                   DUPLICATE
                                      82
Fortenberry, G. Z., Meeker, J. D., Sánchez, B. N., Bellinger, D., Peterson, K., Schnaas, L., et al. (2014). Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. [Article]. Environmental Research, 132, 342-349.
                                      EPI
                                      83
Fortenberry, G. Z., Meeker, J. D., Sanchez, B. N., Bellinger, D., Peterson, K., Schnaas, L., et al. (2014). Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. Environ Res, 132, 342-349.
                                   DUPLICATE
                                      84
Fortenberry, G. Z., Meeker, J. D., Sanchez, B. N., Bellinger, D., Peterson, K., Schnaas, L., et al. (2014). Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. Environmental Research, 132, 342-349.
                                   DUPLICATE
                                      85
Fortenberry, G. Z., Meeker, J. D., Sánchez, B. N., Bellinger, D., Peterson, K., Schnaas, L., et al. (2014). Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. Environmental Research, 132, 342-349.
                                   DUPLICATE
                                      86
Fortenberry, G. Z., Meeker, J. D., Sánchez, B. N., Bellinger, D., Peterson, K., Schnaas, L., et al. (2014). Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. Environmental Research, 132, 342-349.
                                   DUPLICATE
                                      87
Fromme, H., Lahrz, T., Kraft, M., Fembacher, L., Mach, C., Dietrich, S., et al. (2014). Organophosphate flame retardants and plasticizers in the air and dust in German daycare centers and human biomonitoring in visiting children (LUPE 3). [Article]. Environment International, 71, 158-163.
                                     EXPO
                                      88
Furlong, M. A., Engel, S. M., Barr, D. B., & Wolff, M. S. (2014). Prenatal exposure to organophosphate pesticides and reciprocal social behavior in childhood. Environ Int, 70, 125-131.
                                      EPI
                                      89
Furlong, M. A., Engel, S. M., Barr, D. B., & Wolff, M. S. (2014a). Prenatal exposure to organophosphate pesticides and reciprocal social behavior in childhood. Environment International, 70, 125-131.
                                   DUPLICATE
                                      90
Furlong, M. A., Engel, S. M., Barr, D. B., & Wolff, M. S. (2014b). Prenatal exposure to organophosphate pesticides and reciprocal social behavior in childhood. [article]. Environment international, 70, 125-131.
                                   DUPLICATE
                                      91
Gonza'lez-Alzaga, B., Lacasan, tild, a, M., Aguilar, G., o, C., et al. (2014). A systematic review of neurodevelopmental effects of prenatal and postnatal organophosphate pesticide exposure. [Article]. Toxicology Letters, 230, 104-121.
                                      REV
                                      92
Gonzalez, V., Huen, K., Venkat, S., Pratt, K., Xiang, P., Harley, K. G., et al. (2012). Cholinesterase and paraoxonase (PON1) enzyme activities in MexicanAmerican mothers and children from an agricultural community. [Article]. Journal of Exposure Science and Environmental Epidemiology, 22, 641-648.
                                     EXPO
                                      93
Gonzalez-Alzaga, B., Lacasana, M., Aguilar-Garduno, C., Rodriguez-Barranco, M., Ballester, F., Rebagliato, M., et al. (2013). A systematic review of neurodevelopmental effects of prenatal and postnatal organophosphate pesticide exposure. Toxicol Lett, 230, 104-121.
                                   DUPLICATE
                                      94
González-Alzaga, B., Lacasaña, M., Aguilar-Garduño, C., Rodríguez-Barranco, M., Ballester, F., Rebagliato, M., et al. (2014). A systematic review of neurodevelopmental effects of prenatal and postnatal organophosphate pesticide exposure. Toxicology Letters, 230, 104-121.
                                   DUPLICATE
                                      95
Goodman, J. E., Prueitt, R. L., & Rhomberg, L. R. (2013). INCORPORATING LOW-DOSE EPIDEMIOLOGY DATA IN A CHLORPYRIFOS RISK ASSESSMENT. [Article]. Dose-Response, 11(2), 207-219.
                                      REV
                                      96
Goodman, J. E., Prueitt, R. L., & Rhomberg, L. R. (2013). Incorporating low-dose epidemiology data in a chlorpyrifos risk assessment. [Article]. Dose-Response, 11, 207-219.
                                   DUPLICATE
                                      97
Gowda, V. K. N., Bannigidad, N. B., Kumar, P., Srikanteswara, P.-K., Shivananda, Govindraj, et al. (2014). Predictors of non-traumatic coma in a pediatric cohort from a South Indian tertiary care center: Results of a multivariate analysis. [Article]. Journal of Pediatric Neurology, 12, 35-43.
                                      ACR
                                      98
Grandjean, P., Harari, R., Barr, D. B., & Debes, F. (2006). Pesticide exposure and stunting as independent predictors of neurobehavioral deficits in Ecuadorian school children. [Research Support, Non-U.S. Gov't]. Pediatrics, 117(3), e546-556.
                                      EPI
                                      99
Guha, N., Ward, M. H., Gunier, R., Colt, J. S., Suzanne Lea, C., Buffer, P. A., et al. (2013). Characterization of residential pesticide use and chemical formulations through self-report and household inventory: The northern California childhood leukemia study. [Note]. Environmental Health Perspectives, 121, 276-282.
                                     EXPO
                                      100
Guiñazú, N., Rena, V., Genti-Raimondi, S., Rivero, V., & Magnarelli, G. (2012). Effects of the organophosphate insecticides phosmet and chlorpyrifos on trophoblast JEG-3 cell death, proliferation and inflammatory molecule production. [Article]. Toxicology in Vitro, 26, 406-413.
                                      TOX
                                      101
Guodong, D., & Yixiao, B. A. O. (2014). Revisiting pesticide exposure and children's health: Focus on China. [article]. Science of the total environment, 472, 289-295.
                                      REV
                                      102
Guodong, D., Pei, W., Ying, T., Jun, Z., Yu, G., Xiaojin, W., et al. (2012). Organophosphate pesticide exposure and neurodevelopment in young Shanghai children. [Article]. Environmental Science and Technology, 46, 2911-2917.
                                   DUPLICATE
                                      103
Guodong, D., Pei, W., Ying, T., Jun, Z., Yu, G., Xiaojin, W., et al. (2012). Organophosphate pesticide exposure and neurodevelopment in young Shanghai children. Environ Sci Technol.
                                      EPI
                                      104
Handal, A. J., Harlow, S. D., Breilh, J., & Lozoff, B. (2008). Occupational exposure to pesticides during pregnancy and neurobehavioral development of infants and toddlers. Epidemiology, 19(6), 851-859.
                                      EPI
                                      105
Handal, A. J., Lozoff, B., Breilh, J., & Harlow, S. D. (2007a). Effect of community of residence on neurobehavioral development in infants and young children in a flower-growing region of Ecuador. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't]. Environ Health Perspect, 115(1), 128-133.
                                      EPI
                                      106
Handal, A. J., Lozoff, B., Breilh, J., & Harlow, S. D. (2007b). Neurobehavioral development in children with potential exposure to pesticides. Epidemiology, 18(3), 312-320.
                                      EPI
                                      107
Harari, R., Julvez, J., Murata, K., Barr, D., Bellinger, D. C., Debes, F., et al. (2010). Neurobehavioral deficits and increased blood pressure in school-age children prenatally exposed to pesticides. Environ Health Perspect, 118(6), 890-896.
                                      EPI
                                      108
Harley, K. G., Huen, K., Schall, R. A., Holland, N. T., Bradman, A., Barr, D. B., et al. (2011). Association of organophosphate pesticide exposure and paraoxonase with birth outcome in Mexican-American women. PLoS One, 6(8), e23923.
                                      EPI
                                      109
Hoffman, K., Daniels, J. L., & Stapleton, H. M. (2014). Urinary metabolites of organophosphate flame retardants and their variability in pregnant women. [Article]. Environment International, 63, 169-172.
                                     EXPO
                                      110
Hoffman, K., Daniels, J. L., & Stapleton, H. M. (2014). Urinary metabolites of organophosphate flame retardants and their variability in pregnant women. [Journal article]. Environment International, 63, 169-172.
                                   DUPLICATE
                                      111
Hore, P., Robson, M., Freeman, N., Zhang, J., Wartenberg, D., Ozkaynak, H., et al. (2005). Chlorpyrifos accumulation patterns for child-accessible surfaces and objects and urinary metabolite excretion by children for 2 weeks after crack-and-crevice application. Environ Health Perspect, 113(2), 211-219.
                                     EXPO
                                      112
Hore, P., Zartarian, V., Xue, J., Oezkaynak, H., Wang, S.-W., Yang, Y.-C., et al. (2006). Childrens residential exposure to chlorpyrifos: Application of CPPAES field measurements of chlorpyrifos and TCPy within MENTOR/SHEDS-Pesticides model. Science of the Total Environment, 366(2-3), 525-537.
                                     EXPO
                                      113
Horton, M. K., Kahn, L. G., Perera, F., Barr, D. B., & Rauh, V. (2012). Does the home environment and the sex of the child modify the adverse effects of prenatal exposure to chlorpyrifos on child working memory? Neurotoxicol Teratol, 34(5), 534-541.
                                      EPI
                                      114
Horton, M. K., Kahn, L. G., Perera, F., Barr, D. B., & Rauh, V. (2012). Does the home environment and the sex of the child modify the adverse effects of prenatal exposure to chlorpyrifos on child working memory? Neurotoxicology and Teratology, 34, 534-541.
                                   DUPLICATE
                                      115
Horton, M. K., Kahn, L. G., Perera, F., Boyd Barr, D., & Rauh, V. (2012). Does the home environment and the sex of the child modify the adverse effects of prenatal exposure to chlorpyrifos on child working memory? [article]. Neurotoxicology and teratology, 34, 534-541.
                                   DUPLICATE
                                      116
How, V., Hashim, Z., Ismail, P., Said, S. M., Dzolkhifli, O., & Shamsul Bahri, M. T. (2014). Exploring cancer development in adulthood: cholinesterase depression and genotoxic effect from chronic exposure to organophosphate pesticides among rural farm children. [Journal article]. Journal of Agromedicine, 19, 35-43.
                                      EPI
                                      117
Huen, K., Bradman, A., Harley, K., Yousefi, P., Barr, D. B., Eskenazi, B., et al. (2012). Organophosphate pesticide levels in blood and urine of women and newborns living in an agricultural community. [Article]. Environmental Research, 117, 8-16.
                                     EXPO
                                      118
Huen, K., Bradman, A., Harley, K., Yousefi, P., Boyd Barr, D., Eskenazi, B., et al. (2012). Organophosphate pesticide levels in blood and urine of women and newborns living in an agricultural community. [Article]. Environmental Research, 117, 8-16.
                                   DUPLICATE
                                      119
Huen, K., Harley, K., Beckman, K., Eskenazi, B., & Holland, N. (2013). Associations of PON1 and Genetic Ancestry with Obesity in Early Childhood. [Article]. PLoS ONE, 8.
                                      EPI
                                      120
Iftikhar, B., Siddiqui, S., & Rehman, S. (2014). Assessment of the dietary transfer of pesticides to dairy milk and its effect on human health. [Journal article]. African Journal of Biotechnology, 13, 476-485.
                                     EXPO
                                      121
Jafarzadeh, M., Nasri Nasrabadi, Z., Sheikhazadi, A., Abbaspour, A., Vasigh, S., Yousefinejad, V., et al. (2013). Is there a role for progesterone in the management of acute organophosphate poisoning during pregnancy? [Article]. Medical Hypotheses, 80, 804-805.
                                      REV
                                      122
Jamkhande, P. G., Chintawar, K. D., & Chandak, P. G. (2014a). Teratogenicity: a mechanism based short review on common teratogenic agents. Asian Pacific Journal of Tropical Disease, 4, 421-432.
                                      REV
                                      123
Jamkhande, P. G., Chintawar, K. D., & Chandak, P. G. (2014b). Teratogenicity: A mechanism based short review on common teratogenic agents. [Article]. Asian Pacific Journal of Tropical Disease, 4, 421-432.
                                   DUPLICATE
                                      124
Jones, K., Everard, M., & Harding, A. H. (2014). Investigation of gastrointestinal effects of organophosphate and carbamate pesticide residues on young children. [Journal article]. International Journal of Hygiene and Environmental Health, 217, 392-398.
                                      EPI
                                      125
Kajta, M., & Wojtowicz, A. K. (2013). Impact of endocrine-disrupting chemicals on neural development and the onset of neurological disorders. [Literature Review]. Pharmacological Reports, 65, 1632-1639.
                                      REV
                                      126
Karki, R. K., & Risal, A. (2012). Study of poisoning cases in a tertiary care hospital. [Article]. Kathmandu University Medical Journal, 10, 70-73.
                                      ACR
                                      127
Kauffman, R., Marcus, M., Wold, J. L., & Lu, C. (2009). Pesticide exposure and neurodevelopment in migrant farmworker children of South Georgia. International Journal of Child and Adolescent Health. Special Issue: Break the cycle of adverse health and developmental disparities for children, 2(3), 303-312.
                                      REV
                                      128
Kezios, K. L., Liu, X. H., Cirillo, P. M., Cohn, B. A., Kalantzi, O. I., Wang, Y. Z., et al. (2013). Dichlorodiphenyltrichloroethane (DDT), DDT metabolites and pregnancy outcomes. [Article]. Reproductive Toxicology, 35, 156-164.
                                      EPI
                                      129
Khan, H. A. A., & Waseem, A. (2014). The effect of temperature on the toxicity of insecticides against Musca domestica L.: implications for the effective management of diarrhea. [Journal article]. PLoS ONE, 9, e95636.
                                      TOX
                                      130
Khan, K., Ismail, A. A., Abdel Rasoul, G., Bonner, M. R., Lasarev, M. R., Hendy, O., et al. (2014). Longitudinal assessment of chlorpyrifos exposure and self-reported neurological symptoms in adolescent pesticide applicators. [Article]. BMJ Open, 4.
                                   DUPLICATE
                                      131
Khan, K., Ismail, A. A., Abdel Rasoul, G., Bonner, M. R., Lasarev, M. R., Hendy, O., et al. (2014). Longitudinal assessment of chlorpyrifos exposure and self-reported neurological symptoms in adolescent pesticide applicators. [Article]. BMJ Open, 4.
                                      EPI
                                      132
Kicinski, M., Vrijens, J., Vermier, G., Hond, E. D., Schoeters, G., Nelen, V., et al. (2015). Neurobehavioral function and low-level metal exposure in adolescents. International Journal of Hygiene and Environmental Health, 218, 139-146.
                                      EPI
                                      133
Kim, H.-H., Lim, Y.-W., Yang, J.-Y., Shin, D.-C., Ham, H.-S., Choi, B.-S., et al. (2013a). Health risk assessment of exposure to chlorpyrifos and dichlorvos in children at childcare facilities. [Article]. Science of the Total Environment, 444, 441-450.
                                     EXPO
                                      134
Kim, H.-H., Lim, Y.-W., Yang, J.-Y., Shin, D.-C., Ham, H.-S., Choi, B.-S., et al. (2013b). Health risk assessment of exposure to chlorpyrifos and dichlorvos in children at childcare facilities. Science of The Total Environment, 444, 441-450.
                                   DUPLICATE
                                      135
Kofman, O., Berger, A., Massarwa, A., Friedman, A., & Jaffar, A. A. (2006). Motor inhibition and learning impairments in school-aged children following exposure to organophosphate pesticides in infancy. Pediatr Res, 60(1), 88-92.
                                      EPI
                                      136
Koirala, D. P., Rao, K. S., Malla, K. K., & Malla, T. (2013). A study of clinical features, management and outcome of organophosphate and carbamate poisoning in children. [Article]. Journal of Nepal Paediatric Society, 33, 85-90.
                                      ACR
                                      137
Koureas, M., Tsakalof, A., Tsatsakis, A., & Hadjichristodoulou, C. (2012). Systematic review of biomonitoring studies to determine the association between exposure to organophosphorus and pyrethroid insecticides and human health outcomes. [article]. Toxicology letters, 210, 155-168.
                                      REV
                                      138
Koutroulakis, D., Sifakis, S., Tzatzarakis, M. N., Alegakis, A. K., Theodoropoulou, E., Kavvalakis, M. P., et al. (2014). Dialkyl phosphates in amniotic fluid as a biomarker of fetal exposure to organophosphates in Crete, Greece; association with fetal growth. [Article]. Reproductive Toxicology, 46, 98-105.
                                      EPI
                                      139
Kr, M. Z., Öztürk, G. l., Gürler, M., Karaarslan, B., Erden, G. n. l., Karapirli, M., et al. (2013). Pesticide poisoning cases in Ankara and nearby cities in Turkey: An 11-year retrospective analysis. [Article]. Journal of Forensic and Legal Medicine, 20, 274-277.
                                      ACR
                                      140
Krishnamurthy, L., Rohini, K., & Shashikala, P. (2012). Socio-demographic profile of organophosphorous poisoning in a tertiary care hospital. [Article]. Indian Journal of Public Health Research and Development, 3, 169-171.
                                      ACR
                                      141
Kristensen, P., Irgens, L. M., Andersen, A., Bye, A. S., & Sundheim, L. (1997). Gestational age, birth weight, and perinatal death among births to Norwegian farmers, 1967-1991. Am J Epidemiol, 146(4), 329-338.
                                      EPI
                                      142
Kumar, M., Vignan Kumar, G., Babu, P., Kumar, S., Subrahmanyam, B., Veeraprasad, M., et al. (2014). A retrospective analysis of acute organophosphorus poisoning cases admitted to the tertiary care teaching hospital in South India. [Article]. Annals of African Medicine, 13, 71-75.
                                      ACR
                                      143
Kumar, M., Vignan Kumar, G., Babu, P., Kumar, S., Subrahmanyam, B., Veeraprasad, M., et al. (2014). A retrospective analysis of acute organophosphorus poisoning cases admitted to the tertiary care teaching hospital in South India. [Article]. Annals of African Medicine, 13, 71-75.
                                   DUPLICATE
                                      144
Kumar, N., Gupta, A. K., Najotra, D. K., Digra, S. K., Gupta, S., Gupta, P. K., et al. (2013). Incidence and pattern of acute poisoning among children aged 1 month to 18 years: An experience from Jammu. [Article]. Indian Journal of Forensic Medicine and Toxicology, 7, 135-138.
                                      ACR
                                      145
Lee, D., Jung, K., Choi, Y., & Cheon, Y. (2014). Body mass index as a prognostic factor in organophosphate-poisoned patients. [Journal article]. American Journal of Emergency Medicine, 32, 693-696.
                                      ACR
                                      146
Lee, J., & Freeman, J. L. (2014). Zebrafish as a model for investigating developmental lead (Pb) neurotoxicity as a risk factor in adult neurodegenerative disease: A mini-review. Neurotoxicology, 43, 57-64.
                                      TOX
                                      147
Lee, W. J., Ko, Y., & Cha, E. S. (2014). Acute pesticide poisoning among children in south korea: Findings from national health insurance claims data, 2006-2009. [Article]. Journal of Tropical Pediatrics, 60, 4-9.
                                      ACR
                                      148
Li, A. A., Levine, T. E., Burns, C. J., & Kent Anger, W. (2012). Integration of epidemiology and animal neurotoxicity data for risk assessment. [Article]. NeuroToxicology, 33, 823-832.
                                      REV
                                      149
Li, A. A., Lowe, K. A., McIntosh, L. J., & Mink, P. J. (2012). EVALUATION OF EPIDEMIOLOGY AND ANIMAL DATA FOR RISK ASSESSMENT: CHLORPYRIFOS DEVELOPMENTAL NEUROBEHAVIORAL OUTCOMES. [Literature Review]. Journal of Toxicology and Environmental Health Part B Critical Reviews, 15, 109-184.
                                   DUPLICATE
                                      150
Li, A. A., Lowe, K. A., McIntosh, L. J., & Mink, P. J. (2012). Evaluation of epidemiology and animal data for risk assessment: chlorpyrifos developmental neurobehavioral outcomes. J Toxicol Environ Health B Crit Rev, 15(2), 109-184.
                                      REV
                                      151
Li, J., Yu, N., Zhang, B., Jin, L., Li, M., Hu, M., et al. (2014). Occurrence of organophosphate flame retardants in drinking water from China. [Article]. Water Research, 54, 53-61.
                                   DUPLICATE
                                      152
Li, J., Yu, N., Zhang, B., Jin, L., Li, M., Hu, M., et al. (2014). Occurrence of organophosphate flame retardants in drinking water from China. [Article]. Water Research, 54, 53-61.
                                     EXPO
                                      153
Liu, P., Wu, C., Chang, X., Qi, X., Zheng, M., & Zhou, Z. (2014). Assessment of chlorpyrifos exposure and absorbed daily doses among infants living in an agricultural area of the Province of Jiangsu, China. [Journal article]. International Archives of Occupational and Environmental Health, 87, 753-762.
                                     EXPO
                                      154
Lizardi, P. S., O'Rourke, M. K., & Morris, R. J. (2008). The effects of organophosphate pesticide exposure on Hispanic children's cognitive and behavioral functioning. J Pediatr Psychol, 33(1), 91-101.
                                      EPI
                                      155
Llop, S., Julvez, J., Fernandez-Somoano, A., Santa Marina, L., Vizcaino, E., Iñiguez, C., et al. (2013). Prenatal and postnatal insecticide use and infant neuropsychological development in a multicenter birth cohort study. Environment International, 59, 175-182.
                                      EPI
                                      156
London, L., Beseler, C., Bouchard, M. F., Bellinger, D. C., Colosio, C., Grandjean, P., et al. (2012). Neurobehavioral and neurodevelopmental effects of pesticide exposures. [Article]. NeuroToxicology, 33, 887-896.
                                      REV
                                      157
Lopez-Carrillo, L., & Cebrian, M. E. (2013). Cognitive Function: BLACKWELL SCIENCE PUBL, OSNEY MEAD, OXFORD OX2 0EL, UK.
                                      REV
                                      158
Lovasi, G. S., Quinn, J. W., Rauh, V. A., Perera, F. P., Andrews, H. F., Garfinkel, R., et al. (2011). Chlorpyrifos exposure and urban residential environment characteristics as determinants of early childhood neurodevelopment. Am J Public Health, 101(1), 63-70.
                                      EPI
                                      159
Lozowicka, B. (2015). Health risk for children and adults consuming apples with pesticide residue. [Journal article]. Science of the Total Environment, 502, 184-198.
                                      REV
                                      160
Lu, C., & Andres, L. (2012). Reconstructing organophosphorus pesticide doses using the reversed dosimetry approach in a simple physiologically-based pharmacokinetic model. [Article]. Journal of Toxicology, 2012.
                                     EXPO
                                      161
Lu, C., Essig, C., Root, C., Rohlman, D. S., McDonald, T., & Sulzbacher, S. (2009). Assessing the association between pesticide exposure and cognitive development in rural Costa Rican children living in organic and conventional coffee farms. Int J Adolesc Med Health, 21(4), 609-621.
                                      EPI
                                      162
Marks, A. R., Harley, K., Bradman, A., Kogut, K., Barr, D. B., Johnson, C., et al. (2010). Organophosphate pesticide exposure and attention in young Mexican-American children: the CHAMACOS study. Environ Health Perspect, 118(12), 1768-1774.
                                      EPI
                                      163
Marri, M. Z., Mirza, F. H., & Hasan, Q. (2012). Pattern, incidence and mortality rate of acute poisonings at Karachi- one year study. [Article]. Medical Forum Monthly, 23, 6-10.
                                      ACR
                                      164
Martin, I. G., Thottempudi, S. R. K., & Frattarelli, D. A. C. (2013). Changes to Prescription Drug Pediatric Labeling: Awareness by Practicing Pediatricians. [Article]. Drug Information Journal, 47, 214-218.
                                      OTH
                                      165
Mascarelli, A. (2013). Growing up with pesticides. [Note]. Science, 341, 740-741.
                                      REV
                                      166
McGinnis, W. R., Audhya, T., & Edelson, S. M. (2013). Proposed toxic and hypoxic impairment of a brainstem locus in autism. [Article]. International Journal of Environmental Research and Public Health, 10, 6955-7000.
                                      REV
                                      167
Memon, A., Shaikh, J. M., Kazi, S. A. F., & Kazi, A. (2012). Changing trends in deliberate self poisoning at Hyderabad. [Article]. Journal of the Liaquat University of Medical and Health Sciences, 11, 124-126.
                                      ACR
                                      168
Meyer-Baron, M., Kim, E. A., Nuwayhid, I., Ichihara, G., & Kang, S.-K. (2012). Occupational exposure to neurotoxic substances in Asian countries - Challenges and approaches. [Article]. NeuroToxicology, 33, 853-861.
                                      REV
                                      169
Michalakis, M., Tzatzarakis, M. N., Kovatsi, L., Alegakis, A. K., Tsakalof, A. K., Heretis, I., et al. (2014). Hypospadias in offspring is associated with chronic exposure of parents to organophosphate and organochlorine pesticides. [Article]. Toxicology Letters, 230, 139-145.
                                      EPI
                                      170
Mink, P. J., Kimmel, C. A., & Li, A. A. (2012). POTENTIAL EFFECTS OF CHLORPYRIFOS ON FETAL GROWTH OUTCOMES: IMPLICATIONS FOR RISK ASSESSMENT. [Article]. Journal of Toxicology and Environmental Health Part B Critical Reviews, 15, 281-316.
                                      REV
                                      171
Mink, P. J., Kimmel, C. A., & Li, A. A. (2012). Potential effects of chlorpyrifos on fetal growth outcomes: implications for risk assessment. J Toxicol Environ Health B Crit Rev, 15(4), 281-316.
                                   DUPLICATE
                                      172
Moreno-Banda, G., Blanco-Munoz, J., Lacasana, M., Rothenberg, S. J., Aguilar-Garduno, C., Gamboa, R., et al. (2009). Maternal exposure to floricultural work during pregnancy, PON1 Q192R polymorphisms and the risk of low birth weight. Sci Total Environ, 407(21), 5478-5485.
                                      EPI
                                      173
Morgan, M. K., & Jones, P. A. (2013). Dietary predictors of young children's exposure to current-use pesticides using urinary biomonitoring. [Article]. Food and Chemical Toxicology, 62, 131-141.
                                     EXPO
                                      174
Morgan, M. K., Wilson, N. K., & Chuang, J. C. (2014). Exposures of 129 preschool children to organochlorines, organophosphates, pyrethroids, and acid herbicides at their homes and daycares in North Carolina. [Article]. International Journal of Environmental Research and Public Health, 11, 3743-3764.
                                     EXPO
                                      175
Morgan, M. K., Wilson, N. K., & Chuang, J. C. (2014). Exposures of 129 preschool children to organochlorines, organophosphates, pyrethroids, and acid herbicides at their homes and daycares in North Carolina. [Journal article]. International Journal of Environmental Research and Public Health, 11, 3743-3764.
                                   DUPLICATE
                                      176
Mosaddegh, M. H., Emami, F., & Asghari, G. (2014). Evaluation of residual diazinon and chlorpiryfos in children herbal medicines by headspace-SPME and GC-FID. [Article]. Iranian Journal of Pharmaceutical Research, 13, 541-549.
                                     EXPO
                                      177
Munoz-Quezada, M. T., Lucero, B. A., Barr, D. B., Steenland, K., Levy, K., Ryan, B., et al. (2013). Neurodevelopmental effects in children associated with exposure to organophosphate pesticides: A systematic review. Neurotoxicology, 39, 158-168.
                                      REV
                                      178
Munoz-Quezada, M. T., Lucero, B. A., Barr, D. B., Steenland, K., Levy, K., Ryan, P. B., et al. (2013). Neurodevelopmental effects in children associated with exposure to organophosphate pesticides: A systematic review. [article]. Neurotoxicology (Park Forest South), 39, 158-168.
                                   DUPLICATE
                                      179
Naseh, M., & Vatanparast, J. (2014). Enhanced expression of hypothalamic nitric oxide synthase in rats developmentally exposed to organophosphates. [Journal article]. Brain Research, 1579, 10-19.
                                      TOX
                                      180
Neal, R. E., Chen, J., Jagadapillai, R., Jang, H., Abomoelak, B., Brock, G., et al. (2014). Developmental cigarette smoke exposure: Hippocampus proteome and metabolome profiles in low birth weight pups. Toxicology, 317, 40-49.
                                      TOX
                                      181
Nevison, C. D. (2014). A comparison of temporal trends in United States autism prevalence to trends in suspected environmental factors. [Article]. Environmental Health: A Global Access Science Source, 13.
                                      EPI
                                      182
Oates, L., Cohen, M., Braun, L., Schembri, A., & Taskova, R. (2014). Reduction in urinary organophosphate pesticide metabolites in adults after a week-long organic diet. [Journal article]. Environmental Research, 132, 105-111.
                                     EXPO
                                      183
Ostrea Jr, E. M., Reyes, A., Villanueva-Uy, E., Pacifico, R., Benitez, B., Ramos, E., et al. (2012). Fetal exposure to propoxur and abnormal child neurodevelopment at 2 years of age. [Article]. NeuroToxicology, 33, 669-675.
                                   DUPLICATE
                                      184
Ostrea, E. M., Bielawski, D. M., Posecion, N. C., Corrion, M., Villanueva-Uy, E., Bernardo, R. C., et al. (2009). Combined analysis of prenatal (maternal hair and blood) and neonatal (infant hair, cord blood and meconium) matrices to detect fetal exposure to environmental pesticides. Environ Res, 109(1), 116-122.
                                     EXPO
                                      185
Ostrea, E. M., Bielawski, D. M., Posecion, N. C., Corrion, M., Villanueva-Uy, E., Jin, Y., et al. (2008). A comparison of infant hair, cord blood and meconium analysis to detect fetal exposure to environmental pesticides. [Evaluation Studies Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't Research Support, U.S. Gov't, Non-P.H.S.]. Environ Res, 106(2), 277-283.
                                     EXPO
                                      186
Ostrea, E. M., Morales, V., Ngoumgna, E., Prescilla, R., Tan, E., Hernandez, E., et al. (2002). Prevalence of fetal exposure to environmental toxins as determined by meconium analysis. NeuroToxicology, 23(3), 329-339.
                                     EXPO
                                      187
Ostrea, E. M., Reyes, A., Villanueva-Uy, E., Pacifico, R., Benitez, B., Ramos, E., et al. (2011). Fetal exposure to propoxur and abnormal child neurodevelopment at 2 years of age. NeuroToxicology.
                                      EPI
                                      188
Ostrea, E. M., Villanueva-Uy, E., Bielawski, D. M., Posecion, N. C., Corrion, M. L., Jin, Y., et al. (2006). Maternal hair--an appropriate matrix for detecting maternal exposure to pesticides during pregnancy. Environ Res, 101(3), 312-322.
                                     EXPO
                                      189
Oulhote, Y., & Bouchard, M. F. (2013). Urinary metabolites of organophosphate and pyrethroid pesticides and behavioral problems in Canadian children. [Article]. Environmental Health Perspectives, 121, 1378-1384.
                                      EPI
                                      190
Padmanabha, T. S., Gumma, K., & Kulkarni, G. P. (2014). Study of profile of organophosphorus poisoning cases in a tertiary care hospital, North Karnataka, Bidar, India. [Article]. International Journal of Pharma and Bio Sciences, 5, P332-P339.
                                      ACR
                                      191
Papoutsis, I., Mendonis, M., Nikolaou, P., Athanaselis, S., Pistos, C., Maravelias, C., et al. (2012). Development and Validation of a Simple GC-MS Method for the Simultaneous Determination of 11 Anticholinesterase Pesticides in Blood-Clinical and Forensic Toxicology Applications. [Article]. Journal of Forensic Sciences, 57, 806-812.
                                     EXPO
                                      192
Patel, A. B., Dewan, A., & Kaji, B. C. (2012). Monocrotophos poisoning through contaminated millet flour. [Article]. Arhiv za Higijenu Rada i Toksikologiju, 63, 377-383.
                                      ACR
                                      193
Perry, L., Adams, R. D., Bennett, A. R., Lupton, D. J., Jackson, G., Good, A. M., et al. (2014). National toxicovigilance for pesticide exposures resulting in health care contact-An example from the UK's National Poisons Information Service. [Article]. Clinical Toxicology, 52, 549-555.
                                      ACR
                                      194
Peshin, S. S., Srivastava, A., Halder, N., & Gupta, Y. K. (2014). Pesticide poisoning trend analysis of 13 years: A retrospective study based on telephone calls at the National Poisons Information Centre, All India Institute of Medical Sciences, New Delhi. [Article]. Journal of Forensic and Legal Medicine, 22, 57-61.
                                      ACR
                                      195
Petit, C., Chevrier, C., Durand, G., Monfort, C., Rouget, F., Garlantezec, R., et al. (2010). Impact on fetal growth of prenatal exposure to pesticides due to agricultural activities: A prospective cohort study in Brittany, France. [Research Support, Non-U.S. Gov't]. Environ Health, 9, 71.
                                      EPI
                                      196
Phillips, M. L. (2006). Registering skepticism: Does the EPA'S pesticide review protect children? Environmental Health Perspectives, 114(10), A592-A595.
                                      REV
                                      197
Pillai, H. K., Fang, M. L., Beglov, D., Kozakov, D., Vajda, S., Stapleton, H. M., et al. (2014). Ligand binding and activation of PPARγ by Firemaster(R) 550: effects on adipogenesis and osteogenesis in vitro. [Journal article]. Environmental Health Perspectives, 122, 1225-1232.
                                      TOX
                                      198
Polanska, K., Jurewicz, J., & Hanke, W. (2013). REVIEW OF CURRENT EVIDENCE ON THE IMPACT OF PESTICIDES, POLYCHLORINATED BIPHENYLS AND SELECTED METALS ON ATTENTION DEFICIT/HYPERACTIVITY DISORDER IN CHILDREN. [Literature Review]. International Journal of Occupational Medicine and Environmental Health, 26, 16-38.
                                      REV
                                      199
Potera, C. (2012). Newly discovered mechanism for cchlorpyrifos eeffects on neurodevelopment. [Article]. Environmental Health Perspectives, 120, A270.
                                      REV
                                      200
Prapamontol, T., Sutan, K., Laoyang, S., Hongsibsong, S., Lee, G., Yano, Y., et al. (2014). Cross validation of gas chromatography-flame photometric detection and gas chromatography--mass spectrometry methods for measuring dialkylphosphate metabolites of organophosphate pesticides in human urine. [article]. International journal of hygiene and environmental health, 217, 554-566.
                                     EXPO
                                      201
Prapamontol, T., Sutan, K., Laoyang, S., Hongsibsong, S., Lee, G., Yano, Y., et al. (2014a). Cross validation of gas chromatography-flame photometric detection and gas chromatography-mass spectrometry methods for measuring dialkylphosphate metabolites of organophosphate pesticides in human urine. International Journal of Hygiene and Environmental Health, 217(4-5), 554-566.
                                   DUPLICATE
                                      202
Prapamontol, T., Sutan, K., Laoyang, S., Hongsibsong, S., Lee, G., Yano, Y., et al. (2014b). Cross validation of gas chromatography-flame photometric detection and gas chromatography - mass spectrometry methods for measuring dialkylphosphate metabolites of organophosphate pesticides in human urine. International Journal of Hygiene and Environmental Health, 217, 554-566.
                                   DUPLICATE
                                      203
Quiros-Alcala, L., Alkon, A. D., Boyce, W. T., Lippert, S., Davis, N. V., Bradman, A., et al. (2011). Maternal prenatal and child organophosphate pesticide exposures and children's autonomic function. Neurotoxicology, 32(5), 646-655.
                                      EPI
                                      204
Quirós-Alcalá, L., Bradman, A., Smith, K., Weerasekera, G., Odetokun, M., Barr, D. B., et al. (2012). Organophosphorous pesticide breakdown products in house dust and children's urine. [Article]. Journal of Exposure Science and Environmental Epidemiology, 22, 559-568.
                                     EXPO
                                      205
Radwan, N. M., & El-Nabarawy, N. A. (2014). The role of muscle injury in organophosphate poisoning and development of intermediate syndrome. [Article]. International Journal of Pharmaceutical Sciences Review and Research, 28, 61-66.
                                      ACR
                                      206
Rajanandh, M. G., Santhosh, S., & Ramasamy, C. (2013). Prospective analysis of poisoning cases in a super specialty hospital in India. [Article]. Journal of Pharmacology and Toxicology, 8, 60-66.
                                      ACR
                                      207
Rajasekharan, C., Renjith, S. W., & Jayapal, T. (2012). Opsoclonus and lingual myoclonus due to organophosphate poisoning: images in clinical medicine. [Article]. BMJ case reports, 2012.
                                      ACR
                                      208
Ram, P., Kanchan, T., & Unnikrishnan, B. (2014). Pattern of acute poisonings in children below 15 years - A study from Mangalore, South India. [Article]. Journal of Forensic and Legal Medicine, 25, 26-29.
                                      ACR
                                      209
Ramanath, K. V., & Naveen Kumar, H. D. (2012). Study the assessment of poisoning cases in a rural tertiary care teaching hospital by a clinical pharmacist. [Article]. Asian Journal of Pharmaceutical and Clinical Research, 5, 138-141.
                                      ACR
                                      210
Rauch, S. A., Braun, J. M., Barr, D. B., Calafat, A. M., Khoury, J., Montesano, M. A., et al. (2012). Associations of prenatal exposure to organophosphate pesticide metabolites with gestational age and birth weight. [Article]. Environmental Health Perspectives, 120, 1055-1060.
                                      EPI
                                      211
Rauh, V. A., Garfinkel, R., Perera, F. P., Andrews, H. F., Hoepner, L., Barr, D. B., et al. (2006). Impact of prenatal chlorpyrifos exposure on neurodevelopment in the first 3 years of life among inner-city children. Pediatrics, 118(6), e1845-1859.
                                      EPI
                                      212
Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X. J., et al. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. [Article]. Proceedings of the National Academy of Sciences of the United States of America, 109(20), 7871-7876.
                                   DUPLICATE
                                      213
Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., et al. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. [Article]. Proceedings of the National Academy of Sciences of the United States of America, 109, 7871-7876.
                                   DUPLICATE
                                      214
Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., et al. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. Proc Natl Acad Sci U S A, 109(20), 7871-7876.
                                      EPI
                                      215
Rauh, V., Arunajadai, S., Horton, M., Perera, F., Hoepner, L., Barr, D. B., et al. (2011). Seven-year neurodevelopmental scores and prenatal exposure to chlorpyrifos, a common agricultural pesticide. Environ Health Perspect, 119(8), 1196-1201.
                                      EPI
                                      216
Redmond, E., Egeston, C., & Bralley, J. A. (2012). Low level prenatal exposure to organophosphate pesticides significantly lowers IQ in children. Townsend Letter(342), 58-61.
                                      REV
                                      217
Ridano, M. E., Racca, A. C., Flores-Martin, J., Camolotto, S. A., De Potas, G. M., Genti-Raimondi, S., et al. (2012). Chlorpyrifos modifies the expression of genes involved in human placental function. [article]. Reproductive toxicology (Elmsford, NY), 33, 331-338.
                                      TOX
                                      218
Ridano, M. E., Racca, A. C., Flores-Martín, J., Camolotto, S. A., de Potas, G. M., Genti-Raimondi, S., et al. (2012). Chlorpyrifos modifies the expression of genes involved in human placental function. Reproductive Toxicology, 33, 331-338.
                                   DUPLICATE
                                      219
Roberts, J. R., & Karr, C. J. (2012). Pesticide Exposure in Children. [Article]. Pediatrics, 130, E1765-E1788.
                                      REV
                                      220
Roca, M., Leon, N., Pastor, A., & Yusà, V. (2014). Comprehensive analytical strategy for biomonitoring of pesticides in urine by liquid chromatography-orbitrap high resolution mass spectrometry. [Article]. Journal of Chromatography A, 1374, 66-76.
                                     EXPO
                                      221
Roca, M., Miralles-Marco, A., Ferré, J., Pérez, R., & Yusà, V. (2014). Biomonitoring exposure assessment to contemporary pesticides in a school children population of Spain. [Article]. Environmental Research, 131, 77-85.
                                   DUPLICATE
                                      222
Roca, M., Miralles-Marco, A., Ferré, J., Pérez, R., & Yusà, V. (2014). Biomonitoring exposure assessment to contemporary pesticides in a school children population of Spain. [Article]. Environmental Research, 131, 77-85.
                                     EXPO
                                      223
Rodriguez, T., Van Wendel De Joode, B., Lindh, C. H., Rojas, M., Lundberg, I., & Wesseling, C. (2012). Assessment of long-term and recent pesticide exposure among rural school children in Nicaragua. [article]. Occupational and environmental medicine (London), 69, 119-125.
                                   DUPLICATE
                                      224
Rodriguez, T., van Wendel de Joode, B., Lindh, C. H., Rojas, M., Lundberg, I., & Wesseling, C. (2012). Assessment of long-term and recent pesticide exposure among rural school children in Nicaragua. Occup Environ Med, 69(2), 119-125.
                                     EXPO
                                      225
Rogers, T. D., McKimm, E., Dickson, P. E., Goldowitz, D., Blaha, C. D., & Mittleman, G. (2013). Is autism a disease of the cerebellum?: An integration of clinical and pre-clinical research. [Article]. Frontiers in Systems Neuroscience.
                                      REV
                                      226
Rohitrattana, J., Siriwong, W., Tunsaringkarn, T., Panuwet, P., Ryan, P. B., Barr, D. B., et al. (2014). Organophosphate pesticide exposure in school-aged children living in rice and aquacultural farming regions of Thailand. [Journal article]. Journal of Agromedicine, 19, 406-416.
                                     EXPO
                                      227
Rohlman, D. S., Arcury, T. A., Quandt, S. A., Lasarev, M., Rothlein, J., Travers, R., et al. (2005). Neurobehavioral performance in preschool children from agricultural and non-agricultural communities in Oregon and North Carolina. NeuroToxicology, 26(4 Spec. Iss.), 589-598.
                                      EPI
                                      228
Rohlman, D. S., Ismail, A. A., Abdel-Rasoul, G., Lasarev, M., Hendy, O., & Olson, J. R. (2014). Characterizing exposures and neurobehavioral performance in Egyptian adolescent pesticide applicators. [Article]. Metabolic Brain Disease, 29, 845-855.
                                      EPI
                                      229
Rohlman, D. S., Lasarev, M., Anger, W. K., Scherer, J., Stupfel, J., & McCauley, L. (2007a). Neurobehavioral performance of adult and adolescent agricultural workers. Neurotoxicology, 28(2), 374-380.
                                      EPI
                                      230
Rohlman, D. S., Lasarev, M., Anger, W. K., Scherer, J., Stupfel, J., & McCauley, L. (2007b). Neurobehavioral performance of adult and adolescent agricultural workers. Neurotoxicology, 28(2), 374-380.
                                   DUPLICATE
                                      231
Rojas, M., Agreda, O., & Infante, S. (2008). A preliminary statistical study of whether pesticide use could be related to birth defects in a rural area of Venezuela. Rev Salud Publica (Bogota), 10(1), 85-93.
                                      EPI
                                      232
Ross, M. K., Borazjani, A., Mangum, L. C., Wang, R., & Crow, J. A. (2014). Effects of toxicologically relevant xenobiotics and the lipid-derived electrophile 4-hydroxynonenal on macrophage cholesterol efflux: Silencing carboxylesterase 1 has paradoxical effects on cholesterol uptake and efflux. [Article]. Chemical Research in Toxicology, 27, 1743-1756.
                                      TOX
                                      233
Ross, S. M., McManus, I. C., Harrison, V., & Mason, O. (2013). Neurobehavioral problems following low-level exposure to organophosphate pesticides: a systematic and meta-analytic review. [Literature Review]. Critical Reviews in Toxicology, 43, 21-44.
                                      REV
                                      234
Roth, N., & Wilks, M. F. (2014). Neurodevelopmental and neurobehavioural effects of polybrominated and perfluorinated chemicals: A systematic review of the epidemiological literature using a quality assessment scheme. Toxicology Letters, 230, 271-281.
                                      REV
                                      235
Ruckart, P. Z., Kakolewski, K., Bove, F. J., & Kaye, W. E. (2004). Long-term neurobehavioral health effects of methyl parathion exposure in children in Mississippi and Ohio. [Comparative Study]. Environ Health Perspect, 112(1), 46-51.
                                      EPI
                                      236
Samarawickrema, N., Pathmeswaran, A., Wickremasinghe, R., Peiris-John, R., Karunaratna, M., Buckley, N., et al. (2008). Fetal effects of environmental exposure of pregnant women to organophosphorus compounds in a rural farming community in Sri Lanka. Clin Toxicol (Phila), 46(6), 489-495.
                                      EPI
                                      237
Santhosh, C. S., Kumar, S., & Nawaz, B. (2012). Profile of poisoning cases autopsied at district government hospital, Davangere. [Article]. Indian Journal of Forensic Medicine and Toxicology, 6, 104-106.
                                      ACR
                                      238
Savitz, D. A., Arbuckle, T., Kaczor, D., & Curtis, K. M. (1997). Male pesticide exposure and pregnancy outcome. Am J Epidemiol, 146(12), 1025-1036.
                                      EPI
                                      239
Schug, T. T., Barouki, R., Gluckman, P. D., Grandjean, P., Hanson, M., & Heindel, J. J. (2013). PPTOX III: Environmental stressors in the developmental origins of disease-Evidence and mechanisms. [Article]. Toxicological Sciences, 131, 343-350.
                                      REV
                                      240
Selvam, V., Panneer Selvam, G., & Vijayanath, V. (2012). Study of death incidence by insecticide poisoning in Salem. [Article]. International Journal of Medical Toxicology and Forensic Medicine, 2, 20-26.
                                      ACR
                                      241
Senarathna, L., Jayamanna, S. F., Kelly, P. J., Buckley, N. A., Dibley, M. J., & Dawson, A. H. (2012). Changing epidemiologic patterns of deliberate self poisoning in a rural district of Sri Lanka. [Article]. BMC public health, 12, 593.
                                      ACR
                                      242
Sexton, K., Salinas, J. J., McDonald, T. J., Gowen, R. M. Z., Miller, R. P., McCormick, J. B., et al. (2013). Biomarkers of maternal and fetal exposure to organochlorine pesticides measured in pregnant hispanic women from brownsville, texas. [Article]. International Journal of Environmental Research and Public Health, 10, 237-248.
                                     EXPO
                                      243
Shelton, J. F., Geraghty, E. M., Tancredi, D. J., Delwiche, L. D., Schmidt, R. J., Ritz, B., et al. (2014). Neurodevelopmental disorders and prenatal residential proximity to agricultural pesticides: The charge study. [Article]. Environmental Health Perspectives, 122, 1103-1109.
                                      EPI
                                      244
Shelton, J. F., Geraghty, E. M., Tancredi, D. J., Delwiche, L. D., Schmidt, R. J., Ritz, B., et al. (2014). Neurodevelopmental Disorders and Prenatal Residential Proximity to Agricultural Pesticides: The CHARGE Study. Environmental Health Perspectives, 122(10), 1103-1109.
                                   DUPLICATE
                                      245
Shomar, B., Al-Saad, K., & Nriagu, J. (2014). Mishandling and exposure of farm workers in Qatar to organophosphate pesticides. Environmental Research, 133, 312-320.
                                     EXPO
                                      246
Simescu, M., Igna, C. P., Nicolaescu, E., Ion, I., Ion, A. C., Caragheorgheopol, A., et al. (2014). MULTIPLE PESTICIDES EXPOSURE OF GREENHOUSE WORKERS AND THYROID PARAMETERS. [article]. International journal of sustainable development and planning (Print), 9, 15-28.
                                      EPI
                                      247
Simescu, M., Igna, C. P., Nicolaescu, E., Ion, I., Ion, A. C., Caragheorgheopol, A., et al. (2014). Multiple pesticides exposure of greenhouse workers and thyroid parameters. [Journal article]. International Journal of Sustainable Development and Planning, 9, 15-28.
                                   DUPLICATE
                                      248
Singh, A. K., Kashyap, M. P., Jahan, S., Kumar, V., Tripathi, V. K., Siddiqui, M. A., et al. (2012). Expression and inducibility of cytochrome P450s (CYP1A1, 2B6, 2E1, 3A4) in human cord blood CD34+ stem cell-derived differentiating neuronal cells. [Article]. Toxicological Sciences, 129, 392-410.
                                      TOX
                                      249
Slomski, A. (2012). Chronic mental health issues in children now loom larger than physical problems. [Short Survey]. JAMA - Journal of the American Medical Association, 308, 223-225.
                                      REV
                                      250
Slotkin, T. A., Card, J., & Seidler, F. J. (2013). Adverse benzoapyrene effects on neurodifferentiation are altered by other neurotoxicant coexposures: Interactions with dexamethasone, chlorpyrifos, or nicotine in PC12 cells. [Article]. Environmental Health Perspectives, 121, 825-831.
                                      TOX
                                      251
Slotkin, T. A., Card, J., & Seidler, F. J. (2014). Prenatal dexamethasone, as used in preterm labor, worsens the impact of postnatal chlorpyrifos exposure on serotonergic pathways. [Journal article]. Brain Research Bulletin, 100, 44-54.
                                      TOX
                                      252
Smith, J. N., Hinderliter, P. M., Timchalk, C., Bartels, M. J., & Poet, T. S. (2014). A human life-stage physiologically based pharmacokinetic and pharmacodynamic model for chlorpyrifos: Development and validation. [Article]. Regulatory Toxicology and Pharmacology, 69, 580-597.
                                      TOX
                                      253
Smith, J. N., Hinderliter, P. M., Timchalk, C., Bartels, M. J., & Poet, T. S. (2014). A human life-stage physiologically based pharmacokinetic and pharmacodynamic model for chlorpyrifos: Development and validation. Regulatory Toxicology and Pharmacology, 69(3), 580-597.
                                   DUPLICATE
                                      254
Snijder, C. A., Heederik, D., Pierik, F. H., Hofman, A., Jaddoe, V. W., Koch, H. M., et al. (2013). Fetal growth and prenatal exposure to bisphenol A: The generation R study. [Article]. Environmental Health Perspectives, 121, 393-396.
                                      EPI
                                      255
Stapleton, H. M., Misenheimer, J., Hoffman, K., & Webster, T. F. (2014). Flame retardant associations between children's handwipes and house dust. [Article]. Chemosphere, 116, 54-60.
                                     EXPO
                                      256
Suarez-Lopez, J. R., Himes, J. H., Jacobs Jr, D. R., Alexander, B. H., & Gunnar, M. R. (2013). Acetylcholinesterase activity and neurodevelopment in boys and girls. [Article]. Pediatrics, 132, e1649-e1658.
                                      EPI
                                      257
Suarez-Lopez, J. R., Jacobs Jr, D. R., Himes, J. H., & Alexander, B. H. (2013). Acetylcholinesterase activity, cohabitation with floricultural workers, and blood pressure in Ecuadorian children. [Article]. Environmental Health Perspectives, 121, 619-624.
                                      EPI
                                      258
Suarez-Lopez, J. R., Jacobs, D. R., Himes, J. H., Alexander, B. H., Lazovich, D., & Gunnar, M. (2012). Lower acetylcholinesterase activity among children living with flower plantation workers. [Article]. Environmental Research, 114, 53-59.
                                      EPI
                                      259
Tajima, S., Araki, A., Kawai, T., Tsuboi, T., Ait Bamai, Y., Yoshioka, E., et al. (2014). Detection and intake assessment of organophosphate flame retardants in house dust in Japanese dwellings. [Article]. Science of the Total Environment, 478, 190-199.
                                     EXPO
                                      260
Tajima, S., Araki, A., Kawai, T., Tsuboi, T., Bamai, Y. A., Yoshioka, E., et al. (2014). Detection and intake assessment of organophosphate flame retardants in house dust in Japanese dwellings. [Journal article]. Science of the Total Environment, 478, 190-199.
                                   DUPLICATE
                                      261
Tan, J., Loganath, A., Chong, Y. S., & Obbard, J. P. (2009). Exposure to persistent organic pollutants in utero and related maternal characteristics on birth outcomes: A multivariate data analysis approach. Chemosphere, 74(3), 428-433.
                                      EPI
                                      262
Taxvig, C., Rosenmai, A. K., & Vinggaard, A. M. (2014). Polyfluorinated alkyl phosphate ester surfactants - current knowledge and knowledge gaps. [Article]. Basic and Clinical Pharmacology and Toxicology, 115, 41-44.
                                      REV
                                      263
Thompson, B., Griffith, W. C., Barr, D. B., Coronado, G. D., Vigoren, E. M., & Faustman, E. M. (2014). Variability in the take-home pathway: farmworkers and non-farmworkers and their children. [Journal article]. Journal of Exposure Science and Environmental Epidemiology, 24, 522-531.
                                     EXPO
                                      264
Torres-Sanchez, L., Rothenberg, S. J., Schnaas, L., Cebrian, M. E., Osorio, E., Del Carmen Hernandez, M., et al. (2007). In utero p,p'-DDE exposure and infant neurodevelopment: A perinatal cohort in Mexico. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't]. Environ Health Perspect, 115(3), 435-439.
                                      EPI
                                      265
Torres-Sanchez, L., Schnaas, L., Cebrian, M. E., Hernandez Mdel, C., Valencia, E. O., Garcia Hernandez, R. M., et al. (2009). Prenatal dichlorodiphenyldichloroethylene (DDE) exposure and neurodevelopment: a follow-up from 12 to 30 months of age. [Research Support, Non-U.S. Gov't]. NeuroToxicology, 30(6), 1162-1165.
                                      EPI
                                      266
Trairatvorakul, C., & Detsomboonrat, P. (2012). Success rates of a mixture of ciprofloxacin, metronidazole, and minocycline antibiotics used in the non-instrumentation endodontic treatment of mandibular primary molars with carious pulpal involvement. [Article]. International Journal of Paediatric Dentistry, 22, 217-227.
                                      OTH
                                      267
Trunnelle, K. J., Bennett, D. H., Tulve, N. S., Clifton, M. S., Davis, M. D., Calafat, A. M., et al. (2014). Urinary pyrethroid and chlorpyrifos metabolite concentrations in northern California families and their relationship to indoor residential insecticide levels, part of the Study of Use of Products and Exposure Related Behavior (SUPERB). [Article]. Environmental Science and Technology, 48, 1931-1939.
                                   DUPLICATE
                                      268
Trunnelle, K. J., Bennett, D. H., Tulve, N. S., Clifton, M. S., Davis, M. D., Calafat, A. M., et al. (2014). Urinary pyrethroid and chlorpyrifos metabolite concentrations in northern California families and their relationship to indoor residential insecticide levels, part of the Study of Use of Products and Exposure Related Behavior (SUPERB). [Journal article]. Environmental Science &amp; Technology, 48, 1931-1939.
                                   DUPLICATE
                                      269
Trunnelle, K. J., Bennett, D. H., Tulve, N. S., Clifton, M. S., Davis, M. D., Calafat, A. M., et al. (2014). Urinary Pyrethroid and Chlorpyrifos Metabolite Concentrations in Northern California Families and Their Relationship to Indoor Residential Insecticide Levels, Part of the Study of Use of Products and Exposure Related Behavior (SUPERB). [Article]. Environmental Science & Technology, 48(3), 1931-1939.
                                     EXPO
                                      270
Ueyama, J., Saito, I., Takaishi, A., Nomura, H., Inoue, M., Osaka, A., et al. (2014). A revised method for determination of dialkylphosphate levels in human urine by solid-phase extraction and liquid chromatography with tandem mass spectrometry: application to human urine samples from Japanese children. [Journal article]. Environmental Health and Preventive Medicine, 19, 405-413.
                                     EXPO
                                      271
van Balen, E. C., Wolansky, M. J., & Kosatsky, T. (2012). Increasing use of pyrethroids in Canadian households: Should we be concerned? [Note]. Canadian Journal of Public Health, 103, 404-407.
                                      REV
                                      272
Van Dyke, M., Martyny, J. W., & Serrano, K. A. (2014). Methamphetamine Residue Dermal Transfer Efficiencies from Household Surfaces. [Article]. Journal of Occupational and Environmental Hygiene, 11(4), 249-258.
                                     EXPO
                                      273
Van Thriel, C., Westerink, R. H. S., Beste, C., Bale, A. S., Lein, P. J., & Leist, M. (2012). Translating neurobehavioural endpoints of developmental neurotoxicity tests into in vitro assays and readouts. [Article]. NeuroToxicology, 33, 911-924.
                                      REV
                                      274
Van Wendel De Joode, B., Barraza, D., Ruepert, C., Mora, A. M., Cordoba, L., ÖBerg, M., et al. (2012). Indigenous children living nearby plantations with chlorpyrifos-treated bags have elevated 3,5,6-trichloro-2-pyridinol (TCPy) urinary concentrations. [article]. Environmental research (New York, N.Y. : Print), 117, 17-26.
                                   DUPLICATE
                                      275
Veale, D. J. H., Wium, C. A., & Müller, G. J. (2013). Toxicovigilance I: A survey of acute poisonings in South Africa based on Tygerberg Poison Information centre data. [Article]. South African Medical Journal, 103, 293-297.
                                      ACR
                                      276
Venerosi, A., Ricceri, L., Tait, S., & Calamandrei, G. (2012). Sex dimorphic behaviors as markers of neuroendocrine disruption by environmental chemicals: The case of chlorpyrifos. [Article]. NeuroToxicology, 33, 1420-1426.
                                      TOX
                                      277
Vera, B., Cruz, S. S., & Magnarelli, G. (2012). Plasma cholinesterase and carboxylesterase activities and nuclear and mitochondrial lipid composition of human placenta associated with maternal exposure to pesticides. [Article]. Reproductive Toxicology, 34(3), 402-407.
                                      TOX
                                      278
Vera, B., Santa Cruz, S., & Magnarelli, G. (2012). Plasma cholinesterase and carboxylesterase activities and nuclear and mitochondrial lipid composition of human placenta associated with maternal exposure to pesticides. [article]. Reproductive toxicology (Elmsford, NY), 34, 402-407.
                                   DUPLICATE
                                      279
Vogt, R., Bennett, D., Cassady, D., Frost, J., Ritz, B., & Hertz-Picciotto, I. (2012). Cancer and non-cancer health effects from food contaminant exposures for children and adults in California: A risk assessment. [Article]. Environmental Health: A Global Access Science Source, 11.
                                      REV
                                      280
Wang, P., Tian, Y., Wang, X. J., Gao, Y., Shi, R., Wang, G. Q., et al. (2011). Organophosphate pesticide exposure and perinatal outcomes in Shanghai, China. Environ Int.
                                   DUPLICATE
                                      281
Wang, P., Tian, Y., Wang, X.-J., Gao, Y., Shi, R., Wang, G.-Q., et al. (2012). Organophosphate pesticide exposure and perinatal outcomes in Shanghai, China. [Article]. Environment International, 42, 100-104.
                                      EPI
                                      282
Wason, S. C., Julien, R., Perry, M. J., Smith, T. J., & Levy, J. I. (2013). Modeling exposures to organophosphates and pyrethroids for children living in an urban low-income environment. [Article]. Environmental Research, 124, 13-22.
                                     EXPO
                                      283
Wason, S. C., Smith, T. J., Perry, M. J., & Levy, J. I. (2012). Using physiologically-based pharmacokinetic models to incorporate chemical and non-chemical stressors into cumulative risk assessment: A case study of pesticide exposures. [Article]. International Journal of Environmental Research and Public Health, 9, 1971-1983.
                                     EXPO
                                      284
Whyatt, R. M., Rauh, V., Barr, D. B., Camann, D. E., Andrews, H. F., Garfinkel, R., et al. (2004). Prenatal insecticide exposures and birth weight and length among an urban minority cohort. Environ Health Perspect, 112(10), 1125-1132.
                                      EPI
                                      285
Wickerham, E. L., Lozoff, B., Jie, S., Kaciroti, N., Yankai, X. I. A., & Meeker, J. D. (2012). Reduced birth weight in relation to pesticide mixtures detected in cord blood of full-term infants. [article]. Environment international, 47, 80-85.
                                      EPI
                                      286
Wickerham, E. L., Lozoff, B., Shao, J., Kaciroti, N., Xia, Y. K., & Meeker, J. D. (2012). Reduced birth weight in relation to pesticide mixtures detected in cord blood of full-term infants. [Article]. Environment International, 47, 80-85.
                                   DUPLICATE
                                      287
Williams, A. L., & DeSesso, J. M. (2014). Gestational/ Perinatal chlorpyrifos exposure is not associated with autistic-like behaviors in rodents. [Literature Review]. Critical Reviews in Toxicology, 44, 523-534.
                                      REV
                                      288
Wolff, M. S., Engel, S., Berkowitz, G., Teitelbaum, S., Siskind, J., Barr, D. B., et al. (2007). Prenatal pesticide and PCB exposures and birth outcomes. Pediatr Res, 61(2), 243-250.
                                      EPI
                                      289
Xiang, H., Nuckols, J. R., & Stallones, L. (2000). A geographic information assessment of birth weight and crop production patterns around mother's residence. Environ Res, 82(2), 160-167.
                                      EPI
                                      290
Ye, X., Pierik, F. H., Hauser, R., Duty, S., Angerer, J., Park, M. M., et al. (2008). Urinary metabolite concentrations of organophosphorous pesticides, bisphenol A, and phthalates among pregnant women in Rotterdam, the Netherlands: The Generation R study. [Research Support, N.I.H., Extramural]. Environ Res, 108(2), 260-267.
                                     EXPO
                                      291
Yolton, K., Cornelius, M., Ornoy, A., McGough, J., Makris, S., & Schantz, S. (2014). Exposure to neurotoxicants and the development of attention deficit hyperactivity disorder and its related behaviors in childhood. [Journal article]. Neurotoxicology and Teratology, 44, 30-45.
                                      REV
                                      292
Young, J. G., Eskenazi, B., Gladstone, E. A., Bradman, A., Pedersen, L., Johnson, C., et al. (2005). Association between in utero organophosphate pesticide exposure and abnormal reflexes in neonates. Neurotoxicology, 26(2), 199-209.
                                      EPI
                                      293
Yuan, Y., Chen, C., Zheng, C., Wang, X., Yang, G., Wang, Q., et al. (2014). Residue of chlorpyrifos and cypermethrin in vegetables and probabilistic exposure assessment for consumers in Zhejiang Province, China. [Article]. Food Control, 36, 63-68.
                                     EXPO
                                      294
Yusà, V., Coscollà, C., & Millet, M. (2014). New screening approach for risk assessment of pesticides in ambient air. [Article]. Atmospheric Environment, 96, 322-330.
                                      REV
                                      295
Zhang, X., Wallace, A. D., Du, P., Lin, S., Baccarelli, A. A., Jiang, H., et al. (2012). Genome-wide study of DNA methylation alterations in response to diazinon exposure in vitro. [Article]. Environmental Toxicology and Pharmacology, 34, 959-968.
                                      TOX
                                      296
Zhang, Y., Han, S., Liang, D., Shi, X., Wang, F., Liu, W., et al. (2014). Prenatal exposure to organophosphate pesticides and neurobehavioral development of neonates: A birth cohort study in Shenyang, China. [Article]. PLoS ONE, 9.
                                      EPI
                                      297
Zhang, Y., Song, H., Liang, D., Shi, X., Wang, F., Liu, W., et al. (2014). Prenatal exposure to organophosphate pesticides and neurobehavioral development of neonates: a birth cohort study in Shenyang, China. [Journal article]. PLoS ONE, 9, e88491.
                                   DUPLICATE
                                      298
Zhao, Q., Gadagbui, B., & Dourson, M. (2005). Lower birth weight as a critical effect of chlorpyrifos: A comparison of human and animal data. Regul Toxicol Pharmacol, 42(1), 55-63.
                                      REV
                                      299
Zhou, S., Rosenthal, D. G., Sherman, S., Zelikoff, J., Gordon, T., & Weitzman, M. (2014). Physical, Behavioral, and Cognitive Effects of Prenatal Tobacco and Postnatal Secondhand Smoke Exposure. Current Problems in Pediatric and Adolescent Health Care, 44, 219-241.
                                      REV
 
Appendix 5.	Table of Systematic Review Analysis:  Second Tier Exclusion Criteria.
Among the 79 potentially relevant epidemiologic studies, 41 were excluded; 17 articles were previously reviewed in 2012; 16 were epidemiological methods papers including exposure validation studies without an original epidemiological risk estimate; and 8 were otherwise not relevant for various reasons. Among the 40 remaining studies, 2 were additionally excluded (one was a duplicate study published a second time; the other did not make a measure of an OP pesticide. Therefore, 38 articles are included in the 2015 literature review.
                                    Number
Citations (Of 79: 17 - part of 3 cohorts, previously reviewed 24-excluded (NOT OP, NOT ND, NOT EPI) 38-include (9 no direct OP measure)
Category (EPI-epi only; EXPO- exposure only; ACR - acute case report or case series; TOX-animal study; REV - review or commentary, no original data; OTH - other, non-English
Include? Outcome==pediatric ND/NB/motor control/morphology/motor control; Exposure==OP (biomarker, questionnaire, env media); epi study (cohort/case control/cross sectional (not ecologic))
                              Include? Rationale
                                       1
Burdorf, A., Brand, T., Jaddoe, V. W., Hofman, A., Mackenbach, J. P., & Steegers, E. A. (2011). The effects of work-related maternal risk factors on time to pregnancy, preterm birth and birth weight: the Generation R Study. Occup Environ Med, 68(3), 197-204.
                                      EPI
                                      NO
                           NO OP; ALL PESTICIDE ONLY
                                       2
Xiang, H., Nuckols, J. R., & Stallones, L. (2000). A geographic information assessment of birth weight and crop production patterns around mother's residence. Environ Res, 82(2), 160-167.
                                      EPI
                                      NO
                             No pesticide measure
                                       3
Ferreira, J. D., Couto, A. C. z., Pombo-de-Oliveira, M. S., & Koifman, S. (2013). In utero pesticide exposure and leukemia in Brazilian children < 2 years of age. [Note]. Environmental Health Perspectives, 121, 269-275.
                                      EPI
                                      NO
                                    NOT ND
                                       4
How, V., Hashim, Z., Ismail, P., Said, S. M., Dzolkhifli, O., & Shamsul Bahri, M. T. (2014). Exploring cancer development in adulthood: cholinesterase depression and genotoxic effect from chronic exposure to organophosphate pesticides among rural farm children. [Journal article]. Journal of Agromedicine, 19, 35-43.
                                      EPI
                                      NO
                                    not ND
                                       5
Huen, K., Harley, K., Beckman, K., Eskenazi, B., & Holland, N. (2013). Associations of PON1 and Genetic Ancestry with Obesity in Early Childhood. [Article]. PLoS ONE, 8.
                                      EPI
                                      NO
                                    not ND
                                       6
Jones, K., Everard, M., & Harding, A. H. (2014). Investigation of gastrointestinal effects of organophosphate and carbamate pesticide residues on young children. [Journal article]. International Journal of Hygiene and Environmental Health, 217, 392-398.
                                      EPI
                                      NO
                                    not ND
                                       7
Khan, K., Ismail, A. A., Abdel Rasoul, G., Bonner, M. R., Lasarev, M. R., Hendy, O., et al. (2014). Longitudinal assessment of chlorpyrifos exposure and self-reported neurological symptoms in adolescent pesticide applicators. [Article]. BMJ Open, 4.
                                      EPI
                                      NO
                                    not ND
                                       8
Michalakis, M., Tzatzarakis, M. N., Kovatsi, L., Alegakis, A. K., Tsakalof, A. K., Heretis, I., et al. (2014). Hypospadias in offspring is associated with chronic exposure of parents to organophosphate and organochlorine pesticides. [Article]. Toxicology Letters, 230, 139-145.
                                      EPI
                                      NO
                                    not ND
                                       9
Rohlman, D. S., Ismail, A. A., Abdel-Rasoul, G., Lasarev, M., Hendy, O., & Olson, J. R. (2014). Characterizing exposures and neurobehavioral performance in Egyptian adolescent pesticide applicators. [Article]. Metabolic Brain Disease, 29, 845-855.
                                      EPI
                                      NO
                                    not ND
                                      10
Rojas, M., Agreda, O., & Infante, S. (2008). A preliminary statistical study of whether pesticide use could be related to birth defects in a rural area of Venezuela. Rev Salud Publica (Bogota), 10(1), 85-93.
                                      EPI
                                      NO
                                    not ND
                                      11
Simescu, M., Igna, C. P., Nicolaescu, E., Ion, I., Ion, A. C., Caragheorgheopol, A., et al. (2014). MULTIPLE PESTICIDES EXPOSURE OF GREENHOUSE WORKERS AND THYROID PARAMETERS. [article]. International journal of sustainable development and planning (Print), 9, 15-28.
                                      EPI
                                      NO
                                    not ND
                                      12
Cecchi, A., Rovedatti, M. G., Sabino, G., & Magnarelli, G. G. (2012a). Environmental exposure to organophosphate pesticides: Assessment of endocrine disruption and hepatotoxicity in pregnant women. [Article]. Ecotoxicology and Environmental Safety, 80, 280-287.
                                      EPI
                                      NO
  not ND directly; maternal ED/thyroid change->adverse fetal (unmeasured)
                                      13
Rohlman, D. S., Lasarev, M., Anger, W. K., Scherer, J., Stupfel, J., & McCauley, L. (2007a). Neurobehavioral performance of adult and adolescent agricultural workers. Neurotoxicology, 28(2), 374-380.
                                      EPI
                                      NO
                                not ND, not OP
                                      14
Carmichael, S. L., Yang, W., Roberts, E. M., Kegley, S. E., Wolff, C., Guo, L., et al. (2013). Hypospadias and residential proximity to pesticide applications. [Article]. Pediatrics, 132, e1216-e1226.
                                      EPI
                                      NO
                   not ND; measured all pesticides, some OP 
                                      15
Carmichael, S. L., Yang, W., Roberts, E., Kegley, S. E., Padula, A. M., English, P. B., et al. (2014). Residential agricultural pesticide exposures and risk of selected congenital heart defects among offspring in the San Joaquin Valley of California. [Article]. Environmental Research, 135, 133-138.
                                      EPI
                                      NO
                   not ND; measured all pesticides, some OP 
                                      16
Eskenazi, B., Chevrier, J., Rauch, S. A., Kogut, K., Harley, K. G., Johnson, C., et al. (2013). In Utero and Childhood Polybrominated Diphenyl Ether (PBDE) Exposures and Neurodevelopment in the CHAMACOS Study. [Article]. Environmental Health Perspectives, 121(2), 257-262.
                                      EPI
                                      NO
                                    not OP
                                      17
Kezios, K. L., Liu, X. H., Cirillo, P. M., Cohn, B. A., Kalantzi, O. I., Wang, Y. Z., et al. (2013). Dichlorodiphenyltrichloroethane (DDT), DDT metabolites and pregnancy outcomes. [Article]. Reproductive Toxicology, 35, 156-164.
                                      EPI
                                      NO
                                    not OP
                                      18
Kicinski, M., Vrijens, J., Vermier, G., Hond, E. D., Schoeters, G., Nelen, V., et al. (2015). Neurobehavioral function and low-level metal exposure in adolescents. International Journal of Hygiene and Environmental Health, 218, 139-146.
                                      EPI
                                      NO
                                    not OP
                                      19
Ostrea, E. M., Reyes, A., Villanueva-Uy, E., Pacifico, R., Benitez, B., Ramos, E., et al. (2011). Fetal exposure to propoxur and abnormal child neurodevelopment at 2 years of age. NeuroToxicology.
                                      EPI
                                      NO
                                    not OP
                                      20
Snijder, C. A., Heederik, D., Pierik, F. H., Hofman, A., Jaddoe, V. W., Koch, H. M., et al. (2013). Fetal growth and prenatal exposure to bisphenol A: The generation R study. [Article]. Environmental Health Perspectives, 121, 393-396.
                                      EPI
                                      NO
                                    not OP
                                      21
Tan, J., Loganath, A., Chong, Y. S., & Obbard, J. P. (2009). Exposure to persistent organic pollutants in utero and related maternal characteristics on birth outcomes: A multivariate data analysis approach. Chemosphere, 74(3), 428-433.
                                      EPI
                                      NO
                                    not OP
                                      22
Torres-Sanchez, L., Rothenberg, S. J., Schnaas, L., Cebrian, M. E., Osorio, E., Del Carmen Hernandez, M., et al. (2007). In utero p,p'-DDE exposure and infant neurodevelopment: A perinatal cohort in Mexico. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't]. Environ Health Perspect, 115(3), 435-439.
                                      EPI
                                      NO
                                    not OP
                                      23
Torres-Sanchez, L., Schnaas, L., Cebrian, M. E., Hernandez Mdel, C., Valencia, E. O., Garcia Hernandez, R. M., et al. (2009). Prenatal dichlorodiphenyldichloroethylene (DDE) exposure and neurodevelopment: a follow-up from 12 to 30 months of age. [Research Support, Non-U.S. Gov't]. NeuroToxicology, 30(6), 1162-1165.
                                      EPI
                                      NO
                                    not OP
                                      24
Acosta-Maldonado, B., Sanchez-Ramirez, B., Reza-Lopez, S., & Levario-Carrillo, M. (2009). Effects of exposure to pesticides during pregnancy on placental maturity and weight of newborns: a cross-sectional pilot study in women from the Chihuahua State, Mexico. Hum Exp Toxicol, 28(8), 451-459.
                                      EPI
                                      YES
                                       
                                      25
Andersen, H. R., Debes, F., Wohlfahrt-Veje, C., Murata, K., & Grandjean, P. (2015). Occupational pesticide exposure in early pregnancy associated with sex-specific neurobehavioral deficits in the children at school age. Neurotoxicology and Teratology, 47, 1-9.
                                      EPI
                                      YES
                                       
                                      26
Barr, D. B., Ananth, C. V., Yan, X., Lashley, S., Smulian, J. C., Ledoux, T. A., et al. (2010). Pesticide concentrations in maternal and umbilical cord sera and their relation to birth outcomes in a population of pregnant women and newborns in New Jersey. Sci Total Environ, 408(4), 790-795.
                                      EPI
                                      YES
                                       
                                      27
Berkowitz, G. S., Wetmur, J. G., Birman-Deych, E., Obel, J., Lapinski, R. H., Godbold, J. H., et al. (2004). In utero pesticide exposure, maternal paraoxonase activity, and head circumference. Environ Health Perspect, 112(3), 388-391.
                                      EPI
                                 YES-3COHORTS
                                       
                                      28
Bouchard, M. F., Bellinger, D. C., Wright, R. O., & Weisskopf, M. G. (2010). Attention-deficit/hyperactivity disorder and urinary metabolites of organophosphate pesticides. [Research Support, N.I.H., Extramural]. Pediatrics, 125(6), e1270-1277.
                                      EPI
                                      YES
                                       
                                      29
Bouchard, M. F., Chevrier, J., Harley, K. G., Kogut, K., Vedar, M., Calderon, N., et al. (2011). Prenatal exposure to organophosphate pesticides and IQ in 7-year-old children. Environ Health Perspect, 119(8), 1189-1195.
                                      EPI
                                 YES-3COHORTS
                                       
                                      30
Dabrowski, S., Hanke, W., Polanska, K., Makowiec-Dabrowska, T., & Sobala, W. (2003). Pesticide exposure and birthweight: an epidemiological study in Central Poland. Int J Occup Med Environ Health, 16(1), 31-39.
                                      EPI
                                      YES
                                       
                                      31
Ding, G. D., Wang, P., Tian, Y., Zhang, J., Gao, Y., Wang, X. J., et al. (2012). Organophosphate Pesticide Exposure and Neurodevelopment in Young Shanghai Children. [Article]. Environmental Science & Technology, 46(5), 2911-2917.
                                      EPI
                                      YES
                                       
                                      32
Engel, S. M., Berkowitz, G. S., Barr, D. B., Teitelbaum, S. L., Siskind, J., Meisel, S. J., et al. (2007). Prenatal organophosphate metabolite and organochlorine levels and performance on the Brazelton Neonatal Behavioral Assessment Scale in a multiethnic pregnancy cohort. Am J Epidemiol, 165(12), 1397-1404.
                                      EPI
                                 YES-3COHORTS
                                       
                                      33
Engel, S. M., Wetmur, J., Chen, J., Zhu, C., Barr, D. B., Canfield, R. L., et al. (2011). Prenatal exposure to organophosphates, paraoxonase 1, and cognitive development in childhood. Environ Health Perspect, 119(8), 1182-1188.
                                      EPI
                                 YES-3COHORTS
                                       
                                      34
Eskenazi, B., Harley, K., Bradman, A., Weltzien, E., Jewell, N. P., Barr, D. B., et al. (2004). Association of in utero organophosphate pesticide exposure and fetal growth and length of gestation in an agricultural population. Environ Health Perspect, 112(10), 1116-1124.
                                      EPI
                                 YES-3COHORTS
                                       
                                      35
Eskenazi, B., Huen, K., Marks, A., Harley, K. G., Bradman, A., Barr, D. B., et al. (2010). PON1 and neurodevelopment in children from the CHAMACOS study exposed to organophosphate pesticides in utero. Environ Health Perspect, 118(12), 1775-1781.
                                      EPI
                                 YES-3COHORTS
                                       
                                      36
Eskenazi, B., Kogut, K., Huen, K., Harley, K. G., Bouchard, M., Bradman, A., et al. (2014a). Organophosphate pesticide exposure, PON1, and neurodevelopment in school-age children from the CHAMACOS study. Environmental Research, 134, 149-157.
                                      EPI
                                 YES-3COHORTS
                                       
                                      37
Eskenazi, B., Marks, A. R., Bradman, A., Harley, K., Barr, D. B., Johnson, C., et al. (2007). Organophosphate pesticide exposure and neurodevelopment in young Mexican-American children. Environ Health Perspect, 115(5), 792-798.
                                      EPI
                                 YES-3COHORTS
                                       
                                      38
Fortenberry, G. Z., Meeker, J. D., Sanchez, B. N., Barr, D. B., Panuwet, P., Bellinger, D., et al. (2014). Urinary 3,5,6-trichloro-2-pyridinol (TCPY) in pregnant women from Mexico City: distribution, temporal variability, and relationship with child attention and hyperactivity. Int J Hyg Environ Health, 217(2-3), 405-412.
                                      EPI
                                      YES
                                       
                                      39
Fortenberry, G. Z., Meeker, J. D., Sánchez, B. N., Bellinger, D., Peterson, K., Schnaas, L., et al. (2014). Paraoxonase I polymorphisms and attention/hyperactivity in school-age children from Mexico City, Mexico. [Article]. Environmental Research, 132, 342-349.
                                      EPI
                                      YES
                                       
                                      40
Furlong, M. A., Engel, S. M., Barr, D. B., & Wolff, M. S. (2014). Prenatal exposure to organophosphate pesticides and reciprocal social behavior in childhood. Environ Int, 70, 125-131.
                                      EPI
                                      YES
                                       
                                      41
Grandjean, P., Harari, R., Barr, D. B., & Debes, F. (2006). Pesticide exposure and stunting as independent predictors of neurobehavioral deficits in Ecuadorian school children. [Research Support, Non-U.S. Gov't]. Pediatrics, 117(3), e546-556.
                                      EPI
                                      YES
                                       
                                      42
Guodong, D., Pei, W., Ying, T., Jun, Z., Yu, G., Xiaojin, W., et al. (2012). Organophosphate pesticide exposure and neurodevelopment in young Shanghai children. Environ Sci Technol.
                                      EPI
                                      YES
                                       
                                      43
Handal, A. J., Harlow, S. D., Breilh, J., & Lozoff, B. (2008). Occupational exposure to pesticides during pregnancy and neurobehavioral development of infants and toddlers. Epidemiology, 19(6), 851-859.
                                      EPI
                                      YES
                                       
                                      44
Handal, A. J., Lozoff, B., Breilh, J., & Harlow, S. D. (2007a). Effect of community of residence on neurobehavioral development in infants and young children in a flower-growing region of Ecuador. [Research Support, N.I.H., Extramural Research Support, Non-U.S. Gov't]. Environ Health Perspect, 115(1), 128-133.
                                      EPI
                                      YES
                                       
                                      45
Handal, A. J., Lozoff, B., Breilh, J., & Harlow, S. D. (2007b). Neurobehavioral development in children with potential exposure to pesticides. Epidemiology, 18(3), 312-320.
                                      EPI
                                      YES
                                       
                                      46
Harari, R., Julvez, J., Murata, K., Barr, D., Bellinger, D. C., Debes, F., et al. (2010). Neurobehavioral deficits and increased blood pressure in school-age children prenatally exposed to pesticides. Environ Health Perspect, 118(6), 890-896.
                                      EPI
                                      YES
                                       
                                      47
Harley, K. G., Huen, K., Schall, R. A., Holland, N. T., Bradman, A., Barr, D. B., et al. (2011). Association of organophosphate pesticide exposure and paraoxonase with birth outcome in Mexican-American women. PLoS One, 6(8), e23923.
                                      EPI
                                 YES-3COHORTS
                                       
                                      48
Horton, M. K., Kahn, L. G., Perera, F., Barr, D. B., & Rauh, V. (2012). Does the home environment and the sex of the child modify the adverse effects of prenatal exposure to chlorpyrifos on child working memory? Neurotoxicol Teratol, 34(5), 534-541.
                                      EPI
                                 YES-3COHORTS
                                       
                                      49
Kofman, O., Berger, A., Massarwa, A., Friedman, A., & Jaffar, A. A. (2006). Motor inhibition and learning impairments in school-aged children following exposure to organophosphate pesticides in infancy. Pediatr Res, 60(1), 88-92.
                                      EPI
                                      YES
NO: LONG TERM EFFECTS OF  ONE TIME POISONING
                                      50
Koutroulakis, D., Sifakis, S., Tzatzarakis, M. N., Alegakis, A. K., Theodoropoulou, E., Kavvalakis, M. P., et al. (2014). Dialkyl phosphates in amniotic fluid as a biomarker of fetal exposure to organophosphates in Crete, Greece; association with fetal growth. [Article]. Reproductive Toxicology, 46, 98-105.
                                      EPI
                                      YES
                                       
                                      51
Kristensen, P., Irgens, L. M., Andersen, A., Bye, A. S., & Sundheim, L. (1997). Gestational age, birth weight, and perinatal death among births to Norwegian farmers, 1967-1991. Am J Epidemiol, 146(4), 329-338.
                                      EPI
                                      YES
                                       
                                      52
Lizardi, P. S., O'Rourke, M. K., & Morris, R. J. (2008). The effects of organophosphate pesticide exposure on Hispanic children's cognitive and behavioral functioning. J Pediatr Psychol, 33(1), 91-101.
                                      EPI
                                      YES
                                       
                                      53
Llop, S., Julvez, J., Fernandez-Somoano, A., Santa Marina, L., Vizcaino, E., Iñiguez, C., et al. (2013). Prenatal and postnatal insecticide use and infant neuropsychological development in a multicenter birth cohort study. Environment International, 59, 175-182.
                                      EPI
                                      YES
                                       
                                      54
Lovasi, G. S., Quinn, J. W., Rauh, V. A., Perera, F. P., Andrews, H. F., Garfinkel, R., et al. (2011). Chlorpyrifos exposure and urban residential environment characteristics as determinants of early childhood neurodevelopment. Am J Public Health, 101(1), 63-70.
                                      EPI
                                 YES-3COHORTS
                                       
                                      55
Lu, C., Essig, C., Root, C., Rohlman, D. S., McDonald, T., & Sulzbacher, S. (2009). Assessing the association between pesticide exposure and cognitive development in rural Costa Rican children living in organic and conventional coffee farms. Int J Adolesc Med Health, 21(4), 609-621.
                                      EPI
                                      YES
                                       
                                      56
Marks, A. R., Harley, K., Bradman, A., Kogut, K., Barr, D. B., Johnson, C., et al. (2010). Organophosphate pesticide exposure and attention in young Mexican-American children: the CHAMACOS study. Environ Health Perspect, 118(12), 1768-1774.
                                      EPI
                                 YES-3COHORTS
                                       
                                      57
Moreno-Banda, G., Blanco-Munoz, J., Lacasana, M., Rothenberg, S. J., Aguilar-Garduno, C., Gamboa, R., et al. (2009). Maternal exposure to floricultural work during pregnancy, PON1 Q192R polymorphisms and the risk of low birth weight. Sci Total Environ, 407(21), 5478-5485.
                                      EPI
                                      YES
                                       
                                      58
Nevison, C. D. (2014). A comparison of temporal trends in United States autism prevalence to trends in suspected environmental factors. [Article]. Environmental Health: A Global Access Science Source, 13.
                                      EPI
                                      YES
NO: this is a review; ecol comparison
                                      59
Oulhote, Y., & Bouchard, M. F. (2013). Urinary metabolites of organophosphate and pyrethroid pesticides and behavioral problems in Canadian children. [Article]. Environmental Health Perspectives, 121, 1378-1384.
                                      EPI
                                      YES
                                       
                                      60
Petit, C., Chevrier, C., Durand, G., Monfort, C., Rouget, F., Garlantezec, R., et al. (2010). Impact on fetal growth of prenatal exposure to pesticides due to agricultural activities: A prospective cohort study in Brittany, France. [Research Support, Non-U.S. Gov't]. Environ Health, 9, 71.
                                      EPI
                                      YES
                                       
                                      61
Quiros-Alcala, L., Alkon, A. D., Boyce, W. T., Lippert, S., Davis, N. V., Bradman, A., et al. (2011). Maternal prenatal and child organophosphate pesticide exposures and children's autonomic function. Neurotoxicology, 32(5), 646-655.
                                      EPI
                                      YES
                                       
                                      62
Rauch, S. A., Braun, J. M., Barr, D. B., Calafat, A. M., Khoury, J., Montesano, M. A., et al. (2012). Associations of prenatal exposure to organophosphate pesticide metabolites with gestational age and birth weight. [Article]. Environmental Health Perspectives, 120, 1055-1060.
                                      EPI
                                      YES
                                       
                                      63
Rauh, V. A., Garfinkel, R., Perera, F. P., Andrews, H. F., Hoepner, L., Barr, D. B., et al. (2006). Impact of prenatal chlorpyrifos exposure on neurodevelopment in the first 3 years of life among inner-city children. Pediatrics, 118(6), e1845-1859.
                                      EPI
                                 YES-3COHORTS
                                       
                                      64
Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., et al. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. Proc Natl Acad Sci U S A, 109(20), 7871-7876.
                                      EPI
                                 YES-3COHORTS
                                       
                                      65
Rauh, V., Arunajadai, S., Horton, M., Perera, F., Hoepner, L., Barr, D. B., et al. (2011). Seven-year neurodevelopmental scores and prenatal exposure to chlorpyrifos, a common agricultural pesticide. Environ Health Perspect, 119(8), 1196-1201.
                                      EPI
                                 YES-3COHORTS
                                       
                                      66
Rohlman, D. S., Arcury, T. A., Quandt, S. A., Lasarev, M., Rothlein, J., Travers, R., et al. (2005). Neurobehavioral performance in preschool children from agricultural and non-agricultural communities in Oregon and North Carolina. NeuroToxicology, 26(4 Spec. Iss.), 589-598.
                                      EPI
                                      YES
                                       
                                      67
Ruckart, P. Z., Kakolewski, K., Bove, F. J., & Kaye, W. E. (2004). Long-term neurobehavioral health effects of methyl parathion exposure in children in Mississippi and Ohio. [Comparative Study]. Environ Health Perspect, 112(1), 46-51.
                                      EPI
                                      YES
                                       
                                      68
Samarawickrema, N., Pathmeswaran, A., Wickremasinghe, R., Peiris-John, R., Karunaratna, M., Buckley, N., et al. (2008). Fetal effects of environmental exposure of pregnant women to organophosphorus compounds in a rural farming community in Sri Lanka. Clin Toxicol (Phila), 46(6), 489-495.
                                      EPI
                                      YES
                                       
                                      69
Savitz, D. A., Arbuckle, T., Kaczor, D., & Curtis, K. M. (1997). Male pesticide exposure and pregnancy outcome. Am J Epidemiol, 146(12), 1025-1036.
                                      EPI
                                      YES
                                       
                                      70
Shelton, J. F., Geraghty, E. M., Tancredi, D. J., Delwiche, L. D., Schmidt, R. J., Ritz, B., et al. (2014). Neurodevelopmental disorders and prenatal residential proximity to agricultural pesticides: The charge study. [Article]. Environmental Health Perspectives, 122, 1103-1109.
                                      EPI
                                      YES
                                       
                                      71
Suarez-Lopez, J. R., Himes, J. H., Jacobs Jr, D. R., Alexander, B. H., & Gunnar, M. R. (2013). Acetylcholinesterase activity and neurodevelopment in boys and girls. [Article]. Pediatrics, 132, e1649-e1658.
                                      EPI
                                      YES
                                       
                                      72
Suarez-Lopez, J. R., Jacobs Jr, D. R., Himes, J. H., & Alexander, B. H. (2013). Acetylcholinesterase activity, cohabitation with floricultural workers, and blood pressure in Ecuadorian children. [Article]. Environmental Health Perspectives, 121, 619-624.
                                      EPI
                                      YES
                                       
                                      73
Suarez-Lopez, J. R., Jacobs, D. R., Himes, J. H., Alexander, B. H., Lazovich, D., & Gunnar, M. (2012). Lower acetylcholinesterase activity among children living with flower plantation workers. [Article]. Environmental Research, 114, 53-59.
                                      EPI
                                      YES
                                       
                                      74
Wang, P., Tian, Y., Wang, X.-J., Gao, Y., Shi, R., Wang, G.-Q., et al. (2012). Organophosphate pesticide exposure and perinatal outcomes in Shanghai, China. [Article]. Environment International, 42, 100-104.
                                      EPI
                                      YES
                                       
                                      75
Whyatt, R. M., Rauh, V., Barr, D. B., Camann, D. E., Andrews, H. F., Garfinkel, R., et al. (2004). Prenatal insecticide exposures and birth weight and length among an urban minority cohort. Environ Health Perspect, 112(10), 1125-1132.
                                      EPI
                                 YES-3COHORTS
                                       
                                      76
Wickerham, E. L., Lozoff, B., Jie, S., Kaciroti, N., Yankai, X. I. A., & Meeker, J. D. (2012). Reduced birth weight in relation to pesticide mixtures detected in cord blood of full-term infants. [article]. Environment international, 47, 80-85.
                                      EPI
                                      YES
                                       
                                      77
Wolff, M. S., Engel, S., Berkowitz, G., Teitelbaum, S., Siskind, J., Barr, D. B., et al. (2007). Prenatal pesticide and PCB exposures and birth outcomes. Pediatr Res, 61(2), 243-250.
                                      EPI
                                      YES
                                       
                                      78
Young, J. G., Eskenazi, B., Gladstone, E. A., Bradman, A., Pedersen, L., Johnson, C., et al. (2005). Association between in utero organophosphate pesticide exposure and abnormal reflexes in neonates. Neurotoxicology, 26(2), 199-209.
                                      EPI
                                 YES-3COHORTS
                                       
                                      79
Zhang, Y., Han, S., Liang, D., Shi, X., Wang, F., Liu, W., et al. (2014). Prenatal exposure to organophosphate pesticides and neurobehavioral development of neonates: A birth cohort study in Shenyang, China. [Article]. PLoS ONE, 9.
                                      EPI
                                      YES
                                       

Appendix 6.	Plausible hypotheses on MOA/AOP for neurodevelopmental outcomes (Extracted from Section 4.4.3. RHHRA for chlorpyrifos) 

Numerous studies on the possible mechanistic aspects of neurodevelopmental effects have been published.  The results have led some research groups to propose that changes in brain connectivity and/or neurochemistry may underlie the long-term in vivo neurobehavioral changes observed into adulthood.  While multiple biologically plausible hypotheses are being pursued by researchers, no one pathway has sufficient data to be considered more credible than the others.  The SAP concurred with the Agency in 2008 and 2012 about the lack of definable key events in a MOA/AOP leading to neurobehavioral effects.  The Agency has considered the new literature since the 2012 SAP related to mechanistic hypotheses as described below (Appendix 11), and note that such a MOA/AOP still cannot be established. 
      
 Acetylcholinesterase (AChE) as a morphogen:  The classically understood role of AChE is the rapid hydrolysis of acetylcholine at synapses in the brain and at neuromuscular junctions, thereby regulating cholinergic neurotransmission. Consistent with this role, AChE is predominant at cholinergic synapses at neurons and in muscle, and inhibition of its catalytic activity results in the signs and symptoms of cholinergic overstimulation. Several lines of evidence suggest that AChE can also serve as a morphogen, influencing the growth of cells during neurodevelopment distinct from its role as an esterase. Alterations in the expression or structure of the AChE protein can disrupt various aspects of neuronal differentiation and growth, as has recently been shown in vitro (using NG108-15 cell line) following exposure to another OP, paraoxon (Campanha et al., 2014). While perturbation of the morphogenic activity of AChE is a plausible adverse outcome pathway for chlorpyrifos, a number of questions remain, including effective concentrations compared to those that inhibit catalytic activity of AChE. There is, however, no direct evidence showing that disruption of the morphogenic function of AChE can alter axon or dendritic growth in vivo. While limited in vivo studies using zebrafish indicate that chlorpyrifos or its metabolite chlorpyrifos oxon can disrupt axonal growth (Yang et al. 2011), it has not been demonstrated that this effect is due to alteration of the morphogenic function of AChE versus other potential mechanisms.
      
 Cholinergic system:  There are several lines of evidence showing that signaling through cholinergic receptors is involved in neurodevelopment. Activation of muscarinic and/or nicotinic cholinergic receptors can regulate neural progenitor cell proliferation and differentiation (Resende & Adhikari, 2009), and in vivo studies demonstrate that cholinergic signaling is likely involved in brain morphogenesis (Hohmann & Berger-Sweeney, 1998). While ChE inhibitors can affect cholinergic signaling by inhibition of the catalytic activity of AChE and subsequent increase in acetylcholine, some inhibitors, including chlorpyrifos and chlorpyrifos oxon, can also directly interact with cholinergic receptors. Thus, direct interaction with cholinergic receptors by chlorpyrifos represents a potential adverse outcome pathway for disruption of neurodevelopment distinct from AChE/ChE inhibition.  Some OPs have been shown to directly interact with cholinergic muscarinic receptors at relatively low concentrations. The muscarinic receptors are members of the G-protein receptor family and five subtypes (m1-m5) have been identified.  Ward et al. (1993) examined the relationship between ChE inhibition and direct binding to muscarinic receptors for a series of OPs and their active oxon metabolites. The results indicated a strong correlation between AChE activity of OPs, including chlorpyrifos and chlorpyrifos oxon, and the ability to compete for CD binding sites (m2 receptors) in rat brain homogenates. Binding affinities of the oxons were in the nanomolar range, at or below concentrations that inhibited AChE (Huff et al, 1994); specifically, chlorpyrifos oxon had a binding affinity of 22 nM in rat striatum and 2 nM in rat cortex (Huff, et al., 1994; Ward & Mundy, 1996).  In total, these studies suggest that direct interactions with muscarinic receptors, and especially the m2 subtype, represent an alternative site of action for OPs including chlorpyrifos and chlorpyrifos oxon, with the oxon forms having high affinity. Together, the studies cited above outline a plausible adverse outcome pathway for chlorpyrifos and chlorpyrifos oxon to affect brain development via actions at the m2 subtype of muscarinic receptors. However, while there are studies showing that chlorpyrifos oxon can affect neurite outgrowth in vitro and decrease cell proliferation and differentiation both in vitro (Jameson et al , 2006; Qiao et al, , 2001; Song, et al., 1998) and in vivo (Dam et al, , 1998; Qiao, et al., 2003), there is no experimental evidence that these effects are a result of direct actions on the m2 receptor.
      
 Endocannabinoid system:   Several lines of research have suggested that disruption of the endocannabinoid (EC) system due to chlorpyrifos exposure could play a role in its acute and/or long-term toxicity, and could also be extended to potential developmental toxicity.  The EC system modulates neurotransmission as well as playing a morphogenic role during development of the nervous system.  Chemicals (e.g., drugs of abuse) which act on this system, produce long-term neurodevelopmental disorders in animal models and human studies.  Chlorpyrifos also interacts with this system, both in vitro and in vivo.  By this hypothesis, the EC system represents a possible adverse outcome pathway for developmental effects of chlorpyrifos.  There is a body of studies on the interaction of OPs with relevant enzymes but only two studies have examined the effects of chlorpyrifos on the EC system in developing animals.  Carr et al. (2011, 2013) has dosed preweanling rats for 5 or 7 days (1-5 mg/kg/day, p.o.), and showed that endocannabinoid-related enzymes were inhibited in rat brain tissue taken 4 to 48 hours after the last dose.  Interestingly, fatty acid amid hydrolase (FAAH) showed a greater degree and more persistent inhibition compared to AChE inhibition measured in the same rats.  A more recent publication (Carr et al., 2014) repeated these findings using a lower dose (0.5 mg/kg/d for 7 days), still showing significant FAAH inhibition but with no measurable AChE inhibition.  This suggests a greater sensitivity of the EC system, at least in terms of the hydrolase compared to AChE activity, in the pups.  However, there were no other ages tested, no downstream or correlative measure of changes in EC system function, and no subsequent neurodevelopmental effects that could be linked to the action.  Additional studies along these lines are needed.
      
 Reactive Oxygen Species:  The production of reactive oxygen species (ROS) and resulting cellular damage has been proposed as a mechanism for a wide variety of neurotoxicants. Due to lower levels of protective enzymes and antioxidants, and relatively low numbers of glia relative to the adult, the developing brain may be particularly sensitive to neural cell damage caused by oxidative stress. In addition, recent work suggests that ROS can act as second messengers. Relatively small changes in the oxidative status of the cell (redox potential) can lead to changes in redox sensitive signaling pathways that regulate cell physiology. In the nervous system, redox signaling is involved in the regulation of neurodevelopmental processes including neural stem cell proliferation and differentiation (Le Belle et al., 2011; Vieira et al,  2011). A number of studies suggest that chlorpyrifos and chlorpyrifos oxon can induce oxidative stress in various neural cell types. Thus, generation of reactive oxygen species and/or alteration of cellular redox potential by chlorpyrifos represent a possible initiating event leading to developmental neurotoxicity.  Data from both in vitro studies with neuronal cells (including neural precursors) and in vivo studies in developing brain demonstrate that chlorpyrifos can induce oxidative stress. The in vitro data suggests that this effect may not be due to AChE inhibition, since the parent compound chlorpyrifos is either equipotent or more potent than the oxon (for example, Crumpton et al., 2000). There was, however, no concurrent analysis of AChE inhibition in most of these studies. Several known developmental neurotoxicants have been shown to disrupt neural precursor cell proliferation in vitro through a common pathway that is initiated by increasing the oxidative state of the cell (Li et al,  2007), and the antioxidant vitamin E protected PC12 cells from the anti-proliferative effect of chlorpyrifos (Slotkin et al,  2007). Thus, the in vitro data suggest that chlorpyrifos can affect a critical neurodevelopmental process, at least in part, via generation of ROS. Though limited, in vivo studies show both direct evidence (lipid peroxidation) and indirect evidence (alteration in the expression of oxidative stress response genes) of oxidative stress in the developing brain after exposure to chlorpyrifos. Recent evidence suggests that oxidative stress can alter neurodevelopment in vitro and in vivo by the dysregulation of signaling pathways controlling neuroprogenitor cell function (Le Belle, et al., 2011; Vieira, et al., 2011). It has been demonstrated in vivo that antioxidant treatment can attenuate the induction of oxidative stress produced by chlorpyrifos in adult rats (Singh and Panwar, 2014), but there are as yet no such studies addressing its developmental neurotoxicity. Thus, there is the potential for initiation of an AOP via induction of oxidative stress, but supportive studies in developing animals have not been reported. 
      
 Serotonergic system:  Beyond its classical neurotransmitter actions, serotonin has other roles during development. In their review, Thompson and Stanwood (2009) described serotonin as a pleiotropic molecule, meaning that it can produce multiple, diverse effects, regulating different functions at different times during development. The serotonergic system is integral in many developmental processes including, but not limited to, neurogenesis, migration, and differentiation, synaptogenesis, and cardiac development before assuming its more well-known function as a neurotransmitter in the adult nervous system (reviewed in Frederick & Stanwood, 2009). Serotonin also plays crucial roles in thalamocortical patterning (reviewed in (Frederick & Stanwood, 2009). As serotonin is present extremely early in development, it is thought that it modulates cellular function even before neurogenesis. Later in development, serotonin is temporarily taken up by so-called transient serotonergic neurons mainly involved in sensory processing, and is involved in activity-dependent patterning of the brain. Later in development, serotonin has also been shown to modulate differentiation in the brain. There are numerous studies of the effects of perinatal chlorpyrifos administration on the patency of the serotonergic system coming from both Duke University and Istituto Superiore di Sanita in Italy; however, there have been no additional reports since the 2012 SAP.  Endpoints in various brain regions include serotonin levels, serotonin turnover, serotonin receptor levels, serotonin reuptake receptor levels, serotonin elicited second messenger activity, gene expression of serotonin receptor and metabolism related genes, serotonin related behavioral assessments, and behavior after serotonergic drug challenge.  All the data indicate that there are acute, as well as permanent, effects of neonatal chlorpyrifos treatment on the maturation of the serotonergic nervous system. The effects are often gender-specific, region-specific and dose-related.
      
      There is ample evidence that chlorpyrifos exposure during development causes permanent changes in the serotonergic nervous system; there are, however, few papers that assessed concurrently the ChE inhibition (either brain or blood) in those same animals. In some cases, although ChE activity was not assessed concurrently, a dosing regimen was used that had been characterized previously with regard to ChE activity. It does appear, however, that most of the studies on the effects of chlorpyrifos on the serotonergic nervous system were conducted with doses of chlorpyrifos that likely produced inhibition of ChE activity.
      
      As many steps in this chlorpyrifos AOP are possible and plausible, and in laboratory animals the serotonergic nervous system is sufficiently sensitive to low doses of chlorpyrifos during development to alter its function, it is plausible that exposure to chlorpyrifos during development could alter brain development and the function of the serotonergic nervous system. Although chlorpyrifos effects on the serotonergic nervous system in laboratory animals likely is initiated within 24 hours (Slotkin & Seidler, 2007), the actual initiating event of this potential adverse outcome pathway is unknown.
      
 Tubulin, Microtubule Associated Proteins and Axonal Transport:  Microtubules, one component of the dynamic cytoskeletal scaffolding within each cell, are composed of heterodimers of α- and β-tubulin, as well as microtubule associated proteins. The microtubule associated proteins appear to have three main functions: (1) to stabilize the microtubules; (2) to aid in tubulin dissociation and (3) to act as motor proteins moving substances forward and backward along the microtubules (Avila et al, , 1994; Pellegrini & Budman, 2005; Sánchez et al, 2000).  Not only does the microtubule cytoskeleton determine neuronal morphology (Matus, 1988, 1990; Sánchez, et al., 2000), but the dynamic reorganization of the microtubules and microtubule associated proteins within a cell may also coordinate neurite extension/retraction, as well as growth cone advancement.  In addition to these integral roles in brain structure and growth, microtubules and the microtubule associated motor proteins kinesin (Hirokawa & Noda, 2008) and dynein (Vallee et al,  2004) also provide a "railway" for  transport of materials throughout the cell, i.e., axonal transport (Fukushima et al,  2009), another process which is integral to the health of the central and peripheral nervous system, playing a pivotal role in neuronal network formation and synapse maturation (Hirokawa & Takemura, 2004). The construction of an adverse outcome pathway using chlorpyrifos-induced effects on tubulin and microtubule associated proteins is still in its infancy. While it is thought that tubulin, microtubule associated proteins and axonal transport are integral to nervous system development and maintenance, there is no experimental evidence that perturbations of these endpoints by chlorpyrifos during development has neurotoxic outcomes. 

Overall, a definitive mode of action or adverse outcome pathway leading to effects on the developing brain cannot yet be established because of insufficient data establishing the causal linkages among different levels of biological organization to adversity.  For example, while there is in vitro evidence relating binding of chlorpyrifos or the chlorpyrifos oxon to AChE and the subsequent decrease in neurite outgrowth at the cellular level, the relationship between neurite outgrowth and neurodevelopmental consequences has not been established. As described in the NRC report, "Toxicity Testing in the 21[st] Century" (NRC, 2007), to develop an adverse outcome pathway not only is it necessary to establish plausible relationships among the key events, but quantitative relationships also need to be established.  In other words, how much of a change in one key event is needed to result in an adverse effect at the next level of biological organization? Thus, certain exposures to a chemical may impact normal physiological responses in a way that may not necessarily be adverse, and thus, the AOP concept requires an understanding of adaptive/homeostatic capacity of biological systems and their limits, relative to concentration and duration of exposure.

References Cited

Avila, J., Dominguez, J., & Diaz-Nido, J. (1994). Regulation of microtubule dynamics by microtubule-associated protein expression and phosphorylation during neuronal development. Int J Dev Biol, 38(1), 13-25.

Campanha HM, Carvalho F, Schlosser P. (2014) Active and peripheral anionic sites of acetylcholinesterase have differential modulation effects on cell proliferation, adhesion and neuritogenesis in the NG108-15 cell line. Toxicol Lett epub ahead of print

Carr, R. L., Borazjani, A., & Ross, M. K. (2011). Effect of developmental chlorpyrifos exposure, on endocannabinoid metabolizing enzymes, in the brain of juvenile rats. Toxicol Sci, 122(1), 112-120.

Carr, R. L., Adams, A.L., Kepler, D.R., Ward, A.B., & Ross, M. K. (2013). Induction of endocannabinoid levels in juvenile rat brain following developmental chlorpyrifos exposure.  Toxicol Sci, 135(1), 193-201.

Carr RL, Graves CA, Mangum LC, Nail CA, Ross MK. (2014) Low level chlorpyrifos exposure increases anandamide accumulation in juvenile rat brain in the absence of brain cholinesterase inhibition. Neurotoxicol. 43:82-89.

Crumpton TL, Seidler FJ, Slotkin TA. (2000). Is oxidative stress involved in the developmental neurotoxicity or chlorpyrifos?. Bran Rev Dev Brain Res. 12:189-195.

Dam, K., Seidler, F. J., & Slotkin, T. A. (1998). Developmental neurotoxicity of chlorpyrifos: delayed targeting of DNA synthesis after repeated administration. Brain Res Dev Brain Res, 108(1-2), 39-45.

Fredrick, A. L., & Stanwood, G. D. (2009). Drugs, biogenic amine targets and the developing brain. Dev. Neurosci, 31(1-2), 7-22.

Fukushima, N., Furuta, D., Hidaka, Y., Moriyama, R., & Tsujiuchi, T. (2009). Post-translational modifications of tubulin in the nervous system. J Neurochem, 109(3), 683-693.

Hirokawa, N., & Takemura, R. (2004). Molecular motors in neuronal development, intracellular transport and diseases. Curr Opin Neurobiol, 14(5), 564-573.

Hirokawa, N., & Noda, Y. (2008). Intracellular transport and kinesin superfamily proteins, KIFs: structure, function, and dynamics. Physiol Rev, 88(3), 1089-1118.

Hohmann, C. F., & Berger-Sweeney, J. (1998). Cholinergic regulation of cortical development and plasticity. New twists to an old story. Perspect Dev Neurobiol, 5(4), 401-425.

Huff, R. A., Corcoran, J. J., Anderson, J. K., & Abou-Donia, M. B. (1994). Chlorpyrifos oxon binds directly to muscarinic receptors and inhibits cAMP accumulation in rat striatum. J Pharmacol Exp Ther, 269(1), 329-335.

Jameson, R. R., Seidler, F. J., Qiao, D., & Slotkin, T. A. (2006). Chlorpyrifos affects phenotypic outcomes in a model of mammalian neurodevelopment: critical stages targeting differentiation in PC12 cells. Environ Health Perspect, 114(5), 667-672.

Le Belle, J. E., Orozco, N. M., Paucar, A. A., Saxe, J. P., Mottahedeh, J., Pyle, A. D., et al. (2011). Proliferative neural stem cells have high endogenous ROS levels that regulate self-renewal and neurogenesis in a PI3K/Akt-dependant manner. Cell Stem Cell, 8(1), 59-71.

Li, Z., Dong, T., Proschel, C., & Noble, M. (2007). Chemically diverse toxicants converge on Fyn and c-Cbl to disrupt precursor cell function. PLoS Biol, 5(2), e35.

Matus, A. (1988). Microtubule-associated proteins: their potential role in determining neuronal morphology. Annu Rev Neurosci, 11, 29-44.

Matus, A. (1990). Microtubule-associated proteins and the determination of neuronal form. J Physiol (Paris), 84(1), 134-137.

NRC (2007). Toxicity Testing in the 21st Century: A Vision and a Strategy. http://www.nap.edu/catalog.php?record_id=11970 

Pellegrini, F., & Budman, D. R. (2005). Review: tubulin function, action of antitubulin drugs, and new drug development. Cancer Invest, 23(3), 264-273

Qiao, D., Seidler, F. J., & Slotkin, T. A. (2001). Developmental neurotoxicity of chlorpyrifos modeled in vitro: comparative effects of metabolites and other cholinesterase inhibitors on DNA synthesis in PC12 and C6 cells. Environ Health Perspect, 109(9), 909-913.

Qiao, D., Seidler, F. J., Tate, C. A., Cousins, M. M., & Slotkin, T. A. (2003). Fetal chlorpyrifos exposure: adverse effects on brain cell development and cholinergic biomarkers emerge postnatally and continue into adolescence and adulthood. Environ Health Perspect, 111(4), 536-544.

Resende, R. R., & Adhikari, A. (2009). Cholinergic receptor pathways involved in apoptosis, cell proliferation and neuronal differentiation. Cell Commun Signal, 7, 20.

Sanchez, C., Perez, M., & Avila, J. (2000). GSK3beta-mediated phosphorylation of the microtubule-associated protein 2C (MAP2C) prevents microtubule bundling. Eur J Cell Biol, 79(4), 252-260.

Singh V, Panwar R. (2014). In vivo antioxidative and neuroprotective effect of 4-allyl-2-methoxyphenol against chlorpyrifos-induced neurotoxicity in rat brain. Mol Cell Biochem. 388:61-74

Slotkin, T. A., & Seidler, F. J. (2007). Comparative developmental neurotoxicity of organophosphates in vivo: transcriptional responses of pathways for brain cell development, cell signaling, cytotoxicity and neurotransmitter systems. Brain Res Bull, 72(4-6), 232-274.

Slotkin, T. A., MacKillop, E. A., Ryde, I. T., & Seidler, F. J. (2007). Ameliorating the developmental neurotoxicity of chlorpyrifos: a mechanisms-based approach in PC12 cells. Environ Health Perspect, 115(9), 1306-1313.

Song, X., Violin, J. D., Seidler, F. J., & Slotkin, T. A. (1998). Modeling the developmental neurotoxicity of chlorpyrifos in vitro: macromolecule synthesis in PC12 cells. Toxicol Appl Pharmacol, 151(1), 182-191.

Thompson, B. L., & Stanwood, G. D. (2009). Pleiotropic effects of neurotransmission during development: modulators of modularity. J Autism Dev Disord, 39(2), 260-268.

Vallee, R. B., Williams, J. C., Varma, D., & Barnhart, L. E. (2004). Dynein: An ancient motor protein involved in multiple modes of transport. J Neurobiol, 58(2), 189-200.

Vieira, H. L., Alves, P. M., & Vercelli, A. (2011). Modulation of neuronal stem cell differentiation by hypoxia and reactive oxygen species. Prog Neurobiol, 93(3), 444-455.

Ward, T. R., Ferris, D. J., Tilson, H. A., & Mundy, W. R. (1993). Correlation of the anticholinesterase activity of a series of organophosphates with their ability to compete with agonist binding to muscarinic receptors. Toxicol Appl Pharmacol, 122(2), 300-307.

Ward, T. R., & Mundy, W. R. (1996). Organophosphorus compounds preferentially affect second messenger systems coupled to M2/M4 receptors in rat frontal cortex. Brain Res Bull, 39(1), 49-55.

Yang D, Lauridsen H, Buels K, Chi LH, LaDu J, Bruun DA, Olson JR, Tanguay RL, Lein PJ. (2011) Chlorpyrifos-oxon disrupts zebrafish axonal growth and motor behavior. Toxicol. Sci 121:146-159