OFFICE OF CHEMICAL SAFETY AND POLLUTION PREVENTION UNITED STATES ENVIRONMENTAL PROTECTION AGENCY WASHINGTON, D.C. 20460



### RE : Federal Letter

- Review o f Chlorpyrifos Epidemiology Studies

Summary

Between July and September 2012, EPA solicited input from several individuals in the federal government with expertise related to (1) the assessment of neurodevelopmental outcomes and (2) the use of magnetic resonance imaging (MRI) for the purpose of research in order to review 

### Brain Anomalie s in Children Exposed Prenatally to a Common Organophosphate Pesticide (2012) study

. This document contains a list of the Federal reviewers, the questions EPA asked the reviewers to consider during their review of the Rauh et al. study, EPA’s brief synopsi s of Rauh et al. 2012 and its evaluation of the epidemiological aspects of the MRI study, the letters sent to each reviewer soliciting their input, and their review.

List of Documents

1. List of Reviewers Page 2 2. Questions for Reviewers Page 3 3. EPA’s Synopsis of Rauh et al. and Evaluation of Epidemiological Aspects of MRI Study Page 6 4. Dr. Basser Letter and Review Page 1 6 5. Dr. Liachenko Letter and Review Page 2 1 6. Dr. Freund Letter and Review Page 2 4 7. Dr. Bitsko Letter and Review Page 2 8 8. Dr. Chelonis Letter and Review Page 3 3Federal External Review Panel Members

Peter J. Basser, Ph.D. NICHD (National Institute of Child Health and Human Development)
TISSUE BIOPHYSICS & BIOMIMETICS

Rebecca Bitsko CDC/ DHHS/CDC

Lisa Freund, Ph.D. (NICHD) NICHD (National Institute of Child Health and Human Development) CHILD DEVELOPMENT & BEHAVIOR BRANCH (HNT3D)

Serguei Liachenko, MD, PhD, Director of Bioimaging, National Center for Toxicological Research Food and Drug Administration

John J. Chelonis Division of Neurotoxicology, National Center for Toxicological Research Jefferson, ARQuestions for External Peer Review of MRI Paper

1) In April 2012, the FIFRA SAP 1 , an external review body designated by Congress to advise EPA concerning science policy issues, reviewed the epidemiology studies produced by researchers
with the Columbia Center on Children’s Environmental Health (CCCEH) regarding the relation between prenatal chlorpyrifos exposure and adverse neurodevelopmental consequences through childhood (FI FRA Scientific Advisory Panel, 2012) . In May 2012, researchers with the CCCEH produced the paper under evaluation in this written peer review; the paper on brain abnormalities using MRI technology by (Rauh et al., 2012 2 )

. Because Rauh et al (2012) was available after the April 2012 SAP review, this study was not consider ed by the Panel. For the current peer review, EPA is soliciting comments on the epidemiological aspects of the study design which are unique to Rauh et al (2012) (e.g., sample selection and inclusion/exclusion criteria; socioeconomic status (SES) consider ations; sample size and statistical power, etc.) EPA’s has reviewed of the epidemiological context of the MRI study by Rauh et al (2012) (see attached). Please comment on the Agency’s review with respect to the epidemiological aspects of the study.

2) The authors us ed sophisticated MRI imaging studies to measure difference in 3D size of cerebral surface and cortical thickness by chlorpyrifos exposure group (high/low). Please comment on the MRI imaging procedures. In addition to other issues you identify a s relevant for EPA to
consider , please include in your comments a consideration of the following:

a. Authors report that 20 of 28 eligible participants in the “high” exposure group were enrolled with usable MRIs, and 20 of 38 eligible participants in the “ low” exposure group were enrolled with usable MRIs in the study. Is th is a reasonable eligibility rate, i.e. , is the rate of lack of usable MRI remarkable in this study?
b. Did authors process and pre

- process images appropriately according to standard practic e in research studies? Were there deviations to common practice in MRI studies, and if so, what are the potential impacts on the interpretation of the findings ?

c. Did authors use appropriate methods to categorize “grey matter” thickness by gradations of colo r of the cerebral cortex across the MRI images? If not, how could methods be improved and what are the implications?
d. Authors utilized a tw o

- step process to identify an individual “template” brain against which to compare other brain images by CPF exposure status. Please comment as to the use of an individual versus an average or amalgamated “template.”

3) Please comment on the statistical procedures used in Rauh et al (2012)

.

1 FIFRA Scientific Advisory Panel. (2012). Draft Issue Paper Scientific Issues Concerning the Health Effects of Chlorpyrifos.

2 Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., . . . Peterson, B. S. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. Proc Natl Acad Sci U S A, 109 (20), 7871

- 7876. doi: 1203396109 [pii] 10.1073/pnas.12033961094) The authors reported several findings (i.e., areas of brain enlargement

, qualitative statistical interaction between full scale intelligence quotient (FSIQ) and brain morphology ; aberrations in
the normal sexual differentiation of the brain

, and sporadic cortical thinning )

. Please provide your interpretation of the results re ported by Rauh et al (2012). Based upon your interpretation of the study, what, if any, neurodevelopmental consequences would you anticipate in these
children?

5) What are the uncertainties around the MRI imaging findings in Rauh et al (2012) ? Child age range in the MRI study was 6

- 11 years. Are there changes in child brain development in this period of rapid brain growth that could affect study results and interpretation?

6) In the developmental neurotoxicity study conducted in laboratory rats (Hoberman, 1998a, 1998b, 1999a, 1999b 3 ) EPA’s Data Evaluation Review (DER) of this study , pp. 23

- 25 and 29 of
DER for MRID 44556901 & pp. 3

- 7 and 13

- 15 of DER for MRID 44787301 4 ]

, effects in brain morphometry w ere observed in rats exposed to chlorpyrifos during gestation. To what degree, if any, are the results of Rauh et al (2012) co mparable with Ho hmann ?

7) What additional comments would you like to provide us that may assist us in better interpreting the results and characterizing the uncertainty?

8) Are there any additional analyses that should be conducted? And what would be the value or implications of these analyses?

3 Hoberman, A. M. (1998a) Developmental neurotoxicity study of chlorpyrifos administered orally via gavage to Crl:CD®BR VAF/Plus® presumed pregnant rats. Argus Research Laboratories, Inc., Horsham, Pennsylvania, Laborato ry study No. 304

- 001, Sponsor study No. K

- 044793

- 109, May 1, 1998. MRID 44556901. Unpublished. Hoberman, A.M. (1998b) Developmental neurotoxicity study of chlorpyrifos administered orally via gavage to Crl:CD®BR VAF/Plus® presumed pregnant rats; Report S upplement 1 (pathology data). Argus Research Laboratories, Inc., Horsham, Pennsylvania, Laboratory study No. 304

- 001, Sponsor study No. K

- 044793

- 109, September 23, 1998. MRID 44661001. Unpublished. Hoberman, A.M. (1999a) Developmental neurotoxicity stud y of chlorpyrifos administered orally via gavage to Crl:CD®BR VAF/Plus® presumed pregnant rats; SAS Statistical Output. Argus Research Laboratories, Inc., Horsham, Pennsylvania, Laboratory study No. 304

- 001, Sponsor study No. K

- 044793

- 109, Undated. MRID not assigned. Unpublished. Hoberman, A.M. (1999b) Developmental neurotoxicity study of chlorpyrifos administered orally via gavage to Crl:CD®BR VAF/Plus® presumed pregnant rats; Report Supplement 2

- Reanalysis of Morphometric Data. Argus Research Labora tories, Inc., Horsham, Pennsylvania, Laboratory study No. 304

- 001, Sponsor study No. K

- 044793

- 109, March 19, 1999. MRID 44787301. Unpublished.

4 Makris, S. Chlorpyrifos Toxicology Data Review.1 Summary and Critical Review of Rauh et al. (2012) “Brain anomalies in children exposed prenatally to a common organophosphate pesticide”

This document includes a brief summary of the research paper “Brain anomalies in children exposed prenatally to a common organophosphate pesticide” published by Rauh et al. (2012) as well as EPA’s critical analysis of the epidemiologic context of this stud y. The study utilizes magnetic resonance imaging (MRI) technology to identify and contrast anatomical characteristics of the cortical region of the brain including cortical surface area and thickness in relation to prenatal pesticide exposure. EPA is seeki ng additional expert input as to the quality of the MRI data collection procedures used in this study as well as the appropriateness of the authors’ interpretation of the MRI studies in association with the environmental exposure of
interest. EPA’s critica l analysis follows the brief synopsis of the study below.

Study Summary

In this study, authors with the Columbia Center for Children’s Environmental Health (CCCEH) birth cohort study evaluated whether there were areas of morphological change in the pediat ric brain in regions of the brain known to be associated with learning, cognition and social behavior in association with prenatal chlorpyrifos (CPF) exposure (V. A. Rauh et al., 2012) . These are functional neurodevelopmental outcomes that have been advers ely associated with chlorpyrifos exposure in previous observational research (V. A. Rauh et al., 2006; V. Rauh et al., 2011)

, as well as some experimental studies, as noted by the authors. Specifically, in this study, authors evaluated 1) whether morpholog ical changes in the brain including cortical thickness and differences in the cerebral brain surface area and volume were related to prenatal chlorpyrifos exposure (the main effect), and, 2) whether the relation between the exposure and morphological chang es in the brain were modified by either full scale intelligence quotient (FSIQ) score (evidence of effect modification by IQ) and, 3) in preliminary analyses, whether changes in the normal sexual differentiation of the brain exist by

exposure

- level (V. A. Rauh et al., 2012)

.
Authors concluded that while overall brain size did not differ by prenatal chlorpyrifos exposure, certain areas of the brain were enlarged among those more highly exposed to chlorpyrifos including superior temporal, posterior middle te mporal, and inferior post

- central gyri, and superior frontal gyrus, gyrus rectus, cuneus, and pre

- cuneus in the mesial views of the right hemisphere of the cerebrum. Further, they noted a difference exposure

- response relation between cerebral surface dista nces and CPF exposure, by FSIQ wherein the relation between brain area enlargements and FSIQ were positively correlated in the low CPF group (greater regional brain size correlated with greater IQ), but an inverse relation among the high CPF group. In supp lemental analyses, authors illustrated that areas of regional brain enlargement among those more highly exposed to chlorpyrifos were due to underlying white matter enlargements. Researchers also noted sporadic differences in cortical thickness wherein high er CPF exposure was related to reduced cortical thickness. Preliminary analyses also displayed a disruption
of normal male

- larger

- than

- female areas as well as female

- larger

- than

- male in certain areas of the brain known to differ by child sex in normal brai n. Overall, researchers concluded that differences in brain structure (regional cerebral size and thickness) exist between CPF exposure groups, and the differences2 (high>low CPF) in regional brain size may be due to enlargement of underlying white matter. However, authors did not speculate as to the underlying biological explanation of the observed qualitative interaction between CPF exposure and FSIQ with morphological changes in key areas of the brain linked with learning, cognition and social behavior.

D esign and Methods

Children selected for this MRI study (n=40) were participants in the larger Columbia Center for Children’s Environmental Health (CCCEH) birth cohort study (n=369). The study was initiated in the late
1990’s to evaluate the relation betwe en air pollution and pesticide exposure and neurodevelopmental effects in children. The cohort is comprised of a low income, inner

- city population primarily of Dominican and African American ethnicity. Mothers were recruited among those who sought prenatal care by 20 weeks of pregnancy at one of the participating study hospitals. Participants were non

smoking women, ages 18

- 35, with no other co

- morbidities including diabetes, hypertension, known HIV infection, and no documented drug use. Therefore, the coho rt represents a population of women at
low

- risk for adverse neurodevelopmental outcomes among their children. Participants were residents of NYC for at least one year prior to enrollment in the research study. EPA’s review of all the epidemiological studie s from the CCCEH (the Columbia studies) is contained in a recent presentation to the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) Science Advisory Panel (FIFRA SAP), April
2012 (FIFRA Scientific Advisory Panel, 2012)

.

Authors measured chlor pyrifos and other organophosphate pesticides, cotinine, lead and mercury in umbilical cord blood samples taken at delivery using CDC analytic methods with published quality assurance and control procedures. Prenatal environmental tobacco smoke (ETS) was me asured as
maternal self

- report, and verified through cotinine levels (≤15 ng/mL). Polycyclic aromatic hydrocarbon (PAH), another environmental chemical possibly related to adverse neurodevelopmental outcomes, was measured through personal air monitoring in the third trimester of pregnancy. In the current (MRI) study, high chlorpyrifos exposure was defined as the upper tertile of the exposed group, ≥4.39 pg/g, and low chlorpyrifos exposure was defined as values below the upper tertile, <4.39 pg/g, including those not exposed to CPF.

To measure intelligence among school aged children, authors used the Wechsler Intelligence Scale for Children, 4th edition (WISC

- IV). The instrument measures four areas of mental functioning: the Verbal Comprehension Index, the P erceptual Reasoning Index, the Working Memory Index, and the Processing Speed Index. These indices are associated with, but distinct from, overall intelligence quotient (IQ) and are sensitive to cognitive deficits related to learning and working memory. A Full

- Scale IQ score combines the four composite indices. A General Ability Index score is a summary score of general intelligence, similar to Full

- Scale IQ, but excludes contributions from both Working Memory Index and Processing Speed Index. WISC

- IV score s are standardized against U.S. population

- based norms for English and Spanish

- speaking children.3 MRI

This section provides some broad detail as to the method of MRI data collection employed in this research paper; this is not a critical review of metho ds. Health outcomes of interest in this study were morphological changes in the brain in regions of the pediatric brain that are related to learning, cognition and social behavior, as noted in the traditional epidemiology studies. Researchers performed
bra in imaging studies using Magnetic Resonance Imaging (MRI) technology among a sub

- sample of the cohort, and measured and compared morphological characteristics by chlorpyrifos exposure status. Details concerning the MRI imaging procedures, processing and pr e

- processing image modifications, methods to measure cortical thickness and methods used to identify the “template” brain against which individual child participant brain images were compared are presented in the Supplementa l Information

.
Briefly, researc hers used a high

- resolution MRI, whole body scanner, with 8

- channel head scanner for this study. Authors processed MRI images using biomedical imaging software (ANALYZE 8.0) while technicians were blinded to participant characteristics, and whether left or right hemisphere of the brain was imaged. Before morphological analyses were performed, images in the dataset were re

- sliced to correct for residual head rotation, tilt, and flexion/extension by participants during imaging

. MRI images

were pre

- processed t o correct for large

- scale variation in color intensity. Extra

- cerebral tissues captured in the MRI brain image were manually removed from the cerebral image before analyzing differences in brain morphology among participants by chlorpyrifos exposure; autho rs hypothesized difference in regional brain areas of the cerebrum only, other parts of the brain were therefore excised from the image before processing. Other similar modifications to the brain image were made to enhance measurement of key/pre

- identified endpoints (cerebral surface morphology and cortex thickness) (See Supplemental Information). Variation in cortical grey matter was assigned by authors as four categories of color changes across images; authors selected color categories that adequately gro uped the participants as to degree of cortical grey matter ( i.e. , creation of four

- level categorical variable to measure cortical thickness) and to allow for a robust statistical analysis, particularly considering the small study sample (n=40).

To measure morphological characteristics within cerebral areas associated with learning, cognition and social behavior, authors identified a “template” brain image against which all participant images were compared and measured. Using a two

- step process, authors ini tially identified the individual in the study sample that most closely approximated average demographic features (child age, height and weight). Using this brain image, all other participant brain image studies were co

- registered, i.e. , similar points on

t he cerebral surface were matched according to the International Consortium on Brain Imaging. Researchers then measured distances of correlated points on the cerebral surface between the “template” brain image and all participant images. Using these measure d distances, authors then identified a final “template” brain based upon the brain image series which most closely matched the average distances among cortical surface measures. Authors advocated that use of one “template” brain as opposed to an amalgamate d “average” brain is superior because of the enhanced clarity and more well defined tissue interface with which to make the measurements for the study. All sets of brain image
studies were then re

- calibrated to the final “template” brain image for statisti cal analyses.4 Statistical Analyses

Regarding the statistical analyses employed in this paper, the analyses involved comparison of the Euclidean distances measured between correlated points on the cerebral surface or cortical thickness by chlorpyrifos (high /low) exposure level, adjusting for age and sex. Supplemental information reported that thousands of statistical tests were performed. Authors used generalized linear equations and linear regression modeling to estimate average differences in morphological characteristics by change in chlorpyrifos exposure level (high/low). Because authors performed multiple statistical tests, they utilized the False Discovery Rate (FDR) methods to adjust the p

- value against Type I error. Authors tested the statistical sign ificance of interaction terms of cerebral surface measures and FSIQ, and by child sex at p<0.05, adjusted for multiple testing.

Study Results & Authors’ Interpretation

For this study, authors selected child participants who had complete measures for CPF, PAH, ETS, and who completed the WISC IV testing. There were 70 participants who met these criteria and who also had cord blood levels in the upper tertile of 4.39 pg/g (High CPF exposure), and among these 28 met the additional criteria of having low or no exposure to PAH and ETS. Twenty of these participants completed the MRI series (20 of 28 eligible participants enrolled in study). Similarly, there were 99 participants who met the enrollment criteria (completed the environmental exposure and IQ testing b attery) and were in the lower exposure group (<4.39 pg/g) and had no or low ETS and PAH. Among this group of low CPF exposed participants, 38 were randomly selected to participate in the MRI study, and 20 participants completed the series (20 of 38 eligibl e participants enrolled in study).

Authors reported no significant differences between those in the high and low chlorpyrifos exposure group, or among the larger cohort according the common socioeconomic indicators including maternal income and education, maternal IQ, race, gestational age and birth weight of child, and blood lead measure (See Table 1, main text). The correlation between chlorpyrifos and blood lead was low (ρ<0.20)
and non

- significant (p>0.20).

Authors concluded that while overall brain s ize did not differ by prenatal chlorpyrifos exposure, certain areas of the brain were enlarged among those more highly exposed to chlorpyrifos including superior temporal, posterior middle temporal, and inferior post

- central gyri, and superior frontal gyru s, gyrus rectus, cuneus, and pre

- cuneus in the mesial views of the right hemisphere of the cerebrum. Further, they noted interaction, or a different exposure response relation between cerebral surface distances and FSIQ, by CPF exposure wherein the relatio n between brain area enlargements and FSIQ were positively correlated in the low CPF group (greater regional brain size equated to greater IQ), but an inverse relation among the high CPF group (greater regional brain size related to lower FSIQ scores).
Aut hors noted statistical interactions in the superior temporal, inferior frontal, inferior pre

- central, and inferior post central gyri bilaterally, and the pre

- cuneus of the left hemisphere.

In supplemental analyses, authors illustrated that areas of region al brain enlargement among those more highly exposed to chlorpyrifos were due to underlying white matter enlargements. This conclusion was supported by the observation of sporadic differences in cortical thickness wherein higher CPF5 exposure was related to reduced cortical thickness, or thinning of the cortex in certain areas. The qualitative interaction between CPF exposure and FSIQ with cerebral brain region enlargements may be explained by size of underlying white matter of the cerebrum, but authors did not speculate as to a biological explanation of the observed qualitative interaction. Preliminary analyses also displayed a disruption of normal male

- larger

- than

- female areas as well as female

- larger

- than

- male in certain areas
of the brain known to differ by child sex in normal brain.

Overall, researchers conclude that differences in brain structure (regional cerebral size and thickness) exist between CPF exposure groups, and the differences (high>low CPF) in regional brain size is likely due to enlargemen t of underlying white matter. Authors also observed an inverse relation between regional brain enlargements and IQ among those highly exposed to CPF, and positive association between regional brain size and IQ among those in the low/no exposed group. While authors do not speculate as to a possibly biological explanation of this observed statistical interaction, EPA notes that given that total brain size did not differ by exposure group, the enlargement of the white matter in
certain areas could have been at the decrement of the size, quality or other morphological changes in the grey matter in other areas; neurons in grey matter of cerebrum affect learning and cognition. Observed inward deformations (reduced brain area) in the dorsal and mesial surfaces of t he left superior frontal gyrus as also reported the authors suggest may also be part of the explanation of the qualitative interaction. However, the authors indicate that additional study is needed to elucidate these possibilities.6 EPA Review

This study by Rauh et al. 2012 follows upon both previous epidemiological research and experimental toxicology studies that suggest an association between chlorpyrifos exposure in the pre

- natal and early life periods with adverse neurodevelopmental outcomes observed in children through school age, including intelligence deficits. Within this CCCEH cohort, authors reported links between prenatal chlorpyrifos exposure (concentration of CPF in cord blood at delivery (pg/g)) with attentional difficulties, behavioral prob lems and measures of intelligence particularly working memory (V. A. Rauh et al., 2006; V. Rauh et al., 2011)

. EPA’s review of these previous epidemiology studies from the CCCEH cohort were presented to the FIFRA SAP in April 2012 (FIFRA Scientific Advis ory Panel, 2012) . Regarding this study, EPA notes a limited and somewhat unbalanced depiction of the available rodent experimental data was presented in the paper. A brief review of references seems to provide some support for the authors’
assertion that t he epidemiological findings are consistent with those in the experimental literature ; however, a fuller evaluation of the data concerning animal to human concordance is needed. For example, the authors’ stated support for their finding regarding enlargemen ts in underlying white matter is not strongly supported by existing experimental data concerning CPF induced glial scarring.

In this study, authors evaluated morphological characteristics of cortical brain regions which have been linked to disrupted learn ing, cognition and social behavior in previous epidemiologic and experimental studies. The cerebrum has been described as the “seat of intelligence” and provides the ability to read, write, and speak ; to make calculations ,and to compose music ; and to reme mber the past, to plan for the future, and to imagine thin gs that have never existed before (Tortora & Grabowski, 2003)

. However,
while the cerebrum ’s influenc e upon human intelligence is generally supported, the study lacked specific hypotheses regarding particular areas of the cerebrum affected (the largest of the four components of the brain). Therefore, as noted in supplemental information, many thousands of statistical tests were performed evaluating areas of the cerebrum that have been mapped. Authors appropriately employed methods to control for Type I error, however in many ways this research requires replication, as noted by the authors. Therefore, use of these data to inform specific underlying biological mechanisms or pathways of the potential tox ic action of CPF is limited.

EPA is seeking external peer review of this specific aspect of the current study to verify that the MRI methods are sound and appropriate. Briefly, however, the main text and supplemental information states that technicians (MRI image readers) were blinded as to exposure status, as well as right or left hemisphere. This design feature adds to the validity and reliability of the study. However, EPA notes authors do not state whether multiple MRI image readers were employed in identifying and measuring
cortic al surface thickness and cerebral surface area, etc. of interest in this study, and whether quality
control statistics such as the inter

- rater reliability coefficient were calculated and considered. Supplemental information notes that validation studies we re performed, however does not describe the aspects of methods that were validated. EPA particularly seeks input from experts in this area as to
whether these methods and validation studies, if known, are sound.

Authors appropriately described the inclusi on and exclusion criteria used in this MRI study. Authors excluded participants for who measureable levels of both PAH and ETS were reported in the total cohort7 study. In this way, by design, authors eliminated the chance of confounding by some other
envir onmental chemicals known to effect neurodevelopment in the relation between chlorpyrifos and brain morphology. Authors also included only those participants who provided full information regarding prenatal chlorpyrifos exposure, other environmental chemica l exposure and who completed the WISC intelligence testing. Given the relatively small sample size (n=40) for this relatively expensive imaging study these decisions eliminated the issue of missing data, and the need to add degrees of
freedom to statistica l models for control of other measured environmental chemicals. These decisions added to the validity and statistical power of the study. Although inclusion and exclusion criteria were somewhat rigorous in this study, authors still achieved a 60% participa tion rate among those eligible to participant (HIGH: 20/28, LOW: 20/38). Further, Table 1 indicates adequate comparability between the exposure groups and the total cohort, although those in the high exposed group were slightly albeit not statistically sig nificantly more likely to be in families in which maternal education and income were slightly greater than the low exposure group. Therefore, in general, EPA believes that authors were able to identify an unbiased sample of cohort, representative of popula tion sample of low income, inner city kids. However, given the rigorous enrollment criteria, t he external validity of the study findings may be limited if the organophosphate pesticide exposure

- brain morphology association is modified by factors
that are m ore, or less, prevalent in the study populations relative to the population(s) to which inference is being made, or to populations with a substantially different exposure range.

In the larger cohort, authors measured concentration of chlorpyrifos parent c ompound in the cord blood of infants at delivery. Generally, biomarkers are considered superior measures than questionnaire based self report or environmental monitoring. The assays used in the study were highly sensitive allowing detection of very low lev els of chlorpyrifos, in the pg/g range (CDC published methods). However, these measures are comparable to those in other samples of the general population, as noted by authors. Uncertainties remain however, as to the extent the use of one biomarker measure ment reflects exposure(s) over critical windows of development during pregnancy, the temporality or periodicity of exposure, over the relevant time period of gestation. These cord blood measures have been used in several studies to estimate pre

- natal expos ure to chlorpyrifos, i.e. , during the entire period of gestation. Validations studies performed within the CCCEH cohort support that use of a one

- time measure is an accurate and reliable measure with which to rank participants (high/low). In this study, authors categorized chlorpyrifos exposure as high (greater than or equal to the upper tertile of 4.39 pg/g) and low (all values below 4.39 pg/g including those values below the LOD) which exposure methodology can accurately and reliably support. PAH in the ambient air was measured using stationary air monitors and ETS was measured through maternal self

- report, validated by cotinine level. A noted limitation in the measurement of pesticide exposure in these studies is the lack of measurement during the post

- natal period through the time period of the epidemiologic evaluations, i.e. , school age. Chemical exposure in the post

- natal period may confound or alter the relation between prenatal chlorpyrifos and neurodevelopmental outcomes, as noted in EPA’s evaluati on (FIFRA Scientific Advisory Panel, 2012) . In this study, child age and sex were the only co

- variables considered; these variables are simple to accurately and reliably measure using linkage with birth/medical records.8 The selection of few co

- variables f or the statistical models used to evaluate the relation between prenatal chlorpyrifos exposure and brain morphology changes is appropriate. For another variable to be a confounder in this relation, the other variable would have to be related to both chlorp yrifos exposure and (causally) related to brain morphology ( e.g. , surface area, cortical thickness, gyri dimensionality). Child age and sex are appropriate confounders as these factors would influence brain size, growth and
differentiation. While some rese arch has noted a link between social characteristics and brain development and possibly morphological characteristics, the authors’ decision to restrict CCCEH participation to those with very similar SES characteristics reduces that possibility of confound ing the association of interest in this MRI study by SES. Further, the small sample size supports the use of few
additional co

- variables in the statistical model to maximize statistical power. Similar, within the MRI study, authors restricted participation to those without other co

- exposures such as PAH and ETS thus eliminating the need to consider co

- adjustment for these variables. Authors additionally measured blood lead in this sample and the overall, total cohort and observe the correlation between chlo rpyrifos and blood lead (necessary factor for positive confounding bias to occur) was very low (ρ<0.20) and not statistically significant (p>0.20).

Other biases that may be present in epidemiology studies include selection and information biases.
Selectio n bias in prospective cohort studies may occur if participants drop out of a study and the decision to drop out was related to either the CPF exposure or neurodevelopmental outcome. In the CCCEH study the retention rate is high for a birth cohort study. Th e participation rate in the MRI study was good (60%). Authors illustrated that those included and excluded were comparable, and the sample and total cohort were also comparable. This ameliorates the concern. Temporal bias, a question as to whether exposure preceded disease, is not a large concern in a prospective study. The prospective nature of the cohort study also reduces that chance of differential information bias or the misclassification of exposure, confounder, or outcome information by key study fac tors. Non

- differential exposure measurement error may have occurred: 1) the single measurement chlorpyrifos during the third trimester, 2) error arising from differences between measured biomarker levels and actual chlorpyrifos at the one

- time point, 3) un measured time

- and space

- dependent patterns of chlorpyrifos exposure, 4) uncertainty regarding the critical period for chlorpyrifos effects on development, 5) missing exposure data, 6) laboratory errors, and 7) imputation of missing exposure levels. Meas urement errors in the ascertainment of chlorpyrifos exposure are likely to have occurred. However, because of the prospective study design employed, the errors are unlikely to result in falsely positive findings, because the probability and magnitude of th ese errors are likely to be independent of the outcome status of participants (MRI characteristics). The degree to which measurement error in the imaging studies performed using MRI technology are of keen interest to EPA.

Authors employed generalized linea r models to estimate the association between CPF exposure (high/low) and “voxels” or three

- dimensional data reflecting morphological characteristics of the cerebral cortex, specifically cortical thickness, distances on the cerebrum, size and dimension of k ey gyri important to learning, cognition, and social behavioral, adjusting for child age and sex. Use of a two

level exposure variable is preferable as it conserves study power, and is supported by the one

- time measure of CPF at delivery (cord blood CPF co ncentration pg/g). Researchers utilized a method to9 correct for multiple statistical tests (FDR) which was reasonable and appropriate in this instance. Authors acknowledged low statistical power to evaluate statistical interaction was likely including gene by environment interactions. Linear modeling is a preferred method when sample size is small as the method conserves statistical power. Authors did not present a table of model results, only graphical depictions of areas of morphological differences of th e cerebrum by CPF exposure, color coded to reflect the statistical significance of the differences in specific, region brain size or volume or area. Additional information reflecting model fit statistics, as well as display of evidence of effect modificati on in both exposure strata (high and low CPF exposure) would enhance clarity and presentation of the research results. While authors appropriately characterized the evaluation of the interaction with child sex and CPF and brain morphology as preliminary, g iven the number of statistical tests performed and lack of hypotheses regarding particular areas of the cerebrum and cortical surface, results require replication before firm conclusions regarding underlying biological mechanisms at play can be posited.

A uthors concluded that the evidence from the study illustrated changes in brain morphology in association with higher CPF exposure, and that changes observed were in areas of the brain that sub

serve those learning, cognition and social behavioral, supporte d by previous observational and experimental literature.10 References

FIFRA Scientific Advisory Panel. (2012). Draft Issue Paper Scientific Issues Concerning the Health Effects of Chlorpyrifos. http://www.regulations.gov/#!documentDetail;D=EPA

- HQ

- OPP

- 2012

- 0040

- 0002

.
Rauh, V. A., Garfinkel, R., Perera, F. P., Andrews, H. F., Hoepner, L., Barr, D. B., . . . Whyatt, R. W. (2006). Impact of prenatal chlorpyrifos exposure on neurodevelopment in the first 3 years of life among inner

- city children. Pediatrics, 118 (6), e1845

- 1859. doi: 10.1542/peds.2006

- 0338

Rauh, V. A., Perera, F. P., Horton, M. K., Whyatt, R. M., Bansal, R., Hao, X., . . . Peterson, B. S. (2012). Brain anomalies in children exposed prenatally to a common organophosphate pesticide. Proc Natl Acad Sci U S A, 109 (20), 7871

- 7876. doi: 1203396109 [pii] 10.1073/pnas.1203396109

Rauh, V., Arunajadai, S., Horton, M., Perera, F., Hoepner, L., Barr, D. B., & Whyatt, R. (2011). Seven

- year neurodevelopmental scores and prenatal exposure to chlorpyrifos, a common agricult ural pesticide. E nviron Health Perspect, 119( 8), 1196

- 1201. doi: 10.1289/ehp.1003160

Tortora, Gerard J., & Grabowski, Sandra Reynolds. (2003). Principles of Anatomy and Physiology (pp. page 467). United States of America: John Wiley & Sons.

### 1. I have read and reviewed the material presented in the Summary and Critical

Review of Rauh et al. (2012) “Brain anomalies in children exposed prenatally to a common organophosphate pesticide” , as well as the Rauh et al. (2012) paper itself. This Summary fairly represents the method s and findings reported in the Ruah (2012) paper. The Critical Review portion seems fair and balanced.

2. The Ruah paper uses well

- established image acquisition and processing techniques. It is important to point out that MRI at lasing is not an exact science, for instance

, as is the determination of weights and measures

. In the latter case sources of systematic experimental artifacts and instrumental biases are known ; these can be accounted for and corrected

, and accurate and precise measurements can be made. MRI atlasing is an evolving science in which a particular MRI acquisition protocol is used to produces imaging data that is processed by a “pipeline” with which this MRI data is analyzed and from which inferences are drawn

. S ystematic and non

- systematic artifacts and bias c an be introduced at any point in this pipeline. To streamline and help standardize processing of multi

- subject or multi

- site brain MRI data, several neuroimaging resea rch centers (e.g., Massachusetts Gene ral Hospital (MGH), UCLA

- LONI, Oxford University, Montreal Neurological Institute ) have developed and disseminated integrated MRI processing pipelines. These are typically based on different assumptions and use different mathematical and computational stra tegies to create brain atla ses from MRI data and report the statistical significance of their similarities or differences. Based on the assumptions and routines described in the Rua h paper to analyze their MRI data, it appears that Ruah et al. relied on the ir own image

- processing pipeline that uses some visualization routines provided in Analyze 8.0

. The use of thei r own software for this purpose is in itself, no particular source of concern. a) The inclusion of 20 out of 28 eligible participants in the high exposure and 20 out of 38 of the low risk subjects does not raise any raise any red flags. Young children present many challenges to radiologists. Unless sedated, they are often non

- compliant and a complete patient series may not be acquired. S ubject motion during the MRI acquisition can be a major confound when scanning children. Termination of MRI scanning session before an entire data set specified in the protocol can be acquired could explain why as little as half the data could be salvaged. b) The authors’ pre

- processing and processing of MR images conformed to standard methods and practices used in this field. There did not appear to be deviations to common practice in the MRI studies . That being said, this reviewer would have preferred to see the data analyzed using one or two standard processing pipelines. I suspect that the qualitative findings would have been the same but that there would have been quantitative differences among different approaches. c) The set of steps used in the acqui sition and image

- processing pipeline are all reasonable. T1

- weighted MRI brain data using a fast spoiled gradient

### recall ed echo (SPGR) MRI sequence (referred to incorrectly in the text) is the standard in the field providing both excellent anatomical fidel ity and high spatial resolution. Segmentation of gray and white matter areas using this MRI data was performed on a slice by slice basi s using differences in signal intensity values. This segmentation step, by itself leads to inac c uracies

, and is notorious ly non

- robust, but it is followed up by a manual correction step, which in principle should be adequate. The large deformation fluid registration method i s considered as one of the best approaches

. While there was an error in the ir stated formula of the Navier

- Stokes equation in the supplementary material (missing plus sign) this was deemed to be a typo rather than an error in the computer software used to implement this approach

. It would have been preferable for the authors to compare th e ir registration to other methods, such as Freesurfer, which wa s not done here, but this should not be considered an omission and is not necessarily a flaw in the study. M any people in the

c ommunity use FreeSurfer or CARET to perform cortical segmentation and thickness extraction. However, such method s depend on the existence of an average brain, and such an average is often difficult to obtain in pediatric subjects. d) Some people in the community construct an average brain from the group of brains within t he study. This can be problematic when there is significant variability in this sample. The authors choose a representative brain and then refine d their average. The first step is more similar to finding a median brain. There does not appear to be anything problematic with this approach. 3. In determining statistical significance, the authors used the False Discov ery Rate (FDR) test. This test is more permissive than the standard Bonferroni multiple comparison test, and is widely used in atlasing applications. The entire problem of assessing the significance of similarities or differences between signal intensities found within a large number of voxels in MRI is a developing area of statistical analysis

. 4. I don’t possess the knowledge of the social an d psychol ogical aspects of child development to comment on this question. 5. There are significant changes in brain structure occurring during this period in development; however, many of the primary white matter p athways that one sees in the brain of a ten year old a re present in skeletonized form already at birth. The topology or relative location of different brain structures does not appear to change. The large change in brain size can make it difficult to find an average brain to which to register other brains. 6. No comment here. 7. This appears to be a competently done study by a qualified group. Most of my reservations are not with a particular method or approach used, but with the overall problem of validating a particular image processing pipeline. 8. As I alluded to above, it would be valuable to reanalyze the MRI data using Freesurfer or CARET, which use different methods and assumptions, as a way to establi sh consistency in the results, at least in the qualitative findings

.

### Sadly, in such atlasing studies, as of y et, there is no “ground truth”. In this

case consistency and convergence of findings can help improve their credibility

.

### 2) a . Authors report that 20 of 28 eligible participants in the “high” exposur e group and

20 of 38 eligible pa rticipants in the “low” exposure group had usable MRIs. Such high rate of exclusion due to the quality of MRI scans is not unusual when young chi ldren are the patients. The quality of MRI data is strongly dependant on the motion stability of the subject. Any kind of movement during scan is creating artifacts. Low amplitude repetitive motion (h e art beats, respiration) could be often dealt with using variety of approaches, however , larger amplitude sporadic motion (especially related to claustrophobia or panic inside the closed space of the MRI) can easily render MRI data useless. It would be very helpful if authors had described the MRI sequence in m ore detail especially concerning the duration of the scan and steps taken to reduce motion artifacts if any. b. There is no defined ‘standard practice’ in research studies dealing with MRI morphometric analysis. The methods of image process and pre

- process described in Rauh et al (2012) seem reasonable and justified . The analysis software (ANALYZE 8.0) used in this study is well accepted and widely used in biomedical imaging research.

c. Authors used appropriate methods for gray matter segmentation and cort ical thickness measurements. d. Authors used elaborate procedure to select the final template brain from the studied population. While it is not obvious why the selection o f the most representative brain might benefit the statistical analysis, authors righ tfully point out that the use of individual brain as a template is preferable to averaged template due to better resolved white matter/gray matter, CSF/gray matter, and gyri/sulci interfaces. The bottom line is that the laborious method of two

- step selecti on of the template brain might have been overkill, but this did not adversely affect the actual image analysis.

6) It is difficult to compare the developmental neurotoxicity study conducted in laboratory rats (Hoberman, 1998a, 1998b, 1999 a, 1999b3) with t he data presented in Rauh et al (2012)

. Hoberman’s preclinical study does not seem to support clinical Rauh’s data, however there are significant differences in the methodology (dosing, morphometric methods, rats vs. humans, imaging vs. histology). Rats sh owed decreased brain size as well as the thickness of hippocampus gyrus and parietal cortex in high dose group of rats. Rauh et al did not see change in the size of the brain and the cortex thickness. The changes in the brain surface morphology were majorl y attributed to the changes in underlying white matter and the morphology of hippocampus was not analyzed.

### Response to Specific questions:

4) The authors reported several findings (i.e., areas of brain enlargement, qualitative statistical interaction between full scale intelligence quotient (FSIQ) and brain morphology; aberrations in the normal sexual differentiation of the brain, and sporadic cortical thinning). Please provide your interpretation of the results reported by Rauh et al (2012). Based upon your interpretation of the study, what, if any, neurodevelopmental consequences would you anticipate in these children?

The Rauh et al (2012) reported results show a diffuse pattern of morphological differences in the surface areas of the brains of the children exposed prenatally to high levels of CPF when compared to a representative template brain. Both significant enlargements, bilaterally, occurred across temporal, parietal, and frontal regions of the brain found to be primarily the result of enlarged white matter. Some areas in the left superior frontal gyrus of high level CPF brains showed depressed deformations of matter (presumeably less white matter). The high level CPF group showed a significant positive dose response between exposure level and enlargement of the superior frontal gyrus, bilaterally suggesting that as CPF exposure levels increased, so did the enlarged white matter in the superior frontal region of the brain. This pattern of morphological differences was not reported for the children exposed to low levels of CPF prenatally. When the groups were compared in overall cortical thickness, there was the presence of reduced cortical thickness in scattered areas across the brain surface in the high CPF group. Overall the pattern of results suggests a diffuse or generalized morphological effect on the brain white matter integrity and cortical thickness of children exposed to higher levels of CPF.

When IQ scores were added to the analyses, the low CPF group showed that lower IQs were associated with greater depressed deformations while higher IQs were closer to the template standard in areas of pre and post central gyri, the inferior frontal gyrus, the precuneus and the fusiform gyrus. Alternatively, the high CPF group showed morphological enlargements associated with lower IQs in the these same areas. These results support the contention that exposure to CPF, even for some within the low CPF exposure level group, is related to general cognitive deficits.

When sex was analyzed by CPF exposure level in sexually dimorphic brain regions, some reversal of usual female vs. male differences occurred. Specifically, females typically have larger right parietal lobes than males and this pattern was shown in the low CPF group; however, the high CPF group showed the reverse pattern with a larger right parietal lobe (and greater surface distance from the template) in males when compared to females. It is not clear what statistical analyses were used to determine this result. The results must be interpreted very cautiously since there were only 6 males in the high exposure group and 9 in the low exposure group (total males = 15 vs. total females = 25). Thus CPF exposure may also affect typical sex dimorphic development in the brain.

Overall the interpretation of the data from this study appears correct but when the authors list the various cognitive processes associated with the CPF effects on specific brain regions, the interpretation is reasonable but still an extension beyond what the results actually show. The study only used a general IQ measure to quantify cognition in this study and more specific cognitive and behavioral tests would be needed to pinpointspecific cognitive processes affected by CPF exposure. In general, among the children with higher CPF

exposure, and perhaps to some degree those with the lower CPF exposure, a range of cognitive deficits or differences might be expected depending upon the level of brain function integrity or general intelligence the child’s genetics provide. Those children with greater brain function integrity but higher CPF exposure might show soft signs of neurological impairment or non-specific cerebral dysfunction. Those with less functional integrity or other, non-CPF-related brain insults, lesions, or genetic predispositions, may show more severe but non-specific cerebral dysfunction even with lower CPF exposure. The results of this study suggest that one might expect that the most common effects of CPF exposure would be similar to the most common effects associated with a range of developmental brain insults, effects such as attention deficits, learning disabilities, and deficits in social development.

6) In the developmental neurotoxicity study conducted in laboratory rats (Hoberman, 1998a, 1998b, 1999a, 1999b 3 ) EPA’s Data Evaluation Review (DER) of this study, pp. 23 -25 and 29 of DER for MRID 44556901 & pp. 3-7 and 13-15 of DER for MRID 44787301 4 ], effects in brain morphometry were observed in rats exposed to chlorpyrifos during gestation. To what degree, if any, are the results of Rauh et al (2012) comparable with Hoberman ?

The only indirect comparison between the Hoberman study and the Rauh study is in regard to cortical thickness of the dorsal parietal cortex. The Hoberman study found that both males and female rats with a high dose of prenatal exposure to CPF showed decreased thickness of the parietal cortex compared to

contro ls with no prenatal exposure to CPF

. There was no analysis of the low dose prenatal exposure to CPF in the Hoberman study. In the Rauh study, the high CPF group included males and females and within that group there was a negative correlation such that greater CPF exposure was associated with thinner dorsal parietal cortex, bilaterally. The Rauh and Hoberman studies are not completely comparable but they provide some convergent evidence of a post-natal effect of higher CPF exposure on cortical thickness in the parietal
lobe.Question 1 : For the current peer review, EPA is soliciting comments on the epidemiological aspects of the study

design which are unique to Rauh et al (2012) (e.g., sample selection and inclusion/exclusion criteria; socioeconomic status (SES) considerations; sample size and statistical power, etc.) EPA’s has reviewed of the epidemiological context of the MRI study by Rauh et al (2012). Please comment on the Agency’s review with respect to the epidemiological aspects of the study.

- Sample selection and generalizability of the findings: Two racial/ethnic groups were represented: African American and Dominican. The sample of mothers is described as “low risk” based on beingfree of diabetes, hypertension, known HIV, and documented drug abuse, as well as being nonsm okers. The sample was described as low-income and urban. The average IQ of the full cohort
was 85.0. The authors state the findings are generalizable “to a larger urban population because it was a representative community- based sample.”

- In agreement with the EPA review, the homogeneity of the sample in terms of race/ethnicity and poverty/urban status is appropriate for detecting differences in a small sample of women/children, but may not be appropriate for generalizing to a larger urban population.

- The sample is not “low risk” based on the association between maternal poverty (or SES/environment) and child neural development.[ 2-4 ] This further impacts the generalizability of the findings, and need for future research with larger samples and other populations.

- It is not clear how representative the sample is of the community, because of possibly low participation rate, especially among the low exposure group. It is stated that 20/28 (71.4%) eligible individuals in the high exposure group completed the MRI and had useable data, while 20/38 (52.6%) in the low exposure completed the MRI and had usable data. It isn’t stated specifically how many individual’s chose not to participate or had unusable data, and whether that was different between the two groups. It is stated based on data in table 1 that there were no differences between those who participated and the overall cohort; however, based on small sample size (see below), it is not possible to conclude this with certaintly.

- Sample size.

- The sample size (20/group) appeared adequate for detecting differences in specific brain

areas.

- It is not clear whether the sample size is adequate for concluding that the high chlorpyrifos exposure caused the changes in the brain. In table 1, the sociodemographic characteristics of the study population are presented, showing no significant differences between the high and low exposure groups, as well as between the full MRI sample and the larger cohort from which they were selected. It would have been informative to include the actual p-values, rather than the notation NS for non-significant. In general, the high exposure group seems to have greater risk, and when comparing the high and low exposure groups, the sample sizes are small and power is low to detect what may be meaningful differences between the groups. Specifically, values in the high exposure group are lower for maternal education, maternal IQ, and birth weight, but higher for prenatal lead. Power analyses suggest the sample size was not adequate to detect whether the differences observed on these measures were significant. The highchlorpyrifos group had average prenatal lead exposure of 1.4 μg/dl compared to 0.8 μg/dl in the low exposure group. While these levels are not considered elevated, there is evidence that even low levels of lead can impact neurodevelopment, and even that the observed neurobehavioral deficits are more pronounced at lower blood lead levels when compared with higher blood lead levels. [ 5-8 ]



### Question 3 : Please comment on the statistical procedures used in Rauh et al (2012).

- The authors included age and sex of the child as covariates in the analyses, which is appropriate. It was stated that there was no association between lead and 

### chlorpyrifos, but any association between lead and brain differences was not studied or reported. In addition, it seems that one analysis used a continuous measure of chlorpyrifos exposure, while all others used a dichotomous variable (high/low, with no medium group). It isn’t clear why the continuous measure wasn’t used, or whether this would be more informative to demonstrate a relationship between chlorpyrifos exposure and neurodevelopment.

Question 7 : What additional comments would you like to provide us that may assist us in better interpreting the results and characterizing the uncertainty?

- A recent publication by the same group suggested there might be differences in the relationship between prenatal exposure to chlorpyrifos and working memory by sex, with males being more susceptible.[ 9 ] The authors controlled for sex in their analyses, and were also able to present preliminary results by sex, even with only 6 males in the high exposure group. Additional studies with larger samples are needed to better understand differences by sex.

Question 8 : Are there any additional analyses that should be conducted? And what would be the value or implications of these analyses?

- Differences in prenatal lead exposure should be considered, even though they were not statistically different between the two groups. Again, this may be due to inadequate sample size to detect a difference. The high chlorpyrifos group had almost twice the prenatal lead exposure as the low group. It is reported that lead was not associated with chlorpyrifos; however, given the sample size, this may not be a reliable finding. Therefore, lead should be included as a covariate, or analyses run determine whether lead levels are independently associated with the observed brain differences, in order to tease apart a potential role of lead versus chlorpyrifos.

- Also, if there are data available on other environmental exposures (other pesticides, etc.), the association between these exposures and the outcomes should be studied.

- In general, it is agreed that the research requires replication (as stated by authors and in the EPA review).

REFERENCES

1. Rauh, V.A., et al., Brain anomalies in children exposed prenatally to a common organophosphate pesticide. Proc Natl Acad Sci U S A, 2012. 109 (20): p. 7871- 6. 2. Farah, M.J., et al., Childhood poverty: specific associations with neurocognitive development. Brain Res, 2006. 1110 (1): p. 166-74. 3. Hackman, D.A. and M.J. Farah, Socioeconomic status and the developing brain. Trends in Cognitive Science, 2009. 13 (2): p. 65- 73.4. Tucker-Drob, E.M., et al., Emergence of a gene x socioeconomic status interaction on infant mental

ability between 10 months and 2 years Psychological Science, 2010: p. 1- 9. 5. Bellinger, D.C., Very low lead exposures and children's neurodevelopment. Curr Opin Pediatr, 2008. 20 (2): p. 172- 7. 6. Canfield, R.L., et al., Intellectual impairment in children with blood lead concentrations below 10 microg per deciliter. N Engl J Med, 2003. 348 (16): p. 1517- 26. 7. Chiodo, L.M., S.W. Jacobson, and J.L. Jacobson, Neurodevelopmental effects of postnatal lead exposure at very low levels. Neurotoxicol Teratol, 2004. 26 (3): p. 359-71. 8. Tellez-Rojo, M.M., et al., Longitudinal associations between blood lead concentrations lower than 10 microg/dL and neurobehavioral development in environmentally exposed children in Mexico City. Pediatrics, 2006. 118 (2): p. e323-30. 9. Horton, M.K., et al., Does the home environment and the sex of the child modify the adverse effects of prenatal exposure to chlorpyrifos on child working memory? Neurotoxicol Teratol, 2012.

### Answers to Questions for Federal Child Development Experts

1a. Was the type of testing age appropriate for the outcome measures of memory and IQ testing in children reported in the Columbia Mothers and Newborn Study as discussed in the papers by Rauh et a l., (2006 and 2011)?

There were no direct measures of memory or IQ in the Rauh et al., 2006 paper. Children were tested at th e ages of 12, 24, and 36 months. Two of the three age groups were younger than the youngest appropriate age for many of the more common measures that are used to assess IQ since many of these IQ tests require the child to be able to use and comprehend language. The youngest age to assess IQ for four widely used measures of IQ are as follows: The Wechsler 

### Intelligence Scale for Children-Fourth Edition
age 

### 6 years ; The Kaufman Brief Intelligence Test, Second Edition age 4 years 

### ; The Wechsler Abbreviated Scale of Intelligence age 

### 6 years, and The Wechsler Preschool and Primary Scale of Intelligence-Third Edition 

### age 2 years 6 months. While the Wechsler Preschool and Primary Scale of Intelligence-Third Edition may be administered to children as young as age 2 years 6 months, children in the younger age group of 2 years 6 months to 3 years 11 months are tested on only four core subsets as opposed to seven core subsets administered to children ages 4 years to 7 years 3 months.



### The Rauh et al., 2011 paper did directly assess memory and IQ using the Wechsler Intelligence Scale for Children-Fourth Edition (WISC- IV ). Th e use of this test was appropriate for the sample because all 265 subjects had reached the age of seven and the WISC-IV was designed for examinees aged 6 years 0 months to 16 years 11 months.

1b. Was the type of testing conducted correctly for the outcome measures of memory and IQ testing in children reported in the Columbia Mothers and Newborn Study as discussed in the papers by Rauh et al., (2006 and 2011)?

The Rauh et al., 2006 paper provided extensive detail regarding how the psychological tests were administered, and the efforts made to maximize reliability of testing and scoring. However, no infor mation was provided regarding the qualifications of the individuals who administered and scored the tests. The Rauh, et al., 2011 paper did not provide any information regarding the administration, scoring,

or qualifications of the individuals who administered the IQ test. Although it should be mentioned that
t he WISC- IV generally has very understandable administration and scoring instructions and that the manuals are easy to use, the manual does suggest that examiners be familiar with the Standards for Educational and Psychological Testing and that examiners complete graduate training in assessment or be
super vised by such a person. In addition the manual suggests that examiners have specialized training when testing persons from unique linguistic, cultural, or clinical backgrounds. Unfortunately, none of this infor mation about the examiners was provided in either study.

1 c. Are there population norms associated with these measures?

There are population norms associated with the measures used in both Rauh, et al. studies. Below is some infor mation regarding the population norms for each of these measures.

Bayley Scales of Infant Development, Second Edition (BSID-II)

The BSID- II was standardized on 1,700 infants and children between the ages of 1 and 42 months. The sample spanned 17 age levels and contained 100 individuals (50 males and 50 females) at each level. Age

### intervals occur monthly up to 12 months (with the exception of ages 7, 9, and 11) and every 3 months

thereafter to the age of 30 months, and then at 6-month intervals. The standardization sample closely approximates the regional, ethnicity, and parent education characteristics of children 1 to 42 months of age as determined by the 1988 U.S. Census data. The test developers made an effort to limit the standa rdization sample to children who were not diagnosed with medical, psychological, or behavioral problems in order to allow examiners to compare individual child performance to that of normal developing peers (Gyurke, 1994).

Wechsler 

### Intelligence Scale for Children

- Fourth Edition ( WISC

- IV )

The standardization sample used for the WISC

- IV included 2,200 examinees divided into 11 age groups, 200 per group, but the Arithmetic subtest was normed on a sample of 1,100 children. The sample was representative of the March 2000 U.S. Bureau of the Census data in terms of age, gender, race ethnicity, parent education level, and four geographical regions.

1 d. If so, how can these be used to help characterize the findings in the (Rauh et al., 2006 and 2011) stud ies?

The answer to this question depends on the question that is specifically being addressed in the research studies. The BSID- II and the WISC- IV can be used in the current study to determine if mental development, psychomotor development, and IQ are worse in children who are exposed to higher levels of chlorpyrifos compared to children who are exposed to lower levels of chlorpyrifos. However, given that the sample of children used in the Rauh et al. studies had very different characteristics than that of the

sa mples on which these measures were standardized, caution should be used when describing the results, especially when attempting to generalize these findings to the general population.

F or example, here is some information regarding the approximate percentages from the U.S. Census March 2000 Population Survey: School Enrollment Supplemental File which is similar to the sample used
to st andardize the WISC- IV compared to the sample in the Rauh et al. studies. A similar comparison could be made for the Rauh et al

. studies compared to the BSID- II that were standardized based on the 1988 U

. S. census data.

Race Ethnicity: Population Survey = White – 63%, Hispanic – 16%, African American – 16%, Asian – 3.5%, Other – 1.5%

. Rauh studies = Dominican – 55%, African American – 45%.

Parent Education Level: Population Survey = High School Graduate or higher – 85% Rauh studies = High School Graduate or higher – 69%.

Ge o graphic Regions: Population Survey = Northeast – 18%, South – 34%, Midwest – 24%, West – 24%. Rauh studies = Northeast – 100%.

Household Income Population Survey = < $20,000 – 22.1%. >= $20,000 – 77.9% Rauh studies = < $20,000 – 52%. >= $20,000 – 48%

### 2 a . For the outcome measures in the selected papers, how do the distributions compare to

population norms or other studies using these outcome measures (memory, IQ) in the Columbia cohort?

The distributions of the BSID- II scores for both psychomotor and mental development are reported in the R auh et al., 2006 paper for both the low and high exposure groups. The distributions are based on standard scores that have a mean of 100 and a standard deviation of 15. The distribution of the scores for the Rauh sample for the psychomotor development index appear to be relatively similar to those of the population norms for

this instrument. In contrast, the mental development index scores for the lower

e xposure group range from about 6 points less at age 12 months to almost 1 standard deviation (15 points) be low the mean for the test at age 24 months. Therefore, this sample of children are already below (and at 24 months well below) the average score for this measure based on the sample on which the measure was
st andardized. Unfortunately, it is impossible to compare the WISC- IV scores to the population norms because no information was provided regarding the distribution of WISC- IV scores (i.e., mean and standard deviation) in the Rauh et al., 2011 paper. There is a scatterplot showing the raw scores for overall IQ and for each of the subtests, but it is not possible to obtain the necessary information to compare the distributions of these scores with the norms for the test or with any other study sample. Ideally, the means and standard deviations for these scores should be presented for either a non-exposedor a non-exposed combined with low exposed group and these should be compared to a moderate or highexposed group as was done for the BSID- II in the Rauh et al., 2006 paper. It should also be noted that the IQ composite scores were reported as natural logs of the obtained composite scores. Hence in order to obtain the IQ scores the values that were reported would need to be converted back to their original

co mposite scores.

2b. With respect to the domains that make up the outcome measures, how do they compare to
p opulation norms or other studies populations?

S ee 2a. above. It should be mentioned that for each of the subtests of the WISC- IV no information w as reported regarding the means or standard deviations of the samples, hence it is not possible to compare these data to the population norms for this measure.

2c . A re there domains which are considered to be the more/less important in cognitive development?

This is a difficult question to answer because there is little or no consensus regarding what domains are or are not as important in cognitive development. However, the assessments that were used in the Rauh et

a l. research to assess infant development and IQ have been widely used in a variety of studies.

3a. Were the tests used to measure the outcome valid

The Bayley Scales of Infant Development II (BSID-II) manual presents extensive information regarding the construct and criterion validity of this instrument. The strong correlations between the Wechsler Preschool and Primary Scale of Intelligence-Third Edition (WPPSI-III) (.73), and the McCarthy Scales of Children's Abilities(MSCA) (.79) provide evidence of criterion validity for children at the older age range (36 to 42 months) of the

BSID-II. The manual presents data from a series of studies with children demonstrating a variety of developmental disabilities. These results seem to support the discriminant validity of the BSID-II. Collectively, the validity information provided in the manual tends to support the construct and criterion-related validity of the BSID-II.



### The Wechsler Intelligence Scale for Children-Fourth Edition (WISC- IV ) has been found to have good correlations with a variety of other measures of IQ and memory including the Wechsler Preschool and Primary Scale of Intelligence-Third Edition (WPPSI-III), the Wechsler Abbreviated Scale of Intelligence (WASI), the Children's Memory Scale (CMS ), and the second edition of the Adaptive Behavior Assessment System (ABAS-II).

3b. Were these tests applicable to the research study question?

B oth the Bayley Scales of Infant Development II and the Wechsler Intelligence Scale for Children IV are wide ly used research tools for measuring mental development and IQ. A text search on the PsychInfo database for the Bayley Scales of Infant Development yielded 188 results. A similar search for the Wechsler Intelligence Scale for Children IV yielded 341 results. This indicates that both measures have been, and are still widely used to assess mental development and IQ, which were both outcomes of the Rauh et al. research studies.

4 a . What are the uncertainties around the outcome measurements in this cohort?

The main uncertainties around the outcome measures stem from the fact that those measured were standa rdized on a sample that was very different than the sample that was used in the Rauh et al. research. This is not necessarily problematic when the means for the composite scores (i.e., standard scores) are directly compared between two groups within the Rauh et al. sample (as was done for the Bayley Scales of Infant Development II in the Rauh et al., 2006 paper). However, caution should be taken when interpreting the data from analyses that rely heavily on these standard scores. For example, in the Rauh et al., 2006 paper, one potentially interesting and important finding was that a sig nificantly greater percentage of children in the higher exposed group had scores of less than or equal to 85 (i.e., 1 standard deviation below the mean) for the Mental Development Index (MDI) component of the BSID- II at 36 months of age and this value was also nearly significant at 24 months of age. This was despite the fact that the actual scores on this measure were not significantly lower for the higher exposed than for the lower exposed group. This inconsistency may have been due to the fact that the scores for the MDI section of the BSID- II for the lower exposed group were near the threshold of 85 to begin at both the 24 and 36 month age groups so that even a small decrease in scores would cause more children to be below this threshold than if the scores on this measure were closer to 100. It is interesting that for the P sychomotor Development Index (PDI) scores at the ages at which significant differences occured is sim ilar whether one compares the means of the scales between the exposure groups or the percentage of

c hildren who scored below 85. This may be due to the fact these scores were closer to the standardized mean of 100 on this scale to begin with.

Unfortunately, the Rauh et al., 2011 study did not report any means or standard deviations for overall IQ or for any of the subtests. Here the uncertainties stem from the assumptions that are made when

re gression analyses are performed. The main issue here is that outliers can greatly influence the slope of the function. For this study the bulk of the data were obtained from children who had exposures of less than 10 pg/g. In addition only 7 children had exposures of 15 pg/g or greater, and 2 had exposures of

### 20pg/g or greater. Hence the regression (i.e., the slope) of the function would be greatly influenced by the

data from these children. A second issue regarding the analysis of the data from the Rauh et al., 2011 paper is that although the correlation between full scale IQ and exposure was significant, as was that of working memory and exposure, the correlations were not particularly high (-.13 for full scale IQ, and -.21 for working memory). In fact, it has been suggested that a correlation should be .5 or higher to be large, .3 or higher to be moderate, and .1 to be small (Cohen, 1988). And in fact, the coefficient of determination which is a measure of the proportion of variability in the data that is explained by the relationship between the two variables would be .0169 for IQ and .0441 for working memory which,

a ccording to Gravetter and Wallnau (2007), would be indicative of small correlations (a value of .09 (i.e.,

r = .3 or -.3) or greater would be required for medium correlations and .21 or greater for large correlations).

4b. Are socio-economic status and other exposures adequately addressed?

The investigators did an excellent job of addressing the issues of other exposures and socio-economic status of the children in their sample.

5. What additional comments would you like to provide us that may assist us in better interpreting the results and characterizing the uncertainty?

It should be mentioned that the results from the Child Behavior Checklist in the Rauh et al., 2006 study

we re analyzed in a manner that was similar to the analysis conducted on the Bayley Scales of Infant Development II with regard to the proportion of children who had a score at or below a certain value. The da ta fo r the Child Behavior Checklist are reported as the percentage of children who met criteria for being in the clinical problem range on one of several dimensions with higher scores indicating a greater problem in t hat domain. Once again, this sample of children may have had scores that were already well above the

mea n for this measure such that even a small increase in scores would be enough to be above the critical point on that measure. Unfortunately, this could not be determined because the authors did not provide the me ans and standard deviations for the children in their sample on this measure for either the lower or higher exposed group.

6. Are there any additional analyses that should be conducted? And what would be the value or implications of these analyses?

A between group analysis using inferential statistics, as was done for the Bayley Scales of Infant De velopment II in the Rauh et al., 2006 paper, should be performed on each variable in both studies (i.e., the Chi ld Behavior Checklist in Rauh et al., 2006, and the full scale IQ and subscales for the WISC- IV in the Rauh et al., 2011 study). This would be the most direct and least problematic method for determining whether exposure to chlorpyrifos resulted in significant decreases in IQ or significant increases in be havioral problems. If significant differences were found between the groups, this would provide much stronger evidence than the evidence that is currently provided by the analyses that were presented in the R auh, et al., 2006 and the Rauh, et al., 2011 papers.

### References:

Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New Jersey: Lawrence Erlbaum.

Gravetter, F.J. & Wallnau, L.B. (2007). Statistics for the behavioral sciences (7 th 

### ed.). California: Thomson Wadsworth.

Gyurke , J. S. (1994). A reply to Bagnato and Neisworth: intelligent versus intelligence testing of preschoolers. School Psychology Quarterly, 9 (2), 109-112.