A WEIGHT-OF-EVIDENCE ASSESSMENT OF RECENT EPIDEMIOLOGY STUDIES LINKING
CHLORPYRIFOS WITH HEALTH OUTCOMES IN INFANTS AND CHILDREN

Douglas L. Weed, M.D., M.P.H., Ph.D.

Founder and Managing Member

DLW Consulting Services, LLC

Kensington, MD 20895

September 8, 2008

Comments to the U.S. Environmental Protection Agency

Docket ID number EPA---HQ---OPP---2008---0274

TABLE OF CONTENTS

INTRODUCTION 4

SUMMARY OF FINDINGS 5

Birth Outcomes

Neurodevelopmental (Behavioral) Outcomes

Comments on EPA Preliminary Conclusions

CONTEXT 6

Scientific Evidence on Chlorpyrifos Published Prior to 2003

Children's Health as a Societal Value

The Distinction between Regulatory Action and Scientific Claims of
Causation

The Role of Industry in Environmental Health Science

METHODS 9

A Weight-of-Evidence Approach for Assessing Causation

Weight-of-Evidence Methods as Interpretative Methods

Systematic Narrative Review

Criteria-based Methods

The General Scientific Method as a Component of a Weight-of-Evidence
Approach

Study Design and Analytical Methods

RESULTS OF PUBLISHED EPIDEMIOLOGICAL STUDIES 15

Epidemiological Studies of Chlorpyrifos and Children's
Neurodevelopmental and Birth Outcomes: A Descriptive Summary of Designs
and Results

The Studies

Columbia Cohort

Mount Sinai Cohort

Salinas Valley, CA Cohort

Similarities and Differences in Study Populations

Birth Outcomes: Results Organized by *in utero* Chlorpyrifos Exposure

Neurodevelopmental Outcomes in Childhood: Results Organized by Exposure
Assessment and Age at Testing

CRITICAL REVIEW OF PUBLISHED COMMENTARY 21

Published Reviews of the Epidemiology Studies

A Critique of the Published Reviews

WEIGHT-OF-EVIDENCE ASSESSMENT 24

Chlorpyrifos and Gestational Measures

The Mount Sinai Cohort

The Salinas Valley Cohort

The Columbia Cohort

Conclusion

The Causal Criteria Applied to the Chlorpyrifos Studies: Birth Outcomes

Conclusion

Chlorpyrifos Exposure and Neurodevelopmental (Behavioral Outcomes)

Conclusion

SUMMARY OF CONCLUSIONS 29

ACKNOWLEDGEMENTS 30

REFERENCES 31

Commentaries, Reviews, Editorials, and Letters

Epidemiological Studies

Other Literature Cited

TABLES 35

Table 1. Description of Cohorts Exposed to Pesticides during Pregnancy

Table 2. Chlorpyrifos Exposure as measured by urine TCPy and birth
outcomes in the Mount Sinai and Salinas Valley cohorts

Table 3. Relationship between Chlorpyrifos in cord blood and Birth
Outcomes: Columbia Cohort

Table 4. Neurobehavioral Scale Scores at 12 and 24 months in two
Cohorts: Columbia and Salinas Valley

Table 5. TCPy in Prenatal Urine Samples and Neurodevelopment at 12 and
24 months in two Cohorts: Columbia and Salinas Valley

Table 6. Chlorpyrifos Exposure and Neurodevelopment at 36 months among
inner city children: Columbia Cohort

INTRODUCTION

On April 18, 2008, the Federal Register announced that the U.S.
Environmental Protection Agency (EPA) was seeking comments on the
interpretation of recent epidemiological studies associating in utero
and/or post-natal chlorpryifos exposure with health outcomes (Federal
Register, 2008a). In a memo dated August 21, 2008, the EPA also asked
for comments on preliminary conclusions made by the Agency, specifically
that these same studies "provide valuable information on the effects in
children of high exposure to pesticides," and that "chlorpyrifos likely
played a role" in both birth and neurodevelopmental outcomes. In
response to these requests, I provide in this report an interpretative
assessment of the epidemiology studies examining possible links between
chlorpyrifos, a widely-used pesticide, and neurodevelopmental (including
birth) outcomes in children. By "interpretation" I am referring to a
"weight-of-evidence" (WOE) approach, that uses several well-established
methodologies for interpreting scientific evidence (Weed, 2005). These
will be described in detail below. By "recent epidemiological studies" I
am referring to a series of scientific studies published from 2003
through 2006 by investigative groups affiliated with the Centers for
Children's Environmental Health at the University of California,
Berkeley, the Mount Sinai Medical School (Landrigan et al., 1999), and
at Columbia University (Perera et al., 2002). These studies examine
putative relationships between chlorpyrifos exposure and health outcomes
in three cohorts: the Salinas Valley cohort, the Mt. Sinai cohort, and
the Columbia cohort.

The key interpretative questions to be examined in this report involve
whether it can be or should be claimed that chlorpyrifos causes
neurodevelopmental (and/or birth outcome) effects (more broadly, health
outcomes) in children given the current scientific evidence. My
interpretation will be based on a critical methods-based examination of
published studies. Inasmuch as others have provided published opinions
on these questions, I will also provide critical assessments of those
claims. These opinions are typically found in published reviews,
commentaries, and to some extent, the discussion sections of published
studies.

SUMMARY OF FINDINGS

Birth Outcomes

Given the available epidemiological evidence, exposure to chlorpyrifos
does not cause adverse birth outcomes, defined as gestational measures
(i.e. birth weight, birth length, head circumference, and gestational
age). Despite a concerted effort on the part of three different
investigative groups, the evidence regarding the relationship between
gestational measures and chlorpyrifos exposure is remarkably
inconsistent. There is a fundamental question in these epidemiological
studies regarding whether a statistical association exists between
chlorpyrifos and any gestational outcome much less a causal association.
Finally, there is no strength of association, a lack of temporality, no
dose-response relationship, and only a relatively weak plausibility
argument.

Neurodevelopmental (Behavioral) Outcomes

Given the available epidemiological evidence, exposure to chlorpyrifos
does not cause adverse neurodevelopmental outcomes in children who have
been exposed in utero. The evidence shows no indication of any adverse
effect at one year or at two years of age. At three years of age,
several measures of neurodevelopment were significantly lower in the
children of one of the three cohorts. These children may have been
exposed to "high" levels of chlorpyrifos, although these results were
obtained by redefining exposure categories post hoc to achieve
statistical significance. Furthermore, these findings were restricted to
a single study in which the measurement of chlorpyrifos exposure was
made at birth. No testable scientific explanation was provided for why
differences in neurodevelopmental and behavioral outcomes should be
observed in this cohort at 36 months but not at 12 and 24 months. A
causal claim regarding chlorpyrifos exposure and neurodevelopmental
(behavioral) outcomes from this evidence is not warranted.

Comments on EPA Preliminary Conclusions

According to a memo dated August 21, 2008, the EPA preliminarily
"believes that all three studies (i.e. the cohorts from Columbia
University, Mt. Sinai School of Medicine, and the University of
California, Berkeley) provide valuable information on the effects of
children of high exposure to pesticides, particularly OPs." As discussed
in some detail in this document, these studies do not provide sufficient
information to attribute adverse "effects" to chlorpyrifos. The EPA also
preliminarily "concludes that chlorpyrifos likely played a role in these
outcomes." There is insufficient evidence provided in these studies to
make such a claim. In particular, there is insufficient evidence to
claim that statistical associations exist between chlorpyrifos and
several birth and neurodevelopmental outcomes much less causal
associations. In the absence of consistent statistical associations, a
conclusion that chlorpyrifos "likely" played a role in these outcomes is
not warranted.

CONTEXT

The interpretation of scientific evidence regarding human health is
never done in a vacuum. Contextual issues can potentially influence how
one interprets scientific evidence. Issues of particular importance to
this assessment of the relationship between chlorpyrifos and children's
health include:

-   the scientific evidence on chlorpyrifos and health published prior
    to 2003

-   the societal value placed on children's health

-   the distinction between regulatory actions and scientific claims
    regarding causation

-   the role of industry in environmental health science

Each of these issues has its own published literature too vast to be
systematically reviewed here. Nevertheless, each of these issues could
influence how one answers the question facing the EPA: how should recent
epidemiological studies regarding chlorpyrifos exposure and health
outcomes in children be interpreted?

*Scientific Evidence on Chlorpyrifos Published Prior to 2003 and its
Interpretation*

A substantial scientific literature exists on chlorpyrifos exposure
assessments and possible health effects in humans (e.g. reports of acute
poisoning as well as investigations of exposed occupational
populations); much of it was published prior to 2003. A literature also
exists on the effects of chlorpyrifos in laboratory animals as well as
the possible biological mechanisms of the actions of chlorpyrifos. And
finally, there is a scientific literature on the topic of children's
neurodevelopment. Systematic reviews of these various topics are beyond
the scope of this report. Nevertheless, some general conclusions and
specific publications from that literature are relevant to the topic at
hand.

Indeed, published studies have, on occasion, stimulated controversy
regarding the putative health effects of chlorpyrifos and its
regulation.

Consider, for example, a chlorpyrifos exposure assessment study that
appeared in the journal, Environmental Health Perspectives in 1998
(Gurunathan et al., 1998); interpretations of the results of this study
were remarkably different.

The study examined the accumulation of chlorpyrifos on children's toys
after the time suggested for reentry following application of the
pesticide. A news item in the same journal accompanied the study's
publication, entitled "Playing with Pesticides" (EHP, 1998). It
concluded that the results of the study were "likely to shed doubt on
whether a June 1997 agreement between the EPA and industry to reduce
consumer exposure to the pesticide will be sufficient to protect
children." Six months later, two commentaries appeared in the same
journal (Davis and Ahmed, 1999; Gibson et al., 1999). One alleged that
the study results suggested that "exposures from indoor spraying of
chlorpyrifos pose greater health risks to children than currently
estimated" (Davis and Ahmed, 1999, p. 299). The authors listed their
affiliation as the World Resources Institute. The second commentary was
written by scientists working at Dow AgroSciences, a chemical company
that produces chlorpyrifos-containing pesticides (Gibson et al., 1999).
The authors noted that the "scientific weight of evidence strongly
supports (the) safety (of chlorpyrifos) when used in the indoor
environment" (Gibson et al., 1999, p. 303).

It is important to point out that both groups of authors believe that a
"weight-of-evidence" approach is the best way to assess the scientific
literature. In Davis and Ahmed's words, for example, weight-of-evidence
can be used to "resolve issues involving the safety of widely used
compounds such as chlorpyrifos" (1999, p. A583). Gibson et al. also
clearly supports the use of weight-of-evidence approaches, as noted
above.

More to the point of this section, however, this brief example---only a
small fraction of the available literature on chlorpyrifos and health
outcomes prior to the epidemiological studies published from
2003-2006---serves to underscore the relevance and importance of several
of the contextual issues mentioned above. Regulatory issues are
addressed, for example. But there is also a difference of opinion in
1999 regarding the extent to which chlorpyrifos, when used as an indoor
pesticide, causes adverse health effects in children. And that fact---so
obviously documented in the competing assessments by Davis and Ahmed
(1999), on the one hand, and Gibson et al. (1999), on the other---brings
up a key contextual issue: society's interest in children's health.

*Children's Health as a Societal Value*

An important public health challenge is to protect children's health
(Faustman, et al., 2000). Children's health is a key indicator of a just
and beneficent society. From a scientific perspective, considerable
interest has been generated about the extent to which children (and so
their response to environmental toxicants) are different from adults
(Tilson, 1995; Landrigan et al., 2004). As Bearer (1995, p. 10) writes:
there are many "reasons why children cannot be considered little adults
in the area of environmental medicine." Putting aside the difficult
scientific issue of making claims about children's health outcomes from
studies of adult health outcomes, the key concept here is that
children's health is of undeniable importance to our society.

The science of children's environmental health is best served by not
allowing one's heightened concerns about children's health, itself a
legitimate and vital societal value, to influence scientific assessments
and claims regarding causation. It is critically important to emphasize
that scientific claims about causation of adverse outcomes in children
are just that: *scientific* claims. Ideally, these claims should not be
influenced by the fact that children are being studied. Put another way,
methodologically sound scientific studies and interpretations should
stand on their own.

*The Distinction between Regulatory Action and Scientific Claims of
Causation*

Making claims about causation and determining what regulatory actions
should be taken (e.g. such as the 1997 agreement by the EPA and industry
to reduce chlorpyrifos exposures mentioned above) are very different
activities that require overlapping but different values and
methodologies (Weed and McKeown, 2003). On the other hand, scientific
claims and regulatory actions are not independent of one another.
Regulatory actions rely on scientific assessments, which in turn, may
provide important information on the impact of regulatory actions. For
example, regulatory actions may be taken in the absence of firm causal
claims in circumstances wherein a judgment is made that the potential
harms (of exposure) outweigh the potential benefits. In this report, I
will make no comments regarding the advisability of past, current, nor
future regulatory actions regarding the use of chlorpyrifos. Nor will I
comment on public health recommendations or clinical practice
recommendations regarding the use of chlorpyrifos.

*The Role of Industry in Environmental Health Science*

As discussed above, the Gurunathan et al. (1998) exposure assessment
study and the dueling commentaries from the World Resources Institute
and Dow AgroSciences, reveals a tension regarding competing causal
claims. It also reveals a tension that centers upon the role of industry
in environmental health science. This tension has at least three
dimensions: (1) some critics believe that industry science is less
transparent than it should be, (2) other critics maintain that industry
science has, as one of its central purposes, to create additional
uncertainty around scientific claims that should be considered
established, and (3) still other critics contend that industry funding
produces inherently biased scientific results and interpretations of
those results (Davis and Ahmed, 1998b, Michaels, 2006). Importantly,
industry scientists have discussed and countered such accusations
(Acquavella, 1997), including scientists at Dow Chemical Company (Barrow
and Conrad, 2006).

I will not comment further on the fairness much less validity of the
attacks on industry science. I mention this issue because I have been
asked to provide this assessment by a company that manufactures
chlorpyrifos, Dow AgroSciences, and I have been compensated by that
corporation for preparing this report. With these facts in mind, and in
the interest of full and transparent disclosure, it is important to
point out that I agreed to undertake this project only if I had full and
final control over the content of this report, a condition to which Dow
AgroSciences both agreed to and upheld.

METHODS

A Weight-of-Evidence Approach for Assessing Causation

A weight-of-evidence approach is the methodological foundation for an
assessment of the epidemiological studies on the association between
chlorpyrifos and children's neurodevelopmental and birth outcomes.

"Weight-of-evidence" has several meanings in the practice of
contemporary biomedical science (Weed, 2005; Krimsky, 2005).
"Weight-of-evidence," for example, can be a metaphorical concept without
reference to methodology. Alternatively, "weight-of-evidence" can
emphasize that all---rather than selected "positive" studies---are to be
evaluated. The second of these meanings has some relevance to this
report, as mentioned later, but another, more comprehensive and nuanced
meaning is more important.

The weight-of-evidence approach used in this report refers to
well-established interpretative methodologies: the systematic narrative
review and the criteria-based methods of causal inference. These will be
described in detail below. In addition, it is important to emphasize
that weight-of-evidence interpretations of scientific evidence always
involve a number of other methods. These include the general scientific
method---the foundation for the acquisition of scientific knowledge---as
well as the familiar study design and statistical (analytical) methods
such as cohort studies, case-control studies, and regression techniques.
In an effort to provide the EPA with a clear sense of the full range of
methods used in this assessment, I offer the following descriptions.

Weight-of-Evidence Methods as Interpretative Methods

Several methods are generally recognized in this category (Weed, 2000):

1.  The systematic narrative review

2.  Meta-analysis

3.  Criteria-based methods of causal inference

Two of these are particularly important in a discussion of the
relationship of chlorpyrifos to children's neurodevelopmental health
outcomes: the systematic narrative review and the criteria-based methods
of causal inference.

*Systematic Narrative Review*

The systematic narrative review is a critically important methodology in
the practice of interpreting evidence and therefore in causal inference
(Weed, 1997). Systematic collection, summarization, and interpretation
of relevant scientific evidence are key components of this methodology.
As such, the systematic narrative review acts as a template or framework
for the remainder of the methods used in a comprehensive
weight-of-evidence approach. Typically, medical library databases are
searched with a description of the search techniques made sufficiently
transparent that the search could be repeated by others with similar if
not exactly the same results. In addition, it is common for authors of
systematic reviews to supplement the searches with additional studies
found in the reference lists of published papers or textbook chapters on
the topic, government reports, and possibly, unpublished studies. The
purpose of the review, the conditions (or criteria) for including and
excluding the studies to be summarized and interpreted, and the criteria
(or other methods) to be used in making causal claims are
important---indeed, essential---components of a systematic narrative
review. The word "narrative" is often used to describe this methodology,
because it is common for authors to describe each study before the
results are summarized and finally interpreted in terms of causation.

For this report, I searched PubMed for all articles published between
2003 and the present using the terms, "chlorpyrifos" and "epidemiology."
From the reference lists of those articles and from the "related
literature" section of PubMed, I found an additional set of relevant
publications. The cited literature can be found in the bibliography of
this report. It should be noted that the identity of the "recent
epidemiological studies" referred to in the EPA's request for comment
are not really in question. In the following sections, the studies
selected for summarization and interpretation will be carefully
described.

Within a systematic narrative review of epidemiology studies, it is
common to find some mention of (or a comprehensive application of)
criteria-based methods of causal inference. A description of these
methods follows.

*Criteria-based Methods*

The criteria of causation, often referred to as "Hill's criteria," are
traditionally the most important step in causal inference involving both
studies of human populations and laboratory-based studies of biological
mechanisms (Hill, 1965; Weed and Gorelic, 1996; Weed, 1997; Weed, 2003).
Typically, these criteria are only applied to a body of evidence once a
statistical association has been established, i.e. once at least one
(and typically several) epidemiological studies have revealed increased
risks of a disease (or condition) that could be called "statistically
significant," often at the level of (p \< 0.05) (Weed, 2006).

It will be helpful and important to describe these so-called criteria,
their characteristics, and some of their strengths and limitations. I
use the word "so-called" because so many commentators have emphasized
the fact that these are not criteria in the classic sense of the word,
with one exception. The criterion of [temporality]{.underline}---that a
causal factor must precede its effect---is a true criterion. In the
absence of temporality, i.e. if the presumed effect precedes its
putative cause, then causation can, for practical purposes, be ruled
out. In some epidemiology studies, the exposure and the disease are
measured simultaneously, calling into question whether the criterion of
temporality has been satisfied. But the existence of the appropriate
temporal order---a putative cause followed by its presumed effect---does
not guarantee causation, because many alternative explanations remain to
be considered and because causation is not guaranteed because one event
follows another. For example, it is fair to say that temporality may
support causation when it exists, but there are at least eight other
so-called criteria to be considered. The remaining eight are presented
in roughly the same order that Austin Bradford Hill, the medical
statistician whose 1965 paper on this topic remains a classic source,
discussed them:

[Strength of Association]{.underline}: refers to the magnitude of the
relative risk estimates. Typically, it is believed that the larger the
relative risk, the more likely the observed association is causal. For
example, relative risk estimates can be obtained from well-designed RCTs
and cohort studies comparing the incidence of a condition in those
exposed to the putative cause to the incidence of the same condition in
those unexposed. The magnitude of any comparison (including, for
example, comparisons of physical characteristics such as head
circumference, birth weight, or birth length) is a key issue in
characterizing the extent to which the association can be considered
"strong." Traditionally, investigators examine the role of confounding
(both known and unknown confounders) in altering (or not) the strength
of the association.

[Consistency of Association]{.underline}: refers to the extent to which
scientific results are similar (e.g. in direction) across the entire
body of evidence. Commonly, this consideration includes the extent to
which the basic causal hypothesis (e.g. does chlorpyrifos exposure cause
neurodevelopmental effects) has been tested in different populations,
using different study designs, different exposure metrics, and in
studies designed and managed by different investigators. Typically, it
is believed that the more consistent the results the more likely the
observed association is causal.

[Biologic Gradient (Dose-Response]{.underline}): refers to the extent to
which the relative risk estimates (or other outcome measures) increase
in magnitude as the dose of the exposure increases. Typically, it is
believed that a regularly increasing relationship between exposure and
risk estimate is more likely to represent a causal relationship than
other patterns.

[Biologic Plausibility]{.underline}: refers to the extent to which a
mechanism of action has been proposed and/or studied, typically in
laboratory studies involving animals and/or human tissues. This
"criterion" has been examined in some detail (Weed and Hursting, 1998).
There are prominent differences among the following definitions of this
consideration: whether the association "makes sense," whether evidence
exists (e.g. in biological systems) that precisely the same outcome
occurs in other species at comparable exposure levels, and finally, the
extent to which a mechanism of action has been demonstrated rather than
simply hypothesized.

[Specificity]{.underline}: refers to the precision with which the
exposure and/or the outcome can be defined and characterized in the
causal hypothesis undergoing testing. For example, in the case of
chlorpyrifos and neurodevelopmental outcomes, there are important
distinctions to be made among these various versions of a basic causal
hypothesis: that "chlorpyrifos exposure causes neurodevelopmental
effects," "chlorpyrifos exposure causes adverse neurodevelopmental
effects," "chlorpyrifos exposure causes reductions in birth weight,
birth length, head circumference, and gestational age," and
"chlorpyrifos exposure causes adverse behavioral changes in children no
sooner than 36 months after birth." To each of these, it is possible to
make more specific who is being exposed: the pregnant mother, the fetus,
or the newborn. As noted above, measuring exposure and outcome in a
newborn makes a determination of temporality impossible.

Nonspecific hypotheses are easier to confirm, more difficult to refute,
and therefore, test hypotheses less stringently. Put another way,
studies that do not carefully specify in advance what hypotheses they
are testing (and why) should be considered closer to
"hypothesis-generating" studies than "hypothesis-testing" studies.
Studies that test many hypotheses (without pre-specifying which of these
they intend to carefully examine) can fall prey to multiple-hypothesis
testing concerns.

[Coherence]{.underline}: refers to the extent to which the evidence and
hypotheses for the results fit together into a reasonable and
well-tested explanation. Explanation, as discussed below, is the aim of
any science.

[Experimentation]{.underline}: refers to the extent to which a
randomized clinical trial (e.g. a prevention trial) has been undertaken.
This is an uncommon condition to be satisfied, due to the high cost of
prevention trials. Note that this criterion traditionally does not refer
to the animal experimentation, although some commentators have
mistakenly said so. Nor does this criterion refer to adverse events
uncovered during the course of an experiment, e.g. a randomized clinical
trial.

[Analogy]{.underline}: the extent to which the purported
exposure-disease relationship under consideration is similar (in types
and characteristics of evidence) to other relationships, known to be
causal or not.

Several issues should be understood when these criteria (or
considerations) are invoked. First, the selection of which criteria to
use is more a matter of customary practice than theoretical rigor. In
epidemiology, for example, commonly used criteria appear to be:
strength, consistency, biological plausibility, and dose-response (Weed
and Gorelic, 1996). Other criteria are also used, however, but in a less
regular way. Nevertheless, all of Hill's original criteria are relevant
to making causal claims. It is also the case that there are no
well-accepted approaches to prioritizing these nine so-called criteria.
Put another way, selecting and ranking these criteria is more a matter
of personal preference than methodological rigor. Nevertheless, these
are popular criteria; the method has been used for at least forty years
in hundreds, if not thousands, of applications involving many different
exposures and many different diseases and/or conditions. Furthermore,
many causes of diseases have been identified using this methodology,
since its development nearly fifty years ago, although no formal test of
its reliability or validity has been undertaken (Weed, 2006).

If users of this methodology select some of the criteria but not others,
they should provide an explanation and justification for their choices.
These explanations and justifications are subject to critical appraisal.
Finally, it is important that users of this criteria-based methodology
explain in detail what evidence is needed to satisfy each of the
criteria they select. For example, if "consistency" is important to
their use of the method, they should specify how much, what kinds, and
what characteristics of the evidence satisfy that criterion. Without an
explanation and justification for their individual choices in the use of
the criteria-based method, the user's causal claims are suspect.

The General Scientific Method as a Component of a Weight-of-Evidence
Approach

The general scientific method is a key component of the process of
making causal claims about human health and disease. The aim of science
(and so its methodology) is explanation, typically causal explanation.
The methods of science provide the objectivity so critical to the
validity and reliability of scientific claims.

The basic structure of the scientific method includes: a causal
hypothesis, observable predictions of that hypothesis, the observations
themselves, alternative hypotheses, and tests to distinguish between the
causal hypothesis of interest and its alternatives. The process of
making causal claims using this basic (although over-simplified)
scientific method proceeds as follows: a causal hypothesis is proposed
(often, by observing a phenomenon that requires explanation, such as the
occurrence of a condition or disease in a single individual (a patient)
or a series of patients who have been exposed to some chemical).
Predictions that can be observed in studies are then made (for example,
that in a population of individuals, some of whom have been exposed and
others who are not, the incidence of this condition should be greater in
those exposed than in those unexposed). Alternative hypotheses are also
important. These include but are not limited to: chance, other factors
that may also cause the same outcome, and bias. Finally, tests are
proposed and undertaken to attempt to distinguish the causal hypothesis
of interest from its alternatives. For example, a test of statistical
significance provides the scientific investigator with information that
may allow him to exclude chance as an alternative explanation.
Alternatively, if a test of statistical significance is not performed
(or fails to achieve some pre-specified level of significance, such as p
\< 0.05), then chance cannot be excluded as a possible explanation for
the results. There are many competing explanations for any scientific
observation. The goal of the scientific method is to discern which
explanation is the best explanation.

This methodology is implicitly applied every time a scientific study is
undertaken. It is not always the case that investigators explicitly
describe each of these steps, although it is important to keep in mind
that each step is a critical part of this general scientific method. It
is especially important to recognize the importance of alternative
hypotheses when general causation is the issue.

Furthermore, the general scientific methodology is vitally important in
the process of examining a body of evidence for causation. At the heart
of the matter---when causation is under consideration---is the extent to
which the causal hypothesis of interest has, in fact, been adequately
tested. Such an assessment may include but is not limited to:
assessments of the accuracy of measurement (outcomes and exposures) and
the extent to which exposure measurements accurately reflect the
exposures experienced by the study participants. It is important to
assess the extent to which all the published results can be explained.
Finally, it is important to assess the extent to which alternative
hypotheses have been tested. The most common alternative hypotheses are:
confounding, biases, chance, and alternative causes.

Study Design and Analytical Methods

Study design and analytical (typically, statistical) methods involving
human subjects (i.e. study participants) are important means for
identifying and testing causal hypotheses. Study design methods are
prioritized as follows from strongest to weakest:

Randomized Controlled Trials

Well-designed Controlled Trials without Randomization

Well-designed Cohort Studies

Well-designed Case-Control Studies

Multiple Time Series

Descriptive Studies or Case Series

Expert Opinion

By "strongest," I mean the strongest test of a causal hypothesis, i.e.
the study design that best distinguishes between the hypothesis of
interest and alternative hypotheses. A randomized controlled trial (RCT)
is generally considered to be the strongest test. By "strongest," I do
not necessarily mean the study from which the strongest recommendations
for some action---e.g. a regulation regarding exposure to a
chemical---should be taken, even though that may, in some specific
circumstances, be the case. Any action taken on the basis of the
scientific evidence is a complex matter, involving issues well beyond
the results of scientific studies and their interpretation and beyond
the purpose of this report.

By "weakest" I mean the weakest test of a causal hypothesis. For all the
reasons discussed above, expert opinion in the absence of evidence is an
extraordinarily unreliable way to make causal claims. Expert opinion in
the absence of evidence is no test of a causal hypothesis. Expert
opinion with evidence but in the absence of a clearly stated methodology
for making causal claims is another concern.

There are many analytical statistical methods appropriate to the testing
of causal hypotheses in epidemiology. These I will not comment on
further inasmuch as standard textbooks can provide not only the basics
but also the more nuanced issues in their application.

In order to apply these methodologies to the epidemiology studies on
chlorpyrifos and children's neurodevelopmental outcomes, those studies
must be described and their results summarized. The next section
provides that information.

RESULTS OF PUBLISHED EPIDEMIOLOGICAL STUDIES

Epidemiological Studies of Chlorpyrifos and Children's
Neurodevelopmental and Birth Outcomes: A Descriptive Summary of Designs
and Results

*The Studies*

Three cohort studies from which a total of nine papers have been
published on possible health effects associated with chlorpyrifos
exposure during pregnancy or early in life are the focus of this
section. Study methods are described briefly, followed by a discussion
of the similarities and differences among the cohorts follows. The
results are presented by exposure measurement (e.g. cord blood, personal
air, or urine) and by outcomes measured at birth as well as
neurodevelopmental outcomes during childhood.

*Columbia Cohort (Whyatt et al., 2004, Whyatt et al., 2005, Perera et
al., 2003, Whyatt et al., 2002, Rauh et al., 2006)*

This prospective study evaluates the potential effects of prenatal
exposures to ambient and indoor pollutants on birth outcomes and
neurocognitive developmental outcomes in a cohort of mothers and
newborns from minority communities in New York City (NYC). Initiated in
1997 to evaluate polycyclic aromatic hydrocarbon and environmental
tobacco smoke exposure, the study began to gather information on
prenatal pesticide use in 1998, in response to concerns over the extent
of residential insecticide use in NYC (Perera et al., 2003). The study
was restricted to non-smoking pregnant African American and Dominican
women 18-35 years of age who did not use illicit drugs or have an
illness that might affect the birth weight of their child. Investigators
measured exposure in four ways: 1) self-reported history of residential
pesticide use, 2) chlorpyrifos levels in personal air measured in the
third trimester, (3) chlorpyrifos in maternal blood drawn within two
days postpartum, and (4) chlorpyrifos in umbilical cord blood at birth.

*Mount Sinai Cohort (Berkowitz et al., 2004, Berkowitz et al., 2003)*

This prospective study examines the potential effects of in utero
pesticide exposure on fetal growth and neurodevelopment in a
multi-ethnic cohort of infants delivered at Mount Sinai Hospital in New
York City. Investigators restricted the study to primiparous women with
singleton births. Excluded from the study were women who used illegal
drugs, drank alcohol regularly, and who had an illness that might affect
birth weight of their child. Chlorpyrifos exposure was measured as
self-reported exposure to indoor pesticides as well as the levels of a
chlorpyrifos metabolite, 3,5,6-trichloro-2-pyridinol (TCPy), in maternal
urine obtained in the third trimester of pregnancy.

*Salinas Valley, CA Cohort (Eskenazi et al., 2004, Eskenazi et al.,
2007)*

The Center for the Health Assessment of Mothers and Children of Salinas
(CHAMACOS) prospective study was designed to determine whether
organophosphate pesticide exposure, as assessed by biologic markers, was
associated with poorer fetal growth and shortened length of gestation in
a cohort of primarily Mexican-born pregnant women living in a low-income
agricultural community in the Salinas Valley of California. Study
participants were women 18 years of age or older and eligible for
California's public health insurance program, Medi-Cal. Chlorpyrifos
exposure was measured using levels of the chlorpyrifos metabolite, TCPy,
in maternal urine obtained at two interviews during pregnancy
(approximately 13 and 26 weeks of gestation) and at postpartum
interviews.

Similarities and Differences in Study Populations (Table 1)

Table 1 describes the cohorts exposed to pesticides during pregnancy.
Two of the cohorts were urban, located in New York City, and exposed
primarily to pesticides used for indoor pest control. Exposure in the
Salinas Valley cohort was a result of working in the fields or living
with an agricultural worker, and constituted a different mix and
possibly route of exposure for organophosphates.

The differences between cohorts outweigh the similarities with the only
similarities among all three cohorts exposed to organophosphate
pesticides during pregnancy were the average age of enrollment
(approximately 25 years) and gestational age at birth, which on average
was 39 weeks. Birth weight differed by less than 5% between cohorts and
was lowest among the Mt. Sinai cohort, which had an unspecified number
of smoking mothers (mean = 3289g). Birth weights were highest among
women living in the Salinas Valley (mean = 3449g).

The cohorts differed considerably on socioeconomic status. Although
women enrolled in the two studies conducted in New York City were
similar with respect to educational attainment (between 30-34% with \<
high school education), the Columbia cohort enrolled only non-smoking
African American and Dominican women. Fifty per cent (50%) of the Mount
Sinai cohort was Hispanic and included smokers. Educational attainment
among the predominately Mexican-born Salinas Valley Cohort was the
lowest, with 79% of women having less than a high school education.

The types of pesticide exposure and the biomarkers used to measure
pesticide exposure were not comparable between any two studies (Needham,
2005). Fifty-seven percent (57%) of the Columbia cohort reported using
indoor pest control methods such as sprays by an exterminator, can
sprays, and pest bombs (defined as "highly toxic methods" in (Whyatt et
al., 2005)) compared to 72% of the Mount Sinai cohort who reported any
indoor pesticide use. These studies used two different biomarkers of
exposure: cord blood (parent compound) and urine TCPy, respectively.
Although the Mount Sinai study and the Salinas Valley cohort had the
same prenatal urine biomarker measured (TCPy in urine), the exposure
differed (indoor pest control versus agricultural) and urine samples
were collected at different times during the course of the pregnancy.
For a more complete description of the exposure measurement differences,
see (Needham, 2005).

Birth Outcomes: Results Organized by *in utero* Chlorpyrifos Exposure

*Chlorpyrifos Exposure: Self-reported exposure to pesticides during
pregnancy*

*Summary: Not associated with birth outcomes*

Self-reported exposure to pesticides during pregnancy was not associated
with differences in birth outcomes (gestational age, weight, length, or
head circumference) in either of the New York City studies (Whyatt et
al., 2004, Berkowitz et al., 2004). Maternal self-reported exposure to
pesticides was not analyzed in the Salinas Valley Cohort (Eskenazi et
al., 2004).

*Chlorpyrifos Exposure: Metabolites in urine:
3,5,6-trichloro-2-pyridinol (TCPy)*

*Summary: Not associated with birth outcomes*

Table 2 summarizes findings from the two studies that measured
chlorpyrifos exposure as TCPy levels in prenatal urine (Whyatt et al.,
2004, Eskenazi et al., 2004). Both studies reported similar birth
weight, length, head circumference, and gestational age for the exposed
and unexposed groups. It is noteworthy that TCPy levels and their limits
of detection (LOD) were markedly varied for the two studies and,
consequently, so were the groups defined as exposed and unexposed
because they were based on levels above and below the LOD. For example,
the LOD in the Mount Sinai study was 11 µg/L and the 57% of the
population that had TCPy levels below this constituted the unexposed
group. In comparison, the LOD for the Salinas Valley study was 0.26
µg/L, resulting in a much smaller proportion (24%) of the population at
a much lower exposure level classified as unexposed.

Investigators of the Mount Sinai study (Berkowitz et al., 2004) have
suggested that the relationship between prenatal TCPy levels and head
circumference was more pronounced among women with low paraoxonase
(PON1) activity, an enzyme that detoxifies organophosphate metabolites.
Specifically, women with detectable TCPy levels (\> 11 µg/L) and low
PON1 activity as measured in maternal blood had babies born with
slightly smaller heads (circumference = 33.3 cm) when compared to women
with either medium (34.0 cm), or high PON1 activity (34.1 cm; *P*~trend~
= 0.014). Although they do not report the p-value, the author's state
that the interaction between PON1 activity and TCPy level with head
circumference was not statistically significant, calling into question
whether these findings may simply be due to chance.

Additionally, investigators of the Mount Sinai study reported findings
from fifteen (15) other possible interactions assessed in this analysis
(Berkowitz et al., 2004). They found that the relationship between TCPy
and birth outcomes was not significantly modified by maternal PON1
activity (for length, weight and gestational age -- head circumference
discussed in the previous paragraph), infant PON1 activity (for head
circumference, length, weight, and gestational age), or maternal or
infant genotype (for head circumference, length, weight, and gestational
age).

*Chorpyrifos Exposure: Chlorpyrifos measured in personal air*

*Summary: Not associated with birth outcomes*

During the third trimester of pregnancy women in the Columbia cohort
wore a personal ambient air monitor for 24 hours for two consecutive
days, with an average chlorpyrifos level of 15.3 ng/m^3^ for the study
sample (Whyatt et al., 2004). Using multivariable analyses adjusting for
potential confounders, the investigators report no association between
maternal personal air insecticide levels (including an individual
measure of chlorpyrifos) and birth weight, length, or head
circumference.

*Chlorpyrifos Exposure: Chlorpyrifos measured in cord blood*

*Summary: Associations found with birth weight and birth length. No
association with head circumference at birth.*

The Columbia Cohort is the only study that assesses the relationship
between chlorpyrifos levels in cord blood and birth outcomes, reporting
borderline significant findings for deficits in birth weight and length
(Whyatt et al., 2004). Table 3 shows that each log unit increase in
chlorpyrifos exposure was associated with a 42.6 g decrease in birth
weight (p = 0.03) and a 0.24 cm decrease in birth length (P = 0.04).
Chlorpyrifos exposure was not associated with head circumference at
birth. These findings should be reviewed with the understanding that
p-values were derived from a linear regression model of non-linear data
(log transformed).

In addition to assessing chlorpyrifos levels continuously (see above),
in the case of a significant association, exposure was also
operationalized as a four level categorical variable for the significant
associations. Subjects below the LOD comprised the reference group
(31%). Those with detectable limits were divided into tertiles, but the
corresponding chlorpyrifos values were not provided in the publication.
As shown in Table 3, women with the highest exposure levels relative to
those below the LOD had babies with lower birth weight (on average 150
g; P = 0.03) and shorter birth length (on average 0.75 cm; P = 0.07).
Parameter estimates (β) oscillated from positive to negative for the
intermediate exposure groups and no p-value for trend (e.g.
dose-response) was reported.

Stratified analyses were also conducted to evaluate the potential
effects of pesticide exposures on birth outcomes among newborns born
before versus on or after Jan 1, 2001. On that date the US EPA began a
regulatory action to phase out residential uses of chlorpyrifos. In June
2000, the US EPA entered into an agreement with the registrant to begin
restricting residential uses of chlorpyrifos and to terminate all retail
sales for indoor use by December 2001. Indeed, the results of these
stratified analyses show that decrements in birth weight and length are
more pronounced among babies born prior to regulatory action (Table 3),
although a formal test of interaction was not reported. The authors
recognize that fewer babies in this analysis were born after regulatory
action and that additional analyses may illuminate these results as the
cohort matures.

Neurodevelopmental Outcomes in Childhood: Results Organized by Exposure
Assessment and Age at Testing

The Columbia Cohort (Rauh et al., 2006) and the Salinas Valley Cohort
(Eskenazi et al., 2007) assessed early life and *in utero* exposure to
chlorpyrifos and neurodevelopment in childhood. Outcomes measured at
annual visits included the Bayley Scales of Infant Development (BSID-II)
and the Child Behavioral Check List (CBCL). As show in Table 4, the
average Mental Development Index (MDI, a subscale of BSID-II) and
Psychomotor Development Index (PDI, a subscale of the BSID-II), as well
as the proportion of children with problems were similar for both
cohorts when measured at 24 months of age (based on an interview with
each child conducted by trained staff). At 24 months, according to
Bayley scales, approximately half of these children have clinical
problems with mental development and 13-15% have problems with
psychomotor development.

Table 4 also summarizes clinical problems as measured by Attention,
Attention-Deficit/Hyperactivity Disorder (ADHD), and Pervasive
Development Disorder (PPD) on the CBCL completed by the mother. Because
this instrument was collected at different time periods for the Columbia
Cohort (36 months) and the Salinas Valley Cohort (24 months), the
populations cannot be compared. Nonetheless, fewer than 5% of mothers
report issues in the clinical range with attention or ADHD problems.
Fourteen (14%) per cent of children in the Salinas Cohort were found to
have clinical PPD problems at 36 months (Eskenazi et al., 2007).

*Chlorpyrifos exposure and neurodevelopment at 12 and 24 Months*

*Summary: Not associated with neurodevelopment at 12 and 24 months*

Table 5 summarizes measures of chlorpyrifos exposure and
neurodevelopment at 12 and 24 months in two cohorts. As noted above,
chlorpyrifos was measured in cord blood in the Columbia Cohort (Rauh et
al., 2006) and as TCPy from prenatal urine in the Salinas Valley Cohort
(Eskenazi et al., 2007). No significant associations were observed
between prenatal TCPy levels and any Bayley (Table 5) or CBCL outcomes
(data not shown). Likewise, no significant associations were observed
between chlorpyrifos in cord blood and any Bayley outcomes measured at
12 and 24 months.

*Chlorpyrifos exposure and neurodevelopment at 36 Months*

*Summary: Association observed at 36 months using categorical exposure
variables after no association observed at 36 months using continuous
exposure variables.*

Only the Columbia Cohort assessed chlorpyrifos exposure (as measured in
cord blood at birth) and neurodevelopment at 36 months in their
published report (Rauh et al., 2006). As shown in Table 6, children in
the high compared to the low exposure group consistently reported lower
neurodevelopment scores and experienced more behavioral problems. When
compared to measures at 12 and 24 months, neurodevelopment deficits were
more pronounced at 36 months and frequently statistically significant.
It is noteworthy that general linear models for repeated measures were
generally consistent with the age-specific findings, but the effect of
chlorpyrifos over time was only significantly related to PDI score (P =
0.01).

Some discussion on how cut points for the exposure variable were made is
warranted (details in the methods section of Rauh et al., 2006). Because
the investigators found no indication of either a linear or nonlinear
dose response relationship between chlorpyrifos levels and developmental
outcomes, the exposure variable was then assessed categorically. These
categories were as follows: undetectable levels, and tertiles of
detectable levels: low, medium, and high exposure. The authors only
noted statistical differences if chlorpyrifos levels were used
dichotomously as the highest tertile of detectable levels (\> 6.17 pg/g,
n= 44, high exposure) versus all other exposure (≤ 6.17 pg/g, n=184, low
exposure). Therefore the exposure variable was chosen for its ability to
classify high and low chlorpyrifos levels and was ultimately selected
based on its ability to produce a statistically significant finding of
an association between chlorpyrifos exposure and neurodevelopmental
outcomes.

CRITICAL REVIEW OF PUBLISHED COMMENTARY

Published Assessments of the Epidemiology Studies of Chlorpyrifos and
Children's Birth and Neurodevelopmental Outcomes

Three recent reviews assess some of the epidemiology studies discussed
above and make interpretative statements about the relationship between
chlorpyrifos and children's neurodevelopmental outcomes (Perera et al.,
2005; Rosas & Eskenazi, 2008; and Eskenazi et al., 2008). In this
section, I will briefly describe and critique these review papers,
including: their purpose and methodologies, the chlorpyrifos studies
that were summarized, and most importantly, the reviewers' conclusions
and recommendations.

Perera et al. (2005) describe three previously published analyses from
their Columbia Center for Children's Environmental Health, including the
studies published by Whyatt et al. (2002, 2003) on chlorpyrifos exposure
and neurodevelopmental outcomes in minority communities in New York
City. They also describe their studies on environmental tobacco smoke
(ETS) exposure and polycyclic hydrocarbon (PAH) exposure. No methodology
for assessing or interpreting the results in causal terms is described.
Indeed, no purpose of the review is provided. Basically, the authors
restate the designs, methods, and selected results of the various
studies. From these restatements, they conclude the following:

> "The results from these three analyses...support a causal relationship
> between prenatal environmental toxicant exposure and the adverse
> effects on fetal growth and child development...." (Perera et al.,
> 2005, p. 584).

No discussion nor methodological justification for this conclusion is
provided.

Rosas and Eskenazi (2008) cite a purpose for their review: to "summarize
the current literature regarding neurodevelopmental effects from
exposure of children to pesticides" (p. 191). They examine both
organochlorine and organophosphate pesticides, providing brief
descriptions of their uses, mechanisms for their effects on the nervous
system, and extent of exposure assessments in the United States, based
on biomonitoring studies. In addition, they briefly cite a "substantial
body of animal research (that) supports the notion that low levels of
prenatal and postnatal pesticide exposure can affect neurodevelopment in
children" (p. 193). The studies summarized regarding organophosphates
and neurodevelopment are limited to those examining neonatal reflexes
and results of developmental tests such as the Bayley Scales of Infant
Development among other details. Put another way, these authors do not
summarize any of the published studies examining gestational measures,
such as birth weight, birth length, and head circumference. Indeed, they
note that "...there is a paucity of data on the potential short- and
long-term developmental effects of low-level exposure to children from
in-utero and childhood exposure." (Rosas and Eskenazi, 2008, p. 196).

Nevertheless, they conclude the following:

> "There are some inconsistencies across studies which may arise from
> differences in exposure and in the method of exposure assessment, but
> overall there is surprising consistency in the few studies that have
> been conducted. These studies suggest that there is reason to be
> cautious about exposure of pregnant women to DDT/DDE and
> organophosphates because of the potential effect on the
> neurodevelopment of their children. Policy-makers and pregnant women
> should be educated accordingly." (Rosas and Eskenazi, 2008, p. 196)

Eskenazi et al. (2008) published a review entitled, "Pesticide Toxicity
and the Developing Brain." They describe the purpose of the review as
summarizing "the results of research from the CHAMACOS study, from the
Center for Children's Environmental Health Research at the University of
California, Berkeley, in the context of existing literature on pesticide
exposure and neurodevelopment in infants and children." (p. 228). In the
review, these authors describe the CHAMACOS study, and they also
describe in tabular form literature on DDT and DDE as well as literature
on organophosphate exposure and neurodevelopment, precisely the same
studies reviewed in the publication described above in this report by
Rosas and Eskenazi (2008). Similarly, they do not mention, much less
examine, the studies on chlorpyrifos exposure and gestational measures.

They conclude the following:

> "Previous literature and recent work by the Centers for Children's
> Environmental Health Research provide evidence of pesticide toxicity
> to the developing human brain." (Eskenazi et al., 2008, p. 235)

An Assessment and Critique of the Published Reviews: (Perera et al.,
2005; Rosas and Eskenazi, 2008; Eskenazi et al., 2008*)*

The quality of any review of the literature can be assessed using the
following criteria (Weed, 1997; Breslow et al., 1998):

> Stated Purpose
>
> Stated Methods for Searching the Literature
>
> Stated Inclusion and Exclusion Criteria for Literature Reviewed
>
> Stated Interpretative Methods (appropriate to the Purpose)

In addition, the arguments the reviewers use to make claims and
recommendations must be examined for reasonableness, logic, and
methodological support.

Two of the three reviews assessed state their purpose: to summarize the
literature. The same two reviews, however, go beyond this stated
purpose---to summarize the studies---in their conclusions. Rosas and
Eskenazi (2008), for example, make clinical practice and policy
recommendations regarding exposure to pesticides. Eskenazi et al.,
(2008) also state that their purpose is to summarize the literature, but
they make claims regarding causation. Perera et al., (2005), on the
other hand, do not state a purpose for their review, but make a claim
nevertheless regarding the causal nature of the relationships.

None of these reviews provide methodological support for their claims
and recommendations. As described in some detail in this report, methods
for causal claims are widely-known and frequently employed. Methods (or
guidelines) for making clinical practice and policy recommendations are
similarly well-known in the broader clinical, public health, and policy
communities. Nevertheless, Rosas and Eskenazi (2008) make clinical
recommendations but provide no explicit methodology for how they arrived
at their conclusions. Neither Eskenazi et al., (2008) nor Perera et al.,
(2005) provide a methodology for their causal claims. They simply assert
their conclusions after reviewing the evidence.

Finally, none of the reviews describes why they chose not to include
published studies---the studies on gestational measures and chlorpyrifos
exposure---that had direct relevance to the causal claims or policy (and
practice) recommendations they published.

It should be noted that a possible response to this critique is that the
causal claims made in these reviews were rather broad, not specifically
focused on chlorpyrifos exposure per se. Perhaps, it could be argued,
some of the reviewers meant to emphasize what they believe are the
DDT/DDE (organochlorine) pesticide effects on neurodevelopment rather
than what they believe the organophosphate pesticide effects to be.
Eskenazi et al., (2008), for example, use the term "pesticides" in their
causal claim. It is not possible to sort out this lack of clarity.
Perera et al., (2005), are even broader in their claim, using the term
"environmental toxicants," which apparently includes ETS, PAH, and/or
chlorpyrifos. It is also not clear if they believe all (or some) of the
reviewed exposures cause adverse effects.

In summary, these reviews are of questionable quality, based on
generally recognized criteria for the quality of systematic narrative
reviews in epidemiology (Breslow et al., 1998) as well as in the
scientific literature more generally. Extreme caution in accepting their
causal conclusions is warranted.

WEIGHT-OF-EVIDENCE ASSESSMENT

Having described the epidemiology studies and the commentaries and
reviews interpreting their results, it is time to apply a
weight-of-evidence approach. As described above, such an approach
employs a number of methodologies, including study design and
statistical methods, the systematic narrative review, criteria-based
methods, as well as the overarching (and critically important) general
scientific method. Some of these methods have already been applied,
namely, the systematic narrative review (in collecting and summarizing
the existing evidence) as well as the epidemiology study designs (in the
three cohorts) and, to a lesser extent, the statistical methods applied
to the collected data in those studies.

The purpose of this section is to guide the reader through the
application of the remaining methods, including the general scientific
method and the criteria-based methods of causal inference. Note that the
results from these studies were not such that a quantitative "weighting"
of evidence across studies was possible. A meta-analysis, in other
words, is not a part of this approach.

First, I will examine how well the basic causal hypotheses regarding
chlorpyrifos exposure and neurodevelopmental outcomes were tested in the
epidemiology studies. As noted above, there are a number of specific
hypotheses to be considered. Exposure measures (both content and timing)
are important to evaluate, including the methodologically questionable
technique of manipulating the exposure measure to find a statistically
significant result (as discussed earlier regarding Rauh et al., 2006).
Furthermore, outcomes can include various gestational measures or
behavioral measures. And within each of those categories, there are more
specific hypotheses to be tested, such as the hypothesis linking
chlorpyrifos exposure and head circumference, or the hypothesis linking
chlorpyrifos exposure with birth weight. To put it another way, given
the broad range of specific causal hypotheses, what were the results of
all the tests of those hypotheses? How well were these hypotheses
tested? This is an application of the general scientific method.

Second, I will examine the reported (statistically significant, i.e.
positive) associations between chlorpyrifos exposure and
neurodevelopmental outcomes in terms of the criteria-based method of
causal inference described in some detail above.

With regard to the criteria-based methods, it is clear from the
published reviews and commentary that to date no formal (i.e. explicit)
application of these methods of inference has been applied to the
results of the epidemiology studies. On some occasions, however,
investigators have selectively applied one or more of these criteria to
their findings, typically without any accompanying definitions, calling
into question the validity of their approach. A good example can be
found in Berkowitz et al. (2004) who reported a single statistically
significant finding, what they described as an adverse effect on head
circumference, and then argued that this "effect" was biologically
plausible. No definition of "plausibility" was provided and no causal
claim was made. It is, therefore, not clear why plausibility was
mentioned, other than perhaps to provide a warrant for more research?
Such lack of clarity and purpose is common in the practice of causal
inference in epidemiology (Weed, 2001).

These two approaches---one employing the general scientific method and
the other using criteria-based methods---are not entirely independent.
When applying the criteria, for example, issues regarding the
testability of hypotheses may (indeed, should) emerge. Nevertheless, it
will be important to examine the findings found in the epidemiology
studies from both perspectives.

A Weight-of-Evidence Assessment of Chlorpyrifos and Gestational Measures

Assume that at the heart of the scientific investigations of
chlorpyrifos and gestational measures these hypotheses are of interest:

Chlorpyrifos exposure (in utero) causes reduced birth weight.

Chorpyrifos exposure (in utero) causes reduced birth length.

Chlorpyrifos exposure (in utero) causes reduced head circumference.

Chlorpyrifos exposure (in utero) causes reduced gestational age.

Note that these are all declarative statements. A more precise
description of a causal hypothesis would be: if chlorpyrifos is a
neurotoxin for infants in utero, then those infants who have been
sufficiently exposed will experience reductions in birth weight, birth
length, head circumference, and gestational age.

Testing these hypotheses in epidemiology studies involves a number of
conditions and concerns too numerous to describe in detail. Yet it is
reasonable to presume that three central tenets apply to testing these
hypotheses:

1.  That exposure to chlorpyrifos (in utero) can be accurately measured.

2.  That the birth outcomes (i.e. gestational measures) can be
    accurately measured.

3.  That the exposure to chlorpyrifos (as measured and incorporated into
    the analyses) must precede the birth outcomes in order for that
    exposure to be causally relevant.

There are, of course, a number of additional presumptions needed for a
careful and comprehensive---a "critical" test---of a causal hypothesis
in an epidemiological study, not the least of which is the extent to
which alternative causes of the outcome associated with the exposure
have been controlled (i.e. the known and unknown confounders). For the
moment, however, I will focus primarily on the basic tenets of the
epidemiological version of the general scientific method: measure
exposure accurately before the expected outcome and then compare the
outcome in those exposed versus those unexposed.

*The Mount Sinai Cohort*

Maternal exposure to chlorpyrifos was measured (using urinary TCPy)
during pregnancy and thus prior to the measured birth outcomes. Infant
exposure to chlorpyrifos---cord blood TCPy---was measured simultaneously
with the birth outcomes. The extent to which the infants (as fetuses)
were exposed to chlorpyrifos during pregnancy was not measured.

Tests of the basic hypotheses (for every outcome) showed no differences
between exposed and unexposed.

*The Salinas Valley Cohort*

Maternal exposure to chlorpyrifos (TCPy) was measured during pregnancy
and thus prior to the measured birth outcomes. Infant exposures were
taken (cord blood) at birth.

Tests of the basic hypotheses, between (urinary) exposure and the
unexposed, revealed increases in head circumference and in birth length
of the infants; these were not statistically significant. These tests,
therefore, suggested contradictory results given the investigators'
stated expectations. Tests of the hypotheses using blood levels revealed
one statistically significant comparison of decreased gestational
duration (3 days shorter) among many other comparisons.

Tests of the basic hypotheses (for every outcome) showed no differences
between exposed and unexposed.

*The Columbia Cohort*

Maternal exposure to chlorpyrifos was measured by questionnaire
(regarding events during pregnancy) as well as by personal air sampling
techniques (during pregnancy). Infant exposures were measured postpartum
in cord blood.

For all hypotheses examining the exposure prior to the outcome, there
were no differences in birth weight, birth length, and head
circumference.

For exposure coincident with outcome, there were statistically
significant differences in birth weight and birth length but not head
circumference.

Conclusion

The reasonable scientific inference to be made from this evidence---and
from the tests of these many hypotheses---is that exposure to
chlorpyrifos does not cause adverse neurodevelopmental
outcomes---defined as gestational (birth) measures---in these
populations.

The results are overwhelmingly negative, where by "negative" I mean both
null findings as well as occasional findings in the opposite direction
(that are not statistically significant). The "weight-of-the-evidence"
taking into account all the findings---rather than an occasional
"positive" finding---warrants no claim of causation.

The Causal Criteria Applied to the Chlorpyrifos Studies: Birth Outcomes

Given the above conclusion, it may seem unnecessary to apply a
criteria-based approach to these same results. A brief description may,
nevertheless, be instructive.

There is a fundamental question here regarding the existence of a
statistical association between chlorpyrifos and gestational outcomes,
much less a causal association. The majority of (nearly all) comparisons
were not significant. If "association" is to be used with these results,
then "association" can only mean that at least one statistically
significant difference was described regardless of how many null or
negative comparisons were reported. That sort of meaning for an
"association" was clearly not what was intended by those who designed
the criteria-based methods of causal inference.

The evidence, nevertheless, is remarkably inconsistent, if by
"consistent" one means that a specific association (e.g. chlorpyrifos
exposure and any of the gestational measures) is seen across many
studies. There is no strength, a lack of temporality, no dose-response
relationship, and only a relatively weak plausibility argument,
specifically, that chlorpyrifos causes adverse events in animal systems
and therefore could also in humans.

Conclusion

Given the available evidence, chlorpyrifos does not cause adverse
gestational outcomes in these populations.

A Weight-of-Evidence Assessment of Chlorpyrifos Exposure and
Neurodevelopmental (Behavioral) Outcomes

The scientific hypothesis of interest here, generally stated, is that
exposure to chlorpyrifos causes adverse neurodevelopmental and
behavioral outcomes in children. The scientific hypotheses (or
explanations) advanced by the investigators include the possibilities
that these adverse effects can (should) occur at any age subsequent to
birth; the investigators examined the children in these studies at 1, 2,
and 3 years. As noted above, however, there was no evidence of adverse
effects at 1 year or at 2 years of age in two different cohorts of
children. At 3 years (36 months), however, several measures of
neurodevelopment and behavior in the Columbia cohort were significantly
lower in children exposed to "high" levels of chlorpyrifos (wherein the
exposure categories were defined to achieve statistical significance, as
described earlier) than those exposed to "low" levels. It seems
reasonable to question any causal conclusion given that no explanation
is provided by the investigators for why differences in
neurodevelopmental and behavioral outcomes should be observed at 36
months but not at 12 and 24 months.

Why is this omission important? In the section of their published paper
in which their results are discussed, the Columbia University-based
investigators write that "...the magnitude of the pesticide effect on
MDI scores reported in this study was modest." (Rauh et al., 2006, p.
1855). Put another way, these investigators label their results as an
"effect" implying that causation has been established. Elsewhere, they
note that "...the effects of chlorpyrifos were not attributable to the
effects of potential known confounders..." again implying that these
results reveal causal relationships (Rauh et al., 2006, p. 1855).

Causation, however, has clearly not been established. First, there is
the problem of known confounders that were not measured. In addition,
there is the problem of unknown confounders. Known and unknown
predictors of adverse neurodevelopmental effects in children must be
controlled for before a causal claim can be made regarding chlorpyrifos
exposure and neurodevelopmental outcomes. Second, these "positive"
findings are restricted to a single study in which the measurement of
chlorpyrifos was made at birth. It is not clear, therefore, if the
results reported here are false-positive results, a well-recognized
problem in epidemiological studies, especially when early studies are
published (Boffetta et al., 2008). Additional studies of the
relationship between chlorpyrifos and neurobehavioral outcomes, as well
as longer follow up in the Columbia cohort, will likely prove helpful in
this regard. Finally, inasmuch as these "positive" results are limited
to the Rauh et al., (2006) report, no application of the criteria-based
methods of causal inference should be undertaken.

Conclusion

Given the available epidemiological evidence, chlorpyrifos exposure does
not cause adverse neurodevelopmental outcomes in children who have been
exposed in utero.

SUMMARY OF CONCLUSIONS

Given the available scientific evidence, no claim of causation can be
(nor should be) made regarding chlorpyrifos and neurodevelopmental and
birth outcomes in infants and children. This assessment arises from a
systematic review of the available epidemiological studies using a
"weight-of-evidence" approach, as discussed in detail in this report.
Additional research on these matters may (or may not) change this
assessment in the future.

According to a memo dated August 21, 2008, the EPA "believes that all
three studies (i.e. the cohorts from Columbia University, Mt. Sinai
School of Medicine, and the University of California, Berkeley) provide
valuable information on the effects of children of high exposure to
pesticides, particularly OPs." As discussed in some detail in this
document, these studies do not provide sufficient evidence that effects
attributable to chlorpyrifos exist. In addition, the EPA "concludes that
chlorpyrifos likely played a role in these outcomes." There is
insufficient evidence provided in these studies to make such a claim. In
particular, there is not sufficient evidence in any of these studies
that statistical associations exist between chlorpyrifos and several
birth and neurodevelopmental outcomes. In the absence of consistent
statistical associations, a conclusion that chlorpyrifos "likely" played
a role in these outcomes is not warranted.

ACKNOWLEDGEMENTS

The epidemiological studies on chlorpyrifos exposure and health were
assessed and summarized by Michelle Althuis, Ph.D., an epidemiologist in
private professional practice. I am indebted to her careful review and
summarization of these studies as well as to her critical review of an
earlier draft of this report.

REFERENCES

*Commentaries, Reviews, Editorials, and Letters*

Davis DL, Ahmed AK. Exposures from indoor spraying of chlorpyrifos pose
greater health risks than currently estimated. Environ Health Perspect
1998;106:299-301.

Davis DL, Ahmed AK. Independent review of industry-generated data.
Environ Health Perspect 1998;106:A583-584.

Eskenazi B, Bradman A, Castorina R. Exposures of children to
organophosphate pesticides and their potential adverse health effects.
Environ Health Perspect 1999;107(suppl 3):409-419.

Eskenazi B, Rosas LG, Marks AR, Bradman A, Harley K, Holland N, Johnson
C, Fenster L, Barr DB. Pesticide toxicity and the developing brain.
Basic Clin Pharmacol Tox 2008;102:228-236.

Gibson JE, Peterson RKD, Shurdut BA. Human exposure and risk from indoor
use of chlorpyrifos. Environ Health Perspect 1998;106:303-306.

Landrigan PJ, Claudio L, Markowitz SB, Berkowitz GS, Brenner BL, Romero
H, Wetmur JG, Matte TD, Gore AC, Godbold JH, Wolff MS. Pesticides and
inner-city children: exposures, risks, and prevention. EHP
1999;107(suppl 3):431-437.

Lemus R, Abdelghani A. Chlorpyrifos: an unwelcome pesticide in our
homes. Rev Environ Health 2000;15:421-433.

Needham LL. Assessing exposure to organophosphorus pesticides by
biomonitoring in epidemiologic studies of birth outcomes. Environ Health
Perspect 2005;113:494-498.

Perera FP, Illman SM, Kinney PL, Whatt RM, Kelvin EA, Shepard P, Evans
D, Fullilove M, Ford J, Miller RL, Meyer IH, Rauh VA. The challenge of
preventing environmentally related disease in young children:
community-based research in New York City. Environ Health Perspect
2002;110:197-204.

Perera FP, Rauh V, Whyatt RM, Tang D, Tsai WY, Bernert JT, Tu YH,
Andrews H, Barr DB, Camann DE, Diaz D, Dietrich J, Reyes A, Kinney PL. A
summary of recent findings on birth outcomes and developmental effects
of prenatal ETS, PAH, and pesticide exposures. NeuroToxicol
2005;26:573-587.

Playing with pesticides. Environ Health Perspect 1998;106:A10.

Rosas LG, Eskenazi B. Pesticides and child neurodevelopment. Curr Opin
Pediatrics 2008;20:191-197.

Zhao Q, Badagbui B, Dourson M. Lower birth weight as a critical effect
of chlorpyrifos: a comparison of human and animal data. Reg Toxicol
Pharmacol 2005;42:55-63.

*Epidemiological Studies*

Berkowitz GS, Wetmur JG, Birman-Deych E, Obel J, Lapinski RH, Godbold
JH, Holzman IR, Wolff MS. *In utero* pesticide exposure, maternal
paraoxonase activity, and head circumference. Environ Health Perspect
2004;112:388-391.

Berkowitz GS, Obel J, Deych E, Lapinski R, Godbold J, Liu Z, et al.
Exposure to indoor pesticides during pregnancy in a multiethnic, urban
cohort. Environ Health Perspect 2003;111:79-84.

Eskenazi B, Harley K, Bradman A, Weltzien E, Jewell NP, Barr DB, Furlong
CE, Holland NT. Association of *in utero* organophosphate pesticide
exposure and fetal growth and length of gestation in an agricultural
population. Environ Health Perspect 2004;112:1116-1124.

Eskenazi B, Marks AR, Bradman A, Harley K, Barr DB, Johnson C, Morga N,
Jewell NP. Organophosphate pesticide exposure and neurodevelopment in
young Mexican-American children. Environ Health Perspect
2007;115:792-798.

Gurunathan S, Robson M, Freeman N, Buckley B, Roy A, Meyer R, Bukowski
J, Lioy PJ. Accumulation of chlorpyrifos on residential surfaces and
toys accessible to children. Environ Health Perspect 1998;106:9-16.

Perera FP, Rauh V, Whyatt RM, Tsai WY, Tang D, Diaz D, Hoepner L, Barr
D, Tu YH, Camann D, Kinney P. Effect of prenatal exposure to airborne
polycyclic aromatic hydrocarbons on neurodevelopment in the first 3
years of life among inner-city children. Environ Health Perspect
2006;114:1287-1292.

Rauh VA, Garfinkel R, Perera FP, Andrews HF, Hoepner L, Barr DB,
Whitehead R, Tang D, Whyatt RW. Impact of prenatal chlorpyrifos exposure
on neurodevelopment in the first 3 years of life among inner-city
children. Pediatrics 2006;118:e1845-e1859.

Whyatt RM, Rauh V, Barr DB, Camann DE, Andrews HF, Garfinkel R, Hoepner
LA, Diaz D, Dietrich J, Reyes A, Tang D, Kinney PL, Perera FP. Prenatal
insecticide exposures and birth weight and length among an urban
minority cohort. Environ Health Perspect 2004;112:1125-1132.

Whyatt RM, Camann D, Perera FP, Rauh VA, Tang D, Kinney PL, Garfinkel R,
Andrews H, Hoepner L, Barr DB. Biomarkers in assessing residential
insecticide exposures during pregnancy and effects on fetal growth.
Toxicol Appl Pharmacol 2005;206:246-254.

Young JG, Eskenazi B, Gladstone EA, Bradman A, Pedersen L, Johnson C, et
al. Association between in utero organophosphate pesticide exposure and
abnormal reflexes in neonates. Neurotoxicology 2005;26:199-209.

*Other Literature Cited*

Acquavella JF. The politics of identity in epidemiology. Ann Epidemiol
1997;7:431-433.

Barrow CS, Conrad JW Jr. Assessing the reliability and credibility of
industry science and scientists. Environ Health Perspect
2006;114:153-155.

Bearer CF. How are children different from adults? Environ Health
Perspect 1995;103(Suppl 6):7-12.

Boffetta P, McLaughlin JK, LaVecchia C, Tarone RE, Lipworth L, Blot WJ.
False-positive results in cancer epidemiology: a plea for
epistemological modesty. JNCI 2008;100:988-995.

Breslow RA, Ross SA, and Weed DL. Quality of reviews in epidemiology. Am
J Public Health 1998;88:475-477.

Faustman EM, Silbernagel SM, Fenske RA, Burbacher TM, Ponce RA.
Mechanisms underlying children's susceptibility to environmental
toxicants. Environ Health Perspect 2000;108(Suppl 1):13-21.

Federal Register. April 18, 2008 (Volume 73, Number 76) Notices. Page
21125-21128.

Federal Register. July 9, 2008 (Volume 73, Number 132) Notices. Page
39292-39294.

Hill AB. The Environment and Disease: Association or Causation? Proc Roy
Soc Med 1965;58:295-300.

Krimsky S. The weight of scientific evidence in policy and law. Am J Pub
Health 2005;95(Suppl 1)S129-136.

Landrigan PJ, Kimmel CA, Correa A, Eskenazi B. Children's health and the
environment: public health issues and challenges for risk assessment.
Environ Health Perspect 2004;112:257-265.

Michaels D. Manufactured uncertainty: protecting public health in the
age of contested science and product defense. Ann NY Acad Sci
2006;1076:149-162.

Needham LL. Assessing exposure to organophosphorus pesticides by
biomonitoring in epidemiologic studies of birth outcomes. Environ Health
Perspect 2005;113:494-498.

Slotkin TA. Developmental cholinotoxicants: nicotine and chlorpyrifos.
Environ Health Perspect 1999;107(Suppl 1):71-80.

Tilson HA. The concern for developmental neurotoxicology: is it
justified and what is being done about it? Environ Health Perspect
1995;103(Suppl 6):147-151.

U.S. Environmental Protection Agency. Memo to Sharlene Matten, Ph.D.
from the Office of Prevention, Pesticides, and Toxic Substances. August
21, 2008.

Weed DL and Gorelic LS. The practice of causal inference in cancer
epidemiology. Cancer Epidemiol Biomark Prev 1996;5:303-311.

Weed DL. Methodologic guidelines for review papers. J Nat Cancer Inst
1997;89:6-7.

Weed DL and Hursting SD. Biologic plausibility in causal inference:
current method and practice. Am J Epidemiol 1998;147:415-425.

Weed DL. Epidemiologic evidence and causal inference.
Hematology/Oncology Clin N America 2000;14:797-807.

Weed DL and McKeown RE. Science, ethics and professional public health
practice. J Epidemiol Commun Health 2003;57:4-5.

Weed DL. Weight of evidence: a review of concepts and methods. Risk
Analysis 2005;25:1545-1557.

Weed DL. Evidence synthesis and general causation: key methods and an
assessment of reliability. Drake Law Review 2006;54:639-650.

Weed DL. The nature and necessity of scientific judgment. J Law Policy
2007;15:135-164.

![](media/image1.wmf)

![](media/image2.wmf)

![](media/image3.wmf)

![](media/image4.wmf)

![](media/image5.wmf)

![](media/image6.wmf)