Appendix F: 

Critique of Syngenta Corporation’s Probabilistic Implementation of 

CASMATZ for Ecological Risk Assessment 

(30 Pages) 
Page 1 of 29 

Critique of Syngenta Corporation's 

Probabilistic Implementation of CASMATZ for 

Ecological Risk Assessment of Atrazine 

Russell Erickson 

U.S. Environmental Protection Agency 

Mid-Continent Ecology Division 

Duluth, Minnesota 

July 27, 2009
Page 2 of 29 

1 Introduction 

This document critiques the merits of the atrazine risk assessment methodology proposed by 

Syngenta Crop Protection, Inc. during public comments at the May 12-14, 2009 FIFRA 

Scientific Advisory Panel (SAP) Open Meeting regarding the "The Ecological Significance of 

Atrazine Effects on Primary Producers in Surface Water Streams in the Corn and Sorghum 

Growing Region of the United States". This proposal was in response to EPA presentations at 

this meeting and to the EPA briefing document provided for this meeting (hereafter referred to as 

the "SAP Briefing Document"). 

The material from Syngenta examined for this critique included: 

(1) Copies of the Powerpoint slides in the presentation given on May 12, 2009 at the SAP 

meeting by David Volz (Syngenta) and Steven Bartell (E2 Consulting Engineers under contract 

to Syngenta) entitled "Probabilistic CASMATZ2: Refined Aquatic Ecological Risk Assessments 

for Atrazine." 

(2) A June 1, 2009 report from Steven Bartell and Shyam Nair of E2 Consulting Engineers 

entitled "Distributions of Environmental Input Data, Toxicity Data, and Bioenergetic Parameter 

Values for the Probabilistic CASMATZ2 Presented at the 2009 FIFRA SAP Meeting, " which 

summarizes the procedures used to develop parameter sets for probabilistic CASMATZ. 

(3) Electronic files containing the 200 parameter sets used in Syngenta's probabilistic CASMATZ

analyses, provided by David Volz of Syngenta. Due to an error by EPA regarding toxicity 

values and an error by Syngenta regarding light intensities in these sets, these sets were modified 

in coordination with Syngenta and E2. These new, modified sets will be used for the analyses in 

this critique. Consequently, model results will no longer match those presented at the SAP 

meeting. 

In addition, exchanges of information were made between EPA and E2 Consulting Engineers to 

establish consistency in model inputs and verify identical outputs when using the same inputs. 

As part of this exchange, it was also determined that E2 was not using the final experimental 

ecosystem exposure time-series EPA included with the SAP Briefing Document, and a 

consistent, updated set of time-series for model input was provided by EPA to E2, resulting in 

additional differences in results from those presented by Syngenta at the May SAP. 

This document is organized into three sections: 

(1) To better understand the review comments on Syngenta's proposed methodology, the general 

assessment problem and framework for atrazine assessments is described. 

(2) Various shortcomings of Syngenta's methodology and analyses are identified. 

(3) Important implications of Syngenta's methodology to atrazine risk assessments are discussed. 
Page 3 of 29 

2 Framework for Atrazine Assessments 

Appendix I provides the problem definition and general strategy for EPA's atrazine assessments, 

as presented by EPA at the SAP meeting and in the SAP Briefing Document. This assessment 

framework must be understood to provide necessary context for the critique of Syngenta's 

proposed methodology. In addition, some aspects of this framework have been a source of 

confusion and merit some additional explanation here: 

2.1 Extrapolation of Atrazine Effects Information Across Exposure Time-Series 

A major element of this framework is the need to extrapolate effects across different exposure 

time-series, which involves the use of an effects model that can provide information on such 

extrapolations. Figure 1 depicts a simple example of this extrapolation problem and how it is 

addressed. 

In this example, an experimental ecosystem establishes the level of concern (LOC) for a 30-day 

exposure to be 20 μg/L. For the purposes of this example, the effect of concern is specified to be 

a 20% reduction in average total plant biomass for some appropriate measurement period. 

However, the exact nature and magnitude of this effect is not important to the general problem 

definition and the conceptual model for extrapolation being explained here, so other types and/or 

magnitudes of effect could have been used here instead. 

This experimental ecosystem-based LOC is assumed to be relevant to the natural ecosystem; i.e., 

the natural ecosystem would experience a comparable effect if exposed to 20 μg/L for 30 days. 

However, this long experimental exposure is quite different than the exposure of interest in the 

natural ecosystem, which is specified for this example to be a 3-day, 100 μg/L exposure. The 

basic question is whether this shorter, more intense exposure exceeds an LOC; i.e., if the 

Figure 1. A simple example of problem definition for extrapolating an LOC based on an experimental 

mesocosm to a natural ecosystem with a different exposure time-series. 

Experimental Ecosystem

20 μg/L atrazine determined

to be LOC (20% effect) for

a 30-day exposure

Natural Ecosystem

Exposure of interest is 100 μg/L

for 3-days; this exceeds modelextrapolated LOC of 80 μg/L.

Mathematical Effects Model

20 μg/L, 30-day exposure 

produces a 30% effect 

(=model effects index LOC)

Mathematical Effects Model

To produce the 30% effect 

for a 3-day exposure

requires 80 μg/L atrazine

Uncertain Extrapolation

to 3-day Exposure

Determine Model

Effects LOC Based on

Experimental System

Use Model to Extrapolate

Between Exposures

Apply Model 

Extrapolation to

Natural System
Page 4 of 29 

experimental ecosystem was tested with this shorter exposure, would effects exceed the 20% 

benchmark? Unfortunately, the experimental ecosystem study does not have information 

pertinent to this shorter, more-intense exposure. 

The extrapolation model is intended to bridge this gap between the experimental ecosystem and 

natural ecosystem exposures by providing a means to assess the effects of both exposure timeseries for the same (model) system. The model need not exactly describe either the experimental 

or natural ecosystem, or even produce the same impairment at the same exposure. For example, 

Figure 1 shows the model to produce a 30% effect at 20 μg/L, in contrast to the 20% effect in the 

experimental ecosystem. What the model does need to provide is an extrapolation between these 

two exposure time-series that is informative of what would be expected in the experimental 

ecosystem if it was tested at the shorter duration.

In the example here, the model is first used to determine that there is a 30% modeled effect at the 

LOC (20 μg/L) established for the 30-day exposure in the experimental ecosystem. This 30% 

effect is designated the LOCMEI – the value for a model effects index (MEI) at the LOC 

concentration established in the experimental ecosystem study. The model is then used to 

estimate that reaching this LOCMEI for a 3-day exposure requires 4-times the concentration (80 

ug/L). 

Under the assumption that this concentration shift is relevant to the experimental ecosystem, the 

LOC concentration for the experimental ecosystem would also be 80 μg/L for a 3-day exposure. 

And because it is already assumed that the experimental ecosystem is relevant to the natural 

ecosystem, the LOC for a 3-day exposure would also be 80 μg/L for this system as well. Thus, 

the 3-day, 100 μg/L exposure in the natural ecosystem does exceed the LOC. 

The above simple example establishes the basic nature of the assumptions being made, the 

extrapolations being conducted, and the (limited) role of the extrapolation model. However, the 

assessment framework must accommodate more complexities regarding effects and exposure 

information than in Figure 1. This more complex situation is described in Appendix I and only 

summarized here. There is not just one, definitive experimental ecosystem that defines an LOC, 

but rather a large number of experimental ecosystem studies with exposures that are simply 

classified as exceeding or not exceeding an effects level of concern (Figure I-1). There is not 

just one exposure time-series in the experimental ecosystems and one in the natural ecosystems, 

but rather many different exposure time-series shapes (e.g., Figure I-2). Nonetheless, the 

concepts in the above simple example still apply. As described in Appendix I, the model is run 

for each experimental ecosystem exposure to determine an LOCMEI that best discriminates 

exposures with and without an effect of concern (Figure I-3). Once the LOCMEI is determined, 

the model can be run for any natural ecosystem exposure to determine if the modeled effects 

exceed the LOCMEI, and also to determine the factor (risk factor) by which the exposure must be 

reduced to produce modeled effects exactly equal the LOCMEI (Figure I-4). 

To end this section, it should be reiterated and emphasized that the effects model is not being 

asked to predict specific responses in the various ecosystems of interest (experimental or natural 

ecosystems); rather, these various systems are being used to define the variety of exposure timeseries over which the effects model is to assess relative effects. The main requirement for the 
Page 5 of 29 

model is that the relative model-calculated effects among exposure time-series be acceptable 

estimates of the relative effects among time-series in the real systems, so that the effects of 

concern in the experimental ecosystems can be extrapolated to exposure time-series in natural 

ecosystems. An analogous situation would be if the natural ecosystem exposures were all at 

lower temperature than the experimental ecosystems, and that independent information was 

available that this temperature difference would reduce sensitivity by two-fold. It is not 

necessary for this information on temperature effects to pertain directly to the absolute effects in 

any of these systems, but rather just that the two-fold relative effect of temperature be relevant to 

the extrapolation of effects from the experimental ecosystems to the natural ecosystems. 

2.2 Generic Nature of Time-Series Extrapolation 

One option for the extrapolation effects model that was proposed by Syngenta and evaluated by 

EPA was a version of the mathematical aquatic community simulation model CASM 

(Comprehensive Aquatic Simulation Model) developed by Dr. Steven Bartell of E2 Consulting 

Engineers for atrazine effects in a generic U.S. Midwest corn belt stream (CASMATZ). This 

model has a large number of parameters and input variables regarding community 

structure/bioenergetics and physicochemical conditions that must be provided specific values. 

The values of these parameters/variables would be quite different among the different 

experimental ecosystems and the different natural ecosystem systems of interest. 

However, the very nature of the time-series extrapolation explained above requires that the 

model not have one parameterization for one time-series and another parameterization for 

another time-series, because then the relative effect is not just of the time-series. Rather, 

CASMATZ parameterization must be generic – meaning that the same parameter and input 

variable values are used for all exposures, regardless of the nature of the system for which the 

exposure was observed. For a probabilistic assessment, this means that there would be multiple 

parameterization sets selected from ranges for the parameters/variables encompassing the 

systems of interest, but that all exposures (whether they be from an experimental ecosystem or a 

natural ecosystem site) would be analyzed with the same parameterization sets. Again, the 

model is just addressing the relative effects of different exposure time-series, not trying to 

simulate specific systems. The real ecosystems (experimental or natural) provide the exposure 

time-series across which extrapolations are desired, not specific community structures and 

physicochemical conditions that the model must simulate effects for. 

Thus, with regard to CASMATZ, the assessment thus would not be site-specific except with 

respect to the characteristics of the exposure time-series. However, values for the various 

bioenergetics parameters and physicochemical conditions required by CASMATZ must still be 

specified, and the selection of these values should be based (generically) on the experimental and 

natural ecosystems being assessed so that the extrapolations are as relevant as possible. 

Furthermore, variability of risk estimators due to the ranges from which these values might be 

selected from should be assessed. If this variability is significant, there must be consideration of 

where risk decisions are positioned within this variability, but this generic application will not 

attempt to adjust risk estimates based on system-specific attributes. 
Page 6 of 29 

To end this section, it should be noted that more site-specific uses of CASMATZ are conceivable, 

in which CASM would not just address relative effects of different exposure time-series, but 

would be expected to predict absolute effects of atrazine not just as a function of exposure timeseries but other system attributes as well. This would require creating individual versions 

CASMATZ tailored to each experimental and natural ecosystem, including relevant uncertainty 

analysis for each version and validation that these individual versions adequately simulate 

community dynamics with and without atrazine exposures. The modeled effects for the 

experimental ecosystems still would be used to establish an LOCMEI based on the observed 

effects in the experimental ecosystem, and the modeled effects in natural ecosystems would then 

be compared to this LOCMEI. However, such an effort is not feasible. 

2.3 Assessing Uncertainty and Variability 

As EPA discussed at the SAP meeting and in the SAP Briefing Document, this framework also 

allows for uncertainty analyses of the final risk determinations (e.g., whether an exposure 

exceeds the LOC; the value of the risk factor), by conducting Monte Carlo analyses in which 

various data inputs to the assessment are varied according to information on their individual 

uncertainties. In particular, parameters and input variables for the extrapolation model could be 

varied to determine how uncertain/variable these extrapolations might be. In the SAP Briefing 

Document, EPA described in general how such uncertainty analyses should be done, and 

provided some example calculations of uncertainties, although the uncertainty analyses were not 

fully developed. 

CASMATZ requires specifying a large number of parameters and input variables, and EPA 

conducted sensitivity analyses to determine how risk assessments would vary with the values 

specified for certain of these parameters/variables. Based on these sensitivity analyses, EPA 

concluded there would be too much uncertainty in the risk assessments, and thus rejected 

CASMATZ as the extrapolation model and adopted a different approach (which will not be 

discussed here). The limited uncertainty analysis provided by EPA in the SAP briefing 

document also demonstrated how the variable results from CASMATZ would result in 

conservative restrictions on atrazine of questionable validity. 

Syngenta disagreed with EPA's conclusions regarding the utility of CASM. They asserted that 

what EPA treated as uncertainty should rather be viewed as meaningful variability and would not 

preclude CASM's use in the assessment, and that the sensitivity analysis conducted by EPA was 

too incomplete (with respect to the identity, distribution, and correlations of parameters/variables 

being varied) to support EPA's conclusions. Syngenta therefore conducted probabilistic analyses 

similar to EPA's but involving more extensive parameter variations, and proposed how these 

should be incorporated into the risk assessments. 

Three categories of model parameters and inputs were varied by Syngenta. 

(a) Toxicity relationship EC50s and slopes were varied in accord with procedures developed by 

EPA. The relationships developed by Syngenta were based on erroneous mean EC50s for some 

taxonomic groups in the Briefing Document, but still supported a useful demonstration of their 

proposed methodology. This critique will present analyses using the correct mean EC50s. 
Page 7 of 29 

(b) Field data on temperature, light, nutrients, suspended solids, and water depth/velocity were 

analyzed by Syngenta to develop distributions for these variables, including certain correlations. 

This information was used to generate hypothetical input time-series for the model, although 

there were errors in the maximum light values used for these series. 

(c) Selected bioenergetics parameters were assigned distributions based on the variability of 

values reported in the literature. 

Based on this information, Syngenta developed 200 sets of model parameters and input 

variables, differing with respect to toxicity and bioenergetics parameters, and environmental 

conditions (these input sets have subsequently been updated to correct for errors noted above). 

For each input set, Syngenta's analyses were conducted using EPA's assessment framework as 

described above and in Appendix I. Syngenta summarized the results of their 200 analyses in a 

distribution of the LOCMEIs for the 200 cases and distributions of the MEIs for each field 

exposure time-series of interest. Risk was determined simply by whether the median of the these 

distributions differed; i.e., if the median of the MEIs for a particular exposure time-series 

exceeded the median of the LOCMEIs. This differed from EPA analyses, in which risk was 

characterized by the risk factor described above. 
Page 8 of 29 

3 Shortcomings of Syngenta Methodology and Analyses 

It first should be first noted that Syngenta's proposed probabilistic methodology is very similar to 

the uncertainty analysis described by EPA, in that both methods (a) use Monte Carlo analysis, 

modifying model parameters and input variables to generate probability distributions of effect 

outcomes and (b) follow the basic extrapolation methodology from experimental ecosystems to 

field exposures described in Appendix I for each Monte Carlo case. The only differences in the 

methodologies presented at the SAP meeting regarded (a) how risks were characterized based on 

these probability distributions and (b) various specifics of model parameter/input variable 

modifications. This section of the critique will first address these two areas of differences, and 

then address the issue of ecological realism in probabilistic CASMATZ results. 

3.1 Risk Characterization Statistic 

Syngenta determined whether the LOC was exceeded based simply on comparison, for the 200 

Monte Carlo cases, of the median LOCMEI to the median MEI for a field exposure of interest. In 

contrast, EPA based their assessment on risk factors and addressed percentiles other than the 

median. Some deficiencies regarding Syngenta's risk characterization are: 

3.1.1 Syngenta's methodology ignores the pairwise nature of the data. 

By modifying model parameters and input variables, the 200 probabilistic cases define 200 

different model configurations that vary markedly in their sensitivity to atrazine. The MEI for a 

specific probabilistic case should be compared only to the LOCMEI for that case; i.e., it is 

inappropriate to compare the MEI for an atrazine-tolerant configuration to the LOCMEI

determined for an atrazine-sensitive configuration. Consequently, comparing the medians (or 

any other percentile) of the separate MEI and LOCMEI distributions actually does not have a 

defined meaning with regard to how often and how markedly the MEI exceeds the LOCMEI. For 

example, it is possible for the LOC to be exceeded a large fraction of the time, and the medians 

of these distributions to still indicate that the LOC is not exceeded. In contrast, the risk factors 

used by EPA as the basis of the assessment inherently address the pairwise nature of the data 

because they are based on adjusting exposure so that the MEI for a probabilistic case equals the 

LOCMEI for that case. 

Instead of the risk characterization procedure used by Syngenta, a single risk statistic needs to be 

derived based on the MEI and LOCMEI for each case, thus recognizing the pairwise nature of this 

analysis. The distribution of this single risk statistic should be examined, not the separate 

distributions of the MEI and LOCMEI. Three possibilities for this statistic are: 

(a) A simple yes/no regarding whether the MEI exceeds the LOCMEI. This can then 

be summarized as a probability of the MEI exceeding the LOCMEI based on all 200 

cases. This ignores information on the severity of exceedences, but at least provides a 

legitimate expression of risk for a specific exposure, because the frequency with which 

the LOC is exceeded across the 200 probabilistic cases is actually determined, unlike 

Syngenta's methodology. 
Page 9 of 29 

(b) The ratio of the MEI to the LOCMEI ( or other measure of this deviation). Such 

an effect ratio retains information on the relative severity of the exceedence and risk can 

be summarized as a distribution of these ratios. The percentiles of this distribution 

would provide useful summary information (e.g., what ratio is exceeded in 50% of the 

cases? In 10% of the cases?). This distribution could also be used to summarize how 

often a particular value for the ratio is exceeded (e.g., how often is the ratio above 1 to 

indicate some effect, or above 2 to indicate a major effect?). 

(c) The risk factor as described in Section 2 and in Appendix I. This changes the 

perspective to the concentration scale rather than the effects scale and is analogous to 

risk quotients commonly used in risk assessment, which contrast an exposure 

concentration to an effects concentration. Like the effects ratio in option (b) above, this 

provides information on the severity of the exceedence and can be summarized based 

on its distributional properties. EPA elected to focus its analyses on this statistic 

because risks defined using the concentration scale can be more directly related to 

management needs. 

3.1.2 Syngenta methodology bases risk just on the median.

Basing risk characterization on median sensitivities constitutes a risk management decision for a 

lower level of protection than typical of OPP assessments. When the median result shows the 

exposure to be just above the LOC, this is tantamount to allowing the exposure to exceed the 

LOC 50% of the time (often by a substantial amount, given the variability of CASM results). 

This is true whether the variability of CASM-based results is considered to be uncertainty or 

meaningful variability among sites. If uncertainty, using the median rather than a confidence 

limit allows a high rate of error. If meaningful variability, using the median allows a substantial 

percentage of sites to be at risk. 

3.1.3 The statistical test for the difference between the distributions is inappropriate. 

These model-based distributions involve an arbitrary number of simulation cases. A statistical 

test is not appropriate to address an inadequate number of model simulations. Enough 

simulations should be run to characterize the median well enough without a statistical test. 

Furthermore, even if a statistical test were appropriate, the null hypothesis used by Syngenta is 

not, because it imposes a antiprotective bias in that the MEI must exceed the LOCMEI by a certain 

amount before it is concluded to exceed the LOCMEI at all. 

3.1.4 Syngenta's methodology is not a probabilistic risk assessment. 

Although based on a probabilistic analysis, Syngenta's methodology does not support meaningful 

probability statements regarding whether a given exposure exceeds a level of concern (i.e., risk = 

probability of effect). Syngenta's simple comparison of median MEIs and LOCMEIs has 

uncertain meaning because it ignores the pairwise nature of the data, and comparisons based just 

on the median provide little information beyond that from a single, deterministic risk assessment. 

In contrast, probabilistic analyses using the risk statistics options discussed above provide a more 

meaningful characterization of risk, and allow useful statements about percentiles other than the 
Page 10 of 29 

median. Figure 2 illustrates how these statistics would support an actual probabilistic risk 

assessment, in contrast to Syngenta's methodology. This figure is based on new EPA analyses 

using CASMATZ2 code that was confirmed to be functionally the same as that used by Syngenta 

for the SAP meeting, with updates to the 200 sets of parameters/input variables and experimental 

ecosystem exposure time-series to address errors in the original sets discussed earlier. 

The first (top) panel of Figure 2 compares the LOCMEI distribution to the MEI distribution for 

four of the example chemographs in Figure I-2 of Appendix I. These chemographs were selected 

to provide MEI distributions that were shifted, relative to the LOCMEI distribution, far the left, 

slightly to the left, slightly to the right, and far to the right. This is the type of comparison 

proposed by Syngenta, with the risk characterization simply being whether the median of the 

MEI distribution is shifted to the left or right of the median for the LOCMEI distribution. Such a 

comparion provides incomplete and misleading information about whether the LOC is exceeded. 

As discussed above, the magnitude of the shift between the distributions has uncertainty meaning 

because it doesn't address the pairwise nature of the data. These figures often provide a 

misleading impression that the MEI is either above or below the LOCMEI for all 200 cases, when 

in fact this is far from true, as the rest of Figure 2 illustrates. Even if direct comparison of these 

distributions was meaningful, Syngenta also ignores the degree to which the distributions 

overlap. In general, such a comparison is not informative about the frequency with which MEIs 

exceed their corresponding LOCMEIs, or the actual magnitude of these exceedences. 

The second panel of Figure 2 shows the cumulative probability distribution of the effects ratio 

(option b above), which provides well-defined information on both (a) the magnitude of the 

MEIs relative to their corresponding LOCMEIs and (b) how frequently the LOC (or some multiple 

of the LOC) is exceeded. The median effect ratios are 0.16, 0.74, 1.10, and 2.48 for 

chemographs #10, 2, 6, and 16, respectively, which provides a better-defined statistic regarding 

the degree with which the MEI exceeds the LOC for the median case. More importantly, this 

distribution of effects ratios provides information other than whether a simple median shift 

occurs. For example, Chemograph 2 would be characterized as not posing a risk by Syngenta, 

but in 36% of the cases, the MEI exceeds the LOCMEI (effect ratio > 1), and in 25% of the cases 

this effect ratio exceeds 2. Such information warrants consideration in characterizing and 

managing risk, and should not be ignored as it is in Syngenta's approach. 

The third panel of Figure 2 shows the distribution of risk factors (option c above), which also 

provides useful information on the magnitude and probability of effects, similar to that for the 

effects ratios. In fact, both the effects ratio and the risk factors have identical cumulative 

probabilities when their value is 1.0 (i.e., equal probabilities of the MEI exceeding the LOCMEI). 

The median risk factors are 0.28, 0.75, 1.08, and 3.09, differing somewhat from the median 

effect ratios because of the nonlinear relationship between effects and exposure concentration. 

The bottom panel of Figure 2 summarizes the distributions of the effect ratios and risk factors as 

box plots. This format will be used for subsequent analyses in this critique because it allows 

more effective comparison of many assessments, and more clearly shows the consequences of 

different risk management decisions with respect to the frequency and magnitude of LOC 

exceedences. In Figure 2, this format makes clear how, although the majority of the MEIs do not 
Page 11 of 29 

Figure 2. Top panel: comparison of LOCMEI and MEI distributions for four example chemographs, based on 

simulations with 200 different cases for CASM parameters/input variables. Second panel: cumulative 

probability distributions for effect ratios. Third panel: cumulative probability distributions for risk factors 

Bottom panel: probability distributions for effects ratios and risk factors summarized as box plots (box specifies 

25th, 50th, and 75th percentiles; whiskers specify 10th and 90th percentiles; dots specify 5th and 95th percentiles).

Model Effects Index

0.0 0.2 0.4 0.6

Cumulative Probability

0.0

0.2

0.4

0.6

0.8

1.0

LOCMEI 

Chemograph 16 

Chemograph 2 

Chemograph 6

Chemograph 10 

Effects Ratio

0.1 0.2 0.5 2 5 1 10

Cumulative Probability

0.0

0.2

0.4

0.6

0.8

1.0

Risk Factor

0.1 0.2 0.5 2 5 1 10

Cumulative Probability

0.0

0.2

0.4

0.6

0.8

1.0

Chemograph #

16 6 2 10 16 6 2 10

Risk Factor

0.05

0.2

0.5

2

5

20

0.1

1

10 Effect

Ratios

Risk

Factors
Page 12 of 29 

exceed the LOCMEI for Chemograph 2, this exposure could be considered to entail considerable 

risk because the variability/uncertainty of probabilistic CASM results in substantial exceedences 

of the LOCMEI for a substantial fraction of the cases. 

3.2 Nature of Model Parameter and Input Variable Modifications 

Several differences existed in the variation of model parameters/input variables in the 

probabilistic analyses presented by Syngenta and EPA in the SAP meeting materials. 

(a) For bioenergetics, Syngenta varied several selected parameters, based on the range of 

values reported in reviewed literature. EPA did not vary any bioenergetic parameters in 

their sensitivity or uncertainty analyses. Analyses for this critique use the 200 bioenergetics 

data sets developed by Syngenta for the analyses they presented at the SAP meeting. 

(b) For toxicity, EPA established distributions for the EC50s and slopes of effectconcentration relationships for four major plant taxonomic groups (vascular plants, diatoms, 

green algae, blue-green algae) and randomly selected EC50s and slopes from these 

distributions for each case in its sensitivity and uncertainty analyses. Syngenta used these 

same distributions, except toxicity parameters used by Syngenta were in error due to EPA 

misreporting the mean EC50s for the taxonomic groups at one place in the SAP Briefing 

Document. Analyses for this critique use new toxicity parameter sets reflecting correct taxa 

means. 

(c) For environmental conditions, EPA started with the default time-series in the CASMATZ

code for each variable (developed by Syngenta based on their analysis of field data) and 

changed the entire time-series based on just two randomly selected factors – one that shifted 

all the default values of the time-series up and down and one that changed variability around 

the 30-day running averages of these default values. This allowed the development of timeseries that differed in their mean and in their variability, while still retaining seasonal trends 

and auto-correlation of daily values similar to the default time-series. Syngenta used the 

original data for the default time-series to develop monthly distributions for each variable, 

including simple correlations among the variables. They then randomly generated 

individual daily values from these distributions to develop different environmental 

parameter time-series, without considering temporal autocorrelation except at the monthly 

time scale. As noted above, the time-series for light were based on erroneous upper limits, 

and Syngenta developed 200 new environmental time-series sets to correct this error. The 

analyses in this critique used these new sets, but also included some variations to illustrate 

some shortcomings of Syngenta's methodology. 

(d) EPA modified the exposure start date (default = Julian day 105) for model simulations 

using experimental ecosystem exposures and demonstrated that this had some effect on the 

risk determination. Syngenta did not consider this parameter in its probabilistic analyses. 
Page 13 of 29 

Some shortcomings of Syngenta's approach and analyses are: 

3.2.1 The set of model parameters/input variable being modified is inadequately defined. 

Although EPA manipulated only a limited number of model parameters and input variables in 

their sensitivity and uncertainty analyses, these analyses were only intended to illustrate that 

there was sufficient uncertainty to contraindicate the use of CASM. A more extensive and 

refined modification of model parameters/input variables would not change this conclusion and 

was therefore not necessary. 

However, if CASM is to be used for a probabilistic risk assessment, as proposed by Syngenta, 

there needs to be a complete effort to identify what parameters/variables need to be manipulated 

and to appropriately define their distributions and interactions for the Monte Carlo analysis. In 

this respect, Syngenta's efforts are inadequate. There was no effort to demonstrate (through 

sensitivity analyses) which parameters/variables should be included, so that it is unknown 

whether other CASM model parameters/variables should be included in the probabilistic 

manipulations and whether the parameters/variables that were manipulated could actually be 

omitted. One major omission of Syngenta, as noted above, was not considering the effect of 

exposure start date for the experimental ecosystem exposure simulations used to establish the 

LOCMEI. Other notable omissions were the degree-day parameters used to determine when each 

species becomes biologically active, the initial assumed biomasses which affect eventual 

biomass potential and competitive relationships, and light attenuation parameters, which is a 

major factor in probabilistic CASM's daily primary production. There is also the issue of the 

community structure in CASMATZ – how sensitive are results to this and how well does it 

encompass the diversity of sites to be assessed? 

3.2.2 Parameter/variable distributions are inadequately justified.

Syngenta's environmental variable distributions and correlations are based on only two Ohio 

streams. Although these streams are not dissimilar to many in the Midwest corn growing region, 

they would not fully capture the variation and interactions of environmental parameters that 

should be considered. The distributional shape (uniform or log uniform) also needs better 

evaluation and justification. A defensible probabilistic implementation of CASM applicable to 

the region would need considerably more development in this regard. 

Regarding the bioenergetics parameters, the sources of the distributions are inadequately 

characterized, without citations and with reference to professional judgment in parameter 

distribution development. Any actual implementation would require specification of all 

information sources considered, and better established and defined decision criteria. 

Although Syngenta's probabilistic analyses used distributions for toxicity parameters developed 

by EPA, if these are to be used for actual risk assessments (rather than just to establish CASM as 

too variable for use as EPA analyses concluded), more development is needed regarding these 

distributions. There needs to be more consideration of what toxicity data are most applicable to 

each CASM species, with possible development of different distributions for smaller or different 

taxonomic groups. In addition, the probabilistic analyses need to consider not just random 
Page 14 of 29 

selection from a distribution given a nominal mean and standard deviation, but also the 

uncertainties of these means and standard deviations. In particular, the implications of limited 

data for the diatoms and blue-green algae should be addressed. EPA addressed this uncertainty 

issue in their analyses of the effects model they proposed as an alternative to CASM. 

3.2.3 Correlations among parameters and variables are inadequately developed. 

Although Syngenta criticized EPA for not considering correlations among parameters, they only 

considered simple linear correlations and only within their limited environmental data set, and 

did not demonstrate how effectively variability/uncertainty in model results was addressed by 

this treatment. Considering the limited available data, such shortcomings in the analysis are 

understandable and largely unavoidable, but still result in weaknesses and uncertainties in the 

overall procedure that affect model credibility and warrant better consideration. Again, 

considerable more development work and documentation in this regard would be needed if this 

methodology is to be justified and implemented. One important consideration would be whether 

plant community structure and bioenergetics should vary with nutrient, temperature, light, etc. 

This would be expected to be the case and EPA's sensitivity analysis in the SAP briefing 

document raised this issue. As difficult as it would be to accomplish, this needs to be addressed 

in some manner and degree to establish the credibility of the probabilistic analyses. Another 

important consideration would be the shape of relationships between parameters other than the 

currently-assumed linear relationship. 

3.2.4. Environmental variable time-series have questionable characteristics. 

Syngenta's procedure for developing environmental time-series resulted in questionable 

characteristics for the time-series. These problems arose due to combining data from multiple 

sites/years into distributions for each month, and then using the distributions to randomly select 

daily values without regard to how daily values would vary due to temporal autocorrelation and 

the differences between years and sites. This results in three problems in the time-series, 

illustrated in Figure 3 using the first five Syngenta Monte Carlo cases for temperature. 

(a) Because each case uses, for each month, a random sample of 28-31 daily values from 

the same distribution, monthly means are virtually identical among the different cases 

(upper panel of Figure 3). Syngenta's probabilistic analysis thus does not account for 

variation in average temperatures across sites and years, which is important if this 

analysis is to credibly describe variability across the Midwest corn belt. Rather, the cases 

differ just in the random variations within a month for a single set of monthly means. For 

contrast, the upper panel of Figure 3 also shows the range over which 30-day running 

average temperatures varied in EPA's analyses, covering a range of possibilities for 

variation among sites and years. 

(b) Because temporal effects other than monthly means are not considered (especially 

autocorrelation among days) and because the contribution of variability between sites and 

years are treated as part of the daily variation, random day-to-day variations are 

unrealistically extreme – up to 15°C between adjacent days (middle panel of Figure 3), in 

contrast to the more realistic default temperature time-series. 
Page 15 of 29 

Figure 3. Temperature inputs for simulations. In top panel, solid lines with numbered circles denote monthly 

means for first five cases of Syngenta's probabilistic analysis, and dashed lines denote range of 30-day running 

averages in EPA's uncertainty analysis. In middle and bottom panels, the variability for the default time-series 

(bold line) is contrasted with that for the first five cases of Syngenta's probabilistic analysis (numbered lines on 

middle panel, dotted lines on bottom panel). 

Month

1 2 3 4 5 6 7 8 9 10 11 12

Average Monthly Temperature

0

5

10

15

20

25

30

35

1 1

1

1

1

1

1 1

1

1

1

2 1

2

2

2

2

2

2

2

2

2

2

3 3 2

3

3

3

3

3

3

3

3

3

3

4 4

4

4

4

4

4

4

4

4

4

5 4

5

5

5

5

5

5 5

5

5

5

5

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Daily Temperature

0

5

10

15

20

25

30

Julian Day

90 95 100 105 110 115 120

Daily Temperature

0

5

10

15

20

25

30

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1 1

2

2 2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2 2

2 2

3

3

3

3 3

3

3

3

3

3

3 3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

4

4

4

4

4

4

4

4

4

4

4

4 4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5 5

5

5

5

5

5

5

5

5

5

5

5
Page 16 of 29 

(c) Using simple monthly stratification results in a disjoint, "stair-step" time-series 

(bottom panel of Figure 3). This again should be contrasted to the more realistic 

variations of the default temperature time-series (bold line). 

What value and legitimacy does Syngenta's probabilistic analysis have if it is based on 

unrealistic temperature fluctuations and does not cover the range of environmental conditions 

that likely occur in the assessment region? And similar problems exist for other environmental 

variables – Figure 4 shows even more extreme and unrealistic variations for suspended solids 

compared to the default time-series. The default series provides a more reasonable temporal 

pattern, suggesting events that last for a few to several days rather than wild day-to-day 

variations. However, even this default series warrants some further evaluation because by using 

average values over multiple years it has too many events and the events would be less 

pronounced than original due to averaging. In any event, more realistic time-series are needed, 

because primary productivity responds greatly to this variable in a highly nonlinear way. 

Figure 4. Total suspended solids inputs for simulations. The variability for the default time-series (bold line) is 

contrasted with that for the first five cases of Syngenta's probabilistic analysis (numbered lines on top panel, 

dotted lines on bottom panel). 

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Total Suspended Solids

10

100

1000

Julian Day

90 95 100 105 110 115 120

Total Suspended Solids

10

100

1000

1

1

1

1 1

1

1

1

1

1 1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

1

2

2

2

2 2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

2

3

3

3

3

3

3

3

3

3

3

3 3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

3

4

4 4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4

4 4

4

4

4 4

4

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5

5 5

5
Page 17 of 29 

3.3 Ecological Realism of CASMATZ Simulations

A potential benefit of CASM is that it might introduce more ecological realism into atrazine risk 

assessments. However, although CASM does include credible descriptions of certain ecological 

processes, it does not necessarily follow that these provide meaningful and significant inferences 

regarding atrazine risk assessments. As George Box said, "all models are wrong, some are 

useful." CASM is necessarily a gross simplification of an actual ecosystem, and whether its 

inferences about how atrazine toxicity is translated into community effects as a function of 

model parameters and input variables are "useful" is far from established. This section of the 

critique will discuss various aspects of this issue, some of which apply to CASMATZ generally 

and some of which are more specific to Syngenta's probabilistic approach. 

This question of ecological realism is especially important given the large variation of risk 

estimates from Syngenta's probabilistic CASM methodology. Figure 5 summarizes the 

variability of risk factors for all 16 example chemographs from Figure I-2 across the 200 (new) 

probabilistic cases. The difference between the 5th and 95th percentiles varies from 6 to 50 

depending on the chemograph, and would probably be larger if these cases addressed the 

deficiencies discussed in Section 3.2. If CASMATZ is "ecologically realistic" the implication of 

this is that atrazine risk varies with how system properties differ across the probabilistic cases. 

This has great implications to risk assessments, as discussed in Section 4, dependent on how 

meaningful this variation actually is. For such a large variability to be accepted in the risk 

assessments, there must be confidence that this CASM-predicted variability is meaningful. 

3.3.1 Simplifications in CASMATZ have uncertain effect on assessment realism. 

CASMATZ involves various simplifications that conceivably can affect the ecological realism of 

the simulations and thus atrazine effects assessments. At a minimum, there must be some 

demonstration that these simplifications do not materially affect assessments. Some examples of 

these simpliccations are: 

Figure 5. Box plots of the distributions of risk factors for sixteen example chemographs, based on using 

probabilistic CASMATZ with Syngenta's 200 new cases. The box denotes the 25th, 50th, and 75th percentiles; the 

bars denote the 10th and 90th percentiles; and the dots denote the 5th and 95th percentiles. 

Chemograph #

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Risk Factor

0.2

0.5

2

5

0.1

1

10
Page 18 of 29 

(a) Although 26 plant species are a lot for such a model, this is far more simple than natural 

ecosystems and often results in time periods in which only 1-3 species are significant. Plant 

community dynamics thus might be unduly affected by the toxicity parameters assigned to 

certain critical species, which is not necessarily true to the same extent in natural systems. This 

is especially a concern given uncertainties regarding what the toxicity parameters for individual 

species should be. 

(b) CASM is a single-compartment (zero-dimensional) model trying to describe a complex threedimensional system. Although it does address certain depth effects (such as light attenuation), 

each species type is assigned a specific depth and the light at that depth is applied to all the 

bioenergetic calculations for the species. One specific ramification of this for streams is that, 

under turbid conditions, benthic plants are often uniformly subject to enough light attenuation to 

make primary production negligible, without consideration of plants growing at different depths, 

including shallow depths were sufficient light exists for primary production. 

(c) CASM does not consider diurnal variations in community dynamics and other conditions. 

This might or might not affect estimations of atrazine risks, but still is an ecologically unrealistic

element of CASM that warrants better evaluation with regard to whether its results are unduly 

distorted by this simplification. 

3.3.2 Mass balance is not satisfied in CASMATZ. 

CASM originated as a lentic model in which horizontal exchanges of material are much slower 

and more limited than in streams. For the stream implementation of CASMATZ, advective mass 

balance is not handled properly. Partly for this reason, phytoplankton were dropped from 

CASMATZ. However, for Sygenta's methodology, CASM still models nutrient dynamics despite 

these mass balance problems, and there are questionable depletions of nutrients within the stream 

reach being simulated that affect community dynamics. In addition, field nutrient concentrations 

are treated as inputs to the simulated compartment (stream reach), when they might arguably be 

better treated as the actual ambient concentrations, so that their dynamics should not be 

simulated. Due to concerns raised by EPA, CASMATZ does have the option of not simulating 

nutrient dynamics, which EPA has used in some analyses, demonstrating that these two different 

options can produce substantially different reference simulations and risk calculations (see SAP 

Briefing Document). Such different outcomes underscores uncertainties of CASMATZ that cast 

doubt on its ecological realism. Which option, if either, provides realistic nutrient concentrations 

to drive the dynamics of the plant community and its susceptibility to atrazine? 

3.3.3 Effects of specific parameters and variables are inadequately defined and validated. 

As already noted, Syngenta failed to provide sensitivity analyses that define which model 

parameters and input variables are actually needed in the probabilistic analysis. Another reason 

for such sensitivity analyses is to understand what parameters/variables are most important for 

the large variability of risk estimates in Figure 5. If some credibility of the nature and magnitude 

of the effects of specific parameters/variables can be established, this would support the overall 

credibility of the probabilistic CASMATZ results. EPA's sensitivity analysis do this to some 

degree. However, rather than supporting CASMATZ credibility, such analyses often reveal 
Page 19 of 29 

shortcomings. For example, EPA's sensitivity analysis showed that temperature can have a large 

effect on CASM-computed risks. Is there a reasonable expectation that risks in real ecosystems 

would be similarly temperature-dependent, or is this an artifact of the model that stems from 

incomplete consideration of important temperature-related factors? As noted by EPA, this 

temperature sensitivity raises questions about correlations between temperature and community 

structure/bioenergetics, exposure timing, toxicity, etc., that are not addressed in Syngenta's 

probabilistic analyses. Overall, there is no empirical or theoretical basis for saying this 

variability is meaningful, and it would be necessary to more fully explore Syngenta's results in 

this regard. 

3.3.4 Probabilistic parameter modifications produce unrealistic reference simulations. 

Considerable attention has been paid to whether CASM produces realistic reference (i.e., no 

atrazine exposure) simulations of biomass time-series for a midwestern stream, using the default 

bioenergetics parameters and environmental variable time-series. To some degree, the default 

CASM parameters were tailored to produce credible reference simulations. If this probabilistic 

analysis is to be meaningful, the reference simulations for all 200 probabilistic cases also need to 

be realistic. 

Figure 4 shows the variability of the reference biomass time-series for different plant taxa for 

Syngenta's first ten probabilitic cases, compared to the reference simulation for the default case. 

In some of the probabilistic cases, taxa biomasses don't even increase beyond their initial 

assumed values. Among these cases, peak taxa biomasses vary by up to three orders of 

magnitude. If they do not provide more realistic reference simulations than these, what is the 

value of this probabilistic variation of bioenergetics parameters and environmental variables? 

Another aspect of the reference simulations (including for the default situation) that must be 

questioned before accepting CASMATZ as a foundation for risk assessments is the extended 

period of constant plant biomass early in the year (controlled by a degree-day parameter for plant 

dormancy), and the significant declines for most taxa following this flat portion before 

significant growth occurs. This behavior is, per se, unrealistic. How much this behavior, and the 

associated selection of values for initial biomass and the degree-day parameter, might affect the 

risk assessments must be addressed. 

3.3.5 Risk factors are overly sensitive to methodology for environmental time-series. 

It was already noted that Syngenta's time-series for environmental variables were unrealistic in 

that they involved excessive day-to-day variations around the same monthly mean, rather than 

providing appropriately autocorrelated day-to-day changes and means that covered the range in 

the Midwest corn belt. That the nature of these time-series can affect risk assessments is 

illustrated on Figure 7. These boxplots summarize analyses of the 200 probabilistic cases in 

which toxicity and bioenergetics values are unchanged, but environmental variables are treated 

in four different ways – (a) the 200 (new) cases using Syngenta's approach, (b) fixed at the daily 

averages of Syngenta's time-series, (c) fixed at the default values, and (d) 200 cases using EPA's 

approach as described in the SAP briefing document. 
Page 20 of 29 

Figure 7 demonstrates considerable variability in risk estimates depending on how the 

environmental time-series are treated. For a particular chemograph, the four alternative 

treatments have medians that vary by up to 6-fold (typically be at least 2-fold), and can have 

greatly different variabilities. Most notably, for all the chemographs, Syngenta's 200 

environmental time-series result in risks markedly less than those for the average of these timeseries. This is not necessarily inappropriate, but underscores that the nature of the probabilistic 

time-series will affect risk assessments, so that it is important to have something more realistic 

than what Syngenta's methodology provides. Interestingly, the default time-series often produce 

the lowest risk estimates. The point here is not that one of these alternatives is better than the 

others, but rather that the sensitivity of risk estimates to assumptions about environmental 

variabilities requires that there be more effort to have better estimates for these variables. 

Figure 6. Variability of reference simulations in Sygenta probabilistic analysis. Bold lines depict default 

environmental conditions and bioenergetics parameters; other lines depict first 10 of 200 probabilistic cases. 

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Green Algae Biomass 0.005

0.02

0.05

0.2

0.5

2

5

20

50

0.01

0.1

1

10

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Bluegreen Algae Biomass

0.005

0.02

0.05

0.2

0.5

2

5

20

50

0.01

0.1

1

10

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Diatom Biomass 0.005

0.02

0.05

0.2

0.5

2

5

20

50

0.01

0.1

1

10

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Other Algae Biomass 0.005

0.02

0.05

0.2

0.5

2

5

20

50

0.01

0.1

1

10

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Total Plant Biomass 0.005

0.02

0.05

0.2

0.5

2

5

20

50

0.01

0.1

1

10

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Vascular Plants 0.005

0.02

0.05

0.2

0.5

2

5

20

50

0.01

0.1

1

10
Page 21 of 29 

Figure 7. Distributions of risk assessments for all 16 example chemographs using four alternatives for 

environmental variable time-series (still using Syngenta's 200 probabilistic sets for bioenergetics and toxicity 

parameters). Numbers without a letter suffix denote simulations with chemograph using Syngenta's 200 

probabilistic environmental time-series; suffix A denotes using average of these time-series; suffix D denotes 

using default time-series; suffix E denotes using 200 probablistic environmental time-series using EPA

approach. 

Chemograph #

1 1A 1D 1E 2 2A 2D 2E 3 3A 3D 3E 4 4A 4D 4E

Risk Factor

0.2

0.5

2

5

20

0.1

1

10

Chemograph #

5 5A 5D 5E 6 6A 6D 6E 7 7A 7D 7E 8 8A 8D 8E

Risk Factor

0.2

0.5

2

5

20

0.1

1

10

Chemograph #

9 9A 9D 9E 10 10A 10D 10E 11 11A 11D 11E 12 12A 12D 12E

Risk Factor

0.2

0.5

2

5

20

0.1

1

10

Chemograph #

13 13A 13D 13E 14 14A 14D 14E 15 15A 15D 15E 16 16A 16D 16E

Risk Factor

0.2

0.5

2

5

20

0.1

1

10
Page 22 of 29 

4 Implications of Sygenta Methodology to Risk Assessments 

Section 3 discussed a variety of issues regarding establishing CASMATZ as a credible tool for 

atrazine risk assessments that would require a tremendous effort, and might not be possible for 

some of the specifics. However, assuming that this could be done, the risk estimates of 

CASMATZ would still range widely depending on how the model is parameterized, which would 

have implications to risk assessments that would still contraindicate its use. 

4.1 Syngenta method risk characterizations are excessively uncertain. 

At the SAP meeting, both Syngenta and some SAP panel members objected to EPA 

characterizing the range of CASM results as uncertainty, when it could to some degree 

represents meaningful model inferences regarding how atrazine risk varies with respect to system 

characteristics. 

However, probabilistic CASM's variability can be argued to be predominantly uncertainty. A 

large fraction of this variability is due to uncertainty regarding what toxicity parameter to assign 

CASM species. As noted earlier, this involves even more uncertainty that isn't yet accounted for 

in probabilistic CASM. Even if the toxicity parameter distributions could be better resolved for 

particular taxa, there is also uncertainty about whether individual CASM species should be 

treated as just representing particular species, or are surrogates for a broader set of species with 

certain bioenergetic properties. For example, EPA's sensitivity analysis demonstrated that 

changing the nutrient status shifts the late-season dominant species from blue-green algae to 

what is based on a euglenoid. Thus, with respect to toxicity parameters, variability must be 

treated as uncertainty. The same is also true for the bioenergetics parameters, which reflect the 

variability of values reported in the literature without any clear links to system characteristics. 

With respect to the environmental variables, there is a potential for CASM results to reflect some 

meaningful differences between systems (e.g., lower versus higher nutrients, temperatures, etc.). 

However, as discussed earlier, this potential is not realized in the current probabilistic CASM 

because the environmental time-series are very close regarding their monthly means and just 

differ by unrealistic daily variations about those means. Even if this shortcoming of probabilistic 

CASM was corrected, there is inadequate substantiation that CASM variability is a meaningful 

function of system attributes. As discussed earlier, EPA sensitivity analysis reported in the SAP 

Briefing Document did show risk to vary as a function of mean temperature, but various factors 

and interactions not considered in probabilistic CASM make this effect of uncertain validity. 

However, even if there are some elements of probabilistic CASM that reflect meaningful 

variation of risk due to system characteristics, this doesn't matter because both uncertainty and 

meaningful variability have the same implications to risk assessment for this generic assessment. 

If the variability is uncertainty, risk needs to be managed based on some location in this range of 

uncertainty. If the variability is meaningful, the implication is that some systems are more at risk 

than others, but generic risk then must be managed based on a similar location in the 

distributions of systems to be protected. 
Page 23 of 29 

4.2 Risk managment using Syngenta methodology will be unjustifiably conservative. 

As noted above, whether probabilistic CASM variability is uncertainty or variability, it requires a 

decision regarding where risk is managed within this variability. The median is arguably 

inappopriate, because it allows a 50% change of a system not being protected. Furthermore, 

given the wide variability of CASMATZ risk estimates (Figure 5), this also involves considerable 

risk of substantial effects, not just minor risks. Use of the median is contrary to current OPP 

practice of using more sensitive endpoints. 

Table 1 lists the 50th and 90th risk factor percentiles for all 16 example chemographs from Figure 

5. These represent the factor by which exposure would need to be reduced so that the MEI is at 

the LOCMEI for the selected percentage of probabilistic cases. Using the 90th percentile to 

provide moderate conservatism in face of probabilisitic CASM variability/uncertainty results in 

15 of these chemographs exceeding a level of concern (as opposed to 8 if the median was used), 

and would result in restricting exposure by 2.0- to 3.5-fold more than if the median was used. 

This more restrictive risk management is a necessary consequence of the variability/uncertainty 

of probabilistic CASM. However, whether these risk restrictions can be justified must be 

questioned given various shortcomings and insufficient validation of CASM for this application. 

This creates an untenable situation for EPA – of suggesting conservative regulations that cannot 

really be defended and of not being able to justify less conservative CASM-based risk 

assessments because of various uncertainties about this model. 

4.3 Syngenta methodology requires site-specific application that is uncertain and 

impractical. 

A final point that should be made regarding the implications of the proposed Syngenta 

methodology to risk assessments is that, if probabilistic CASMATZ variability is truly a 

meaningful function of definable system attributes, then the generic assessment approach should 

be dropped. If CASMATZ can truly be trusted to provide meaningful information on the 

relationship of atrazine risk to system attributes, then a site specific approach should be used as 

discussed at the beginning of this critique, in which a version of CASM is developed for each 

experimental and natural ecosystem of concern and the MEIs/LOCMEIs and risk factors reflect 

these system differences, not just model extrapolations across the time-series of concerns. Even 

if possible, this is not practical regarding the level-of-effort, money, and time needed to address 

the many development issues/questions with CASM for so many systems. The successful uses 

of CASM have involved sufficient efforts to tailor CASM to answer assessment questions for 

specific sites. The problems discussed here with probabilistic CASM underscore how it is not 

justified for the desired generic atrazine assessments. 

Table 1. 50th and 90th percentile risk factors for CASM-based assessment of 16 example chemographs. 

Chemograph # 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

50th Pct Risk Factor 2.0 0.85 0.68 0.78 0.66 1.27 1.49 1.41 1.62 0.32 0.64 0.59 1.33 1.61 4.1 3.3

90th Pct Risk Factor 7.0 2.1 1.82 2.1 1.76 3.4 3.5 3.5 3.8 0.64 1.33 1.22 3.1 3.6 9.5 8.1
Page 24 of 29 

Appendix I. Atrazine Assessment Framework 

1. Problem Definition 

A set of microexperimental ecosystem and mesoexperimental ecosystem (experimental 

ecosystem) studies with atrazine serve are to serve as the foundation for determining LOCs for 

atrazine exposures in natural freshwater systems. Figure I-1 summarizes these data. Each datum 

on Figure I-1 is characterized by the exposure duration at which effects were assessed, the 

average exposure concentration over this duration, and an effects score (Brock score) using the 

system of Brock et al. (2000). A fundamental assumption in using these experimental ecosystem 

data is that they collectively describe a relationship of effects to exposure that is relevant to 

natural freshwater systems. In other words, natural aquatic plant communities will react 

adversely if subjected to the same exposures that elicited significant adverse effects in the 

experimental ecosystem studies. This assumption is inherent in any assessment that extrapolates 

toxicity experiments to the field. The use of experimental ecosystems arguably provides a better 

basis for such extrapolations than do single-species toxicity tests. 

An important consideration in using the experimental ecosystem data to establish LOCs is the 

diversity of exposure concentration time-series. Not only do the experimental ecosystem studies 

have different durations, but the exposure concentrations within these durations can range from 

being roughly constant to declining substantially with a half-life as short as 3 days. This 

variability makes if difficult to compare effects among different experimental ecosystem studies, 

but exposure variability is an even greater problem for extrapolating these effects to the field. 

For natural freshwater systems, atrazine exposures tend to be even more variable than in the 

Figure I-1 Brock scores in atrazine cosm exposures as a function of exposure duration and 

average concentration. Dashed lines denote the 10-20 μg/L LOC concentration range designated 

by the 2003 IRED. 

Exposure Duration (days)

2 5 20 50 200 10 100

Average Concentration (

μg/L Atrazine)

0.1

1

10

100

1000

10000

5

4

3

2

1

Brock

Score
Page 25 of 29 

Figure I-2. Examples of atrazine exposure time-series for natural freshwater systems. 

0 50 100 150 200 250 300 350

0

50

100

150

200

250

IN11_05_1Peak

0 50 100 150 200 250 300 350

0

10

20

30

40

50

0 50 100 150 200 250 300 350

0

5

10

15

20

25

0 50 100 150 200 250 300 350

0

2

4

6

8

10

0 50 100 150 200 250 300 350

0

10

20

30

40

50

0 50 100 150 200 250 300 350

0

10

20

30

40

50

0 50 100 150 200 250 300 350

0

10

20

30

40

Julian Day

0 50 100 150 200 250 300 350

0

10

20

30

40

50

0 50 100 150 200 250 300 350

0

10

20

30

40

0 50 100 150 200 250 300 350

0

5

10

15

20

25

0 50 100 150 200 250 300 350

Concentration (

μg/L Atrazine)

0

10

20

30

40

50

0 50 100 150 200 250 300 350

0

10

20

30

40

0 50 100 150 200 250 300 350

0

10

20

30

40

0 50 100 150 200 250 300 350

0

50

100

150

200

RegA_Res_1Peak

IL08_06_HiLo IL08_06_HiHi

IL08_06_LoHi Sandusky_HiLo

Sandusky_HiMed IN03_05_Orig_HiMedLo

IN03_05_HiMedLo IN03_05_MedHiLo

Maumee_MedHiLo Sandusky_MedHiLo

RegG_Res_longdur MO01_05_MO01

1

3

5

8

11

13 14

12

10

6

4

2

15 16
Page 26 of 29 

experimental ecosystems and do not have a fixed duration. Atrazine enters natural freshwater 

systems largely in rainfall-driven runoff, resulting in highly variable and episodic exposures 

(Figure I-2) that depend on rainfall distribution, atrazine application patterns, topography, and 

soil properties. Therefore, a primary goal for this assessment methodology is to address the 

diversity of exposure time-series both in the experimental experimental ecosystems and natural 

systems. The experimental ecosystem data do not provide sufficient information for defining 

such time effects because the scoring system is semiquantitative and the study conditions and 

responses do not fully address the issues of duration and time-variability. Thus, another issue 

facing this methodology is to base time effects on considerations other than the experimental 

ecosystem data. 

Atrazine LOCs are to be based on discriminating the Brock score group of 3 to 5 (i.e., significant 

effects) from the score group of 1 or 2 (i.e., no to slight effects). As shown in Figure I-1, these 

two score groups are largely separated, although some overlap also exists, particularly over the 

10-20 μg/L range.. Some of this overlap is due to the exposure time-variability issue (which 

cannot be fully addressed in the two-dimensional graphic format of Figure I-1); for example, the 

two lowest squares involve exposures with fast decays, so that the plotted average exposure 

concentrations are much lower than the peak concentrations. However, this overlap is also 

reflective of other differences among the experimental ecosystems. Unfortunately, these 

differences are not sufficiently understood to support methods for better discriminating the score 

groups. In any event, this overlap means that any LOC methodology will need to address the 

occurrence of false negatives (i.e., Brock scores of 3 to 5 which lie below the LOC) and false 

positives (i.e., Brock scores of 1 or 2 which lie above the LOC). The positioning of the LOC 

relative to the number of false positives and negatives is a key risk management decision. 

To illustrate this decision, Figure I-1 shows horizontal lines at 10 and 20 μg/L (the LOC range 

specified in the 2003 IRED). The upper line corresponds to false positives and false positives 

being about equal, while the lower line has considerably more false positives than negatives. It 

is important to emphasize that these horizontal LOC lines are presented here only to illustrate the 

concept of using the experimental ecosystem data to position the LOC, and that the methods 

described in this document will not produce simple, fixed-concentration LOCs. 

Therefore, the general problem is to extrapolate effects observed under one exposure time-series 

to another time-series. This extrapolation will allow effects observed for different experimental 

ecosystem exposures to be compared and synthesized into an LOC, and for this LOC to be used 

to assess the types of exposures observed in natural systems. For example, if 20 μg/L atrazine is 

an LOC for a 60-day constant exposure, what would be the LOC for a 30-day constant exposure? 

Or what would be the LOC based on the highest peak (or some average concentration) for a 

chemograph with a complex shape, such as those in Figure I-2? 

The LOC thus must be a function that can be computed for the entire range of exposure timeseries encountered in experimental ecosystem studies and in natural freshwater systems of 

interest. This requires a model that can provide, for any exposure time-series, a comparable, 

quantitative measure of expected effects, which will be referred to as the model effects index

(MEI). For example, this might be a model-calculated percent reduction in total plant biomass 

over a specified duration. 
Page 27 of 29 

By correlating the MEI to the Brock scores for the various experimental ecosystem exposures, an 

LOC for this MEI (LOCMEI) can be determined, so that the LOC is on a modeled effects scale

rather than on a concentration scale. Evaluation of the exposure time-series in natural 

freshwater systems is then simply done by computing the MEIs for these time-series and 

comparing them to the LOCMEI. One consequence of such a "calibration" of the LOCMEI to the 

experimental ecosystem data is that the model is not required to make absolute predictions of 

effects in an actual aquatic system, but rather is used to support comparisons of the relative

severity of different exposures time-series. This general strategy for assessments is described in 

further detail below. 

Because the experimental ecosystem data do not contain sufficient information on timedependence of effects, this model also must provide an acceptable basis for this timedependence. In addition, the model should provide extrapolations among different exposure 

time-series that are not adjusted for physical, chemical, and biological properties attributes (other 

than the exposure time-series) of the experimental ecosystems and natural systems of interest. 

Even if a model is a function of such attributes, it must be applied with generic values for the 

attributes because there is insufficient empirical information on the relationship of effects to such 

system attributes to validate model predictions and because the model must be applicable to 

systems for which these system attributes are not defined. With the exception of exposure 

variability, the influence of system attributes on differences among experimental ecosystem 

studies and on extrapolations from experimental ecosystem studies to natural systems cannot be 

addressed. 

2. General Strategy for Assessment Methodology 

This section describes the general strategy for atrazine assessments in terms of two major steps: 

(1) establishing an LOCMEI based on experimental ecosystem data and (2) using the LOCMEI to 

evaluate exposure time-series for natural freshwater systems. This general strategy can 

accommodate MEIs of widely different natures and is described here without specifying the 

exact nature of the MEI. However, for illustrative purposes, it is useful to provide actual values 

for some variables, so some calculations were made for the description. The effects model is 

only characterized as being able to convert an exposure time-series into a time-series of "percent 

effects". The MEI is set to the maximum 90-day average of these daily percent effects, this 90-

day assessment period being an arbitrary example. The LOCMEI is determined by equalizing 

false positives and negatives, and the LOCMEI value is for illustrative purposes only. 

Establishing a LOCMEI Based on Experimental ecosystem Data 

(1) The MEI is determined for each of the atrazine exposures in the experimental ecosystem data 

set. This MEI must be for a fixed assessment period to provide comparability among different 

exposures (i.e., effects confined to a short exposure period cannot be equated to effects extending 

over a longer exposure period; rather, average effects of each exposure over the same period 

should be compared). The selected assessment period must be appropriate to the nature of the 

MEI, the experimental ecosystem studies, and risk assessment goals. 
Page 28 of 29 

(2) The observed Brock scores (1 through 5) for each experimental ecosystem treatment are 

correlated to their respective MEIs, to determine what levels of the MEI are associated with 

observed effects in the experimental ecosystem. An LOCMEI is determined that best 

discriminates between the score groups (i.e., 1 or 2 vs. 3 to 5), based on a desired relative 

incidence of false positives and negatives. Note that the LOC is now on the modeled effects 

scale rather than an exposure concentration scale. 

Figure I-3 depicts this process. The top panel shows the experimental ecosystem data and 

highlights one of the data points, whose exposure time-series is shown in the left middle panel. 

The effects model converts this exposure data to daily percent effects in the right middle panel 

and these daily values are in turn processed into 90-day running averages for this example. The 

MEI is set to the maximum (9.3%) of these running averages. In the lower panel, the Brock 

scores versus the MEIs are plotted for all of the experimental ecosystem treatments. The 

LOCMEI is set to the MEI value that equalizes false positives and negatives (vertical line at 

1.9%). 

Using a LOCMEI to Evaluate Exposure Time-Series from Natural Freshwater Systems 

(3) For each exposure time-series from natural freshwater systems of interest, the MEI is 

calculated and compared to the LOCMEI to see if it is exceeded, and by how much. 

Figure I-3. Strategy for establishing the level of concern for a model effects index 

based on cosm data 

Exposure Duration (days)

2 5 20 50 200 500 10 100

Average Concentratio

(μg/L Atrazine)

0.1

1

10

100

1000

10000

Time from Start of Cosm Exposure (d)

0 20 40 60 80 100

Concentration

0

10

20

30

40

50

Time from Start of Cosm Exposure (d)

0 20 40 60 80 100

Perce nt Effect

0

5

10

15

20

25

Daily Value

90-Day Running

Average

Model Effects Index

0.2 0.5 2 5 20 50 1 10 100

Brock Score

1

2

3

4

5

LOCMEI=1.9

Effects

Model

Model Effects

Index = 9.3

5

4

3

2

1

Brock

Score
Page 29 of 29 

(4) If desired, iterative model calculations can be conducted to determine the factor ("risk 

factor") by which the exposure concentration time-series would need to be decreased for the 

MEI to exactly equal the LOCMEI, thereby providing further information for risk assessment. 

Figure I-4depicts this process. The top left panel provides an example field exposure time-series. 

The effects model converts this exposure data to daily percentage effects in the top right panel 

and these daily values are in turn processed into 90-day running averages, the maximum of 

which (6.3%) is the MEI. In this case, the MEI substantially exceeds the LOCMEI (1.9% average 

effect). The bottom panels illustrate the process by which a risk factor can be determined. The 

model is iteratively applied to scale the exposure time-series (left bottom panel) so that the MEI 

exactly equals the LOCMEI (right bottom panel). The ratio of the original exposure time-series to 

the adjusted one equals the risk factor, which is 3.5 in this case. 

For an exposure time-series of this exact shape, the bottom left panel of Figure I-4 defines an 

LOC on a concentration scale basis, and the original exposure time-series would need to be 

reduced by 3.5-fold to be at the LOC. This concentration-based LOC can be characterized by 

any single (e.g. the peak) concentration or any average concentration over the time-series, but if 

the relative shape of the time-series changes, this concentration-based LOC would not be valid 

and new model calculations would be needed to determine when exposure is at the LOCMEI. 

Figure I-4. Strategy for using the level of concern for a model effects index to 

evaluate field exposures 

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Concentration

0

20

40

60

160

180

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Concentration

0

10

20

30

40

50

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Percent Effect

0

2

4

6

8

10

12

14

16

Daily Value

LOC 90-Day Running Average MEI

Julian Day

0 30 60 90 120 150 180 210 240 270 300 330 360

Percent Effect

0

5

10

15

20

25

50

55

Daily Value

90-Day Running Average LOC

MEI

Risk Factor

is Ratio of

Exposures

Risk Factor = 3.5

Iteratively Apply Model 

to Scale Exposure so

MEI=LOCMEI

Apply Effects Model

to Determine if

LOCMEI Exceeded

MEI = LOCMEI =1.9%

MEI(=6.3%) > LOCMEI (=1.9%)